Learning Awk Is Essential For Linux Users
ฝัง
- เผยแพร่เมื่อ 15 มิ.ย. 2021
- One of my favorite command line utilities is "awk" which is a text-processing program. It is mostly used for pattern scanning and processing. In this video, I will give examples of some of the basic awk commands, and show you some of the ways that I often use awk.
REFERENCED:
► www.gnu.org/software/gawk/man... - GNU Awk
WANT TO SUPPORT THE CHANNEL?
💰 Patreon: / distrotube
💳 Paypal: www.paypal.com/cgi-bin/webscr...
🛍️ Amazon: amzn.to/2RotFFi
👕 Teespring: teespring.com/stores/distrotube
DONATE CRYPTO:
💰 Bitcoin: 1Mp6ebz5bNcjNFW7XWHVht36SkiLoxPKoX
🐶 Dogecoin: D5fpRD1JRoBFPDXSBocRTp8W9uKzfwLFAu
📕 LBC: bMfA2c3zmcLxPCpyPcrykLvMhZ7A5mQuhJ
SOCIAL PLATFORMS:
🗨️ Mastodon: distrotoot.com/@derek
👫 Reddit: / distrotube
📽️ LBRY/Odysee: odysee.com/$/invite/@DistroTu...
DT ON THE WEB:
🕸️ Website: distrotube.com/
🐿️ Gemini Capsule: gemini://distro.tube
📁 GitLab: gitlab.com/dwt1
FREE AND OPEN SOURCE SOFTWARE THAT I USE:
🌐 Brave Browser - brave.com/dis872
📽️ Open Broadcaster Software: obsproject.com/
🎬 Kdenlive: kdenlive.org
🎨 GIMP: www.gimp.org/
🎵 Audacity: www.audacityteam.org/
💻 VirtualBox: www.virtualbox.org/
🗒️ Doom Emacs: github.com/hlissner/doom-emacs
Your support is very much appreciated. Thanks, guys! - วิทยาศาสตร์และเทคโนโลยี
![แข่งแลกของ 1 วัน!! ของใครแพงสุดชนะ!! [Ver.2024]](http://i.ytimg.com/vi/joRT59ilPHg/mqdefault.jpg)
![[4K] TREASURE(트레저) “KING KONG” Band LIVE Concert 킹콩은 라이브를 찢어🦍 [it’s KPOP LIVE 잇츠라이브]](http://i.ytimg.com/vi/p8bLLOxPDD8/mqdefault.jpg)
AWK is not just a text processing utility, awk is a complete programming language!!
yes, remove Perl, Python, and Node JS (notice no mention of Ruby - ah how far they do fall) - awk and bash is all you need
@@TheSulross you need Perl to keep your sanity, you can remove the rest
Calm down. It's not that exciting.
@@gordongoodman8342 it is, a lot!
yup! I was going to comment the same thing. you could even use it as your shell.
DT: AWK is important for linux users
Luke Smith 1 hour later: Why AWK is useless for linux users
_Awk is the predictable result of misguided Enlightenment philosophies._ 🤭
Brodie 2 hours later: Boomers fighting over usefulness of linux utilities
_Why learning awk when you can learn grep_
Luke Smith 1 hour later: "You can replace Awk with Emacs"
10hs later: "You can replace your wife with Emacs"
@@blogattacker 10.5 hours later...I did. 😂
awk is a must. So powerful.
Btw. You actually did just square. Not square root :)
Could have done the square root of i*i is i
I tried it with sqrt(i) and that worked. I kind of wonder how much math you can do with awk... Yes, exp(var), log(var), cos(var), sin(var) - but not tan(var); var**x gave exponent, as did var^x. Lots of different functions...
I have poorest knowledge of maths but I paused and rushed to comment section to check I am not wrong. But tutorial is very good.
At 8min, when you pipe uniq, note that 'uniq' does not detect repeated lines unless they are adjacent.
You may want to sort the input first, or use 'sort -u' without 'uniq'.
so it would be | sort | uniq OR | sort -u
indeed
quite
rather
In the square root example, very time DT said square root, he means square!
Great video, DT!
Just swap the 2 fields and pretend you didn't see that:
"10 is the square root of 100" is a completely valid statement 😃
@@antoninperonnet6138
I could do that! 😉
It's AWK math
@@KeithPeters 🤣🤣🤣🤣
This is a great tutorial about awk. I've learned quite a bit. The only thing that red flagged me was that NONE of those were square roots. They were squares.
I just posted this same thing, before seeing you posted it.
He just got the order backwards. "The square root of i*i is i". Anyone could have made that mistake.
14:00 - /^[b,c]/ matches anything that begins with a "b" or a "c" *or a comma*.
neat
So true.
Regex yeet
Had to process a 120k line csv at work the other day. Awk was a god send
Great TUT. I ran into an issue because I don't use uniq and sort often enough; uniq doesn't work if dupes are not adjacent so you need to sort first.
I just came to say that I can see that you listened to the comments and now you are sharing your knowledge on simple but technical stuff that's really helpfull. I apreciate that and also can feel that effort
That is the most clear demonstration of AWK I have ever seen. Amazing.
Glad you made this video. I was stumbling upon awk in everywhere and wanted to learn it but didn't have time to do so. Thanks
This is great, this video arrived just in time since currently I am on problem automating a lot of stuff with bash scripting and using awk for text-processing. Thanks for the content!
Thank you Derek I appreciate the time and effort you put into making these videos. You help us "middle users" take things up a notch. Middle as in we are no longer beginners but won't ever be sysadmins. We do however want more out of their Linux system.
Sir, your a wizard with awk, i only knew half of this 😂
Needed a quick refresher on awk as I had no used it in years. This was a great video, covering the essentials. Cheers.
This is one of the best explanations of awk I've seen! I'll save this one for some people I know that are still just learning Linux.
Good introduction to awk. Back in 2013 i had to parse huge log files that had no field separator, even worst each line could be completely different depending on certain conditions. Awk saved my life.
Thanks for the share. I used use text editors like Textpad to do these things with regex, but this is so much better since not just find/replace, but perform operations on each result, and being able to easy have in a shell pipeline of commands. Always too lazy to learn it for some reason, but now I'm motivated.
Straight to the point & excellent as always. Keep up the good work DT ! 👏
DT, you are amazing man. Yours are the only videos I have been watching on a daily basis. Thanks for sharing all the knowledge!
Hey DT. Great review of the power of "awk". You cover all the basic functions for a new user. I use "awk" in a shell script to sort through my datalog's results that i need. Then dump it to a csv file.
Good video. Some of these, I didn't knew.
An error in the video: it wasn't the "square root", but just the "square", or "power of two". The square root of 4 is 2, the square root of 9 is 3, the square root of 25 is 5, and so on.
i came to the comments expecting every comment to be this. I am highly disappointed in the YT comments shit posting today.
@@christopherdavies7403 And this guy says 'I didn't knew', grammar mistake.
@@gordonbai5320 ok boomer
@@charlieoocharlie9466
FLAME WAR!
And you live in your mother's basement.
awk is awesome, but if you just want the number of lines, using wc is less characters to type :)
yeah, that was one example that did not resonate at all because wc will also provide totals when applied against multiple files - much less to type to just use wc
The point is to show what you can do with awk =)
The simple fact that fish is your default shell deserves a like and a subscription.
Nice, thank you. I am beginning my journey on bash and this is awesome.
This is really helpful and practical, thank you DT!
Great video. An accessibility note: Your console’s active line is at the very bottom of the screen, which is where TH-cam on Smart TVs puts the subtitles, so the subs block the thing you’re typing. It would be better if your terminal was sized so that the bottom 10-20% of the screen isn’t used.
I bet someone is already rewriting awk in rust for the memes.
you mean for speed and safety? /s
@@alessandroferrari4699 people use rust because it's fast and easier to solve errors and even rivals c in speed while being easier.
yes because obviously all these 40 year old Unix tools are buggy and broken after thousands upon thousands of users applying them daily over many decades
@@TheSulross exactly 😂😂
This is truly amazing, hope to have another new Awk video in the furure!
Great tutorial DT! AWK is very powerful!
Thanks, Josh!
@Earthling-Z3R0 Thanks man. You're right.
DT, this is an excellent video! Thanks!
Hell yeah!!! AWK arrived at the right time!! Thank you Derek!
I have been putting this off for a long time thanks for encouraging me to finally learn it.
Learned more about awk in 20 minutes than in the previous 40 years
I love you man, you're the only one who was able to explain awk in a simple yet effective way. I finally understand how awk works!
Very nice introduction to awk features. Thanks for sharing this useful knowledge!
That was some awesome awk-fu bro, thank you!
Very useful info DT! I only knew how to print columns in awk.
Excellent presentation!! to the point and straight forward! great job!
Great video DT! Thanks for the awesome awk tutorial
Except:
s/square\ root/square 😀
Thanks for this video. It helps me understand how to use awk better.
Thank you professor DT for the lecture .
This is a great review of some practical usages.
Thanks buddy for such clear explanations. Now I'm comfortable to start write some crazy scripts.
I have been using linux for years living on a few commands. I had only a faint knowledge of the command/programs such as awk or sed. Mainly because I had no ideas of their capabilities and use cases. Now I can see how useful they can be. I usually recommend your channel to my PhD students and observed a real improvement in their computer skills. IMO your channel is vital for linux wannabe and "familiar but not expert" users. So thank for your work.
He also inspires me to learn about this powerful utilities.
My payment will be available in 2 weeks, then I'll be a patreon. Your content is amazing.
This is a great video on AWk..thanks
Awk is awesome, great vid I learned a lot
I have always had trouble with awk because of all the syntax involved, this helped sooo much.
When you pressed enter on the square root part then said that the output was correct, this is the task list I had to perform to get back on track:
1. Facepalm
2. Pull fire alarm
3. Crying session
4. Shower with clothes on
5. Call Mom
6. Eat some sweets
7. Finish the video
Thank you for doing this video. Very helpful.
Great tutorial! I do prefer the WTF command. It covers all my confusion when trying to navigate the command line. 😜
This is the kind of stuff I love to watch
I learned so much more here in 20 mins then I did in my last two classes about this
Awk ... very meaningful name !!!
Looks like an assembly instruction.
This was helpful, thank you 😊
Great vid DT!
This was helpful…. Thanks to u and ur team
Heaviest 20 minutes in my life. Worth it
i really appreciate your effort , thank you for the great lesson MAESTRO .
I enjoyed your presentation on awk and I subscribed to this channel. I have a question: Can awk to the following job -- send the first line of a file1, delimited by default space, to another file2, file2 that would take its name from the first few fields of the line 1 just transferred from the file1? Thank you
that is a very cool podcast, very useful. thank you
A little bit of Awk can be useful.
My standard example: I have a list of shares I track. The website I use lets me download data, but holds more data (open, close, max, min,volume ) than I want.
Awk is perfect for filtering a single column (close values).
Great tutorial. I learned more in 20 mins w/ awk then in 7 years in IT.
What did you do then in 7 years of IT? :))
Amazing! Basically I can programming using awk. Thanks
Yea, you can actually use "awk" as a scripting language. Might have been useful in past decades. But now, I'm not sure why anyone would script with awk when things like bash and python exist.
@@DistroTube maybe awk faster than python and more specific programming language bash or posix shell
love both Sed and Awk
thanks awk command that help me a lot in my project after watching this video ^_^
Nice Luther insult at 8:20! I for real laughed out loud on that one. I'm surprised no one called that out in another comment...😁
Great video, helpful comments and even better resourceful and corrective people in the comments
Your audio sound 1000x better then before
small suggestion, bring the bottom of your terminal window up a bit so the youtube progress bar doesn't cover up the command you're entering when the video is paused. Thank you for making this!
I know in most window managers you can change the size of a window from left or right but not on the top or bottom at least from what I have seen unless you are in floating mode than you can change it to any size. Most of the space around each window is configured hard coded in your window manager config. You could change it but than would have to change it back after a video.
@@warhawk_yt clearly he just needs to increase the edge padding to something crazy like 48-64 pixels for the [ M A X I M U M R I C E ]
Always great content DT! Is there a way around the case where the field separator is inside the entries? Like the ps -ef command output has spaces inside the content as well as spaces being the field separator.
I'm sure there is an awky way to do it, but in this case i would be lazy and choose sed.
Awsome presentation 👍
Awesome... good to know how versitile awk is.
Excellent, thank you very much!
Great channel, great information.
Very Powerfull. Thx for the video.
Yup, it's a complete language, as it is taught in CS degree courses (when taught). However this guy, otherwise, is a far more interesting teacher. Thanks, DT! This app/language confusion here is, like with Python, there is the language, AND there is ERIC, a free IDE that comes with Python. ERIC executes Python, but it's NOT Python. The AWK IDE is more invisible, but there. You can write independently, then execute.
USEFUL!! Thank you!
Yup, awk is cool. Just as an aside that was squares, not square roots. For square roosts you would need something like (n^(1/2)). AWK does have ^ which is raise to the power function. Raising a number to a fractional power is taking the root.
Thanks for making this video! I've tried a couple of times to add AWK to my arsenal, but gave up halfway. This has inspired me to give it a more serious go. I'm generally pretty comfortable with grep and sed but looks like I can make this work for replacing those two for many use cases.
Hi, thx for ur great videos reagarding Linux! I have a question: r u still using trackballs?
Quality content 💯
The problem with awk is that most people know it but all seem to think its whole purpose is just to extract columns. This Video gives many practical example that it's so much more.
A couple months ago I wrote a little shell script to
gather system information such as
CPU model, memory, computer model, shell, WM
(using lscpu, dmidecode, etc)
and I had to learn a little bit of Awk, Sed, and Grep
to clean it up and make it look pretty.
This video would have came
in handy back then, instead of spending 30 minutes
reading StackExchange articles on them!
Could you share your script? I would like to use it!
I really need to push myself to use awk more. A lot of this can be done by just combining grep, sed, and cut, but it’s kinda nice having it in one process.
Your example for the line count can be reproduced with the command "wc -l /foo/bar/file_a /foo/bar/file_b"
As I found out, you have to sort _before_ running the _uniq_ , otherwise it wouldn't work.
Useful video thanks 🙏
Good Morning Sir,
I just wanted to know which Distro and terminal you're using here?.. it's so dope! 🔥 Also that fun message after entering wrong syntax LOL that's where I lost it..
like you said i should have learned this earlier. Good tutorial..
Thanks! What mic are you using?
Also 7:40 , you could use awk -F "/" '{print $NF}' /etc/shells for the same result
Your df command gives output equivalent to df -h , I suppose you configured an alias?
@Distrotube Great video !!! I tried to extract livescores of footbal matches by using AWK but don't know how to do it.
I found mobile version of website flashscore and there are scores in source code but how can I pull
time of event / home team / score / away team to display in terminal ?
because I don't want to use crappy API's if that could be done by core utils like AWK.
Never needed awk in last 6-8 years or so... but thanks!
Thanks for sharing.
David Draiman loves this program.
Please do jq tutorial next: it's so awesome to manipulate json data, combined with curl it's very powerful for working with REST apis
Is jq a GNU utility?
@@ghodawalamoamanzahirabbas8996 it isn't (according to short duckduckgo search :) ) - but it's amazing nonetheless!
Geeeewd, you have a nice voice sir!
14:24 Why did it print the indented line? I would have expected it to not do it, based on the behavior of 11:04, which counted indentation as characters and this one checking the first character being 'b|c'. Are whitespace characters only pseudo-characters or something like that? It would sort of make sense, because of the whole column thing, but I'm still quite a bit confused.
i always found it easy to use because i'm AWKward.