Why Use CAT Or GREP When You Can AWK?
ฝัง
- เผยแพร่เมื่อ 1 ธ.ค. 2019
- Enough of the bloat! You don't need to use cat, grep, head, sed, etc. All you need is AWK!
REFERENCED:
► www.gnu.org/software/gawk/man...
WANT TO SUPPORT THE CHANNEL?
💰 Patreon: / distrotube
💳 Paypal: www.paypal.com/cgi-bin/webscr...
🛍️ Amazon: amzn.to/2RotFFi
👕 Teespring: teespring.com/stores/distrotube
SOCIAL PLATFORMS:
🗨️ Mastodon: mastodon.technology/@distrotube
💬 IRC: irc://freenode #distrotube
👫 Reddit: / distrotube
DT ON THE WEB:
🕸️ Website: distrotube.com/
🌐 Gopherhole: gopher://distro.tube
📁 GitLab: gitlab.com/dwt1
FREE AND OPEN SOURCE SOFTWARE THAT I USE:
📽️ Open Broadcaster Software: obsproject.com/
🎬 Kdenlive: kdenlive.org
🎨 GIMP: www.gimp.org/
🔊 Audacity: www.audacityteam.org/
💻 VirtualBox: www.virtualbox.org/
Your support is very much appreciated. Thanks, guys! - วิทยาศาสตร์และเทคโนโลยี
![คุณอาเรียโต๊ะข้างๆ พูดรัสเซียหวานใส่ซะหัวใจจะวาย ตอนที่ 4 [ซับไทย] 【Ani-One Thailand】](http://i.ytimg.com/vi/_vK_cden2f8/mqdefault.jpg)
Just one day after Luke Smith's video.
th-cam.com/video/82NBMvx6vFY/w-d-xo.html
Things are heating up. Next: Why awk sucks
Only SIX YEARS after Conner McDaniel's awesome SED series >:P
@@eritert Next on chris titus... Why Awk is the Devil and Why So Many No Longer Use It :P
who's the noob now? very awkward
;)
@@eritert Why awk is bloated.
2:54 - why bother with cat when you can awk and make the command 8-10 characters longer? I get that awk is an interesting program, but this is absurd. Come on DT, these videos are great but this is silly.
He would use "sed" instead..
Yeah people don't use akw because a lot longer commands. Even the first users of unix like Brian still use cat, is perfectly fine. Also people use grep because is shorter. So cat and grep are the pure essence of Linux programs in the 70s be great at one thing. Awk has its purpose but it isn't to replace grep.
Here's a video of Brian the creator of AWK which says grep is his favorite command lmao th-cam.com/video/O9upVbGSBFo/w-d-xo.html
Couldnt disagree more.
@@ArdieMejia83 care to elaborate? Why would you still use awk when cat is way shorter and easier to type?
My suggestion is to not waste time showing how a tool can awkwardly emulate another, but what it can do better. Splitting fields is one of the most common uses for awk.
you mean awk wardly dab
Exactly what I was thinking. "Why use "head" when you can type out this much more verbose syntax to accomplish the same thing?" hmm
And splitting fields with awk is not _awk_ wardly (haha) emulating other tools? Like ... _cut_ for cutting fields?
Well i think what's he's highlighting here is that, maybe those tools could share more of they're functionalities in a UNIX mentality, instead of having their own ways of doing some same things. awk, sed and grep could use cat binary and have it in dependency to get text from file. It would make more sense.
But i think that's kinda the goal of busybox
why carry around a hamer when you can hammer things with your shoe?
DistroTube vs Luke Smith : Civil War 😂😂
Than-O.S. is coming! LoL
No it's not. Unaboomer while immensely popular, doesnt rely on YT - he is a community builder (without probably realizing it) around i3, as he gets real life problems and shows the solutions. He's a freaking eye opener. He's not a news outlet and doesnt search for topic to cover - he discover something useful, teaches himself, improving a toolkit and shares with us.
He shows you realtime what you can do with it from the point of view of regular (yeah, right...) staff of the university of Arizona (or in whichever hole is he know).
Picking up on him without understanding his point (which is 20-30%% performance boost, 6 minutes instead of 9 for longer scripts) is childish and as much pathetic as calling him "another creator"
To be fair, Luke specifically was talking about piping cat into grep for standard grep uses, he never proposed it as a replacement for cat in it's whole. Interesting video though!
3D terminal games have been written in Awk. It’s that powerful.
Tetris games have been written in awk. True story.
@@DistroTube I like Tetris, since it was released in europe, because I'm born in the former GDR.
@@msdosm4nfred are you sure this is the right channel for you? just going by the name.
Really? Where what?
@@urugulu1656 MS-DOS is now open source, atleast 1.25 and 2.0. So isn't it technically the best non-POSIX os now. repo link github.com/microsoft/MS-DOS
This was actually hilarious. Thanks DT
I believe that the Unix philosophy is small single purpose tools that do one thing well. That is why you would use grep instead of awk, not to mention that the grep command line was very much simpler and easier to remember than the awk equivalent. That said, this was a very interesting video on how the same things can be accomplished with different tools.
"Why use awk and sed if you can use Perl?"
Just kidding, Perl is awesome.
Why use Perl when you can use Node.js?
@@alexxx4434 Because js it is a soydev language.
You though I was serious?
@@alexxx4434 Yes, sorry. I wooshed my self.
This video is spot on. Awk is seriously underutilized. One of my favorite commands.
If you use awk for this, you shouldnt be allowed anywhere near scripting in any production environment :D
@@meyimagalot9497 lol
awk is underutilized, _as a scripting language_ .
It’s not very useful as a command, but as a scripting language for all sorts of text manipulation. Other stuff often done using awk as a command have their own commands (cutting fields with cut, _grepping eith grep_, replacing characters with tr, simple stream operations with sed, ...). awk is a scripting language!
Humm semantics and pedantics, eh? I enter the awk command and use the awk scripting language daily :)
@@mitchelvalentino1569 Pff. awk is for longer scripts (although not getting _too_ long ... usally ~50 loc.)
All depends on the task ... and the size of the data-set and the time allowed to do it !!
"This got AWKward."
This video is such a flex that he is literally flexing in the thumbnail, love it!
Thank you, Derek. "Bloat" is bloat. Use "blo" instead.
Gnu/Bloat
The cat replacement can be shortened to awk 1 new.sh. I do this instead of cat when I think I will want to perform an action on the file using awk after I view the contents. Now I can recall the last command and simply replace the 1 with the AWK program.
Yea I think I recall seing this video recently of some boomer talking about using sed instead of cat.
Only way Luke can redeem himself is if he does a NixOS series
I know what you are doing there! Lol
Some people use TurboTax to file their taxes. I file them with AWK.
Thanks and learned a lot
Awk is like using a sledgehammer to drive in a nail. LoL. I will continue to use cat and grep.
Not piping from cat into grep makes sense. Using awk instead of quicker alternatives does not. grep, head, tail, and sed, are shorter and easier to type and give the same results. I bet they complete quicker as well.
Of course!
Also great way to view firtst few lines using awk.
awk ''FNR
I think we need to include all that stuff into systemd )))
"why use a cap when you can use an umbrella on a sunny day"
At some point these classic Unix tools just become verbs for what we want to do - when we just want to dump files as is, or combine several files into one, we achieve that via the verb of 'cat'. When we want to scrape things per line out of a file, we 'grep' it. If we want to scrape, filter/transform a file, we 'awk' it. Even if there were one uber tool that could do all these things, I'd have to create aliases for 'cat', 'grep', 'awk' so those verbs would still be there
Why use grep when you can just print the file out on a dot matrix printer and use white-out on all the lines you don’t want to see?
For beginners 'bc', 'cat', 'curl', 'head', 'grep', 'sed', 'tail', 'wc', and 'wget' are easier to learn for text retrieval and processing; the subprocess operator $() comes in handy too. In contrast, 'awk' is a text-processing utility with a built-in programming language. Those 5 utilities plus 'gnuplot' are sufficient for data analysis and much of data science. If programming language constructs such as variables, loops, and user-defined functions are needed, a standard *nix shell like 'bash' provides these features. By the way, I use all of these utilities and sometimes only the full capabilities of ''awk' on a daily basis to perform all stages of the data science workflow, including machine learning. Whether I choose to go awk-less depends on the project. Okay, I am sadistic. By the way, I use Python and R too but when introducing data analysis or the stages of the data science workflow up to exploratory data analysis. The 'bc', 'cat', 'curl', 'head', 'grep', 'sed', 'tail', 'wc', and 'wget' utilities allow for an interactive, step-wise refinement approach known as REPL without the cognitive overhead of programming language constructs.
1:40 "why bother with grep? grep is sort of a neat program, but it really only does one thing"
have you heard of the unix philosophy?
I see where this is going.
_Glenda’s so cuuute!_
I didn't know the "tolower()" parameter in the awk command. I learned it with you. Thanks man. Greetings from Brazil.
using awk it feels like im using dvorak keyboard because of those special letters, eapecially when i need to finish jobs faster, i only use awk for missing features like print. time is gold.
As an awk lover, this video put a smirk on my face. But seriously, awk really shines when manipulating data in a limited number of potential sources. In real life scenarios, searching through large git/mercurial repos, where there are thousands of source files and a lot more build files, awk is not really your friend. All searching operations will be much much slower than grep. Grep will also be much slower than SilverSearcher or RipGrep, but those tools are not a part of any Gnu/Linux distribution.
But you are absolutely on point about replacing sed with awk. Even when awk syntax looks longer, it is more meaningful and readable.
Master this is reveling to me now I'll use AWK for my log and Metadata work.
The reason we use cat at the beginning is that it moves the file being referenced to the start of the line before any processing commands. Otherwise you've got grep followed by a random pattern match, then the file you're processing, often followed by a long sequence of other things. It's easier to read and conceptualise when you have "take file X | search for content y | ..." especially if you're chaining many commands together and later you have more greps or sorts and you switch the mode of their use from including a filename to taking stdin. It makes the $1 stick out and easy to understand rather than burying it somewhere down the commandline.
Enough of the bloat you old goat.
You don't need Cat or exfat.... a head or sed even if the font is in red.
Don't use grep, or WEP and especially don't use zipgrep.
In the terminal or youtube as you gawk, while you listen to DT talk
He says all you need is AWK.
-----Dr. Suess
If I ever have kids, I'm reading this to them.
beautiful
Wow. Just....wow.
For replacing head, you only need: "awk 'NR
why say 'content creator'? Just name Luke Smith. good video apart from that dt
YOu know, not to shout out the competition... OMG how low-key, jealous and petty it is.... I cant help it, but to me tells a lot about the character.
Felt almost insulting tbh.
Chill Luke is giving him ideas for new content and he's not bothered to pay respect or acknoledge a gnu user fellow.
Great video... but you answered yourself really. The simplest solution (or more readable) is usually the best one... so which do you prefer to use in a script?
This: head file.sh
Or this: awk '(NR>=0 && NR ... etc etc about 20 more characters... ' file.sh?
Which would be more readable and maintainable while also allowing fellow programmers looking at your code to 'get it' quickly?
awk is awesome, I use it to update pip installs
pip list --outdated | awk 'NR >2 { print $1}' | xargs pip install --upgrade
Nice!
@@DistroTube forgot xargs, but, edited the post
@Peter Andrijeczko for using awk? No, I was sharing an example for newcomers. Its aholes like you who scare people away from linux man. How in the F, am I going to look for congratulations for simple line of awk, when I've written emulators in ASM for both 6805 and 6502. Go bug someone else if you're the bored man.
@Peter Andrijeczko this isn't a penis measuring contest, I'm just saying, you came off as an ahole and you know you did.
I'm sure you're way more capable than me but honestly, years don't equate to ability. I found and exploited a security whole in nagravision cards used by popular tv providers. A loop counter was off by one and overwrote the counter itself in ram. The loop itself was decrypting memory with IDEA cipher with a key of zero for a few loops then they got over written with a constant key. I had to encrypt my payload with 2 keys and then used another command to cause a jump to the payload in ram. I've never worked with linux as an employee and I'm self taught but. People will have their strengths and weaknesses all over the place. Like I said this isn't a pecker measuring contest.
@Peter Andrijeczko this why i wrote emulators for both the 6805 and 6502, these cards where using these same architectures depending on the rom version. for the 2nd part of the exploit I wrote code in C++ so i could brute force a set of bytes that would give me a jump to my payload since what was being overwritten in ram was psuedo-random. I need a jump to anywhere from $0080 to $00D0 in ram. but yeah im not pro or expert.. I learn as I go and when it comes to linux I'm still a beginner. Hell I stick to debian or ubuntu for that very reason.
BEGIN { print "Luke Smith" }
😉
sed "1 i Luke Smith"
The one caveat with replacing grep with awk is that it doesn’t support extended or Perl-like regex the way modern GNU grep does.
Did he just declare war?
I'm using only awk right now for the Advent of Code. It can really do anything.
*grabs popcorn* I am enjoying seeing Luke and DT go back and forth with these commands. It's like watching an awesome game of chess.
sed s/Luke/DT ... *shots fired*
It's a tool in the toolbox. It's good to be aware of what the strength of each tool is, and use it in a meaningful way. -Capt. Obvious.
How would you do to substitute "grep -A" or "grep -B" with awk? cause that something I use a lot in my daily life when I'm extracting data from one particular kind of files I have. Is there any way to do it with awk or any different and more efficient wat than "grep -A/-B"?
Thanks!
Thanks for the comparison but to me it remains more tedious to use awk before ripgrep, grep...you name it..in most cases. Anyway still learned something new, thanks for that.
awk is great for work with data tables
True, that's really where it shines.
Blargh! Captain Pedantic here... it's not "case insensitive" it's "case indifferent". It's the same difference as "uninterested" and "disinterested".
awk '{print $0} (NR==11){exit}' new.sh seems easier to understand and use than what you used for the head example . great video by the way!!
Using awk for everything is surely doable, but sounds like a gimmick to my ear. Interesting, though.
I think that's the joke. There is nothing wrong with using cat or grep if that's what you're used to doing.
"We can replace head with awk"
7:37:
Me: Nope
6:10 "finds every line that contains 'dir' " : $1 finds every line with 'dir' in the first item
echo "When The Band goes Filing in" | awk '{print substr($1,1,1)substr($2,1,1)substr($5,1,1)}'
Is it „wtf“?
@@eNNercY Yes :)
@@eNNercY LOL unix / linix is cool ... but scripting is always quick and dirty ... but will be in no way fast ...
Arguing between cat and awk is just clickbait !!
7:42
How that did appear? What kind of sorcery you are doing?
I shall give thee my like Mister Derek Distrotube!
The magic of Kdenlive. I wonder if awk can edit video? Hmm...
@DistroTube I was wondering, did you manage to customize stumpwm to the point you could use it or did you lose interest?
I haven't logged into stumpwm in awhile. I may get back into it at some point. Was too damn frustrating to deal with right now.
Why use head when with an extra 40 characters you can use awk to do the same thing. It took him that long to type out the command he jump cut it.
As for head replacement, you can simplify command to "awk '{ if(NR < 11) print; else exit }' new.sh" :)
@@bigpod I meant simplification of command he provided as replacement for head. But, in general, it is quite simple command, there's really not much to it :)
@@bigpod Sure it is, can't argue with that, but I think that was not a point of the video :)
This was an interesting video. However, if I want the head of a file, I will type: head t.txt not the long awk command you suggested. Have you counted how many keystrokes are involved in the awk command? The awk utility is only to be used for complex operations, which other simpler utilities cannot perform. It is not a replacement for all other commands.
One less dependency............but a few extra characters to invoke. This is the simplest possible example of a tradeoff in programming I can think of.
or just write your own text parser in c/c++ XD XD XD
I seldom use cat. Usually "less" or "more" for large files, sometimes with grep. The pipe is the Unix way.
If you have a long chain of commands, it may assist in readability to cat file | at the beginning.
Looks like a good to possible replace a lot of grep and sed uses, by when on earth would you replace cat with awk, that's working harder not smarter?
Awk is 671KB, plus 34K for the links from /usr/bin/awk -> /usr/bin/gawk. Cat, grep, sed, head, and tail are 475KB combined. Awk is teh B|0@t!
Grep is still the least cumbersome and most elegant for every day searches. The bloated commands, sed and awk, are better for more specialized cases. Awk is exceptional when it comes to tabular data.
Imagine a carpenter making a video on using solely one specific hammer for all. You’d laugh about that.
I wouldn’t abandon all other CLI tools.
Besides the tool to replace them all .. wasn’t that Perl?
Have you ever maintain a GNU/Linux package, and which one?
It's like why use _reduce, map, forEach, filter, every, some, includes,_ where you can use _for loop_ for everything.
Pretty soon you'll be browsing the web with awk and curl /s
Laughs in busybox
me, watching videos on awk because my coworker who retired always used awk, and in the end he used awk for everything.
so I had a really good laugh in the end :D.
I use awk a lot in my scripts. But one thing I wanna do is for example printing from the first column to the end of line. It is very basic thing but I couldn't find the syntax for it. Can you guys help me out?
Use cut instead...
$ cut -d' ' -f2- file.txt
War begins .
100 years from now:
-- granpa, how the WWIII begun?
-- you'll see grandkid, all it started with this Linux videos on TH-cam..
Awesome 😊
and now for the bigger question... Y use vi when u got awk, sed, grep and the whole 9 yards or bash utils. nano ftw!
It would be really cool if you showed things awk can do that others can't. It sounds like awk is bringing a sledgehammer to hammer a small nail a lot of the time. It's less characters typically to do something with sed or grep, so when they do the job (which is often) why would you type more characters? I would only use awk when I can't do the same thing faster. Is there any performance difference between these commands?
I like this channel
Do all the awk commands you showed in this video are POSIX compliant script?
I stuck with POSIX for this vid. There some non-POSIX ways of doing alot of this stuff too.
Awk is great and is one of the commands I love the most... Only problem being I don't even know how to use it.
If you do not use awk yet consider gawk (GNU awk) or nawk (New awk). They are both more powerful.
"grep only does one thing" Linux Axiom "do one thing do it well" 🤔
The legend of Zelda : link's AWK-ning
I think Luke has a point (performance) whereas DistroTube is looking for some topic to talk about
Even I have this kind of doubt in my mind
Well although awk can replace cat or grep even if it's bulkier, saying that it can replace sed is quite a bit statement. I do a lot of parsing using sed quick and simple and I can't see how would you trade this to an awk experience. For splitting lines into columns it's fine though.
awk ‘{print $0}’ file.txt was even more AWKward than cat file.txt | grep “search”
Head can be a lot shorter: awk 'NR
It's legit to use cat to just print out a file.
Cat is bloated. Use "
@@bendover4728 No, use '>'. Files are bloated.
@@TheGruselmops LoL! Yo can use 'sudo find / -type f -delete' to get rid of bloating..
@@bendover4728 whahaha
ohyeah!! Thats my favorite Anime-Crossover :3
grep has a builtin -r, or recursive, flag, which is very useful. These days I use "The silver searcher" though.
Check out Ripgrep!
@@xthebumpx I actually have! I use this vim plugin for searching (vimgrep is so slow) that uses ripgrep. But on the command line I prefer to use ag, even though rg is definitely faster.
2:54 Typing `awk '{ print $0 }' new.sh` is bloat. Just type `awk // new.sh`
I may need to test this one...
$ time awk '{print $0}' new.sh
real 0m0.005s
user 0m0.002s
sys 0m0.002s
$ time awk '//'' new.sh
real 0m0.003s
user 0m0.003s
sys 0m0.000s
@@DistroTube For full autism, make sure to time how long it takes to type each one. :-D You don't need quotes around // in this particular case.
Why use awk when you can use perl?
10:16 “if you’re going to be an autist...” of course he was being sarcastic, guys.
How about using the right tool for the job? Usually that's whatever requires the least code.
cat and grep and more and tail are the basis of UNIX systems. Do one thing and do it well. Pipe through the different things you are doing. For me it is easier to remember that cat echoes the file and grep gets the input and filters it in that to remember that grep has all those parameters one of witch is the filename. It sure has ANDs also, yet I use to pipe one grep to another if I am filtering on two conditions. In any case, the examples you give are longer to type and harder to remember that easy cat/grep equivalents.
Offtopic: After 30 years of using grep for different piped tasks i discovered "grep -r 'someCodeSnippet'" that looks into a folder and shows you all files and file lines that contains a given text. That is soo usefull, I cannot believe I never used it.
Its sort of like using shovel instead of spoon imo
I use grep, when I need to grep only.
sed 11q is not the same as head it seems. Shouldn't it be sed 10q?
Big brain time: trade awk for perl
echo "cares one No" | cat | cat | awk '{ print $3, $2, $1 }'
Ha ha ha.... awkward ;-)