When I worked in IT I remember we used to do all sorts with pipes. The Windows guy thought it was magic. We will still be playing with *nix OS's in 50 years time.
@@michallasan3695 Bro, pipes are beautiful. I wish every day that other languages had the feature. There is nothing for program flow control so readable as pipes.
@@coreygossman6243 They are now also in C++. I am more of a performance guy than readability guy. Thus, I gladly obfuscate my bash scripts for speed. This is why I do not care much about readability of pipes, but rather about the silly thing that every pipe spawns a subshell which seems like wasting resources to me. This is why I prefer subsequent method calls which, after all, is pretty readable, too.
Dude.... THANK YOU. I wish you continued to create more of these videos more recently vs so many of the rant/vlog videos. Now, don't get me wrong: you do provide valuable info in (most... if not all) of those videos, however you're so great at teaching things like this so succinctly, in easy to understand/follow videos, I simply would appreciate more of them. Regardless, I appreciate all of the content you create and post for the community & any general viewer. But these happen to fall into the "you don't know what you don't know" catagory. Meaning this info is out there for anyone to learn, but the reason we find out about it or understand it better is because you made a video about it and we come across them coincidentally, or happen to be subscribed. Anyway, cheers.
just read the first chapter the "The awk programming language" and you can be productive with it. For git read the second chapter of this git-scm.com/book/en/v2
Thanks! It was kind of obscure to understand pipes the first time around form the text description only. This demonstration would have helped a lot then and even now it is just so clear.
@@rickyoswald Yes it does. You have to set binary and noeol for it to not do this. Try it yourself, create a single line in a text file, save it and cat it and you'll see that it will end in a line break, causing your prompt to appear on the next line, not on the same line as the contents of the file.
Here is a better Awk script: awk '{print substr($2,2,length($2)-2) " " $1}' This script will print the section number as well as the name of the manual. Luke's script only prints the name, and if a manual resides in multiple sections only the first one will be chosen. If anyone has a simpler way of saying "everything from $2 except the first and last character" let me know.
@Luke Great piping! However, one small problem with this example is that there might be more than one man entry in different sections for a given string. For example, "read" is (1) a shell utility that gets input, (2) a Linux system call, (3) a C library function. Each of these has a separate entry, and your script would always select the first one. We can fix it with even MORE piping! man -k . | dmenu -l 30 | sed 's/[()]//g' | awk '{print $2 " " $1}' | xargs -r man -Tpdf | zathura -
For those who using `while read`. Read will delete leading whitespaces in string by default. What I usually do to fix that is making alias for read like this: alias read='IFS= read -r' Empty IFS means that there is no Field Splitting needs to be performed (by default it looks for any whitespaces and splits everything in between into "$@" variable, kinda). -r flag says that string must be saved as is, without treating backslash as special character.
I tried doing it your way with zathura but since I run Fedora, the packages are not the same and zathura was missing some obscure plugin package. There is a way to do it with evince though: man -Tpdf ls > temp.pdf ; evince temp.pdf ; rm temp.pdf So the whole command ends up being: apropos . | dmenu -l 30 | grep -Eom1 '^[^ ]+' | xargs -r man -Tpdf > temp.pdf ; evince temp.pdf ; rm temp.pdf
wifish list | awk 'FNR>2 {print $1}' | rofi -dmenu | xargs wifish connect Wrote my first script ever, yayyyy, Thank you luke smith. so my script takes the output of wifish list (wifish is a frontend to wpa_supplicant) pips it into awk which reads from line 3 and up and only the first column and pips that into rofi with dmenu functinality (you could just use dmenu though) then it pips it into xargs with the flag -r so it won't execute anything if you hit ESCAPE, but if you hit one of the APs it will connect to it. I am sooo happy hehehe
6:50 Why use awk when you can just use: cut -d ' ' -f1 output spacing is predictable so cut might be a bit faster than invoking awk for something so simple. (awk is bloat) unix.stackexchange.com/questions/132313/what-are-the-exact-differences-between-awk-and-cut-with-grep
Funny I was messing around with doing this with groffer (1) the other day.. But I like your solution a lot better! I made a couple changes though: man -k . | dmenu -l 30 | awk '{gsub("[()]", "", $2); print $2, $1}' | ( read c || exit; man -Tpdf $c | zathura - ) a) If you don't want to have zathura open up when the script starts, but only when a item is chosen, you can pipe the choice to a subshell an use read instead of xargs. b) This will always open the first match for a man page found in any section. Thus, it will open groff (1) even if the user selects groff (7). A simple addition to the awk command fixes this by including the section number in the man command. c) If if there's no choice, read returns an error code and bails out before opening zathura.
My mom once shoulder surfed me while I was chatting on IRC (back in the day when it was still cool :P) and asked me what I meant when asking some guy in a support channel, "How fat is your pipe?". I told her, "Woman, return to the kitchen if you know what's good for you..." Later that day, the ISP called, they were tryin' to sell me some crap-ware "security package" and I was arguing with them about something I can't remember, and told the tech, "Buddy, if you send me another one of those floppy disk with your garbage software on it, I'm gonna: dd if=/dev/zero of=/dev/fd0 bs=4k till the cows come home! Just feed us the pipe and stay out of the security industry". Funny day :P
This video inspired me to make make life so much easier. Honestly anyone looking into scripting on Linux, should watch it. I made a more or less one-liner VPN switcher with dmenu, which made my life so much easier.
Would be cool to add a flag to mainsplain to let the user "prechoose" the man page. Maybe "mainsplain -s cd" automatically opens the man page for cd in a pdf.
I actually made this script and use it pretty often, though I made some changes. I noticed that you can only open the default section of any manpage regardless of which one you choose, so I changed it to: man -Tpdf "$(man -k . | dmenu -l 30 | awk '{print ($1, $2)}' | sed 's/ //')" | zathura - that way it pipes "man(1)" or "man(7)", instead of just 'man' for both. Also since I use this on a very old computer, running 'man -k .' takes some time which is annoying, so I created a file 'manlist' that stores the result of that command and update it through my .bash_profile. All in all I ended up with: man -Tpdf "(dmenu -l 30 < ~/.scripts/manlist | awk '{print ($1,$2)}' | sed 's/ //')" | zathura -
@@ajwadjaved9909 just saved and reopened the file. When Vim saw the #!/bin/sh leader it recognized it as a script and turned on the highlighting. That's why :e is more efficient, it gets Vim to rescan the file, thereby reading the script header and enabling highlighting.
I wish I knew about dmenu much sooner. I used to use a zenity based bash function and index files that do what dmenu does. Now I'm gonna have to update a whole bunch of scripts. Thanks ..?
Your videos are great. Thanks! Quick question: you mentioned there are reasons why you prefer using scripts over functions. What are those reasons? Thanks.
Thanks Luke vor this video. Suggestion for the topic of a future video, maybe about job queuing on linux. How to make a queue of jobs, independent of the terminal window. I use personnaly task spooler.
New to your channel, love this video and the others I've seen. Quick question, I'm getting an error using man -Tpdf saying that groff thinks pdf is an invalid device. Is there some additional program I need to get this piece working?
Won't this cause a problem with e.g. complex(7), because this will always pull the first page on MANPATH? I think you might want to further use awk (or if you're me, find a way to do it without awk ;) to pull out the section number and ask for e.g. complex.7.
This is very good pipes example, but man to pdf script has some problems. Take a 'printf' for example - there is printf(1) and printf(3) /man 1 printf & man 3 printf/ man -k . | grep printf //not very useful in this case. wanna pipes? drill, baby, drill ;>
I attempted that one and what I came up with is a udev event that sends a signal to a C program which sends a notification. I did that because a command caused by a udev event runs as root but the notification has to be sent as user
Any reason you are using a hidden name for your scripts directory ( .scripts )? Just trying to understand if this is used for something productivity-related. Thanks
I have two problems: 1. When i run this script before even i choose my man page zathura is opened. 2. I put this script in a PATH folder but everytime i have to source .profile to use it in my bash terminal. Os: debian (sid)
Very cool. But why do we have to write "xargs -r man -Tpdf" instead of just "man -Tpdf". Like we didn't have to use xargs in the command after the first pipe
`man xargs` will explain it. But it takes in stdin and appends it to the command. So `echo ls | xargs -r man -Tpdf` = `man -Tpdf ls`. The man script can't use/read stdin for an argument.
@@ehllie are you on arch? if yes, use the command "pacman -Qi zathura" it will show you what packages you need (optional Deps). I personally use zathura-pdf-mupdf for PDF support (you need only one package for PDF support).
% man: why did you get a divorce?
man:: Too many arguments.
lmao
I tried this and it open a POSIX Programer's Manual. lol
@@sethsrc792 its actually the manpage for get
@@aprameynaganur7734 ?
@@sethsrc792 when you write man why did you get a divorce it shows you the manpage for get
I hope you make more of these basics videos.
Never been so close to subscribing to this channel.
Lol
10:37 "The biggest recomendation I can give you is just git gud"
git: 'gud' is not a git command. See 'git --help'.
The most similar command is
gui
@@gydo1942 git: 'gud' is not a git command. See 'git --help'.
The most similar command is
add
I’m tempted to alias something like that
alias git-gud=“git”
@@uuu12343 lol
When I worked in IT I remember we used to do all sorts with pipes. The Windows guy thought it was magic. We will still be playing with *nix OS's in 50 years time.
@@youtindia Do you have/know of any good source to learn PowerShell?
No magic in pipes, they are just an unnecessarily wasteful way to replace a sequence of function calls.
@@michallasan3695 What a sad boring twerp you are...........
@@michallasan3695 Bro, pipes are beautiful. I wish every day that other languages had the feature. There is nothing for program flow control so readable as pipes.
@@coreygossman6243 They are now also in C++. I am more of a performance guy than readability guy. Thus, I gladly obfuscate my bash scripts for speed. This is why I do not care much about readability of pipes, but rather about the silly thing that every pipe spawns a subshell which seems like wasting resources to me. This is why I prefer subsequent method calls which, after all, is pretty readable, too.
Btw, Friedrich Nietzsche also used Zathura for his pdfs: thus spake zathura
Wow this a very Luke Smith meme
@@Fooftilly bash: zarathustra: command not found
This video increased my testosterone by 53%!
Dude.... THANK YOU. I wish you continued to create more of these videos more recently vs so many of the rant/vlog videos. Now, don't get me wrong: you do provide valuable info in (most... if not all) of those videos, however you're so great at teaching things like this so succinctly, in easy to understand/follow videos, I simply would appreciate more of them. Regardless, I appreciate all of the content you create and post for the community & any general viewer. But these happen to fall into the "you don't know what you don't know" catagory. Meaning this info is out there for anyone to learn, but the reason we find out about it or understand it better is because you made a video about it and we come across them coincidentally, or happen to be subscribed.
Anyway, cheers.
Up next
*What is the worst Linux distro?*
Brian Lunduke
Reeee
I mean... he's not a Linux distro, but he is fucking terrible.
I think he's talking about Lunduke's video on where he answers what the he thinks the worst linux distro is
Everyone knows it's Gentoo, when referencing Lunduke's idea of the worst distro.
@@sbrazenor2 Yah, fuck Gentoo. I use Arch btw
the Windows Virgin vs. the Unix Chad
* Windows Incel
Wincel
Windows just got a terminal lol
The virgin Unix vs the chad TempleOS
@@MrEdrftgyuji RIP Terry
a video on awk would be useful. maybe one on git too
Ray Garner do you need him chew your food for you before you eat?
@@Clutter.monkey yeah...and wipe our ass too, thanks.
lmao dude just read the man page
just read the first chapter the "The awk programming language" and you can be productive with it.
For git read the second chapter of this git-scm.com/book/en/v2
If you can't figure out git... You probably shouldn't be using Linux to be honest.
"man -k ." returned "nothing appropriate" when I first tried to run it. I needed to run mandb as root in order to update the man cache.
Thanks!
It was kind of obscure to understand pipes the first time around form the text description only. This demonstration would have helped a lot then and even now it is just so clear.
Luke exiting vim instead of :set filetype=sh??? Has the bloat gotten to him???
typing letters is bloat
:e
Even better: `:filetype detect`
I'm pretty sure the fastest way is:
:w|:e
@@realEchoz Great!
my pipes were not working properly, getting stuck most of the time, but then I was really happy you made a video on plumbing as well
uh oh someones planning on moving to dwm...
dwm installed, openbsd on thumbnail, YOU FALL AGAIN FOR MEMES LUKE
Absolute chad.
luke pipes em the right way...
Manual of the manual 🤔
It's not POSIX compliant because line 2 is not terminated by a newline character.
3.206 :D
⠀
Vim automatically terminates the last line with a newline char.
@@realEchoz no it does not. it might be an option but that is not the typical behavior.
@@rickyoswald Yes it does. You have to set binary and noeol for it to not do this.
Try it yourself, create a single line in a text file, save it and cat it and you'll see that it will end in a line break, causing your prompt to appear on the next line, not on the same line as the contents of the file.
@@realEchoz >You have to set
Oh so what I said then.
I was like "oh, he's gonna explain fifo files and '' " but no. I was wrong oh so wrong.
Great video. You could make a series out of this format
"man up" could be used like RTFM.
Here is a better Awk script:
awk '{print substr($2,2,length($2)-2) " " $1}'
This script will print the section number as well as the name of the manual. Luke's script only prints the name, and if a manual resides in multiple sections only the first one will be chosen. If anyone has a simpler way of saying "everything from $2 except the first and last character" let me know.
sed 's/^\([^ ].*\) (\(.\)).*/\2 \1/'
For my Ubuntu and Linux Mint brethren out there, to have -Tpdf option you need to have groff installed.
Thanks bro, u just saved me from an hour of figuring out what went wrong.
Thanks Luke! 🤗
very useful! I save my manpages because of portability. or have the man's to review anywhere. Good Job!! Many thanks!!!
YT comment section is bloat.
I really like the syntax highlighting in your terminal. Is that done through st config?
Great, especially, for when you have to print something! Nice video.
such a chad
Typing sounds like a stampeding herd of horses.
@Luke Great piping! However, one small problem with this example is that there might be more than one man entry in different sections for a given string. For example, "read" is (1) a shell utility that gets input, (2) a Linux system call, (3) a C library function. Each of these has a separate entry, and your script would always select the first one. We can fix it with even MORE piping!
man -k . | dmenu -l 30 | sed 's/[()]//g' | awk '{print $2 " " $1}' | xargs -r man -Tpdf | zathura -
Even easier way, using only awk instead of sed, awk and xargs.:
man -k . | dmenu -l 30 | awk 'system("man -Tps " substr($2,2,1) " " $1)' | zathura -
For those who using `while read`. Read will delete leading whitespaces in string by default. What I usually do to fix that is making alias for read like this:
alias read='IFS= read -r'
Empty IFS means that there is no Field Splitting needs to be performed (by default it looks for any whitespaces and splits everything in between into "$@" variable, kinda). -r flag says that string must be saved as is, without treating backslash as special character.
‘Git Gud’ at laying pipe.
Even Mario can't handle that many pipes
I tried doing it your way with zathura but since I run Fedora, the packages are not the same and zathura was missing some obscure plugin package. There is a way to do it with evince though: man -Tpdf ls > temp.pdf ; evince temp.pdf ; rm temp.pdf
So the whole command ends up being:
apropos . | dmenu -l 30 | grep -Eom1 '^[^ ]+' | xargs -r man -Tpdf > temp.pdf ; evince temp.pdf ; rm temp.pdf
Amazing!!! Great help! Thanks a lot!
5:55 I don't think pipes are temporary files (unless Zathura created that file on the fly )
This title is underated AF UwU
What a friggin chad bruh
man oh man is man a good program
wifish list | awk 'FNR>2 {print $1}' | rofi -dmenu | xargs wifish connect
Wrote my first script ever, yayyyy, Thank you luke smith.
so my script takes the output of wifish list (wifish is a frontend to wpa_supplicant) pips it into awk which reads from line 3 and up and only the first column and pips that into rofi with dmenu functinality (you could just use dmenu though) then it pips it into xargs with the flag -r so it won't execute anything if you hit ESCAPE, but if you hit one of the APs it will connect to it.
I am sooo happy hehehe
Dude you are amazing, I love these videos so much. How did you start to learn all of this stuff? Is there a certain channel or blog?
Great tricks, thanks!
Very nice! I love these types of videos
6:50
Why use awk when you can just use:
cut -d ' ' -f1
output spacing is predictable so cut might be a bit faster than invoking awk for something so simple. (awk is bloat)
unix.stackexchange.com/questions/132313/what-are-the-exact-differences-between-awk-and-cut-with-grep
Funny I was messing around with doing this with groffer (1) the other day.. But I like your solution a lot better! I made a couple changes though:
man -k . | dmenu -l 30 | awk '{gsub("[()]", "", $2); print $2, $1}' | ( read c || exit; man -Tpdf $c | zathura - )
a) If you don't want to have zathura open up when the script starts, but only when a item is chosen, you can pipe the choice to a subshell an use read instead of xargs.
b) This will always open the first match for a man page found in any section. Thus, it will open groff (1) even if the user selects groff (7). A simple addition to the awk command fixes this by including the section number in the man command.
c) If if there's no choice, read returns an error code and bails out before opening zathura.
Awsome knowlege... Would like to see more of such things
My mom once shoulder surfed me while I was chatting on IRC (back in the day when it was still cool :P) and asked me what I meant when asking some guy in a support channel, "How fat is your pipe?". I told her, "Woman, return to the kitchen if you know what's good for you..." Later that day, the ISP called, they were tryin' to sell me some crap-ware "security package" and I was arguing with them about something I can't remember, and told the tech, "Buddy, if you send me another one of those floppy disk with your garbage software on it, I'm gonna: dd if=/dev/zero of=/dev/fd0 bs=4k till the cows come home! Just feed us the pipe and stay out of the security industry". Funny day :P
Saddest thing in POSIX: exit status of pipeline is exit status of last command...
Why is that an issue?
too often I find myself wanting the status of the first one... the posix compliant way of doing is a pain:(
This video inspired me to make make life so much easier. Honestly anyone looking into scripting on Linux, should watch it. I made a more or less one-liner VPN switcher with dmenu, which made my life so much easier.
Would be cool to add a flag to mainsplain to let the user "prechoose" the man page. Maybe "mainsplain -s cd" automatically opens the man page for cd in a pdf.
based
nice one luke!
2:28 zathura can read from stdin - woww!!!
I actually made this script and use it pretty often, though I made some changes. I noticed that you can only open the default section of any manpage regardless of which one you choose, so I changed it to:
man -Tpdf "$(man -k . | dmenu -l 30 | awk '{print ($1, $2)}' | sed 's/ //')" | zathura -
that way it pipes "man(1)" or "man(7)", instead of just 'man' for both. Also since I use this on a very old computer, running 'man -k .' takes some time which is annoying, so I created a file 'manlist' that stores the result of that command and update it through my .bash_profile. All in all I ended up with:
man -Tpdf "(dmenu -l 30 < ~/.scripts/manlist | awk '{print ($1,$2)}' | sed 's/ //')" | zathura -
how do you run the command at 9:41 without any terminal open
xargs is glorious
8:49 You can actually do :e to reload the file, without exiting vim ;')
How did he get the syntax highlight on?
@@ajwadjaved9909 just saved and reopened the file. When Vim saw the #!/bin/sh leader it recognized it as a script and turned on the highlighting. That's why :e is more efficient, it gets Vim to rescan the file, thereby reading the script header and enabling highlighting.
:w|:e to save and re-edit in one line
Another cool trick you can do in Vim is type “:cia” to make your text glow in the dark
@@realEchoz Again, truly great! Haha
There are arguments against the idea of pipes but I like them.
What's really cool (for Windows losers) is powershell pipes, which pass whole objects. Next level stuff...
great content! keep it up!
Just want to say this video is freakin awesome
I wish I knew about dmenu much sooner. I used to use a zenity based bash function and index files that do what dmenu does. Now I'm gonna have to update a whole bunch of scripts. Thanks ..?
Hi! Great tip!
on Debian I had to install groff to get man -Tpdf to work.
Your videos are great. Thanks! Quick question: you mentioned there are reasons why you prefer using scripts over functions. What are those reasons? Thanks.
He explained that xargs can read from scripts, not functions
Thanks Luke vor this video. Suggestion for the topic of a future video, maybe about job queuing on linux. How to make a queue of jobs, independent of the terminal window. I use personnaly task spooler.
New to your channel, love this video and the others I've seen. Quick question, I'm getting an error using man -Tpdf saying that groff thinks pdf is an invalid device. Is there some additional program I need to get this piece working?
Won't this cause a problem with e.g. complex(7), because this will always pull the first page on MANPATH? I think you might want to further use awk (or if you're me, find a way to do it without awk ;) to pull out the section number and ask for e.g. complex.7.
That's why terminal is more usefull than GUI.
Maybe instead zathura you can use LaTex, with a custom font like baskerville
Newb here. Any way to make zathura open *after* the desired manpage has been selected?
When I run the script and if I press escape on dmenu to select nothing, zathura still opens a blank window. Can I solve this somehow?
This is very good pipes example, but man to pdf script has some problems.
Take a 'printf' for example - there is printf(1) and printf(3) /man 1 printf & man 3 printf/
man -k . | grep printf //not very useful in this case.
wanna pipes? drill, baby, drill ;>
Functions are good. I had one but decided to go with scripts. If you do it right you can call a specific script multiple times
How about script that allerts you when you reach crittical battery level? Btw thanks for the video and Larbs!
I attempted that one and what I came up with is a udev event that sends a signal to a C program which sends a notification. I did that because a command caused by a udev event runs as root but the notification has to be sent as user
#02:21 - you can just do 'pacman -Qqs zathura' if you didn't know about that
10:16 the -r flag for xargs isn't POSIX
you mentioned that xargs creates a sub=shell is there any process that doesn't do that i wonder it's the old fork & exec
Liked just for
Man -k .
Thanks boomer Luke
Any reason you are using a hidden name for your scripts directory ( .scripts )?
Just trying to understand if this is used for something productivity-related.
Thanks
Nice syntax highlighting on your shell. Have you switched to fish??
Sweet fucking lord. One question. When PDF is displayed, is there a file somewhere, or is it read from memory?
wow thats amazing
Subscribed immediately :)
I have two problems:
1. When i run this script before even i choose my man page zathura is opened.
2. I put this script in a PATH folder but everytime i have to source .profile to use it in my bash terminal.
Os: debian (sid)
A command named "man"... Feminist outrage intensifies. REEEEEEE 🐸 👌🏽
Can I get a link to buy your keyboard Luke?
www.pckeyboard.com/page/product/UB40PGA
But why? Why would you need these manual in pdf?
I noticed that as dmenu pops open empty zathura window starts. I presume if you then cancel dmenu it stays open. This makes me a bit uneasy.
😢
awk is pretty chonker tho, so `cut -f 1` is a good alternative for such a simple usecase
the virgin Luke tutorial
vs the chad ` cat /dev/urandom | base64 | lolcat `
I really like your squishy brain
Very cool. But why do we have to write "xargs -r man -Tpdf" instead of just "man -Tpdf". Like we didn't have to use xargs in the command after the first pipe
`man xargs` will explain it.
But it takes in stdin and appends it to the command. So `echo ls | xargs -r man -Tpdf` = `man -Tpdf ls`. The man script can't use/read stdin for an argument.
youre my favorite unix chad
I have an issue with zathura. When piping a pfd to it, it just opens a black window that shows just [No name]
Do you have a pdf backend installed?
@@LukeSmithxyz I have only those packages with the name pdf in them installed:
poppler
poppler-data
potrace
texlive-core
zathura-pdf-mupdf
@@ehllie are you on arch? if yes, use the command "pacman -Qi zathura" it will show you what packages you need (optional Deps).
I personally use zathura-pdf-mupdf for PDF support (you need only one package for PDF support).
You might be missing the '-' at the end, telling zathura to read from standard input.
xargs with an s , not xarg
You could cover pipes with redirections in a future video ^^
-l stands for --lines
cool but no need for dmenu - 'select' is builtin everywhere i think