Lecture 2: Shell Tools and Scripting (2020)
ฝัง
- เผยแพร่เมื่อ 31 ม.ค. 2020
- You can find the lecture notes and exercises for this lecture at missing.csail.mit.edu/2020/sh...
Help us caption & translate this video!
amara.org/v/C1Efd/ - วิทยาศาสตร์และเทคโนโลยี
![[Live สด] การออกรางวัลสลากกินแบ่งรัฐบาล งวดวันที่ 2 พฤษภาคม 2567](http://i.ytimg.com/vi/GO21E3C-kXE/mqdefault.jpg)
If anyone here is having difficulties understanding him, you can turn on English captions which work perfectly.
2x speed :(
Basically I can’t understand what he had said without captions....
I can understand well
the lecture notes saved me
why did i scroll down at 47:40 time stamp
00:01 - introduction
01:00 - defining variables: = for assignment, $ for accessing value
01:25 - spaces in bash reserved for separating arguments
02:15 - defining strings: "", ''
03:25 - function example: mcd.sh (mkdir and cd into it); $1 = first argument
04:46 - source
05:24 - reserved commands: $1-$9, $?, $_, !!
07:35 - detailed explanation on $_ (0 or 1 value)
08:24 - true, false commands
08:43 - || OR operator
09:26 - importance of hydration
09:32 - && AND operator
09:53 - ; command separator
10:15 - getting command output into variable: foo=$(pwd)
10:45 - command substitution
11:15 - process substitution
12:15 - script example: example.sh; $(date), $0, $#, $$, $@; grep command
14:30 - redirecting STDOUT: > /dev/null
15:02 - redirecting STDERR: 2> /dev/null
15:37 - comparison operator -ne; other operators like -f etc.
16:44 - >> to append
18:27 - * for any sequence of characters
19:00 - ? for any single character
19:56 - expanding with {}
21:48 - diff
22:37 - python script example script.py; sys.argv in python similar to $1...9 in bash
25:01 - importance of hydration
25:30 - shellcheck
27:00 - remark on loading scripts into shell (sourcing)
28:00 - tools: man, ripgrep, tldr, find, locate, grep, ack, ag
41:20 - tools for search: history, Ctrl-R, fzf
44:53 - importance of hydration
45:12 - tools for navigation: tree, broot, nnn
importance of hydration :)
@@saisaske1 lol I got confused as well at first XD
Thanks
thanks a lot ☺
Thanks! very useful summary!
The amount of info he squeezed in this short amount time is staggering. These tips will save you countless hours. His accent is rough but English captions work well. Got a beautiful terminal now with loads of nifty tools after a day of intense googling
The series is fantastic - but I fear that after Lecture 1 the jump in difficulty here will put off a huge number of people. If you're struggling with this, move on to the following lectures, they are all much easier to follow.
Yeah, I hope so. This one was really STREAM OF CONSCIOUSNESS feel. Thought I was learned from James Joyce. But, yes, I did enjoy it and learn so much. I just wish that I could have had a little more arrangement, because I had to pause so much -- to look up what he was talking about ... because there were no screenshots nor references on screen. And, I couldn't really understand what he was saying exactly -- in some places.
Thanks a lot for providing your reassurance in this comment - I've been trying to follow along with the exercises at home, and I've been messing around for an hour, but that's only translated to five minutes in the video!
It really helped to look at the lecture notes at missing.csail.mit.edu/2020/shell-tools/
@@ConorJTobin Yeah, same with me, when I watched it first time, it took me long time to keep up with Speaker. Because, I actually tried and verified everything that he did.
Thanks man
There should have been made a distinction between standard/shell commands and individually installed tools (like e.g. 'convert' or 'tldr').
It's good to mention that 'cat' is a command that concatenates files that are specified in the arguments, but also when no arguments are passed it takes STDIN as the (sole) "input file". By mixing examples between Lecture 1 and 2 where both use cases are shown, students may be confused about the role of process substitution "
Thank you for making these videos available. These really do help expose all the critical loose ends of computer science. I’ve been lucky to have some professor who quickly went over a tool or two but never a dedicated lecture on the basics in and out.
This lecture requires time to be digested but its content is extremely valuable. When starting learning Linux and scripting the topic is so broad that you can easily get lost. This lecture is a solid list of things that can be used to start the journey. Thanks!
It took me two days to grasp the contents covered in this 48 min lecture :'). Thank You so much for this course!
Thank you very much team@missingsemester for improving my skills! Keep up the good work🙏
Wow, I thought I knew the shell quite well, but clearly there's still so much more to learn. This is a great lecture, even if the instructor can be a little tough to understand at times (although the captions are pretty well spot on so that helps).
I didn't know tldr, what a great progrom which make linux and unix lot easier, thanks for sharing!
Thanks for the tips on the subtitles. I thought only auto generated would be available but happy it's the real deal there. I really didn't want to miss out on all the cool information. Took me far too long to realise what 'bass' and 'cell' were meant to be and that it's actually 'command' and not 'comment' that he is saying. Great teacher, just added an extra layer of confusion with some of the pronunciations.
This is such an amazing series. Administrating Linux servers is one of my many responsibilities at my job, but I have been stuck for quite a while now on my existing workflows and commands I know. This opened my eyes for many techniques and commands I did not know about.
The lecturer actually comes up these very good interactive examples to demonstrate the quirk of shell. For example, the conditional execution using && and || is based on the fact that 1 is treated as failure and 0 is treated as successful which is counter intuitive for beginner. His example actually shows the reasoning behind this. I have seen most other tutorials just showing some commands without explaining why.
If others feels this lecture is unclear, then it is because it requires you to actually think. Not a cookbook style lecture, but fundamental understanding of why. If you are struggling, then you have not spent enough time or have done enough hard work to think.
As for the English part, his English is pretty understandable by US standard. In the real world software development workplace culture, especially the outsourcing culture, you will inevitably communicate with Indian, east European, Chinese, etc that speak imperfect English.This is also part of thing we need to learn and accept.
The best channel in TH-cam
Initially, I was having a little difficulty understanding (because of different accent). However, I tried a little bit more and now I feel pretty comfortable :)
Great job
This was a really incredible presentation. It took me twice as long or more, because I had to keep pausing it, thinking about what he said, and reviewing to understand the wording/s. It'd have been nice if he would have paused, said it clearly in some places, and then began a new topic. But, I wish it'd be a little more arranged, like a sequential presentation versus stream of consciousness. Yet, don't take what I said negative. I got more from this than sitting through a week of college lectures. Thanks for sharing these videos. I hope to find more like this from the MissingSemester.
Also, let me add that I looked up EVERY SINGLE REFERENCE that he gave to other tools, read about them, and thought how they would be used. So, I just wish that I had a download of this presentation in (like) a PDF or Powerpoint, where it could useful to review later. But, I wasted 2 to 3 times as long to make all of the notes myself to make sure that I could remember it.
Glad you found it useful! FYI lectures notes are online with the examples and links to mentioned tools and concepts. missing.csail.mit.edu/2020/shell-tools
@@MissingSemester Again, I loved it. I learned so much from this guy. He did a really good job, but I just wished that I could walk-away with less effort than I did. If that makes sense. Because, honestly, I paused and reviewed every website that he suggested, and MAN OH MAN is there some good tools out there. New to Linux, but man, I see what I was missing from the world of Windows.
Every lesson I've started was with the same complain, 'too much to learn'....... But becoms easier as I keep on trying .... Thanks for the great lesson .
well, that escalated quickly
Great lecture! What websites do you use to keep up-to-date with new tools for shell, etc.?
missing.csail.mit.edu/2020/shell-tools/
Amazing talk. Which theme are you using in your zsh terminal
I'm working through these. Thank you.
Great work! Thank you for making this content free!
Since this is an introductory course I would have preferred he used simple bash and not zsh since I do not really get the difference, or teach us how to do it and make the terminal as pretty as his.
Its the same with zsh have more human friendly collors .
Fantastic class. Although it's a little difficult for someone who is just starting with bash in my opinion. I was able follow along probably because I'm already familiar with many of the concepts that are explained. But the explanation is really awesome. It's definitely my go to reference of bash for now on. Congratulations MIT and thanks for opening this to the World.
He is explaining bash but using zsh -- similar but not the same. This may be confusing for beginners. His examples are also unnecessarily complex.
Yeah, between the over complicated examples, his accent, and the break neck speed, I was having a hard time following along.
Oh, I definitely appreciate that it's free! My purpose wasn't to just complain, but to let them know so they can adjust their content in the future. If I'm not being effective at teaching something, I 'd like to know so I can adjust; and not waste my or the learner's time.
@Dat Nguyen There's nothing wrong with someone stating they're having a difficult time understanding the lecture; feedback isn't always positive. Everyone appreciates that they're providing free lectures, but without honest feedback, how can they improve?
@@LipStickBPopin And that's why you didn't get into MIT. Remember that the lecture is actually not for us.
@@thechronic4twenty I don't see the need to insult me. Nothing I have said has been rude like your statement. Please understand that even those who accomplish great things also have room for improvement. My intention here is not to hurt anyone's feelings.
very nice series - big jump in difficulty from the first lecture, could have been a bit more progressive. Also it would be great if we could download some ample files to be able to replicate in real time
Thanks for this amazingly useful lecture!
This is awesome, thank you so much! I really appreciate this!!
If you're on a Mac, some of the tools he mentioned (tldr, shellcheck, ripgrep, etc) aren't installed by default - easy to fix tho! While it may be possible to download and install them via a browser and the finder, please don't do that. Instead, instead install the homebrew package manager (brew.sh) and then use it to install all of your command line apps. ie I installed tldr with "brew install tldr". Even better, you can easily manage your installs with "brew list" or "brew uninstall".
Scott BREWer :P
Great initiative and superb work by Anish, Jose, and Jon. Quick question though, is there a solutions page to the questions? Would be great if there was
Tips: if this accent is kinda hard to understand immediately, you can use CC. This will make it much easier to follow.
Thanks a lot for sharing such a great series. I have learned many useful things from this lecture. I think the comments complaining about his speech are unfair.
awesome lectures again. Thank you
Awesome content!
Awesome Lecture! The Instructor has good knowledge on the topic. People here are complaining about his accent. But its fine to me! The Amount of Knowledge shared in this short video is so amazing! Kudos to the instructor and @missingsemester to sharing these lectures to students who can't experience such quality of lectures! Heartly Thanks!
Lecture was really good, but without captions was really hard to understand lecturer's speech..
What keyboard shortcut are they using to preview the expanded command? For instance how does he visualize "touch foo1 foo2" after typing "touch foo{1,2}"?
For how clear, structured and beginner friendly parts one and then three are, this is really quite a poor followup. A great resource in its own right, but as part of a series it seem quite out of place. When Lecture 1 opens with explaining what a terminal is and how ls works, this is a heck of a jump.
Using a customised zsh shell when teaching a bash lecture is a bit of a no to me too.
missing.csail.mit.edu/2020/shell-tools/
WhiskeyBurns thanks. I was also struggling a bit with his exposition
The lectures weren't meant for complete beginners. It is meant for people who have work with the shell before and wanted to deepen their understanding.
@@brucejimenez814 But the structure is questionable.
These lectures are not for beginners. It feels more like a compilation of things that you can learn, you know. You won't finish these playlist mastering version control nor shell scripting, but you will know that those tools exists, are quite handful and that if you follow the tips they gave, you can learn those things.
I really like the lectures, thanks @missingsemester. Is there a way to find the solutions for the questions/homework in the lecture for comparing and guide if we are stuck?
Solution set would be cool!
I love this lecture
im struggling to keep up but can infer most of what's taught from prior knowledge - this lecture needs more explanation of basic concepts. but i do appreciate the lecturer's efforts in the sheer amount of information provided: all those examples means he spent a lot of time putting the lecture together, but i think he's taking a bit too much of his own knowledge for granted ;)
anyone knows how to use the letters of the bash with that colours?? In his bash diferent commands has diferent colours.
What shortcut used in 20:30 to expand foo{,1} to foo foo1?
AWESOME!
Nice lecture!.....
If possible can you post the zsh config file please, your terminal looks cool!
Thanks! All of the instructors' dotfiles are already online and linked in the lecture notes under missing.csail.mit.edu/2020/command-line/#dotfiles
Thanks for the reply! What I'm looking for is instructor's computer terminal setup file(zsh config - theme, plugins etc.). I couldn't find them in lecture notes.
They are linked at the bottom of the section I linked to. Look for the text "All of the class instructors have their dotfiles publicly accessible on GitHub: Anish, Jon, Jose."
Can somebody confirm if the ~/m/tools >>> ...{master } screen is git repository from git hub? {I am new to vim, git and Linux).
Great Lecture,
Wondering how can I set up my shell like the one you have?
I have "oh my zsh", how can I update the prompt like yours?
Hey I can help you to do so I have done something similar
@@abdoulayebalde2139 please tell me..
Hi, I also want to know. I have used OhMyZsh with powerLevel10k. @Abdoulaye Balde Please help
Someone said zprezto, somebody else said ohmyzsh with floating around
Anyone know what formater he is using in his terminal? I really like the colors and the code completion and execution status.
Most likely zsh and oh-my-zsh
Sir, what's your oh my zsh theme, please?
man this session was hard!
anyone know what theme he is using?
What is that zsh theme?
20:23 is there a keyboard shortcut in bash to expand this curly brackets as in zsh to see the combine command ?
tab key
Hi, can someone explain how at 15.06 "2" is used? I did not understand how it is used for the standard error. Thanks
I think that idea is very simple:
each program has 2 output streams (may have more). The first one is stdout, which usually gives output on the screen, for example, echo "Hello" gives Hello.
The second is stderr (standard error), the stream that says about execution is succeeded or not. If yes it gives 0, which means 0 errors, else gives 1.
So simply echo "Hello" has 2 output streams. stdout is Hello, stderr is 0.
The 2> /dev/null means redirect stderr to the /dev/null
The > /dev/null means redirect stdout to the /dev/null
@@kemalbidzhiev1948 Thanks. Makes sense now :)
It makes me sad to admit, that I struggled to understand what he was saying and that he was making unnecessary references (such as the $-sign and its representation in other languages) here and there.
You will get used after a while. There are also excellent subtitles.
The "unnecessary" references you say aren't unnecessary at all.
You can have your opinion. :)
@Username More people speak spanish in America.
anyone know that where i could config this teachter's shell color？
Can you please tell me exact details about this setup? Zsh? Themes? Plugins?
specially l liked the git plugin, command highlighting even inside man page there. Wow
look for zprezto
Any updated stuff?
this is concentrated essence
i'm wondering how -exec rm {} \; i mean after rm works, not mentioned
`-exec` runs the following command on each of the results of the `find` command, substituted at each `{}`. You need to tell bash where the end of the command is with a semicolon, but since semicolon has special meaning in bash, it needs to be escaped here as `\;`
which theme are you using over zsh?
Jose is using github.com/sorin-ionescu/prezto. His configuration of prezto is available here, in his dotfiles: github.com/JJGO/dotfiles
@@MissingSemester Thanks!
Total beginner when it comes to bash scripting, but I had to figure out why his program was not working for me. I had to rewrite that
if statement as if(("$?" != 0)) ; then
#rest of the statement
It took me a bit to figure this out but I can imagine some other students interested in learning shell scripting and then getting discouraged at this point :(..
What is the shell being used in this class and what is the theme ?
pretty sure they said the shell being used was bash but I don’t know the theme
@@jimmoriarty3381
No it was zsh.
Z shell
@@xrafter true
didn't they use bash in the first vid tho?
and I know for sure when they redid this lecture they used bash
@@jimmoriarty3381
Bash was the target however this guy for some reason uses zsh
you don't need to install all of these or learn all of them, when these tools come inside your daily use and eventually you'll get the hang of it.
How to install broot on linux ubuntu 20.04 ?
I think this was great, and bash is just weird, so do not blame the lecturer.
There is only one way to learn the shell and that is to use the shell.
At 16:55 in line 7 there is “$@“. In the resource file $@.
In line 11 in the video there is “$?” in the condition, in the source code at the homepage only $?.
I tried both ways, but the first 5 rows worked for me.
I installed then ZSH and them changed the first row to #!/bin/zsh, but it didn’t help.
No offense, but I found the first video easier to understand (the accent). It was well planned
becoming step by step a bit more complex. The pace was better there as well.
Here we suddenly jump into a lot of new commands, not testing them separately first.
(I have Ubuntu 18.04.4 LTS)
The [[ and ]] is a built-in operator so maybe zsh won't work with it.
@@puneetgupta87 love your song choices mate
I don't understand what the teacher does to expand what's in between the {}.
For example, at 21:31, how is touch {foo bar}/{a..j} expended in all the variations foo/a... bar/j before he executes the command ?
tab key
I'm rather new to Unix. I tried the commands (shellcheck, tldr, rg, fd) and none of them are on my system. I know that I can install them, but I don't understand the mindset that it is "OK" to blindly install packages on production servers. In the Windows world, I wouldn't even consider installing some unknown software (viruses, etc.). Why is it OK to install these utilities? Especially when it requires sudo to install (which again tells me it's something you should give serious thought to before doing).
Well Welcome to Unix. Everything is open source here. If there would've been issues, people would have reported and stopped using it. Moreover, you can always build the tools from the source just to be safe :D
is it worth to studying bash script? nowadays python is heavily being used.
Yes, it's worth it. Bash relies on the standalone utilities (like ls, cat, and so on) to a much greater extent than Python ever would, and it's actually designed to do so. Python replaces all that, usually with much slower alternatives. Python was also never intended to be used as a replacement for Bash/any other shell.
An analogy would be if you were asking whether you should use a shopping cart or a truck. It depends on the task at hand. ;)
@@herrpez
Unknown refrence .
I almost did everything in lecture 1 in Mac, Can I do the rest of the course in Mac book?or I need Linux?
As far as I know most of this bash stuff will work because Mac and Linux are Unix-like, but if you want to be sure you can do all of it, I recommend trying Linux in a virtual machine or through some other method, it's very easy if you look on TH-cam. If you don't want to I'm sure you'll be completely fine though, good luck!
Sure, I have windows but I use Git Bash or Ubuntu from the Windows Store to follow along on my machine.
which keyboard stroke expands curly brackets command?
With the right completion setup, tab completion expands ${variables} along with other things, like $(commands)
Not sure about parameter expansion (curly brackets), but ALT-8,SHFT-8 will expand glob patterns.
if you use zsh, then add setopt BRACE_CCL to .zshrc file, after that you just press TAB and brackets will expand
How is this "history substring search" (min 44:00) called?
it's history-substring-search plugin for Zsh. github.com/zsh-users/zsh-history-substring-search . Or if you install oh-my-zsh (ohmyz.sh/), it comes with this and many other plugins, themes etc for Zsh out of the box.
@@destsk Thanks
Where he was clicking to expand the globing pattern Inline ?
tab key
6:39 sudo !!, where the !! auto replace with previous command does not work on my bash, after i type !! and press enter, its execute previous command and output the result, it's only work on zsh?
You don't have a history file like .bash_history .
Usually when you start bash and zsh for the first time it will ask you do you want it to setup this for you usually you will say yes.
why "mcd test" gives you the "~/m/t/test" while the pwd is "~/m/tools"? Shouldn't that be "~/m/tools/test"?
It's probably a matter of his terminal, in that it shortens the paths. When he does cd .. he goes back to m/tools, meaning that the test directory was indeed created inside the m/tools directory. Cheers!
I think that he abbreviated his $(pwd) in .bashrc file like the one here
check this
unix.stackexchange.com/questions/381113/how-do-i-shorten-the-current-directory-path-shown-on-terminal
check this too if you interested, he showed how to customize your own terminal and how you can use other peoples dotfiles
th-cam.com/video/c5RZWDLqifA/w-d-xo.html
```pip install tldr``` , to install tldr.
Hope you will find this helpful.
Does anyone know what shell he was using? It's colorful and could automatically complete the commands.
Zsh the name is floating around.
Same question
ZSh, but Theme?
Does anyone know what terminal he's using? I know it's in linux..
It looks like he's running macOS, and the shell is zsh.
what language is this in?
Hello guys, how to make it display the whole command when I input like touch {1..9}.sh I can only run it but can't see the whole command. Thanks~
I think he's using some sort of extension,
I don't know which one but if you are interested in an good extension for python programming there's one called as "KITE AI" which auto completes the code for you, it supports editors like vim,vscode,pycharm etc
Just wondering... How did Google fully understand what he said and generate captions? Marvelous technology....
It's probably a human being who edited the transcript.
Is anyone doing the exercises for these? I am stuck on number two. Is there an answer guide anywhere?
You could try storing the path in a file with the marco-program and read it from the file(with it's absolute path) with the polo-program. If you don't know how to store and read variables from files with bash, read this: stackoverflow.com/questions/5965737/how-can-one-store-a-variable-in-a-file-using-bash
@@noway4517 Thanks a lot! I was trying to do something similar to that but didn't have exactly the right idea. Appreciate your help.
Any way to get the author shell configure? (theme)
Ditto that! I'm a big fan of the theme they're using
Email and ask for that portion of his .bashrc file.
Milk Phongtharin The theme looks pretty darn similar to what you can get with prezto (github.com/sorin-ionescu/prezto)
42:22 Control+R, command history search
How did you guys solve the marco-polo problem?
You could try storing the path in a file with the marco-program and read it from the file(with it's absolute path) with the polo-program. If you don't know how to store and read variables from files with bash, read this: stackoverflow.com/questions/5965737/how-can-one-store-a-variable-in-a-file-using-bash
I'm guessing you came across similar problems as me, I know this is late, but this continues to be useful to people 3 years later, so, here goes nothing!
I created marco and polo and they did exactly what I needed them to, except my directory wouldn't change back. That's when I realized what the difference between sourcing a script and the functions within meant. When you run a script, all the commands are followed properly but they are all executed in a brand new shell, and the shell is closed after execution. Thus creating a function and sourcing it into our current shell, and running them in our current shell will make them work. I know you're probably past this issue already, but to those who aren't, I hope this helps!
The OS parse #!, NOT the shell.
Good catch, it is indeed the kernel that reads the #! part to determine what program to use. I updated the notes to reflect this.
Please please please share your dotfiles. The shell is awesome
All of the instructors' dotfiles are already online and linked in the lecture notes under missing.csail.mit.edu/2020/command-line/#dotfiles
@@MissingSemester this course is amazing, I can't believe it
That autocomplete in the command tool is just annoying.
how is he fish like autocompletions etc while using the bash
how is he getting*
Zsh not bash
Zsh have plugins for that
Too bad xargs(1) wasn't covered, together with find -exec. It is VERY common in shell scripts.
you know it is not easy to follow when he use foo bar as example.
Hi guys, how to understand 2> in the sentence "grep foobar "$file" > /dev/null 2> /dev/null". I tried many different ways in my shell, there is nothing special.
there are two streams, one is standard output and other is stderr, the regular output gets printed to the stdout and errors will be send to stderr stream. so 2> is for directing the stderr output to /dev/null which is like a black hole in linux.
Still curious why 2> just replace the /dev/null to some file on you disk and see the ouput.
**for more detailed comparison refer to this answer**
stackoverflow.com/questions/3385201/confused-about-stdin-stdout-and-stderr#:~:text=If%20my%20understanding%20is%20correct,all%20the%20exceptions%20are%20entered.
anybody know how i can make my bash as pretty as his?
Ισίδωρος Τσαούσης He is not using BASH. He is using ZSH. You can install zsh and then use oh-my-zsh scripts to make it pretty as his. Zsh provides a lot more features than BASH to make your work easier.
There is Oh-my-bash available on Github too...you can use it for bash
We need a dollar sign cheat sheet.
if you want to convert multiple png files to jpg.
find . -name '*.png' -exec convert {} "{}.jpg" \;
Impressive. Wish I was better at Linux, like some of you guys in this lecture. But, I hope to get there.
Wouldn’t this convert let’s say abc.png into abc.png.jpg?
jose looks like captain spock
Can't believe I spent an hour to watch this.:) I think I might take more time to try-n-error on those bash commands, especially my shell is zsh that has slightly different than bash.:) Quick feedbacks here: 1.) It will be good if someone put times tamp for each bash command he introduce. 2.) Just wondering if this overview helps students more OR just highlight some practical commands and go deeper with interaction of students.:) Just my two cents...
oh h is silent in spanish, that's why