So many videos on internet but this will have a special place amongst all of them since none of them deep dive with such clarity. I finally understand the magic behind docker and more confident in using it. Thank you very much!
I spent hours researching this specific thing the other day, mostly comparing "containers" to FreeBSD Jails. It's so hard to find useful information wading through all the devops marketing nonsense. Thank you for taking an interest in this topic and sharing your research process.
Great video! The effort put into nifty graphics, scripting, editing and overall quality is *IMMENSELY* appreciated. Most videos on topics like this are just one take (with a bad microphone to boot). Videos like this make containers more accessible and understandable, especially for inexperienced users. You got yourself a new sub!
Had to learn these things the hard way while doing some container escape CTF, this video would have been a very solid starting point. Thank you so much for the effort you put on this, I know the lots of docs you had to review to explain this way.
This really clears it up for me. I just started looking into what a docer is and how it differs from a VM and I started reading about namespaces and I was like "What!??"
A container/namespace is essentially a pocket dimension. You have the main universe or dimension (your computer), but then you can create a dimension within it that can be seen from the main one, but not vice versa.
Congrats Sr. for this wonderful video explaining the underlying "magic" of the containers which definitely helps to understand better the differences between VM and containers and what does it mean in terms of performance etc. Just for curiosity I checked the wikipedia and the "namespaces" feature was introduced in 2002 in the Linux kernel (inspired by a wider namespaces feature from Bell-Labs's Plan9 OS). It's amazing to see how could a "good feature " can create so much around after almost 20 years :D
Great video! I am currently also working with docker and in the beginning had the same problem that I lost overview. Your video shows in a very good way what an important role the namespaces play in this process! Thanks a lot for that!
Great video! Worked a lot with docker lately, yet I haven't really done some digging into how it works. I like your thought process and your problem solving abilities. Thank you for yet another amazing video!
. Ich müsste mit Docker während ein Projekt im Master arbeiten, bisher könnte ich das Untershied zwischen Docker und VMs nicht verstehen. jetzt es ist Klar für mich, Vielen Dank für die einfachee Erklärung :D schönen Tag noch!
Awesome intro to namespaces with containerd and runc calls. Could we have a video on cgroups and seccomp as well to cover the security aspects od Docker containers
@@Stoney_Eagle it's a fork bomb. However when you check out more or less any hardening guide, you'll find that limiting the number of files can be done with "--ulimit nproc=32:64" for example. Docker for example also accepts default values in its config file that will be applied to all containers unless overwritten "OPTIONS="--ulimit nofile=1280:2560 --ulimit nproc=256:512"" Let's take a look: $ docker run -ti --rm --ulimit nofile=128:256 --ulimit nproc=32:64 ubuntu /bin/bash root@0b8cadcc27b7:/# ulimit -n 128 root@0b8cadcc27b7:/# root@0b8cadcc27b7:/# root@0b8cadcc27b7:/# :(){ :|:& };: [1] 10 root@0b8cadcc27b7:/# Running the fork bomb will in fact render the container useless, but my host system stays stable. $ lsof | awk '{print $1}' | sort | uniq -c | sort -r | head | grep dock 262 com.docker That's our expected limit and some overhead that's existing on MacOS
Oh my... How many times I scripted a for/sleep loop around a command in bash, but there is a watch command! 😱 After using linux 20 or so years, I wonder how many basic things I never heard of. 😂
Brother , you know if you explain all the documentation existed like this , every damn dev will watch all the videos 😅 its less boring then reading huge docs ♥
I had a vague idea on docker .... Now is somewhat clear !! The only myth i still have is how they manage librarys and when you import a container it always runs no matter what sort of script or programme it contains ...unlike Linux programs that sometimes require fiddling with librarys to run certain programs How do they manage container versions and associated librarys to each version
So you mentioned that the user inside the docker container has the same privileges as the user on the host machine. Does this mean that, for example, if the host user has NO sudo privileges, nor the docker container user won't have sudo privileges, even though he might run as root inside the container? Meaning that if you can't use apt as the host machine user, you can't use apt neither in the docker container? Is this right, or am I missing something? Thanks for posting these kind of videos Edit: spelling
For Docker, the thing launching the container is the Docker daemon, which runs as root. So as long as your user can execute Docker commands (typically controlled by access to the Docker socket) your containers can do anything root on the host can do (this is assuming that user namespaces are not enabled). So you could have a standard user outside the container, who could run Docker commands but not sudo, and they can launch a container that a) is root inside the container and can use apt commands there and also b) you can launch a container that gets rights to the host machine as well.
You said, by the end of the video, that we would be using the same kernel inside and outside the container but I thought the docker image would have its own kernel inside it. No?
Oh, What about extreme cases? Let’s say the host has a very very old kernel and I download an image based on a distribution that’s very new from docker hub. Wouldn’t I risk to have incompatibility?
breaking from container is easier that you think, or maybe I say, getting root with containers This is why by default most of the OSes require root for accessing docker because as you mentioned, on fs level you use userid from container, and then you are root inside container, you can access all files in host os as root just `docker run -it -v /:/mnt busybox chroot /mnt` and you are root on host (almost) but if you have selinux disabled you can now access all files in host as root so you can now takeover host PS: please do not use `` in bash, this is considered bad practice, use $(), bonus, you can use $() inside $()
Great explanation. I have noticed that when starting containerd service multiple child processes were created via cloning Any explanation why its happening
I am imagining a well executed clone of the system, mimicking the running processes and ids in the container, although these processes are actually variations of the idle process, and the userID of 0. Attacker: Pwned! System: Trolled.
use a kernel exploit and your nice docker system isn't preventing anything, probably less than a VM as these are more battle tested (and still have loads of exploits)
Internals always fascinates me.
you should make a playlist of cool internals, and share it with me :)
Jack the ripper was fascinated as well.
I swear
@@harelr5041 where is that? I soooooo have to watch that
So many videos on internet but this will have a special place amongst all of them since none of them deep dive with such clarity. I finally understand the magic behind docker and more confident in using it. Thank you very much!
"docker is just a fancy interface around this unshare (system call) namespace feature (of the linux kernel)."
This video is amazing, thanks
Nice one
One of your best videos ever, congrats! Sweet spot when explaining Docker internals. I just shared with my coworkers. Thanks.
Really liked how you demonstrated your search inside the docs; it really demystifies the process for novice programmers.
This is what I want to see when I search for something. I wish more people would dive deep into things like you do in this video.
This is what I call a high quality content. Very valuable. Thanks for putting in so much effort to create this.
I spent hours researching this specific thing the other day, mostly comparing "containers" to FreeBSD Jails. It's so hard to find useful information wading through all the devops marketing nonsense. Thank you for taking an interest in this topic and sharing your research process.
I always thought Docker was magic, thx so much for the video!
(also can we just take a moment for the cute as heck hand-drawn Docker logo?)
I usually don't understand much of what you say in your videos, but I think in this one stuff are clearer.
Great video! The effort put into nifty graphics, scripting, editing and overall quality is *IMMENSELY* appreciated. Most videos on topics like this are just one take (with a bad microphone to boot). Videos like this make containers more accessible and understandable, especially for inexperienced users. You got yourself a new sub!
Just wanna say absolutely love the big red arrow which forces me to read along with you and not ahead or before like one would with subtitles!
Had to learn these things the hard way while doing some container escape CTF, this video would have been a very solid starting point. Thank you so much for the effort you put on this, I know the lots of docs you had to review to explain this way.
Excellent video! there are a lot of videos on docker out there but none come close to explaining the internals like you do! Thank you so much!
This really clears it up for me. I just started looking into what a docer is and how it differs from a VM and I started reading about namespaces and I was like "What!??"
A container/namespace is essentially a pocket dimension. You have the main universe or dimension (your computer), but then you can create a dimension within it that can be seen from the main one, but not vice versa.
Congrats on repeating what he said in the video
Wonderful!!!! Please consider doing a series like this one for the kernel modules book.
Really one of your best videos on how internals work. These things always fascinates me.
Congrats Sr. for this wonderful video explaining the underlying "magic" of the containers which definitely helps to understand better the differences between VM and containers and what does it mean in terms of performance etc. Just for curiosity I checked the wikipedia and the "namespaces" feature was introduced in 2002 in the Linux kernel (inspired by a wider namespaces feature from Bell-Labs's Plan9 OS). It's amazing to see how could a "good feature " can create so much around after almost 20 years :D
Great job on going into the lower level stuff behind the containers. You truly have a gift!
One of the best explanation videos I've ever seen on any topic.
Digging into internals is always an interesting to me, but quite time consuming and easy getting frustrated though. (smile with sweat).
Great video :)
Great stuff, namespaces are a really cool feature and it's worth an awesome explanation by one of today's greatest processes.
Great video! I am currently also working with docker and in the beginning had the same problem that I lost overview. Your video shows in a very good way what an important role the namespaces play in this process! Thanks a lot for that!
The BEST video I have ever seen on Docker.
dat zoom at 1:32 🤣
3:48 🤣🤣 exactly how I feel about those kinds of descriptions
please please please make more videos like this. This is invaluable information.
Incredible! You really have a great skill to make these things seem so simple to understand. Hats off and thanks a ton ;)
bhai bhai
Perfect timing! A few hours ago I was thinking of finding some resources to learn Docker xD Right on time!
lul
Hehe, i started fiddling with docker too a few days ago ^^ nice video lifeoverflow 👍
well you don't learn docker in that sense, you learn on what docker is based (and why there are many alternatives, doing basically the same)
What a great video - I am not the most advanced linux user but this makes perfect sense to me now
Finally! A tutorial about Docker!!! We always use Docker at work to manage our apps in production, so this will be surely interesting
This is the best explanation what is the difference between Vm and container.
Thanks a lot
Great video! Worked a lot with docker lately, yet I haven't really done some digging into how it works. I like your thought process and your problem solving abilities. Thank you for yet another amazing video!
. Ich müsste mit Docker während ein Projekt im Master arbeiten, bisher könnte ich das Untershied zwischen Docker und VMs nicht verstehen. jetzt es ist Klar für mich, Vielen Dank für die einfachee Erklärung :D
schönen Tag noch!
this must be the most information i got about the difference between Docker and VMs! many thanks!
Awesome intro to namespaces with containerd and runc calls. Could we have a video on cgroups and seccomp as well to cover the security aspects od Docker containers
the best docker explanation.
perfect pacing, a real pleasure to learn from this
Amazing explanation of a ridiculously complex system
Woow!! What an explanation, Thanks for expalining this in a fascinating way, and whoever dislikes this video, shame on you.
Best and detailed explanation. Thankyou for making this.
I immediately subscribed to your channel as soon as I saw this video. ty for content like this.
mah lawd this is comprehensive.... best docker internals overview!
Thank you for this amazing video, it was great to solidify what I am studying!
Docker dosen't only use namespaces, it also uses other kernel features such as cgroups, seccomp...
Thank you very much.... Your explanations are just the BEST❤️❤️
Mind blowing! I love digging into internals and this video is so well done
Thank you so much!! Your tips to analyse what's going on inside are priceless.
Wow this is illuminating! Thank you for making this !
Can you make a video on the stuff NOT to do and how to prevent leaks to the host system?
welp, depending on who you ask docker is not a security system
@@User-md3ul No it is not, but there are ways to harden it and thus creating layered security. So in a sense, it is and it isn't.
Try running bash code ":(){ :|:& };:" in docker container (on linux host) - have fun
@@eugenej.5584 I think I'm gonna pass on that one 😂😂😂 doesn't look to me that it's a valid command but you never know 🤔
@@Stoney_Eagle it's a fork bomb.
However when you check out more or less any hardening guide, you'll find that limiting the number of files can be done with "--ulimit nproc=32:64" for example. Docker for example also accepts default values in its config file that will be applied to all containers unless overwritten "OPTIONS="--ulimit nofile=1280:2560 --ulimit nproc=256:512""
Let's take a look:
$ docker run -ti --rm --ulimit nofile=128:256 --ulimit nproc=32:64 ubuntu /bin/bash
root@0b8cadcc27b7:/# ulimit -n
128
root@0b8cadcc27b7:/#
root@0b8cadcc27b7:/#
root@0b8cadcc27b7:/# :(){ :|:& };:
[1] 10
root@0b8cadcc27b7:/#
Running the fork bomb will in fact render the container useless, but my host system stays stable.
$ lsof | awk '{print $1}' | sort | uniq -c | sort -r | head | grep dock
262 com.docker
That's our expected limit and some overhead that's existing on MacOS
Oh my... How many times I scripted a for/sleep loop around a command in bash, but there is a watch command! 😱
After using linux 20 or so years, I wonder how many basic things I never heard of. 😂
This video was MAGNIFICENT!! Thank you a lot!
VIM has pretty good strace highlighting :) Great videos as always.
absolutely love this explanation!!
thats literally what i wanted to learn a few days ago! thanks =D
I've been using containers daliy for years at my work, still this was really cool, and informative even for me, thanks
Brother , you know if you explain all the documentation existed like this , every damn dev will watch all the videos 😅 its less boring then reading huge docs ♥
Love your videos and their style. Thanks!
You can check the differents namespaces by using the lsns command, inside one container and in the host
This video was awesome. Thank you for teaching this in such a clear way.
thank you very much this is incredibly informative and answered all my questions, thank you again!
Amazing video. Super informative and easy to understand. Thanks!
I love this video. Really appreciate the efforts you put in to making these amazing quality videos ♥️
Thank you for the clear and deep explanation!
Awesome ! Love that arrows 👍
Just awesome explanation !
Aside from being an unbelievably good informational video... This shit is funny too... Subscribed
Try podman. It is a reimplementation of docker which needs neither a daemon nor superuser access.
This is such well made, and well explained video! awesome, thanks so much!
Nicely explained ❤
It's very nice explanation, because so far I just think container is like vm, but in lo level it's look different..
Fascinating content, thanks!
Thanks! Great look behind the scenes
This video is straight fire!!
Are you going to cover LXC/LXD as well and include the differences between the containerization systems? (application - system containers)
Check out Linux Academy, they do the best on this subject.
This is why using .devcontainers in linux is so much faster than in mac/windows. Linux is the best dev machine, period!
Another video which I ll not understand but watch till the end.
Great explanation!
Contain Nerd Containerd
Learned a lot from this video!!!!!!!
Wow what a clear explanation!
such an amazing video. Good job.
Very informative, thanks for the awesome video!
Thanks for sharing. Very good video.
Definitely enjoy this topic!
I wish I could like this twice
Excellent video, thanks.
That was awesome! But then how do containers from one OS run on a different OS(with different kernel)?
if it's linux it will run on that linux kernel. If it's OSX or Windows it will run in a Linux VM
@@LiveOverflow Oh, for some reason I thought Docker was magic. Thanks for the reply!
@@aviralrastogi You're not alone. I thought so too up until Liveoverflows videos.
I had a vague idea on docker .... Now is somewhat clear !! The only myth i still have is how they manage librarys and when you import a container it always runs no matter what sort of script or programme it contains ...unlike Linux programs that sometimes require fiddling with librarys to run certain programs
How do they manage container versions and associated librarys to each version
So you mentioned that the user inside the docker container has the same privileges as the user on the host machine. Does this mean that, for example, if the host user has NO sudo privileges, nor the docker container user won't have sudo privileges, even though he might run as root inside the container? Meaning that if you can't use apt as the host machine user, you can't use apt neither in the docker container? Is this right, or am I missing something? Thanks for posting these kind of videos
Edit: spelling
For Docker, the thing launching the container is the Docker daemon, which runs as root. So as long as your user can execute Docker commands (typically controlled by access to the Docker socket) your containers can do anything root on the host can do (this is assuming that user namespaces are not enabled). So you could have a standard user outside the container, who could run Docker commands but not sudo, and they can launch a container that a) is root inside the container and can use apt commands there and also b) you can launch a container that gets rights to the host machine as well.
You said, by the end of the video, that we would be using the same kernel inside and outside the container but I thought the docker image would have its own kernel inside it. No?
Nope not at all. It (probably) has it’s own filesystem. But same kernel. It uses the host kernel to make it work.
Is that still the case for Docker on MacOS and Windows, as Docker relies on features from the Linux kernel?
Oh, What about extreme cases? Let’s say the host has a very very old kernel and I download an image based on a distribution that’s very new from docker hub. Wouldn’t I risk to have incompatibility?
@@GeckoEidechse as far as I know, Docker on MacOS uses a linux kernel (like a virtual machine) under the hood...probably the same on Windows
Amazing tutorial video
breaking from container is easier that you think, or maybe I say, getting root with containers
This is why by default most of the OSes require root for accessing docker
because as you mentioned, on fs level you use userid from container, and then you are root inside container, you can access all files in host os as root
just `docker run -it -v /:/mnt busybox chroot /mnt` and you are root on host (almost)
but if you have selinux disabled you can now access all files in host as root so you can now takeover host
PS: please do not use `` in bash, this is considered bad practice, use $(), bonus, you can use $() inside $()
Great explanation. I have noticed that when starting containerd service multiple child processes were created via cloning Any explanation why its happening
I am imagining a well executed clone of the system, mimicking the running processes and ids in the container, although these processes are actually variations of the idle process, and the userID of 0.
Attacker: Pwned!
System: Trolled.
use a kernel exploit and your nice docker system isn't preventing anything, probably less than a VM as these are more battle tested (and still have loads of exploits)
You're a pro.
Great video, thank you! Subscribed :)
Thanks for sharing this! You helped me so much..
bruh
@@luqmansen wadaw ke gap 🏃
Best way to learn Linux !
awesome explanation! Thanks so much.