Building a container from scratch in Go - Liz Rice (Microscaling Systems)
ฝัง
- เผยแพร่เมื่อ 20 ส.ค. 2024
- Everyone has heard of Docker, but what is a container? Is it really "a lightweight VM"? In this talk we'll dispel the magic by writing a container in about 100 lines of Go.
Liz Rice has a wealth of software development, team, and product management experience from her years working on network protocols and distributed systems as well as in digital technology sectors including VOD, music, and VoIP. When not building startups and writing code, Liz loves riding bikes in places with better weather than her native London.
I finally think I understand what a Docker container is! Thanks.
Her 50 lines of GO lang code hides over a 100 levels of OS level abstraction, and you confidently say that you think you understand what a Docker container is. That's laughable.
@@sussus4914 so why don’t you give a talk on those 100 lines of OS abstraction?
@@sussus4914precisely why they are abstractions, so you don’t get distracted with boilerplate
The best part of this demo is that it makes containers more like jail and addresses the trust issue.
Wow the way you make things feel simple, now I'm even a go pro developer
Loving demos cause it shows the real things in practical world, always intrigued to watch her demos;
this is by far very short and well explained how the container can expand the possibilities how GO can do it.
one of the best container talk i've heard
Looks like my favorite presentations are those that start so simple you almost think they are jokes, until suddenly they are not.
This was amazing. Simple and easy to digest, but packed with information.
I loved this talk!
That's very inspiring, especially Liz shows her charm by unique humor and coding flow.
Thanks, It was the best way someone can show me concept of a container.
This was an excellent presentation, thanks!
Made me want to learn Go. Thanks.
Note that child processes can call chroot() again and break outside of this container easily. Docker doesn't use chroot.
All big words should be explained like this.
Awesome video and a great intro to how containers are constructed
"You are my peer reviewers"... what a lass
why can't I give 1000 likes to this one? :)
Woah... very nice explanation 🔥
Thanks, That was cool and easy to understand!
The talk really begins at 2:25
You're welcome.
This is awesome!
Great Demo!
Did I just watch Go programing in Downtown Abbey?
Great work !
Great tutorial thank you!
This is a fantastic presentation, thank you! I notice that you quickly gloss over installing a root file system ("I just happen to have one lying around"). I'm not familiar with installing linux file systems and my attempts at doing so are apparently too naive to work. Is there a straightforward way to download or copy one?
For ubuntu:
apt-get install lxc
sudo lxc-create -t ubuntu -n yakkety
# now use this for the chroot /var/cache/lxc/yakkety/rootfs-amd64
# I still had to `run mount -t proc proc /proc` to get ps to work though, not sure how to get around that
I could do this by creating a chroot filesystem. help.ubuntu.com/community/BasicChroot
debootstrap stretch test1
really good
great explanation. Thank you 👍
Really good talk
Great impressive Demo
hope to see you at kubecon nice video
Awesome stuff. Any reason that calling /proc/self/exe was done rather than syscall.ForkExec? She mentioned that it does the same thing. Just because the copy-paste portion is faster for the demo? Minimizing the Go-specific parts since ForkExec is a convenience method and takes a Go struct? Not criticizing, just curious
Thanks
very good, thanks
............Excellent .................
Nice
excellent video
awesome lizrice
Does anyone have the link to the talk by Julian Friedman she is talking about ?
awesome, thanks!
Badass
Where is the network stack+virtualization for that? How does that work?
Badass!
wait this is so easy
very impressive demo
good afterneen
🤯
Really good walk through.
The container doesn't have internet access. Is there a way to provide the container with internet access?
yes there is a way, how to do that i have no idea
wow..this is super awesome!! ps not showing host processes was very nice. But why Go though. As someone who doesnt speak Go, what I understood was you did some syscalls, cloned UTS namespace, changed rootdir, and invoked a new /bin/bash as a fork process. Same thing can be done in any language cpp/python/java right?
Efficiency. You can build a completely self-contained binary that does everything. If you add https or a straight TLS socket you only need the exe and a cert file... it's super clean.
With Python you may have versioning/suppor issues.
Java is a pig - it latches onto cpu+memory resources. Not really apropos.
C++ is ok for this, but younger engineers may not know it + unix command programming in C++ can be a bit tricky. Golang makes it (and certain other tasks) pretty straightforward.
I would use either C or golang. Maybe Rust but I don't know it yet.
@@RockwellAIM65 yeah I think outside of Go, c++ would be the best choice
@@piyushsingh178 A C based solution would be clean; you'd have to add lots of external libraries tho' ... would have been nice if C had a standard add-on for managing databases, doing all the simple network type transactions w/ a second thread perhaps (application+background processing thread) + an easy-to-interface string based hierarchical data store. C++ wasn't really necessary... it turned into the Cobol of the 1990s!
Why am I watching 7yo video about sth I don't do very often in a language I don't use, I don't know. But it''s good xD
I thought Linux supported containers/virtualization natively (LXC) - why is she faking container isolation by e.g. mounting alternative file systems?
How is she faking it ?
When did she mount an alternative fs? When she mounted proc? Or do you mean the chroot? The chroot source dir already contains a file system hierarchy because the child process quite literally has a different root, and would not be able to access anything “outside”. Without this, calling /bin/bash would do nothing since that path wouldn’t even exist. Mounting proc is required purely because of the chroot. LXC isn’t a “native” feature, cgroups are. LXC is just one of several system virtualization frameworks such as libvirt or systemd-nspawn. Her talk was about making containers from scratch, which I’ll interpret as a LSB compliant OS with a vanilla Linux kernel. LXC under the hood does pretty much the same thing as her code. Go ahead and check out their github; it’s all open-source. LXC has more features, but I’m pretty sure it’s quite a bit bigger than 60-ish lines
TL;DR: chroot && mount proc
Likee
Too much weird sounds from Liz (don't how to say it since I am not good with english) but good presentation.
I'm not sure it really explained what a container is or how it works. I just showed what result a few lines of go code had, but what really happened?
For me it was the fact that I always sort of imagined containers as this mystical thing. I use them all the time, but I never really thought about how they are created or work under the hood. Seeing this video really opened up my mind to how simple it really is. Of course I know there is so much more to containers than that, but just seeing how she could give the processes their own namespace, their own processes, their own “isolation”, is really interesting and impressive for just 56 lines of code.
If you follow what it shows, you'll also see what happened. It gave the program it run isolation (not messing with the environment outside), it's own root (not seeing outside a particular directory), and a few other similar things. That's what a container like Docker does, using several OS provided utilities (like chgroups, chroot, etc), like this program does (in a more basic way)
Min 05:00 and I don't know what the heck she is doing. And yes as she said, this is quite dull. And Go, why?
“Sublime” editor?
Yis
Good presentation.
One thing, however: Why the need for go? Couldn't all this have been done in plain bash? After all, all she's doing is calling system commands.
Why the need for programming languages at all? After all, all they do is translate human readable code to bunch of machine instructions. Just write those instead.
Docker is written in go.
She compared the length of her code to Docker's in the end, so it had to be written in Go.
Yes, it could be done in plain bash. But it would be hell to maintain, scale, and make able to run arbitrary containers. Whereas this shows how a container manager like Docker does it (of course with much less functionality here), so it can be more easily extended configurable etc.
06:35 cringe, with respect for trying
What was the point of this program again?
Trust me, nobody is surprised that a flat earth proponent is not able to understand something very basic.
The point was to run a container without docker.
Isn't it OBVIOUS? To show how the basics with which a container system like Docker can be implemented...
What was the point of this comment again?
Was expecting a token female the managers stuck out front then she started coding live lol
Very skewed view of the tech world where most of the innovation is done by women since the beginning of the time.
@@shailynortiz def not true
@@shailynortiz lol, not true
@@shailynortiz I think both views are skewed. Females are not recognized for their accomplishments in the tech world, but saying they've done the majority of innovation in that field is simply not true.
Came for Vim, saw Sublime, leave.
I stopped listening once she said,"... on my Mac..."
that's a cool story
You got to be running the TempleOS or something to be that arrogant.
Alex Kozadaev nope, just not a proprietary os on proprietary hardware both designed to restrict your freedoms.
An Enemy nope. Linux.
Corey Reichle makes sense. I also prefer Linux and OpenBSD and cannot see myself ever buying a Mac, however as we can see in this presentation despite of what you said it won’t stand in a way of doing cool stuff either :)
Thanks