How I learned to love build systems
ฝัง
- เผยแพร่เมื่อ 7 มิ.ย. 2024
- Follow me on Mastodon: hachyderm.io/@fasterthanlime
Support me on Patreon or GitHub: fasterthanli.me/donate
Why in the world did they want me to run ./configure?
Contents
00:00 How Amos fell into the magic potion
04:44 "Real" Windows development
07:23 Enter CMake, then Meson
10:34 Now scale it up
14:15 Be humble, stay curious
15:28 Obligatory tunes of the chip - วิทยาศาสตร์และเทคโนโลยี
I have no idea how you got on my TH-cam recommended feed, but I'm oh so glad you did. Fantastic, clear, informal videos that honestly excite me to go out and learn -- which is an incredible resource. Subscribed... and thanks for what you do! :)
Thanks so much! This is exactly why I do these :)
for me i got his RSS feed from some personal site, and goddamn i fell in love with his yt channel.
same, i follow on twitter, for some reason i forgot now and i wont pretend i understand all of this but still enjoy it. weird
"I also write articles that are equally long"
Well-
"Part of the answer is that, games are not food" @ 2:29
- FasterThanLime
Fantastic video, really mirrors my experience of going from the comfortable but illusory plug-and-play nature of clicking buttons in an IDE to understanding the intricacies and complexities of building source code for multiple platforms
Kudos to you for keeping a good attitude about them and even coming to appreciate them! I’m at a point in my career where I understand why they’re needed, but dread every minute I spend modifying a jenkins file/web pack/grunt/maven/make/gradle/{whatever} config.
It’s time that I should be spending on the actual problem I’m trying to solve.
Don’t think I’ll ever be able to say I love them.
This is a fantastic video. Its educational and gives a clean abstracted overview of build systems. Personally I find that abstracted overview is hard to come by because most people convoluted the general overview of what a build system does. The only issue I have is your title. I searched for "most used build systems" and found this video but I think this video belongs more in the "what is a build system and why do I need one" category which I feel like your title excludes you from. Either way I am happy I found this video!
Great video, Amos. Thank you!
Well, it was fun to watch you do such a deep dive into this niche software dev topic. Very informative stuff, thanks for sharing!
I thought this was going to be tips on what you have to learn to build systems but it literally was how you learned systems. It's amazing and helpful to hear your journey!
Back around 1997, I used to walk to/from university (around 3 miles each way) with a Zip drive and 2 or 3 Zip disks to download stuff that I could take home in my bag, configure, make, run. Seemed so amazing at the time that I could even do that, compared to how it was before with 1.44M floppies. 256MB USB sticks circa 2005 were magic.
And now, we have NixOS which might download 5GB of data when you modify your mounts 😅
(Disclaimer: NixOS is amazing and I run it on all of my machines. Nonetheless it's reliant on a fast network connection and would be absolute hell to use in the university / zip disk situation)
@@antonliakhovitch8306 me: plugs in a usb drive
NixOS: well, time to backup everything and reinstall the OS. Like they say "better safe than sorry"
I remember in around 1993 we got an Amiga 1200 and we got a couple CD's of AmiNet, by snail mail from Germany to Greece, as it was faster and cheaper than over the practically nonexistent internet.
I knew nothing about build systems until I got into embedded development. I fumbled around with SCons for a bit but now have learned to love CMake. Great video!
Wow, incredible video. Great job :)
Great video! I'd love a course/tutorial by you where you show step by step how to setup a build system like this.
I have no idea on js history, I only know that there are a lot of import-styles, build systems and complexity but it would be really interesting to know about all of that history (and it would be enlightning to know why some dependencies only like some build system for whatever reason)
Many of the weird behavior of javascript is due to its original focus on interacting with HTML, which has things like bools and numbers all stringified so in order to be ergonomic, JS would need to be able to treat strings like numbers or booleans.
reflecting a bit what have me drawn to these kind of "lower-level" tools like the cli and vim, was a kind of appreciating of exposure to these, often difficult, parts to lessen the uncomfortableness of leaky abstractions. and on further reflection it gave me confidence that I so felt the need for, quite often as a coping mechanism for feeling most of the time insignificant from childhood events. now after more or less being able to move beyond that need I find myself sensing a more natual curiosity emerge to explore these dimension. Thank you for this video which had much value in it for me.
Thanks, Amos. Great video!
Great video! I could use a vide about compiler optimizations. But it has to be so simple that even it can be used in programming game like ShenzhenIO or Human Resources Machine.
cool video. i loved the summary
Loved the video!
As a DevOps I recently encountered a sort of build engineer(ish) task of gluing together a C++/ML desktop app, which allowed me to make a deep dive into build systems during work hours. Started to read a blog of a Ninja-build creator, Professional CMake book, etc. Man, build systems stuff is sooo interesting, deeply cool / technically beautiful and at the same time not unappreciated/ viral enough!
Thank you for spreading the word and chad content in general!
Lol I thought this video was about how you're gonna takle a task with a certain framework, thinking in systems or whatever, but I also enjoyed what you had to say :) Noted the quoted regarding accidental complexity, amazing!
I vividly remember when I first started to play around with Linux and the command line in general, I was very overwhelmed by it. I was also the only one in my family who had any computer experience at all, so if something wasn't working, I had no one to rely on. It was unpleasant, but it also taught me alot of stuff.
This video would've been so useful like 5 years ago, I went through pretty much the same learning process (even started with a ti84 lol)
that's so cool, man! I have completely different experience, but I came to similar questions...
This video is great please make more.
I initially didn't want to watch this video, but I figured I should, and I learned so much about computer development! Definitely grateful for your experiences
Such a great storyteller
"There's only one of you, so make it last" -- love it!
Fascinating video topic, i am going to have to revisit this. i subbed
This was recommended to me
and now im subbed.
You know that you're on the right path to becoming a computer expert if this video shows up in your feed, loved it😊
As an eternal amateur who never went past console-run games made using the Curses library and some Unity stuff... I want to believe :D
Tutorial hell
that just means you watch youtube videos about computers
More of this content please
ah this was great! "car no go space, cargo road" will be stuck in my head for a while :P
"sccache (which by the way is looking for a maintainer right now)" 😅
Great video!
such a good watch!
great video
I've loved CMake ever since I started using it. It's awesome to be able to have a simple, easy to read project file that can also manage all my unit tests and build for both Windows and Linux
The syntax is appalling though. It's a great tool but Meson has such better ergonomics and I wish people would switch.
Recent CMake is better in ergonomics, but it's the same old ugly syntax.
i never understood how people move through life so uncritically. CMake is one of the worst things we ever forced upon hardware to execute.
But peoples only criteria seems to be "it works in my small frame of reference" but they are wholly uncritical and uneducated on how things should be.
for c/c++ i write my own build systems. bc its a brutally trivial task. even cross platform. maybe 200 lines tops. thats probably 2-3 orders of magnitude less than the average cmake code used to just configure a project.
This is a great video.
great video.
you earned subscriber, a bit for the great video but mainly for the car go road meme lol
15:23 Loved to watch it
Currently bashing my head into this mess while going on a multi-hour grind marathon of trying to compile rtl_433 for Windows. Great video and good points. However, I think the end moral of the video leaves out one important point: build systems (like cmake) are very often implemented half-way. It's only as good as the devs/maintainers make it. So, quite often I find cmake setups that were bent to wrap-around a platform-specific quirk (or several), or that leave-out dependencies their code relies on (or several), with no clear way to identify or mitigate these issues on other platforms. (example: rtl_433 needs linker options "-lws2_32", "-lwsock32", "-lnetapi32" added to some obscure mid-build "linker.txt" file after linker errors are produced -- oh and if you build using the 32-bit mingw platform, it'll fail to initialize USB when you think you got it all compiled - you have to use the 64-bit x86_64-w64-mingw-* tools instead, etc etc)
I often still end up finding it's just the same problems in slightly different packaging. Devs gonna dev.
I ended up getting it working in the end (as of 5 minutes ago, after about 6 total hours across 3 days, trying countless different approaches), and I could write a small novel on the problems in the build system that it revealed. But Windows is the least-loved of all platforms for open source software, so I mostly just expect to get roundly mocked or ignored or "closed::willnotfix"'d if I write this stuff up.
So hey, I solved it for myself at least. The last binary Windows build of the software in the wild is still from 2019.
great storytelling ez subscribe
CMake can also generate Ninja project files, it's very nice
Gracias Alexby
this is great
It clicked for me when i was working on a complete full-stack solution involving TypeScript, Websocket, Go, gRPC and Swagger. You make one single change in the API and you need that change to propagate to entire stack. Running every single tool by hand would be nightmare. Build system solves this issue and solves it in 2s.
So many different projects..cmake, meson, maven , gradle , bazel
Is there a latest-and-greatest set of tools that provide the most flexibility in building for calculators, windows, linux, etc?
I think most of these complex build systems exist to solve real problems that should have never been created in the first place.
it's because people have competing visions of computing and the different visions have gained their respective mindshares. If you had just one vision that had the entire world's mindshare, then you wouldn't need any build systems. 😂
@@KirbyZhang Sometimes I wonder if forking entire projects would be a better solution to handling different visions of how things should be implemented. Forking would bring some redundant work, but maybe it would still be more efficient than dealing with bloated dev tools. Plus, people would probably abandon most forks rather quickly; it might help rapidly determine which implementation vision is really the best.
I also started with my graphics calculator in the mid 90s :)
I need to know the name of the game at 2:01, i've played it as a kind and have never been able to find it online!
This was really good! Also, who was the person you wanted to share your first game with. :)
Just some schoolmates!
@@fasterthanlime love your passion and drive. It led you to some great things! :)
i ❤ this channel
I don't know why I'm watching this. I'm using CI/CD since more than a decade, and probably started earlier than this you. Still interesting seeing how others started their journey.
Good vid.
The biggest issue I have with Chrome is not technical at all. It's that Chrome is a vessel for Google to dominate the browser market, and that hurts us all, because they can push standards and practices that benefit their business while hurting everything else. Because most of the users will just accept it, everyone else has to adapt. It's the main reason I came back to using Firefox (which I am aware Google is also sponsoring indirectly).
I had the exact same experience just not in the same way but I had the same revelations
This is a fascinating video which really complements your articles I was already reading. I can't wait to see more !
For your next videos, maybe you could set up some sort of prompter, it is kind of weird to not have you looking directly into the camera, maybe it's because of the global trend that do it that cause it to me, but it's my feeling. I hope you won't take it personally :)
I bought a teleprompter recently and use it in my latest video. Enjoy!
Geez it took you longer than it should have to come to these realizations
the internet has definitely changed this video, i needed to send a executable from a rust program to my friend on windows and all i had to do was just look up mingw
Is Objective-C++ actually a thing?
spark-sxala kyro classes require so much builders or they dont exist at run time
i think they now have an option to say hey cache these before run time.
And now we're at wsl2 and wsa
I don't know doesn't every book on the subject explain where the executables come from, where the compiler is etc?
Have you checked out tup build system you can use lua alongside the build system
Yeah I know tup! We used it to build everything when I worked at itch.io
"Sure, I had to learn English, but I also got a lifetime of trauma for being extremely online. So it's impossible to tell if that was good or not"
I didn't sign up for this vicious targeted attack
my first build tool was Maven with Java and then gradle again with Java, at I first I hated the idea of using a build tool, I only realised the importance of build tools when I discovered MinGW and cross-compiling
7:27 “That’s new… and also annoying!” 😂😂
I'm only in my third semester of computer engineering so I don't understand shit, but the wizard in the video makes me feel eager to learn things
Kojima is a genius
Uh I had that CD, too!
Had the same experience discovering github actions
I was a bit surprised to see ikenna as a programmer
This video is not about build systems. I think that the concluding question here - When something doesn't make sense: Is it incidental complexity OR have you had a unique insight OR Is your current worldview missing some key pieces? - is critical. No matter what the answer is, asking the question when you reach a "WTF?!" moment is enlightening. I can easily project this onto my science education. Wish someone had told me this much earlier in my education and career.
i want to make things that are cross platform too... i wish i could figure out all of those things as well
The Photoshop/3DS Max/Cubase screenshots loool
I Hope I get this video 5 years ago
what font is that on the terminal ??
It's Iosevka!
@@fasterthanlime thanks 🙌
Build systems are a science and an art. But once you understand how much time these things can save you, you adhere to the Law.
Here's a weird one, why do CCSDS standards exist?
CCSDS - Consultative Committee for Space Data Systems, is a committée that does standards for satellites and rockets and space stuff. And most of space digital infrastructure, especially legacy and/or government infra, which in space is like 99% of it, relies on it.
What's curious is they felt the need to redo the entire communications stack, from the physical layer through TCP and IP and beyond. They have their packet definitions and whatnot. And my question is, why? What need were they solving that required them to redo everything as opposed to say, take the normal internet stacks and do a couple adjustments for the intermittent long-delay radiowave-reliant communications typical of the space environment?
TCP/IP has been used in space applications, I suspect the reason for this divergence is historic. But I don't know, and I wonder!
Literally my experience with computers and Linux/Java/OGL
it was so painful to include LWJGL to your java build and make it work on more than your system, and linux build will always crash/fail on other linux because binary LWJGL library include other library that no presented on target linux...
And you endup making something that currently Steam on Linux use - just pack half of Ubuntu libraries with your libraries, but even in 2010 size still mattered, so having 50Mb of binary libraries +90Mb portable binary java build because not a single linux had Java preinstalled... for your 500Kb jar file - it was not an option
And yes - dont use opensource software it has no value and Elon Musk hate it.
Thank goodness thimgs like arch linux's aur exist so that work can be delegated to the build system
Well right now im wondering why bash and sh exists. Why are there two slightly different systems?
There's a bit of history here: www.aosabook.org/en/bash.html
sh is the Bourne shell, bash is the Bourne again shell. There's plenty of other shells! dash, zsh, fish, nushell, etc. etc.
One of the many revolutionary aspects of Unix (compared to other systems of the time) was that the command processor was not built into the OS. It was built like any ordinary user program, and ran as an ordinary user process, with no special privileges. So if you didn’t like it, you could write your own. And lots of people did.
I'm going to take you on that call at the end of the video and ask a question to you (or any other knowledgeable stranger) that has been bothering me for a while - what are linker scripts and why do they exist?
The place I've seen them (from afar) is for embedded development, for hardware that expects the image to have sections in a certain fixed place - I believe the linker script defines the structure of the generated file, so it matches what the piece of hardware expects.
They are needed for special purposes. For example, say you are building a ROM for an embedded processor. Or you are building the Linux kernel. These are pieces of code that need special memory layouts, initialization/loading procedures etc. You control that in part via the linker script.
Type “info ld” to find out more than you wanted to know about it.
Nearly all things made sense in the context or "world view" how you called it, they were made in. Otherwise people would not make them. Things only get bad when we use things in a context they were not made for. Because there is no more suitable tool (yet).
I am a package maintainer for a linux distro. Maintainers often complain about the modern languages like Go and Rust because with their hang for static linking they do not quite fit into this package world that was mainly made with C in mind. Yet I still like coding in Rust and I can see how much this static linking simplifies the whole dependency process.
Its interesting to see both "world views". Both are correct in their own context, yet
"These things you bang your head against the wall about have a place." Yea I agree. Most of the times that place will be to be replaced with Rust.
I started the video thinking its topic was "How I learned to love to build systems". But no. :D
11:40 now I know which facebook group you're in
The joke about trauma of being on the internet made me chuckle.
and .exe that run on windows mac and linux are a reality!
I really liked this video as someone who is learning to program and isn't that knowledgeable
There's clearly a reason why build systems exist and how they got more and more complicated over time. But it doesn't have to be this way and that's a terrible oversight from the whole SW industry. Starting from OS libraries management and language architectures, it's a mess. I don't want to learn a different language just to build a program, we shouldn't tolerate unnecessary complexity.
why dont we just use ftp for everything? i dont get networking... i have no clue...
Joy Wave
Video title: How I learned to love build systems
What I've read: How I learned to build love systems
Close enough 😂
This is awesome! Can't wait for your next uploads, hopefully more Rust hacking 😄
Well that comment gave me an idea for a Rust video... now to actually, erm, make it. That's the hard part!
just realised I've spent the last 20 years pronouncing MinGW like "ming-wee" and maybe most people don't
The last point about cargo is a big one. Its really annoying that it wants to do everything for you and its highly non-trivial to integrate it with a meta build system/package manager... As a non Rust programmer just using Rust dependencies (e.g. webgpu) you'll understand this very quickly.
I don't entirely agree with this, it still makes it so that all the dependencies exist in one repository, and you have the benefit of always knowing that a tool that can compile this code for anymachine actually exists, which is not true of C++ projects, and that all the configuration exists in toml files that define everything specific to a project, so that tools exist to integrate it with bazel. Don't underestimate standardization as a way to avoid complexity. It makes it so that you don't need to download a vm image with the exact build env of the author
@@BosonCollider check out how you have to use cargo with tools like nix or Spack and I think you'll see my point. It's just about cargo adding some options to support it rather than the hacks needed now with e.g. bazel and nix
As someone who cannot stand the excessive complexity of managing C/C++ dependencies and build systems, Cargo is great. Like C# or NodeJS, Rust/Cargo "just works". Grabs dependencies for you, compiles things, no long cli arguments, just "build this thing as the developer intended" and away it goes.
Some Times I hate how humans are different.
But there were textures
These days, you can go into IDEA, make a new project from a template, and it will make a build script which can even build the installers.
For a long time, tools would give us ways to get to a .jar file and then actually distributing the app in a useful way was still annoying. I feel like we have only just made it back to the convenience we used to have when visual development tools first appeared. We're still missing the drag and drop UI designer for Compose, so actually writing the code still takes time, but once that's in place we'll be back to what Visual Basic used to feel like when it first came out.
Holy shit you look like Hideo Kojima