The automatic recognition of changes to header files is a godsend and something I did not know. Just implemented it in all of my makefiles and I love it! Thank you!
this was EXTREMELY HELPFUL !!! Ive been needing to understand build systems because I am designing on for an FPGA, this definitely helped me clean up my confusion. Thank you brotha !
This is THE BEST tutorial on make out there. I've looked at so many other resources but I couldn't understand a thing reading/watching them. Thanks a lot for your effort :)
I wish I had a professor like you , Such clear and concise teaching everything is motivated , actually interesing and applicable information, thank you sir
Love this video. Helped me get from "makefiles are black magic" to "alright this is kind of fun". I wish there was some info about target-specific variables but lovely regardless, love your way of speaking. Thanks!
This is the greatest make file tutorial. Thanks for utilizing your talent to help and teach beginners. I hope you continue to produce tutorials on other topics. Bash, docker, yml pipelines, etc...?
Great, fantastic, love it. The only part missing for my prog was a way to include a pre-compiled library; in my case the "mosquitto" library. I just added a 4th line to your first three. OTHERLIBS=mosquitto Then LLIBS=$(foreach D,$(OTHERLIBS), -l$(D)) and lastly chg'd the rule for $(BINARY): $(OBJECTS) to [tab]$(CC) -o $@ $^ $(LLIBS)
What an awesome, concise, holistic, easy-to-follow, and, truly fruitful lecture. I literally watched and learned all the Make skills I needed for my new job while watching the lecture as a movie. Thanks for sharing this amazing knowledge you've gained through the years in just an hour. It's quite remarkable. THANK YOU!
Having inherited a project that uses make on Linux, and coming from a Visual Studio on Windows background, this has been so useful in helping me understand what is going on. Thanks.
What a great tutorial. I’m only jumping back into Linux recently and not only did you teach me about make, but you also provided a great refresher regarding C, lib, includes etc. thanks a lot
Why did it take me a month to hear about .d files. I swear nobody mentions those but it’s so confusing trying to understand something when people omit information like that. Thanks boss :)
Lovely explanation , Brilliant and I mean brilliant !! I wish there more of you about who can explain this with the well defined , well thought out descriptions that you have used in your presentation I understood everything you were doing Thank you PS for other people as we now in 2023 who feel AI will give you these answers then Im going to say , "No AI no matter how good you think this is will and no AI will help you and guide you to this depth and understanding This is why we need people like Gabriel Parmer especially in poorer and less educated sectors where this is no help and all controlled by finances " You need to understand the essentials of this Im pleased we still have some who have recognised this and know what is required Its not a case of how quick , how fast or , how many cuts to education our officials can make or where others see AI as a method of doing away with people especially in education Its very much a case of how the use of AI where people in positions of power need to be very clearly show just how much they dont understand and why persons such as Gabriel Parmer are very much in demand I hope this justifies all comments I make whilst I highly recommend tutors such as this which we are lucky to have If only many other people in many an industry would actually wake up to the damage they do by making the cuts they have in conjunction with the loss of real professionals in all industry even more so in teaching and training
Thanks so much for the comment! I'm planning on a series of videos about system tools in the coming months. You've provided me some motivation, so thanks!
this is a really cool and useful video, help me a lot, thank you for doing it!, may i ask you which theme were you using for this video if you recall it?
One peculiar error I came across is that if you use the line %.o:%.c and you happen to not have a correlating .c file for one of your .o files (such as not have a x.c when x.o is a dependency for the binary) then it will say there was no rule to make x.o which is required for the binary. You might think that you do have a rule for that because you have %.o:%.c but because there is no x.c it shows that error that might be a little difficult to track down. Alternatively if you list out all the .o's instead of using % then it will tell you that you don't have a rule to make the TARGET x.c needed by x.o which is still kind of weird because x.c is never a target but it does make a whole lot more sense and it is closer to the root cause which is no x.c file.
Genuinely the best video about makefiles. In embedded environment is it okay to have a rule for flashing into target hardware or considered a bad practice?
Generally it is fine. However, depending on the development environment, the commands to perform the flashing might differ. So it is not uncommon to have a shell/python script to do the flashing, and simply invoke the script from the makefile.
First of all, thank you ever so much for your valiant effort, it really is appreciated. However, I rather think that I have little choice, but to accept that I'll never really grasp this subject. I like to place myself in the category of "actual programmer", by which I imply nothing in regards to occupation, knowledge, talent, experience or intellect. I understand the language of the computer, and it is not abstract, not in the slightest. It is procedural and exceedingly simplistic. For crying out loud, I have issues with something as fundamental as the foreach loop, whereas I comprehend the workings of a "normal" for loop perfectly. The mechanics are visible, for the most part. I can bit twiddle like no other, but ask me to do something simple and mundane to a string, or char array, which obviously I much prefer, I'm done for, at least you won't get a quick answer, and don't even get me started on OOP, polymorphism and god knows what. I suppose this is just who I am. It has it's pros, like being able to do and comprehend certain things that many, very experienced programmers/developers, don't, and cons, like not being particularly useful to the industry, in the modern world, but I digress. To simplify things, I have a few very simple questions, which I know there are no simple answers to, but I'd like to hear your opinion. Splitting a large project into multiple smaller modules, if that is indeed the term, or alternatively libraries, and preferably a main source file, with a single, or at least a few headers, what would be a good hierarchical folder structure, not least to make Make, easier to work with? Is it in any way shape or form, a good ide to have the main Makefile, so to speak, invoke other Makefiles, which could also, to my mind at least, simplify things(?) I'm thinking that treating each "module" as it's own separate project, with a dedicated Makefile, which can them be invoked as needed, in the larger scheme of things, What are the potential pitfalls? It this something that is actually done? As far as I know, it can be done, but as of yet, I have seen no examples of it, so presumably it is something that is not actually done, and presumably there are good reasons why. I've very much like to know. Last, but not least, do you know of any source, online or otherwise, which takes you from the very basics, through advanced, and actually explains things on the way? This is by no mean meant as a criticism of you, if anything, you're the first of many, who have actually attempted to explain anything, otherwise it has just been: "Do this, and magic happens.", and in a few key areas, you've had some success, which is impressive. Alas, actual comprehension still seems unattainable, thus this rather longwinded diatribe, hoping to for some... well, hope. I firmly believe that the praise expressed by just about everyone other than me, is well deserved, but ya, there you have it. Thanks and best regards.
Thank you so much for the thought out comment!!! I don't know how long you've been programming, but it honestly sounds like you're riding the learning curve up programming, bit by bit. Very little of what we do is natural, and it all takes a concerted effort to learn. You also can't really just learn all of it at once, as there are many conceptual stepping stones to get to many concepts. Even for those going through a well-designed curriculum that walks them through those stepping stones, it is very hard, and takes a lot of time. All of that is to say this: have patience, and cut yourself some slack. It is hard, and takes time, but is all learnable with time and effort. How you break up the C/C++ and header files, and how to you break up the Makefiles is very project specific. For the source files, you kind of want to think of breaking up c/c++ files into logically cohesive units. Think: the functions that "bin" together, should be in the same file. There are few rules around how big files should be, or any of that, but generally keeping the logic within them manageable and all self-related is the goal. We split up Makefiles, practically, pretty rarely. A single, top-level makefile can do quite a bit. I can build files in subdirectories, move them around, create executables, etc... The exception to this is if different parts of your project require vastly different ways of being built. For example a library might be separate from your main application code, might be separate from your testing code, each using different Makefiles. But honestly, I don't see this often. In our OS, we have huge hierarchies of Makefiles and I still don't know if that was the right idea ;-) If you're in the root of your project, and you cant to cause a Makefile in a subdirectly (foo) to the rule "all", you use the following in one of your rules: make -C foo/ all For example, github.com/gwsystems/composite/blob/main/src/Makefile#L20-L21 will execute the Makefile in the "components" directory, and execute the "component" rule. All that said, I'd recommend sticking to a single makefile till it feels like it is being pulled into too many directions. Only then use multiple. I don't know of great resources to learn a lot of this. Because of that, I recommend simply using it in one of your projects. Start out simple, then dive further in when you have to. The make references are complex, and hard to understand. But it often gives you enough information to try some stuff out, and see what sticks. I wish I had better news on this, but build systems are one of those things that most people learn by doing. I don't know if any of this is kinda what you're looking for, but a class I taught a year ago was on system programming. Using UNIX APIs and what-not. Might be useful? Might not. Find links here: gwu-cs-sysprog.github.io/ github.com/gwu-cs-sysprog Best of luck! Make sure to allow yourself the time to learn. We all start off not knowing much, and not being able to dive deep into complex topics (I always learned quite slowly). Before you know it, you'll know enough to start learning quickly. Your rate of learning only increases as you learn more. But to get to the part of that learning curve that feels "good" takes time. Some thoughts on this: www2.seas.gwu.edu/~gparmer/posts/2018-06-22-compounded-self-improvement.html
@@nothing6001 Well, that was a heck of a lot more than I expected, though of course I didn't expect much, given the age of the video. I have been developing and programming for decades really, though apart from 3 years in a Job were my official title was indeed IT developer, though I didn't do much programming, never professionally, for the very simple reason, that I simple cannot handle the excessive, and in my not so humble opinion unnecessary and counter productive, abstraktion. Perhaps it has something to do with my autism, who know, but it is what it is. At any rate, I started out some 25 years ago or so, I even did C++, when when it was still the case that one could actually manage to learn it, before it changed into something utterly unrecognizable, and when that happened, I landed on C, and have never looked back. However, Build Systems have never been foremost among my interest. If it has been warranted, I have used make, simply by copying what others have done. There has never been any doubt in my mind, that were I to attempt to comprehend the subject, I would fail miserably, but now I actually rather need to, if I want to get anything done in regards to my newest project, which is the most complex I have attempted so far, and that is actually saying something. At any rate, I am beyond grateful for your effort, answering my, probably not to easy to comprehend ramblings, and I will delve deeper into your answer, when I get the time, and perhaps post some follow up questions, if you don't mind. I have of course subbed, and hope to see some activity at some point. Thanks again, and best regards.
@@FelixNielsen Yeah, I want to find the time to make a series of videos that are shorter, and cover "levels" of different technologies. Time is always the problem. This video was just a twitch stream for a class, but I think I could do much better if I focused on the content. Regardless, we'll see if I find the time. If you're interested in OS topics, my video series on that topic are better thought out.
@@nothing6001 Sounds good. Should you need some sparing or ideas, I am actually, though it doesn't always seems like it, quite the tool, in a positive way, of course. I myself, have been wanting to do a series for a long time now, focusing not so much on a language, paradigmes, specific tools, etc., but on the very fundamentals, on achieving comprehension, problem solving, alternative "out of the box" thinking, heavily biased by my disgust for the prevalent abstraction of the day, obviously. Pretty much inspired by Charles Petzold and his book "Code: The Hidden Language of Computer Hardware and Software", that I read so many years ago, and was the one book that truly lead me to comprehend the inner working of computers, well, comprehend may be a strong word, but strong fundamentals at least, that I fear has not been prioritized for quite some time, and also, more than anyting, fueled my entusiasm. I would describe it as a children's book, to be honest. Even the most tech illiterate, can understand it, and I dare say that even the most praised and talented programmers/developers, might well learn something. Then again, it has been a very long time ago, so perhaps it is utter crap. All I can say is, that had it not been for that book, I probably wouldn't be able to tell and Iphone from a remote control today. Arg, I'm doing it again. I just cannot sit down in front of a computer, without writing a small novel, my apologies, but my central point remains. Chances are that I will never actually get to that point, but it would please immensely to experience someone else doing so. There was a time where I was very much into exactly that, OS programming. I stil visit osdev.org, once in a blue moon, and should you ever find yourself there, you might want to look up the ancient, though presumably still incoherent, ramblings, of one Zacariaz. I may well watch all you have to offer, though chances are *(NULL&(void*)0)--, that I'll ever convince myself, that starting down that path again, is a good idea, in any way shape or form. Best of luck with it all. I'll be watching.
Would creating a dependency something like: %.c:$(foreach D,$(INCDIRS),$(D)/%.h) have the same affect as using -MP -MD? (other than the fact that it doesn't create .d files of couree... I guess I'm curious the difference btwn using .d files and the above dependency.)
Big difference, actually. 1. You're adding a dependency on all header files, which means a change in any of them will require recompiling all C files. What the .d files do is recompile the C files only if their specific headers are modified (i.e. not all C files include all headers). 2. it won't include the headers outside of your path. This might not matter, but does if you use libraries *and* modify those libraries. Likely this is not a big deal for you.
Very good video! A couple of things though; how properly to place all build files (*.o, *.d) in its own "build" directory and the use of .PHONY. Keep up the good work!
i have a question abut the declaration of CODEDIRS and INCDIRS. What is the meaning of the one dot "." or 2 dot ? CODEDIRS=. lib #Source code directory INCDIRS=. ./include/ # Include directory Is it important to include since i saw other people do not use it
This is an amazing tutorial, very concisely explained. But I have a small question. I'm running on a windows machine, so if I want to utilize the rm -rf command, I have to open a WSL Ubuntu terminal, and everything works just fine. However, if I want to do "make clean" in PowerShell, using either of del or remove-item commands, "make clean" is unable to find the files its supposed to remove. I've tried to make it work but have been unsuccessful. Could I have some suggestions as to how to modify the code so that it works in PowerShell as well?
I don't know any powershell, unfortunately. But this linuxhint.com/delete-files-in-powershell/ says you can simply `rm -fo names*`. Not sure if it is right, or what limitations it has. Sorry!
@@nothing6001 Damn, thank you so much for the answer, big fan of your videos here. Apparently rm is just an alias of the remove-item cmdlet, and it does indeed have some limitations (dammit windows), so it looks like ill be sticking to wsl instead. Again, thank you so much for the answer and the content you put out, if it wasn't for your passion I wouldn't have grasped the concept of makefiles at all.
It pulls in the C/C++ dependencies on header files. When you compile the C/C++ with, for example, "-MMD", it will output a dependency file postfix with *.d. It includes rules for which .c files depend on which .h files. The line you ask about simply includes the .d files in the makefile. Long story short: if you modify a .h file used by a .c file, you want to recompile the .c file. This enables make to understand which .h files each .c file depends on!
simply says "look for this header file in any of the 'include paths'". When gcc compiles a program, you can set up the include paths using the "-I" compiler flag. For example, `gcc -I. file.c` would look in the current directory (see the "." there?) for any header files included with . Another example github.com/gwu-cs-os/evening_os_hour/blob/master/f19/10.2-makefiles/02_simple_makefile/Makefile#L2 and a more complex one github.com/gwu-cs-os/evening_os_hour/blob/master/f19/10.2-makefiles/03_featureful_makefile/Makefile#L10
Hi there, Why is there a difference in declaring the directories? I mean, why are both directories not addressed in the same way? Like either: CODEDIRS=. lib INCDIRS=. include or else: CODEDIRS=. ./lib/ INCDIRS=. ./include/
No fundamental reason. Your suggestion that they should be consistent is correct! There are always improvements to be made to code, and you point out a good place where the current Makefile should be cleaned up.
I'll leave this up in case anyone else has this question. It looks like % will run on all the .o files listed as dependencies for the binary not on all the .c files in the directory. I copied his setup and added in a random .c file and it didn't get compiled.
@@jasongharvery If you want to generate a .o, it would have to be a dependency for another rule. You often see a top level rule, something like: result.exe: $(wildcard *.o) Which effectively says: this rule will produce result.exe, but requires all the .o files. This will trigger the rule with the %.o on the left of the colon!
It was a great lecture, and I liked it. But one more thing I want to know is, will the same Makefile work if I have c and c++ files in my project? Or do I need to change it in some way?
Great question. Generally, you can just change the makefile from using gcc to instead using g++ to compile c++ (or whatever c++ compiler you use). Other than that, I'd expect everything should be the same.
No, surprisingly! When we generate the *.d files for each .c file using the gcc -MP -MD flags, it automatically creates the dependencies from the .c file to their corresponding headers. So you only need to include the .d files, and they take care of the dependencies.
Great video, thank you! :D I only have one question left. If you want to have tests for your code/module, how would you do it? I was thinking about using only the necessairy .o files when compiling a given test, with additional .c files, that are needed for the test to compile (like test suites, Test Cases, etc.). So when you want to make a test, you'd say something like 'make DataStructsTest' and it would create the executable of the test (in this example) for some Data Structure module.
Great question. When you're testing an application, you might manually specify only the dependencies you need (assuming the application is multiple object files). However, as you want a "compile test" to be part of your tests, I'd simplify in most cases, and just have your test dependent on all of the .o files of the application. In the case that you're testing a library, you want to have a dependency on the .a or .so resulting binary, which will likely compile/link the entire library. You want this complete set of dependencies as you want your test to interact with the library in exactly the same way that a client would. In short, I'd make tests dependent on the closest thing to a final output possible.
You don't need a rule to build the .c file because it already exists! You only need the rules to compile the source files into the objects. Hope that helps.
This is not as interesting as you'd think. It is my username (of course), and I usually choose my username on different computers based on programming language constructs (like the y-combinator that is the theoretical foundation for repetition).
Believe me, he has put his entire heart creating this video, flawlessly just amazing :)
Great video. Other tutorials only ended up in confusion for beginners. This is THE makefile tutorial. Thanks for doing this.
Dude … this is so good. The stuff you can find FOR FREE on the internet.
Thank you for taking the time to record this. It is very hard to find coherent information on the higher level topic in programming.
the remaining 5% would take you the next 10 years to learn
Amazing literally the best tutorial, can't believe this hasn't blown up
The automatic recognition of changes to header files is a godsend and something I did not know. Just implemented it in all of my makefiles and I love it! Thank you!
Really glad to hear it helped!!!
I've looked for a long time, finally I've found the ultimate Makfile tutorial. This was so great, thanks so much
1hr video that cleared up weeks of confusion for me. Thanks for sharing this publicly.
best makefile tutorial ever
this was EXTREMELY HELPFUL !!! Ive been needing to understand build systems because I am designing on for an FPGA, this definitely helped me clean up my confusion. Thank you brotha !
This is THE BEST tutorial on make out there. I've looked at so many other resources but I couldn't understand a thing reading/watching them. Thanks a lot for your effort :)
I can't believe I finally found a powerful lecture about makefile, it was a hopeless case before watching this video, thanks @Gabriel
I wish I had a professor like you , Such clear and concise teaching everything is motivated , actually interesing and applicable information, thank you sir
Great tutorial on makefiles. Learned everything I needed. Thank you!
Absolutely flawless video, and flawless teaching style. Never stop teaching, you have what it takes!
Love this video. Helped me get from "makefiles are black magic" to "alright this is kind of fun". I wish there was some info about target-specific variables but lovely regardless, love your way of speaking. Thanks!
Thanks for the request about target-specific variables! I'm thinking of remaking a series of these, and that is useful!
This is the greatest make file tutorial. Thanks for utilizing your talent to help and teach beginners. I hope you continue to produce tutorials on other topics. Bash, docker, yml pipelines, etc...?
Thanks for the suggestions! Keep 'em coming! Within the next 6 months, I do want to make a series of videos about systems tools.
@@nothing6001 can't wait to see them coming
An amazing lecture that was both thorough and concise.
Wow! That's i call a very great explanation. Thanks for putting so much efforts
This video is very very good starting point for those beginners who are learning build systems. Thanks for such amazing content
You're welcome; glad you like it!
thank you for explaining in a way that is so easy to understand
Great, fantastic, love it. The only part missing for my prog was a way to include a pre-compiled library; in my case the "mosquitto" library. I just added a 4th line to your first three.
OTHERLIBS=mosquitto
Then
LLIBS=$(foreach D,$(OTHERLIBS), -l$(D))
and lastly chg'd the rule for
$(BINARY): $(OBJECTS)
to
[tab]$(CC) -o $@ $^ $(LLIBS)
Best resource on Make on the internet !
Just incremented this like counts to 300. Very useful content, thanks a lot for sharing!
What an awesome, concise, holistic, easy-to-follow, and, truly fruitful lecture. I literally watched and learned all the Make skills I needed for my new job while watching the lecture as a movie. Thanks for sharing this amazing knowledge you've gained through the years in just an hour. It's quite remarkable. THANK YOU!
Thanks a lot!
Got my makefile up and running in less than 15 minutes
Glad to hear it!
Thank you for this video! This is exactly what I need.
Having inherited a project that uses make on Linux, and coming from a Visual Studio on Windows background, this has been so useful in helping me understand what is going on. Thanks.
You are... THE GOAT!!!
Thank you for a very informative and complete tutorial. Much appreciated.
Thanks for the video!! super helpful
What an amazing and informative tutorial for makefiles! Thank you.
What a great tutorial. I’m only jumping back into Linux recently and not only did you teach me about make, but you also provided a great refresher regarding C, lib, includes etc. thanks a lot
This looks great. I saved it to watch later when I have more time. Thanks.
Great video. came with no information and left with all. thank you
this is such a helpful video Gabriel thank you
Thanks for the video and all the great information! A higher resolution than 480p would be lovely for fluent readability.
Why did it take me a month to hear about .d files. I swear nobody mentions those but it’s so confusing trying to understand something when people omit information like that. Thanks boss :)
Yes, just what I need. Thank you
Thank you for making this video. I got a good understanding on make. Thank you very much
Great video and nicely explained. Thank you!
Clear and concise. Thank you!
Great video. Thank you!
Love your tutorials, this is awesome!
Thank you for this video it was really helpful, and I got many insights about makefiles.
extremely useful video.
I could not understand, what is target, rules, dependencies and all that weird symbols...but now I have understood!
Great video of the basics
Excellent video! Thank you for sharing, much appreciate it
Great video!
really awesome lecture video regarding makefiles...
Thanks!
Lovely explanation , Brilliant and I mean brilliant !!
I wish there more of you about who can explain this with the well defined , well thought out descriptions that you have used in your presentation
I understood everything you were doing
Thank you
PS for other people as we now in 2023 who feel AI will give you these answers then Im going to say , "No AI no matter how good you think this is will and no AI will help you and guide you to this depth and understanding This is why we need people like Gabriel Parmer especially in poorer and less educated sectors where this is no help and all controlled by finances "
You need to understand the essentials of this
Im pleased we still have some who have recognised this and know what is required
Its not a case of how quick , how fast or , how many cuts to education our officials can make or where others see AI as a method of doing away with people especially in education
Its very much a case of how the use of AI where people in positions of power need to be very clearly show just how much they dont understand and why persons such as Gabriel Parmer are very much in demand
I hope this justifies all comments I make whilst I highly recommend tutors such as this which we are lucky to have
If only many other people in many an industry would actually wake up to the damage they do by making the cuts they have in conjunction with the loss of real professionals in all industry even more so in teaching and training
Thanks so much for the comment! I'm planning on a series of videos about system tools in the coming months. You've provided me some motivation, so thanks!
EXACTLY what I needed
Great introduction to build systems!
You are a genius educator!
Nice tutorial. Thanks for sharing :)
Thank you so much for these detailed information. I now understand a lot better this mess ^^
Thanks for making this:)
Amazing explanation, I love your teaching style. Thank you
Great job!!! Really appreciate it :D
this is a really cool and useful video, help me a lot, thank you for doing it!, may i ask you which theme were you using for this video if you recall it?
I don't remember the theme, but it was likely "cyberpunk" (github.com/n3mo/cyberpunk-theme.el)
the best tatorial on makefile good job thank you sir
this is a great video. Thank you very much!
Really nice video indeed on makefiles.
That's fantastic thank you very much, I hope you create more content (;
One peculiar error I came across is that if you use the line %.o:%.c and you happen to not have a correlating .c file for one of your .o files (such as not have a x.c when x.o is a dependency for the binary) then it will say there was no rule to make x.o which is required for the binary. You might think that you do have a rule for that because you have %.o:%.c but because there is no x.c it shows that error that might be a little difficult to track down. Alternatively if you list out all the .o's instead of using % then it will tell you that you don't have a rule to make the TARGET x.c needed by x.o which is still kind of weird because x.c is never a target but it does make a whole lot more sense and it is closer to the root cause which is no x.c file.
Thanks for the explanation.
Genuinely the best video about makefiles. In embedded environment is it okay to have a rule for flashing into target hardware or considered a bad practice?
Generally it is fine. However, depending on the development environment, the commands to perform the flashing might differ. So it is not uncommon to have a shell/python script to do the flashing, and simply invoke the script from the makefile.
@@nothing6001 Appreciate your guidance, thank you!
Thanks sir ☺️
Great tutorial for beginners. 👍👍
Wonderful 👏
Great video. Should have explained PHONY just in case someone has a file with the name as the target.
Great video! Informative enough.
I rarely comment on videos, but this one slaps
Great video and description
First of all, thank you ever so much for your valiant effort, it really is appreciated.
However, I rather think that I have little choice, but to accept that I'll never really grasp this subject.
I like to place myself in the category of "actual programmer", by which I imply nothing in regards to occupation, knowledge, talent, experience or intellect. I understand the language of the computer, and it is not abstract, not in the slightest. It is procedural and exceedingly simplistic. For crying out loud, I have issues with something as fundamental as the foreach loop, whereas I comprehend the workings of a "normal" for loop perfectly. The mechanics are visible, for the most part. I can bit twiddle like no other, but ask me to do something simple and mundane to a string, or char array, which obviously I much prefer, I'm done for, at least you won't get a quick answer, and don't even get me started on OOP, polymorphism and god knows what.
I suppose this is just who I am. It has it's pros, like being able to do and comprehend certain things that many, very experienced programmers/developers, don't, and cons, like not being particularly useful to the industry, in the modern world, but I digress.
To simplify things, I have a few very simple questions, which I know there are no simple answers to, but I'd like to hear your opinion.
Splitting a large project into multiple smaller modules, if that is indeed the term, or alternatively libraries, and preferably a main source file, with a single, or at least a few headers, what would be a good hierarchical folder structure, not least to make Make, easier to work with?
Is it in any way shape or form, a good ide to have the main Makefile, so to speak, invoke other Makefiles, which could also, to my mind at least, simplify things(?) I'm thinking that treating each "module" as it's own separate project, with a dedicated Makefile, which can them be invoked as needed, in the larger scheme of things, What are the potential pitfalls? It this something that is actually done? As far as I know, it can be done, but as of yet, I have seen no examples of it, so presumably it is something that is not actually done, and presumably there are good reasons why. I've very much like to know.
Last, but not least, do you know of any source, online or otherwise, which takes you from the very basics, through advanced, and actually explains things on the way? This is by no mean meant as a criticism of you, if anything, you're the first of many, who have actually attempted to explain anything, otherwise it has just been: "Do this, and magic happens.", and in a few key areas, you've had some success, which is impressive. Alas, actual comprehension still seems unattainable, thus this rather longwinded diatribe, hoping to for some... well, hope.
I firmly believe that the praise expressed by just about everyone other than me, is well deserved, but ya, there you have it.
Thanks and best regards.
Thank you so much for the thought out comment!!! I don't know how long you've been programming, but it honestly sounds like you're riding the learning curve up programming, bit by bit. Very little of what we do is natural, and it all takes a concerted effort to learn. You also can't really just learn all of it at once, as there are many conceptual stepping stones to get to many concepts. Even for those going through a well-designed curriculum that walks them through those stepping stones, it is very hard, and takes a lot of time. All of that is to say this: have patience, and cut yourself some slack. It is hard, and takes time, but is all learnable with time and effort.
How you break up the C/C++ and header files, and how to you break up the Makefiles is very project specific. For the source files, you kind of want to think of breaking up c/c++ files into logically cohesive units. Think: the functions that "bin" together, should be in the same file. There are few rules around how big files should be, or any of that, but generally keeping the logic within them manageable and all self-related is the goal.
We split up Makefiles, practically, pretty rarely. A single, top-level makefile can do quite a bit. I can build files in subdirectories, move them around, create executables, etc... The exception to this is if different parts of your project require vastly different ways of being built. For example a library might be separate from your main application code, might be separate from your testing code, each using different Makefiles. But honestly, I don't see this often. In our OS, we have huge hierarchies of Makefiles and I still don't know if that was the right idea ;-)
If you're in the root of your project, and you cant to cause a Makefile in a subdirectly (foo) to the rule "all", you use the following in one of your rules:
make -C foo/ all
For example, github.com/gwsystems/composite/blob/main/src/Makefile#L20-L21 will execute the Makefile in the "components" directory, and execute the "component" rule.
All that said, I'd recommend sticking to a single makefile till it feels like it is being pulled into too many directions. Only then use multiple.
I don't know of great resources to learn a lot of this. Because of that, I recommend simply using it in one of your projects. Start out simple, then dive further in when you have to. The make references are complex, and hard to understand. But it often gives you enough information to try some stuff out, and see what sticks. I wish I had better news on this, but build systems are one of those things that most people learn by doing.
I don't know if any of this is kinda what you're looking for, but a class I taught a year ago was on system programming. Using UNIX APIs and what-not. Might be useful? Might not. Find links here:
gwu-cs-sysprog.github.io/
github.com/gwu-cs-sysprog
Best of luck! Make sure to allow yourself the time to learn. We all start off not knowing much, and not being able to dive deep into complex topics (I always learned quite slowly). Before you know it, you'll know enough to start learning quickly. Your rate of learning only increases as you learn more. But to get to the part of that learning curve that feels "good" takes time. Some thoughts on this: www2.seas.gwu.edu/~gparmer/posts/2018-06-22-compounded-self-improvement.html
@@nothing6001 Well, that was a heck of a lot more than I expected, though of course I didn't expect much, given the age of the video.
I have been developing and programming for decades really, though apart from 3 years in a Job were my official title was indeed IT developer, though I didn't do much programming, never professionally, for the very simple reason, that I simple cannot handle the excessive, and in my not so humble opinion unnecessary and counter productive, abstraktion. Perhaps it has something to do with my autism, who know, but it is what it is.
At any rate, I started out some 25 years ago or so, I even did C++, when when it was still the case that one could actually manage to learn it, before it changed into something utterly unrecognizable, and when that happened, I landed on C, and have never looked back.
However, Build Systems have never been foremost among my interest. If it has been warranted, I have used make, simply by copying what others have done. There has never been any doubt in my mind, that were I to attempt to comprehend the subject, I would fail miserably, but now I actually rather need to, if I want to get anything done in regards to my newest project, which is the most complex I have attempted so far, and that is actually saying something.
At any rate, I am beyond grateful for your effort, answering my, probably not to easy to comprehend ramblings, and I will delve deeper into your answer, when I get the time, and perhaps post some follow up questions, if you don't mind.
I have of course subbed, and hope to see some activity at some point.
Thanks again, and best regards.
@@FelixNielsen Yeah, I want to find the time to make a series of videos that are shorter, and cover "levels" of different technologies. Time is always the problem. This video was just a twitch stream for a class, but I think I could do much better if I focused on the content. Regardless, we'll see if I find the time. If you're interested in OS topics, my video series on that topic are better thought out.
@@nothing6001 Sounds good. Should you need some sparing or ideas, I am actually, though it doesn't always seems like it, quite the tool, in a positive way, of course.
I myself, have been wanting to do a series for a long time now, focusing not so much on a language, paradigmes, specific tools, etc., but on the very fundamentals, on achieving comprehension, problem solving, alternative "out of the box" thinking, heavily biased by my disgust for the prevalent abstraction of the day, obviously. Pretty much inspired by Charles Petzold and his book "Code: The Hidden Language of Computer Hardware and Software", that I read so many years ago, and was the one book that truly lead me to comprehend the inner working of computers, well, comprehend may be a strong word, but strong fundamentals at least, that I fear has not been prioritized for quite some time, and also, more than anyting, fueled my entusiasm. I would describe it as a children's book, to be honest. Even the most tech illiterate, can understand it, and I dare say that even the most praised and talented programmers/developers, might well learn something. Then again, it has been a very long time ago, so perhaps it is utter crap. All I can say is, that had it not been for that book, I probably wouldn't be able to tell and Iphone from a remote control today.
Arg, I'm doing it again. I just cannot sit down in front of a computer, without writing a small novel, my apologies, but my central point remains. Chances are that I will never actually get to that point, but it would please immensely to experience someone else doing so.
There was a time where I was very much into exactly that, OS programming. I stil visit osdev.org, once in a blue moon, and should you ever find yourself there, you might want to look up the ancient, though presumably still incoherent, ramblings, of one Zacariaz.
I may well watch all you have to offer, though chances are *(NULL&(void*)0)--, that I'll ever convince myself, that starting down that path again, is a good idea, in any way shape or form.
Best of luck with it all. I'll be watching.
Would creating a dependency something like: %.c:$(foreach D,$(INCDIRS),$(D)/%.h) have the same affect as using -MP -MD? (other than the fact that it doesn't create .d files of couree... I guess I'm curious the difference btwn using .d files and the above dependency.)
Big difference, actually.
1. You're adding a dependency on all header files, which means a change in any of them will require recompiling all C files. What the .d files do is recompile the C files only if their specific headers are modified (i.e. not all C files include all headers).
2. it won't include the headers outside of your path. This might not matter, but does if you use libraries *and* modify those libraries. Likely this is not a big deal for you.
Very good video! A couple of things though; how properly to place all build files (*.o, *.d) in its own "build" directory and the use of .PHONY. Keep up the good work!
i have a question abut the declaration of CODEDIRS and INCDIRS. What is the meaning of the one dot "." or 2 dot ?
CODEDIRS=. lib #Source code directory
INCDIRS=. ./include/ # Include directory
Is it important to include since i saw other people do not use it
A dot means "this current directory".
Is Make able to detect changes in .h files from system or compiler libraries?
This is an amazing tutorial, very concisely explained. But I have a small question. I'm running on a windows machine, so if I want to utilize the rm -rf command, I have to open a WSL Ubuntu terminal, and everything works just fine. However, if I want to do "make clean" in PowerShell, using either of del or remove-item commands, "make clean" is unable to find the files its supposed to remove. I've tried to make it work but have been unsuccessful. Could I have some suggestions as to how to modify the code so that it works in PowerShell as well?
I don't know any powershell, unfortunately. But this linuxhint.com/delete-files-in-powershell/ says you can simply `rm -fo names*`. Not sure if it is right, or what limitations it has. Sorry!
@@nothing6001 Damn, thank you so much for the answer, big fan of your videos here. Apparently rm is just an alias of the remove-item cmdlet, and it does indeed have some limitations (dammit windows), so it looks like ill be sticking to wsl instead. Again, thank you so much for the answer and the content you put out, if it wasn't for your passion I wouldn't have grasped the concept of makefiles at all.
At the bottom of your last makefile there was a line that started -include $(Depfiles). What did this do? Great video by the way.
It pulls in the C/C++ dependencies on header files.
When you compile the C/C++ with, for example, "-MMD", it will output a dependency file postfix with *.d. It includes rules for which .c files depend on which .h files. The line you ask about simply includes the .d files in the makefile.
Long story short: if you modify a .h file used by a .c file, you want to recompile the .c file. This enables make to understand which .h files each .c file depends on!
That was my question as well. Thanks
@@nothing6001 Thanks
@5:30 how r u able to use instead of "" to #include a user created .h file?
simply says "look for this header file in any of the 'include paths'". When gcc compiles a program, you can set up the include paths using the "-I" compiler flag. For example, `gcc -I. file.c` would look in the current directory (see the "." there?) for any header files included with .
Another example github.com/gwu-cs-os/evening_os_hour/blob/master/f19/10.2-makefiles/02_simple_makefile/Makefile#L2 and a more complex one github.com/gwu-cs-os/evening_os_hour/blob/master/f19/10.2-makefiles/03_featureful_makefile/Makefile#L10
@@nothing6001 ahhh thank you! got it working!
Nice video, thx for sharing
The first half of this video is great, the second half is OK but too much "This is a magic incantation, I'm not gonna explain how or why"
there is a reason this is 95%, not 100%
Hi there,
Why is there a difference in declaring the directories?
I mean, why are both directories not addressed in the same way?
Like either:
CODEDIRS=. lib
INCDIRS=. include
or else:
CODEDIRS=. ./lib/
INCDIRS=. ./include/
No fundamental reason. Your suggestion that they should be consistent is correct!
There are always improvements to be made to code, and you point out a good place where the current Makefile should be cleaned up.
@@nothing6001 oh OK. I thought I was missing something and that there was a hidden purpose to it. My bad 😅
Amazing ❤
When you do %.o:%.c is it running on all the .c files in the directory or is it running on all the .o files listed as dependencies for the binary?
I'll leave this up in case anyone else has this question. It looks like % will run on all the .o files listed as dependencies for the binary not on all the .c files in the directory. I copied his setup and added in a random .c file and it didn't get compiled.
@@jasongharvery Yup. What's on the left of the : is the *result*, the thing to be created. The "dependencies" are on the right.
@@jasongharvery If you want to generate a .o, it would have to be a dependency for another rule. You often see a top level rule, something like:
result.exe: $(wildcard *.o)
Which effectively says: this rule will produce result.exe, but requires all the .o files. This will trigger the rule with the %.o on the left of the colon!
Thank you good sir
It was a great lecture, and I liked it. But one more thing I want to know is, will the same Makefile work if I have c and c++ files in my project? Or do I need to change it in some way?
Great question. Generally, you can just change the makefile from using gcc to instead using g++ to compile c++ (or whatever c++ compiler you use). Other than that, I'd expect everything should be the same.
WOW ,Thank you !
Thanks man!
In %.o:%.c shouldn't we need to add a dependency of header files. We aren't going to use the header file in the rule [$(cc) ......$
No, surprisingly! When we generate the *.d files for each .c file using the gcc -MP -MD flags, it automatically creates the dependencies from the .c file to their corresponding headers. So you only need to include the .d files, and they take care of the dependencies.
Great video, thank you! :D
I only have one question left.
If you want to have tests for your code/module, how would you do it?
I was thinking about using only the necessairy .o files when compiling a given test, with additional .c files, that are needed for the test to compile (like test suites, Test Cases, etc.). So when you want to make a test, you'd say something like 'make DataStructsTest' and it would create the executable of the test (in this example) for some Data Structure module.
Great question. When you're testing an application, you might manually specify only the dependencies you need (assuming the application is multiple object files). However, as you want a "compile test" to be part of your tests, I'd simplify in most cases, and just have your test dependent on all of the .o files of the application.
In the case that you're testing a library, you want to have a dependency on the .a or .so resulting binary, which will likely compile/link the entire library. You want this complete set of dependencies as you want your test to interact with the library in exactly the same way that a client would.
In short, I'd make tests dependent on the closest thing to a final output possible.
Wow super 👌
HI Gabriel parmer
I am facing an issue I think my CFILES variable are not updated, Can u help me
Which Linux Distro are you using in this video?
Nothing fancy, just Ubuntu
well the rule for x.o calls the rule for x.c... but there is no rule for x.c. How does this work then?
You don't need a rule to build the .c file because it already exists! You only need the rules to compile the source files into the objects. Hope that helps.
I’m not an expert (that’s what I’m here for), in this case doesn’t x.c count as a dependency?
wait how did you get there mine doesn't look like that what
why is your /home file called ycombinator ?
This is not as interesting as you'd think. It is my username (of course), and I usually choose my username on different computers based on programming language constructs (like the y-combinator that is the theoretical foundation for repetition).