Wow! I was looking around for exactly this - i.e. echo a bunch of bytes to file, chmod u+x and execute. Super cool. The ELF format turned out to be a bit more complex than I expected and the conversion from assembly to machine code as well. I think the video is a little bit too fast, I'd 1) look at the ~1hr version of it so you can explain each part and maybe add a few more examples, and 2) add ability to sponsor your channel directly here on TH-cam to make it very frictionless. Really appreciate the ELF framework Python code too. Super cool!! 👏
There was a question on Usenet ages ago about what the minimum thing you had to include in a linux distribution It came down to the fact that `cat` could append binary things to a file. { echo is a shell command, so cat was the smallest needed executable }. {Edit: Though, all that said, Im not sure what the most minimal potential shell is. I presume its something that can only run executables}
Thank you for producing this content. There aren't enough resources that cover the lower levels in a manner that's well articulated. I'm no low level expert but everything I do know I had to learn by gathering bits and pieces from across the internet. Sometimes I'd just have to assume certain things, and there are times when those assumptions were very wrong and then my whole mental model changed, several times. I'm glad all this knowledge is being condensed and distilled in such an elegant format all in one place, I wish I'd have had this years ago. Yeah TH-cam videos covering the subject aren't new but they tend to leave relevant details out, aren't well articulated, or sometimes straight up contradict one another. This on the other hand does it perfectly. I struggle to come up with any sort of complaint.. well, there is one actually, just one: the pronunciation of Linux. You earned a sub
What a pleasure finding your presentations. I've written boot sectors and master boot records. One of my favorite tricks is to cast a string as a function pointer and call it. I am fond of computed goto. Many of your stories are like my own. Were we not across an ocean, I'd raise a pint with you and we'd swap stories.
This is the type of video where I constantly have to pause in the middle of you saying something while my mind is blown by what you said one sentence before. I then go back and really go in to the mind explosion. Then continue.
This is a really interesting subject, and really well presented. I think that's the most intutitive way I've ever seen ELF structures presented and explained. I've found that commentary, macros and labels make writing machine language (for any ISA) easier, and with complex instruction sets like x86/amd64 having an interactive assembler for single shot {pseudo,}instruction to machine code is way more convenient than having to manually assemble as required. Look up "mli" (machine language interpreter), "mle" (machine language encoder), and "ias" (the poor man's interactive assembler) if you're interested in some proof of concept.
Brilliant video, thank you so much for making this! I still struggle to get out of the comfortable abstractions of OOP but this will give me a fun way to start breaking through. I have a strange idea of fun. Can't wait to binge all your other videos too!
I feel at 14:03, the instruction shown should be :: mov 0x6000f0, esi Since :: [C7] = OPCODE for "mov" instruction [C6] = Mod R/M Byte => [11] [000] [110], where [110] refers to "esi" register [0xf00060] = Little endian for memory address 0x6000f0 Also because :: Linux syscall for write() takes character buffer in "esi" register. Thank you for this series, Kay. Really loving it!!
Wow, I'll have to rewatch this video a couple of times for all of it to sink in, but this was awesome, thanks! :) For Q&A: Would you share a bit more of your personal journey from starting to work in tech to getting into these kinds of low level topics? Was it a necessity for your job or was it more of your personal curiosity that drove you to them? You mentioned in your introduction video how people get into tech and start working on kind of boring stuff, and how easy it is to just stay on that side of programming because that's what you're being paid to do, but I wonder how this was for you in your particular situation.
In the 8-bit days a lot of us used to program machine code kinda this way. The lucky ones had assemblers but many had to POKE bytes into memory. Usually from a BASIC program. But the OSes, file formats, and CPUs were all a lot less complex too.
Great content and outstanding presentation skills! Not sure if you know anything about demoscene, but this kind of content is closely related to what we (well, personally not that much...) used to do in late 80s and early 90s to get out some visually stunning effects in assembly (e.g., with MC68k). Now, I might be actually referring more to your other assembly videos, not particularly to this ELF. It definitely is worth understanding how computer and microprocessor works at the very low level, even though you're writing your code at the highest level of abstraction, or even using prompt engineering as we do nowadays. One interesting detail to cover could be how to demonstrate visual effects with assembly by writing to and manipulating video memory and possibly some specific registers. Then this would get even closer to what we kids did or tried to do in the 80s and 90s. At that time, there were people (gurus and wizards) who just did that magic with their computers and code, but almost no one was able to pedagogically explain well what actually had to be done to achieve those cool effects! It was like secret science that one had to find out by trial and error, with some code snippets, and by reading those dull hardware reference manuals.
You know I was actually mulling over doing a series on demoscene today! This has added to my resolve. Looking at the code can be pretty intimidating so it'd be cool to break it down a bit for people. Thanks for the comment :)
I suggest using the Intel syntax as used in the table of opcodes. Albeit the AT&T syntax is more common on Linux, it is easy to forget swapping source and destination when manually translating assembly to machine code.
5:53 - yewgh. I remember my first time doing actually _machine language_ coding was in my college CS Computer Architecture class but there we used good ole' Arduino with its AVR4 which is soo much simpler - fixed-length instruction word (though some instructions take 2 words, typically if they involve long addresses). Never did machine language code on messy ass x86. (Right now I primarily run ARM, which I believe also has a fixed length word for most instructions but I have not checked that as I've only looked at the assembler level.)
Thank god you are out here getting into the weeds of these low level mechanisms. It's really difficult to find material on these topics this easy to follow. Please keep going~ Maybe a video about linkers/linker scripts in the future??? :D
@@neoeno4242 yIPIEEE thanks a lot, i had to really dive into linker scripts once i was working on embedded systems, but i still feel i don't understand them that much lol also love the little bits of history you sprinkle throughout your videos~
I recently discovered your channel and really enjoy the content you're creating. I'm looking to learn C and need a textbook that doesn’t overly simplify things-something in line with your teaching style. Any recommendations would be greatly appreciated. Thanks, Kay.
also any register starting with R in x86_64 ISA is a general-purpose register. the one sticking point i have with this convention, and the ISA itself, is that RSP, RBP and RIP qualify as general-purpose. changing them has implicit effects for other instructions--and RIP can only be changed with a jump or call--so they're not really general-purpose in my mind.
@@channelgogrvk Ridiculous that RIP is considered a general purpose register. RBP at least can be used as a general purpose register as long as you don't use it to implement stack frames. So that is something.
@@jamesgray6804 -fomit-frame-pointer 😀 i need to clarify, because the Rxx convention isn't real, it's one i made up after thinking about it for only two seconds, because it is true that every two- and three-letter register that starts with R is general-purpose with the only exception of RIP. i feel like the names should be meaningful and indicate to the programmer what the purpose is and the R-convention (that i made up) comes very close to an exclusive set of general-purpose registers. the x86 ISA is a bit of a monstrosity anyway, so a meaningful naming convention was probably always impossible. as for RBP being a GPR, when i write assembly by hand i tend to always use a frame pointer--just because it's easier for me to keep track of any registers i'm spilling, or any objects i'm using directly on the stack. if you're writing in C you can just tell the compiler -fomit-frame-pointer and it would sometimes generate faster code because now you have an extra register! that was a bigger problem in ia32, though, since for position-independent code the compiler can't use EBX and benefits from the extra GPR. since we have R8-R15 on x64 now, it's a little less useful these days. i often don't see codegen using that many registers anyway (unless you have a function call with like 6 parameters, but i wouldn't write that function)
'Register' is what I figured as well. @@channelgogrvk what do you mean? where did you see anyone consider rip as a general-purpose register just because it starts with 'r'?
I was going to ask if you saw the same video as me, but then you wrote up a conversion program in Python. The one I watched a couple of weeks ago, the guy converted hex bytes using `xxd` and `cut` to strip comments. His program, such as it was, was just a `bash` script. Still, I think it'd be a more rewarding project if you just wrote your own assembler. I keep thinking about doing that and perhaps using a completely scratch designed mnemonics set, because I really don't like some of the instruction names on x86 and both nasm and fasm have weird quirks I don't like. Probably wouldn't be a bad idea overall since I need to have an inline assembly mechanism for my compiler anyway.
this video is cool! what id recommend doing is making this longer and splitting it into parts so you can go more in depth into how this works. for an idea of how to do that, i highly recommend looking into ben eater's hello world 6502 tutorial. also wanted to ask if there's any way you identify, such as if you're an enby or trans. i couldn't find any info on your about section of your account to make sure i get it correct. ❤️ from a transfem who's researching how computers work and going deeper and deeper into the electron level coming from higher level downward
Too bad I am repulsed and disgusted by the speaker and THEIR CHOICE to appear in such a way, because I really enjoy this excellent content, but I just cant stomach it.
Firstly, know your fucking history, we owe a lot to queer folks in tech, including trans women. Secondly. since when do computer nerds give a shit what we look like. This isn't a fashion show. You're a fucking poser. Fuck off.
![⭐️( ลอยกระทง 2024 ) Ver. แดนซ์ Remix BY [ ดีเจกิต รีมิกซ์ ]](http://i.ytimg.com/vi/rMWOLhgXUHc/mqdefault.jpg)
Wow! I was looking around for exactly this - i.e. echo a bunch of bytes to file, chmod u+x and execute. Super cool. The ELF format turned out to be a bit more complex than I expected and the conversion from assembly to machine code as well. I think the video is a little bit too fast, I'd 1) look at the ~1hr version of it so you can explain each part and maybe add a few more examples, and 2) add ability to sponsor your channel directly here on TH-cam to make it very frictionless. Really appreciate the ELF framework Python code too. Super cool!! 👏
Holy shit, Karpathy!
Now looking forward to seeing some magic of transformers generating/manipulating machine code file formats directly!
Noo way I'm working on the same type of things as AK! Nice1, OP. Elf quines are (skyward) **peenchy_fingers**.
There was a question on Usenet ages ago about what the minimum thing you had to include in a linux distribution
It came down to the fact that `cat` could append binary things to a file. { echo is a shell command, so cat was the smallest needed executable }.
{Edit: Though, all that said, Im not sure what the most minimal potential shell is. I presume its something that can only run executables}
Next up, create Neural networks with machine code !
Your content blows me away. I work at the totally other end of things, as high level as you get. You are a gifted educator. Thank-you!
Thank you for producing this content. There aren't enough resources that cover the lower levels in a manner that's well articulated. I'm no low level expert but everything I do know I had to learn by gathering bits and pieces from across the internet. Sometimes I'd just have to assume certain things, and there are times when those assumptions were very wrong and then my whole mental model changed, several times. I'm glad all this knowledge is being condensed and distilled in such an elegant format all in one place, I wish I'd have had this years ago. Yeah TH-cam videos covering the subject aren't new but they tend to leave relevant details out, aren't well articulated, or sometimes straight up contradict one another. This on the other hand does it perfectly. I struggle to come up with any sort of complaint.. well, there is one actually, just one: the pronunciation of Linux. You earned a sub
I pretty much watch these and take notes hoping that I can accidentally learn something via pure osmosis. Awesome content!
What a pleasure finding your presentations. I've written boot sectors and master boot records. One of my favorite tricks is to cast a string as a function pointer and call it. I am fond of computed goto. Many of your stories are like my own. Were we not across an ocean, I'd raise a pint with you and we'd swap stories.
This is the type of video where I constantly have to pause in the middle of you saying something while my mind is blown by what you said one sentence before. I then go back and really go in to the mind explosion. Then continue.
This is a really interesting subject, and really well presented. I think that's the most intutitive way I've ever seen ELF structures presented and explained. I've found that commentary, macros and labels make writing machine language (for any ISA) easier, and with complex instruction sets like x86/amd64 having an interactive assembler for single shot {pseudo,}instruction to machine code is way more convenient than having to manually assemble as required. Look up "mli" (machine language interpreter), "mle" (machine language encoder), and "ias" (the poor man's interactive assembler) if you're interested in some proof of concept.
Brilliant video, thank you so much for making this! I still struggle to get out of the comfortable abstractions of OOP but this will give me a fun way to start breaking through. I have a strange idea of fun. Can't wait to binge all your other videos too!
I feel at 14:03, the instruction shown should be ::
mov 0x6000f0, esi
Since ::
[C7] = OPCODE for "mov" instruction
[C6] = Mod R/M Byte => [11] [000] [110], where [110] refers to "esi" register
[0xf00060] = Little endian for memory address 0x6000f0
Also because ::
Linux syscall for write() takes character buffer in "esi" register.
Thank you for this series, Kay. Really loving it!!
Only seen one of these so far but they content and editing are terrific. Keep em coming, thanks
Thorough introduction, presented so eloquently, thank you!
Wow, I'll have to rewatch this video a couple of times for all of it to sink in, but this was awesome, thanks! :)
For Q&A: Would you share a bit more of your personal journey from starting to work in tech to getting into these kinds of low level topics? Was it a necessity for your job or was it more of your personal curiosity that drove you to them?
You mentioned in your introduction video how people get into tech and start working on kind of boring stuff, and how easy it is to just stay on that side of programming because that's what you're being paid to do, but I wonder how this was for you in your particular situation.
In the 8-bit days a lot of us used to program machine code kinda this way. The lucky ones had assemblers but many had to POKE bytes into memory. Usually from a BASIC program.
But the OSes, file formats, and CPUs were all a lot less complex too.
Thanking you most kindly for all you're information
i really hope you keep making programming videos for a long run.
its very unlikely, they tend not to last very long
The sickest channel with the nicest teacher! Love your explanation, love how 'simple yet pretty complicated' the things you teach are :) Thank you!
I subscribed for the low level programming tutorials.
OMG!!! All your videos are so amazing!!! Thank you so much. I've learned so much from this entire series.
enjoying your lovely video with a broken leg😊
Get well soon!
I really enjoy these videos. Thanks!
How exciting, a new 0DE5 video from only 4 hours ago! 🎉
Really enjoying these!!! Awesome work. Thanks.
Great content and outstanding presentation skills! Not sure if you know anything about demoscene, but this kind of content is closely related to what we (well, personally not that much...) used to do in late 80s and early 90s to get out some visually stunning effects in assembly (e.g., with MC68k). Now, I might be actually referring more to your other assembly videos, not particularly to this ELF. It definitely is worth understanding how computer and microprocessor works at the very low level, even though you're writing your code at the highest level of abstraction, or even using prompt engineering as we do nowadays.
One interesting detail to cover could be how to demonstrate visual effects with assembly by writing to and manipulating video memory and possibly some specific registers. Then this would get even closer to what we kids did or tried to do in the 80s and 90s. At that time, there were people (gurus and wizards) who just did that magic with their computers and code, but almost no one was able to pedagogically explain well what actually had to be done to achieve those cool effects! It was like secret science that one had to find out by trial and error, with some code snippets, and by reading those dull hardware reference manuals.
You know I was actually mulling over doing a series on demoscene today! This has added to my resolve. Looking at the code can be pretty intimidating so it'd be cool to break it down a bit for people. Thanks for the comment :)
This was excellent, thank you so much for sharing your knowledge!! 🙂
I loved crafting the smallest possible elf's. Cool video
I suggest using the Intel syntax as used in the table of opcodes. Albeit the AT&T syntax is more common on Linux, it is easy to forget swapping source and destination when manually translating assembly to machine code.
A lot better than the ELF "spec".
5:53 - yewgh. I remember my first time doing actually _machine language_ coding was in my college CS Computer Architecture class but there we used good ole' Arduino with its AVR4 which is soo much simpler - fixed-length instruction word (though some instructions take 2 words, typically if they involve long addresses). Never did machine language code on messy ass x86. (Right now I primarily run ARM, which I believe also has a fixed length word for most instructions but I have not checked that as I've only looked at the assembler level.)
I like my executables artisanally crafted
Thank god you are out here getting into the weeds of these low level mechanisms. It's really difficult to find material on these topics this easy to follow. Please keep going~ Maybe a video about linkers/linker scripts in the future??? :D
Thanks! Such a good idea too - I got really interested in linkers when making this video so we'll definitely come back to that topic.
@@neoeno4242 yIPIEEE thanks a lot, i had to really dive into linker scripts once i was working on embedded systems, but i still feel i don't understand them that much lol also love the little bits of history you sprinkle throughout your videos~
Keep it going, Kay. 👍 (:
Peak programmers creates executables in raw bytes.
A squectangle is a rectangular squircle.
I recently discovered your channel and really enjoy the content you're creating. I'm looking to learn C and need a textbook that doesn’t overly simplify things-something in line with your teaching style. Any recommendations would be greatly appreciated. Thanks, Kay.
Want to get a good grounding without the crap ? Get the K&R text and read it. That's how I learned C with the help of Borland C.
yashwant kanetkar
My personal recommendation is C How to Program by Deitel
very very nice vid, thank you very much.. high quality video.
Great video
AFAIK, the 'r' in RAX just stands for register.
also any register starting with R in x86_64 ISA is a general-purpose register. the one sticking point i have with this convention, and the ISA itself, is that RSP, RBP and RIP qualify as general-purpose. changing them has implicit effects for other instructions--and RIP can only be changed with a jump or call--so they're not really general-purpose in my mind.
@@channelgogrvk Ridiculous that RIP is considered a general purpose register. RBP at least can be used as a general purpose register as long as you don't use it to implement stack frames. So that is something.
@@jamesgray6804 I prefer to think of it as *really* extended (:
@@jamesgray6804 -fomit-frame-pointer 😀
i need to clarify, because the Rxx convention isn't real, it's one i made up after thinking about it for only two seconds, because it is true that every two- and three-letter register that starts with R is general-purpose with the only exception of RIP. i feel like the names should be meaningful and indicate to the programmer what the purpose is and the R-convention (that i made up) comes very close to an exclusive set of general-purpose registers. the x86 ISA is a bit of a monstrosity anyway, so a meaningful naming convention was probably always impossible.
as for RBP being a GPR, when i write assembly by hand i tend to always use a frame pointer--just because it's easier for me to keep track of any registers i'm spilling, or any objects i'm using directly on the stack. if you're writing in C you can just tell the compiler -fomit-frame-pointer and it would sometimes generate faster code because now you have an extra register! that was a bigger problem in ia32, though, since for position-independent code the compiler can't use EBX and benefits from the extra GPR. since we have R8-R15 on x64 now, it's a little less useful these days. i often don't see codegen using that many registers anyway (unless you have a function call with like 6 parameters, but i wouldn't write that function)
'Register' is what I figured as well.
@@channelgogrvk what do you mean? where did you see anyone consider rip as a general-purpose register just because it starts with 'r'?
Ooooooh my goodness, thank you so much, i was looking for this whole my life
Awesome!!
Good lord these videos are so great!
I want to learn this party trick, but I'm doubtful anyone I know would be patient enough for it 😂
I was going to ask if you saw the same video as me, but then you wrote up a conversion program in Python. The one I watched a couple of weeks ago, the guy converted hex bytes using `xxd` and `cut` to strip comments. His program, such as it was, was just a `bash` script. Still, I think it'd be a more rewarding project if you just wrote your own assembler. I keep thinking about doing that and perhaps using a completely scratch designed mnemonics set, because I really don't like some of the instruction names on x86 and both nasm and fasm have weird quirks I don't like. Probably wouldn't be a bad idea overall since I need to have an inline assembly mechanism for my compiler anyway.
Where was this video 5 years ago when I was writing my assembler
hell yeah awesome video, when are you going to do linux kernel from scratch
Can’t wait
he likely won't last that long
Wow , this is even lower than Assembly 😊
@@robertkiestov3734 weirdos cant stand seeing a trans woman thriving
@@robertkiestov3734cope
@1:27 please, what software did you use to make these diagrams in your videos
Thanks!
I needed this. Thanks!
this video is cool! what id recommend doing is making this longer and splitting it into parts so you can go more in depth into how this works. for an idea of how to do that, i highly recommend looking into ben eater's hello world 6502 tutorial. also wanted to ask if there's any way you identify, such as if you're an enby or trans. i couldn't find any info on your about section of your account to make sure i get it correct. ❤️ from a transfem who's researching how computers work and going deeper and deeper into the electron level coming from higher level downward
what the fuck
Hi! Thanks for the feedback :) I am a trans woman yes. Good fortune on your quest to the electron!
@@neoeno4242 you are a man!
@@neoeno4242 not a woman…
@@neoeno4242 Have you gone to therapy?
This channel is awesome
What do you use to create illustrations in the vid? 🙏
This one was After Effects:)
Thank you for sharing this amazing content. Which books do you suggest to learn this low level stuff?
Can we support you on Patreon or similar?
💖💖💖💖
ty
Alright but how can I center a div?
Having done digital forensics for years, I’ve come to the conclusion after watching all these videos …. This is the woman I should marry.
Uhh... that's not a woman...
@@robertkiestov3734can you please make an effort to not make people's lives worse with your actions, when you have a choice to do so?
@@justawatchin2 this man’s “life” has a coin-toss-chance of ending within the next 5 years anyway
@@robertkiestov3734 are you maybe projecting a bit?
@@chfourchfour it just went up to 70%
what do you use to do the visuals?
This is amazing
Very interesting video, but why do you say Linux like that?
Is that Griffith?
Wow
how is rhis the first video im seeing from this channel
Can you do for c
Possibly! I made something a couple videos ago which you may find useful - th-cam.com/video/5aZiRjgSGQU/w-d-xo.html
Never heard anyone pronounce Linux like that.
now do linking with libc
I knew
lAinux - thanks, I hate it xd
I get it, but really? executables from bytes, seriously?
look at smithforth
have u considered starting a discord
Oh these seductive click baits with binary data... 😊
truly hardcore hehehe
85th
crafting raw bytes from using your fingernails to turn each transistor on and off
why does your mouth not really move?
Parece o felca
Felca depois do estrogênio
wat
Sounds like reading an x86 assembly language book at high speed. Not a great idea for a topic as complex as this.
What the ff sound and vouce totally un match
Brother you must either cut your hair or grow your beard
It's an emergency
Im not sure but i believe they are trans
@@weakness9343 unfortunately it might be true but i don't want to think of him like this
what a disgusting comment to put under such a wonderful teacher.
go bring your hate somewhere because this is not the place.
Too bad I am repulsed and disgusted by the speaker and THEIR CHOICE to appear in such a way, because I really enjoy this excellent content, but I just cant stomach it.
pluh
You could have chosen to not watch and not comment. Instead, apparently it matters so much to you that you watched and commented. Priorities.
Firstly, know your fucking history, we owe a lot to queer folks in tech, including trans women. Secondly. since when do computer nerds give a shit what we look like. This isn't a fashion show.
You're a fucking poser. Fuck off.
Who gives a shit what you think?
I loved crafting the smallest possible elf's. Cool video