If the instructions of the .EXE files are for the processor, why there are differences with the unix/linux executable files? Or there isn't? Windows/DOS filters those instructions? Or there is a kind of instructions for OS, and others for CPU?
Because the EXE files doesn’t just contain instructions for the processor. It contains a structure. These structures are different in different operating systems. Additionally, even though the _processor_ is capable of executing the same instructions no matter which OS, how those instructions are actually _used_ differs. You can’t build a cabinet with instructions that show you how to build a chair, even if you are able to understand both instructions. Any time your program needs to do anything other than raw computation, it has to ask the operating system to do that. This is called a “system call”, or _syscall_ for short. For example, to open a file, you need a system call. The exact means to perform that system call differ from operating system to the next. There are also conventions and rules you have to follow, which vary from OS to the next. Rules such as which registers are used to pass function parameters. This set of rules is called ABI, application binary interface. If your program is compiled for the wrong ABI, no good things happen.
Rudimentarily, yes, the code in EXE could be run by all OSs if it's the right CPU, but that would be a particularly useless or dangerous code. If an executable wants 4 GB of memory and access to your GPU to draw something on screen it doesn't just go and grab them from who's currently using them, instead it asks nicely the OS, to which the OS might reply here you go or nope in various ways. If some code doesn't follow this procedure: - it's doing something rather useless like adding 1 to register A a dozen times, without the ability to ask the user/OS for anything or tell them anything back - it's asking the OS in the wrong way therefore the OS will shut it down for doing something illegal - it's actually interacting with the user, but without asking the OS, so it's probably a virus targeting an exploit which gives it power over hardware resources and it's hopefully caught by someone(user)/something(antivirus) before it can do that And even before all that, the EXE file must first ask the OS to load the code which will run on the CPU, if the EXE file does that wrong it fails before even starting.
Different OSs have often different structures in their executables, as well as different sets of system calls (APIs) and ABIs, as Bisqwit described. And other things, too, such as filesystem organization. However, if the code itself is for the same processor, it may be possible to run a program from another OS, without emulation and therefore with usually little loss of performance, by providing a compatibility layer that implements the other OS's APIs, ABIs, behaviors, etc. A well-known example of such a layer is Wine, software for Linux and Mac OS that allows to run Windows software. It has varying degrees of success, depending on the APIs used (the most iffy IME is video playback, which requires appropriate codecs that frequently don't work well), but works pretty well for a lot of things. Besides implementing Windows APIs, Wine also creates directory trees to show to applications so they see something resembling a Windows filesystem (most would freak out if they don't find certain things they expect).
Good question. At the end of the day, the programs do not run all on their own necessarily, they also make calls to the OS itself routinely to tell them that they are still working, to request resources, to offload complex operations such as handling networking protocols and moving images on screen, making sounds, etc. This is all handled by the OS itself, and while all of the instructions will complete successfully regardless if you're running Linux or Windows or DOS or even Mac OS on some machines, that does not mean the programs will be able to do anything because they still need those basic "Primitives" that are part of the OS itself.
I am relieved. I was worried with the virus around... I contemplated emailing him but saw that some of his videos have 4-5 months time difference. Honestly I'm happy he is back.
Thank you for your concern, both of you. I have been fine. But my stress levels have been quite high and energy levels quite low, lately. Thankfully I was able to put this video together nonetheless!
@@Bisqwit Coffee is good at restoring physical and mental energy(antidepressant), but it can cause a slight spike to blood pressure(might contribute to stress?) and the possibility of manic behavior.
What is heavily complicated for me to understand is how these assembly programs in 80s and 90s could access video functions to draw things on the screen. I have a huge curiosity on how these binary and hex calculations become the first user interfaces and games inside companies like Atari, Nintendo, the Xerox interfaces and Apple with Macintosh's, etc . WIsh someone could make a video about that....
@@Bisqwit the algorithm kind of brought me here and I loved that part the most 🖤I might even look for more old computer booting up...Its sounds, tempo, code... its relaxing and amazing to watch
A little disappointed the MZ EXE header/structure wasn't touched on, as that is what I thought the video was about, but it's still an entertaining look at what x86 code actually is.
Sorry about that. The 16-bit DOS EXE file is simple enough that it would be possible to cover in a time about the same as this video, but when we get into NE/PE, the background concepts become far less approachable and require a lot of explanation, limiting the options to 1) a very long video or 2) a video that most viewers don’t understand.
Thanks @Dzouje! I really appreciate the sentiment. However, the problem is, longer videos take _much_ longer time to produce… Because it’s not just the labor, but also the time needed to think of a good script / representation. The relationship between video length and production time is almost exponential in most cases, at least of my genre.
4:30 "Good afternoon this is your captain speaking, welcome onboard Flight 4B7 with service from San Francisco to New York. We are currently third in line for take-off and are expected to be in the air in approximately seven minutes time. We ask that you please fasten your seat-belts at this time and secure all baggage underneath your seat or in the overhead compartments. We also ask that ... "
That was highly informative for a laymen such as myself who only deals in environments. Looking at numbers makes my head spin. ASCII looks daunting, the 8008 chip really does simplify what is today a maze of a processing instruction. It's amazing anyone could make a program let alone multiplayer servers etc with tons of stuff happening both inside and outside. Just all looks so daunting...
I've been working on a small game and that section with the old PC perfectly fit the vibe I wanted to go for in it, and ever since I've been down a rabbit hole of foley work, digging up videos of the music of FDDs. thank you greatly.
Glad to see more content from you, Bisqwit. I've been a fan of your content for years, but never commented on your videos until now. Your content helped to rekindle my love of programming and I've recently had time to dive back into it. I look forward to more videos and I wish you the best. //Draco
I may not fully understand everything you said, but it has blown my mind and made me appreciate the advancement of modern computers so much more. Thank you for making this!
I fear an Unicode video would need to be 1 hour long :D I think the other parts of the .exe file could be just as interesting, think self extracting archives, etc.
I'm so sorry but I legit thought at 8:53 you said "Inside the minecraft processor" and I got a bit excited :D Awesome explanation as always. Love the little soundeffects and clipart. It really adds to the whole!
I first opened a binary around age eight-to-twelve, via a plaintext editor, and was presented with gobbledygook just as you describe. I recall querying Google around 2002 with queries like "how to make an EXE" and "how to make a DLL." Ahahaha.
I remember opening an exe file with notepad as an 8yo trying to circumvent the WPA process for WMP10 back then 😆 hoping that I would find the if statement in there to comment out lol 😂
Jumping in here, just to let you know that I love this content, and that the work you put into Simon's Quest is absolutely transformative in making a definitive edition of the game. Thank you sir!
Thank you! I still wish one day I could finish the two big parts I am working on: - Changing the town doors visually to close for night & open for day - Music for the prologue
It is so nice to see Bisqwit videos :) As many people here are happy to see you again. this topic is really interesting. it is interesting how many emulators tries to mimic not only the instructions but the number of cycles for each. some effects on some emulators were just possible due to some hacky abuse of how the processor worked. that's fascinating. imho, it is even more incredible because with very few resources, no internet (not as complete as today) people accomplished amazing deeds. one of my favorites is Rollercoaster simulator. there is something magical about going from some wiring specifications at processor level to a full experience we carried for life.
Bisqwit, inbetween this video and the last, I built an 8-bit CPU across many breadboards and wrote my own little assembly language for it. The whole process was such an eye-opener. This video was really terrific in solidifying what I learned, thank you :)
I love your videos! You have a very good and original way of teaching this passion. You are a very good teacher! Thank you very much for making such great videos!
That was amazing, great editing and very informative. I also love that you do videos where you reverse engineer old games. I heard Pokemon was first written in the Gameboy specific ASM lang and that is just insane. Hope to see more from you. BTW I liked hearing the sound of that old computer. Gained a subscriber today!
Most games on the 8-bit consoles, including Gameboy, were completely written in assembler. It is not even that hard, when you gain experience. Laborsome, sure. But then again, you can achieve certain things with surprisingly little amount of assembler code when your entire hardware is designed with that point in mind.
It was very nostalgic to hear the familiar sounds of a Tandy 1000 trying hard to follow simple instructions. I miss struggling to convince it to play games.
Cool! I really liked the video. I knew about many of the different pieces but somehow you managed to put all the knowledge together in a way that made sense. Thanks!
I did. Nobody is stalking you specifically. This is what I spend 15 minutes doing 3~4 TIMES A DAY nowadays on my channel. th-cam.com/video/YPU4Galm9yk/w-d-xo.html Thanks TH-cam, for doing NOTHING.
8:56 I understand that: 1) in 8086 there is a Table containing bit sequences at an irregular interval that represent commands 2) the 8086 loops through the Table to match each read byte from the .exe 3) Therefore the program runs according to the equivalent assembler's instructions All thanks to the Table commands and the fact 8086 can match the next read .exe command to one of them. Is that correct? If that^^^ is true, does it mean Windows 11 won't work on 8086 Mostly because Windows 11 is an .exe that has non-compatible .exe commands?
Well the CPU uses something quite a bit more effective than _looping_ through the table. It’s electronics, so it can just try hundreds of transistors (of the table cells) _simultaneously_ instead of sequentially. Those transistors that fire will contribute towards the operation, and those that don’t, don’t. EDIT: For clarity, I mean the process of identifying a single instruction. It doesn’t need to do a "is it this? No, how about this? Well, how about this?” process to identify the instruction; it can just test all alternatives simultaneously. But the instructions themselves are still processed sequentially, as the CPU usually needs to finish executing the previous instruction before it can do the next one. (Modern CPUs find ways to cheat this limitation, but it was still very true on the 8086.) The reason why 32-bit or 64-bit EXEs don’t work on MS-DOS is that the file has a particular structured format, that the operating system has to parse, before it can let the CPU unravel the raw data within. If the file does not have the expected format, the OS will reject it. This is why DOS programs won’t work on Windows, Windows programs won’t work on DOS, and neither will work on Linux and so on. (It’s crazy because the OS is a computer program too. It is a program that analyzes and controls other programs.) But let’s assume that all this was magically sorted out, and the OS accepted the file. Then, the CPU would try to run the code. The CPU would interpret the code according to its _own_ lookup tables. Those lookup tables are different for 16-bit code, 32-bit code and 64-bit codes. A processor that is running in 64-bit mode (such as a modern Intel or AMD processor) would interpret the same bytes very differently than a processor running in 16-bit mode (such as the 8086). It would do different things. If the code was not designed for that processor, it would do things that do not make sense, such as access invalid memory addresses, perform nonsense calculations and so on, like a person who returns homework for the wrong week or wrong course. The answers don’t make any sense together with the questions. Modern processors are designed to interrupt when invalid operations are performed, and to signal the operating system that something went wrong. On Windows or Linux, this would appear as the program crashing (while the operating system will continue operating normally). On MS-DOS, which does not have any protection measures, it would appear as the entire computer rebooting or becoming totally unresponsive, possible with some data corruption.
Really apreciated the emulator reference. I vaguely remember that video, but, back then, I thought it was just an emulator, I hadn't realised the "spin“ on it. Also you made me realise I probably never tried to open a binary file with a text editor. Windows helped with that. But I have broken many a terminal (emulator) sessions since I started using Linux, by `cat`ing binaries.
Not all bytes of a mashine code are visible as an ASCII, but with a group of 8 bits, or as a hexadicimal value for example. The hexadicimal value of one byte is divided into 4 bits with the high nibble and 4 bits with the low nibble of a byte. And the bytes can combined together to build a 16 bit or 32 bit or 64 bit, ....value depending on the instructions of the mashine code. Some bytes are a part of the instructions and some other bytes can contain a table with simple values for a routine.
If you don't mind me asking, how did you create the nice effects (like the numbers flying at the screen) in this video? Did you use a commercial video editing package or write your own (or maybe even the flying numbers is a screensaver's output), etc.? For my videos (channel associated with this username), I pretty much only use effects/transitions/annotations filters I have written, but I am open to using other programs as well (for example, hopefully I will have a reason to use manim (from 3Blue1Brown) in the future (even though it means coding in Python)). (I have used LaTeX to generate nice looking equations. Manim also uses LaTeX) I don't need to know about the annotations - I already have that covered in my filter code. BTW, I can totally empathize and agree to an answer you gave to one of the comments - it does take a long time to create these type of videos (script, editing, effects, etc.).
The numbers flying at the screen effect is actually from here: www.alldesigncreative.com/video-footage/green-effects/animated-numbers-background-green-screen-effect/ I first considered rendering it in POV-Ray, but finding an existing animation shortened my production time.
@@Bisqwit Thanks for responding (and for the link)! Coincidentally, in January I added green-screen capability to my video overlay filter, so that site's effects will really come in handy.
One thing you might cover next @Bisqwit is how to actually make and use .so and .dll files. Many C++ programming books just lead the reader to compile simple examples pertaining to syntax and features of C++ but do not expand on what it means to compile a modular reusable library. They are taught that you can include code with your main.cpp file but do not elaborate on the use and creation of shared objects.
Shared libraries are an operating system level concept, not a language concept. At best, it’s compiler-specific. I might cover that topic at some time, but to be honest I’m not exactly familiar with the details either. Especially when we get to library versioning…
This exact question has been in my mind for the last 2 days or so. Can you read minds? Whenever I have a question, I can now just pray that god bisqwit will upload his answers.
There's some forum threads scattered around the Internet from Mac users asking how to get out of "DOS mode", in the mistaken belief that this mode is preventing a program they want to run from functioning. Take a wild guess how they came to this conclusion.
What is heavily complicated for me to understand is how these assembly programs in 80s and 90s could access video functions to draw things on the screen. I have a huge curiosity on how these binary and hex calculations become the first user interfaces and games inside companies like Atari, Nintendo, the Xerox interfaces and Apple with Macintosh's, etc . WIsh someone could make a video about that....
One of my future video plans has actually been to explain how graphics worked in CGA, EGA and VGA (and possibly MDA and Hercules, even though I have not much experience with those). As for video game consoles, each of them is a separate case. There already exist videos in TH-cam that explain how graphics work on the 8-bit NES (Nintendo), if you want to look them up.
The "Retro Game Mechanics Explained" channel has an absolutely brilliant series of videos on how the SNES works and how it's programmed, if that's not too modern for you :)
Wow, this was way more involved than I was expecting. I graduated with a Bachelors in Computer Engineering last year(poor timing) and I have actually developed an instruction set architecture from scratch and implemented it in Verilog code AND also ran that on an FPGA AND also ran a program written with that ISA. Very fun projects, I wish the classes weren't so crunched for time. Sadly, I still haven't found a job(actually I have, but I quit because it wasn't what I thought it was) so none of this knowledge has thus far proven practically useful. I'm not trying to discourage anyone from getting a Computer Engineering degree, maybe my bad luck is just due to the pandemic, but also, most companies right now just aren't interested in hiring people that even know what an ISA is. They need front end web developers, and mobile app developers. And those are the exact kind of jobs that I have gotten offers(and have stupidly turned down, but I'm stubborn I know) from.
Hi Bisqwit, I have a curiosity: have you ever worked with screensavers? I just finished constructing a working prototype, and among my first discoveries are that a screensaver is actually a Windows executable, and that its entry point is predefined inside a static library (ScrnSave.lib, specifically). Programming one from scratch has been proven to be challenging in unusual ways. Many blessings to you too 🙏. P.S.: A friend of mine recently built an 8086 computer on a breadboard; he's going to love this.
This is so fascinating. Can you make another video considering ARM Arch 64, like the one in the new Microsoft Surface or the Apple Macbook Pro? It'll be really interesting to compare the instructions given they have a different OS.
I love these kinds of videos from you. Simple and understood. That said at some point a hope you do cover NE/PE. Maybe in parts? So its not one long video. Either was thank you for this video.
You are back, woho ah the unfortunate consequence of programming for your daily job is that after you get mentally tired home you cannot do anything more then lay down and sleep, watching you channel history (when you drove a bus you had a lot more time and energy to program at home) but you get more money now so thats a good thing even if the job is a bit more stresful
I used to copy the content of the notepad to another one and saved the exe and try to run in and it didnt work, I also used to try "mix" programs by combining the contents shown in notepad and it also didnt work
@5:30, haha what a gentle beast! My ears are still ringing from the CRT though. Please apply a low pass filter in future videos with a CRT, it’s so loud! :)
I remember doing this myself in the "old" days when I still had 486DX2 and later Pentium machines, but since windows took over it kinda took a backseat. I miss the good old DOS days sometimes.
*There is no text, only numbers* I think this might be the most important thing to understand when working with software in general, understanding that everything is just binary data representing other forms of data isn't explained often enough. This may be why a lot of the developers I know personally have struggled with languages like C when first getting in to them, it's not so much a lack of skill or competency but merely a different perspective you have to adopt when working with them that feels very esoteric at first. I'm really glad to see someone going over this though, understanding how binary data works is probably one of the cooler parts of programming.
When I was a kid, the name I used to refer to these types of files that aren't meant to be interpreted as text was, quite ironically, "ASCII files", because I saw a whole bunch of extended ASCII characters and knew that's what those characters were called. I thought "ASCII" (read: raw machine code interpreted as text) was the language that all the pros coded in, and I thought it was (figuratively) magic. And I didn't realize that not all "ASCII" files were created equal-I remember one time I suddenly realized I could draw a smiley face in Paint, save it as a ".exe" file, and I'd have a valid "ASCII" file that would somehow run as a program. I was excited to see what the program would do, and naturally a bit disappointed when I got the inevitable "not a Win32 application" error. (Though my guess wasn't anything spectacular; I was picturing it just displaying the image in an otherwise-featureless window.
I have a Tandy 1000sx that is kinda dead, and that boot up sequence brought me back to my childhood (Yes my first computer was this when i was 4 and im a 90s kid)
This is fascinating on so many levels. Think about this: You have 8 light bulbs and 8 switches in another room. That plus a lookup table is practically a CPU. Well, since two humans on either end would be involved and the instructions would be executed on paper probably the slowest CPU of all time, but you could send instructions and the other human in the other room could execute them. Now think about this a bit more. Think about your smartphone and all the incredible things it can do: render 3D graphics, connect wirelessly to the internet, snap high resolution images (and even edit them in real time) and many many more. Now.. think about, that all these things are fundamentally possible because of simple switches. If some decades ago you pointed at a electric switch and said: "Some day these switches will enable us to fly to the moon." you would be called crazy. Even today, if you told most people that their phones and computers work with billions of switches and not much else (okay, very simplified), they would'nt believe it. Now, for all the god-beleivers: Does your computer die, when you turn it off? Does it have a soul? Most would say 'no'. So why should a human have a soul? If you see what incredible complex things can be made from a lot of simple things, don't you begin to question that humans are nothing else? A machine run by billions and trillions of very small machines which run on relativly simple, chemical and physical rules. Maybe it's just me, but the technical advancements of the last years have shown me one thing: We are not as special as we think and complexity is created from simplicity. Food for thought. Sorry for my bad English, it's not my first language.
Could you do something similar, only with like multithreaded work? I'm interested in how the other cores "know" from the instructions that this should wait etc.
Hmm, I’m not sure I can make a good video around that idea, but I can maybe answer shortly in a comment. There are two levels of mutual exclusion going on. First, when multiple cores access the same data, the CPU itself ensures cache coherency. You can read Ulrich Drepper’s article on memory here: lwn.net/Articles/250967/ It touches the issue, particularly in part 2. Second is operating system support. The operating system offers facilities for mutual exclusion. An application program requests for a lock, and the operating system puts the process on a waiting queue, from which it wakes the process once the awaited event has transpired.
it doesnt quite use a look up table to look up the instruction and then perform it. the binary value of the number, the 1´s and 0s actually turn on and off specific logic units within the processor and at precise timing intervals causing it to perform logical operations. for example the first 1 in the instruction 11010011 might go to a wire that turns on the ALU. the next 1 might go to a wire that latches a certain registers contents onto the wire. the CPU is performing these things dumb and blind. its just logic gates turned on and off with the cpu clock.
If you have studied the 6502 through Visual6502 or for example this diagram www.witwright.com/DonPub/6502-Block-Diagram.pdf , you will notice there is in fact a lookup table. The lookup table contains 130 × 21-bit wide patterns, that match the opcode and the clock cycle and some flags using a bitmask, and each either enable or disable certain paths in the CPU. Modern CPUs are more complex because of superscalar execution and a microcode engine, and the exact techniques are not disclosed publicly. My explanation was a trivialization.
Kiitos tästäkin videosta! 8086:n valtakaudella assembler oli vielä ulkoa opittavissa. 386 toi hirveän määrän lisää rekistereitä ja mahdollisuuksia - alkoi mennä liian monimutkaiseksi mulle, joten vaihdoin C:hen. AMD64 on jo sellainen himmeli, että on turvallisinta jättää assembler-koodaus harvoille ja valituille erikoishenkilöille. :) Onneksi konetehot ovat nykyään sillä tasolla, että tavallisia ohjelmia ei tarvitse lähteä suunnittelemaan binäärikoodia vääntäen.
Kiitos! Joo, assembler-koodausta tarvitsee nykyään tosi vähän x86:lla. Lähinnä sellaiset tapaukset, missä SIMDistä on hyötyä, tarvitsevat usein asmia, koska kääntäjät ovat vieläkin aika heikkoja optimoimaan SIMDiä tehokkaasti, ja intrinsiceissäkin on toisinaan ongelmia.
If the instructions of the .EXE files are for the processor, why there are differences with the unix/linux executable files? Or there isn't? Windows/DOS filters those instructions? Or there is a kind of instructions for OS, and others for CPU?
Because the EXE files doesn’t just contain instructions for the processor. It contains a structure. These structures are different in different operating systems.
Additionally, even though the _processor_ is capable of executing the same instructions no matter which OS, how those instructions are actually _used_ differs. You can’t build a cabinet with instructions that show you how to build a chair, even if you are able to understand both instructions. Any time your program needs to do anything other than raw computation, it has to ask the operating system to do that. This is called a “system call”, or _syscall_ for short. For example, to open a file, you need a system call. The exact means to perform that system call differ from operating system to the next.
There are also conventions and rules you have to follow, which vary from OS to the next. Rules such as which registers are used to pass function parameters. This set of rules is called ABI, application binary interface. If your program is compiled for the wrong ABI, no good things happen.
Rudimentarily, yes, the code in EXE could be run by all OSs if it's the right CPU, but that would be a particularly useless or dangerous code. If an executable wants 4 GB of memory and access to your GPU to draw something on screen it doesn't just go and grab them from who's currently using them, instead it asks nicely the OS, to which the OS might reply here you go or nope in various ways.
If some code doesn't follow this procedure:
- it's doing something rather useless like adding 1 to register A a dozen times, without the ability to ask the user/OS for anything or tell them anything back
- it's asking the OS in the wrong way therefore the OS will shut it down for doing something illegal
- it's actually interacting with the user, but without asking the OS, so it's probably a virus targeting an exploit which gives it power over hardware resources and it's hopefully caught by someone(user)/something(antivirus) before it can do that
And even before all that, the EXE file must first ask the OS to load the code which will run on the CPU, if the EXE file does that wrong it fails before even starting.
Alot of the instructions call system api resources. Search up how to write hello world using x86 assembly.
Different OSs have often different structures in their executables, as well as different sets of system calls (APIs) and ABIs, as Bisqwit described. And other things, too, such as filesystem organization. However, if the code itself is for the same processor, it may be possible to run a program from another OS, without emulation and therefore with usually little loss of performance, by providing a compatibility layer that implements the other OS's APIs, ABIs, behaviors, etc. A well-known example of such a layer is Wine, software for Linux and Mac OS that allows to run Windows software. It has varying degrees of success, depending on the APIs used (the most iffy IME is video playback, which requires appropriate codecs that frequently don't work well), but works pretty well for a lot of things. Besides implementing Windows APIs, Wine also creates directory trees to show to applications so they see something resembling a Windows filesystem (most would freak out if they don't find certain things they expect).
Good question. At the end of the day, the programs do not run all on their own necessarily, they also make calls to the OS itself routinely to tell them that they are still working, to request resources, to offload complex operations such as handling networking protocols and moving images on screen, making sounds, etc. This is all handled by the OS itself, and while all of the instructions will complete successfully regardless if you're running Linux or Windows or DOS or even Mac OS on some machines, that does not mean the programs will be able to do anything because they still need those basic "Primitives" that are part of the OS itself.
The legend has come back.
I am relieved. I was worried with the virus around... I contemplated emailing him but saw that some of his videos have 4-5 months time difference. Honestly I'm happy he is back.
Thank you for your concern, both of you. I have been fine. But my stress levels have been quite high and energy levels quite low, lately. Thankfully I was able to put this video together nonetheless!
@@Bisqwit try smoking weed or taking mushrooms!
@@Bisqwit Coffee is good at restoring physical and mental energy(antidepressant), but it can cause a slight spike to blood pressure(might contribute to stress?) and the possibility of manic behavior.
@@EasyBCLS what in the world bro?
I loved that you showed off those computer sounds btw.
Thank you! I debated it a little, but eventually I decided that at least _I_ love to hear those sounds, and I don’t like to be a slave of algorithms.
@@Bisqwit and that's why your channel is great :)
What is heavily complicated for me to understand is how these assembly programs in 80s and 90s could access video functions to draw things on the screen. I have a huge curiosity on how these binary and hex calculations become the first user interfaces and games inside companies like Atari, Nintendo, the Xerox interfaces and Apple with Macintosh's, etc . WIsh someone could make a video about that....
@@raf.nogueira Explanation is in this video at 1:38.
@@Bisqwit the algorithm kind of brought me here and I loved that part the most 🖤I might even look for more old computer booting up...Its sounds, tempo, code... its relaxing and amazing to watch
I really like how you show that same numbers can be a text, a sound, a bitmap, depending of its interpretation !
And an infinite number of other interpretations as well! These were just the first ones that came to my mind.
Yeah its kinda philosophical!
Before the Tandy 1000 appeared I was thinking, "Only Bisqwit could explain Windows, on Linux" ❤️
What did you think after the Tandy 1000 appeared?
@@Bisqwit as always, never comment before the end :)
This explains the processor operations and not windows though, right?
Linux wasn't even a glimmer in Linus daddy's groins when we grew up with old school TRS80 and older computers.
When you enable subtitles : [Tandy1000: bukiikiikiikii kiikiikii].. Gold!
Them: No! You can't use an old Tandy computer for anything useful today!
Me: Hehehe floppy drive go bururu kiikiikibukiibkiburru
A little disappointed the MZ EXE header/structure wasn't touched on, as that is what I thought the video was about, but it's still an entertaining look at what x86 code actually is.
Sorry about that. The 16-bit DOS EXE file is simple enough that it would be possible to cover in a time about the same as this video, but when we get into NE/PE, the background concepts become far less approachable and require a lot of explanation, limiting the options to 1) a very long video or 2) a video that most viewers don’t understand.
@@Bisqwit Maybe you could make a video discussing EXE metadata, or if that's not enough content, computer file metadata in general?
@@Bisqwit I think I can say most of us wouldn't mind a very long video :)
This was a really nice one btw.
Thanks @Dzouje! I really appreciate the sentiment. However, the problem is, longer videos take _much_ longer time to produce… Because it’s not just the labor, but also the time needed to think of a good script / representation. The relationship between video length and production time is almost exponential in most cases, at least of my genre.
I also expected this to be about executable format metadata like MZ and ELF. Still, it was interesting all the same!
4:30 "Good afternoon this is your captain speaking, welcome onboard Flight 4B7 with service from San Francisco to New York.
We are currently third in line for take-off and are expected to be in the air in approximately seven minutes time.
We ask that you please fasten your seat-belts at this time and secure all baggage underneath your seat or in the overhead compartments.
We also ask that ... "
Clever use of the Papers, Please sound effects when comparing the Tandy disassembly code to the datasheet, it fits quite well!
I knew the sound effects sounded familiar, but couldn't quite put my finger on from what game they were borrowed from; thanks for the answer! :D
Great simple explanation of a somewhat scary thing, I love it! Glad to see more videos from you!
8:41 - For those wondering, this bit of code is just parsing the command-line arguments. 😉
Indeed. You are very good! It is a program I wrote entirely in assembly. You can find its source code here: bisqwit.iki.fi/src/miscasm/becho.asm
I think including the sounds is GOOD for channel growth
This is the stuff I am subscribed for. Actually, this is the stuff why I use youtube at all. I am glad you exist and I love your content 👍
they are made out of cat fish
Hey ! Nice to see you back. Hope you're doing well...
That was highly informative for a laymen such as myself who only deals in environments. Looking at numbers makes my head spin.
ASCII looks daunting, the 8008 chip really does simplify what is today a maze of a processing instruction.
It's amazing anyone could make a program let alone multiplayer servers etc with tons of stuff happening both inside and outside.
Just all looks so daunting...
I've been working on a small game and that section with the old PC perfectly fit the vibe I wanted to go for in it, and ever since I've been down a rabbit hole of foley work, digging up videos of the music of FDDs. thank you greatly.
Glad to see more content from you, Bisqwit. I've been a fan of your content for years, but never commented on your videos until now. Your content helped to rekindle my love of programming and I've recently had time to dive back into it. I look forward to more videos and I wish you the best. //Draco
Oooooooh today I learned something new! Thanks Bisqwit. Also the editing was really nice. God bless you!
Thank you very much!
I may not fully understand everything you said, but it has blown my mind and made me appreciate the advancement of modern computers so much more. Thank you for making this!
We missed you! Welcome back! 😉
Speak for yourself!!!
Your editing skills have improved. That really helps with greenhorn programmers just learning about low-level stuff. Keep it up!
I fear an Unicode video would need to be 1 hour long :D
I think the other parts of the .exe file could be just as interesting, think self extracting archives, etc.
God! This was the first time in my life that I was able to understand a piece of assembly code. Thanks!
My brain increases just a bit for every Bisqwit video I watch. Thanks!
I'm so sorry but I legit thought at 8:53 you said "Inside the minecraft processor" and I got a bit excited :D Awesome explanation as always. Love the little soundeffects and clipart. It really adds to the whole!
The sea of tabs though 0:47
I first opened a binary around age eight-to-twelve, via a plaintext editor, and was presented with gobbledygook just as you describe. I recall querying Google around 2002 with queries like "how to make an EXE" and "how to make a DLL." Ahahaha.
He is back! We miss you.
I was just thinking of him two days ago. Not the biggest coincidence, but a coincidence nonetheless. 🤷
I remember opening an exe file with notepad as an 8yo trying to circumvent the WPA process for WMP10 back then 😆 hoping that I would find the if statement in there to comment out lol 😂
Well you had the principle right, just not the right language and the right tools.
@@EvilSapphireR later on, I realized that I need more tools and knowledge to do that.. fast forward, I am now a software engineer 😄
Jumping in here, just to let you know that I love this content, and that the work you put into Simon's Quest is absolutely transformative in making a definitive edition of the game. Thank you sir!
Thank you! I still wish one day I could finish the two big parts I am working on:
- Changing the town doors visually to close for night & open for day
- Music for the prologue
It is so nice to see Bisqwit videos :) As many people here are happy to see you again.
this topic is really interesting.
it is interesting how many emulators tries to mimic not only the instructions but the number of cycles for each.
some effects on some emulators were just possible due to some hacky abuse of how the processor worked.
that's fascinating.
imho, it is even more incredible because with very few resources, no internet (not as complete as today) people accomplished amazing deeds.
one of my favorites is Rollercoaster simulator.
there is something magical about going from some wiring specifications at processor level to a full experience we carried for life.
The typing game you made, You changed the Dos stub text on it :) I love finding secrets in files.
Thanks! But there is no DOS stub in it, because the game _is_ a DOS game.
I already knew all the stuff in this video, but I watched it anyways for the calming computer noises.
I love that Tandy ASMR. Thank you for leaving that in!
Bisqwit, inbetween this video and the last, I built an 8-bit CPU across many breadboards and wrote my own little assembly language for it. The whole process was such an eye-opener. This video was really terrific in solidifying what I learned, thank you :)
Congratulations on your endeavor! It sounds like a lot of fun.
Thats crazy
ben eater?
wow very impressive! What instructions did you follow to make it?
Nice to have you back! Was missing your content. Hope you've stayed safe during the pandemic.
Super interesting, super insightful! Thanks Bisqwit!
I love your videos! You have a very good and original way of teaching this passion. You are a very good teacher! Thank you very much for making such great videos!
My favourite TH-camr is back at it again and I couldn't be happier!
4:34
Computer: old af
Also computer: *transcendence noises*
10:47 I see you are like living in the edge without parking your HD before turning it off.
There is no parking utility on this computer that I am aware of.
Great video again Bisqwit, you really are excellent at explaining these topics in understandable terms.
That was amazing, great editing and very informative. I also love that you do videos where you reverse engineer old games. I heard Pokemon was first written in the Gameboy specific ASM lang and that is just insane. Hope to see more from you. BTW I liked hearing the sound of that old computer. Gained a subscriber today!
Most games on the 8-bit consoles, including Gameboy, were completely written in assembler. It is not even that hard, when you gain experience. Laborsome, sure. But then again, you can achieve certain things with surprisingly little amount of assembler code when your entire hardware is designed with that point in mind.
It was very nostalgic to hear the familiar sounds of a Tandy 1000 trying hard to follow simple instructions. I miss struggling to convince it to play games.
Love it. Would have also loved a longer version more in-depth.
Cool! I really liked the video. I knew about many of the different pieces but somehow you managed to put all the knowledge together in a way that made sense. Thanks!
You can hear the passion you have for computers. Great content.
Looking forward to some more videos, but in the meantime I'm rewatching some of your earlier uploads
4:30 that sound of the coolant system for the fusion core in a Tesla personal VTOL warming up circa 2040
How does someone even dislike a video like this? Call me a nerd but I actually find these types of videos entertaining.
i just found this channel, i love it, it's super relaxing
omg whoever deleted or repotlrted the scam comment thanks, a group of scammers are stalking me
I did. Nobody is stalking you specifically. This is what I spend 15 minutes doing 3~4 TIMES A DAY nowadays on my channel. th-cam.com/video/YPU4Galm9yk/w-d-xo.html Thanks TH-cam, for doing NOTHING.
2:05 When you see that Sierra is in the Hex editor and your heart gets warmed!
8:56 I understand that:
1) in 8086 there is a Table containing bit sequences at an irregular interval that represent commands
2) the 8086 loops through the Table to match each read byte from the .exe
3) Therefore the program runs according to the equivalent assembler's instructions
All thanks to the Table commands and the fact 8086 can match the next read .exe command to one of them.
Is that correct?
If that^^^ is true, does it mean Windows 11 won't work on 8086 Mostly because Windows 11 is an .exe that has non-compatible .exe commands?
Well the CPU uses something quite a bit more effective than _looping_ through the table. It’s electronics, so it can just try hundreds of transistors (of the table cells) _simultaneously_ instead of sequentially. Those transistors that fire will contribute towards the operation, and those that don’t, don’t.
EDIT: For clarity, I mean the process of identifying a single instruction. It doesn’t need to do a "is it this? No, how about this? Well, how about this?” process to identify the instruction; it can just test all alternatives simultaneously. But the instructions themselves are still processed sequentially, as the CPU usually needs to finish executing the previous instruction before it can do the next one. (Modern CPUs find ways to cheat this limitation, but it was still very true on the 8086.)
The reason why 32-bit or 64-bit EXEs don’t work on MS-DOS is that the file has a particular structured format, that the operating system has to parse, before it can let the CPU unravel the raw data within.
If the file does not have the expected format, the OS will reject it. This is why DOS programs won’t work on Windows, Windows programs won’t work on DOS, and neither will work on Linux and so on.
(It’s crazy because the OS is a computer program too. It is a program that analyzes and controls other programs.)
But let’s assume that all this was magically sorted out, and the OS accepted the file. Then, the CPU would try to run the code. The CPU would interpret the code according to its _own_ lookup tables. Those lookup tables are different for 16-bit code, 32-bit code and 64-bit codes. A processor that is running in 64-bit mode (such as a modern Intel or AMD processor) would interpret the same bytes very differently than a processor running in 16-bit mode (such as the 8086). It would do different things.
If the code was not designed for that processor, it would do things that do not make sense, such as access invalid memory addresses, perform nonsense calculations and so on, like a person who returns homework for the wrong week or wrong course. The answers don’t make any sense together with the questions.
Modern processors are designed to interrupt when invalid operations are performed, and to signal the operating system that something went wrong. On Windows or Linux, this would appear as the program crashing (while the operating system will continue operating normally). On MS-DOS, which does not have any protection measures, it would appear as the entire computer rebooting or becoming totally unresponsive, possible with some data corruption.
Really apreciated the emulator reference. I vaguely remember that video, but, back then, I thought it was just an emulator, I hadn't realised the "spin“ on it.
Also you made me realise I probably never tried to open a binary file with a text editor. Windows helped with that. But I have broken many a terminal (emulator) sessions since I started using Linux, by `cat`ing binaries.
Not all bytes of a mashine code are visible as an ASCII, but with a group of 8 bits, or as a hexadicimal value for example. The hexadicimal value of one byte is divided into 4 bits with the high nibble and 4 bits with the low nibble of a byte. And the bytes can combined together to build a 16 bit or 32 bit or 64 bit, ....value depending on the instructions of the mashine code. Some bytes are a part of the instructions and some other bytes can contain a table with simple values for a routine.
If you don't mind me asking, how did you create the nice effects (like the numbers flying at the screen) in this video? Did you use a commercial video editing package or write your own (or maybe even the flying numbers is a screensaver's output), etc.? For my videos (channel associated with this username), I pretty much only use effects/transitions/annotations filters I have written, but I am open to using other programs as well (for example, hopefully I will have a reason to use manim (from 3Blue1Brown) in the future (even though it means coding in Python)). (I have used LaTeX to generate nice looking equations. Manim also uses LaTeX) I don't need to know about the annotations - I already have that covered in my filter code.
BTW, I can totally empathize and agree to an answer you gave to one of the comments - it does take a long time to create these type of videos (script, editing, effects, etc.).
The numbers flying at the screen effect is actually from here: www.alldesigncreative.com/video-footage/green-effects/animated-numbers-background-green-screen-effect/
I first considered rendering it in POV-Ray, but finding an existing animation shortened my production time.
@@Bisqwit Thanks for responding (and for the link)! Coincidentally, in January I added green-screen capability to my video overlay filter, so that site's effects will really come in handy.
Thats A Beauty, Love It, It Is Great To Have You Back, Biqwit
Thank You, I Was Never Away Though. Why Are We Starting Every Word With A Capital Letter?
cause I Love Writting Like That
It causes headache.
One thing you might cover next @Bisqwit is how to actually make and use .so and .dll files. Many C++ programming books just lead the reader to compile simple examples pertaining to syntax and features of C++ but do not expand on what it means to compile a modular reusable library. They are taught that you can include code with your main.cpp file but do not elaborate on the use and creation of shared objects.
Shared libraries are an operating system level concept, not a language concept. At best, it’s compiler-specific. I might cover that topic at some time, but to be honest I’m not exactly familiar with the details either. Especially when we get to library versioning…
This exact question has been in my mind for the last 2 days or so. Can you read minds?
Whenever I have a question, I can now just pray that god bisqwit will upload his answers.
I SURE WILL SIR BECAUSE ENJOY THE STORAGE
I was actually watching one of his old videos when i receive a notification...welcome back.
Everything is high or low voltage. The numbers are also abstractions😋
Actually everything is just charges flowing through conductors. High or loe voltages are abstractions
A nice return from the one and only Bisqwit!
There's some forum threads scattered around the Internet from Mac users asking how to get out of "DOS mode", in the mistaken belief that this mode is preventing a program they want to run from functioning. Take a wild guess how they came to this conclusion.
Glad to have you back. I loves this video, the edition was on point.
amazing video! this is exactly the type of content that interests me, especially when it is presented by someone as incredible as you.
What is heavily complicated for me to understand is how these assembly programs in 80s and 90s could access video functions to draw things on the screen. I have a huge curiosity on how these binary and hex calculations become the first user interfaces and games inside companies like Atari, Nintendo, the Xerox interfaces and Apple with Macintosh's, etc . WIsh someone could make a video about that....
One of my future video plans has actually been to explain how graphics worked in CGA, EGA and VGA (and possibly MDA and Hercules, even though I have not much experience with those).
As for video game consoles, each of them is a separate case. There already exist videos in TH-cam that explain how graphics work on the 8-bit NES (Nintendo), if you want to look them up.
The "Retro Game Mechanics Explained" channel has an absolutely brilliant series of videos on how the SNES works and how it's programmed, if that's not too modern for you :)
@@omfgmouse God bless you
1:50 THE NUMBERS MASON! WHAT DO THEY MEAN?
1:24 it kind of reminds me of that dream cast sound when you play Track one
I am only guessing what you could possibly mean, but if I guess correctly (trying to play the program track as audio), definitely not a coincidence.
Wow, this was way more involved than I was expecting. I graduated with a Bachelors in Computer Engineering last year(poor timing) and I have actually developed an instruction set architecture from scratch and implemented it in Verilog code AND also ran that on an FPGA AND also ran a program written with that ISA. Very fun projects, I wish the classes weren't so crunched for time. Sadly, I still haven't found a job(actually I have, but I quit because it wasn't what I thought it was) so none of this knowledge has thus far proven practically useful. I'm not trying to discourage anyone from getting a Computer Engineering degree, maybe my bad luck is just due to the pandemic, but also, most companies right now just aren't interested in hiring people that even know what an ISA is. They need front end web developers, and mobile app developers. And those are the exact kind of jobs that I have gotten offers(and have stupidly turned down, but I'm stubborn I know) from.
Really excellent!! I happen to know this stuff, but still learned new things :D Thanks!
Thank you!
Awesome explanation! Thank you for making videos :)
My pleasure!
Hi Bisqwit, I have a curiosity: have you ever worked with screensavers?
I just finished constructing a working prototype, and among my first discoveries are that a screensaver is actually a Windows executable, and that its entry point is predefined inside a static library (ScrnSave.lib, specifically). Programming one from scratch has been proven to be challenging in unusual ways.
Many blessings to you too 🙏.
P.S.: A friend of mine recently built an 8086 computer on a breadboard; he's going to love this.
I have not worked on screensavers for Microsoft® Windows® other than that noticed that they are essentially EXE files.
Awesome content as always, just as the creator. Thanks a lot for creating interesting and insightful content! :)
Thank you for the kind words!
This is so fascinating. Can you make another video considering ARM Arch 64, like the one in the new Microsoft Surface or the Apple Macbook Pro? It'll be really interesting to compare the instructions given they have a different OS.
Fabulous video, really made it clear, thanks also for the video of your retro PC!
So happy to see you back
I love these kinds of videos from you. Simple and understood. That said at some point a hope you do cover NE/PE. Maybe in parts? So its not one long video. Either was thank you for this video.
If I figure out a way to make it in an entertaining format, I will.
You are back, woho
ah the unfortunate consequence of programming for your daily job is that after you get mentally tired home you cannot do anything more then lay down and sleep, watching you channel history (when you drove a bus you had a lot more time and energy to program at home) but you get more money now so thats a good thing even if the job is a bit more stresful
Glad to have you back
You too! :-)
4:30 Ah, the sound. It brings back memories.
This is the first time listening to someone with a thick accent, made the learn experience even more enjoyable.
I miss those sounds. I was just a child when computers used to do sounds like this, so I don't remember much.
"There is no text, only numbers" - Pythagoras
😄
I used to copy the content of the notepad to another one and saved the exe and try to run in and it didnt work, I also used to try "mix" programs by combining the contents shown in notepad and it also didnt work
The return of the king 🤴 so happy to watch a new video!
@5:30, haha what a gentle beast! My ears are still ringing from the CRT though. Please apply a low pass filter in future videos with a CRT, it’s so loud! :)
The amount of tabs in your browser is over 9000
I remember doing this myself in the "old" days when I still had 486DX2 and later Pentium machines, but since windows took over it kinda took a backseat. I miss the good old DOS days sometimes.
Every software made in the company I work for is made out of our tears, coffee and depression
Could you explain how archives work some day ? Especially the compression algorithm which is still a mystery to me ....
Good idea!
*There is no text, only numbers*
I think this might be the most important thing to understand when working with software in general, understanding that everything is just binary data representing other forms of data isn't explained often enough.
This may be why a lot of the developers I know personally have struggled with languages like C when first getting in to them, it's not so much a lack of skill or competency but merely a different perspective you have to adopt when working with them that feels very esoteric at first.
I'm really glad to see someone going over this though, understanding how binary data works is probably one of the cooler parts of programming.
Amazing video, everything is so clear and easy to understand, you are an outstanding teacher!
When I was a kid, the name I used to refer to these types of files that aren't meant to be interpreted as text was, quite ironically, "ASCII files", because I saw a whole bunch of extended ASCII characters and knew that's what those characters were called. I thought "ASCII" (read: raw machine code interpreted as text) was the language that all the pros coded in, and I thought it was (figuratively) magic. And I didn't realize that not all "ASCII" files were created equal-I remember one time I suddenly realized I could draw a smiley face in Paint, save it as a ".exe" file, and I'd have a valid "ASCII" file that would somehow run as a program. I was excited to see what the program would do, and naturally a bit disappointed when I got the inevitable "not a Win32 application" error. (Though my guess wasn't anything spectacular; I was picturing it just displaying the image in an otherwise-featureless window.
I have a Tandy 1000sx that is kinda dead, and that boot up sequence brought me back to my childhood (Yes my first computer was this when i was 4 and im a 90s kid)
This is fascinating on so many levels. Think about this: You have 8 light bulbs and 8 switches in another room. That plus a lookup table is practically a CPU. Well, since two humans on either end would be involved and the instructions would be executed on paper probably the slowest CPU of all time, but you could send instructions and the other human in the other room could execute them.
Now think about this a bit more. Think about your smartphone and all the incredible things it can do: render 3D graphics, connect wirelessly to the internet, snap high resolution images (and even edit them in real time) and many many more. Now.. think about, that all these things are fundamentally possible because of simple switches. If some decades ago you pointed at a electric switch and said: "Some day these switches will enable us to fly to the moon." you would be called crazy. Even today, if you told most people that their phones and computers work with billions of switches and not much else (okay, very simplified), they would'nt believe it. Now, for all the god-beleivers: Does your computer die, when you turn it off? Does it have a soul? Most would say 'no'. So why should a human have a soul? If you see what incredible complex things can be made from a lot of simple things, don't you begin to question that humans are nothing else? A machine run by billions and trillions of very small machines which run on relativly simple, chemical and physical rules.
Maybe it's just me, but the technical advancements of the last years have shown me one thing: We are not as special as we think and complexity is created from simplicity. Food for thought.
Sorry for my bad English, it's not my first language.
Could you do something similar, only with like multithreaded work? I'm interested in how the other cores "know" from the instructions that this should wait etc.
Hmm, I’m not sure I can make a good video around that idea, but I can maybe answer shortly in a comment.
There are two levels of mutual exclusion going on.
First, when multiple cores access the same data, the CPU itself ensures cache coherency. You can read Ulrich Drepper’s article on memory here: lwn.net/Articles/250967/ It touches the issue, particularly in part 2.
Second is operating system support. The operating system offers facilities for mutual exclusion. An application program requests for a lock, and the operating system puts the process on a waiting queue, from which it wakes the process once the awaited event has transpired.
Now I know where Autechre got his inspiration
it doesnt quite use a look up table to look up the instruction and then perform it. the binary value of the number, the 1´s and 0s actually turn on and off specific logic units within the processor and at precise timing intervals causing it to perform logical operations. for example the first 1 in the instruction 11010011 might go to a wire that turns on the ALU. the next 1 might go to a wire that latches a certain registers contents onto the wire. the CPU is performing these things dumb and blind. its just logic gates turned on and off with the cpu clock.
If you have studied the 6502 through Visual6502 or for example this diagram www.witwright.com/DonPub/6502-Block-Diagram.pdf , you will notice there is in fact a lookup table. The lookup table contains 130 × 21-bit wide patterns, that match the opcode and the clock cycle and some flags using a bitmask, and each either enable or disable certain paths in the CPU.
Modern CPUs are more complex because of superscalar execution and a microcode engine, and the exact techniques are not disclosed publicly. My explanation was a trivialization.
the narrator is so authentic for a documentary/ghost story
Kiitos tästäkin videosta!
8086:n valtakaudella assembler oli vielä ulkoa opittavissa. 386 toi hirveän määrän lisää rekistereitä ja mahdollisuuksia - alkoi mennä liian monimutkaiseksi mulle, joten vaihdoin C:hen. AMD64 on jo sellainen himmeli, että on turvallisinta jättää assembler-koodaus harvoille ja valituille erikoishenkilöille. :) Onneksi konetehot ovat nykyään sillä tasolla, että tavallisia ohjelmia ei tarvitse lähteä suunnittelemaan binäärikoodia vääntäen.
Kiitos! Joo, assembler-koodausta tarvitsee nykyään tosi vähän x86:lla. Lähinnä sellaiset tapaukset, missä SIMDistä on hyötyä, tarvitsevat usein asmia, koska kääntäjät ovat vieläkin aika heikkoja optimoimaan SIMDiä tehokkaasti, ja intrinsiceissäkin on toisinaan ongelmia.