It's really rare that I watch TH-cam videos this long. Seems these CP/M BIOS customisation videos are the only ones I've watched that exceed 1 hour. They're either really informative and entertaining, or I'm just a weirdo. I reckon both.
...or I am?... I feel bad when I gloss over or edit out the details. I may be overcompensating for something... When I was a kid I found myself frustrated by trying to create variations of things I found interesting only to fail as a result of what I later realized could have been avoided if there was just a bit more detail (equal blame on me for not reading the docs that were available and on the presenter for oversimplifying things.) Correct me if I'm wrong. But I like to think that with the detail left in, you can actually customize a design, make your own board, program it, and debug it. Otherwise it'd be yet another 10-minute "look at what I did!" video... that'd leave the curious with nothing that they really need to succeed on their own. I could make them seem shorter by breaking them up. But that'd take much more time to produce them. I keep thinking I'll get better at that. Apparently I'm a slow learner ;-) At least I've shortened them by disabling mid-roll ads! Side note: When I started this, I followed the advice of some of my students. Then I realized that their advice (shorter videos with my face on screen or in a box all the time) made the videos (intentionally or not) about ME rather than the technology or whatever. It's not about me. It's about what others are capable of doing when they are given what they need to succeed. If it isn't moving along fast enough, you can do what I do and watch at 2X normal speed. If you have any advice on how I could improve these things, I am eager to learn!
@@JohnsBasement Nah, they're fine the way they are. I was born the year my Cromemco Z-1 was introduced to market and I only started programming in assembly language in 1996 on the Motorola 6800 and Intel 8088 when I started my diploma in electronic engineering. My Z-1 is the pride of my collection and, although I have two flavours of CP/M for it already, I've wanted to create/mod my own BIOS for years. Datasheets and memory maps are what I cut my teeth on. Instructional videos on programming run the risk of being a bit too "dry". You're enjoying doing this and it really shows, plus the added humour keeps it engaging. The videos have to be long otherwise we won't get the full picture. Short attention spans belong in TikTok. Keep up the good work.
@@JohnsBasement your detailed video style without talking head are BEST; you are probably best teacher I ever met... skipping again quickly through the videos I already watched several times instead of TV nonsenses... before booting cp/m on the hw;
After the first sentence I was afraid this was heading somewhere else! 😂 I'm glad you liked them! If you take a swing at an old box, I hope you have fun!
56:00 From the docs shown early in the video, I got the impression that the disk track number and sector number is remembered individually for each disk. That way a caller can just set the disk ONLY and continue from where he left off on that disk. So, you can't have a single location to store it in: you need to multiply by the current disk number and offset from the label.
You have an interesting interpretation. I never considered. However, if you look at the alteration guide in the appendix, you'll see a sample CBIOS where where there are reserved areas to hold the track number sector number DMA address and disc number for one and only one setting. Therefore your proposal cannot be correct. In fact, if you implemented what your proposing you could have a problem. It's unlikely that CPM expects to be able to change discs while retaining things like the track number, One could not count on it for sure. In fact, I suspect that CPM might actually leave the DMA address set to something like 80 or the directory buffer address in the bios across multiple calls, possibly transferring data from more than one drive!
I have enjoyed your Retro Z80 series a lot. However, I have been unable to find a description of your Z80 tool chain for CPM. The only note I found was in your assembly intor doc: The assembler and simulator can be installed with these commands: sudo apt install z80asm sudo apt install sdcc sudo apt install sdcc-ucsim Is that all there is to your setup for building the projects you have developed? I believe sdcc is probably not required. Also have you considered using z88dk as a toolchain? Thanks for all your interesting content. David C
You are correct. z80asm is all I've used so far. I chose it because it's trivial to use and does not do anything but output the instructions in the source file. sdcc is something I've tinkered with. I originally considered using its simulator. I will have to look at z88dk.
Many SD Cards don't support wear levelling so when using such cards CP/M with a 1:1 CHS to LBA mapping you are at risk of prematurely wearing out a large capacity SD Card particularly if you're compiling programs where lots of temporary files are created and deleted. To counter this, consider implementing a wear levelling scheme that uses all of spare SD Card space. Back in the 1980's when damned expensive EEPROM was introduced we had to implement a wear levelling algorithms to preserve its life.
Yeah. I have only started thinking about this potential issue. In the near term I have ignored it simply because during development, we can afford to scrap a few $3 SD cards. I'm not an expert in the state of the art here. What does this have to do with "1:1 CHS to LBA?" I must be missing something.
LBA(Logical Block Addressing) is just a replacement technique of using a integer addressed blocks in place of the older Cylinder, Head Sector mapping scheme. Thus when using it in a 1:1 relationship on non-auto wear levelling SD Card's Logical Blocks, only those specific areas area of flash ROM are written. An example mapping is LBA = (C × HPC + H) × SPT + (S − 1) meaning that only a very limited number of blocks of a high capacity drive is ever used. These days, to address this issue, some manufacturers have released auto wear levelling SD cards which have their own algorithms and smarts to evenly distribute their flash ROM usage; Western Digital Purple SD Cards (marketed as "Ultra Endurance") apparently are auto self wear levelling and are affordable.
@@PebblesChan When using a card that lacks an automatic wear leveling feature, we would have to change where blocks are located when they are getting rewritten more often than other blocks on the card. I would call that a re-mapping. I don't see how this has anything to do with the translation of CHS addresses to LBA addresses unless you change the math that does that translating... which is no different than simply translating the LBA addresses into replacement LBA addreses. Thus making me question why we'd care about CHS here.
Changing the math is exactly right by adding another factor to increase the number of replacement LBA addresses used. One might only want to assign multiple blocks for directory entries because these are the most often changed and are normally assigned to specific range of track/cylinders.
OK. I think we are in sync so far. So, the problem then comes down to how the filesystem and/or driver keeps track of the current mapping that the OS should be using at any moment. Reserve blocks to hold the current mappings? Of course that too may have to be rrmapped from time to time as well. How do we keep track of the erase counts? That too would have to be stored. Like I said, I have yet to give this much thought Maybe we only store 3 sectors per block & store an erase counter in the last 128 byte region along with a pointer to where the block has been moved if the count is too high?? That could chain... and become inefficient, but it would work. It may become necessary to peridocally rearrange the whole drive to migrate the stable data into the areas that have gotten more wear in irder to let them settle down. Of course no HC/XC card manages itself in 512 byte blocks internally. We'd accomplish nothing unless we are really counting the number of erase cycles on the internal (larger) physical blocks. What is the generally accepted process of doing this? Is there one?
Nice progression. Are you going to put CP/M sectors into SDCard sectors 1:1 (with the remainder wasted) or bundle 4 CP/M sectors into each SDCard sector? I was thinking, as I was watching this, that it would be interesting to emulate the CP/M disks and sectors as actual files on the SDCard... then they actually can be read and modified by Linux or Windows. Each Disk is a root folder and then each sector is a file in a root folder. It's a new virtualisation or abstraction layer underneath the CP/M BIOS that would have to be written though and the actual partition would then be reformatted as eg. FAT32. A lot of work to implement!... As it is I guess you will be backing up the SDCard just be doing raw reads and writes with 'dd'. It's unfortunate that they put the disks block,sector encoding directly into the BIOS (or DOS controller routines?) Makes it a bit harder to jam some other type of media technology in there... A carry over from the older Disk and tape drive systems... Well at least you don't have to implement this on a reel-to-reel tape drive :) At the end, you edited the asm code and added "call .bios_setdma". Just wondering, is that a legal call? I thought the dot at the start of a label set it... have not assembled Z80 is a looong time, I'm probably wrong. Please excuse if so. Cheers,
You are right. The leading dot is a feature of the z80asm assembler I decided to use. Definitely not an original standard Z80 thing! The dot makes it a file-local symbol. I'm using a few modern-ish features like the 0x prefix and .-local labels. I wanted to make getting this thing working as easy as possible for anyone at any level. So I decided to try to use an assembler without a link-editor. I can't bring myself to accept every label as global. :-/ If It was just me, I'd probably write the lot in C. But I'm having fun re-learning Z80 again. Maybe I'll write a fancy boot monitor for it in C later on. Or maybe even reinvent a version of BASIC for the boot FLASH with an option to also run it under CP/M? SPOILER ALERT... I already implemented an all FLASH-bootable version of CP/M that is pretty nice. But it is not practical until I get the bank-selected RAM working with CP/M... which I have yet to complete. Doing so has a fun race-condition if any IRQ handler tries to touch the memory-bank-select register.. or any RAM under 0x8000 for that matter. I'm not sure I want to mess with FAT. I just recorded a piece that shows how to use some command-line tools for copying files in/out of the CP/M filesystem on the SD card. So that works without resorting to FAT. If anyone wanted to take a swing at FAT, it'd likely be better if it were done after we get the bank-selected RAM going. It might need some elbow-room. As for the sector-mapping, I went with 4 sectors per 512-byte SD block. Each block is a CP/M track. It comes out pretty simple. AND the performance gained by not wasting 75% of the SD blocks is significant... as the bit-banged SPI is about the speed of a single-D 5 1/4" drive!
@@JohnsBasement [Bit Banged SPI] Ouch. I did not get to that video yet, on my watch list.. am bouncing around a bit... hey, maybe a future FPGA project, custom peripherals for the CP/M system (SPI, I2C, faster UART ...) ? [4 sectors per block] - Makes sense, neat. I like it. Cheers,
"I'm running at 9600 baud for that realistic feel." Love it.
🤣
It's really rare that I watch TH-cam videos this long. Seems these CP/M BIOS customisation videos are the only ones I've watched that exceed 1 hour. They're either really informative and entertaining, or I'm just a weirdo. I reckon both.
...or I am?... I feel bad when I gloss over or edit out the details. I may be overcompensating for something... When I was a kid I found myself frustrated by trying to create variations of things I found interesting only to fail as a result of what I later realized could have been avoided if there was just a bit more detail (equal blame on me for not reading the docs that were available and on the presenter for oversimplifying things.)
Correct me if I'm wrong. But I like to think that with the detail left in, you can actually customize a design, make your own board, program it, and debug it. Otherwise it'd be yet another 10-minute "look at what I did!" video... that'd leave the curious with nothing that they really need to succeed on their own.
I could make them seem shorter by breaking them up. But that'd take much more time to produce them. I keep thinking I'll get better at that. Apparently I'm a slow learner ;-) At least I've shortened them by disabling mid-roll ads!
Side note: When I started this, I followed the advice of some of my students. Then I realized that their advice (shorter videos with my face on screen or in a box all the time) made the videos (intentionally or not) about ME rather than the technology or whatever.
It's not about me. It's about what others are capable of doing when they are given what they need to succeed.
If it isn't moving along fast enough, you can do what I do and watch at 2X normal speed.
If you have any advice on how I could improve these things, I am eager to learn!
@@JohnsBasement Nah, they're fine the way they are. I was born the year my Cromemco Z-1 was introduced to market and I only started programming in assembly language in 1996 on the Motorola 6800 and Intel 8088 when I started my diploma in electronic engineering. My Z-1 is the pride of my collection and, although I have two flavours of CP/M for it already, I've wanted to create/mod my own BIOS for years. Datasheets and memory maps are what I cut my teeth on. Instructional videos on programming run the risk of being a bit too "dry". You're enjoying doing this and it really shows, plus the added humour keeps it engaging. The videos have to be long otherwise we won't get the full picture. Short attention spans belong in TikTok. Keep up the good work.
@@snarfusmaximus Thank you for such kind words. Hopefully there are some tid-bits in this series that will help you out!
@@JohnsBasement your detailed video style without talking head are BEST; you are probably best teacher I ever met... skipping again quickly through the videos I already watched several times instead of TV nonsenses... before booting cp/m on the hw;
@@7alken I'm glad there are so many finding value in this series! It is a ton of fun getting to re-live all of this!
Nice series of videos. I binge watched them while recovering from Covid. Makes me want to play around with CP/M on my TRS Model 4.
After the first sentence I was afraid this was heading somewhere else! 😂
I'm glad you liked them! If you take a swing at an old box, I hope you have fun!
56:00 From the docs shown early in the video, I got the impression that the disk track number and sector number is remembered individually for each disk. That way a caller can just set the disk ONLY and continue from where he left off on that disk. So, you can't have a single location to store it in: you need to multiply by the current disk number and offset from the label.
You have an interesting interpretation. I never considered. However, if you look at the alteration guide in the appendix, you'll see a sample CBIOS where where there are reserved areas to hold the track number sector number DMA address and disc number for one and only one setting. Therefore your proposal cannot be correct. In fact, if you implemented what your proposing you could have a problem. It's unlikely that CPM expects to be able to change discs while retaining things like the track number, One could not count on it for sure. In fact, I suspect that CPM might actually leave the DMA address set to something like 80 or the directory buffer address in the bios across multiple calls, possibly transferring data from more than one drive!
Hi John. This is great ! Regards, RJM
Thanks!
Much obliged.
I have enjoyed your Retro Z80 series a lot. However, I have been unable to find a description of your Z80 tool chain for CPM. The only note I found was in your assembly intor doc:
The assembler and simulator can be installed with these commands:
sudo apt install z80asm
sudo apt install sdcc
sudo apt install sdcc-ucsim
Is that all there is to your setup for building the projects you have developed? I believe sdcc is probably not required.
Also have you considered using z88dk as a toolchain?
Thanks for all your interesting content. David C
You are correct. z80asm is all I've used so far. I chose it because it's trivial to use and does not do anything but output the instructions in the source file.
sdcc is something I've tinkered with. I originally considered using its simulator.
I will have to look at z88dk.
Many SD Cards don't support wear levelling so when using such cards CP/M with a 1:1 CHS to LBA mapping you are at risk of prematurely wearing out a large capacity SD Card particularly if you're compiling programs where lots of temporary files are created and deleted. To counter this, consider implementing a wear levelling scheme that uses all of spare SD Card space. Back in the 1980's when damned expensive EEPROM was introduced we had to implement a wear levelling algorithms to preserve its life.
Yeah. I have only started thinking about this potential issue. In the near term I have ignored it simply because during development, we can afford to scrap a few $3 SD cards.
I'm not an expert in the state of the art here.
What does this have to do with "1:1 CHS to LBA?" I must be missing something.
LBA(Logical Block Addressing) is just a replacement technique of using a integer addressed blocks in place of the older Cylinder, Head Sector mapping scheme. Thus when using it in a 1:1 relationship on non-auto wear levelling SD Card's Logical Blocks, only those specific areas area of flash ROM are written. An example mapping is LBA = (C × HPC + H) × SPT + (S − 1) meaning that only a very limited number of blocks of a high capacity drive is ever used. These days, to address this issue, some manufacturers have released auto wear levelling SD cards which have their own algorithms and smarts to evenly distribute their flash ROM usage; Western Digital Purple SD Cards (marketed as "Ultra Endurance") apparently are auto self wear levelling and are affordable.
@@PebblesChan When using a card that lacks an automatic wear leveling feature, we would have to change where blocks are located when they are getting rewritten more often than other blocks on the card. I would call that a re-mapping.
I don't see how this has anything to do with the translation of CHS addresses to LBA addresses unless you change the math that does that translating... which is no different than simply translating the LBA addresses into replacement LBA addreses. Thus making me question why we'd care about CHS here.
Changing the math is exactly right by adding another factor to increase the number of replacement LBA addresses used. One might only want to assign multiple blocks for directory entries because these are the most often changed and are normally assigned to specific range of track/cylinders.
OK. I think we are in sync so far.
So, the problem then comes down to how the filesystem and/or driver keeps track of the current mapping that the OS should be using at any moment.
Reserve blocks to hold the current mappings? Of course that too may have to be rrmapped from time to time as well.
How do we keep track of the erase counts? That too would have to be stored.
Like I said, I have yet to give this much thought Maybe we only store 3 sectors per block & store an erase counter in the last 128 byte region along with a pointer to where the block has been moved if the count is too high?? That could chain... and become inefficient, but it would work.
It may become necessary to peridocally rearrange the whole drive to migrate the stable data into the areas that have gotten more wear in irder to let them settle down.
Of course no HC/XC card manages itself in 512 byte blocks internally. We'd accomplish nothing unless we are really counting the number of erase cycles on the internal (larger) physical blocks.
What is the generally accepted process of doing this? Is there one?
Nice progression.
Are you going to put CP/M sectors into SDCard sectors 1:1 (with the remainder wasted) or bundle 4 CP/M sectors into each SDCard sector?
I was thinking, as I was watching this, that it would be interesting to emulate the CP/M disks and sectors as actual files on the SDCard... then they actually can be read and modified by Linux or Windows. Each Disk is a root folder and then each sector is a file in a root folder. It's a new virtualisation or abstraction layer underneath the CP/M BIOS that would have to be written though and the actual partition would then be reformatted as eg. FAT32. A lot of work to implement!... As it is I guess you will be backing up the SDCard just be doing raw reads and writes with 'dd'.
It's unfortunate that they put the disks block,sector encoding directly into the BIOS (or DOS controller routines?) Makes it a bit harder to jam some other type of media technology in there... A carry over from the older Disk and tape drive systems... Well at least you don't have to implement this on a reel-to-reel tape drive :)
At the end, you edited the asm code and added "call .bios_setdma". Just wondering, is that a legal call? I thought the dot at the start of a label set it... have not assembled Z80 is a looong time, I'm probably wrong. Please excuse if so.
Cheers,
You are right. The leading dot is a feature of the z80asm assembler I decided to use. Definitely not an original standard Z80 thing! The dot makes it a file-local symbol. I'm using a few modern-ish features like the 0x prefix and .-local labels.
I wanted to make getting this thing working as easy as possible for anyone at any level. So I decided to try to use an assembler without a link-editor. I can't bring myself to accept every label as global. :-/
If It was just me, I'd probably write the lot in C. But I'm having fun re-learning Z80 again.
Maybe I'll write a fancy boot monitor for it in C later on. Or maybe even reinvent a version of BASIC for the boot FLASH with an option to also run it under CP/M?
SPOILER ALERT...
I already implemented an all FLASH-bootable version of CP/M that is pretty nice. But it is not practical until I get the bank-selected RAM working with CP/M... which I have yet to complete. Doing so has a fun race-condition if any IRQ handler tries to touch the memory-bank-select register.. or any RAM under 0x8000 for that matter.
I'm not sure I want to mess with FAT. I just recorded a piece that shows how to use some command-line tools for copying files in/out of the CP/M filesystem on the SD card. So that works without resorting to FAT. If anyone wanted to take a swing at FAT, it'd likely be better if it were done after we get the bank-selected RAM going. It might need some elbow-room.
As for the sector-mapping, I went with 4 sectors per 512-byte SD block. Each block is a CP/M track. It comes out pretty simple. AND the performance gained by not wasting 75% of the SD blocks is significant... as the bit-banged SPI is about the speed of a single-D 5 1/4" drive!
@@JohnsBasement [Bit Banged SPI] Ouch. I did not get to that video yet, on my watch list.. am bouncing around a bit... hey, maybe a future FPGA project, custom peripherals for the CP/M system (SPI, I2C, faster UART ...) ?
[4 sectors per block] - Makes sense, neat. I like it.
Cheers,