a trick for restoring difficult tapes: in Audacity apply a very high pass filter, let's say 16000 Hz. This cancels out all volume variations and keeps only the zero crossings, which is what matters for tape decoding. With this trick I was able to restore a rare tape for the Tarbell interface.
Uhhh can you or someone explain me how this works? Because for one there's no 16KHz signal on tape, the frequency response of it fades out before that, so by all logic if you highpass at 16k all you get is noise from the tape machine; and for other the signal that creates the zero crossings is the 1- or 2-KHz squarewave, so when you highpass it, wouldn't you create new zero crossings all over the place at multiples of the original zero crossing rate? Assuming this works as described and there wasn't a accidental error in the description, what am i actually missing?
@@SianaGearz you are missing that the high pass filter hasn't a flat response, the attenuation below 16KHz is gradual, so you still get some of the 8kHz, less of the 4Khz, and so on (with increasing attenuation as you get closer to 0 Hz). So you still retain a good part of the original signal but in a cleaner form where zero crossings have been enhanced (this because the "step" response of the square wave has a very high frequency content).
@@ninoporcino5790 OK wow yeah that's smart. let's say you create a 6db/oct slope; what filter slope do you use? It deletes low end rumble and many sources of offset, we can outright kill anything below the band and it will just centre the crossings better, and it creates an eq which for a 1KHz wave, strengthens its let's say 3k 5k 7k components increasingly which helps sharpen the box edge. I suppose there's a number of increasingly more complex recipes to EQ the tape to beautify the pulse shape but i get it now.
Hi Noel, 2 things to note about cassette : many of them have the foam buffer cooked or dead, making the tape band reading shitty, and errors here and there. Replace the foam buffer by a new one, and oh ! Miracle ! the tape read again without any errors ! Next is the copper arm on which the foam buffer is glued. This one tend to relax too much with old age. Just put some demak up cotton piece under it, and done :D
Very interesting! It sounds like there are even more techniques BEFORE reading the data like that. Super interesting. If I come across some tape that has problems being read, I think I'll try that. Thanks!
@@NoelsRetroLab Yes. When i got sent 150 tapes to dump as WAV and then pass in CDT for the Amstrad CPC from a big collector, i had to find solutions to get the datas read correctly. And the unrewinded band is a classic. with the band properly sticked to the read head, that's it, the squeeze is much more light or even better the signal is again proper, so when processed, it's all good !
Thank you for the valuable tips! I have some old tapes from my Amstrad CPC from my childhood with some self written programs and games I want to record and that could be really helpful advice.
The fact that we can actually "see" the BASIC listing inside a digitized version of an analog tape from 40+ years ago is fascinating.Great video! And yes, it looks that you need some trial & error until you get most of the listing loaded; doesnt matter if the errors are typos, those you can easily correct post-load; the problem are those nasty frequency incorrections that make the load stop amid file.
azimuth screw adjustment may help as the tape was recorded on a different player with likely different alignment. It wasn't uncommon to have to mess with this to get round R Tape load errors on the spectrum with original games.
Normally, you'd get a 3khz reference tone and turn the screw as you listen (and have a spectrum analyzer running) so you can get it just right. Since the tapes used lower frequencies than that, you can be sure that the 3khz reference adjustment will be perfect for the tapes.
It might be worth a try to replicate the circuit from the TI-99 on a breadboard and run the audio into that. **AS I UNDERSTAND IT** the circuit (MAG IN) uses a 4558 dual op-amp (U400) to pre-filter the signal. The first side of the op-amp is configured as a low-pass filter for 18kHz, which is obviously just to kill any non-audio noise. The second op-amp is configured as a low-pass filter at 1.064Khz, with a high gain, which probably would drive the output as a digital switch. Since it is a low-pass filter, it should be producing a positive output on the '0' signal. Any negative output would be snubbed by the diode CR402. This would produce a digitally-inverted output into programmable system interface U300, pin 30, which is an inverting-input interrupt (INT11). That is just from a brief look at the circuit. It would be informative (and good content) to replicate the circuit on a breadboard and look at the output on an oscilloscope. You could even pipe the output from the breadboard to U300-30 with a jumper to see if the circuit works. Anyhow, I really like this kind of content and I'm glad there are folks like you out there helping the community. Cheers!!
That's actually a great idea (and much better insight at what it's actually doing than my cursory glance at the schematics). I may have to try that at some point. Thanks!
That's an excellent idea. It would also be worth scoping out the signal on the existing machine at various points in the circuit to see how it's responding to various input waveforms, volume levels and so on.
...a signal with 2 different frequencies is FSK, frequency shift keying... FM..is modulated across the bandwidth range, eg centre freq is 100Mhz.. FM can slide up and down any part of its bandwidth (in FM radio this is 200khz so it can slide a max of 100khz either side of the centre frequency) in less than hz resolution and is very analog in nature.....FSK is 2 destinct frequencies only... one to represent the "0" one to represent the "1"....completely digital...usually seperated by 800hz (again in radio...which ALL this tech is based off...and even the tech we use today in our modems, the signal is just sent over wires instead of the air) there are some great radio hacking tools out there that would take this signal and generate the actual data... and you can reconstruct the bad data (eg universal radio hacker) ...when i got into radio i really liked to capture and decode Marine Weather Fax... as it reminded me a LOT of loading tapes in the CPC... its 2 distinct frequencies (FSK) and it transferred the image line by line, but very slowly (about 1 line per second) as it was being sent over the air and has to deal with things like ionospheric conditions
Good catch. The one difference from radio, then is that the frequencies in CMT FSK systems are usually separated by one octave, such as the 689.5/1379 Hz pair here, or the more common 1200/2400 Hz pair.
WEFAX doesn't have to just be two frequencies, although it often is due to the black-and-white nature of the charts. But it's fully grayscale capable and you'll see that when they send weather satellite images.
A perfect balance of explanation and demonstration, keeping all the unknowns in play, and humbly making space for new perspectives. Superior tech vid. Bravo Noel.
One thing I didn't see - tape damage might be due to be stretched, which would alter the frequencies. It might be worth lookigng at the damaged portions to see if maybe you need to somehow speed that section up. It would also be interesting to feed the raw signal into a spectrum analyzer. You should have just the two tones. Look for variations where they move.
Good point. I think slight stretching (i.e. frequency shift) is still fine since these computers were not precise at all. Sometimes you can play something at 25% faster speed and they will still load. I think at that point the signal would even look different at a glance. But I love the idea of running it through a spectrum analyzer! I should try that next time.
@@NoelsRetroLab It may depend on the algorithm. It may adjust to a consistent speed difference, but choke on a sudden change. Especially if it shifts one of the frequencies close to the other.
The algorithms used by the machine firmware of the era are resistant to tape stretch. They only need to differentiate between approximately 0.5ms and 1ms zero crossing intervals so there's a lot of room. Adaptive algorithms which try to estimate time stretch were not at all used, so consistency of speed isn't of any consequence, there is simply a timing threshold below which a zero crossing is considered to be a 0.5ms one and above which a 1ms one; however the threshold tends to not be perfectly consistent depending on previous state of the decoder. As resistant as the resulting algorithm is to stretch, nonetheless bias, phase shift and stretch together could conceivably turn the 0.5ms interval into something that looks just above the chosen 1ms interpretation threshold, so in error sections, manual inspection is a good idea. I don't think you'll see any exciting frequency shift per se but it would be curious to double check.
Accoustic modems used a multitude of techniques such as FSK (Frequency Shift Keying), PSK (Phase Shift Keying), QAM (Quadrature Amplitude Modulation) and even QPSK. The bedoing! bedoing! sounds heard are a part of the communications speed negotiation phase.
I am genuinely surprised no one has made better software by now. The hardest part would be to actually translate the audio into actual binary data, but once you have that you could easily make the troubleshooting 1000x easier by showing the code and highlight the record(s) that was not read correctly. Maybe show both copies of the same record in case one is less corrupt. Could even throw in some basic (pun intended) spellchecking to highlight things like "orink" instead of "print" and so on. The retro community is huge, you'd think this would be one of the first thing that was made to ensure digitizing dying physical media is as successful as possible.
Right? And not just for the TI99/4A. That seems like a program it would be super useful for every retro platform with significant cassette library (although to be useful they'll have to be very customized for the platform).
Well, the physical media doesn't need to be decoded right away; saving a decent quality WAV file (22 kHz seems to be the minimum, but 41 kHz works better, 8 bit resolution is fine though) preserves the audio and then you can decode it at leasure any time later. One thing that would be really helpful from the development community would be to open source their software, or at least make it more of a Unix-style modular set of tools than one big GUI program. For a program like CS1er (which looks to me to be pronounced like "scissor") anybody else working on the same thing or something similar has to start again, especially after the author walks away at some point and is no longer available even to ask for the source.
What you showed in your video is band signal squeezed, due to unrewinded tape. The way to read the tape correctly without the squeeze or less squeeze and then able to recover the tape content : use my method bellow. I have countless of faulty tapes with that. (i did more than 2000 original tapes dumped for Amstrad CPC, but the method applies for any other machine).
Thanks for another wonderful video, it really was a blast from the past hearing the sound of loading from tape on the TI 99-4A and seeing the BASIC code. I did a lot of playing around in my teens, writing programs for my TI back in the early 80s, and am very familiar with the sounds of the tape loading/saving and the frustration of having a program disappear after a load failure.
To this day I've never used a tape short of creating daily backups that no one including myself knew how to recover. LMAO. I have the tape adapter for the U2+ on my C64.....but I haven't even used that either. Everything I've found is a D64 or a T64 (which doesn't go through the tape interface)
I've tried to recover two excellent games I made for the TI. Didn't know about CS1ER. I can say they were EXCELLENT games (comparing with others BASIC Games). I'll try again with that program.
Some software houses used a copy protection, one small part of the tape was recorded on purpose with a very low volume. When the tape was read by the computer, that particular part gave an error, and then the computer could not read back that part. The tape with the error was original, then the computer was ordered to continue. But if the tape was copied, most users have an automatic gain in their cassette recorder, then all parts of the tape could be read back, then the tape must have been a copy and the programme refused to play. When I saw the audio file I recognised this very ingenious copy protection, neither a newly saved cassette nor a copy would work, both had perfect audio at the correct levels.
Pulling this out from the deep dark mists of memory, but the TI is particularly volume sensitive as well. I’m not surprised you weren’t able to easily recover and load just using audacity. I seem to recall having only one small volume range where it would read my tapes, but that may have been partially because of the player I was using.
There is 'castool' which comes with MAME. I don't think it's very full-featured but from memory it supports a few systems in a more or less uniform way.
I'm still sad the guy who made the SVI-CAS seems to have made a total of about 7 of them then disappeared off the face of the earth. Been looking for one for.about a year!
Ooff. When I try to google programs like these, nothing comes up and I have to reinvent all the wheels. I recently recovered some undumped DECO Cassette System games, but I literally had to write my own data extraction program. The only information I had was in the MAME driver, telling me that every block starts with 00 FF, has 256 bytes of data, then two bytes checksum (which is encrypted just like the data) and then FF 00. There is silence in between blocks. I also did multiple dumps (since it uses proprietary tapes, the original player had to do) using different approaches like hold the tape in a certain way, play it at different speeds. On some tapes, I couldn't get a single clean dump, but with all the debug information my program output and having four dumps, I could always do a majority vote and the game programs are now in MAME. And yes - looking at the zoomed in waveform, manually decoding was how I started.
Very cool stuff! I love it that it's possible to make sense out of the audio file data like that. Too bad you had to write everything from scratch, but it was probably worth it. I wonder if it's possible to write a "universal" program with different plugins or rules for different platforms, or whether they're too different from each other.
All the tape formats I played with used more than two frequencies, the others are for things like the synchronization tone. And in fact the frequencies are due to how much time passes between sound pulses, the zero crossings. But if the CPU needs to do more instructions for any reason you'll end up with some durations that are not quite the one for the zero or the one for the one. So you often have to tweak them. Also depending on whether you count the wave crossing zero on the way up from negative to positive or down from positive to negative you can get slight differences, like half a wave. I believe some systems actually count the zero crossings in both directions too. It can be challenging and fun and sometimes frustrating. Anyway great to see somebody do such a good video on the topic!
yup there are usually "pilot" signals at the start (and sometimes the end) that synchronize the decoder with the incoming data..but the data itself is usually just in FSK
That's interesting, because a friend and I have been working on some Python code that reads and generates tape saves, and we've never seen more than two frequencies so far. (We've got National JR-200 and Hitachi Basic Master Lv2 working, NEC PC-8001 mostly working, and have looked at FM-7). The leader and separation tones have invariably been at one of the two frequencies used for FSK.
@@Curt_Sampson I haven't played with them for over a year and a half but at that time I played with Sinclair ZX Spectrum, TRS-80 model 1, and Apple 2. I didn't manage to get all three working from the same code. Speccy for sure has a long leader tone/synch tone be the 1s and 0s. I don't remember the others as well.
Is there anything like this avail be for Amstrad CPC? There was a game I made in basic back when I was 14 and I saved it on to tape. Many years later on, I found that tape again but it has un-fortuneateley has deteriorated to the point where it just can't be loaded anymore.
For Atari 8-bit computers there is a program called Turgen, which supports various standard and custom tape loading formats that were used on the platform.
The data is not out of the band surface or demagnetized, it's still there, it's just that due to the unrewind, the foam buffer too old and the copper part too relaxed, the head tape drive can't read it correctly.
Thanks for sharing, it was interesting to know how the data is stored. I think, if all blocks on the cassette are repeating twice and have a checksum at the end, then it may be possible to reconstruct a good block from merging corresponding bits together. If you have a sound file with a good quality, I can try later in Python :)
It's disappointing, but understandable, that those writing the tape loader routines in machines in the early eighties and late seventies didn't seem to know about Hamming codes and other error correction mechanisms and instead resorted to just clumsily duplicating the data.
Right! They could have done so much more. At the same time, this is a real-time process and maybe they couldn't spare the cycles or the RAM to do any fancy computations.
Even if they had just interleaved the blocks, so the two copies of any given block were farther away from each other on the tape, it would have helped a lot. Many over-the-air digital radio protocols do this because errors tend to come in bursts.
i have had to repair some cassettes that broke near the end or beginning, some of them being audio and for vic 20, i still have some vic 20 tapes to go threw and check. that being said they also get brittle.
Resizing that CS1er window would not help even if it was possible because the way its created, it would only expand the background, while all elements in that form would remain in the same place and size. If there is source code available, it would be possible to move things around easily, or increase size of some elements, using Visual Studio.
You mentioned that the game was already archived. Did you compare your recovered version to the 'proper' one? If you can see which bits were unreadable you may be able to match that with the sound wave in order to see why the decoding circuit got confused. Probably more trouble than it's worth though!
also make 8, 10, 12 separate recordings of the tape and then stack the data, average, sigma, what have you, to at least set yourself up to start from the best possible starting point before actually diving into reconstruction. edit: unless cassette recordings are flawless, i have no idea, i usually deal with photons not audio :)
i just recently got a ti99 +4 with a pair of controllers, couple cartridges, power supply and spare video chip? even though its advertised as working. it also has the alps keyboard. i really didnt expect to win that bid, what i really want is the atari 800.
That's a great lot you got. And hang on to that spare video chip, because they're hard to come by (non-fake ones that is). The Atari 800 is great too! Both really fun systems.
That was really interesting, I imagine it gives hope to some people who still have some old games they wrote on cassette. Did you consider using the 'normalise' option in Audacity? it creates a balanced volume.
I am pretty sure that normalisation in Audacity is not creating a "balanced" volume but simply increases the volume to the maximum available within your number of bits of resolution (i.e., it will set the highest amplitude in the file to the highest possible digital value, and scale everything else relative to that). So it's not going to fix areas where the volume changes. It is, however, quite useful for making sure that your playback is loud enough. My phone doesn't have a lot of amplification out the headset jack, apparently, so on some WAV files normalising them made them load where they wouldn't before.
@@HappyCodingZX But it's not more consistent throughout the track. It's just "louder." If you turn the volume down an equivalent amount on your output device, you'll have exactly the same waveform again. Note that it also won't produce more consistent volume between tracks because it normalises to the highest peak, and the average to peak volume varies depending on the track. It's also worth noting that if you use this on a CMT track, you should first cut off any loud clicks at the beginning and the end of the track, as those will lower the amplitude of the actual CMT save audio.
I don't know, if the saved program is saved as "text" on the casette, including the "100", "101" lines, but maybe load it as text in that program, copy that into notepad and join all lines into one. Does Basic save invisible chars like linebreaks? Replace them with a space or underscore or this: § (this should not be on a US keyboard, so copy it) after that, you break the line every 64bytes with ENTER. So, there should be an error on line 14 then, probably? I mean 64 bytes is: XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX Or going the other approach... 14*64 = 896 (0x380). so in a "clean" hexdump with no headers/footers, it's from 0x380 to 0x3C0. Wouldn't hurt to match both hexdumps (of record 14) against each other and compare them manually. My guess is, this is something you cannot leave 100% to a program, it needs human intervention. Sure "ORINT" instead of PRINT can be fixed on auto, but what about numbers or GOTO linetargets? Yeah, this is complicated, but I have been doing strange approaches to problems.
How do you know that "Gogfight" is wrong? It could be that it was saved that way. lol I spelled things wrong in my programs all the time. And sometimes even professional stuff has spelling errors.
Yeah, initially I was convinced it was a bit problem. Then I looked at the bit patterns of D and G and that was 3 bits off. Eventually I realized it was probably a misspelling 😃
What about trying to clean the media to improve reading performance before recording the wave? Would that be a thing? I think Techmoan has made a video on a device that really cleans cassettes...
Nice process on recovering the file. Too bad they didnt rewrite the data twice on another section of tape. I wonder if there was a technical reason or if they just wanted to get it out the door.
When I was a 12 year old kid, lugging my TI-99/4A to grade school, my friends thought it sounded like punk rock! I could eat lunch while loading my version of Space War- the program that caused me to want Extended Basic and the 32 k expansion because I ran out of the 16k
I wonder if...uh-oh...an A.I. could discern the signal from the background noise. In any case, what a wonderfully useful programming project this will make. Excellent video!
The best tool is Audacity itself. Because you can edit the wave form and since a 1 and 0 are the only two expected waves forms, anything with another shape is corrupt. But most of the time what remains can still be identified by a human. The easiest is if you suspect it is a one, just copy a "1" wave pattern from another part and transplant it in the broken area. I recovered a complete ZX81 tape this way. And if you have guess then just try, having a checksum helps. Also if there are 2 copies you can lay them side by side to see what the original bits should have been because it is unlikely the same bits got broken. I wished I had something like Audacity back in the day. Back then tape with a slight corruption or drop-out was lost, sometimes many hours of work. Looking back a lot could have been recovered easly.
Sadly, that is one of the drawbacks of using magnetic media as a storage system. Both cassette tapes and floppy disks use the same technology. The iron oxide on the surface of the media degrades over time. Tiny particles of iron oxide break away from the media plastic surface causing signal "drop outs. One missing piece of machine code, or basic programing instruction can render the entire program useless. The average life expectancy of magnetic media is around 30 years. Most of these games on cassette have gone well beyond 30 years, and are now 40 years at the time when they were recorded. When you listen to an old music recording, you don't really miss any signal or sound "drop outs", but computers DO. ❤️❤️❤️❤️👍👍👍👍👍🫡🫡🫡🫡😊😊😊😊😊😊
a trick for restoring difficult tapes: in Audacity apply a very high pass filter, let's say 16000 Hz. This cancels out all volume variations and keeps only the zero crossings, which is what matters for tape decoding. With this trick I was able to restore a rare tape for the Tarbell interface.
Ooh, very interesting! I didn't think of doing that. I could see how that would work.
Thats ingenious, it took me two minutes to even comprehend what you said 😄
Uhhh can you or someone explain me how this works? Because for one there's no 16KHz signal on tape, the frequency response of it fades out before that, so by all logic if you highpass at 16k all you get is noise from the tape machine; and for other the signal that creates the zero crossings is the 1- or 2-KHz squarewave, so when you highpass it, wouldn't you create new zero crossings all over the place at multiples of the original zero crossing rate? Assuming this works as described and there wasn't a accidental error in the description, what am i actually missing?
@@SianaGearz you are missing that the high pass filter hasn't a flat response, the attenuation below 16KHz is gradual, so you still get some of the 8kHz, less of the 4Khz, and so on (with increasing attenuation as you get closer to 0 Hz). So you still retain a good part of the original signal but in a cleaner form where zero crossings have been enhanced (this because the "step" response of the square wave has a very high frequency content).
@@ninoporcino5790 OK wow yeah that's smart. let's say you create a 6db/oct slope; what filter slope do you use? It deletes low end rumble and many sources of offset, we can outright kill anything below the band and it will just centre the crossings better, and it creates an eq which for a 1KHz wave, strengthens its let's say 3k 5k 7k components increasingly which helps sharpen the box edge. I suppose there's a number of increasingly more complex recipes to EQ the tape to beautify the pulse shape but i get it now.
Hi Noel, 2 things to note about cassette : many of them have the foam buffer cooked or dead, making the tape band reading shitty, and errors here and there. Replace the foam buffer by a new one, and oh ! Miracle ! the tape read again without any errors ! Next is the copper arm on which the foam buffer is glued. This one tend to relax too much with old age. Just put some demak up cotton piece under it, and done :D
Very interesting! It sounds like there are even more techniques BEFORE reading the data like that. Super interesting. If I come across some tape that has problems being read, I think I'll try that. Thanks!
@@NoelsRetroLab Yes. When i got sent 150 tapes to dump as WAV and then pass in CDT for the Amstrad CPC from a big collector, i had to find solutions to get the datas read correctly. And the unrewinded band is a classic. with the band properly sticked to the read head, that's it, the squeeze is much more light or even better the signal is again proper, so when processed, it's all good !
@@dlfrsilver Thanks :). Like the CPC!
Thank you for the valuable tips! I have some old tapes from my Amstrad CPC from my childhood with some self written programs and games I want to record and that could be really helpful advice.
The fact that we can actually "see" the BASIC listing inside a digitized version of an analog tape from 40+ years ago is fascinating.Great video!
And yes, it looks that you need some trial & error until you get most of the listing loaded; doesnt matter if the errors are typos, those you can easily correct post-load; the problem are those nasty frequency incorrections that make the load stop amid file.
I like this content. I get both the concept and technical detail on the subject. You are a great teacher. Excellent job Noel.
Glad you enjoyed it! Thanks for the feedback.
azimuth screw adjustment may help as the tape was recorded on a different player with likely different alignment. It wasn't uncommon to have to mess with this to get round R Tape load errors on the spectrum with original games.
Normally, you'd get a 3khz reference tone and turn the screw as you listen (and have a spectrum analyzer running) so you can get it just right. Since the tapes used lower frequencies than that, you can be sure that the 3khz reference adjustment will be perfect for the tapes.
CS1er, what a powerful tool!
It might be worth a try to replicate the circuit from the TI-99 on a breadboard and run the audio into that. **AS I UNDERSTAND IT** the circuit (MAG IN) uses a 4558 dual op-amp (U400) to pre-filter the signal. The first side of the op-amp is configured as a low-pass filter for 18kHz, which is obviously just to kill any non-audio noise. The second op-amp is configured as a low-pass filter at 1.064Khz, with a high gain, which probably would drive the output as a digital switch. Since it is a low-pass filter, it should be producing a positive output on the '0' signal. Any negative output would be snubbed by the diode CR402. This would produce a digitally-inverted output into programmable system interface U300, pin 30, which is an inverting-input interrupt (INT11).
That is just from a brief look at the circuit. It would be informative (and good content) to replicate the circuit on a breadboard and look at the output on an oscilloscope. You could even pipe the output from the breadboard to U300-30 with a jumper to see if the circuit works. Anyhow, I really like this kind of content and I'm glad there are folks like you out there helping the community. Cheers!!
That's actually a great idea (and much better insight at what it's actually doing than my cursory glance at the schematics). I may have to try that at some point. Thanks!
That's an excellent idea. It would also be worth scoping out the signal on the existing machine at various points in the circuit to see how it's responding to various input waveforms, volume levels and so on.
...a signal with 2 different frequencies is FSK, frequency shift keying... FM..is modulated across the bandwidth range, eg centre freq is 100Mhz.. FM can slide up and down any part of its bandwidth (in FM radio this is 200khz so it can slide a max of 100khz either side of the centre frequency) in less than hz resolution and is very analog in nature.....FSK is 2 destinct frequencies only... one to represent the "0" one to represent the "1"....completely digital...usually seperated by 800hz (again in radio...which ALL this tech is based off...and even the tech we use today in our modems, the signal is just sent over wires instead of the air)
there are some great radio hacking tools out there that would take this signal and generate the actual data... and you can reconstruct the bad data (eg universal radio hacker)
...when i got into radio i really liked to capture and decode Marine Weather Fax... as it reminded me a LOT of loading tapes in the CPC... its 2 distinct frequencies (FSK) and it transferred the image line by line, but very slowly (about 1 line per second) as it was being sent over the air and has to deal with things like ionospheric conditions
Good catch. The one difference from radio, then is that the frequencies in CMT FSK systems are usually separated by one octave, such as the 689.5/1379 Hz pair here, or the more common 1200/2400 Hz pair.
WEFAX doesn't have to just be two frequencies, although it often is due to the black-and-white nature of the charts. But it's fully grayscale capable and you'll see that when they send weather satellite images.
@@flapjack9495 yup youre right... RTTY would have been a better example... but the sound and way WEFAX downloads is alot like tape loading..
A perfect balance of explanation and demonstration, keeping all the unknowns in play, and humbly making space for new perspectives. Superior tech vid. Bravo Noel.
One thing I didn't see - tape damage might be due to be stretched, which would alter the frequencies. It might be worth lookigng at the damaged portions to see if maybe you need to somehow speed that section up. It would also be interesting to feed the raw signal into a spectrum analyzer. You should have just the two tones. Look for variations where they move.
Good point. I think slight stretching (i.e. frequency shift) is still fine since these computers were not precise at all. Sometimes you can play something at 25% faster speed and they will still load. I think at that point the signal would even look different at a glance. But I love the idea of running it through a spectrum analyzer! I should try that next time.
@@NoelsRetroLab It may depend on the algorithm. It may adjust to a consistent speed difference, but choke on a sudden change. Especially if it shifts one of the frequencies close to the other.
The algorithms used by the machine firmware of the era are resistant to tape stretch. They only need to differentiate between approximately 0.5ms and 1ms zero crossing intervals so there's a lot of room. Adaptive algorithms which try to estimate time stretch were not at all used, so consistency of speed isn't of any consequence, there is simply a timing threshold below which a zero crossing is considered to be a 0.5ms one and above which a 1ms one; however the threshold tends to not be perfectly consistent depending on previous state of the decoder. As resistant as the resulting algorithm is to stretch, nonetheless bias, phase shift and stretch together could conceivably turn the 0.5ms interval into something that looks just above the chosen 1ms interpretation threshold, so in error sections, manual inspection is a good idea. I don't think you'll see any exciting frequency shift per se but it would be curious to double check.
Accoustic modems used a multitude of techniques such as FSK (Frequency Shift Keying), PSK (Phase Shift Keying), QAM (Quadrature Amplitude Modulation) and even QPSK. The bedoing! bedoing! sounds heard are a part of the communications speed negotiation phase.
I am genuinely surprised no one has made better software by now. The hardest part would be to actually translate the audio into actual binary data, but once you have that you could easily make the troubleshooting 1000x easier by showing the code and highlight the record(s) that was not read correctly. Maybe show both copies of the same record in case one is less corrupt. Could even throw in some basic (pun intended) spellchecking to highlight things like "orink" instead of "print" and so on.
The retro community is huge, you'd think this would be one of the first thing that was made to ensure digitizing dying physical media is as successful as possible.
Right? And not just for the TI99/4A. That seems like a program it would be super useful for every retro platform with significant cassette library (although to be useful they'll have to be very customized for the platform).
Well, the physical media doesn't need to be decoded right away; saving a decent quality WAV file (22 kHz seems to be the minimum, but 41 kHz works better, 8 bit resolution is fine though) preserves the audio and then you can decode it at leasure any time later.
One thing that would be really helpful from the development community would be to open source their software, or at least make it more of a Unix-style modular set of tools than one big GUI program. For a program like CS1er (which looks to me to be pronounced like "scissor") anybody else working on the same thing or something similar has to start again, especially after the author walks away at some point and is no longer available even to ask for the source.
Very good! I was going over approaches in my head while watching. You hit them all and then some!
What you showed in your video is band signal squeezed, due to unrewinded tape. The way to read the tape correctly without the squeeze or less squeeze and then able to recover the tape content : use my method bellow. I have countless of faulty tapes with that. (i did more than 2000 original tapes dumped for Amstrad CPC, but the method applies for any other machine).
Thanks for another wonderful video, it really was a blast from the past hearing the sound of loading from tape on the TI 99-4A and seeing the BASIC code. I did a lot of playing around in my teens, writing programs for my TI back in the early 80s, and am very familiar with the sounds of the tape loading/saving and the frustration of having a program disappear after a load failure.
Awesome software. I could do with something like that for the ZX81 and the Speccy.
To this day I've never used a tape short of creating daily backups that no one including myself knew how to recover. LMAO. I have the tape adapter for the U2+ on my C64.....but I haven't even used that either. Everything I've found is a D64 or a T64 (which doesn't go through the tape interface)
I've tried to recover two excellent games I made for the TI. Didn't know about CS1ER. I can say they were EXCELLENT games (comparing with others BASIC Games). I'll try again with that program.
Good luck! CS1er is so powerful, I'm hoping you can at least do a partial recovery.
Some software houses used a copy protection, one small part of the tape was recorded on purpose with a very low volume. When the tape was read by the computer, that particular part gave an error, and then the computer could not read back that part. The tape with the error was original, then the computer was ordered to continue. But if the tape was copied, most users have an automatic gain in their cassette recorder, then all parts of the tape could be read back, then the tape must have been a copy and the programme refused to play. When I saw the audio file I recognised this very ingenious copy protection, neither a newly saved cassette nor a copy would work, both had perfect audio at the correct levels.
Pulling this out from the deep dark mists of memory, but the TI is particularly volume sensitive as well. I’m not surprised you weren’t able to easily recover and load just using audacity. I seem to recall having only one small volume range where it would read my tapes, but that may have been partially because of the player I was using.
There is 'castool' which comes with MAME. I don't think it's very full-featured but from memory it supports a few systems in a more or less uniform way.
I'm still sad the guy who made the SVI-CAS seems to have made a total of about 7 of them then disappeared off the face of the earth. Been looking for one for.about a year!
There are plenty of clones implementing the hardware and still compatible with the firmware.
@@talideon Can you point me in the direction of one? Clearly I'm not searching for the right thing!
Fantastic deep dive as always Noel, thank you.
this is unbelievably powerful, wow!
Ooff. When I try to google programs like these, nothing comes up and I have to reinvent all the wheels.
I recently recovered some undumped DECO Cassette System games, but I literally had to write my own data extraction program. The only information I had was in the MAME driver, telling me that every block starts with 00 FF, has 256 bytes of data, then two bytes checksum (which is encrypted just like the data) and then FF 00. There is silence in between blocks.
I also did multiple dumps (since it uses proprietary tapes, the original player had to do) using different approaches like hold the tape in a certain way, play it at different speeds. On some tapes, I couldn't get a single clean dump, but with all the debug information my program output and having four dumps, I could always do a majority vote and the game programs are now in MAME.
And yes - looking at the zoomed in waveform, manually decoding was how I started.
Very cool stuff! I love it that it's possible to make sense out of the audio file data like that. Too bad you had to write everything from scratch, but it was probably worth it.
I wonder if it's possible to write a "universal" program with different plugins or rules for different platforms, or whether they're too different from each other.
ahh, ideal video to watch after supper
All the tape formats I played with used more than two frequencies, the others are for things like the synchronization tone. And in fact the frequencies are due to how much time passes between sound pulses, the zero crossings. But if the CPU needs to do more instructions for any reason you'll end up with some durations that are not quite the one for the zero or the one for the one. So you often have to tweak them. Also depending on whether you count the wave crossing zero on the way up from negative to positive or down from positive to negative you can get slight differences, like half a wave. I believe some systems actually count the zero crossings in both directions too. It can be challenging and fun and sometimes frustrating. Anyway great to see somebody do such a good video on the topic!
yup there are usually "pilot" signals at the start (and sometimes the end) that synchronize the decoder with the incoming data..but the data itself is usually just in FSK
That's interesting, because a friend and I have been working on some Python code that reads and generates tape saves, and we've never seen more than two frequencies so far. (We've got National JR-200 and Hitachi Basic Master Lv2 working, NEC PC-8001 mostly working, and have looked at FM-7). The leader and separation tones have invariably been at one of the two frequencies used for FSK.
@@Curt_Sampson I haven't played with them for over a year and a half but at that time I played with Sinclair ZX Spectrum, TRS-80 model 1, and Apple 2. I didn't manage to get all three working from the same code. Speccy for sure has a long leader tone/synch tone be the 1s and 0s. I don't remember the others as well.
Is there anything like this avail be for Amstrad CPC? There was a game I made in basic back when I was 14 and I saved it on to tape. Many years later on, I found that tape again but it has un-fortuneateley has deteriorated to the point where it just can't be loaded anymore.
Not that I know of, but I was kind of hoping this would encourage someone to do it. If not, I may have to do it at some point 😃
For Atari 8-bit computers there is a program called Turgen, which supports various standard and custom tape loading formats that were used on the platform.
The data is not out of the band surface or demagnetized, it's still there, it's just that due to the unrewind, the foam buffer too old and the copper part too relaxed, the head tape drive can't read it correctly.
Thanks for sharing, it was interesting to know how the data is stored. I think, if all blocks on the cassette are repeating twice and have a checksum at the end, then it may be possible to reconstruct a good block from merging corresponding bits together. If you have a sound file with a good quality, I can try later in Python :)
It's disappointing, but understandable, that those writing the tape loader routines in machines in the early eighties and late seventies didn't seem to know about Hamming codes and other error correction mechanisms and instead resorted to just clumsily duplicating the data.
Right! They could have done so much more. At the same time, this is a real-time process and maybe they couldn't spare the cycles or the RAM to do any fancy computations.
Even if they had just interleaved the blocks, so the two copies of any given block were farther away from each other on the tape, it would have helped a lot. Many over-the-air digital radio protocols do this because errors tend to come in bursts.
Your a Wizard Harry...
Thanks!
Wow, thank you very much! 🙏
Recovering these tapes is very much like the problems of recovering data from deep space spacecraft.
i have had to repair some cassettes that broke near the end or beginning, some of them being audio and for vic 20, i still have some vic 20 tapes to go threw and check. that being said they also get brittle.
Resizing that CS1er window would not help even if it was possible because the way its created, it would only expand the background, while all elements in that form would remain in the same place and size. If there is source code available, it would be possible to move things around easily, or increase size of some elements, using Visual Studio.
Alas, it appears to be closed source. :( Quite unfortunate for a tool like this, and even more so now that it's been abandoned.
You mentioned that the game was already archived. Did you compare your recovered version to the 'proper' one? If you can see which bits were unreadable you may be able to match that with the sound wave in order to see why the decoding circuit got confused. Probably more trouble than it's worth though!
also make 8, 10, 12 separate recordings of the tape and then stack the data, average, sigma, what have you, to at least set yourself up to start from the best possible starting point before actually diving into reconstruction.
edit: unless cassette recordings are flawless, i have no idea, i usually deal with photons not audio :)
i just recently got a ti99 +4 with a pair of controllers, couple cartridges, power supply and spare video chip? even though its advertised as working. it also has the alps keyboard. i really didnt expect to win that bid, what i really want is the atari 800.
That's a great lot you got. And hang on to that spare video chip, because they're hard to come by (non-fake ones that is). The Atari 800 is great too! Both really fun systems.
Gogfight must occur over North Wales.
The microbee had a handy option to be forced to read tapes with checksum errors.
Oh wow! I don't know I've ever seen that option in any computer. Smart!
That was really interesting, I imagine it gives hope to some people who still have some old games they wrote on cassette. Did you consider using the 'normalise' option in Audacity? it creates a balanced volume.
I am pretty sure that normalisation in Audacity is not creating a "balanced" volume but simply increases the volume to the maximum available within your number of bits of resolution (i.e., it will set the highest amplitude in the file to the highest possible digital value, and scale everything else relative to that). So it's not going to fix areas where the volume changes.
It is, however, quite useful for making sure that your playback is loud enough. My phone doesn't have a lot of amplification out the headset jack, apparently, so on some WAV files normalising them made them load where they wouldn't before.
@@Curt_Sampson yes, I wasn't using it in a technical sense, perhaps the better word to use would be 'consistent'.
@@HappyCodingZX But it's not more consistent throughout the track. It's just "louder." If you turn the volume down an equivalent amount on your output device, you'll have exactly the same waveform again.
Note that it also won't produce more consistent volume between tracks because it normalises to the highest peak, and the average to peak volume varies depending on the track.
It's also worth noting that if you use this on a CMT track, you should first cut off any loud clicks at the beginning and the end of the track, as those will lower the amplitude of the actual CMT save audio.
@@Curt_Sampson thanks, that's clear.
14:44 LINE 1710 ….“ENBNDED IN A TIE”… 😂
What a use case for galois forward error correction huh? 😊 Can u imagine these little computers trying to compute all that 😂
The tapes I want to read are mini-cassettes and the data is frequency modulated. How to start with that?
I don't know, if the saved program is saved as "text" on the casette, including the "100", "101" lines, but maybe load it as text in that program,
copy that into notepad and join all lines into one. Does Basic save invisible chars like linebreaks? Replace them with a space or underscore or this: § (this should not be on a US keyboard, so copy it)
after that, you break the line every 64bytes with ENTER.
So, there should be an error on line 14 then, probably? I mean 64 bytes is: XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX
Or going the other approach... 14*64 = 896 (0x380). so in a "clean" hexdump with no headers/footers, it's from 0x380 to 0x3C0. Wouldn't hurt to match both hexdumps (of record 14) against each other and compare them manually.
My guess is, this is something you cannot leave 100% to a program, it needs human intervention. Sure "ORINT" instead of PRINT can be fixed on auto, but what about numbers or GOTO linetargets? Yeah, this is complicated, but I have been doing strange approaches to problems.
How do you know that "Gogfight" is wrong? It could be that it was saved that way. lol I spelled things wrong in my programs all the time. And sometimes even professional stuff has spelling errors.
He mentions that in the video.
The final text also says ENBDED instead of ENDED, clear mistake because is not only a bit shift, is an extra character.
@@PG-gs5vb yeah.....but I hadn't gotten to that part yet. lol
@@GeomancerHT yeah I wondered about that too.
Yeah, initially I was convinced it was a bit problem. Then I looked at the bit patterns of D and G and that was 3 bits off. Eventually I realized it was probably a misspelling 😃
What about trying to clean the media to improve reading performance before recording the wave? Would that be a thing? I think Techmoan has made a video on a device that really cleans cassettes...
Nice process on recovering the file. Too bad they didnt rewrite the data twice on another section of tape. I wonder if there was a technical reason or if they just wanted to get it out the door.
to change a D to a G in DOGFIGHT, the computer would have to get 2 bits wrong. Which seems a bit unlikely. So maybe it was originally typed like that!
gg for the perseverance
When I was a 12 year old kid, lugging my TI-99/4A to grade school, my friends thought it sounded like punk rock!
I could eat lunch while loading my version of Space War- the program that caused me to want Extended Basic and the 32 k expansion because I ran out of the 16k
Classic99 is technically a *simulator* - Win99 is probably a better bet
I wonder if...uh-oh...an A.I. could discern the signal from the background noise. In any case, what a wonderfully useful programming project this will make. Excellent video!
I bet you could use machine learning to repair tape data really really well
That would be an awesome video!
That's actually a great idea!
@@NoelsRetroLab thanks Noel, I loved watching your video :)
The best tool is Audacity itself. Because you can edit the wave form and since a 1 and 0 are the only two expected waves forms, anything with another shape is corrupt. But most of the time what remains can still be identified by a human. The easiest is if you suspect it is a one, just copy a "1" wave pattern from another part and transplant it in the broken area. I recovered a complete ZX81 tape this way. And if you have guess then just try, having a checksum helps. Also if there are 2 copies you can lay them side by side to see what the original bits should have been because it is unlikely the same bits got broken. I wished I had something like Audacity back in the day. Back then tape with a slight corruption or drop-out was lost, sometimes many hours of work. Looking back a lot could have been recovered easly.
Excellent recovery - shame about the game!
❣️ Promo-SM
Can't understand ya....
I thought this was Adrians digital basement. ill kick myself ro the view you got. Fuck am pissed.
????
Guao good learning from your videos Noel a big hello from Spain 😉🥳🦾
Sadly, that is one of the drawbacks of using magnetic media as a storage system. Both cassette tapes and floppy disks use the same technology. The iron oxide on the surface of the media degrades over time. Tiny particles of iron oxide break away from the media plastic surface causing signal "drop outs. One missing piece of machine code, or basic programing instruction can render the entire program useless. The average life expectancy of magnetic media is around 30 years. Most of these games on cassette have gone well beyond 30 years, and are now 40 years at the time when they were recorded. When you listen to an old music recording, you don't really miss any signal or sound "drop outs", but computers DO.
❤️❤️❤️❤️👍👍👍👍👍🫡🫡🫡🫡😊😊😊😊😊😊