Porting Bad Apple to the Homebrew CPU!
ฝัง
- เผยแพร่เมื่อ 17 ก.ค. 2024
- The Bad Apple Demo is one of the most complex demos I’ve implemented on the system with synchronised Audio, Video, SD card and the UART. In this video I discuss many of the technical challenges and my process for solving them.
0:00 Intro
0:22 Overview
0:59 About Bad Apple
2:29 Graphics Rendering
8:28 Time & Streaming
12:06 Lyrics
14:06 Graphics Compression
21:33 Sound
26:38 Outro
27:04 Bad Apple Port Close-up
The Links:
My 4k Bad Apple Recording: • Bad Apple on the Homeb...
Bad Apple Explained: History and Analysis: • Bad Apple Explained: H...
"Bad Apple on everything" Playlist: • Bad apple on everything
Bad Apple Resources: archive.org/details/bad-apple...
The source Midi: • Touhou Bad Apple!! - M... - วิทยาศาสตร์และเทคโนโลยี
![[TH] Esports World Cup : Grand Final](http://i.ytimg.com/vi/sedbKhFYDMo/mqdefault.jpg)
Join us on Discord: discord.gg/jmf6M3z7XS (Over 1024 members!)
Follow me on Twitter: twitter.com/WeirdBoyJim
Support the channel on Patreon: www.patreon.com/JamesSharman
CAT Compression - If it fits, I sits!
It fit. Just!
Brilliant work. The 80x60 version is impressive in itself, but the deep-dive into extending the resolution is fascinating.
Glad you liked it! For me the challenge is what makes it interesting, nice to get something that drives the system!
This is a great examination of many of the old tricks used to fake high res graphics on old limited systems. Really takes me back.
Glad you enjoyed it! There is going to be a lot more "old graphics tricks" in the future.
@@weirdboyjim Great news, I used to love the challenge of coming up with tricks to get around the limitations of the old 8 and 16 bit machines.
Old demo coder here just saying this is awesome and it brought me back!
Glad to be of service!
Prioritizing audio over visual playback makes prefect sense, as humans are way more forgiving of visual stutter/discrepancies than of audio issues. As well as the variability of the amount of processing needed, as you mentioned.
Glad you think so!
It’s impressive to see your home brew running such an easy to recognise demo… I daresay you feel rather chuffed at just how much you can achieve with a machine you’ve created from scratch 😊
Thanks Timothy! I was indeed quite pleased with this, especially getting so many different parts of the system working together.
It is great to see the computer being used just for the sake of doing something. It shows that it really is a whole working system. Congratulations!
Great explanations, as always. Thanks!
Absolutely! It was rewarding to make!
Extremely impressive, James. Seeing how you solve technical / programming challenges has always been the most interesting content on your channel for me, given your expertise. Thanks for sharing your incredible knowledge and skills with us!
It's great that people are finding it interesting!
Saw that amazing demo first. Now time to see all of the backgrounds...
Edit: impressive work! Thank you for your clear explanations and for sharing your work. Very inspirational!
Good to hear you found it interesting!
Can it run Doom, can it play Bad Apple .... I love it!
Of course this time it's Yes! Glad you like it!
Nice work James.
Very clever implementaion. !
Thank you! Cheers!
Ahhhhh finally got to Bad Apple :D Gotta say it's been great watching this long term project grow over time :)
Thanks! I appreciate your support Rae!
Love the work you do. Look forward to every episode. ❤
same
Thanks! I hope I don't disappoint!
Facinating and humbling. Still thoroughly enjoying these vidoes! Great stuff. 👍
Glad to hear it!
Magnificent!
If you do plan to do more music on the synth, it might actually be good to add a channel that's a white notse generator. You can do a much more convincing hat or snare by passing white noise through an ADSR (the snare has something like a tom on a second channel, perhaps).
A lot of people have suggested that. I might have a play at some point.
That's an impressive result on your demo!
Glad you liked it!
very cool, thank you!
Glad you liked it!
That's so impressive !
Glad you enjoyed it!
@@weirdboyjim audio and video compression are some of my "hobbies".
Another is CPU design.
You aced them all 🙂
Awesome job!
Very crisp edges and great job keeping up with the transitions!
I see the 4Mb file size, maybe I missed it but what did you end up with for a compression rate compared to the uncompressed down sampled source frame images?
Have you thought of using your UART as an audio output?
Drop the serial and hook the output to a speaker with a resistor and capacitor to limit the current and smooth it a little?
Since you have a 8 byte output buffer on that already you could output 1 bit audio without any additional hardware and that buffer might make the code not too bad.
If you just left it at the 115,200 you'd need to pump out 14,400 bytes a second, or 240 bytes a frame. I don't know if your SD card transfer rate and free CPU time on the Video decode would allow that without reducing the framerate, but I'd bet 115 KHz 1 bit audio would sound pretty good!
Obviously if you reduce the output rate of your UART you could drastically reduce the bitrate of the audio. I wonder what the minimum acceptable 1 bit audio quality would be?
Think you could squeeze that out?
Thanks for putting this out there!
There is not enough content explaining how to actually DO one of these demos with specifics.
I know this video would have been a HUGE help and time saver when starting work on my own Bad Apple! demo.
It would be neat if you took some time to get into the guts of your encoder for the video and how you sliced and diced the MIDI.
Love this content!
The compression was around 11:1, the compression as as much about speeding the decompression up as well. I generally update more of the frame with the compressed data than I could do with the a raw copy.
@@weirdboyjim That trade-off is harsh. When I started looking at doing this with the 6502 I was planning on just using a preexisting decompression library but all that I looked at were mostly meant to be tape or floppy loaders and were focused on max compression, and therefore really slow for what I needed. Obviously more modern routines are too heavy for a simple processor. Add in the slow nature of a bit banged SD card read on a VIA and now you also have the trade off of read speed to take into account.
Is it faster to decompress a little more or to read another byte?
Will adding that 1 more branch on the decoder use too many cycles to offset the bytes it saves on the encode? etc.
I like how you mention being 'good enough' for the demo. You can really keep going for quite a ways.
absolutely insane achievement. 💪!!!!!
Glad you liked it!
awesome!!😍
Thanks! 😄
An incredible amount of work and time.
More than I want to admit! ;-)
It turned out incredibly well. Kudos! It´s right there with the demos for the best performing 8bit computers I've seen. I wonder if rasterizing vector graphics would compress it even further. There is a version made with a fourier series XD
Thanks Damian! If you had a decent bitmap display vector graphics would work well for much of it, would be a bit more complex to deal with the blurry bits and the flames though. Decompression would be a bit more cpu intensive than I'd want to do on this device though.
James I just had an idea - you quite possibly can play samples on your sound hardware through the volume register. Just send singed integer formatted samples into the volume register!
Yeah, I was thinking the same. Would need to lock the frequency output into a fixed high state though. And the amount of streaming audio data would be way higher - would need to be at least 16kB/sec, continuous, rather than a few bytes per frame.
@@simontillson482 could go as low as 4kHz sampling rate and still get OK speech, 8kHz would be fine for low fidelity sampled instruments. True, it would take a lot of bit banging. Some DMA controller would speed up a lot of IO stuff here. I don't recall the memory access method of JAM-1, but if it only accesses memory at rising edges for example, then that leaves the whole other clock phase for DMA, and that could do sample playback autonomously, just fill up buffers with the CPU, set up memory addresses for the DMA and let it rip.
Indeed! I've actually done a little bit of experimentation with that, I'll make a video at some point. Not practical as an aide here though as it needs constant GPU supervision.
impressive!
Thanks Ivan!
Fascinating journey 😁 when I was young I fall in a rabitt hole of creating fonts or electronic components symbols on millimeter paper 8x8 matrix (for ZX Spectrum)... for a moment I wondered why not create all possibilities but I already know binary system well and realised it be 2^64...😅
Lol! I did a fair bit of of the graph paper design work back then as well!
This is of course amazing as-is. as ever. Good problem solving. Feels so ridiculously modern with a trace of retro. Hats doffed. tipped and whatever else one does respectfully with hats.
And, for once. I am an expert (eh as much as anyone is). and if this makes it into the open... will be happy to fiddle very slightly with some of the values in that demo to give my take on OCD concert-pitchish perfection and percussive filth ;)
so good i watched it thrice
Thanks twobob! Like I said in the video, I know the audio is the weakest bit. Part of that is just down to my musical ear being rubbish, hard for me to chart improvements.
I've not even watched it yet and already hit the like button.
Hoping you didn't unlike it after.
Mind. Blowing. %)
Thanks Igor!
So cool! I wonder if you can stream sprite changes as well. Could create some interesting opportunities.
You could! You have balance the total memory bandwidth though, you only have so much to share between everything.
Nice little project for the CPU! I’d recommend looking into the Heatshrink compression library for future projects, if you haven’t already. I can verify it’s suitable for realtime video streaming on microcontrollers.
I had a quick look at that library, I'll look at it more in the future. It wouldn't be suitable for the decompression here though which extremely custom and very much tuned to the data and cpu.
@@weirdboyjim yes, for something that was also storage space or bandwidth constrained it might be useful. I use it for fitting video frames into single UDP packets. Combined with the same sort of frame delta and RLE you are doing, it works really well.
Great work! I was thinking about the problem of being stuck with the vertical blanking period for writes a while back and the only workaround I could come up with would be a 30Hz mode, where, during one frame, you write everything out to a 1MB frame buffer while you display the frame, and on the second frame, you read back from the frame buffer, freeing up your primary graphics memory during the entire frame for writes. But that increases the complexity of the video hardware so much that I just don't know if it would be worth the engineering on a home project.
Thanks for your videos in general, as I really enjoy thinking about these sorts of issues.
It's worth thinking about the problem, I'll explore some different approaches in future builds.
Interesting.
To be honest your audio setup is decent given the constraints. I really like the retro square wave sound it has. What's missing is a bit of polish on the analogue side of things. In particular you need a low pass filter somewhere after the mixer to get rid of aliasing and any other unwanted noise. You'll probably have to experiment to find a good cut off point.
Thanks for the feedback, I'll look into the filter. The one thing in the audio circuit I didn't realize when sketched it out is the high frequency components you get on a volume adjust. Might have been better to latch in the 4 bits of volume at the end of the a "cycle".
It would be fascinating to see a couple of the frame transitions slowed down significantly so you can see the block-by-block updates happening. Taking the existing encoding and doing one operation every few frames?
Great work and explanation as always! One question: how many LOC do the sources of the player binary have?
Thanks. I use a lot of generic library code, but dedicated lines written for this: The de-compressor for the video stream is in it's own files and totals 195 lines. The main file is just under 300 lines of assembler.
Hell yeah
Thanks!
Stunning work, yet again - great job!
And sympathies on the audio struggles - I think your approach made a lot of sense given the limitations you had, though. And you're actually wrong when you say none of your percussion sounds are going to impress anyone - the bass drum you made at the end is quite good, I like that a lot.
Here on out is unsolicited-suggestion-ville, so take from it what you will: if it were me, I'd like to add a noise oscillator - either replacing one of the existing square-wave channels or augmenting one or more channels with that functionality. There's ways to generate analog noise directly in hardware - transistor noise is one I've heard of and seen from folks like Moritz Klein - but there's also a very famous sound chip that generates 1-bit noise: the General Instrument AY-3-8912 used in (for example) the Amstrad CPC computers. Basically, it has a hardware random number generator (I think it uses a linear-feedback shift register, like the NES APU does) and instead of switching between 1 and 0 at the frequency you set, it switches to the next random bit value at the frequency you select. That should be enough to generate some fairly convincing white-noise-like sounds for your hi-hats and cymbals, and as a bonus you can get some nice rumbly blasts for explosions and such in your games.
yeah, I did think much of this. but honestly he made it work despite all that.
Thanks for the kind words! I am curious about the way noise was used by some earlier synth systems but my knowledge about sound lags far behind my graphics skill. That said generating a random bit sequence in hardware is an interesting challenge, I might have a play at some point.
@@weirdboyjim Sound is definitely a massive rabbithole, for sure - I grew up playing the piano, spent years watching music theory and DIY synthesizer videos on TH-cam, and I've written dozens of completed tracks and hundreds of fragments in the fantasy console PICO-8, and I still feel like I'm missing a lot of knowledge.
The way I think of noise in music composition for 8-bit machines kinda comes from a rock/pop theory perspective: a classic drum kit has low percussion (the kick), mid percussion (toms and snares), and high percussion (cymbals and hi-hats), and you want to be able to make sounds to fill all those roles. Having a pitched oscillator - like your square waves - that drops quickly in pitch - like you did with your bass drum - is a good way to make a low or mid percussion, the kick/bass drum and tom roles, but if you try to start your pitch higher, you get a kind of laser-zappy noise. Which is cool, but it's not a hi-hat. A short burst of noise, on the other hand, *can* sound like a snare or hi-hat - it gives you the high percussion sounds you struggled with. That's why it shows up in analog synthesizers, in classic consoles (NES, ZX Spectrum, Commodore 64, etc.), and even in modern synths.
...come to think of it, I think the ZX Spectrum *also* has an AY-3-8912 chip in it. Is there a ZX Spectrum music tracker you can play with? I feel like there must be.
@@Packbat noise is an entire subfield. since white noise + filter stack = eh, everything, given enough skill.
If we are going to say things that "might be nice" mine would be use the "you get a kind of laser-zappy noise" above at the start of the kicks just like the OP said here. Again totally nailed some of the less obvious, easy oversights to make, in reproducing kicks.
An oscillator that can drop in pitch given a period and a range really can be invaluable,
if we are adding cools things... 74HC164 as an LFSR with a 74HC86 for FB? maybe that would sound good? surely you have that just lying around..mangle the clock division.. surely some noise in that. or some other similar "nod to noise" without succumbing to an actual wave/synth solution. I'm biased. I like sounds
I'd definitely be curious to see what could be done if the pallet was played with as part of the video stream, beyond the colour inversion. My first thought would be to chop the video up into sections with a dedicated pallet each, for higher resolution, though there's probably way further you can go than even that.
This is a seriously impressive demo, I love it
Glad you like it! Well the 2x2 grid works better in some segments than others. I did think of tweaking the entry used for 50% to better match where I was using it.
Back in the day (1992), we used to play with VGA on a 80286 PC and the recently arrived SoundBlaster (that costed a fortune). We were using the 640x480 16 colours mode. You mention 60fps, we were doing half of that, but we were "hiding" the updates: in the 256kB of a standard VGA, there is enough space to "draw" the next image offscreen and reveal it with a simple "out" (I think it was to 0x3c4). Also, in that mode, we only used the 16 grays and the data was pre-rendered (we used PCX as a base standard, minus the header) to only recalculate it into the chroma data (and offset to have more tints in the light shades than in the dark), all that with a bunch of SHLs. Maybe you don't really need to go 60fps?
For the sound, as you already (kinda) went to MIDI instruments, why not going to an OPL4 chip (or did I miss an episode?) such as the YM2515 and simply upload the instruments to the chip, and leave the sound problem for later?
opl series. 1994.
6502 series 1975...
This project is very much about doing things from scratch so it would have felt like cheating. I'll do some more advanced Audio in a future build.
@@twobob No, it's 6502 series 1975, OPL series 1984 (10 years, not 20): The YM3526, introduced in 1984, was the first in the OPL family, providing a nine channel, two operator synthesizer.
@@weirdboyjim I know it sounds cheating if you make it permanent. My point was to start thinking of goals to achieve with that scratch build in the future, but already have an API (or a subset of it) and a comparison point. It's as much cheating as Android being a copy of Sun's JVM: the implementation will be yours, not Yamaha's.
Also... Do you really need 60fps?
That was really interesting, thanks James. I do a lot of retro-gaming conversions and emulation etc, and these kinds of optimisations are always interesting to me! (currently trying to convert Missile Command to my tile+sprite based platform on the old 8-bit Arduino, with composite output, using a similar idea of tiles with partial line segments) Out of interest, did you consider a frame-direction byte, so the data is rendered to the buffer either horizontally or vertically? Also a byte at the start of the frame to choose the tile-set to use, which could be more optimal for the frame? Your audio section reminded me of Tim Follin's ZX Spectrum music - that dude was a master at getting amazing sound from it's little beeper :)
When building this demo I had a number of idea of small hardware tweaks that would improve it but I was keen to do the best demo with the hardware I've had rather than tweak the device.
@@weirdboyjim Ah! I didn't realise you cannot switch tile sets in the software. The frame rendering direction would be purely software. As you say though, it's done and looking good anyway :)
The world is full of unsung genius.
Very kind!
@@weirdboyjim Not at all! It takes one to know one XD
What year do you think this computer could have been built with equivalents of that time. And would it be an impressive computer of the time? Very impressive project from start to finish!
Glad you are enjoying the series. That's a really tough question, the majority of the circuit could have been built in the late 60's. The big "liberty" I take is with memory devices, a 64KB of sram would have cost you a kings ransom.
Could you have somehow programmatically determined the best tiles for video? Maybe split the each frame into tiles and then find the closest ones and combine them? Also could you have changed the pallet frame to frame to be able to have the best tiles for that frame?
In theory you could have determined an optimal set but that would have been a sizeable job compared to the time I wanted to devote to this, and I don't think the improvement would have been significant. I did investigate some dynamic tile modification, the problem with that updating tiledata consumes the same memory bandwidth as updating the tilemap which was already constrained.
It sounds like you need a hw random 8-bit number generator you can hook up to the audio circuit to help with noise style outputs?
It is interesting the way old synth chips used the noise channel.
@@weirdboyjim yeah. The C64 GEOS copy protection was very interesting in how it used the SID chip noise to decode itself. 😎
NES created noise without an RNG, by the way. I don't know the exact details, but I think there was some prewritten data that it would loop through with some feedback mechanism, so that you could also then somehow alter the tune.
The different parts of the song ended up not being in tune with each other in terms of how they’re transposed. Should be fixable? I mean like the long low notes (bass) that play alongside the melody.
Is that in the source midi data? My musical ear is not good.
@@weirdboyjim yeah it should be fixable in the midi data. I cross posted in discord. Would it be possible to play the same scale with each instrument, then we should be able to figure out how to transpose the midi data
@@weirdboyjim no, the midi is fine. the instruments you created are out of tune
the link on the bad apple vid thats supposed be to this vide is broken (on its description)
Not sure what happened, that should be fixed now.
To quote a Monty Python script.... MY BRAIN HURTS!
When you were building the audio device, I always wondered why you did not incorporate timers to generate various tones, paired with digital volume control chips? Sure, they would be square waves, but with volume modulation, you could make a nice sounding output. Add another timer to the actual volume control chips and it could modulate without the need of heavy CPU work. Sure, this would have two timers and half a volume chip per channel, but it would not be very complex to make. I understand not designing a computer to run demos, but it would certainly make games much more interesting.
If I'm understanding your proposal correctly I think that would be about 4-5 extra chips per channel. Something I find is that every circuit, there is always a way to add functionality and make it better and wherever I draw the line someone else would have done it differently.
Yeah, though I think its not just limited to everyone drawing the line at a different place. Its also people not having tried it first before commenting :D its one thing to make an off-hand suggestion, but its another to actually try implementing it and realizing what a pain in the butt it might end up being.
Man, if I ever get this far along in my diy computer project I want to try a "gpu" with sprites that support rotation, scaling and skew. back in the day IIRC, I saw some code that did all three operations at the same time (but line by line) with some "simple" matrix math (I hope I'm not mis-remembering this.. it was 20+ years ago in a little library called allegro 3) and I imagine it could be done "reasonably well" in hardware with fixed point math and a lookup table. Hmm. But then you'd want the gpu to be able to do the math to offload the work from the cpu. so do you actually support math, or just limit the features to some stuff that can be pre-calculated? feh. The imagination runs wild :D
I'm pretty sure the Gameboy Advance had exactly that, with the option to double the size of a sprite in the buffer to include a 2x2 matrix.
The trick is to find the right level to put it into hardware. To implement the scaling and rotation you can boil it all down to a bunch of adds from registers.
Do you really need to decode the entire frame before vblank ends? Don't you just need to stay ahead of the current scan line?
The video circuit is bandwidth limited, It doesn't support reading and writing at the same time. Priority goes to the cpu so if I write outside of blanking it will momentarily corrupt the screen. See my vga series for more information.
hope you decide that you really do need to improve that audio. I've watched computer development over 76 years and don't really remember that rough of audio in any computer I've had. We all have our own definition of "adequate" I guess. Still, would be nice if the sound was less grating - after all, we understand tech better today and you do have precident (ie we didn't have SD cards back then)
I'll be doing a different audio circuit in the future.
Have you had any will, and any success , collaborating with other people on either the software or hardware that interests you?
Largely speaking this project is "I'm doing my own thing". I have been talking to an artists friend about helping me out with some pixel art for games though.
Touhou mentioned
Had to!
"Doujin Circle" isn't a proper name, it's a description of the group. "Doujin" means "same person" meaning "self-published". "Circle" just means "group". In the case of music, you'd translate it as "Indie band". "Alstroemeria Records" is thus the name of the group rather than a record label like the name would suggest.
Ahh, thanks for letting me know. It's difficult getting everything exactly right when it's relates to another language.
why not start a new series, implementing CPU into an ASIC, and sending it to Tiny Tapeout
So many ideas! So little time!
>Hopefully you've had a chance to watch the Bad Apple demo
... Which posted one minute after this video, haha
Deliberate to have the demo as the most recent one in the list.
The graphics is putting the audio to shame.
Well I've always been a graphics specialist.
No accounting for taste, in general I find this demo, weird, but then again that is my opinion of most Japanese (anime) stuff. I do however appreciate the technical merits of you getting it to work on your system. That is a real accomplishment. Hopefully your next choice will sound and look better. To set the bar, the scrolling parrot demo looked better. Your 3d demo looked better.
Well, sorry it's not what you wanted.
@@weirdboyjim don't be, I just find your other demos more impressive and bad apple overrated. As said, it was technically still very impressive. I look forward to seeing what you do next.
why is it always Bad Apple? Bad Apple here, Bad Apple, there, Bad Apple EVERYWHERE. God, I hate Anime.
It’s a video game not anime
As for “why”? The 1-bit friendly black and white animation lends itself to being ported to all kinds of display technologies, and it’s highly compressible.
I really hope we get a lot more Bad Apple, if for no other reason than to piss off this ignorant asshole.
The song is catchy; the art style lends itself very well to being ported to unorthodox or limited systems; and Zun takes a radical stance regarding IP to not only allow others to use his work, but encourage it.
Considering the large overlap between people who watch anime and play anime-inspired games, and people in technical fields, ya. There _are_ a lot of anime references in tech spaces. Bad Apple is just one of them. Another is the phenomenon of "programmer socks".
Fortunately you'll find I have many videos that do not contain any apples or anime.