ฝัง
- เผยแพร่เมื่อ 21 มี.ค. 2023
- Hackaday.io: hackaday.io/project/190161-rp...
Github: github.com/TEC-IST/rp2040-hdm...
Microcontrollers have begun to rival early PCs and retro gaming consoles in measures like speed and memory capacity, and even leapfrog them with new capabilities like multiple cores and wireless connectivity. One area that still seems a bit lacking is video output. Sure, the ability to connect a smallish LCD via i2c or SPI is normal, but can these little marvels do more? With a little spicier circuit, I think so...
And, in fact, we know so: Wren6991 successfully bitbanged DVI on the RP2040 [ github.com/Wren6991/PicoDVI ] and validated the output up to 372Mbps (good enough for HD 720 @ 30Hz). That leaves some resolution milestones unxplored, for example FHD 1920 x 1080 @ 60Hz (1080p) x 24 bit color depth (i.e. 8 bits for each of the red, green, and blue subpixels, aka RGB888) = 2.985984Gbits/second not including control signals and blanking overhead -- approaching an order of magnitude larger. Back to Wren6991: the CPU cycles for QVGA (320 x 240) consume 60% of one of the RP2040's two M0+ cores, i.e. around 30% of its total processing power, so that means there's only headroom to about triple the processing rate. Maybe with a very aggressive overclock it could get in the ballpark, but it seemed like, considering we have up to 30 GPIO pins to play with and DVI's Transition Minimized Differential Signalling (TMDS) encoding uses only 6 pins, there had to be a way of using the 32-bit width to help escape the bottleneck. The RP2040's M0+ cores can execute 1 32-bit move instruction in 1 clock cycle, which means that if all 30 GPIO pins were tasked with sending this output, the theoretical maximum output from an ideal 1 core x 1 ipc code x 30 pins x 133Mhz 'normal' clock speed is around 3.99Gbits/second, although GPIOs are needed for other functions and some processing is necessary to work out what to send (and note that only 26 GPIO pins are exposed in the RPi Pico). I considered an overclock and a Ghz-speed capable shift register, specifically the very interesting MC100EP142FA. I considered a reduced color bit depth, like RGB565 or RGB666. Ultimately, it sememed like an RGB to HDMI encoder would be the best idea to offload the encoding functions, and one can be had cheaper than the ultra-fast shift registers.
Another problem: with only 260KB of RAM onboard, how would the RP2040 store even a single frame that is 1920 x 1080 x 24 bits/pixel = 49,766,400 bits / 8 bits/byte = 6.220800 MB? Display protocols require the frame transmitted many times per second. Clearly the RP2040 needs another ic in the mix: external RAM = 49.7664Mbits+. With this, the RP2040 could store the entire desired output for the current frame at all times, which frees the RP2040 from 'forgetting' and redrawing sections of each frame, i.e. it only has to deal with pixels that need to be updated and it can write those updates into memory over time, sending the current state of the frame in memory each time the video protocol needs a frame. The best part about an independent memory chip is that the RP2040 doesn't necessarily need the data returned to it (supposing the use case doesn't require acting upon the existing frame, but simply overwriting it). The RP2040 only needs to send the RAM the address of the next batch of pixels, which the RAM can then make available to the HDMI encoder/transmitter between their respective outputs and inputs.
...
[ text clipped -- see the full text via Hackaday.io or Github links above ]
...
So what does everyone think?
Maybe something like this would be of interest to retro gamers wanting to plug their RP2040-based emulator into a TV/monitor. Maybe piping a terminal to modern displays via hardware? As mentioned, PyDOS is interesting. Thinking about a GUI, the most feature complete one for microcontrollers I see out there is FabGL [ github.com/fdivitto/FabGL ] for the ESP32 with no clear option for the RP2040. If you're after 1024 x 768 @ 60 FPS or smaller, 'going bigger' with an Allwinner V3S has RAM in the SIP and outputs RGB888. For digital signage, a low end orange/banana/mango {insert tropical fruit here}/Rock pi/RADXA zero variant would be about the same price and validated. Sometimes it's just fun to explore the limits. I'm putting the board schematics and PCB files out there. If you do use these, please double-check EVERYTHING, as I have not had these built/tested!
---
Become a Channel Member: / @tecist
Patreon: / tec_ist
BTC: 3PVeqcvKoQqKRgGDhHvZJ49AumUhnVSqC9
ETH: 0xe74a8e8e76380d3E2048e25Bd927b7880E608Ac8
tec.ist
/ tec_ist
/ tec_ist - วิทยาศาสตร์และเทคโนโลยี
Finally google news recomended some quality content :)
I wonder how far you could get with some kind of decompression in the DMA / PIO?
If you're upscaling low Res graphics that should get you quite far.
My sense is pretty far. Any kind of GUI (even the one-way microcontroller to RAM to encoder/transmitter proposed) would need a way to translate the application/OS state to which parts of the frame should be overwritten with what data, so there is already some logic working to keep the state within the MCU, despite the end product being too big to hold in the MCU all at once.
Before compression, you could go down from RGB888 to RGB565. With some dithering the perceived color depth loss/banding might not be too bad.
Lower frame rates wouldn't help the size of the 'frame buffer' because it only holds a single frame, *but* might free up some clock cycles to do compression / decompression work. Running on bare metal could eliminate memory cost coming from a runtime/interpreter.
Image data tolerates lossy compression, so compression ratio can really vary with your image quality standards. Supposedly JPEG can be compressed down to 5% (although it must look 'potato' at that extreme).
I'm not sure how computationally intense the compression / decompression would be, nor how much could be offloaded to PIO... hmmm... it would be good to keep everything internal to the RP2040.
To continue (without pixel doubling) via the DVI LUT approach brilliantly demonstrated by Wren6991, you are not only RAM-constrained, but you need more compute power and have little overclock headroom left, which implies a need for multiple RP2040s (there are such projects out there).
What about DPRAM (Dual-Ported RAM)?
Thank you for mentioning DPRAM -- I considered it but didn't mention it in the video. The dual ports would make the read/write timing much simpler. I would have gone with DPRAM if I could locate it in stock at JLCPCB (I was trying to stay within their parts assortment, so the PCB could be fully assembled all in one go).
@@TECIST but they do global parts sourcing. i can probably help out
Dual port RAM is fairly expensive, but that was a decade ago when I checked. It's eliminates the issue with bus contention which is only an issue if the processor runs out of time due to waiting for access. You can also fire an interrupt to update the frame.
SDI to HDMI converters exist. 1 GPIO pin for video and audio.
SDI is very interesting. I hadn't come across it before, but looked into it per your comment. I could see that being used to keep the GPIO pin use low while still being able to bitbang SDI-->HDMI video (where you have a very fast microprocessor / serializer). The headache for the RP2040 is that video bit clocks for higher resolutions are so fast the RP2040 can't be OC'd to keep up with low pin counts, so it needs to parallelize across many pins to have a chance. One pin 1080p video means Gb/sec, which means Ghz bitclock/processor clock, and the highest I've seen the RP2040 going is in the 400Mhz range. Raspberry Pi tried 1Ghz [ www.raspberrypi.com/news/dont-try-this-at-home-overclocking-rp2040-to-1ghz/ ], but TL;DR it was fried after a few benchmark runs.
Yeah, I came to the same conclusion. lately I've been struggling with I2S on an ESP32-S3. "E (7) I2S: i2s_set_pin(295): ws_io_num invalid". So much personal tech disappointment over the last few years.
Why would you want to run at 60fps if the module won't be able to update the screen that fast?
Just because you can? 🙃
Lower frame rates (24/30Hz) just feel too laggy to track cursor movement (I'm thinking of a cursor in applications that are not very demanding, perhaps the PyDOS equivalent of the old MS-DOS text editor).
The HDMI encoder (SII9022ACNU) pixel input clock for 1080p is 148.5Mhz, which is below the clock speed of the SDRAM and still in the 'mild' overclock range for the RP2040. There are time windows to write to RAM during the HDMI encoder horizontal and vertical blanking intervals. While in theory this could apply *some* updates to each frame, my sense is that you're right, updates would regularly need to be written over the course of multiple frames, resulting in image tearing.
Also, just because 😂
@@TECIST or you could just do what most modern GPUs do and use a double buffering setup potentially on two ram chips, but could in theory work on just one, That way your display logic could just happily go about what it's doing while your pi is rendering to the next frame. Or am I misunderstanding something
That could prevent tearing, but at the cost of having to redraw the frame continuously. It is best to think of the RAM here as containing what 'should be on screen' at any given time, rather than single frames. That allows the RP2040 to circumvent the impossibility of keeping a full frame in memory and reduces the redrawing to only changed pixels.
@@TECIST That being said, it would allow for significantly more time to process... Everything. In theory though, you could just dump the contents into another chip, but I also understand at that point we are talking much more complexity, it was just a suggestion to get more "bang" for the time you have.
If DVI possible on a rp2040, could we use a esp32 instead? Look more powerful..
Yes, although the ESP32 would still require an external PSRAM to keep a large frame in memory -- the good news is that the EP32 ecosystem is relatively graphics-friendly: some dev boards have 8MB PSRAM onboard, LCDs (to 7" 800x480) with an integrated ESP32 are available, and LVGL targets ESP32: github.com/lvgl/lvgl
I'd rather see a Raspberry Pi zero as GPU with some interface to the less capable processor running my code. That is to say, I want a display and I don't want to do a lot of work.