วีดีโอ

It's on Linkedin, eventually on arXiv. YT is not letting me post more, not sure why

it was cheaper, but kind of slow

Or that I really dislike editing my own audio. AI voice generation took hours rather than days. That is on top of researching, writing the script, and creating the visuals, which took several weeks. Then following that up with tedious spectrogram work is quite an unpleasant experience. The primary content is not the audio, it is only one component to the medium.

Thanks for the feedback! Could you give an example where compute outstripped memory? The only cases I can think of were marketing (i.e. less VRAM was chosen to save cost - which is not a technology / architecture limitation). I disagree with transistors being a good metric, that's similar to comparing software by lines of code. Transistors are used for the compute, but also on-chip memory, data routing (busses), clock distribution, miscellaneous controllers, and I/O buffers. What you really want to use are operations/second, which for fixed function GPUs would be fill-rate. Comparing clock speed and fill-rate gives you an indication of where the performance came from. If fill-rate grows faster tan clock-speed, then the performance comes from scaling horizontally, whereas the contrary is from pipelining or a technology shrink. Bandwidth (memory) does still play a role there, but it's impossible to unlink the two in this domain as it forms an integral part to the processing pipeline. Also note that none of the memory claims (except for the PS2) account for DRAM overhead, which will necessary result in degraded performance compared to the ideal (peak) numbers.

@RTLEngineering sure, transistor count isn't great either, but that doesn't mean clock speed is a good indicator. Current day gpus are WAY more than 15x faster than the PS2 gpu in terms of compute. Regardless, memory and compute are as important as each other, one isn't the main contributor vs. the other. And by compute out stripping memory, I mean memory bandwidth becoming the bottleneck. The geforce 256 was notoriously limited by its memory bandwidth, and they later released the geforce 256 ddr to unlock it's potential. It's simply a matter of balance and bottlenecks. You could possibly chart FLOPs vs memory speed, idk, but anything is better than hz.

Then I guess you're in agreement with me / the videos. The entire premise was that bandwidth was the driving factor for performance, not memory capacity or clock-speed. Every GPU that I am aware of has been limited by the bandwidth in one way or another, the Geforce 256 being no exception. And the Geforce 256 DDR was still limited by the memory bandwidth. Unfortunately we can't plot FLOPs, because most older GPUs didn't execute floating-point operations. Similarly, when considering modern GPUs, FLOPs is not a great metric for render performance since large portion of the pipeline are still fixed function. So fill-rate remains the better metric, which serves as a proxy for "FLOPs". That's also what I used in the videos, not clock speed - the clock speed was shown to indicate that it was not the major contributing factor. Also, what you were describing (FLOPs vs Memory Bandwidth) is called a Roofline, which is the standard method for comparing performance of different architectures and workloads.

@@RTLEngineering My issue is that I'm making a hard distinction between logic units and memory bandwidth, where as I think you've explicitly shown that they are deeply coupled, proving that the line is effectively a lot blurrier than I previously understood. I'm just smoothbrained from all the hardware reviews making hard distinctions between the two. Thanks for your detailed replies!

@@RTLEngineering one word counterarguement: RDNA2

Your engagement by leaving a comment technically is a reward. Luckily, if you spend a few seconds thinking about it or reading the other comments, you will see that your concern is unjustified (i.e. no work was stolen). If you need a hint, this video was posted almost two years ago (pre AI craze), meaning a human would have had to write the script. The AI voice was chosen to save production time on my part, and I did take care to make sure all of the pronunciations were correct. The only issue was "Id" in "Id Software", which is said twice in a 20 minute video. Regardless, you're free to dislike the video and not watch it due to the voice over, but claiming plagiarism is uncalled for!

Probably because the association was for AI/ML workloads which work with tensors (matrices are a special case of the more general tensor object). Though I am not sure why "Tensor Core" was chosen as the name since other AI/ML architectures call them "Matrix Cores" or "MxM Cores" (for GEMM). It might just be a result of marketing. I would say "MFU" or "Matrix Function Unit" would be the most descriptive term, but that doesn't sound as catchy.

None. There's a small cache that's controlled by the hardware (to cover bursting), but otherwise the z-buffer and frame buffer are stored in the shared system memory. The DMEM on the RCP can't be used for z-buffer or color directly. It can be used for it indirectly, but you're going to end up copying stuff in and out of main memory which will perform worse than not using it at all. Alternatively, it's possible to program a software renderer using SIMD on the RCP, but it would leave the RDP idle.

you can do microcode changes directly, maybe a true hardware z-buffer, using the DMEM/IMEM 4kb caches@@RTLEngineering

maybe TMEM could be partially used as local z-buffer cache, while other part is used as normal texture memory@@RTLEngineering

That's what I meant by "software render using SIMD". There's no read/write path between the DMEM and IMEM, nor is there a read/write path between the DMEM and the fixed-function RDP path. All communication between them would need to be done using DMA over the main system bus. Regarding TMEM, it's the same. There's no direct write path, where you can only write to the TMEM using DMA. Worse yet, the DMACs in all cases required that one address be in main memory, so you couldn't DMA between the memories without first going through the main memory.

What is meant with "1 frame of latency" comes down to the fact that all triangles must be submitted before rendering can begin - at least with the older GPUs. The new PVR archs (especially those used in the Apple SoCs) can reload previously rendered tiles, but the GPU used in the Dreamcast had no method to load the tile buffer from VRAM. So in practice, you want to pipeline the entire process, which gives you that extra frame of latency with IMRs don't require (since the render tiles can be revisited there). Submit -> Vertex Transform -> Bin -> Render -> Display. While you could do all of those in a single frame, that necessarily reduces the total amount of work you can do, else you will have to re-render the previous frame (introducing latency). For IMR, you don't need the Bin stage, and can instead interleave them, meaning you have... TBDR: |Submit -> Vertex Transform -> Bin ->| Render ->| Display|. (3 frames) IMR: |Submit -> Vertex Transform -> Render ->| Display|. (2 frames) Note that the Dreamcast was specifically modified to reduce this latency under certain scenarios, in which the tiles can be rendered in scanline order meaning that the next frame can start to be displayed while the pixel visibility was being computed and then shaded. Dreamcast: |Submit -> Vertex Transform -> Bin ->| Render -> Display|. (2 frames)

@@RTLEngineering If the Dreamcast couldn't re-load the tile buffer from VRAM then I don't know how that would be an issue unless the game was trying to use the rendered image as a texture. Outside of that, what gets rendered to the tile-buffer and then out to VRAM is the finished tile for the frame. It only needs to be read by the display controller and sent to the TV. It can still read from it's tile list in the same frame. |Submit -> Vertex Transform -> Bin ->| Render ->| Display| |Submit -> Vertex Transform -> Bin ->| Render -> Display| What you're saying about the Render and Display steps makes complete sense to me. It's the separation between Bin and Render that makes none. It's not reading from the tile-buffer here. The tile buffer is at the end of the pipeline. The Bin -> Render stage is when the tile list is being pulled into the GPU from VRAM to be rendered. There's nothing that would necessitate waiting for the next frame deadline for this to happen. If the GPU can't read the tile buffer from VRAM then that wouldn't cause an issue because the tile buffer isn't the tile list/parameter buffer which is all that's needed to be read in that stage. The tile list can obviously be read VRAM because that's where they're stored. If it couldn't then the GPU wouldn't work at all. I could understand it if you're looking at a example where the last triangle is submitted close to the deadline though. The IMR will have already completed rendering of almost all previous geometry and only need to finish that up. In that same case, yes, the TBDR will not complete rendering before that deadline because it was waiting for the last triangle to start rendering. But by saying that these two stages always happen in different frames would be incorrect. For example, if you're just rendering a menu on the Dreamcast then the amount of submitted geometry would be so little that it could be in counted by hand. The CPU computation and geometry submission could take, lets say, half of a millisecond. The transform and binning stage would take less than that. At that point it's not going to wait for the next 16ms before it starts rasterizing and texturing those triangles though. It's just going to start reading the tile list right after it's done with binning and it will finish rendering far before the next frame deadline and there will be no frames of latency.

The issue is that every triangle for a frame must be binned before rendering can begin. So if you have 4M triangles in a frame, you must first submit, transform, and bin all 4M triangles before the first render tile can be touched. If you start rendering a tile before binning is complete, then you may finish visibility testing and rendering before all of the triangles are known for that tile - that would result in dropping triangles over the screen randomly based on submission order. This is a hard deadline which is not necessary for IMR - IMR can accept new triangles until a few cycles before the new frame must be presented to the display. Even the IMR architectures that do a type of render tile binning do so on a rolling submission basis because they can return to a previously rendered tile. The scenario you described is correct, in that case you have less work to do and therefore the deadline isn't as tight. But in general, a game developer wants to submit as many triangles with as many textures, with as many effects as possible, per frame. If you combine Submit->Vertex->Bin->Render, into a single frame, and target 60 fps, then that 16ms must be divided into the two phases: Submit->Vertex->Bin, and Render. So if Submit->Vertex->bin takes 10ms, then you only have 6ms to render all of the tiles (480p would be 300 tiles, so 20us per tile), which limits the total triangles per frame. Also keep in mind that Submit->Vertex is done on the CPU (for the Dreamcast) and is interleaved in the game logic itself, so that's going to take longer than if all it were doing is pulling from a preset list in RAM. Binning is done on the GPU, but only handles 1 triangle at a time, so that will be slow if there are too many as well. (It's a write-amplification task, meaning it can be done in bounded but not constant time). Regardless, if you take that approach during a game, you're likely going to drop every other frame to catch up with rendering. The alternative is to render the tiles as you display them, but that would mean that all 20 horizontal tiles need to be rendered within 1/15 of a frame, or 53us each. If the row of tiles is not complete by the time they need to be displayed, then you again need to drop the frame or accept screen tearing. While that same number is also true for the entire screen at once, you have 300 tiles to balance out the load rather than relying on 20 (you're more likely to have some tiles that take 2ms and some that take 2us in a pool of 300 than 20). In both cases, if you drop a frame, then you get 1 extra frame of latency. And besides, in your menu example, 1 extra frame of latency is not important... you should be thinking about the cases in which both latency and performance matters.

@@RTLEngineering I think you're misunderstanding where I'm disagreeing with you. I'm not disputing that submitting and binning all the triangles for the scene before rendering is a hard requirement on a TBDR or that "chasing the beam" with tile-render order would be required to get the most work done before the frame deadline. Where my issue lies is with saying that the 1 frame of additional latency is a rule that's built into how the hardware works when it isn't. That's the reason why I mentioned the menu example. It's not representative of the workload of a full 3D game scene, sure, but it demonstrates a real scenario that's not uncommon on the Dreamcast or a PC where the GPU would be needlessly wasting time and adding latency if it were really a hard requirement for the GPU to do it's vertex stage and fragment shading across two different frames. That's not how any GPU designer would design their GPUs and that's not Imagination designed their's. You could say that a frame of latency is a side effect of the architecture when triangles get submitted too close to the deadline and you could even say that that's common but explaining it as if the hardware literally can't avoid the latency in any scenario and is an absolute requirement of the hardware... is wrong. If the hardware has enough time before the frame deadline to finish rendering after it's done binning (like in the menu example or even in the case of a 2D game) then it will do that and it won't have the latency. This is an architectural video so it should describe the architecture. If there's a realistic limitation to that architecture when in use then that should be mentioned, too, but shouldn't be phrased like that limitation is built into the architecture. I mean you said it yourself. If you target 60 fps then the 16.6ms then to render the frame. That could be 10ms for Submit->Vertex->Bin with 6ms for ->Render->Display... but you CAN do that. It's not a hard limitation. Also keep in mind that workloads in a game aren't constant, they vary. If that's how one frame works out then the next frame could be the inverse of that, 6ms for Submit->Vertex->Bin with 10ms for Rending->Display and that assumes that the CPU waiting until the deadline of frame 1 before it started frame 2. If it started the vertex stage for frame 2 right after it was done with the vertex stage of frame 1 then it would be ready to render to start rendering frame 2 around the time of the deadline for frame 1. Sure, frame times aren't often that erratic, you can argue that the scenario I just mentioned isn't common, and you could say that the hardware would be underutilized in that scenario but the hardware IS capable of it. Lastly, any game that hits a stable 60fps likely isn't just hitting it's deadline, it's done way before it and is just capped at 60fps. The same is also true for 30 fps games. Without a cap, they could run at 35 or 40 but they just cap it at 30fps. That means they have 33.33ms between frames but they'll often be finished rendering in 22-28.5ms.

Sure, although I don't think I ever claimed it was fundamental hardware constraint. That would be entirely wrong as the hardware had no interlocks as far as I am aware of - you could have it display a partially drawn tile if you hit the pointer-flip at the right time. Practically, I gave two examples in my previous response in which there would be no extra frame of delay, and mentioned the limitations of doing so. Typically software running on a TBDR do introduce a second frame of delay in how the software controls the GPU though, for those very reasons. It's also a lot simpler to write the game code to account for that delay than to dynamically adjust to it. Note that even with IMR, you don't need to have any frame delay either - you could just render directly to the display buffer (the PS2 did this in some cases), in which case you would be submitting triangles to the frame as it was being rendered. The Nintendo DS was actually notorious for this as that was the only way to draw the triangles (chasing the beam). Regardless that's arguing more semantics than anything since it's more complicated to say that the extra frame delay was introduced by software, as a result of the TBDR architecture's hard bin deadline requirement (a requirement not imposed by IMR), but could be overwritten in cases where the deadline is more relaxed or visual artifacts were tolerable. Some simplifications need to be made for an architecture video / lecture, as it's not reasonable to list all of the nuances. For example, you could use the PVR2 to compute N-body and fluid simulations instead of drawing triangles, same thing with the PS2's GPU (as a hint, you would do so with blending modes). Drawing 3D graphics is not inherent to the architecture itself, but it's the common / primary use case. So the video should discuss the common case where the rest is left as an exercise to the viewer. I disagree with your last comment about 60fps. You could easily write a game that continually just barely hits the 60fps cap as the GPU has two limits - visibility and shading. So you could have more than enough room in visibility, but be compute limited by the shading engine where a poorly ordered texture cache miss causes you to miss the deadline (this is what happened when drawing 2D sprites). The same thing happens can happen in modern 3D GPUs, but didn't usually occur in the older 3D ones like the Voodoo since the rasterizer was tied to the shading pipeline.

Thanks for pointing that out!

Perhaps the limits are more relative? You can always sacrifice speed - if you can fit a RISCV, you could run a software emulator.

@@RTLEngineering that is a most excellent observation. 😌