Hey all! 👋 Just wanted to leave a comment to make a couple corrections/clarifications: Firstly, as many have pointed out, I mentioned the N64 outputting over composite with "a maximum resolution of 480p", but that's not quite accurate. While the footage in this video was captured at that resolution, N64 games were more typically presented in 240p or 288p, with the highest fidelity titles running at a max of 480i (interlaced). Secondly, when speculating about the future of video super sampling, I suggested that someone could theoretically use the Nvidia DLSS SDK to upscale video signals in realtime. In reality, DLSS uses additional data provided internally on a game-by-game basis to achieve its end result and could not be used as-is to upscale generic footage. However, Nvidia has just recently announced their RTX Video Super Resolution tech which is designed to do exactly that! Incidentally, 2kliksphillip has a new video on this tech, too, which I would highly recommend watching if you're interested in the topic. As always, thanks for watching! 😄
Don't you mean four syllable word not four letter word? Because emulation has more than four letters but it does have four syllables. This made me laugh!
i want to see my dream of cgi to fmv via ai , i was thinking we would have this like 15 years ago , like the ai does see wood and does fill it in real wood with images of wood and so on. so old games would look like real life. also upscale Sierra games , upscale does cause brakes in the lines so colors bleed , ai could fix the lines.
Perfect dark, and a few other games have a 480i mode but need the 8MB expansion pak. The 4MB Jumper Pak sucks dont Use it, as soon as you get the 8MB Pak, destroy that stupid and useless 4MB Jumper Pak, or sell it.
@@LennyQUMFIF Yep very true. Pokémon Stadium 2 is another one like that. Though there are some merits to staying at 240p if you don't like interlaced video.
the reason DLSS works, is that it captures as much information from the game as possible, like motion vectors. Without getting extra information (its impossible to get anything more than just the final image from an old console) something like DLSS is completly impossible, since it needs motion vectors to not become a smudgy mess. So no, u can't just use a software and a video capture to be able to upscale n64 games, because DLSS needs a lot more information than video footage to be able to upscale
Well, something like DLSS 2.0 and up. DLSS 1.0 was entirely framebuffer-based, and yeah, a lot of its output basically looked like Topaz or DALL-E 1 in the absolute worst ways. And yeah, trying to work 2.0 into the pipeline wouldn't really be an option without some kind of motion vector-producing software shim, which would probably have to be built on a per-game basis.
This is not how DLSS works. DLSS uses a lot of information beyond just the rendered video frame to achieve good upscaling results. Most notably, DLSS depends on the motion vectors provided by the game engine. It’s not a simple video upscaler.
In all actuality, the fanmade pc ports of these older titles that have been popping up recently are the best answer. Natively upscales to whatever your pc can handle, and same for the aspect ratio. Sure it's not the most accessible option, but it's simultaneously the most cost efficient and gives you the best results. Not to mention that resolution and aspect ratio aren't the limit when it comes to making the games prettier, plus it allows us to push these versions of the games to other consoles. There's already a proper switch port for the fanmade pc version of OOT for example. Rebuilding the games from the ground up is likely going to be the future for these old titles.
Pretty neat, but I'd definitely take sharp, crisp pixels, or CRT simulation, from a scaler like the RetroTink over this. In its current state it really looks like a smeary mess, and can be quite unstable at times in motion. It does certainly have its impressive moments, but overall I would not want to play on this. Personally I play on a real CRT whenever possible, and firmly believe this to be the best way of viewing these low-res signals, but I understand that won't always be practical for everyone, especially as CRTs get scarcer over the decades. I'm hoping highly-accurate CRT emulation will be an accessible option down the road (or, better yet, my unrealistic dream of someone manufacturing new CRTs with retro gaming in mind).
The N64 had universally enabled anti-aliasing and a blur filter applied to every game. The resolution isn’t that big of an issue for the video quality as can be seen by the PS1 and Saturn, but all the blurring really hurts the image. One can remove the blur using a hardware mod and remove anti-aliasing with game patches. And let me say, playing the games without that stuff enabled is a treat.
3:55 It gave Fox a weird sideways Sonic mouth. I think the AI upscale looks quite bad in most cases, unless the thing It's upscalling is filling most of the screen it just kinda takes a blurry image and makes a smudgy image, where some things look ok but the further back it is the worse of a job it does figuring out what to do with it.
the composite video signal and 480p is not why the N64 looks blurry, the N64 uses 3-point bilinear filtering on it's hardware, you'd literally have to solder new hardware for it to be nearest neighbour.
@@mirabilis it looks truer and it would give a more accurate and truer source for AI image upscaling. Bilinear just makes it look blurry which is why the image upscaling represented on the video doesn't look the best, but klikflilip's Nvidia upscaling (DLSS) looks better. his 240p upscaled footage to 720p looked much more accurate.
I would definitely be interested in some kind of real-time upscaling device like that. Being able to do re-stylization would be an amazing option as well; being able to provide an image as a style input which is then automatically applied to the video output. That would be incredible on older and monochromatic consoles like the Gameboy.
I think using neural networks to remake old games in real time is gonna be possible, in the future. But right now cpus (gpu) are way to slow if you dont want a huge input lag. And that is with current models of NN for upscaling. Imagine like a shader that you put on Resident Evil (ps1) and it outputs graphics similar to the GC remake. The computing power would probably be more than ten times a rtx 4080 needed. But I can see it in the future. Until then, 240p lools best in 240p on a crt.
You could always learn C and refactor the entire cartridge like what that one German guy did with Mario 64. Using efficient C techniques you could not only get the textures to be as crystal clear as the hardware will allow but you could also increase the frame rate and keep it high.
For me personally, I'll just go for the HDMI adapter option (highest quality possible) and not get into supercharging it. That way I can also use the same HDMI adapter for my AV Famicom and Super Famicom systems too (if it works with all AV Nintendo systems anyway). Hopefully I can get one with 4:3 and 16:9 stretch options included, for those 16:9 supported games of course.
If you add the mClassic to any adapter you have it has a 4:3 mode switch right on the sucker and it polishes those edges and fixes up the color a little bit. I use a retrotynk adapter or EON Super 64 and slap the mClassic on the back of it for a very nice picture from my N64.
What they really need to do is create a modern display for retro/composite/S-video that matches pitch and resolution of CRT that is displaying the 240P/480 signal. LCD displays have 1 fixed resolution so if you input a low resolution signal then it gets upscaled and turns into a blurry mess. If the panel was created so the native resolution matched the CRT resolution then perhaps it could look original again but probably not. It is for these reasons that I keep a Sony PVM Trinitron around.
btw the most games on the n64 ran at 256x224 or 320x240 if you downscale your captured footage to the native resolution that game outputs then upscale it will look cleaner and less jaggy.
Maybe someone has already pointed this out, but the N64 also has S-Video which is much better quality as composite, which would help with the AI thingy. I believe you can also get an RGB mod for an N64 as well, so again that would improve the picture even more.
Yes and no, see they are not intended to look bad, the developers did as much as posible to look good, for exaple the wind waker and wind waker for wii u both are what the creator were thinking, the wii u just takes it to the next level but still represents the oficial way and also if back in the day the developers could they just straigth up would had use the renders for promotion
Wow that looks incredible! Better than any other attempts I've seen over the years to improve the graphics. Would need one powerful af FPGA to pack something like that in a Retrotink or OSSC
It gets even more impressive when you start using custom-designed models. I’ve been working on a custom model set for the Super Mario Bros Super Show (models available in the Animation Upscale Discord). The results on the animated portion aren’t as good as going back to the source 16mm film would be, but it’s still very impressive. Live action is taking a bit more work since it was shot directly to tape.
Please get an OSSC or Retrotink for your original hardware presentation, the m cable is only "decent" after the video signal has already been processed by a good scaler like the aforementioned. Also N64 was never 480p, it was 240p with a select few games supporting 480i with the 8mb ram expansion, which is even more reason for you to use a real scaler
i personally use a combination of a rgb modded n64, that passes through a hdmi line doubler (with optional smoother) and it makes the games look quite close to the footage you are showing as well :)
Those games weren't meant to run on those clear image screens. The scan lines and the blurryness makes a huge difference. The more high definition that are implemented on those games, the more ugly they look. Now we can count every pixel and every polygon just by looking at it. In CRT TVs you see less geometric figures in the polygons and less sharpness of the pixels.
I don't mind the low resolution, it's the blur that I don't like. Which I take care of with a Gameshark and codes to disable the blur/anti-aliasing. Also get an fps boost on games with unlocked framerates.
the easiest way of doing it would to build a signal processor that intercepts the signal from the console into a small pc with a gpu and use AMD's FSR to uprender the signal before putting it out to a tv or monitor. the only issue with trying to do that now is that FSR would have to be trained on the full library of N64 games for it to work, or any game for that matter since it does not just upscale any video feed it is given. what we need is a AI that is powerful enough to upscale any footage it is given and do it in real time.
I prefer those old games running in their original blurry resolution as they were designed around it when you scale it up or render it at a higher resolution you reveal all the imperfections that the low resolution originally hid so while the game looks sharper the overall image quality ends up looking worse
I tend to agree. It's even worse when it's a game that has pre-rendered backgrounds because the the rendered models get up-scaled, but the background images stay blurry. At the original resolution the pre-rendered backgrounds blend well with the rendered elements and create a pretty convincing effect, but once they're up-scaled it just looks like a bad green screen. Some games handle the up-scaling better than others though, games with a more cartoon-like aesthetic like Mario 64 don't look as bad when up-scaled.
I upscale all the longplays on my channel! Games like parasite Eve I increase the resolution and fps and use all kinds of other effects! Got an old game Drakerider from 2012 uploading now that’s 12 hours long all upscaled. Takes forever but super big difference!
Your theory about DLSS will not work unless you are doing emulation. DLSS requires motion vectors for the pixels in each frame that are not exported by the hardware. It will not work without those. You could try doing motion vector estimation, but I have been told by others that does not work. I had a similar idea too.
Would frame interpolation be possible with emulators? A lot of early 3D games were programmed specifically to run at 30 fps (or 20 in Zelda's case) so being able to play all of them at 60 fps would be great.
@@TFSned This is a complicated question to answer. On one hand, yes it is very possible to make a game look 'smoother' by introducing additional render frames into a game and thereby make it have an effective higher frame rate. The problem is more related to how the games were programmed. Since many console games were designed with a specific set of criteria since they were expected to only be ran on a specific piece of hardware, many console games take shortcuts on game logic programming, and lock the game's logic to the expected frame rate of the game. So even if you can force the game to interpolate extra frames in between a typical game's 30-60 FPS runtime, the game will still only logically function at 30-60 frame intervals. This can mean many things, but usually all of them lead to the game feeling more jank or unresponsive in spite of the smooth gameplay, or in some cases, can entirely break the game's logic due to heavily relying on frame by frame logic. This is why if you uncap the frame rate on an emulator, it will speed the game up instead of simply just having an uncapped frame rate with the game running at the speed it was before. There are certain games on emulators that can change the frame rate cap thanks to hacks or cheats (Such as being able to play Mario Sunshine on Dolphin at 60 FPS using a cheat code), but those are typically the exception and usually only work because the game's engine was designed to be able to theoretically run at that higher frame rate, and it was likely locked at a lower frame rate because the actual consumer hardware couldn't handle it, or because it introduced more bugs and issues that weren't easy to fix in a timely manner, thereby capping the FPS was the easier answer. In short, yes it is theoretically possible, but the result may end up worsening the experience instead of enhancing it, or may just plain and simple break the game you're playing.
What we need is a proper 2.0 HDMI upscaler box with Scart input that can properly handle scanlines. Many of the upscalers we have available introduce some type of issue.
I would expect AI 10 years into the future to be so completely beyond what we currently have that it's unrecognizable. The progress is happening so fast.
It all depends as of right now AI is not self aware and just like Siri can only be programmed meaning it has a human bias behind how it gives/takes info, if they really did manage to get an ai that was even slightly self aware though I'd say things would change in like a few months lol, it would be writing code , upgrading itself , what an amazing time we live in when these are the things we talk about lol
Glad your touching on the (very large) retro subject! Nowadays it's all framemeister, OSSC and retrotink2x/5x and I would love to see more videos on thoses by you!
There's value in using AI to "upscale" textures. A friend of mine gave me an AI-upscaled version of the PC ports of Dino Crisis and (original) Resident Evil 2, which both looked very crisp. However, I think there's a bigger challenge for the N64, which has always had certain graphic issues when emulating game footage due its source hardware. Either way, it's a cool idea.
While it’s one of more harmless uses for AI, it still wouldn’t be that practical. For people that want to use emulation there’s far better texture packs and upscaling options that get the job done, and for people that want to play on the original hardware, I doubt they would want to play through something that is so heavily changing the “original intent” of the graphics. Plus HDMI and upscalers already exist. Interesting I guess, but like a lot of AI things, it feels like they come up with solutions for problems that already have better solutions.
I would prefer a line doubler (like the retrotink or EON Super 64) paired with the mClassic for a lag free polish on original hardware before I would wish to use an AI picture modification which would just be slightly too far from the original in my opinion. The nostalgia of knowing I am looking at the closest thing to true N64 graphics at the moment (when I'm using a modern TV vs my CRT of course) is kind of essential to my interest in playing the N64.
Kliksphillip mentioned. Love to see his very niche style of videos get some spotlight outside those circles. I wonder if it will be possible in the future to hijack the video output directly from the console in this type of manner. Considering there are already console mods for HDMi converters, maybe.
basically the ai video conversion brings the console video capture to emulation quality, maybe one day in real-time from console capture :D neat vid and idea!
I was theorizing this when I first saw stable diffusion drop a couple years ago. Filter all the games with a prompt to change the art style in real time. Can't wait to be honest
1000000% more of these kinds of videos and discussions. We have seen the gadgets of yesteryear and today, but how we preserve and enjoy our favorite pastimes in the future? That's what I've enjoyed in this bit.
Another great video on an interesting topic. I too am a retro gamer and I love playing my games on original hardware. Topics like this give me hope for retro gaming's future. Thank you.
I found you touched on this before with the N64 combining the Mclassic with the E.O.N Super 64. I use this for my current set up and though it's not perfect it's damn good compared to the original and essentially the MClassic's anti aliasing AI combined with the E.O.N's Smoothing.
I bypassed the C sync on my nintendo 64 to get RGB out and have an everdrive to turn off anti-aliasing. I just wonder how an AI or a DLSS filter would do with a cleaner signal.
There is literally a $99 device that does this all, and works great with retro consoles. Sold at BestBuy, Amazon, and honestly does what you are trying to do without all this extra work
I was emulating some ps1 games, started at 4k but found myself turning it back down to 2x resolution because i only wanted an original look but with a little clean up. Im happy with that
if there's something I wanna see for a HDMI upscaling box, it's probably one that uses FSR to upscale HDMI inputs to higher resolutions, I wonder if anyone is attempting this, because I'm most certainly not the first person to think of the idea, but it seems like such a smart one.
I have an M-Classic and I regulary use it with my Switch. It makes a big difference for me considering my TV is 4K and it doesn't upscale a 720p image very well, but a 1080p image looks just fine.
There is a great video called "enhancing photorealistic enhancement" which uses Ai to make GTA footage look like real life. With the progression of AI image models there is the possibility of style transfer. For example you could feed the AI images of the pre-rendered models of the star fox characters and have an AI use them as a base to enhance the N64 footage. Of course, this requires for models with more temporal coherence and we would need much more powerful hardware, but as you said the point of the video was to look at least a decade into the future.
Easiest nowadays is still use a CRT and if you want to push it get an s-video adapter. Also with a flash cart there are rom hacks that reduce the blurriness.
I’m still looking forward to the edge fill when a movie is full screen native. And the AI fills in the edges with something better than what we have now.
As someone who plays even the PS4 with a downscaler, fuzzy images don't bother me all that much. The highest the TV I'm using only goes to 1080i, and not only my Wii U and PS3 are both plugged into the only available component arrays, my PS4 doesn't seem to like running in 1080i when it boots up. The TV itself is capable of S-video, but I don't know if my PS4 will play nice with that, either.
This is exactly the sort of look I’d like to get on real hardware. I know some folks like that blurry, pixelated and scan line look, but it’s not for me. I would much rather have the cleaned up and crisp look of this video. But how close to that can you get with current external solutions like the retrotink 5x (which I have) and some other smoother like a MClassic (which I also have)? I’ve tried the 5x alone, and it’s certainly better than plugging it in directly…but it still looks pretty blurry and pixelated. I wonder about the other smoother Drew mentioned in this video. Either way, I hope it will be possible sooner than later to get this kind of look on real hardware with minimal to no input lag.
What I would be interested to see is one of these adapters/AIs that is specifically trained on something like low res n64 footage and high res emulator footage in an attempt to have a more specialized dataset for upscaling those old polygonal n64 games.
for original hardware, honestly the best option is soldering a HDMI output directly into the motherboard. not that it's easy or non-time consuming, but currently that's the best option. tho you can't really do much for textures, which is a byproduct of the severely limited texture pipeline the n64 has anyways. if textures are an absolute essential upgrade tho, emulation is the only way currently. lot's of community-made texture packs and mods offer a variety of options, which is nice :) it sucks that decompilation takes so much time, because running the games natively on modern devices would be the best way to preserve and "supercharge" them, but alas that's out of the question unfortunately.
I did buy a plug-in device that upscales my games. These kinds of things aren't new, and advancements are constantly being made to improve them. Just get a RetroTink 5X Pro for more accurate upscaling than the AI did that video
No, you can't just slap DLSS on top of emulator or console outputs. DLSS is not a spatial algorithm, it doesn't just take an image and upscales it. It requires a motion vector buffer, a history buffer and a jittered main buffer to reproject previous frames and achieve reconstruction/multisampling with the help of an AI this way. Frame interpolation in DLSS 3 is temporal as well. FSR 1.0 would be more suitable for this as it is actually spatial. FSR 2.0 is not, it's temporal and it requires similar inputs as with DLSS.
I don't think DLSS frame gen works on the output it works by analysing the input geometry so it would not work for your use case but could work for emulation
I'm personally in the camp that I want to keep the original experience alive. But that's only for video games. For movies I want the highest fidelity so I can understand why people would want games in that way too. I'm sure there are some movie buffs out there that say film projection is the only way to experience a film.
So real time ai upscaling for videos exist on the Nvidia shield and it is pretty good. For retro games, I can't imagine you'll ever get reliably good and stable results without access to a depth buffer, it's a big part of why DLSS works so well and why FSR 1.0 was so bad by comparison
DLSS uses motion vectors from the game code, unless you also had some sort of high-speed code transport from the N64 to the "DLSS device" it would not be as effective at up-scaling. That said, AMD FSR and CAS algorithms do a pretty good job without motion vectors or AI so who knows what the future might bring. AI will certainly get faster and better at this. You may even get the option to replace all graphics with newly generated realistic ones similar to who AI art tools can take a child's crayon drawing and turn it into a realistic painting.
Regardless of OG hardware vs emulation, when AI models fully understand how to remaster a game frame by frame in real time, we could be in store for things like UE5 graphics for 6th, 7th and 8th gen games.
Thing is what you describe as the "original experience" isn't the original experience either. You're playing games specifically made to be displayed on CRT TVs on modern LCD and LED displays.
In principal I don't think any screen-video upscale could ever catch up with enhancements that has access to the whole pipeline, and could use upscaled textures, anti-aliasing, less-lossy interpolation and much more. and in the former case too, an algorithm that was trained on live action and high-poly CGI is not ideal.
Hey all! 👋 Just wanted to leave a comment to make a couple corrections/clarifications:
Firstly, as many have pointed out, I mentioned the N64 outputting over composite with "a maximum resolution of 480p", but that's not quite accurate. While the footage in this video was captured at that resolution, N64 games were more typically presented in 240p or 288p, with the highest fidelity titles running at a max of 480i (interlaced).
Secondly, when speculating about the future of video super sampling, I suggested that someone could theoretically use the Nvidia DLSS SDK to upscale video signals in realtime. In reality, DLSS uses additional data provided internally on a game-by-game basis to achieve its end result and could not be used as-is to upscale generic footage. However, Nvidia has just recently announced their RTX Video Super Resolution tech which is designed to do exactly that! Incidentally, 2kliksphillip has a new video on this tech, too, which I would highly recommend watching if you're interested in the topic.
As always, thanks for watching! 😄
Don't you mean four syllable word not four letter word? Because emulation has more than four letters but it does have four syllables. This made me laugh!
@@KevinBlue18 "Four-letter word" was a bit of a joke, but it's used in American English to refer to any word that is crass/rude/offensive. 😛
i want to see my dream of cgi to fmv via ai , i was thinking we would have this like 15 years ago , like the ai does see wood and does fill it in real wood with images of wood and so on. so old games would look like real life. also upscale Sierra games , upscale does cause brakes in the lines so colors bleed , ai could fix the lines.
i want a software in my PC where i can play old games with bit graphic and just let them get updated by my own dlss !!!!!
Antworten
It's worth noting that the N64 isn't actually 480p. It's usually 240p and has a maximum resolution of 480i.
Perfect dark, and a few other games have a 480i mode but need the 8MB expansion pak.
The 4MB Jumper Pak sucks dont Use it, as soon as you get the 8MB Pak, destroy that stupid and useless 4MB Jumper Pak, or sell it.
@@LennyQUMFIF Yep very true. Pokémon Stadium 2 is another one like that. Though there are some merits to staying at 240p if you don't like interlaced video.
@RaniaIsAwesome Yes that's true as well.
Would this AI technology work well with GBA ports. I would love to see Astrix and Oblesk on the GBA
@@LennyQUMFIF Some games break when there's an 8 MB pak installed
I know people love this but it just looks like melted crayons to me. then again, I need glasses so I could use more pixels in my games.
Nah you’re right. It looks pretty bad. Not surprised though since it’s AI lmao.
Nintendrew is allergic to crts lol
Crayons is on point, on top of wobbly edges like the PlayStation.
AI interpolation needs higher base resolution for good results.
the reason DLSS works, is that it captures as much information from the game as possible, like motion vectors. Without getting extra information (its impossible to get anything more than just the final image from an old console) something like DLSS is completly impossible, since it needs motion vectors to not become a smudgy mess.
So no, u can't just use a software and a video capture to be able to upscale n64 games, because DLSS needs a lot more information than video footage to be able to upscale
Well, something like DLSS 2.0 and up. DLSS 1.0 was entirely framebuffer-based, and yeah, a lot of its output basically looked like Topaz or DALL-E 1 in the absolute worst ways.
And yeah, trying to work 2.0 into the pipeline wouldn't really be an option without some kind of motion vector-producing software shim, which would probably have to be built on a per-game basis.
Always great to sit down at lunch and a new nintendrew episode drops
Absolutely
I just sat down to drop one myself. Post lunch.
Indeed
Lol it’s been like a month tho
Hahaha, my microwave just beeped, - ready to watch this episode.
Hell no, fuck AI. I'll take my upscaler until the day I die.
A reasonable person would think with the thumbnail and title that you actually played it at this resolution, not ran it through topaz. Clickbait.
Totally agree. Just emulate it for resolution. The Nintendo Switch is in hd.
Cool story
@@mlassz009 yeah that's not how you use "cool story"
@@mapsgoonthewall5396 Thanks for the hot tip
This is not how DLSS works. DLSS uses a lot of information beyond just the rendered video frame to achieve good upscaling results. Most notably, DLSS depends on the motion vectors provided by the game engine. It’s not a simple video upscaler.
1:14 ah yes, my favorite 4 letter word: emulation
amazing video..however, the footage from the upscaled version looks like someone dumped a bunch of Vaseline and grease on the n64 footage
I swear, every time you use the Game Boy Camera music it puts me in an amazing mood.
It does 😄 nice touch lol
In all actuality, the fanmade pc ports of these older titles that have been popping up recently are the best answer. Natively upscales to whatever your pc can handle, and same for the aspect ratio. Sure it's not the most accessible option, but it's simultaneously the most cost efficient and gives you the best results. Not to mention that resolution and aspect ratio aren't the limit when it comes to making the games prettier, plus it allows us to push these versions of the games to other consoles. There's already a proper switch port for the fanmade pc version of OOT for example. Rebuilding the games from the ground up is likely going to be the future for these old titles.
I wish there were fanmade native PC ports of Banjo-Kazooie and Rare's other 3D platformers, because as of now, there aren't any.
@@gamestation2690 Boy do I have news for you.
Pretty neat, but I'd definitely take sharp, crisp pixels, or CRT simulation, from a scaler like the RetroTink over this. In its current state it really looks like a smeary mess, and can be quite unstable at times in motion. It does certainly have its impressive moments, but overall I would not want to play on this.
Personally I play on a real CRT whenever possible, and firmly believe this to be the best way of viewing these low-res signals, but I understand that won't always be practical for everyone, especially as CRTs get scarcer over the decades. I'm hoping highly-accurate CRT emulation will be an accessible option down the road (or, better yet, my unrealistic dream of someone manufacturing new CRTs with retro gaming in mind).
I was surprised to see the red device for the switch. I backed that and haven’t heard anyone talk about it yet
The N64 had universally enabled anti-aliasing and a blur filter applied to every game. The resolution isn’t that big of an issue for the video quality as can be seen by the PS1 and Saturn, but all the blurring really hurts the image. One can remove the blur using a hardware mod and remove anti-aliasing with game patches. And let me say, playing the games without that stuff enabled is a treat.
3:30 What a squandered opportunity to use a WaveRace 64’s “MAXIMUM POWER!” sound byte! C’mon Drew! Haha 😂
3:55 It gave Fox a weird sideways Sonic mouth.
I think the AI upscale looks quite bad in most cases, unless the thing It's upscalling is filling most of the screen it just kinda takes a blurry image and makes a smudgy image, where some things look ok but the further back it is the worse of a job it does figuring out what to do with it.
the composite video signal and 480p is not why the N64 looks blurry, the N64 uses 3-point bilinear filtering on it's hardware, you'd literally have to solder new hardware for it to be nearest neighbour.
Ew, that would make it look worse.
@@mirabilis it looks truer and it would give a more accurate and truer source for AI image upscaling. Bilinear just makes it look blurry which is why the image upscaling represented on the video doesn't look the best, but klikflilip's Nvidia upscaling (DLSS) looks better. his 240p upscaled footage to 720p looked much more accurate.
@@xehP Honestly I kind of like how it looks. I typically find N64 games more graphically a pleasing than PS1.
Love the video Drew! Great insight into what’s coming in the near future
I would definitely be interested in some kind of real-time upscaling device like that. Being able to do re-stylization would be an amazing option as well; being able to provide an image as a style input which is then automatically applied to the video output. That would be incredible on older and monochromatic consoles like the Gameboy.
I think using neural networks to remake old games in real time is gonna be possible, in the future. But right now cpus (gpu) are way to slow if you dont want a huge input lag. And that is with current models of NN for upscaling. Imagine like a shader that you put on Resident Evil (ps1) and it outputs graphics similar to the GC remake. The computing power would probably be more than ten times a rtx 4080 needed. But I can see it in the future. Until then, 240p lools best in 240p on a crt.
You could always learn C and refactor the entire cartridge like what that one German guy did with Mario 64. Using efficient C techniques you could not only get the textures to be as crystal clear as the hardware will allow but you could also increase the frame rate and keep it high.
Ah, and here I thought you actually, physically did something to your Nintendo 64 to get that footage.
4 letter word....
For me personally, I'll just go for the HDMI adapter option (highest quality possible) and not get into supercharging it. That way I can also use the same HDMI adapter for my AV Famicom and Super Famicom systems too (if it works with all AV Nintendo systems anyway).
Hopefully I can get one with 4:3 and 16:9 stretch options included, for those 16:9 supported games of course.
If you add the mClassic to any adapter you have it has a 4:3 mode switch right on the sucker and it polishes those edges and fixes up the color a little bit. I use a retrotynk adapter or EON Super 64 and slap the mClassic on the back of it for a very nice picture from my N64.
What they really need to do is create a modern display for retro/composite/S-video that matches pitch and resolution of CRT that is displaying the 240P/480 signal. LCD displays have 1 fixed resolution so if you input a low resolution signal then it gets upscaled and turns into a blurry mess. If the panel was created so the native resolution matched the CRT resolution then perhaps it could look original again but probably not. It is for these reasons that I keep a Sony PVM Trinitron around.
btw the most games on the n64 ran at 256x224 or 320x240 if you downscale your captured footage to the native resolution that game outputs then upscale it will look cleaner and less jaggy.
Maybe someone has already pointed this out, but the N64 also has S-Video which is much better quality as composite, which would help with the AI thingy. I believe you can also get an RGB mod for an N64 as well, so again that would improve the picture even more.
why go through all this trouble, when its cheaper, and easier, to get a CRT, which will always look better...
I want my old games to look like ass because that's how they were intended to look.
fr fr
Yes and no, see they are not intended to look bad, the developers did as much as posible to look good, for exaple the wind waker and wind waker for wii u both are what the creator were thinking, the wii u just takes it to the next level but still represents the oficial way and also if back in the day the developers could they just straigth up would had use the renders for promotion
Wow that looks incredible! Better than any other attempts I've seen over the years to improve the graphics. Would need one powerful af FPGA to pack something like that in a Retrotink or OSSC
It gets even more impressive when you start using custom-designed models. I’ve been working on a custom model set for the Super Mario Bros Super Show (models available in the Animation Upscale Discord). The results on the animated portion aren’t as good as going back to the source 16mm film would be, but it’s still very impressive. Live action is taking a bit more work since it was shot directly to tape.
Where did you get the training data for the Mario show?
@@No_True_Scotsman I got stills from the Shout Factory DVD sets released in the 00's.
Please get an OSSC or Retrotink for your original hardware presentation, the m cable is only "decent" after the video signal has already been processed by a good scaler like the aforementioned. Also N64 was never 480p, it was 240p with a select few games supporting 480i with the 8mb ram expansion, which is even more reason for you to use a real scaler
Thanks for sharing this!
i personally use a combination of a rgb modded n64, that passes through a hdmi line doubler (with optional smoother) and it makes the games look quite close to the footage you are showing as well :)
Oh yeah same here with my Retrotink 5X Pro & set the Phase to Bypass.
Those games weren't meant to run on those clear image screens. The scan lines and the blurryness makes a huge difference. The more high definition that are implemented on those games, the more ugly they look. Now we can count every pixel and every polygon just by looking at it. In CRT TVs you see less geometric figures in the polygons and less sharpness of the pixels.
I don't mind the low resolution, it's the blur that I don't like. Which I take care of with a Gameshark and codes to disable the blur/anti-aliasing. Also get an fps boost on games with unlocked framerates.
the easiest way of doing it would to build a signal processor that intercepts the signal from the console into a small pc with a gpu and use AMD's FSR to uprender the signal before putting it out to a tv or monitor. the only issue with trying to do that now is that FSR would have to be trained on the full library of N64 games for it to work, or any game for that matter since it does not just upscale any video feed it is given. what we need is a AI that is powerful enough to upscale any footage it is given and do it in real time.
I prefer those old games running in their original blurry resolution as they were designed around it when you scale it up or render it at a higher resolution you reveal all the imperfections that the low resolution originally hid so while the game looks sharper the overall image quality ends up looking worse
I tend to agree. It's even worse when it's a game that has pre-rendered backgrounds because the the rendered models get up-scaled, but the background images stay blurry. At the original resolution the pre-rendered backgrounds blend well with the rendered elements and create a pretty convincing effect, but once they're up-scaled it just looks like a bad green screen. Some games handle the up-scaling better than others though, games with a more cartoon-like aesthetic like Mario 64 don't look as bad when up-scaled.
I upscale all the longplays on my channel! Games like parasite Eve I increase the resolution and fps and use all kinds of other effects! Got an old game Drakerider from 2012 uploading now that’s 12 hours long all upscaled. Takes forever but super big difference!
Your theory about DLSS will not work unless you are doing emulation. DLSS requires motion vectors for the pixels in each frame that are not exported by the hardware. It will not work without those. You could try doing motion vector estimation, but I have been told by others that does not work. I had a similar idea too.
Would frame interpolation be possible with emulators? A lot of early 3D games were programmed specifically to run at 30 fps (or 20 in Zelda's case) so being able to play all of them at 60 fps would be great.
@@TFSned This is a complicated question to answer. On one hand, yes it is very possible to make a game look 'smoother' by introducing additional render frames into a game and thereby make it have an effective higher frame rate. The problem is more related to how the games were programmed. Since many console games were designed with a specific set of criteria since they were expected to only be ran on a specific piece of hardware, many console games take shortcuts on game logic programming, and lock the game's logic to the expected frame rate of the game. So even if you can force the game to interpolate extra frames in between a typical game's 30-60 FPS runtime, the game will still only logically function at 30-60 frame intervals. This can mean many things, but usually all of them lead to the game feeling more jank or unresponsive in spite of the smooth gameplay, or in some cases, can entirely break the game's logic due to heavily relying on frame by frame logic. This is why if you uncap the frame rate on an emulator, it will speed the game up instead of simply just having an uncapped frame rate with the game running at the speed it was before. There are certain games on emulators that can change the frame rate cap thanks to hacks or cheats (Such as being able to play Mario Sunshine on Dolphin at 60 FPS using a cheat code), but those are typically the exception and usually only work because the game's engine was designed to be able to theoretically run at that higher frame rate, and it was likely locked at a lower frame rate because the actual consumer hardware couldn't handle it, or because it introduced more bugs and issues that weren't easy to fix in a timely manner, thereby capping the FPS was the easier answer.
In short, yes it is theoretically possible, but the result may end up worsening the experience instead of enhancing it, or may just plain and simple break the game you're playing.
What we need is a proper 2.0 HDMI upscaler box with Scart input that can properly handle scanlines. Many of the upscalers we have available introduce some type of issue.
Fascinating stuff! Thank you for painting the future as a bright one for retro gamers!
I would expect AI 10 years into the future to be so completely beyond what we currently have that it's unrecognizable. The progress is happening so fast.
Yep and sadly be used for evil (you're already seeing examples), humans truly cant have nice things without corrupting it somehow.
10 years?
Try 3-4
It all depends as of right now AI is not self aware and just like Siri can only be programmed meaning it has a human bias behind how it gives/takes info, if they really did manage to get an ai that was even slightly self aware though I'd say things would change in like a few months lol, it would be writing code , upgrading itself , what an amazing time we live in when these are the things we talk about lol
Glad your touching on the (very large) retro subject! Nowadays it's all framemeister, OSSC and retrotink2x/5x and I would love to see more videos on thoses by you!
There's value in using AI to "upscale" textures. A friend of mine gave me an AI-upscaled version of the PC ports of Dino Crisis and (original) Resident Evil 2, which both looked very crisp. However, I think there's a bigger challenge for the N64, which has always had certain graphic issues when emulating game footage due its source hardware. Either way, it's a cool idea.
I'd rather play my N64 with low graphics and not have it look like an ai oil-painting
While it’s one of more harmless uses for AI, it still wouldn’t be that practical. For people that want to use emulation there’s far better texture packs and upscaling options that get the job done, and for people that want to play on the original hardware, I doubt they would want to play through something that is so heavily changing the “original intent” of the graphics. Plus HDMI and upscalers already exist. Interesting I guess, but like a lot of AI things, it feels like they come up with solutions for problems that already have better solutions.
I would prefer a line doubler (like the retrotink or EON Super 64) paired with the mClassic for a lag free polish on original hardware before I would wish to use an AI picture modification which would just be slightly too far from the original in my opinion. The nostalgia of knowing I am looking at the closest thing to true N64 graphics at the moment (when I'm using a modern TV vs my CRT of course) is kind of essential to my interest in playing the N64.
Great video man!
it's even funnier when you take into account that the emulators themselves already offer the option to increase the internal render resolution
Kliksphillip mentioned. Love to see his very niche style of videos get some spotlight outside those circles.
I wonder if it will be possible in the future to hijack the video output directly from the console in this type of manner. Considering there are already console mods for HDMi converters, maybe.
Yes New Video! I hope it's good!
Another killer video bro! This is the future of retro gaming right here. Wow!!
basically the ai video conversion brings the console video capture to emulation quality, maybe one day in real-time from console capture :D neat vid and idea!
I was theorizing this when I first saw stable diffusion drop a couple years ago. Filter all the games with a prompt to change the art style in real time. Can't wait to be honest
That's awesome thank you
1000000% more of these kinds of videos and discussions. We have seen the gadgets of yesteryear and today, but how we preserve and enjoy our favorite pastimes in the future? That's what I've enjoyed in this bit.
Bro having emu quality graphics on OG 64 hardware is amazing that's always been goals for me.
Another great video on an interesting topic. I too am a retro gamer and I love playing my games on original hardware. Topics like this give me hope for retro gaming's future. Thank you.
This was very interesting, and the technology could be used to revitalize vintage gameplay! I'd like to experience this myself.
I found you touched on this before with the N64 combining the Mclassic with the E.O.N Super 64. I use this for my current set up and though it's not perfect it's damn good compared to the original and essentially the MClassic's anti aliasing AI combined with the E.O.N's Smoothing.
I bypassed the C sync on my nintendo 64 to get RGB out and have an everdrive to turn off anti-aliasing. I just wonder how an AI or a DLSS filter would do with a cleaner signal.
There is literally a $99 device that does this all, and works great with retro consoles. Sold at BestBuy, Amazon, and honestly does what you are trying to do without all this extra work
I was emulating some ps1 games, started at 4k but found myself turning it back down to 2x resolution because i only wanted an original look but with a little clean up. Im happy with that
On you go Nintendrew, I have no doubt what you have said is correct for the future - but have to press doubt on you doing it yourself.
Love the game room and this video rocks! Thanks for sharing!
if there's something I wanna see for a HDMI upscaling box, it's probably one that uses FSR to upscale HDMI inputs to higher resolutions, I wonder if anyone is attempting this, because I'm most certainly not the first person to think of the idea, but it seems like such a smart one.
A cheap solution for up-scaling video console footage sounds great.
I don't think I'll ever touch an analogue to digital converter or upscaler. Either emulate in HD or grab a CRT from grandmas house
They look really good on crt
No! DLSS works completely different! It has a lot more to work with than only an output picture signal. It has 3D information etc.
I have an M-Classic and I regulary use it with my Switch. It makes a big difference for me considering my TV is 4K and it doesn't upscale a 720p image very well, but a 1080p image looks just fine.
Always nice to see another nintendrew video
There is a great video called "enhancing photorealistic enhancement" which uses Ai to make GTA footage look like real life. With the progression of AI image models there is the possibility of style transfer. For example you could feed the AI images of the pre-rendered models of the star fox characters and have an AI use them as a base to enhance the N64 footage. Of course, this requires for models with more temporal coherence and we would need much more powerful hardware, but as you said the point of the video was to look at least a decade into the future.
Easiest nowadays is still use a CRT and if you want to push it get an s-video adapter. Also with a flash cart there are rom hacks that reduce the blurriness.
i've thought about this a lot for a long time now. analog AI chips trained to upscale retro games. gib now
I’m still looking forward to the edge fill when a movie is full screen native. And the AI fills in the edges with something better than what we have now.
As someone who plays even the PS4 with a downscaler, fuzzy images don't bother me all that much. The highest the TV I'm using only goes to 1080i, and not only my Wii U and PS3 are both plugged into the only available component arrays, my PS4 doesn't seem to like running in 1080i when it boots up. The TV itself is capable of S-video, but I don't know if my PS4 will play nice with that, either.
This is exactly the sort of look I’d like to get on real hardware. I know some folks like that blurry, pixelated and scan line look, but it’s not for me. I would much rather have the cleaned up and crisp look of this video. But how close to that can you get with current external solutions like the retrotink 5x (which I have) and some other smoother like a MClassic (which I also have)? I’ve tried the 5x alone, and it’s certainly better than plugging it in directly…but it still looks pretty blurry and pixelated. I wonder about the other smoother Drew mentioned in this video. Either way, I hope it will be possible sooner than later to get this kind of look on real hardware with minimal to no input lag.
What I would be interested to see is one of these adapters/AIs that is specifically trained on something like low res n64 footage and high res emulator footage in an attempt to have a more specialized dataset for upscaling those old polygonal n64 games.
i would just love to use this tech on my DVD collection! 480p to 4k would be incredible.
for original hardware, honestly the best option is soldering a HDMI output directly into the motherboard. not that it's easy or non-time consuming, but currently that's the best option. tho you can't really do much for textures, which is a byproduct of the severely limited texture pipeline the n64 has anyways.
if textures are an absolute essential upgrade tho, emulation is the only way currently. lot's of community-made texture packs and mods offer a variety of options, which is nice :)
it sucks that decompilation takes so much time, because running the games natively on modern devices would be the best way to preserve and "supercharge" them, but alas that's out of the question unfortunately.
No 480p on the N64, actually. Just 240p or 480i.
I did buy a plug-in device that upscales my games. These kinds of things aren't new, and advancements are constantly being made to improve them. Just get a RetroTink 5X Pro for more accurate upscaling than the AI did that video
It's an interesting time to watch this with the HDMI solutions that are about now.
No, you can't just slap DLSS on top of emulator or console outputs. DLSS is not a spatial algorithm, it doesn't just take an image and upscales it. It requires a motion vector buffer, a history buffer and a jittered main buffer to reproject previous frames and achieve reconstruction/multisampling with the help of an AI this way. Frame interpolation in DLSS 3 is temporal as well.
FSR 1.0 would be more suitable for this as it is actually spatial. FSR 2.0 is not, it's temporal and it requires similar inputs as with DLSS.
I don't think DLSS frame gen works on the output it works by analysing the input geometry so it would not work for your use case but could work for emulation
Retrotink family of products are the answer to your question
So basically you captured some in game footage and upscaled it. 😂 wow so cutting edge
Do you know something new about the EON Super 64 Adapter? It's sold out everywhere for a long time now😩 and I can't get it anymore
I'm personally in the camp that I want to keep the original experience alive. But that's only for video games. For movies I want the highest fidelity so I can understand why people would want games in that way too. I'm sure there are some movie buffs out there that say film projection is the only way to experience a film.
So real time ai upscaling for videos exist on the Nvidia shield and it is pretty good. For retro games, I can't imagine you'll ever get reliably good and stable results without access to a depth buffer, it's a big part of why DLSS works so well and why FSR 1.0 was so bad by comparison
1:16 It's not 4 letter word, it's a 4 syllable word! :) Great video!
Was looking for this
Imagine if emulation could render each frame as a.i upscaling, it would suck but be cool!
DLSS uses motion vectors from the game code, unless you also had some sort of high-speed code transport from the N64 to the "DLSS device" it would not be as effective at up-scaling. That said, AMD FSR and CAS algorithms do a pretty good job without motion vectors or AI so who knows what the future might bring. AI will certainly get faster and better at this. You may even get the option to replace all graphics with newly generated realistic ones similar to who AI art tools can take a child's crayon drawing and turn it into a realistic painting.
Ai in the future of retrogames. Nooooppeeeeee please
Regardless of OG hardware vs emulation, when AI models fully understand how to remaster a game frame by frame in real time, we could be in store for things like UE5 graphics for 6th, 7th and 8th gen games.
I like how the N64 need something like this for its low res textures but we see the ps1 being imitated with modern games even today.
What modern games are imitating PS1? Is there like an indie sub-genre of games made to look like PS1 games?
Thing is what you describe as the "original experience" isn't the original experience either. You're playing games specifically made to be displayed on CRT TVs on modern LCD and LED displays.
it definitely sounds possible i can see the end goal already to play filtered games with a little intelligence inbetween
In principal I don't think any screen-video upscale could ever catch up with enhancements that has access to the whole pipeline, and could use upscaled textures, anti-aliasing, less-lossy interpolation and much more. and in the former case too, an algorithm that was trained on live action and high-poly CGI is not ideal.
I’d love to see an adapter that decompresses the sound.