'Upscaling' framerates in games - Using RIFE to prepare for DLSS 3
ฝัง
- เผยแพร่เมื่อ 30 พ.ค. 2024
- I don't have access to DLSS 3, but now's an excellent time to convert some videos using RIFE's AI and to see what sort of bugs come with framerate conversion- which DLSS 3 will have to somehow combat / mitigate.
0:00 - Framerate conversion
0:29 - Latency?!
2:19 - 5 to 60 FPS
4:38 - Benefits of higher starting real FPS
6:16 - 15 to 60 FPS
7:31 - The magical 24 FPS
8:00 - 30 to 60 FPS
9:43 - 60 to 120 FPS?
10:38 - Conclusion - เกม
Rendering technology is so wild. It's hard to believe how just 30 years ago we had games at a resolution of 160x200 and now computers are just inventing frames whole cloth.
30 years is an eternity in modern technology terms.
Hell 30 years is a significant portion of a human lifetime.
Man insane how we went from the pinhole camera back in 1021 to what we have today in 2022 absolutely incredible
A Pentium from 30 years ago would have produced 10+ quadrillions of cycles by now.
The same amount of cycles today's 8 core cpu can do under 30 mins. That's how quick technology has improved and evolved.
30 years ago was 1992 and we had SNES with much higher resolution and only 4 years before N64 and SM64
As far as I'm aware only pre-8 bit consoles and handhelds ran at resolutions that low. Both the NES and SNES ran at 256x244, although for NTSC regions the image was actually rendered at 256x224. The N64 was capable of 480p image, although the vast majority of games used the 320x240 mode. The original GameBoy and Gameboy Colour both rendered at 160x144. Much older video formats also rendered at 144p.
I mean, yes, the fact that the Atari 2600 had pixels you could count on screen and the PS5 and current PCs are rendering at 3840x2160 (which was always a pet peeve of mine, 4K suggests a 4 times larger resolution but if 1080p is 1920x1080, then 4K is actually only double the resolution, the 4K coming from the horizontal axis and not the vertical axis but that's neither here nor there) but also time moves forward.
I dont know if you tried this already, but how does it look if you extract all the AI generated frames from the 30->60fps video and play them back without any real frames (and then ai generate again based on that). Might be a cool experiment to see how often you can generate until it looks bad or non-playable.
I really like this idea! It wouldn't be necessarily in context but it'd be interesting to watch at the very least
Cool idea
this has 3klik's written all over it, love it
Brilliant idea! Would make a perfect 3kliks video.
3KLIKS, 2KLIKS, KLIKS, ANYONE
One thing to note about 24fps movies are that each frame is filled with motion blur which helps our head basically create detail from what is essentially a blurry mess.
A frame from a video game capped at 24fps would be essentially moments frozen in time with no motion blur, thus looking less smooth.
In fact, shutter angles are a consideration for DPs to vary the length of the motion blur depending on whether it's an action scene (making it look almost superhuman) or to mimic intoxication or being drugged by making the blur extra long.
usually shutter speed is double the fps you capture at so you get smooth more natural realistic motion blur
The rave scene in Blade being a good example of no blur, if I remember correctly
Once again Philip is explaing this technology better than anybody else.
Somehow this CSGO TH-camr is my go to guy when telling people about AI technology.
Great video man. Really shows you're willing to go the extra mile with the effort you put into this stuff. 💙
I enjoy his videos, but Digital foundry is my go to if you want in depth analysis.
@@lolroflmaoization sure, I'd agree with some stuff. But with this very example of DLSS 3 digital foundry seems very restricted by Nvidia about what they can show. And didn't dedicate their video to exploring AI frame generation in its entirety, just showed some percentage increases when using DLSS 2 and DLSS 3 in like 2 Nvidia approved games and scenarios.
So at least for now, I think Philip has them beat in terms of a full comprehensive look at this technology. So hats off to him I say.
you must be thinking of 3kliksphillip. different guy. brother. father?
@@WhizXGames haha true, the CSGO mapper, the AI expert and the jelly baby eater. A true family.
@@eniff2925 ultimately DLSS 3 isn't out yet. So this video isn't trying to be a deep dive and review of DLSS 3 and if you should turn it on or not. It's meant as an overview of what frame interpolation is, how it works, the limitations of the technology and what artifacts to look for, what frame rates the technology works better at and the fact that you see much better results when interpolating smoother looking motion to begin with (such as 24fps+) and explained why with simple and easy to understand graphs about the time between each real frame and AI generated frame and that tells us about what kind of use cases DLSS 3 is going to be best at (60fps to begin with and up)
Like why so negative? This video is great for just expanding on a basic understanding of the technology and without early access from Nvidia is the best we can hope to do, it covers many things that weren't mentioned in digital founday's video.
And you're negative towards DLSS as well saying it's a huge sacrifice in either latency or resolution. Whilst ignoring the benefits which is much smoother motion to drive monitors like 240Hz and 360Hz and 480Hz at 4k. Or allow higher end demand rendering like ray tracing to become smooth looking.
The tech isn't out yet, we don't know how good it is. But even if you take digital founday's numbers on latency at their worst and say that latency is the same as native when you have reflex on and are rendering at 1/4th the resolution. Or a little higher when rendering at native res with frame generation and reflex on presumably. Is that really a massive deal? It's milliseconds more latency for a much much smoother image to drive these new 240Hz+ monitors. If you only focus on the downsides of a few ms higher latency it sounds terrible, but you're just ignoring the positives. Which doesn't really matter at the moment since neither of us can make a good conclusion about the tech yet. But it seems you are instinctively negative and bias against it for whatever reason, a sentiment I've seen a lot of due to the anger with Nvidia over priced and what not.
Just give the tech a chance, stop hating on a video for not being an in-depth look at DLSS 3's specific implementation of frame interpolation before the tech is even out, and stop being so negative. 👍
The biggest difference is that dlss has access to color, normal, depth and motion vectors straight from the engine.
I don't see why DLSS wouldn't have access to the vertex data too
Just wanted to say that I really appreciated the random Oblivion OST towards the end of the video, absolutely wonderful
Same taught. I had to check if anyone else noticed.
@@gsedej_MB same
One thing to keep in mind is that rife can see all the frames while nvidia can only see two, the present frame and the next one.
Edit: IM WRONG rife only uses two frames despite having access to all of them. Thanks Quaterions
@@dan_loeb not only nvidia others upscale too. It's not special thing tô nvidia
Unfortunately RIFE only uses two frames regardless of the fact it has access to all of them...
@@baka2907 What does this have to do with anything? This is not upscaling.
@@baka2907 ???
RIFE does not use more than 2 frames.
Impressive stuff tho, I still remember when fading between frames to increase framerate was the state of the art
My old copy of Sony Vegas sure likes to remind me that resampling framerates was the future, when it makes even my hard cuts out of black blurry.
I've been pretty surprised that MPV seems to do that, while the Smooth Motion option in MPC-HC (video players) doesn't seem to do anything at all.
I can't believe that every single player in existence shouldn't be able to do better real time interpolation on PC and a 5600X and 3060Ti than my 10yo TV on some potato chip.
RIFE 4.0 via Flowframes is great and shockingly fast, compared to my previous experience 2 years ago with DAIN 0.36 and a GTX 960, but I'd still love to know about something that replicates that standard TV feature.
I'm pretty sure the motion vectors in use will make a massive difference. The motion vectors are 3D, so it knows whether things are moving towards or away from you as well.
I just wish games would also use the "time warp" style fps boost that VR uses. With VR you have the drivers doing stuff with the depth buffers, distorting the previous frame, guided by the depth buffer, using your headset's input.
So it allows you to move your head in a scene, while technically the action in the scene isn't newly rendered yet, for a low head-movement input latency?
I never heard of this, cause I don't own a VR headset, but that's what I understand this as.
@@Rainquack yes! I encourage you to look it up on youtube. There are videos demonstrating it. The idea is to never let your headset's movement suffer from any framerate changes. When a frame isn't ready in time, it takes the previous frame and warps it according to the geometry of the scene (stored in the depth buffer), so you can always move around freely in the space in the full framerate, and if things are chugging along you will see items and characters and stuff get choppy, but your own movement in the space is always smooth. It's brilliant!
I really appreciate the amount of effort that has gone into this video. Straightforward and easy to understand, even for someone who isn't very tech savvy. Thanks for the all the hard work, very enjoyable!
Any time I notice music from Oblivion in the background of a video I have to go back and watch parts again because the music is so good it's distracting.
incredible work with the explanations and demos! The filmed video is literally perfect
2:33 this effect could be used purposefully to make some amazing visual stylisation
0:00 You may not like it, but this is what the peak male form looks like.
your videos are dope. been binging everything since i paused watching you from a few yrs ago, good stuff.
Thank you for your research! I've really enjoyed this type of content from you over the last few years!
You released the video just in time. Was rewatching your older ones
मेरे विचार से यह एक बहुत सुन्दर , उपयोगी और महत्वपूर्ण तकनीक है चित्रपट निर्माण के लिए । इस तकनीक द्वारा हम विशेष दृश्य प्रभाव जैसे मैट्रिक्स प्रभाव या टाइम-स्लाइस (जो की कई कैमरों का उपयोग करके बनाया जाता है ) ताकि काल धीमा हो जाये या स्थिर प्रतीत हो , बहुत सरलता और तीव्रता गति से से सम्पन किया जा सकता है।
वैसे आपने इस चित्रपट में गहन शोध , स्पष्ट अवलोकन , प्रज्ञावान विश्लेषण और बहुत सुन्दर प्रस्तुतीकरण किया है ।
Wow, those morphing videos created by RIFE remind me of some rather spiritual experiences
?
@@HaxxorElite the RIFE merging looks really similar to an effect one can get while using psychedelics
@@logisticallogic8673 It has no connection
@@HaxxorElite ? I'm not quite sure what you mean, I'm only saying it looks similar to things I've personally seen before.
Look at that powerful trunk. Phillip is built like a Roman column.
excellent video as always, always a pleasure
DLSS 3 will be HUGE for older games that have physics tied to fps. A great example is Ocarina of Time. Trying to play at 20fps gives me a headache sometimes, but I power through it since it's one of my favorite games ever. If DLSS can properly "upscale" it to 60fps on Project64, I'd buy a 40 series card alone for that.
I would love to see a dlss 3 example that shows the input video on the left, the "upscaled" result in the middle, and JUST the newly generated frames on the right. Would make for a really neat comparison.
Seeing my own animation in your video caught me of guard. Very nice
Great video. I've been waiting for real-time frame interpolation tech in games for years; it's very exciting to see it finally coming to life. Plus, the wise mystical music always gets me in a wise, mystical mood.
love your work!
I think this is the most I have ever critically analyzed and learned from a clip of a half naked man
Other than maybe that unlisted 4kliksphillip video from a while back
I enjoyed this video thank you for making it
I almost didn't click on this video, because the title sounded boring, but I am very glad that I changed my mind. This video is even better than a lot of your recent ones.
Long time SVP user and RIFE user as of this year here. I would like to report that chain link fences are the bane of all things frame interpolation, and you should use it as a worst case scenario instead of grass!
When using Rife with Flowframes, it does have an option to detect scene changes, so it will not interpolate between those 2 frames.
Woah, what is the name of the last song used in Conclusion chapter? It is so good, never heard it on another philip video before.
i also wanna know
Wait... is that Uncle Al? Oh god, he's broken into our physical dimension!
Aside from that, I did find this whole experiment/demonstration super fascinating. I've always been mesmerized with some plugins and software that can take 60fps and attempt to convert them into slow motion. The weird bending and smeary distortion can look really neat.
Frame upscaling and the artifacts that come along with it look really similar to asynchronous space warp (frame interpolation for vr headsets)
This is not upscaling though, it's interpolation
@@theRPGmaster you could argue that its temporal upscaling 🤔
@@AngryApple You could argue it's temporal upscaling when plants grow
@@AngryApple no you cant because then it would be called temporal upscaling
Fricken great video!
I think it could be interesting to just show the 60 AI frames for the 120fps example, to judge what it is doing.
I'm glad you sorted out the city packs for flight sim! Sorry, you were probably flooded with comments on the video on that...
doubling the replay speed to increase fps was the most smart thing i ever heard 😮
Don't undersell yourself - you are always quite mesmerizing to watch ;)
Ray tracing and dlss are the technical computing miracle technologies of my time and I now understand what a lot of older gamers talk about about what I take for granted, it’s such an exciting time
OOOOH the bugs you mention about framerate reminded me of an old bug. when you ran oldest cossacks game on 64 bit architecture it couldnt help but speed up the game so fast it was unplayable it was like running a speedhack on very fast setting. it was meant for 32bit architecture. there were also some differences of gameplay between different screen resolutions iirc.
I'm fairly sure the gpu will utilise more data to interpolate muuuuuuuuch better. A gpu has a z-buffer (depth image), movement vectors for each pixel and more data, basically for free. This additional data can make the output almost perfect, even for the grass situation for example.
The Oblivion score in the background was doing my head in trying to work out if I had another tab open with someone streaming gameplay lol.
it would be cool if there was like an option that would only activate the fps upscaling when your fps goes below a certain point.
for example you would normally play at 60 fps but when the game sometimes dropped to 40 it would generate new frames to hit the 60 fps again
Saving this to my "watch later" pile as I'm going to sleep. 😂
see you in the morning my dude
what looks REALLY good is already high fps footage upscaled to even higher (say 240 to 2x or 3x that speed) then RESAMPLED back down to 60fps with sony vegas smart resample. it looks insanely smooth good video btw
Would love to see this as well
Very interesting test and comparasion! But I wonder if DLSS3 creates its newly interpolated frames by interpolating between grainly DLSS-upscaled frames Or is this new frame interpolation an entirely separate tech which doesn't involve any upscaling at all? At the top of the video you mentioned that it doesn't allow for higher resolutions than before. So I assume its working like DLSS2 plus interpolated extra frames. I am looking forward to a possible future video where you are testing it out to see how it looks and behaves in detail! Big love from Europe!
In the 5 -> 60 flight sim, sure there was a lot of funny business going on over much of the screen, but there was also a lot of surprisingly smooth motion on the more distant objects. Definitely not how I'd want to game, but still super impressive that it can take 5 frames and turn them into 60 with any degree of success even if it's not exactly a good result overall.
I am curious how this program does its thing without itself reducing performance. If the computer is not fast enough to produce 60 real frames, how is it fast enough to invent fake frames to make up the difference? I get that the fake frames aren't as complex to resolve as real ones, but they aren't free either, the computer has to do some calculating to produce them. Is there a point at which the computer is spending so much time inventing fake frames that the real frame rate starts to suffer?
Two things I would say-
The low frame rate interpolation shimmering could be used for some really good music videos
Hypothetically for the corner warping if the area it takes up is consistent enough would rendering the image at a slightly higher resolution and basically cropping to the desired resolution
that intro is gold.
Didn't expect to see David Bowie demonstrations today.
8:25 - It’s impressive technology, but I feel like those camera “jumps” would drive me insane after a while. It’s similar to the later examples where you can’t exactly look for the issues, but you can FEEL their presence.
You made a great point at the end of the video. Past 60fps, traditionally I think the benefit of higher framerates has been the reduction in latency, not that it appears to move smoother. You can try looking over someone's shoulder as they play a video game and honestly anything above 50 looks reasonably smooth, but they can certainly feel that it's not perfect due to the latency. I'm really curious how DLSS 3 is going to feel on 30fps and 60fps games
It looks much nicer ngl.
Disagree, CRT's in motion resolution are equivalent to 1000fps, 60hz on a sample and hold LCD or OLED screen looks like a smeary mess, in fact resolution drops to 360 lines equivalent to 360p when panning the camera which looks ultra terrible on modern 4k displays. Latency is only 1/2 the picture and only really relevant for E-sport titles / twitchy games. Nvidia, Intel have both done research papers/tests that suggest past a certain point people don't care about latency. Also this testing methodology is useless as DLLS 3 has access to the games render pipeline and motion vectors so all those artefacts he experiences in his CS:GO example are a limitation of RIFE and can't be compared 1:1 with DLSS.
@@theemulationportal what bargain bin OLED do you have that smears 60fps content?
@@theemulationportal You're right about the motion vectors, I expect DLSS to outperform the methodology shown in the video in terms of image quality.
My argument was more that the increase in framerate is only really meaningful as long as you can feel improvements in latency. If you can't feel improvements in latency, you probably aren't benefitting anymore from high FPS from a smoothness standpoint either. I suppose we could actually test this empirically by having a 240hz screen and injecting input lag until it has the same latency as if you were playing at 60fps, might be an interesting test to do.
@@mechanicalmonk2020 For a 1000 pixels per second motion tracked by the eye, the object tracked is perceived to have a 16,6 pixel wide symmetrical smear on a sample and hold 60Hz display. Being OLED doesn't change that. A theoretical instant pixel transition technology would not change that either.
these vids are so cool 🕶
5:01 This came out really nice with the Vsauce-like music
such an amazing video. holy shit..
Did philip keep working on his personal version of odgrub?
At 11:57 there are new custom subtitles which werent there in the video
yea the 5-60 csgo scene broke my brain, I have never in my life experienced motion sickness not even when I bought my vr set and instantly started with locomotive movement. But that scene broke me, jesus
Wait I did not get the last part about raytracing. How would DLSS 3 help double the amount of rays fired for a raytracing scene while keeping the framerate same as before? Image interpolation and ray tracing are completely different scenarios of rendering. How is that related? Can someone enlighten me? 12:16 Or does he just mean essentially what Optix is doing? A form of AI denoising which helps get a better looking image with just a few rays fired.
I see what you tried to do here, but all of these technologies are only giving the AI images without motion vectors, whereas dlss3 has all of the motion vectors in thr engine to work with in addition to the visual data. Early comparisons between existing techs show that dlss is far more advanced than off the shelf interpolation AI programs. However what you said about it looking smoother without feeling smoother is probably spot on. That will be interesting to see how it does actually feel versus just watching a game with dlss 3 footage
its going to be quite interesting for sure. With post processing you can't really test what would happen with a game. 5FPS decimated from a 60FPS video would be as you presented it, an even decimation of the frame rate and thusly, the frame times. but with a game, as i'm sure you know, you have a variable frame time. i'm the most interested in where you are getting, by all means, a measurable "60fps" or "120fps" but your frame TIMES are wonky and uneven. THAT is where i'm sure DLSS 3 will shine.
EDIT: and yeah, DLSS3 will also have the motion vectors and depth buffer among other things that can act as hints to it. this is going to be VERY VERY interesting to see.
Well there is a game realm where something like this is already implemented - ASW in VR games, and it gives very convincing results.
Good video!
Very interesting.
Idea: The artifacting in the 60 to 120 FPS conversion becomes fast enough to trigger a kind of survival mechanism in the human brain, related to danger response and reflex. Like if you see something unexpectedly move in the corner of your vision, that may be a danger that you need to react to, so your brain distributes more resources to paying attention to it, and divides your attention between the flickering artifacts and whatever thing your trying to concentrate on.
Nice!
I'm surprised that you, as an avid VR fan, made no comparison to the interpolation technology used in headsets! This kind of "Something is wrong, but you can't exactly tell what" happens ALL the time when you use interpolation to smooth out the framerates in VR. Things seem to "hang" on the edges of the screen sometimes and warp out of proportion. It is why I prefer to have a choppier framerate, as this kind of "wrong" feeling disappears, though I presume people who quickly get motion sick cannot enjoy the same luxury.
Specifically, I'm speaking of "Asynchronous Spacewarp", which adds frames when you move your head, to interpolate where you're about to look.
wondering if this DLSS3.0 feature will be able to hide the shader stutters that we get in most open world games like...that would be "worthy gain" in my book.
in one of DF analysis, the frame interpolation process takes as little as 3ms so it s very flexible, they can also implement frame interpolation as a "substitute" frame., only delivering the interpolated one if the "genuine frame" does not show up on time, that would eliminate pretty much of the occasional stutters, the 1% low for sure, the 0.1% lows not sure
If a frame does not show up, dlss 3 doesn't have 2 frames to interpolate. So you'll get stutter anyways
@@iurigrang you are right, DLSS3.0 became meaning less to me now but thnx
My Samsung smart TV upscales frames already it's called sharpness setting, really it's good as long as you don't go overboard with it (mine is set on 70/100). Especially during circular outer pans of camera, that's when you notice all the gaps in spacetime it can not possibly fill :D
Fallout 3, NV, and Skyrim, have mods/plugings that give high framerate without issues
(Some of these require some tweaking, but I have them all of them working)
waited for this vid
Can you try to test a video using the dain app? I've used that for a lot of my videos and upscaled videos from 24fps to 48 or from 30fps to 60fps and I really liked the results.
artifacts do not worry me. what worries me is the input latency
Have you seen RTX remix phillip? Who am I kidding, of course you have, feel like it's a techology you'll love. I can't wait to go render so many old dx8 and 9 games with new ray tracing capabilities it'll be very neat to see them in a different light.
Cool video! Thanks for making it. For the 30->60fps examples, could you show the real skipped frame and the interpolated frame side by side or even the difference between them? I feel like that might be really cool to look at. Cheers.
the oblivion ost just made this video super
sneaky Oblivion music :)
I just hope that some form of it can be modded in or maybe even down the line officially supported on older cards, DLSS2 is great at what it does but fully raytraced cyberpunk still makes my gpu catch fire a bit even on Balanced
this video reminds me of how much i hated using twixtor for editing haha, could never get the extreme slow motion looking right and now I find out why
I've thought of something that I feel could help with video but potentially hurt with games regarding frame rate interpolation
Render it or record it at a higher resolution/zoomed out/farther away a tad
Interpolate then crop so you can't see the edge artifacts as much or mayhaps at all
This would also help screen space reflections
Yeah, I feel like the data should exist, and am not sure why there isn't a 'TV Overscan-esque' option for games, or in the driver that could help with screen space reflections as well.
Makes me think of that one time where nVidia announced some tech at some keynote 5+ years ago, which 'counter bends' 3D game output on curved monitors to fix FOV/curvature related issues.
So looking to the sides works in perspective.
Which reminds me of 10+ years ago, where people used Kinect to give dimensional depth in relation to your head position on 2D screens.
I don't know how we don't have any of these things today.
Hi philip, do you know the name of the song playing in this video? Thanks. Great vid btw
@@2kliksphilip 0:00-3:04
@@2kliksphilip thanks so much!
First 17 seconds of this video is the best news I've heard since the great war
Loool about the keeping the money for camera comment, imagine in 20 years when cryptography has become so powerful we can run quantum apps on old pentium 3 CPUs that would be hilarious. I always thought the race to better specs was a linear scaling through centralized means of production for profits, rather than a smart scaling through data compression and restructuring.
That Oblivion music tho
0:14 Now I really want to see 30 fps vs 60 fps comparison like that, just normal gameplay (not DLSS or anything like that), that is filmed from the same angle in both recordings so you can crop it and put 30 fps on one side and 60 fps on the other side. Searching for it on TH-cam, but I can only find comparisons where it's crop-split like that.
Is there currently anything better than rife? I need 60fps to 120fps conversion for editing purposes.
M2M_VFI
@@D-K-C How is it better? Is it faster or is it more accurate? Or both?
@@Neekzu both
4:52 shouldn't "will" in the subtitles be capitalized?
Something interesting is going to be if it's like minimum frame times, what if there's 10 frames between some of the "true" frames, after nothing but 1 frame between "true" frames. Something like a new form of stuttering...
Even the 15 real FPS upscaled looked fairly impressive to me. I was thinking about how I played like 80 hours of Oblivion on my crap old PC at 15FPS, and then the Oblivion music kicked in around 9:30 and took me right back there 🕑🕒🕓
The 5 real frames example gave me a headache. Really impressed with 24 though.
I still think the optimal solution in a game for this sort of tech would be to dynamically raise the "pure" framerate based on the frame to frame distance, essentially demanding more frames for the ai to use as reference based on the current speed of camera. After animating for many years, I can tell you that your can make things appear very smooth even at a low fps assuming the image isn't changing a ton frame to frame.
I wonder if this would be adopted in VR, instead of the Asynchronous Space Warp
So the main problem occurs at the edges of the screen right? How about we increase the edges of the screen by an amount that is like 5-10%, i.e. increase the screen resolution, for game footage, let the interpolation do it's thing, and then cut off the excess edges back down to the original resolution so the artifacts are hidden by sheer brute force strategy. I know stuff like the HUD will get cut off, but it might be worth trying.
The problem has nothing to do with the edges of the screen, but with the fact, that the objects closer to the edge tend to move faster inbetween frames... But as he said, the movement vectors will fix all that wobblying, so it will be a close to perfect result anyway
I really didn't understand DLSS as well as I thought till I discovered your channel, and since then I've learnt a lot. This technology is going to get pretty wild as time goes on.
Why are your videos so good?
I still doubt a lot that the frame generation has to wait the next frame, pretty sure he can render new frames based on previous and current frames only and i have engineers tweets to back it.
Now you should try 40 or 50 fps boosted to 60, or like 90->120fps. I believe interpolating less than a 1:1 frame ratio would be perfect, as when it guesses 11 new frames between every 5fps, it struggles so deeply with attempting to create the new images.
Also, if it created frames as going to the halfway points between movement, frame changes, then it could look far more accurate for bringing up low frame rate video.
60->120
Having a 144hz monitor and putting the test on x2 speed I honestly didn't notice any artifacts, aside from a feeling of motion blur
Most of the artifacts in this video are the result of naively estimating motion by finding the nearest patch of similar looking pixels. This is why high frequency detail like grass, rapidly moving objects, or repeating textures look the worst.
DLSS 3 should have access to motion vectors for both frames, so if done properly, most of these artifacts should be avoidable. The only hard part is generating detail that is occluded in input frames.
We have the technology to do this convincingly, but I doubt it is optimized enough to run in real time yet, so it looks like DLSS3 is just smudging surrounding colors to fill the gap. Maybe that’s convincing enough in practice.
2:20 ah what I would give to get up to buttery smooth 5fps!