GPU Reviews: How Relevant Are Apples-to-Apples Benchmarks?
ฝัง
- เผยแพร่เมื่อ 7 ก.พ. 2025
- ► Watch the FULL Video Here: • DF Direct Weekly #199:...
► Support us on Patreon! bit.ly/3jEGjvx
► Digital Foundry TH-cam: / digitalfoundry
► Digital Foundry Merch: store.digitalf...
► Digital Foundry at Eurogamer: eurogamer.net/...
► Follow on Twitter: / digitalfoundry
somebody needs to come up with a system of enumerating image quality
Apple to apple benchmark (native res) still very important to gave perspective on raw power of a hardware. Many will try to convinced you it isnt important, but without it customer will lost and hardware review only become hardware feature showcase
AMD is satisfied on being the mediocre 2nd option for GPUs. They profit anyway
Expect they are better value for money and offer comparable performance.
Imagine saying this in light of the 5000 series paper launch 4000 refresh. Nvidia charges 2x price for +15-30% performance.
@fatrat92 they give you the impression that you are getting more for less, but at the end of the day their software sucks and you know that very well. Pure raster performance is no longer enough in the GPU gaming market.
@ Nvidia makes great products, but they are indeed greedy in their pricing and market strategies. I am not defending them here. I am just criticizing AMD for being incompetent to the point where even when everybody is tired of Nvidia, they still lose by puting those 2nd class RX GPUs on the market.
One thought they might got somewhere in 2020 after considering how strong they were that year and the impressiveness that the 30 series was. + the improvement that ryzen had. I was thinking they made a breakthrough!
Did this happen back in the day, like when the Glide api came out were there people complaining it wasn't "fair" to compare their preferred gpu brand to a Voodoo card because the result made them feel bad?
good question
Given the sole purpose of GPU performance is to improve quality at high fps, ignoring differences in quality would invalidate the performance comparison. It would be like benchmarking some inner stage of the rendering pipeline, and those numbers don’t manifest to the final frames.
Native resolution is now more like a benchmark that shows some result "in a vacuum", but what good is it if the product is not used in such conditions?
For a ton of people, still very useful. I refuse to use upscaling unless I HAVE to. I absolutely notice the graphical issues that come with using it, and it annoys me, and in some titles actually makes the game unplayable. I had to refund Black Myth Wukong because of how absolutely terrible upscaling made that game look.
@truedps8 you had to be using fsr cause dlss looks great
@@truedps8 What you talking about? Dlss is great, its only looks a bit blurry at 1080p, but new dlss 4 super resolution fix that alot, its looks good even at 1080p performance mode upscaled from 540p, i almost not believe what i saw lol
For a GPU *review* native resolution should be tested first -- since it's a *review* about the cards *performance*
But naturally _I assume_ most people would be interested in how it performs when using upscaling and then you as a reviewer -- if you'd like to -- can talk about _your own perceived experience_ or recommendation using whatever software solution.
You can still show your brand loyalty in your reviews by only testing that 10 games that supports Ray Tracing for example, we should never go that low to skip native rendering in a review. Otherwise what's the point even, we can just go and check those trustworthy Nvidia / AMD / Intel slides and reach the same conclusion as Mr Jensen about the 5070. 😆
Native resolution is now more like a benchmark that shows some result "in a vacuum", but what good is it if the product is not used in such conditions?
@@staskozak8118 They are not used, because native resolution is so bad nowadays. This is why we still need check it to keep developers and GPU makers accountable with getting better products. People should stop buying games, which cannot be run on native 60 fps on at least medium setting on resolution GPU cards were made for. Of course FG isn't allowed like they were written in Monster hunter requirements. Of course newer generations should get minimum 60 fps on high settings. Otherwise game should be DOA. With that developers would complain more to GPU companies to get better hardware, but if people are buying games, which run like crap even with newest hardware, than why bother to do more work on your product?
@@staskozak8118 You have an opinionated point of view about _how you feel_ the hardware will be used by the consumer. However we are talking about an objective review of something. Why would you want to paint a picture of a device running proprietary software based techniques *only* when these might change in a week by a driver update -- thus will make your video instantly obsolete?
I'm not saying upscaling should be excluded I'm just replying to a carefully tailored, clearly dumb question that is asked in the video title.
Personally I'm not buying an expensive video card to use upscaling or frame generation, if I have to I'd rather lower the settings a bit or just _vote with my wallet_ , so in general I'm a lot more interested in how the product actually performs, runs a game in a way it supposed to run it.
I also don't really enjoy the current state of the industry, how vendors are marketing their products, how publishers are using these software based techniques to mask their incompetence or laziness.
So please allow me to have a different opinion and raise my concerns instead of giving these companies more excuses if they under deliver.
There is absolutely no value in comparing native resolutions anymore. The games aren't played that way so its completely irrelevant. What consumers actually care about is---when my visual fidelity looks like "X" how many FPS is it running at? Whatever magic you need to do on the back end to make it look like X is meaningless, is it native, is it AI, doesn't matter.
@@chrisbutler8856 I'm aware that I'm commenting under a Digital Foundry video, so I understand that you like talking about DLSS and stuff, but I'm still puzzled why you people are trying to force your opinion on others.
5090 4k tests didn't make a lot of sense to me since, yes, they'd be going from 60fps on a 4090 to 80fps on a 5090 at 4k (33% increase) but... everyone I know playing at 4k with a 4090 (myself included) uses DLSS quality or performance so we can get 100+ fps. So we've never cared about native 4k performance in real world use, since for most games the card was only internally rendering 1440p or 1080p and in those cases the difference between a 4090 and 5090 were much smaller (8-15%). So when I upgrade to a 5090, am I getting 33% or 15% or 8%? (And likely even less if I'm CPU limited!) you know?
There are objective methods to test video codecs, that can be applied here. Even a simple PSNR rating would help. You'd have to have a 100% repeatable benchmark, establish a ground trouth video f.e. by a 16k render and then let software compare the losless recordings of differend upscaling techniques runs.
How about instead of trying to figure out how we compare different technologies against each other, we instead punish the corporations to limit the software to their own hardware in the first place for the soul purpose of monopolizing the market and locking you into their eco system, so situations like this won't happen anymore. "Good job nvidia for yet again developing propriotary technology that locks out all the rest of the industry and that you have full and only control over. Let's give you even more of an advantage by changing the way we review products to better fit your editorial direction". Stop this nonsense. If it's not comparable, then don't compare it. Aknowledge it's there and that should be all there is to it.
My take would be making a standard test with a like for like bench (Say, FSR or XeSS Quality). But then an "Equivalent Quality" test for cards that have access to advanced upscalers (DLSS, XMX XeSS.etc)?
And on the Note of DLSS Transformer matching up to 4K TAA, we could have a test be using that as the baseline standard bench (at least if possible), then check the vendor upscalers for equivalent quality threshold? (so EX, Quality FSR, Perf DLSS, and Balanced XeSS)
There absolutely should be native benchmarks. DLSS will NEVER be perfect, and for some people the graphical artifacts that come with it will never be worth it.
Smaa/msaa is dead, taa is in every game and games look like shit without it, and blurry mess with not enough antialiasing with taa. DLLS looks better then taa, it is what it is. Games are made without any care about how they look without taa, and many games don't even allow you to disable taa.
No one ever is gonna be happy with any sort of comparison that isn't 1 to 1 and that is no longer possible outside of pure raster performance without upscaling. Nvidia, whether people like it or not has moved one and there is no longer parity in image quality or feature set between vendors. Games also more and more demand rt capable hardware (software rt or hardware) which further complicate testing. Even if FSR 4 matches DLSS in image quality there is still a gap in Denoising and other aspects that can affect the perceived value of each product based on pricing tier, games tested, and user general needs.
I'll put it this way just show the native WORST CASE results first and foremost I don't care if something like path tracing is too difficult for even a 5090 it shows the proper raw performance of the card for the resolution it's targeting and if you get like 10 fps at 4k show the native 1440p and 1080p results as well only then do you show the upscailed results because you're still having a trade off with upscailing vs native rendering but at no point should something like frame gen be used in the benchmarks
Apples-to-apples benchmark are non existent. Reviewers have to force Nvidia into scenarios which puts them on equal playing field with AMD. When a large selling point of Nvidia is their features, which put them at an advantage, it's a bit dishonest or lacking context. How can you compare FSR quality and DLSS quality performance figures, when FSR is less than comparable to DLSS performance in many cases. Nobody here can say that upscaling or frame gen experience is equal on the two brands, just because they both have the same name.
Ofc it does if what you are testing is a cards performance at doing x. Then all cards tested must first prove to do x and then their respective performance at that task can be compared.
An AI generated frame for instance, or upscaled graphics can look good... but it is not a real frame intended by the game. Its an approximation of the graphics card of what it thinks could be there. That is not the same thing as what actually is there. It might be close, but not the same thing. So performance should be like for like.
Then upscaling, generated frames etc are potentially useful addons that could be tested as such. I play with DLSS many times and I accept the short comings of it. But I don't mistake it for actual raw perfo4rmance of my GPU.
I’m sure they are comparable but I’m used to NVIDIA GPUs. Did however swap to AMD cpu over intel
FSR 4 is around the corner. This question comes like 4 years too late.
Not really, from the looks of limited testing as well as PSSR, FSR4 will compete with DLSS2, not the DLSS4 transformer model which just came and is infinitely better.
Also people should stop hyping up AMD technologies, they've been 3-4 years behind and have never made any leeway on bridging that gap. They're course correcting but it's too late, it will take at least another whole console generation for machine learning tech to plateau and standardize to the point AMD has a chance to be on equal footing again.
If you both fundamentally misunderstand the question and fundamentally misunderstand FSR4.
@@steel5897
Nobody’s hyping up anything.
I’m in the nvidia boot as I play at 4K and AMD has no competition there.
That being said, wait for FSR 4 and the test results and then we’ll see how much better nvidia’s dlss 4 is. Because the fact is we don’t know.
@@SolarMechanic
When you don’t understand basic English.
Get it? You needed to use “when” instead of “if”.
_Meanwhile in my country you can buy 2 7900 XTX for the price of a single 5080._
Blind quality comparison where you don't know the settings
No, and it's a silly question. Reviews are meant to showcase the best product and the best product for the consumer is the one that performs a certain job the best. You can't ignore the image quality advantages of Nvidia cards in modern games, doing so would be a failure of your job as a reviewer, and some kind of desperate attempt at looking "fair and unbiased" which in practice just means you are going out of your way to protect a brand from looking bad, which is hilariously ironic.
People are really going out of their way to misunderstand this question.
It's not saying you should "ignore" features which make one brand or the other look worse, it's asking how you compare things which, while similar, are actually different.
For instance, a review may compare FSR "Quality" to DLSS "Quality" because they're similarly named and do the same basic thing, but that's not actually "fair" since they produce different qualitative results. In this instance, dropping DLSS down to "Performance" would level it's visuals to that of FSR Quality, providing an experientially "fairer" comparison. One which would, if you think about it for a second, make Nvidia look BETTER since it would boost the FPS of their result.
The DF crew then expand upon this conundrum when considering different technologies which have no equivalent on certain brands, and how this will become an issue for reviewers in the future trying to provide information without their results becoming bloated and convoluted.
I'd ask if you even watched the video, or just read the title and jumped down to the comments, but even if you'd done that, thinking about it should have made you realize the questioner was ON YOUR SIDE.
I think apples to apples ( native resolution no ray tracing)comparison should be the standard. Why ? Because we are seeing games like monster hunter giving system requirements that take performance upscaling at 1080p high settings and achieves 60hz with frame gen.
Upscaling and frame gen should be used to get better performance and not to achieve 30 or 60hz bare minimum.
The two things you said have zero connection to each other, your comment does not make logical sense.
@steel5897 dlss is far superior to fsr. But native resolution is always the real deal. Ofcourse Nvidia user ( like myself ) is going to absolutely use Dlss. The problem now is the lack of optimisation in pc ports. We see system requirement charts put together with upscaling as default. Upscaling tech came into existence to allow greater headroom for ray tracing , more fps and to prolong the life of aging hardware. But you now see a lot of broken games use dlss and fsr as a crutch to reach 30hz. Which is unacceptable.
To explain to you , saying nvidia card is better because dlss performance is almost equal to fsr quality is just wrong even if both are equal in terms of hardware. It only serves to make upscaling the standard instead of native resolution.
I don't think it's that hard at all. DLSS and Frame Gen are stuff that still depend on that native performance to get best results or even to know if they merit being turned on at all (also, that 80% of DLSS usage is misleading at best, as games usually enable it by default, whether you need it or not, and Nvidia App does the same when "optimizing" the game). Ray Reconstruction and RTX Mega Geometry are stuff that, while they impact visual quality, are still confined to a small subset of games and effects that might not have a strong effect the lower you go on the performance tier. All these software features are definitely worth talking about, but as an additional, subjective layer in the review for the consumer to decide how much they will impact their experience.
💯
True, but only the games that struggle with native performance need these features. I don't think many people are upgrading to 40 or 50 series with the intention of getting their old AAA or backlog performance numbers up as a sole reason. These features are needed, and that small subset are games people play for the best visual fidelity. Subjectivity in reviews is a way that influencers can add their "just as good" labels to features that are clearly not as good. That has been AMD's marketing strategy for the last 5 years...pay influencers to talk up raster and talk down hardware driven AI advancements.
@@CuttinInIdaho that's my point, that should be up to YOU to decide. But again, the point is to know what is the baseline performance you can get with the new GPU, and that will allow you to assign what (subjective) value DLSS and such add to the hardware. For example, can you really care about Ray Reconstruction if the card can't even run the game at a decent framerate? Will you upgrade your card if it doesn't give you enough of a performance increase that you have to use an upscaler just like your current one, considering future games are going to be even more demanding? That's the subjective part that has to respond to the objective one, which is: how many frames can this card really render?
@@Diegorskysp17 I completely agree. Baseline is what we need, cause everything derives from it. It's like changing naming of presets for XeSS, where quality is the same resolution as balanced in other two. Someone without knowledge about it could be thinking XeSS is better performance. I do like, when reviewers give native raster performance per resolution, then they get RT performance (preferable native, but mostly quality upscaling, cause RT is too taxing to get playable framerates anyway).
I dislike comparing those other technologies, cause they are mostly dependent on post processing filters. Why do I dislike it? Cause not all games have the same quality of those filters. Some games have really bad TAA implementation and those are better for Nvidia, cause they get the most difference by changing to their DLAA. In those cases FSR suffers from those native artefacts like shimmering. But when there is good implementation od TAA, both upscalers are generally good quality.
I do wonder, what will FSR4 bring to the table. This is made with RDNA4 in mind, but will this be radeon only solution from now on, like with xmp mode for XeSS or will this still be platform agnostic? How would performance and quality look like? We saw samething on note to ray reconstration too at CES, so maybe RDNA 3 and 4 will get better quality image in the future?
What would be ideal would be setting standard for some technologies as a baseline and leaving hardware and driver implementation for GPU manufacturers. With that we would be more sure to have hardware compatible and newer games could be more eager to try new experimental tech behind extreme settings.
Castlevania: Aura of Holy Might (New Content Is Out Now)
This album features 8-bit / 16-bit remixes from the franchise.
not at all
Upscaling shouldn’t be used in benchmarks. Easy…
I totally agree. I don't see the point in testing 1440p native and 1440p DLSS Q instead of 1080p native. Gives the wrong impression amd saying 1080p results are close to what you would get by upscaling is much more accurate. However, I do think mentioning the image quality difference is worth it.
For starters filter out Cyberpunk, Alan wake and similar titles otherwise it's gonna skew towards Nvidia. We already know cdpr are working tightly with NV so amd and Intel will always render with a handicap, these titles are designed to use DLSS. For comparing different NV cards they are a really good fine grained test "suite" though.
Sounds like the question asker just wants to validate their brand preference.
How? They're asking if reviewers can, or should, provide clearer results by comparing experiential results as opposed to base numbers in a world where different GPU brands/models can provide different features and levels of fidelity. Which DF basically agrees with.
The fact that such a system would benefit Nvidia right now isn't "bias" it's... reality.
Complaining about it just shows you want to reinforce your own brand preference.
@@SolarMechanic You can't claim clearer results by comparing apples to oranges based on a subjective analysis of image quality. It's unfortunate, but comparing native resolution results is the only accurate or honest way if you want to put both on the same graph.
If determined to test with upscaling, you could compare both cards running the same upscaler but you would need to use one that works on both like FSR or XESS. This is the double edge sword of Nvidia making their technology completely locked to their own cards.
If you want to compare one to the other they need to be running the same test.
Not using dlss in the benchmarks makes them useless because thats not how a gamer would actually play take game. if people bought gpus to run benchmarks then sure but they buy them to play games. apples to apples is misleading becouse you are using setting ma that are just plainly dumb and reviews of the nvidia product produced by only looking at bar charts arent really worth that much.
Ok, but you can't put something on the same chart that's running a different test, that's my point. Yeah it's not perfect but what do you do? If anything, maybe compare Nvidia to themselves if you want dlss benchmarks.. @@0007JO
@@0007JO the problem with this line of thinking is that a benchmark is literally a standard or a point of reference for comparison. So if you're running two different tests on cards it is not a benchmark anymore. I have no problem with reviewers pointing out the deficit of FSR to DLSS but to put them on the same graph is disingenuous and everyone knows it. That's what it has mostly not been a thing.
Crocodile jacket man: "5070 has 4090 performance for $549" 🤡🤡🤡
Ps5 has destroyed the failbox series x!!!!!!!!!!!!!! Failbox is now the new Sega!!!!!!!
Another boring crap comment from Ryan Miller
Let what resolution, monitors, and hardware the majority people actually own and run be the guide.
Otherwise who are you helping with your review?
Power users don’t need guidance as much, and lower end users can see where they can reasonably aim for if they lower settings.