HERE is what we found, 150 hours of testing - AMD 6000 Series
ฝัง
- เผยแพร่เมื่อ 9 พ.ย. 2024
- 👉Need a repair? krisfix.de/en/...
👉KrisFix Shop: www.gpufix.de
👉Question about repair: service@krisfix.de
👉Follow me on Instagram: / krisfixgermany
👉Need Help? Book your Technical support here: support@krisfix.de
👉 Leave a tip for us if we've helped you out:
-PayPal : www.paypal.com...
-ETH: 0x03FFa9DF33F4edcE958a0729B30Dd5A9241DeFE2
#krisfix #gpu #repair
Pictures: www.techpoweru... - วิทยาศาสตร์และเทคโนโลยี
Because there was a lot of speculation after the first video, I decided to change the title so it wouldn't cause wrong thoughts.
Journalism is a good thing, but maybe some topics become more appetizing when you twist things a bit.
This channel is not meant to create videos with misleading content. The first video garnered a lot of interest and panic after which I was obliged to invest time and resources to give clarity on the topic.
If in the first video I had been sure where the problem was I would not have asked you for information.
I know there will be many more questions but I think the topic is closed.
In the next video we will continue with the repairs
I wonder if it has to do with the carbon thermal interface. Carbon atracts water and it touches the die directly.
The moisture/temperature during storage theory seems possible, but it would imply that this kind of failure should happen all the time because the manufacturer has no control over the transport and storage of the cards until they get to the customer (and even more so for the second-hand market). Also, aren't consumer electronics products required to function under a wide range of ambient conditions (unlike individual components) to avoid exactly that kind of issues? I mean, it would be interesting to see the general statistics of GPU chips popping due to storage conditions because it seems like a very rare failure mode in general. Even if all these faulty cards are from the same storage, it's a bit odd to see such a clustering of this failure with these specific models. It surely can't be the only instance where cards have been stored in this way...
something something, Jay2Cents 😂
It's really unfortunate that we keep having content creators with large platforms distorting the truth before having proper facts or knowledge.
@krisFix-Germany thanks for your awesome work and dedication 😉
This makes TOTAL SENSE!! You are SO SMART!!! I agree, but it is too bad we cannot know where they REALLY came from, or where and how they were stored. This is just a story about 'BUYER BEWARE!" or "PURCHASE AT YOUR OWN RISK" there are ZERO promises the card will work correctly... and we see that they did not. :(
ONCE AGAIN, MINER RUIN MORE LIVES!!! Miners have caused SOOOO MANY PROBLEMS for gamers... and even now it continues.... even AMD and NVIDIA were screwed by miners because when the crypto market crashed they were stuck with TONS of unsold cards (well, that is a risk the manufacturers know they are taking.... so I DO NOT FEEL BAD FOR THEM - I FEEL BAD FOR GAMERS AND OTHER END USERS.... NOT MINERS OR MFGRS... ... oh I also feel bad for retailers too :(
@@RealNovgorod my theory is, the cards are from miners. All day maxed temp, the graphit pad burned in. Then storage in cold condition, the cooler and die a drawing. After using the crack comes by using and get bigger until it dies
In the early days of my career, I worked at telecom factories. All of them had the same common practice regarding electronic parts entering the building: They would all go through a nitrogen moisture purging process. If humidity gets trapped in a part, it will decapsulate either when soldering that part of during the burn-in test. Nice work on that!
How could humidity get trapped inside the GPU die though? It's sealed to the substrate with underfill.
@@busterscrugs water is sneaky, water can make rocks in the dessert to crack, violently. Water can get in everywhere, given enough enough time. Which is why high humidity is not advised for a longer amount of time for most electronics.
@@crankylucifer Water CANNOT "get everywhere". It needs a big enough crack (which rocks tend to have). Hell, reverse osmosis relies on membranes that *don't* pass water through. You need pores larger than a water molecule for water to get through.
@@busterscrugs The underfill might actually be a problem. I remember Apple screwing up the underfill on some Macbook or another.
@@samiraperi467 reverse osmosis does pass water through, as a matter of fact, in reverse osmosis, only water passes through te membrane.
I had my gpu (3090 FE with EK Waterblock) fixed by you! Thanks again for this and the new memorie chips
HA! I knew it! I had a feeling something like this was what was happening. Having so many people with the same issue around the same time was incredibly strange and considering that the driver can not make low level changes to vbios it was very apparent that something else was going on. Well that's a sigh of relief for my 6950xt.
What's the tldr? Currently at work so I can't watch til much later. My new 6800XT wants to know lol
I feel bad for everyone who bought these boards.
@@detractorandroid33 tl;dw
Krisfix believes that the cards weren't stored in a proper environment by the seller. So when they suddenly offloaded a bunch of GPU's, they likely had a high amount of humidity, and with the high thermals of the GPU at stock settings. It likely caused the die to experience a kind of thermal shock(?). A great majority of the customers that sent their cards in for repair, had the same answer to having bought them online, and couldn't do a warranty repair because they didn't have the original invoice.
It's his theory, but it's practically what I had in mind since the news broke out as well.
Tho lower end...a sigh of relief for my rx6600xts ;)
@@AnotherLotte idem
This is a really solid theory, and something that didn't cross my mind. Earlier this month, I had to get a BIOS chip replacement on my laptop's motherboard, and it came factory dried and sealed with desiccant to avoid this exact reason of failure. Well spotted.
It didnt come to your mind that used hardware could have been treated poorly? Because that is pretty much the gist of the story...
@@dovermaskot4441 It looks like it wasn't "used hardware" but closer to like "new old stock". The improper storagw of stockpiled brand new cards sounds like a good probable cause.
Your theory of this being humidity damage makes sense to me.
For the 6000 series at least AFAIK the driver can override the powerlimit and hotspot temp. IDK if it can override the thermal shutdown.
To me this is the only logical explanation
you can override the thermal shutdown using morepowertool
These cards were all under warranty by AMD and yet the users opt'd to send the cards to a 3rd party repair service rather than RMA under warranty?
First of all thank you for your work! :):)
I definitely did not buy my card from a miner, however it has been opened (changed thermal pads and change back to air cooler plus I did not use a gpu support to hold it up). So from your video there are other possibilities here.
However I'm about to send my card back to you today as we have been discussion through mail.
@@RNG-999 take your time to watch the complete video. It may answer your question...
It was sad that some people gave you such a hard time when you asked the community if they were seeing this problem. I am glad you kept investigating what was causing this and came up with a logical conclusion. keep up the good fight!
No it isnt, AMD is going to sue this guy, just watch. His original was taken by everyone.. Linus, Jay2cents, all the youtubers, and reddit, and forums and turned into a big stinko with AMD, and it caused bad rep for the company. I for one wasnt worrie about this, i knew drivers couldnt do this, which is what he should have mentioned in his first video.. instead he goes.. well i got 60 bad cards all cracked cores.. all using the same drivers... think about it. it caused mass histeria for ignorant people..
@@jeepsblackpowderandlights4305 Sue in the basis of what? He didnt make any accusations, he literally asked the community to help with his diagnosis.
@@jeepsblackpowderandlights4305 whatever you're smoking I want some
@@Bramble20322the way he made his video spring a huge negative response online of people smashing AMD making accusations even big youtubers made amd face palm videos, it got overblown. And all because he didnt do any research just went on to say all tge cards blew up and it might have been the drivers..
Am i living on the same planet.. Did you not see the big youtuber channles bashing AMD over this ? EVERYONE even small channles made videis talking about sht they dont understand. It got blown way out of purportion. And i honestly could see AMD suing
lol just having the amd tag on reddit saying that you had no problems or something and you were treated like a pest ofc people would get mad to the guy that started all the drama.
This is amazing professional testing for a 3rd party unheard of a few years ago. X-Ray inspection! Huge respect
Yeah, this is more than a standard inspection. It's like a human health diagnosis
It makes sense. I believe it 100%.
In winter, when I buy any electronic device, it arrives from the courier so cold that it immediately fogs up. To be sure, I let it "dry" on the heating device before turning it on.
You are a GPU god
Interesting question in this case, is how water manages to get under coldplate within thermal compound? If it just stayed in humid place, it is.
I mean, it is really sounding reason, and probably highly possible one.
@@DimkaTsv water always finds a way
makes all the sense in the world same thing in summer; sets out in the sun or hot ass truck, then comes inside to an air conditioned enviroment.
@@lookin4talentt Yeah, i guess you're right with this one
How come your cell phone doesn’t have a problem, are we have had GPUs for 20 years damn near and never had a humidity problem? Why isn’t there gtx10 rtx 20 or 30 series having issues with humidity causing cards to die.
Not saying weather changes can’t harm a card but to think it would only apply to Radeon in this case … idk
As a 6900xt custom model owner i thank you so much for your investigation because to bé honest i was really scared about this "driver" problèm, now i Can reinstall thé last driver. Thanks you so much for your hard work
Super Arbeit, habt euch/hast dir definitiv ein Langzeitabo verdient
As a 6800XT owner I'm so relieved right now. Thanks for the in-depth research and analysis, great work!
Me too, i have the ROG strix LC OC 6800xt and it was not cheap.
Count me in on that relief, I just bought an MSI gaming z trio 6800xt around September and I was starting to worry.
and yet so many 6800xt and 6900xt owners swore that the latest drivers are causing them to black screen enmassed.
@@romanceidiot Well, maybe the last drivers are borked too and causing issues, maybe they don't play nice with certain Win versions, or some other HW those people are using. But hey, at least they're not explosive hahaha
me too but I can't say that I didn't monitored metrics like crazy these past weeks, everything good & normal so far on 22.11.2
I used to repair motherboards and do bga replacements. When we did this we would bake out the humidity from them so when replacing the bga they wouldn't blow up.
We put the motherboard's in an oven at a low temp for up to 24 to 48 hours. This would remove the moisture from the boards for the most part in order to do repairs.
Replacing bga chips like a GPU would require heating them up hot to melt the solder. If there was moisture in the boards from humidity, it would boil and cause damage. This is the reason to bike them to remove the moisture as much as possible.
Humidity is a major component that damages consumer electronics and computers.
I just wanna say thank you and express my admiration for all the work you are putting into finding out what the problems were with these GPU‘s. I really learned a lot from your videos, very interesting stuff.
I wanted to send my GPU to you today but holy smokes, $400 to send from US to Germany was just not in the cards, almost half the cost of the GPU, and a broken one at that. Love your videos btw :P learning a lot. You're company should expand, there is a massive demand for this kind of skilled labor.
I bought my old 1080Ti in 2018 second hand, after being used in a mining rig for 6 months by the previous owner, and it still works to this day with mild overclock. One thing I usually do, when receiving new PC part from shipping is to leave it for a day open in room temperature, to condition (acclimatize) the device, before installing and power it up.
Not possible for most due to excitement
I've always done just 1 hour of this and never had any issues, but 1 day is probably smarter 😂
9 years ago, I stuck a broken hard drive in the freezer for an hour. Came out frosty. Plugged it in as soon as I took it out, it's been working great ever since. Still using it in my NAS to this day lol.
@@busterscrugs I wonder how it will spin if stuck with ice.
u dont need to leave it for 1 day, its too long to wait for playing games, the reason card is still working is the store of mining its self is under 80'c
Unfortunately the damage has been done. I've seen people in comments parroting the narrative that the drivers are cracking 6000 series cards. Even Jayz2cents made a video as well as many other TH-camrs. People stated that they returned 6000 and 7000 series cards simply because of the fear your first video created. I hope that this video gets as much coverage as your first.
I'm honestly upset he never apologized for making the first video nor that he never took it down after realizing the die were all cracked. Because for him to straight up lying saying drivers can break cards on the hardware level is simply wrong. Especially when he's a hardware repair man.
He should have first make this kind of research he did for this video and THEN release a video about it. Not cry wolf first, create a useless commotion (probably to get more publicity) and then release a second video where he goes "ye nah, drivers can't break cards lol". Are you fucking kidding me? As you said, the damage has been done now, especially when such a big channel like jay picks the story up and makes a video about it.
@@davidepannone6021 Cry more.
not surprised. Jay makes no sense tho. To be fair i have stopped watching him a year or so ago. His quality dipped & no more watercooling so i don't care. I remember when around the Skunkworks days almost all of his videos were something about watercooling.
@@davidepannone6021 cry baby
@@davidepannone6021 Drivers / Software have broken many many cards in the past so it's not that stupid to think about that in the first place.
And what he means with "drivers can't break cards" is.. Drivers can't damage cards in the way those cards are damaged.
Jeez a german guy has to explain you everything?!
I would not be surprised if the seller "cleaned" all the cards with a pressure washer to make them look nice.
yep i think you are bang on, it make them look new and extra clean.
I swear I saw video on youtube where some miner cleaned rig with water hose and I cannot find this video again. Now we know where those cards ended.
Yeah idk how dumb you have to be to wash the cards. I've seen miners do it a lot
thats the first thought I had also. FYI you can wash boards in water. After cleaning you can then heat the boards in a kiln, an oven, or even generous use of a hair dryer if you want (worst choice imho) to ensure moisture has been expelled from the board before reapplying power. We did it all the time to motherboard when I was a board level technician. That said Ive seen several videos of someone washing cards with the coolers still attached. Thats a huge no no since the pads will absorb water in the process and will be difficult to ensure all moisture is removed.
No, the thermalpads are to clean for that, dust you can't remove from the pads, i opened a lot of GPU's and replaced thermalpads and thermalpaste. They are all new and not stored the right way apperently. These ones have not beeen used, you will see discolouration on the heatpipes and you can't remoce all the dirt from the fans.
Maybe it was a bad batch of GPU's that is also possible. As Kris has 4 cards new and 1 is crashing and an other one with high tmep. 2 out of 4 not is perfect condition new is bad.
When I worked for a semiconductor company, all our MCUs were sent vacuum packed with desiccants inside the bag. This was to avoid this problem where the devices would absorb water during transit and then crack as the water boiled out during flow soldering. If these cards have been poorly stored (notably in high humidity for a long period, perhaps as a bankruptcy asset) and then been used immediately by the consumer, it's possible a similar event could occur.
Your diagnosis is almost certainly correct. Its called Popcorning damage. The fast expansion of water vapor during reflow causes fractures in the package or die which sounds like popcorn popping. The same thing happens if the die exceeds 100C during operation. You cannot prevent water vapor ingress in customer hands or during transport so the solution is to limit die temps to below 100C in LSI devices. Before reflow, devices must be either baked or stored in a dry cabinet if device packaging is opened.
Congrats, Mike, Master IPC Trainer.
Determination at it's best. Thanx KrisFix for this....
Rog Strix 6900 XT LC Owner here.....this is a relief, i had no issues with the 22.11.2 Driver i was using it since the release and how no issues whatsoever, i was worried after hearing about that and downgraded to the previous Driver version.
I will reinstall the new one again now....after cleaning with DDU of course :D
Amazing! Awesome! You have gone above and beyond! Mad respect!
This is a great stress relief, since I bought my 6800 right around early December here in Ger.
Fantastic Work.
Danke Dir Diggah!
As a 6900xt owner I greatly appreciate you thorough coverage on this. On the upside of this I found your channel, great content!
Excellent follow-up video. Thank you for the tip about 'drying' a card bought from an unknown location prior to use.
Your theory makes a lot of sense. I have a feeling this will be happening more and more over the next few years with all the GPU bulk coming from miners, I bet a lot of miners are still holding on piles of GPU waiting for the right time to sell...
The right time to sell was the moment Ethereum mining ended. All of my GPUs are worth like 50% of what they were when the miners actually dumped them.
Miners underclock their GPUs and take care of them. I had 12 5700 XTs that had all been undervolted since they came out of the box. I bet the gamers that got those will have them for 5+ years.
Now try buying a used budget GPU from some gamer that has had it sitting in a box full of hair for years. See if the people who are making money off their GPUs (and have an financial investment in them and need to protect them) take better care of it than the average gamer. That's hilarious to me that you think Joe Public is taking better care of their GPU than professional miners.
I guarantee you that not a single GPU this happened to was a mining GPU (also because they launched after Eth mining ended). These are not GPUs anyone buys for mining anyways. They draw way too much power.
@@JamesAChambers let me play the devils advocate for a sec
Knowing eth will go proof of stake , would miners really still have cared about their cards lifespan rather than pushing them as long as still possible and then sell to some idiot?
Do gamers let their living room go icecold? Prob not, who would benefit from a bigger delta T? Maybe people running multiple high power gpu’s at once
Unless you buy from someone in Germany for example where energy prices are ridiculously high there’s not much incentive to go really efficient with your undervolt , especially since miners also had limited supply to gpu power due to them shorting each others supply
Yes often mining cards are run undervolted, but they aren’t treated like they are they only gpu and most expensive part in their work machine that has to live a long time, they are a necessity that can be squeezed out like a worker at Amazon
@@pfizerpricehike9747 Undervolting is critical. Mining has been unprofitable at normal electric rates for a long time.
My electric bill was $850 a month in January of 2021. Today it is $250 because all of that mining is turned off.
If any miners weren't undervolting they are absolute morons. That is money just left on the table.
People mine to make money. I made almost $100,000 from 2017 to 2020's bubble.
Those GPUs were my meal ticket man. They were cleaned all of the time. They were undervolted as much as possible. Anyone who was successful at mining did the same.
They gave me my home and my business. Of course I take better care of them then Joe Public. It was my job (not anymore).
The right time to sell was about one year ago...There is no reason for anyone to buy a SH GPU atm, with everything coming out, and the prices gone down for old stock.
OTOH we talking not years, but months, for a lot of SH cards to die...
@James Chambers Copium to the max my guy. Like all things it depends on the miner or the gamer in question. There’s some out there taking care of their hardware and then there’s plenty more flashing incorrect Bioses and running cards wildly out of spec 24/7 in dodgy climates. And then when it’s selling time they lie and try to act like the cards are new, case in point the video above. The death of ethereum mining was a great gift to the hardware market
That’s a careful thought and detailed video. Now is very clear the reason why of some cards it was getting damaged instantly.
Thank you, it make me feel good to see your investigation and i feel confident that i can go back to the newest driver. Excelent work.
I've got a 6750 XT...not in the 6800/6900 class, but it has had no thermal issues of any kind. It has been working flawlessly so far. Hopefully this issue others are having gets figured out. Glad there are channels like this one working on this. Cheers.
i have a used 6800xt on the way and wanted to do some thermal pad mods and repaste for good measure like i did with my 6700xt (which really helped thermals out a lot) but now i'm nervous about opening it up and seeing a broken die. looks like i'm gonna have to do some more careful research on it lol
@@musek5048 I'm no engineer but I think you'd probably know already if your die was broken. Hopefully you don't have one from the bad batch. As I understand it was a very limited number from a specific batch of silicon dies.
@@Eremon1 ok thats good to know, i'll have to do some light testing on it and monitor temps closely once i get my hands on it.
I was planning to buy a second hand graphics card but after watching this video i changed my mind. Thank you for the hard work.
well you can buy it second hand, just let it air out for 24 hours like he says before using it.
dont be a baby
the amount of people i seen say they bought a second hand 3000 series I just thought yeh ......-_- I can imagine them being washed before hand...
The only way I would buy a used card if the person was gamer lived locally and was the original buyer.
@@OLDMANGAMING1970 then be forever a slave to the market
I'm glad you put the time and effort into looking more into this issue.
I would love do grab a perfectly good working 6800xt, and try to "recreate" this issue, leaving them stored in a garage, cold and humid, for a couple of days, and then running these cards to check what would happen
Thanks for the inputs and research
hope it doesn't end up on eBay
I’m thinking more they were cleaned with water and air dried and sold. The boards still saturated with humidity. We will probably never know exactly what happened but still a good lesson in what not to do.
@@brianrobinson3961 i would think that would leave water spots on them, idk
More like couple of months. Mining on these cards hasn't been profitable since summer 2022.
@@Herbertti3 yeah, its possible, but not all people stopped mining at that point, some are still mining
Now that we know that the culprit is moisture, it might be possible to avoid cracking the die by avoiding high stress workload rightaway after you've bought a used card. For example we can subject the card to a drying period by simply idling it or using it in very light work loads. A more risky way would be to put the card inside a dehumidifier or oven at the lowest possible heat setting
Danke mister! Der videos are hugely pro, as I have repairman experience I can by my heart say that I can really imagine all the work that is behind! But - only imagine. Mad props for you and the team!!!
If the card is sealed, and so clean it's probably gone through an ultrasonic cleaner, and the cleaning compounds turn the thermal paste into cement - so the physical stress on the gpu die would have been incredible. I've seen the same thing happen on hardware that was exposed to very low temperatures, but never with chips so big.
Some used hardware resellers use dishwashing machines for "pcb stuff" like motherboards, GPUs etc, seriously ;)
Had 75% suspicion these were miners cards, there you have it!
problem is not that they were used for mining, its that they were stored in a humid and cold storage, and the chips "collected" water particles that when heated to 110c, broke the gpu chip
@@car.gems. miners are not gamers therefor don't care how the cards get treated, this the longer version of my comment.
@@abdulhkeem.alhadhrami you are absolutely wrong! Most gamers just play with their cards at stock settings (Without undervolt and stock fan curve). Miners use their cards undervolted, with high fan speed, to make sure their cards last to make a return on investment and so that they are not using a lot of power
@@car.gems. i will not keep replying after this i meant the physical aspect like cleaning.
@@car.gems. also miner's keep the cards in open benches in storage rooms or warehouses exposed to humidity. Also these cards get bios flash to increase their mining capabilities. There is no guarantee that the card will survive been flashed back to default bios and all the abuse it suffered.
Amazing! Awesome! It is really nice to see a complete troubleshoot.
Very interesting story, thanks for bringing it to the attention of the community and for so thoroughly researching it afterwards.
I was skeptical that the driver theory was the cause as like you pointed out in your video, it can’t change power/voltage parameters. The new theory seems much more logical and also tallies with the fact that it appeared to be such an isolated incident! Good general PC buying advice for the future though, thanks again.
6800 owner. Ty for the work. I’m a happy camper again
I was waiting for the results of the testing and i am glad that the problem is not in the drivers!
Asome work my Sherlock ! Love it keep up the good work
Man I really respect this guy for going through all this trouble. There are lots of things going bad with the GPU market and products that are due to NVIDIA and AMD. As such, I feel like most people's knee-jerk reaction to this is that they are going to blame AMD for the issues. It's easy to say that because of the whole thing with the 7900XTX Reference Models, but I am glad to see someone who is very methodical and takes a reserved approach to problems before coming to conclusions. Good job, and I'm glad to see that your sub count is growing. You definitely deserve it!
8:55
I Never use the factory fan curves anymore, I noticed that many cards will NEVER reach 100% cooler speeds and that is a big problem.
I would recommend adjusting fans to go 100% at 80º the colder you run your cards the less chance for problems, yeah it will sound like a jet engine taking off so maybe go for 100% at 90º depends on your card and noise tolerance.
Just so you know my 1060 fans start at 50º and max out at 70º, I know 1060 is a cold card anyway I've never seen it go above 65º with these settings. Before you say something I live in Brazil and I am poor as hell, can't buy a 3060Ti anytime soon.
thank you for the follow up. The cards are suspiciously clean, not even residue from thermal pads. My best guess would be that they were stored at freezing temperatures right after cleaning.
Tested my 6000 - series for more than 15 months now and I can tell ya, its a BEAST (for a good price) hehe
Why not tell us who the seller(s) were so that other people don't accidentally buy from them? In any case, great video - was very interesting to follow along
In Europe there is GDPR, you cannot disclose any personal data of anyone. If that seller is not legal entity, you cannot do that. And of course you cannot disclose in EU photos and anything about criminals like in US, etc.
Dataprotection laws, DSGVO.
German Laws.
he wasnt the one who bought them even.
probably ebay.
thanks Kris now im feeling better about my purchase in second hand market of a 6700xt that wasnt used in mining. now i know that the amd drivers are fine and can use latest safely
I remember that I saw a video of a stock of miner cards that were cleaned with a high pressure water cleaner. That makes sense that humidity is possibly involved in this case and why the card look relatively clean. Miner cards won't look that clean if not processed with a cleaning like i mentioned before.
look here: th-cam.com/users/shortsyVmetm1RwAM?feature=share
Thanks for taking the time to produce this! Very interesting stuff :)
I wonder if it was a scalper that ended up with stock they couldn't move for a while, had it stored in a non temperature controlled area and then sold cheap to recover costs?
Warehouses costs them a lot of money over time to keep stocks but I don't think they have that amount of products to hold.
Most likely kept in the basement of a badly insulated home.
Remember that throughout the Fall season of Europe, there were energy problems (due to the war) for a while where heating isn't constant, and moisture is sure to build up as a result of a lack of control over the storage condition.
The GPU dedective, thanks fort that!! loved , subscribed!
I have a bit different theory.
Miners flash firmwares and bioses to make the cards more efficient for mining, thise bioses and firmwares usually expand the Vcore limits so they can undervolt them via software and such.
What i'm thinking is that a bios/firmware with an expanded Vcore limit is still on those cards, being used for gaming without the proper manual Vcore managing could be causing them to use extremely high voltages and blow up.
Asa previous miner with some experience in the field, the modded bioses don't run out of the box on windows as they are not signed, you would have an error 43 in your device manager and the driver would not start! So these would be spotted right away! Also, speaking for myself, miners are tech-savvy, and our settings rarely overheat, i had 48x rx570's for example, running for more than one year with temps under 70, and out of those 48, i think only 5 mmessed up , and the problem was a bad PSU! yeah, your argument is done!
It still doesn't leave out the possibility that a 'bad' miner mismanaged the MBT power table and messed up the card. just because one miner or a group is tech savvy to maintain it, doesn't mean the rest of them 'cared' enough.
I mean you had freaking miners who made a video of themselves water pressure cleaning their cards....
@@dra6o0n Although I agree with you, those other ones were not miners, they were savages, and yeah i saw those videos and they gave me a mix of anxiety and OCD! So yeah, I "partly" take back what I said
After watching your first vid it cross my mind that if they had come from the same place it could be crypto mining but I dismissed the idea as I thought they'd come from lots of different people, I hadn't considered that they might have all bought them second hand from the same source. Awesome investigation thanks for making this vid as since I bought mine new it's a relief to know it's not a drive or manufacturing issue.
Reasons to not buy used card from dubious places +1.
Lately I am a bit hesitant to buy used GPUs because of mining. I think thsi hunch might have saved me money and nerves. For now I am sticking to brand new stuff. Thanks for this video!
Very strange indeed. Remember that there were some miners which even sprayed water on the cards when the crypto crashed. Also, have in mind that these card may have ran aftermarket bios i the past for better memory timings.
Over the past decade and half I've encountered a lot of shorted cards, but only NVidia(board vendors have no guilt for faulty designed products from the beginning), especially paired with Samsung memory.. disgusting. Ahh, yes, there are negligible cases of AMD/ATI cards served long without a clean, overheated from prolong or some cases of burned VRMs, mostly caused by faulty PSU. That's why these two companies(3, if we add Samsung Display to Samsung Electronics) are the benchmark for thrash.
p.s. same as my theory. From a miner, though well set mining conditions are using a third of the gpu TDP, so it runs cold. But bad storage and humidity.
This was an excellent and logical investigation. People are so quick to jump to conclusions with insufficient and often contradictory evidence. I have seen a bunch of videos recently that were talking about the drivers being at fault and it didn't seem to make sense. Keep up the great work, this channel has quickly become one of my favourites 😁
Thank you for clearing that up. My guess was mining cards from the start. There is video of dodgy cleaning techniques these guys use before sale and I will not buy used cards anymore unless I know the card has recently come out of a gaming system by a regular guy.
Thanks for this follow up. I just built a new system with a 6900 and was pretty worried about this. Guess I can go back to latest drivers.
Same for me !
Regarding crashes and driver timeouts, multi-monitor setup can cause this on 6000 series with newer versions of drivers
Conditions are
- not matching monitor refresh rates (60hz and 144 hz for example)
- enabled MPO (windows feature)
- Driver version newer than 22.5.2
And to fix it you only need to disable MPO
That describes my situation, I'll try to disable MPO, go back to 22.5.2 and use my two monitors with same refresh rates, thank you!
@@vinstonecold no problem! disabling MPO should make the latest drivers working
if you go back to 22.5.2, you can enable MPO again
and if there are any problems you can ultimately use 22.5.1 - the most stable driver for now, but it lacks some of the optimizations which 22.5.2 has
@@ZoragRingael I have disabled MPO, did I have to use the same refresh rate on both monitors too?
I'm using 22.11.2 driver
@@vinstonecold Nah, no need to change refresh rates. As long as MPO disabled and you don't have crashes, then everything's okay.
@@ZoragRingael yesterday I haven't got any crashes, everything is okay, thanks for your help!
Thanks for all the time and effort u spent on investigating this issue.
It could be that these cards were used in a cold climate and run at max (high temp) for an extended period of time. Then when the mining rigs were decommissioned, were exposed to a lower then normal temp. That extreme temp change could cause the silicone to crack or make it brittle. Like taking hot glass and pouring cold water on it. It shrinks too quick and fractures.
good theory, but IMOP that kind of issue happens when you have a fast change in temp. The equipment being shut down for even a hand full of minutes would allow a stock heatsinks to dissipate the heat without creating a rapid temp change situation that would cause the kind of cracking your theory suggests.
thx for your research and work !
Funny, I would have thought that people will actually realize sooner that the problem is where they got those GPU´s, kinda important information to state when sending you the GPu´s
48 customers, 25 responded, the rest didn't bother to respond. Only 2 has invoices and are original owners. LOL.
I had the same thing going on with my XFX 6900XT WB Zero, It would get driver timeout just watching videos, But it would game for hours on end, Did a ton of research and found out that its something to do with the ULPS Settings with the voltage, Downloaded Techpower up tools and did what the guy said to shut off, Just all the ULPS stuff and bam, It fixed it, Havent had a issue since!
Great work Kris. Thanks for your hard work in solving this problem.
I feel better for my card, knowing it was bought from a reputable store. 🙂👍
Thank you so much!!! I’ll switch back to the newest driver. I have been having strange crashes and events say it is share point failures (win 11 insider install), but I was worried it could have been the driver.
Can you desolder the Bios and Controller Chips from the broken cards and read out them and compare to working cards ?
Maybe it is possible to solder them on working cards and read them.
Awesome work so far 🦊👍
You don't need a card to read the BIOS chips. Anything that can read SPI flash (Raspberry Pi for example) is good enough.
Thank you for the thorough investigation. We owe you big time.
Great video. Thank you very much for all your effort. I'm no longer in a panic😄, now i have the courage to continue using my 6900XT
You are the best ! I learn so many things from you ! Thats amazing what are you figure out !
Maybe it's the TIM material causing uneven pressure on the GPU die. They harden at cold temperatures. I've only ever seen AMD use them.
thanks for your detailed deep dive!!
Goddamn, Kris, what an excellent investigation. You're the Sherlock Holmes of the repair industry. I can't believe people aren't being careful where they purchase their products from. If a deal seems too good to be true, it most likely is. I wouldn't buy such an item unless it were from a reputable vendor. It shows that if there's a buck to be made, people will take advantage of it. Thank you for all your hard work and dedication in cracking this case. Let's hope people learn from their mistakes and stop buying from shady sources. I can't stand to see crypto miners taking advantage of people, but everyone was warned about the incoming GPU flood when mining went bust. Live and learn, people.
why blame "miners" if the problems was where it was stored and not how it was used?
gpu clearly died after heavy load and temps, which dont happen when mining but while gaming. Another reason for the failure is the hard temperature swing between idle and 110c. Which, also dont happen when mining where the temps remain stable (no expansion and contraction) of the chip and pcb solder joints
@@car.gems. Those cards were all purchased from miners who hoarded all those GPUs. Yes, they stored them improperly, but that still doesn't change the fact that they fucked over all those buyers. Greed is greed; no if’s and’s or but’s about it. Miners saw an opportunity and took it. They're the ones who caused the sudden rise in GPU prices, and now that their gravy train has run dry, they're taking advantage of the market once more. It's fucking shameful.
@@st3althyone I dont get why is shameful to sell used cards. They bought them, used them, and now sell them. Its like buying a used car, not everyone treats the car the same way, anyway, all of them have the right to sell their used car to have some return of their investment. What are we supposed to do, dump all used equipment and increase electronic waste? Gaming cards have by far a much more stressful usage than mining, and gamers are allowed to sell them too. I dont understand your point honestly
And btw, its mostly nvidia and amds fault for the prices. Mining is long dead and we still have 2000 plus $ for the higher end on nvidia. They just care about profits
@@car.gems. I never said there was anything wrong with reselling a GPU. I have a problem with hoarding a product to then sell at an exorbitant price, or in this case, dumping a product after the market crashes and selling a product that’s been improperly stored/cared for. I guarantee the seller did not refer to where these cards came from, as most uninformed buyers won’t know what questions to ask. There’s absolutely nothing wrong with reselling a product, but at least be honest about where it came from and what it was used for.
Glad I've discovered your channel after the last video. Sub
I honestly have no idea how someone with your experience could come to the conclusion it could be drivers, it was clear for anyone with just basic knowledge drivers can't overriding bios.
Not only that, basically everyone wich was not a (paid) troll came to the same conclusion (wich surprice! Is the right one):
Those are ex-mining cards dumped in bulk on the used market.
The only questions you didn't ask in the first video was the only reasonable one:
Why not use warranty?
Why all this cards just locally, why not around the world if it's drivers?
Where those cards are coming from?
I want to belive it was a mistake caused by overconfidence, but still.
The work you put to rootcause this is appriciable, but it doesn't change the fact it could be all prevented asking some basic questions before escalating it online.
Well, you want more traffic to your site? Speculate and create drama. Just a theory of mine after some time to ponder.
Thank you very much!
hopefully you got your 5 minutes of fame.
Very interesting! I have a thought that maybe it was not a mining business selling those but some graphics card "scalper", meaning they were new old stock but stored the wrong way.
GPUs! GPUs, NICHT GPU's! Es gibt kein Plural-Apostroph, im Deutschen nicht und im Englischen auch nicht!!!
Ein Apostroph im Deutschen bedeutet: Hier fehlt ein Buchstabe! ("Wie geht's?")
Danke für die Bemerkung
Sorry, einmal Deutschlehrer, immer Deutschlehrer. :-)
Thank you for all your work and information. Have a nice time.
I can't believe i watched the whole video. Very interesting video. Just learned something new. I own the 6900xt buyed from AMD ,so don't need to worry. Weiter so.
hope this will help many customers to choose whisely not from a random person :/ or at least ask some questions and most important ask for warranty and invoice. Even if the scalpers/miners selling price is lower than the msrp (which should rise some questions) it's still represent a lot of money... the prices are much higher this time than 3-4 years ago.
Thanks KrisFix for the well done explanation, this "thing" with the moisture should apply on every electronic cicute/element... Also I think it is kinda strange to buy a second hand pc part and to see there is no dust at all ... this means that it was washed
Well, they are used cards which are not worth half msrp anymore. There has been a massive dump of this type of used garbage ever since crypto crashed. These ones were no doubt mishandled by miners and then sold on ebay.
amazing work detective KRIS :) ...criminal humidity caused the damage on gpu and the drivers are innocents ... thank you so much for your work and time .
I don't think it was "humidity". Someone cleaned this cards in a dishwasher!
Wow learned something new today thank you sir.
As soon as I saw this issue, I assumed it had something to do with crypto mining, but I didn't want to make claims as I do not have hands on these cards myself. Either these cards were kept in an unbalanced temperature environment, or the person that had them before flashed a custom vbios to optimize their hashrate, and did not flash it back. Combine the custom vbios (likely SoC, and Memory controller voltage changes) with stock fan curves, and it all makes sense now. If you wanted to check the vbios of the cracked cards, you could always pull the vbios chip and dump the bios with a programmer, and then check the vbios via MPT and compare it to a stock dump (either your own dump, or a stock dump from TechPowerUp). I had a 5700XT that I bought online that had a shorted 1.8v rail. After repairing it, I noticed that the performance was worse than stock (this was a Red Devil 5700XT), and worse than reference. I dumped the vbios and sure enough, someone randomly did a bunch of MPT changes, and clearly did not know what they were doing. The only thing that seemed proper were the memory timings (proper for mining at 60MH). I have a feeling that we're going to see a lot more of this from AMD cards. MOST nvidia cards are not going to be vbios flashed, and will likely either have dead/dying memory, or memory controllers.
Hoof, thank goodness. I just bought a 6800 xt and got scared thinking it could die at any moment. Mines new from the store, not second hand.
The moment you said: from the 25 reactions, only two had an invoice, I knew enough.
This is why datacenters and telecom outfits run a standard dehumidifying cycle on all incoming equipment.
Especially when the equipment spent time at sea during their transport, they will go through highly varying temperature and humidity levels for weeks.
And with mining cards, some of them are run in areas with moderately warm and moist climates in an environment without climate control or anything of the sort, as that's unnecessary for their purpose and only takes away from mining profits.
Thank you for all your investigative work
What a legend for discovering the reason..a good warning for storing GPU’s..
It was clear from the start that it had nothing to do with drivers. Still it didn't stop some channels from making a bunch of clickbait videos. But yeah, everything else (mainly the time period when broken cards started appearing and the problem not being world-wide) shows that all these cards came from a single place (likely a miner) and had something done to them which caused them to die.
Remember seeing the video floating around of GPUs actually getting hosed down? Would explain why they are so clean and contribute to the moisture problem.
excellent research man, you did the community a great service! hats off for you
Mentioning the fans running really low. I have an MSI Suprim 3090Ti and the fans don't even start running until the card reaches 70C. Had me worried the card was faulty as that seems wierd as all hell to me, but MSI's tech support said that's how it's supposed to run. Tried setting custom fan speeds in Afterburner using the on chip memory setting that it has but it doesn't work unfortunately and the fans only run at the speeds I set while AB is open (which isn't the end of the world).
Just odd to me that the fans don't even start until the card is that toasty. I guess it's because some people don't like fan noise for whatever reason.
Thank you for following up on your findings. I was nervous about my card leading up to this video but i think I’m in the clear. You and your team did a great job and that gets a subscribe from me.
can you add the coast at end? that would be very interessting. Thank you, greetings
Phew, thank you for the update! Time to reinstall the latest drivers again. Not all has to end in drama, I'm happy in this case it didn't.
It blows my mind with how many youtubers took the whole driver was the problem and was breaking the cards and ran with it with no other research or attempt to make sense out of it when he legit just said the most of the cards were on the newest driver and he doesn't know if it is the driver's and to share if it was happening anywhere else like holy cow there were so many if them it was crazy.