NVIDIA Tesla K80 Cheap & Easy Desktop Cooling
ฝัง
- เผยแพร่เมื่อ 1 ต.ค. 2024
- I recently build a new K80 machine and found a much cheaper and easier way to cool this server class GPU in a desktop machine. In this video a use some $16 fans and plug them right into a motherboard fan header. This is much easier to do than my last video where I use a bigger fan and built a custom fan controller.
My previous video when I build a single fan setup is here. • NVIDIA Tesla K80 Cooli...
This video is not sponsored, opinions are my own.
Some links
Fan www.amazon.com...
PCIe to 8Pin CPU Adapter www.amazon.com...
iFixit Toolkit www.ifixit.com... - วิทยาศาสตร์และเทคโนโลยี
One thing to be careful with is the current limit on motherboard fan headers. With a pair of 4.5A (9A total!) fans on one header there's a very real chance that they could burn out a header entirely
9 amps would kill the fan header for sure. You are an order of magnitude out. The fans I am using are 0.45A so in total 0.9A is drawn from the header for these two. This is within spec for a single fan header to supply, generally they can supply around 1A. My motherboard has 6 fan headers only 1 other is connected so its not a huge concern for the board as a whole. The other point is these fans are PWM controlled and are NOT running at full speed, they are at 60% or something under full system load, I've not measured the current but my guess is they are not close the to 0.9A maximum.
@@robingrosset6941 my mistake I thought it said 4.5!
Thanks! Good job! NVIDIAs has specified a 26 cfm flow for the original channel configuration. Thus, as your test showed, two 76 cfm fans in this new configuration should be more than enough leaving margins for flow control. But what about air recirculation? Especially if you are blasting 300 W in a cramped computer?
where did you get this info?
Yet another question: Would a pair of 120mm fans also fit? (Larger size, so don't need to spin so hard and are therefore more quiet)
I tried this. 80mm fit well as do 90mm. 120mm case fans are harder to fit and you need to figure out how to mount them. They do move more air than 80mm at the same speed and are generally more quiet but the 80mm ones I was suggesting are pretty beefy and at normal profile are quiet, but move a lot of air. Check the current rating on the fans as its a good indication of how much they can do. Most case fans don't move that much air.
Hi Robin, I just purchased one of these to support 3D animation rendering. How suitable do you think it would be for the task? Have you tested video/graphics/animation production on this card? Cheers.
It would be good for it. Over 4000 CUDA cores and tons of VRAM. Not the worlds fastest thing, but so what!
2 x Nvidia Tesla K80 + 2 х E5-2650L V2 + 128 Gb RAM + NVMe SSD SWAP + 2 x 960 EVO 500 Gb + 4 x 1 Tb HDD. Profit!
Lol. Curious to know what you're doing with all that kit to profit? :D
@@peacekeepermoe I rent it to an astronomer who calculates the gravitational dynamics of star clusters by processing data from the Gaia-2 orbiting telescope.
10 euros per hour and he is very glad that it is so cheap and only he uses all the resources, unlike the data center of the same Google.
@@Saber_Toothed_Rus Wow, that is quite interesting! Does he rent it via network or does he actually have the machine with him?
He is my friend and I brought him and parked the car at home. If necessary, I come and provide service.
From the machine to his PC, there is a double 10 Gbit cable over copper and there is no problem of access speed. The server board has a built-in x540-t2 card and the same separate one is installed in its PC by me, a cat7 cable from the factory.
Even with such a configuration, its calculations run continuously for 200-300 hours. And in his free time, he rents out a car remotely for video rendering and calculations, so he pays rent for me and has some income himself.
I realize these don't have output, but what kind of gaming perf u get on this? Purely for shits and giggles.
From the GPU, why does it split x1 8 pin -> x2 8 pin, but then, x2 8 pin -> x1 8 pin? A clarificaiton would be appreciated. How is it connected (in detail) from power supply to GPU?
Hi Robin 👋
How’s the cooling been holding up after a year?
I wonder if a pair of 120mm fans would work better still than those 80mm fans. Or even going just slightly larger with 92mm.
Would this card be good for AI training applications?
Does rtx 3060 and tesla k80 work together on the same system?
No reason why they should not. I have had a K80 work with an integrated GPU in the CPU or a discrete GTX Titan or GTX 1080ti. It should work
Could you look into these cheap universal clip on video card coolers on Ebay, mostly from China.
I might take a look as they appear pretty cheap.
@@robingrosset6941 I went ahead with this configuration since it provides even cooling of both GPUs. Solutions that drive air through the closed case don't -- and that includes the standard server chassis and ebay fan configurations.
How can I get my K80 to run in a HP Z840 Workstation? I want to put the CUDA cores to work!
It’s got a 300w TDP, I’m not sure this will quite cut it.
I have HP Z440 workstation and Quadro K2200, want install Tesla K80 on this machine.
plz help me what settings can does run Telsa K80 on my machine
Great video! What tape are you using (that shiny silver one in the video)? Is it some sort of heat-resistant, non-conductive tape?
It’s very conductive tape :-) it’s aluminum foil duct tape. Used in HVAC assembly and repair. It’s very strong and shapes to the fans and heat sinks. There are no electronics to worry about on the heat sink or fan body. Definitely heat resistant and good for high temps
@@robingrosset6941 Hello, good video, could you run blender tests, I want to buy such a card but I can’t understand if it will work with BLENDER.
@@ХолоденкоВладимир Margaritas for all
My power supply has 3 cables for PCI-E power, each with two sets of connectors.
Hi Robin! I see you know the Tesla K80 pretty well... I wanted to try to build a crypto mining rig with them... do you think it can work?
I am not a crypto miner at all. I think it could work. The main use case for these cards is building large machine learning models. The real unique about these cards is their RAM memory is massive compared to most GPUs. They are several generations behind the latest GPUs so consume more power/watts and are slower for the same amount of work. For crypto mining this perhaps mades these cards a bad fit. I think a bunch of lower spec but more recent GPUs might be better for mining.
Go to en.angryminer.com/ and enter Tesla in the search engine and miner results are not great $33/month for the K40 745Mhz and the K80 $3.91/month so that changes all the time but these cards are computation monsters that might do better in a server than a pc but they underperform all 8 gig video cards.
I use my GPU mostly for machine learning. lately my models are too big for my GTX 1070, but thanks to the pandemic I havnt been able to upgrade to anything with more VRAM. Do you have any expirience with the K80 in PyTorch? I have been thinking about getting one as they are fairly cheap.
Did you ever find any info on this? I'm wondering the same thing. Particularly, I want to know if CUDA will spread the load between GPUs or if it will limit it to one of two GPUs on the card?
@@jeffreybowers5646 I abandoned the idea as it would require modding the GPU to add server fans and we were able to acquire some prebuilt machines with rtx 3090's for msrp. Also from what I understand it acts as if it were two gpus but dont quote me on that. Good luck
For those who are still interested. I use a Tesla M40 (~100dollars) to train 24gb models. It's slow, but works out of the box under ubuntu w/ proprietary nvidia drivers and pytorch. It's slow though, only about 1/3 the performance of my gtx 2080, for about the same power draw, but the additional memory makes it up for me. If you're actually doing this professionally, something like the new amd rx6900xtx would probably perform much better, if you can get it to run pytorch, but if you're like me and you just want to play around with large models for the lowest price of entry possible, this is a great alternative. Just keep in mind that even underclocked and underpowered, these cards still need cooling. But if your cooling setup is a bit short, these cards throttle before hitting critical temperatures, so again, if you don't care about performance, even insufficient cooling may suffice to run one of these cards at reduced performance.
Hello, thank you very much for this video. Can you give the motherboard model with number, that you use for the K80.
Would this work for an M40 as well?
Is this solution better than the previous one? I think the previous one will keep the GPU much cooler than this one, correct me if I'm wrong.
I used a riser like yours and my card started sparking on boot and no longer works, so be careful about that
That is most likely the power connector. You need make sure you use a CPU 8 pin connector and NOT a PCIe connector. Its not a normal GPU. Most likely it will kill your PSU or the cables. The card usually survives being plugged into a PCIe connector instead of a CPU connector as it shorts 12v rail to GND, so the PSU will go into protection and not start. If your PSU has short circuit protection it will work once you get the right cable.
@@robingrosset6941 I have had the same problem until I found out that I need to use the 4+4 CPU connector and not the PCIe one.
My question is.. do I need two PSUs? Because if I use the 4+4 for CPU then I don’t have another one for the GPU, or maybe can I use Molex or Sata to something?
Please advise, many thanks
Hi Robin,
I have been having trouble trying to install a K80 in Ubuntu,
I install the drivers but when I install CUDA it messes up the Graphic Drivers
By default it goes to Cuda 11.1 when having 470 as the drivers.
I tried downgrading the drivers and having Cuda 10.1 installed.
And again, no dice. It gets messed up every time.
I should mention that I am trying to install it in a HPE DL380 G9 with Ubuntu 22.04.3.
What is your exeprience with POP OS what version would have an easy driver install and Cuda?
Would appreciate a feedback.
Best regards,
Hi, did you find any solution for install a k80 in Ubuntu ?
Thanks for the post. Specifically, wht kind of tape are you using???
Would a fan shroud still hold? I probably print a holder for all of it anyways, but I wonder.
what temperature does the video card reach? Could you tell me where to attach the fan attachment on the motherboard above the video card? can I also do this with a Tesla M40?
PCIe to CPU connectors? Or GPU?
You need TWO PCIe 8pins to one CPU 8 pins. This does not accept normal GPU 8 pins.
Stupid question: How do you get the motherboard to know the Tesla K80's GPU temperature so that it knows how to adjust the fan speed?
In this version the motherboard does not know the GPU temperature it just sends a standard fan speed setting to the fans. It’s enough in normal fan speed profile to effectively cool the K80s. You can up the fan profile to high to increase cooling but it does not seem necessary. The normal cooling of K80s was passive cooling in rack mount servers so passively cooled from server air flow.
Would a "quiet" fan (of the same size) work? The fan you linked to seems to go up to 4000 RPM. That's pretty high compared to other fans I see. There are "silent" fans that go up to 2000 RPM, for example. Are they not powerful enough for K80?
The difference is CPU fan vs case fan usually. The amount of air flow with the fan I used is definitively enough to cool it. Case fans with lower RPM don’t tend to move enough air.
Cool, I've got a new system now as before my old pci gen 2 wouldn't possibly support it shame it had 128gb of ram a daul 6 core xeon CPU which are quite good, got a newish B-450F ROG board to handle the job and my K80 came today in the post. I will be using this trick, thanks for the update. just waiting on a few things to get setup.
Good luck with it...!
Thanks for the clear video. I like the fan setup... I wonder if there is a possibility of using it in an e-gpu setup (running with a MacBook on bootcamp). My only concerns are the 4 G decoding requirements. Im not sure how to get around that without the ability to set bios). Any ideas? Might have to resort to getting a Titan X
Have you tested it or found a better solution?
Is putting watercool size 240 AIO to tesla M40 on a custom crafted bracket a stupid idea or does it sound reasonably?
A 240mm AIO if you can get the cooling plate attached to the GPU in the right way would be enough to cool card for sure.
@@robingrosset6941 thanks
Hi there,I was wondering is it true that K80 need DDR4 motherboard?
You need a motherboard that can support above 4G decoding and is made after around 2014 to hopefully be able to support a K80. This likely means you are looking for something like a X99 chipset which is DDR4. Server class motherboards or ASUS workstation class motherboards with PCI-E x16 slots are your best bet.
If I don't use fans will it just over heat even if I have a radiator and many fans
It’s designed to be in a server case with air flow pushed through its fins. It will boot with no fans but when you start to use it does generate heat and will thermal throttle without cooling. If your case has really good airflow it might be okay but I would position fans near it and blowing air in the right direction. Removing the outer shroud might also help depending on your case fans.
Nice work
Thanks!
Great video! Super helpful, saved me so much time and hassle! Thanks, Robin!
P.S. I see you are using foil tape here, would Kapton tape work as well? Thanks.
I have not used Kapton tape for this. I find it’s not that sticky compared to 3M aluminum tape. Just be careful with the fans falling off. You need to make sure it’s really on there as it’s warming up and cooling down a lot.
@@robingrosset6941 Okay, thanks for the tip!
Спасибо парень очень хорошая идея
What kind of tape did you use?
I used aluminized high temperature duct tape. 3M make it but you can get generic versions too. Reason is it sticks really well and does not stick less when heated. Careful to just stick on heat sink and not the board as it’s conductive
Do we need another card to run a tesla or this will do it by this on
You need a video output so any other way to get video out is fine including integrated graphics or separate GPU card .
@@robingrosset6941 ok there is a specific one, ou can be any one,,,, what one do you use
I have used an NVidia GPU like a GTX 1050. Or I have a motherboard with integrated graphics which uses Intel graphics. Both work fine.
Hi! What use do you give the tesla? I have a k20x but im thinking about sell it becouse i cant find something useful for me. Any suggestion?
Mostly I use this for machine learning workloads using TensorFlow. The memory size makes it really good for training larger models. I have also used it recently to contribute to the Folding@Home project. These cards are getting older now and the only advantages over other GPUs really is the RAM and the fact they are super reliable. The latest RTX 3080 for example has only 8GB, compared to one of these with 24GB shared between 2 GPUs.
@@robingrosset6941 I use it for holding home too.
@@robingrosset6941 stupid ML/DL question: can you use all 24 gb at once or can you use 12 gb max but in paralel across 2 gpu’s?
@@BeKindPlox 12GB per GPU, You have to use both GPUs to use all 24GB. Typically people would use all 24 GB by doing something like running hyper-parameters searches with the same model but different configurations on each GPU. (or something like that)
@@robingrosset6941 Thanks!
Robin.. How can I get my K80 to run in a DELL Precision T7500? I want to put the CUDA cores to work!
The trick is usually to enable 'Above 4GB Decoding" in the Boot options in the BIOS. From looking at the Precision T7500 manual it does not appear to have the option. I would try updating the bios... see if you can find something related to 'large BAR support' or 'above 4GB decoding' in the bios.. thanks
@@robingrosset6941 Silly question, but.. BAR stands for? Also I see neither of those in the CMOS. I have the latest BIOS, A18. Is it possible for me to edit the BIOS file before I flash the system?
@@epgiovannini2904 From PCI config on wikipedia: To address a PCI device, it must be enabled by being mapped into the system's I/O port address space or memory-mapped address space. The system's firmware, device drivers or the operating system program the Base Address Registers (commonly called BARs) to inform the device of its address mapping by writing configuration commands to the PCI controller.
Unfortunately I don't know with that motherboard how to get past this issue.
@@robingrosset6941 Robin; I have a Tesla K40c that runs in my T7500. Question: Can I flash a K40m with my K40c BIOS and have it run as a K40c? It's basically the same card, but the K40m has no fan. It shows in Device Manager but isn't accessible to the system. Last time I promise!
@@epgiovannini2904 Hi. Sorry to say I have no idea if that would work. If it does let me know.
what pci riser is this
Hi, I have had this riser for a while so had to look it up. I think its this EZDIY-FAB riser. Works great.
www.amazon.ca/dp/B08KT3YSXF/
@@robingrosset6941 awesome man! thank you! i just bought my first k20 kinda been finicky on what i can run it on lately
So quiet lol.
Its a relative thing.... th-cam.com/video/I2zYOdfASFE/w-d-xo.html
@@robingrosset6941 Compared to blower designs, it's quieter.
It is not A Zeus.... It is A SUS (ASUS)..... as in a suspect, A sus. Not the god of lightning, Zeus... LMAO!!!!
ASUS made a video on how to pronounce their name and it’s “EY-soos” Here you go th-cam.com/video/1d0IyLyXCag/w-d-xo.html
Don't use tape to hold fans onto anything. When the card gets warm, the adhesive will fail.
This is 3M metalized tape with long lasting acrylic adhesive used in HVAC applications specifically designed to work in forced air heating and cooling applications. The original video I posted 2 years ago using this is still running fine and has been under heavy load most of the time. The temperatures reached in a computer are actually lower than heat ducts near an HVAC furnace where this tape is designed to work. www.3mcanada.ca/3M/en_CA/p/d/v000150843/
This video duration is only 4:44 and you say "a" or "em" or "am" 62 times.
Damn get rid of that bad habbit, it's pretty anoying.
Write a script and read only that to avoid such.
make a constructive suggestion and skip the criticism. have you gone to Robin's level of effort to help people. most people including myself have this trait. you count the ems and i count valuable wisdoms from Robin.