IT'S WORKING! - Asus ESC4000 G3 Custom Power Cables
ฝัง
- เผยแพร่เมื่อ 19 ก.ย. 2024
- Grab yourself a Pint Glass, Bottle Opener, or even a Wallet at craftcomputing...
A couple weeks ago, I introduced you to the Asus ESC4000 G3, a 2U Server, and the Cloud Gaming Server of my dreams. Today, it's finally working, with a full set of custom GPU power cables, and we're going to go over how they were made, and what's in store for the server in the immediate future.
But first... What am I drinking???
Drop Bear Brewery (Eugene, OR) Wile Ryly IPA (7.2%)
Pinout for the EPS-GPU Cable - drive.google.c...
Links to items below may be affiliate links for which I may be compensated
- Server and GPUs -
Asus ESC4000 G3 GPU Server: ebay.us/ngCIDO ($175 + shipping!) or ebay.us/hAThwr
Asus ESC4000 G3 Sliding Rails: www.neobits.co...
Intel Xeon E5-2697 v4 18-Core: ebay.us/6nlzsD
8x32GB DDR4-2400 REG-ECC Memory: ebay.us/4DNJ00
1.92TB Patriot Burst Elite SATA SSD: amzn.to/4b8Q9jc
Nvidia P100 16GB: ebay.us/OYEgUn
Nvidia P40 24GB: ebay.us/1FurAp
Nvidia Tesla V100 PCIe (Not SXM2): ebay.us/t1vg8h
- Cable Making Gear -
iCrimp 24-16 AWG Crimp Tool: amzn.to/3UDvflL
JRready Pin Extraction Tool: amzn.to/3QI1iQt
Molex Mini (Female): amzn.to/3JYXPsT
4-Pin EPS Extension 4-Pack (Need 2): amzn.to/3K0Eibv
8-Pin EPS Extension (Need 4): amzn.to/3UXufKB
8-Pin EPS to 2x8-Pin PCIe (For AMD/Consumer GPUs): amzn.to/3QL0J8m
Follow me on Mastodon @Craftcomputing@hostux.social
Support me on Patreon and get access to my exclusive Discord server. Chat with myself and the other hosts on Talking Heads all week long.
/ craftcomputing
have you considered feeding your camera more often?
I give it all the glass it wants... fill its batteries daily... I'm just not sure what more I can give.
@CraftComputing you must give it the souls of new born children....... kidding I wonder if you thought you hit the record button when you didn't. I know I have done that
@@CraftComputing This fluke was a blessing in disguise if you got to taste that fabulous beer twice. Is the beer of the day were a dud like last time would you still pop open the same one to re-shoot the video?
Loving this series on data center GPUs. So interesting to see what these older cards can do!
I would really appreciate some local AI benchmarks as well. Tasks like benchmarking Whisper or local LLM tokens per second would be great. A lot of us out here are wondering how well we can run AI at home on these cards.
Poor camera, eating the footage
Yum
Hopefully the footage wasn't so old it was covered in film. 😎
Hopefully the camera is full for the next while
In reference to the male/female designator, you are close. The male is referencing the pin itself and the female what the pin slides into. The plastic around these is called a shroud. So you would say, "a shrouded female connector. " No judgment, just more knowledge! I love your videos! Keep 'em coming.
Looking forward to a future video where all of Jeff's crimpers have been laser engraved with "Meter twice, crimp once."
Measure twice, leave slack, crimp once.
I see a craft upload about servers, I watch, I enjoy and I like.
Awesome!, Looking forward to the GPUs video btw, keep it up!
I’m so stoked for the GPU benchmarks, can’t wait
Im extremely excited for the enterprise gpu results. Thank you Jeff for investing in getting this knowledge so I can spend a lot of money on thiz, but wisely (?( lol
Kizakura Kuromame is my favorite beer. It's a black bean beer, that with certain foods or a pallet cleanser gives an amazing flavor. It's pretty hard to come by here in the US, but there are a small number of select grocery stores that have it, or you can find it in Kyoto's Kizakura Kappa Country if you're in Japan :). They also have a selection of other beers, an IPA, golden ale, and I forget the other (maybe a blonde?). Also, Sapporo Black is decent, but the Sapporo Gold is better imo. I'm less of a hop-heavy IPA (I only like conifers in my gin, thank you much), so these Japanese beers are much more clean and smooth drinkers.
If you want the true experience, just fly to LA and go to Ootoro, spend at least $300, and weep for the lack of divine cuisine in the US of A.
Algorithm is as hungry as your camera, so take this comment.
Thank you for going through all the troubles testing this out. I would love to have a similar setup for my home lab. I choose to do it in 4u instead of 2U as that makes cooling at reasonable noise levels more doable but I love using old server gear for it as it just has so many more features than consumer gear. I am not saying that everyone needs 128 PCIE lanes you get with EPYC but only having 24-28 lanes on consumer sucks because after you add a single GPU and storage you are basically done as far as expansion.
Hey Jeff! Keep up the good work
"I'd rather RAID all those together"
I just said "my man", out loud.
I would be very, very tempted to ARC/L2ARC max this setup and see what it'll do.
I just ordered the whole setup, less some video cards, not sure what to get just yet. I'm rather new to all this other than the beer drinking, so I hope this goes well for a new host machine.
ASUS did something similarly obnoxious on the RS series servers. The "E9" generation had an 8-pin VGAPWR connector that looked like
GND GND GND GND
GND 12V 12V 12V
while the "E10" and later generation had an 8-pin VGAPWR connector that looked like
GND GND GND GND
12V 12V 12V 12V
*with the exact same keying* (which is also the same as EPS12V, but polarity backwards). You can see where that goes. If you go out and buy a surplus "ASUS GPU power cable" without doing your research... you short GND to 12V very nicely.
probably splitting hairs here ;-) but using two 4 drive z1 vdevs should deliver better performance. less resilience tho if two drives in the same vdev dies
Even better to use striped mirrors - IOPS are limiting here, and each striped vdev adds iops, where raidz reduces iops
Looking forward to the data that comes out of this!
Asus carrying on the dell tradition of driving IT geeks insane by changing the bloody pinouts of standardized connectors.
This is just server stuff being server stuff. Standards go out of the window in a lot of server internal parts, risers and cables
The only channel about computers that recommends that you should start drinking like a pro. 🙂 I do not know if that use of words will make you a alcoholic very soon.
considering the price of the stuff he drinks, not really. He needs to keep most of the budget to buy old server stuff
""either way, there's a whole lot of coupling going on"" I LOST it 🤣🤣🤣🤣
The light in one of the buttons on the arcade cab in the back is out
Yeah, and the monitor is dead :'-(
When you test the tesla v100s, you should test them with and without SLI.
Edited because of a stupid typo
I hope you get into more detail how you passthrough the video cards. I have seen so many ways of how to passthrough gpus with proxmox and proxmox has added way more features and I don't know what the "correct" way is anymore...thank you!
ramdisk maybe in addition to, or in lieu of, some of that local storage since you have 2x 8 slots? Likely deals with the random I/O from games better than ssds over sata. Don't dispose of the 8x ssds though; keep them for persistent non-networked storage. Use ramdisks temporarily during a gaming session. Dunno. Just an idea.
Albeit ddr4 is not cheap, I know. Like $600+ bucks for 512gb kit (8x 64gb). At least $1200+ for dual socket is hefty.
Funny - Supermicro also seems to use a cable that doesn't exist for their GPU headers, at least in my X11 platform. They use what seems to be a standard 8 pin EPS, which you think would be great for these Tesla cards, but the pinout is reversed from what it should be! I had to make my own cables as well since the cables to actually make it work don't seem to exist.
Hey mate, I have supermicron X10 platform and in search of these cables... How did you figured out which cables goes where?... Can you point out how you did this... Some tut or diagram will be much appreciated.
Thanks
This is awesome! Thanks for making videos on this 😁 one day I will end up with a server like this 😂
great video where is the written instruction you promised ,i thought the video will be how you made the cables
I love the idea of a test suite for enterprise cards. Just please, include some AI stuff. Text gen web UI and Stable Diffusion. I'm not an AI fan, but it needs to be run locally or the big companies will steamroll everyone and take control.
Just the video I've been waiting on
This is exciting! Would you mind testing some consumer cards in this context? I'm starting to amass a collection of last gen budget GPUs (3060s, 1660 supers, Vega 56); if you have the cards, how do these perform in a virtualized context compared to an enterprise GPU?
Well, what do you expect? Coupling happens in the spring! Lol
Yeahhh! Solid video as always!
How loud is the server under load tho? I can't imagine it's quiet, but is it super loud?
waiting for that all gpu benchmarks!!
Could you give each VM access to a shared storage on HDD where you could bulk store games that are not being played? Then when you need a game you just move it to the VM drive that's on SSD?
For your temperatures, were you using the “air ducts” described on instructions under the lid ?
6:04 it's way easier to add a new disk to a raid than it is to restore a whole system from backups
Wonder how adding a sound card would pan out. Kinda doubt they would be useful
Thank you. This looks interesting.
Wondering if I could do this same-ish set up with a Dell T7910. So same cpus. Same ram and 2 Tesla P100s?
Any input is welcome
Hi how are you finding these Patriot burst 1.92TB SSDs, I see you using them in this server for storage. I need some ~2TB SATASSDs so far using intel d3- S4510 but they are expensive compared to the Patriot burst wonder if it's worth the price difference.
03:23 If the server is a rockin' don't come a knockin'?
ASUS has the best warranty. just don't get it scratched
it's used stuff bought second hand, warranty is long gone
You don't have RAID setup on your camera?
I just got mine put together! 2 X 2697v4 and 128GB DDR4
I haven’t got unraid on it yet (migrating from a previous 2667v2 based rig) but one thing I have noticed compared to my other 1U server: 1) this thing is LOUD and 2) the second CPU runs quite a bit hotter than the first… like 10-20 degrees hotter despite the airflow ears trying to get fresh air to it
you know somethings about nvidia licensing appliance u need right for the newer types of cards?
else hit me up if you need some help
You could probably sell thoose cables. Also since when do you have V100
You can't afford me to make cables for you.
And since about a week ago 🤪
@CraftComputing true, and I don't even own that server, intact I only own hp servers(rack)
It takes 20 minutes to make "properly" with proper tools or less time and no money if you go full gorilla mode and just cut them and splice the wires with a wago inline splicing connector. I hate crimping this shait, I cut and splice.
I have the ESC4000 G4 and the cables are different than G3 I need these GPU power cables for 2 cards I will pay top dollar.
Anyone with this server successfully setup PCIE bifurcation? I've updated the bios/firmware and its still not present as an option (at least that I can find).
Did you go with RAIDZ-2 for this server?
Yes, that was the final config I settled on. 8x1.92TB in Raid-Z2.
Shame those aren't a more common server I could add those to the GPU power cables I make.
Hmmm... what are your thoughts on Skylake-SP vs these old Broadwell systems? I notice that some Skylake-SP processors are nearly as cheap as Broadwell Xeons now so am kind of thinking that its time to move on from Broadwell era.
@Administrator8497 Skylake server platforms are still quite expensive. Obviously there's going to be more performance than Broadwell but is the power consumption any better?
This is crazy cool. :-)
imo it's almost always easier to build your own cables than depin those damn things
I cut them and use wago inline splice connectors
Do you have a wiring diagram for pinout or instructions?
there is a link to the pinout for the cables in the video description
Now make a high availability cloud gaming cluster 🥵
The male-female designation goes like this...male goes into female, duh. But you can have male terminals in a female plug and vice versa. One example of a female terminal in a male plug is the ATX power cable, and the hard drive molex connector on IDE drives.
Gonna tell ya, those terminals SUCK ASS. UL requires a pull of 35 lbs on 16 ga wire but those terminals have a tendency to fail unless it's crimped for a 45+lb pull test. One thing I recommend is to put a dab of solder on the crimp, it helps especially if you're hand crimping.
Sauce: I worked in a wire mill for 10 years.
What is going on with your new studio?
Long-story short, still happening. Lot's of back and forth with the city, then a change of plans, and then another change of plans. Exploring a modular prefabricated structure right now, and hoping the city will OK that. As soon as I have either a contract in place to purchase or a building permit, I'll update.
lots of fun with paperwork and fighting local government obstructionism
nice
a dollar to troll? what a deal!
I gotta ask, for your GPU VMS have you found a workaround for anti cheats or are they limited to just non anti cheat enabled games
Asking cause I have GPU PT VM on proxmox
Some games allow VMs, others don't. There are a couple workarounds that are effective some of the time, but is it worth a ban risk?
@@CraftComputing Coverage of these paths would be helpful.
Jeff, what OS are you running on the Asus GPU server? I am picking mine up this weekend, going to use Tesla P40. Was thinking of Proxmox, but Ubuntu Server and just build in virtualization.
Proxmox v8
Conspiracy theory: you didn’t lose the footage, you just wanted an excuse to drink that beer again
Worth it
Camera: om nom nom nom
Male/Female is always looked at from the metal pins, not the housing. Think, what needs to be stiff....
Can I do vdi with quadro p4000
Not officially, but yes with the vGPU unlock script: github.com/wvthoog/proxmox-vgpu-installer
@CraftComputing is there a way to get around rebooting server after 2 days? Something about the grid driver.
Make a AI server running Ollama using some of these industrial GPUs
Just in time for the great ASUS beat down of 2024.
Hello!!!
General Kenobi!
RAID is a backup.
raid is a backup if on campus is offsite :)
A RAID allows you to recover a lost disk.
A backup allows you to recover lost data.
A RAID that has failed to the point of losing data will not allow you to recover it.
RAID is definitely not a backup
XD
everything is a backup if you are reckless enough
We dont buy asus anymore.
used is fine
it's an ASUS, no wonder it broke
Gender-fluid connectors???
please god no
"why would Asus do this?" - I guess you aren't up to speed on current events when it comes to Asus...
why would the warranty issues would be relevant to cable pinouts
great video but nah not gonna support the scammy ASUS their RMA is attrocious... So sad to see a company go down the drain... when I grew up they were considered the best on the market
You are not supporting Asus at all by buying one of these servers from an ebay seller. Asus makes zero from the sale.
this is used hardware dude
How about we just forego the gender vocabulary and call one end the “pin” side and the other the “receiver” side?
how about we don't