Its nice to see a holistic explanation of designing / building / installing a complex multi-rack system...As someone that has spent years working on both sides of the "analog/digital divide" (physical data center world / digital world's various segments), the un-sexy physical aspects of available rack space / power / cooling / floor loading / network uplink bandwidth are often overlooked (often assumed)...A semi arrives with a pallet: "Hey Carl, you can have this online in a couple days, right?"
Hey Carl, thanks for the kind comment. Glad you like the video. It's always funny how difficult it can be to 'bridge the divide' between the physical world and virtual world. Many SWEs expect to be able to "spin up" 1000 servers with an API call and forget that there are actual physical objects and tons of people that actually make that happen when you're on-prem.
I work at a network vendor and found this video very informative and practical of listing and explaining all the tasks involved in building a gpu cluster. Just two thoughts if you ever get a chance of building an updated version, 1) include Ethernet as another option to infiniband network, 2) talk about the cost (at least CAPEX) of the cluster as the pain points in the intro was heavily around cost of gpu instances in the cloud. Putting myself in the customer shoes, after watching the video, building cluster seems very complex and I am not sure if the benefits were clearly called out. Just my 2 cents, great video!
Best of the best presentation on server clusters. Author presented deep understanding of server clusters so that he explains things in an easy way. thank you!!!
Lots and lots of A100 GPUs. Every single one of them is a monster, almost 2x faster memory than the next best GPU. An entire room full of A100 racks... holy cow.
Hello team lambda labs. Please make an updated video on this topic. I think alot of have changed. Obviously, basics are same but with updated guide video from the CEO himself indepth will be best. Please consider doing that. Waiting for the video.
@@lambdacloud This was way useful and packed of knowledge. Please make updated video on this with more information on how these designs will be in near future. Cause a lot has changed in scale wise and otherwise as well. So yeah, waiting for this.
Hey Stephen, this is highly informative. I work on this clustering. Now am able to connect the dots and get the bigger picture. where can i read about the relationship between numa topology and GPU peering capability.
I have three computers, and a nas, and a external hub. I think that I don’t need a another server because of the NAS. As far as my architecture goes, is there anything else that you can advise?
Meng Xu, you can email support@lambdalabs.com 24/7 or call +1 (866) 711-2025 during business hours. Sorry to hear you're having issues, I'm sure we'll be able to resolve them quickly.
What if I have a model that I just want to run as provided, it hasn't really been optimized to run around the cluster and has memory requirements greater than any individual system I have. I feel safe to assume that for that specific case a shared distributed memory model would be the solution to run that specific app, yes? Is there any distribution of Linux that has support for such a memory model? It doesn't have to be a full-blown single system image. Perhaps a patch to the memory management driver so storage can be treated as an extension of system memory and not swap memory? Does any such software exist?
Extraordinary presentation. Covered all the important topics in depth and with real teaching talent. Many thanks!!
Thanks. I’m planning on building a “massive” 2 GPU system for home use.
How did it go man? I also want to build something like that and then stumbled on this video, which is excellent
Its nice to see a holistic explanation of designing / building / installing a complex multi-rack system...As someone that has spent years working on both sides of the "analog/digital divide" (physical data center world / digital world's various segments), the un-sexy physical aspects of available rack space / power / cooling / floor loading / network uplink bandwidth are often overlooked (often assumed)...A semi arrives with a pallet: "Hey Carl, you can have this online in a couple days, right?"
Hey Carl, thanks for the kind comment. Glad you like the video. It's always funny how difficult it can be to 'bridge the divide' between the physical world and virtual world. Many SWEs expect to be able to "spin up" 1000 servers with an API call and forget that there are actual physical objects and tons of people that actually make that happen when you're on-prem.
What an amazing presentation - one of the better videos I have watched. Great breadth and depth.
One of the best presentations on GPU cluster design, even at 3 years old. Great teaching skills!
I work at a network vendor and found this video very informative and practical of listing and explaining all the tasks involved in building a gpu cluster. Just two thoughts if you ever get a chance of building an updated version, 1) include Ethernet as another option to infiniband network, 2) talk about the cost (at least CAPEX) of the cluster as the pain points in the intro was heavily around cost of gpu instances in the cloud. Putting myself in the customer shoes, after watching the video, building cluster seems very complex and I am not sure if the benefits were clearly called out. Just my 2 cents, great video!
I was talking reference from XAI's new datacetner and how to design such level of datacenter would be really helpful. So waiting for that.
Stay tuned, a refresh is in the works and will cover InfiniBand vs. RoCE.
Most professional and holistic explanation I heard about this topic.
Thank you so much!!
Thank you for highlighting an underrated topic/options that company should re-consider within their compute infrastructure.
Ground level details with all the critical aspects covered nice for GPU Cluster to the last cable length calculation.
Thank you. You got me started years ago with your lambda stack -- the only way I could get TensorFlow installed on Linux.
Best of the best presentation on server clusters. Author presented deep understanding of server clusters so that he explains things in an easy way. thank you!!!
Lots and lots of A100 GPUs. Every single one of them is a monster, almost 2x faster memory than the next best GPU. An entire room full of A100 racks... holy cow.
Very expert suggestions for hpc and compute sizing.
Genius bait and switch. Props!
Lambda needs an explanation on the difference between "building" and "designing".
Really good analysis and presentation!
This is super insightful!
Highly appreciated...TH-cam should have a separate category called Founder's video.
Still most relevant today, 2 years later. Thanks.
Hello team lambda labs. Please make an updated video on this topic. I think alot of have changed. Obviously, basics are same but with updated guide video from the CEO himself indepth will be best. Please consider doing that. Waiting for the video.
Stay tuned! Videos on building the latest GPU clusters are in the works!
@@lambdacloud This was way useful and packed of knowledge. Please make updated video on this with more information on how these designs will be in near future. Cause a lot has changed in scale wise and otherwise as well. So yeah, waiting for this.
This was amazing. Thank you.
Thanks for the video.
What a remarkable video
very informative , thank you.
very informative, thanks!
Tell me how difficult it is so i can buy your solution kind of talk
I want to build a multi dual epyc 7742 based system for goofing around learning this stuff.
Hey Stephen, this is highly informative. I work on this clustering. Now am able to connect the dots and get the bigger picture.
where can i read about the relationship between numa topology and GPU peering capability.
Excellent.
Great insight!
it is a lecture more than a tutorial, Thx.
thanks for the inspiration
I have three computers, and a nas, and a external hub. I think that I don’t need a another server because of the NAS. As far as my architecture goes, is there anything else that you can advise?
Your are insane, thank you
My machine learning team consists of me baby
Our group ordered around 10 lambda PCs 1 year ago. Right now more than 5 have problems. Some of them do not start up. Mine gets stuck randomly....
Have you tried looking into the reasons?
Meng Xu, you can email support@lambdalabs.com 24/7 or call +1 (866) 711-2025 during business hours. Sorry to hear you're having issues, I'm sure we'll be able to resolve them quickly.
Our team has 5 lambda laptops, they work perfectly for over a year now..
We also have a workstation with 3 GPUs, works great too.
Very Based
I just love this kind things. How do i can start this kind bussnes how i can find customer for like small node and start building up
Looking for work would love to help
Does it work in man????
What if I have a model that I just want to run as provided, it hasn't really been optimized to run around the cluster and has memory requirements greater than any individual system I have. I feel safe to assume that for that specific case a shared distributed memory model would be the solution to run that specific app, yes? Is there any distribution of Linux that has support for such a memory model? It doesn't have to be a full-blown single system image. Perhaps a patch to the memory management driver so storage can be treated as an extension of system memory and not swap memory?
Does any such software exist?
Is this opensourced?
Now if only I was a billionaire so I could make use of this great information...
Do you guys have a gpu cluster optimized for 3d rendering.
Hell yes Lambda Lambda Lambda.
This guy is smart as fak
Does lambda products (gpu cluster) ship with a manual to help you set up the servers for use
Half Life man!
Talk about what ur expert.. don’t talk useless stuff without knowing all facts
headeggs
This dudes in full submission mode . Sad
speak UP