I was able to try out VMware vSphere on NVIDIA Bluefield DPUs! In this video, we’ll explore the insane power of these technologies through a hands-on access demo on NVIDIA LaunchPad. Learn more here: ntck.co/nvidialaunchpad INSANE whitepaper covering this craziness deeper: ntck.co/whitepaper SUPPORT NETWORKCHUCK --------------------------------------------------- 🔥🔥Join the Academy: ntck.co/NCAcademy ☕☕ COFFEE and MERCH: ntck.co/coffee **Sponsored by NVIDIA
The past: lots of dedicated chips for 1 or few purposes. The present: few chips for multiple purposes. The future: lots of dedicated chips for 1 or few purposes.
@@pride4928 don't forget expensive and bloated. Just think what we can do if we add a few risc-v processors for just insert a function! The new management overhead, attack vectors and software layers will pale in comparison to the 9% increase in insert previous function particular measurement of choice!
Five years later: Huge advancement in hardware! DPU now becomes integrated to CPU as tech node becomes more advanced and IDPU now uses smaller portion of CPU budget.
The CPU is like a manager who's trained in knowing how to sort and distribute tasks, but isn't very good at the task itself. The DPU is like a specialized employee who's trained in that specific task. The CPU can delegate, and the DPU can perform. That's how it makes sense to me. Pretty sweet.
that's how every expansion or accelerator is supposed to work. Modern system designers have been royally incompetent - or I should say, their bosses have forced them to take on bad designs for penny-pinching.
Cpu actually does task and gpu is cpu who just refuses any task apart of graphical and dpu is mastered of bachelor in cpu 😂 Cpu already has gpu and dpu chip in it you just put another dedicated one beside him. So are we gunna take all the chip out from the cpu and we will have many cpus in the future imagine the motherboard containing gpu dpu rpu spu apu bpu a-z😂
That was awesome. So on my list of things I need (but not actually need) for my computer, I'll add a DPU. Its right up there with the blowy-matron, sub-zero cooler, quantum co-processor, and nuclear fusion cell from Fall Out 4.
If i'm nvidia, i'd send you cards like that, because i think videos like these are better at conveying what's good about the product than corporate videos. nice work, great editing and pacing!
@@bmunday Not so sure. First, Don't you think that the tech guys in companies look at youtube? second, i think that an exec that does not understand all that well would be watching this to be more relevant in the next reunion, and would thus greenlight this product rather than whatever else; and last, who do you think the techs that make recommendations were, 5, 10 years ago? You're young, you look at a lot videos when learning the field, get interested, and down the line, you remember the cool tech from nividia, and push to either have the latest version, or the mainstreamed version. Let's not forget that maybe in 5-10 years, if this trickles down in more down to earth devices, there might be a whole line of stuff available at lower prices. Better be the cool brand that everyone knows for pionering the stuff. look at Linus tech tips. They got 15M subs, so 5X more than this chanel, but way more general audience. ANd yet, they get lent/given crazy tech all the time, like those 30TB SSDs from kiokxia. Who in LTT audience can buy one? Still if i am consulted by my org, i'll think of them as an option. Not necessary the top of the line, but a few models down, yeah.
This is no better than a corporate video presenting an evolutionary products as evolutionary. There are various network offloading vendors out there. Even your embedded laptop network card has some offloading. This is payed video creating fan base.
Enthusiasm 1000% is always the best part of your content. It may be worth including the concept of offloading & discrete hardware in this summary. The microcode, driver, & software is what makes this solution possible. W/o it this is just chips on an easily removable pcb.
He is a good seller. First you overdwelm the person in front of you just with enthousiasm and say this it it, this is the best, this will change your life. A smart person pins right through that. But I like the channel too.
@@bmunday I disagree, he is way more solid on the fundamentals than say someone like Linus, better studio than PewDiePie, better looking than Tim Pool, idk man, I think it's better to say he's punching where most people in his position wouldn't even bother.
That's because you're like a small child. You need constant high stimulation so your social media (and probably also porn) fried brain can retain anything. Pure neotony, but worse.
@@cannaroe1213 seems like yer off in the consumer sector of viewership. He is presenting datacenter toward his typically elite consumer audience, and I am left wondering why, and what he portrayed.
Will be exploring FPGAs instead of SoC for some of my applications thanks to you will definitely be adding DPUs to my system...thanks again Chuck and have a great weekend!
13:09 you know, google has already made that AI processing unit, called TPU or tensor processing unit. Its specific job is processing many many small computations, just like GPU, but without graphic processing. As processor technology is advancing, we will have many processor made specifically for those tasks that is specific, need a lot of computations, and something that CPU just cannot do it better
@@illfordhey aren't niche. Dedicated ML cards like the commercial P100, V100, A100, and H100 are the core of large ML clusters. They are what power the most advanced ML training workloads, up to most of the top 10 supercomputers in the world. They are not GPUs. I ran a $2m cluster with 160 P100 devices and mellanox (before Nvidia bought them) infiniband nics across 42 servers. I had a combined compute capacity of about 2 PetaFLOPS. The cluster I handed the ML engineers allowed them to take training from many days down to a few hours.
@@rian7079 the Google TPU is Google's answer to the P/V/A/H100 and GH200 SOC series of Nvidia cards. Google's TPU is extremely powerful but outclassed by the A100, H100, and GH200. The latest gold standard design from Nvidia, the DGX GH200, creates single row (half-row really), 16 rack architecture (32 15U GH200 chassis, with 8 GH200 trays and 3 NVSwitches each) ExaFLOP scale supercomputers with nanosecond networking latencies at up to 400Gbps (note the bandwidth is far lower and latency is far higher in this video than what the DPU can actually do; nanosecond scale latency with 500x higher throughput). Seriously, go look at the DGX GH200 datasheet. I can confirm it's accurate. This video was good but didn't demonstrate the full power of the DPU, which is intended for non ML virtual workload communication tasks. The GPUs themselves connect directly to dedicated NVSwitches (inter-node NVLink) so they behave as a single mesh and don't use the server bus at all outside of command and control from NCCL. I know I sell like a sales ad, but after having played with the progenitor systems, I have to give credit to the awesome computational power they have brought to bear. It's terrifyingly powerful. With 256 GH200 SOCs, it gives 144TB of virtually shared GPU memory and combined active throughout between all GPUs of nearly 128TBps (1Pbps). This is just a single row. A single row is nearly as powerful as ORNL's Frontier, the current record holder supercomputer (#1 on the TOP500). Imagine an entire set of rows of these interconnected. It would dwarf the most powerful supercomputers at present, including the ORNL's Frontier and Summit and LLNL's Sequoia. Nvidia still holds 9th place with their Selene supercomputer based on the A100, their previous gen cards.
So if the dpu is just a normal computer (with a more efficient arm precessor) but put on a card that can go in a pcie slot, does that mean you could theoretically make a dpu with upgradable parts?
Wasn't always working as intended. I visualized a WISE terminal to act like a VT100, and it locked up the port, and had to take down the mainframe to make it let go.
Man, this video made me smile. I’m glad someone is getting enjoyment out these smartNICs.😅 I had gotten to play around and test those since the first version of the Bluefield cards. I was a member of the Data Center Ops team at one of theVMware’s R&D data center (miss that job in a lot of ways, but full cycle now I’m employed at NVIDA). It’s crazy how the tech had evolved from the first smartNIC vs what has gone to market now. Also, got to help build the infrastructure for Project Monterey (that was a huge undertaking). Thanks for the video Chuck. Even though you made me a little sad, but your content is some of the greatest stuff on the net! Keep up the passion and great digestible content.
I was only aware of the GPU and the CPU and the APU. I'll have to look into this DPU! Back in the days everything was held in separate "containers" now today everything is integrated into one single unit!
I just subscribed, but the TH-cam algorithm sent me your way because of all my work with intelligent NICs and DPU testing. I love my home DPU for what I use it on, but I don’t think these are ready for prime time, or at least where there are being marketed for in the data center. With that stated, if you use them for extending an EVPN fabric to the vNIC or next-gen firewall or some other post-processing of network packets, that would be a reasonable business case justification. It just does not fiduciarily make sense as an accelerator to make servers more powerful. Here is the big issue under absolutely ideal conditions, you get about a 30% increase with a CPU fully utilized at 100%. However, the observed increase under non-ideal is about half that. The problem is that a customer interested in performance will always have many servers and run them at less than full utilization on the CPU, SAN, and NIC. However, a customer concerned about the cost of running their server at full CPU utilization will not spend two to four grand more money per server to upgrade to these cards. Also, something not to mention these DPUs needs to be managed, whether that is automated via software or individually; they need, at the very least, to be patched for security issues and concerns. Also, they did require a complete OOB network. What do I use my home card for, and why do I love it? I love playing computer games, and with rare exceptions, most computer games are single-threaded, meaning they will only use one core. That is one core for all graphic processing, physics, in-game mechanics, and sometimes even preprocessing the networking. I run a 64-core AMD CPU for my home/work machine, and although it has many cores, they run at 2.9GHz. So, in playing a game, it is not unusual to have one random core running the game maxed out at 100%. My DPU has increased my framerate. This is an entirely crazy use of a DPU, but it works. What is odd is that a company in the mid-2000s called Killer NIC made game network alternators on purpose, but they were essentially snake oil, but this BlueField 2 card is the real deal for gaming.
What was once old, becomes new again! We took hardware and made it virtual... now we take the virtual and make it hardware! The king is dead.. long live the king!
Honestly I can't believe this hasn't been more of a thing by now. Accelerator expansions have always been the heart and soul of making advanced computing competitive at scale, but not only have we stagnated as an industry with being limited more or less to CPU, GPU, and DAC, we've actually stopped producing and using dedicated audio solutions to take the load off the CPU or expand the overall capabilities. We're down to 2 primary processing units, taking care of EVERY specialised task other than general intrasystem communications. Half of all system slowness comes from file transfer and data reading. The sheer scale of the old e-waste systems we could resurrect as competent web browsers and 'digital offices' with even an extremely modest DPU are potentially insane.
Chuck, the way you break these concepts down to the level that my peanut brain can actually understand is legendary❤. I'm starting to think Chuck is just the AI model for ChatGPT-9😂
I remember when the first GPU was announced. It was the Voodoo Graphics card. After that, the supposed next big thing was the PPU or Physics Processing Unit, But it never really took off. Now it's the Data Processing Unit. I can see the next thing being a card dedicated just to AI.
10mm x 10mm die full of cuda cores is actually a very powerful chip. You just have to give it the tasks it can understand. The nature of network packets and graphics processing are similar. Lots of small specific tasks to complete. This nic card makes alot of sense.
This is bada$$. Gotta admit even as a 20 plus year linux veteran some of this kinda went over my head but this video was so well made and so articulately said that it really broke down the complexity for me and conceptually I get it. Network chuck well played.
I watched the whole thing, seems interesting, but my mind still wasn't ready to fully digest how exciting or important the DPU might really be for me, even at the rather productive and usually very inspiring time of 3am that I'm currently watching this! Maybe in the morning after some sleep it will all kick in...
Just discovered your content due to looking for a career change and trying the self taught route. I'm loving the way your deliver your content, it gives me modern Bill Nye vibes. Got me excited to learn the content when others can be mind boggling. Can't wait to check out all your content to help guide me to this career that I'm striving for!
Funny thing back in 386/486 times we had what was called a mathematical co processor. Essentially like the DPU thing a card that was put into the PCI slot (wasn't PCI at that time but can't remember name of the slots) that took over some of the work the CPU did. But CPUs got better and the co processor was first integrated and later got obsolete. Now its going back to outsourcing workload to other chipsets again. Sort of full circle ;)
I made something like this a decade ago. I used an ARM SBC and then I used Linux USB Gadgets to provide ethernet and USB Mass Storage to the host. The problem I wanted to solve, was storage virtualization. So I would store files on a drive connected to my SBC, which would be visible to the host as a drive. It could boot from them like a normal USB drive and this would allow me to use external encryption, snapshots, network raid, etc, that would be completely invisible to the host PC. I built the thing in a 5.25" self-made chassis that I would just install and connect the device to a USB header on the host motherboard. It was a very fun project, but I just couldn't get the performance. I got up to 50MB/s or something and that's not useless, but not something that I would've bought, so I gave up. But the project did make me think up a lot of scenarios for such usecases and DPUs are definitely going to be popular among normal users in time. In fact, I think it's going to take over. You'll probably run a small SOC as your host and then just add more SOCs to do actual work so you can just power them up and down depending on how much processing power you need.
Uh...same amount of memory as my gaming pc actually...though mine is DDR4. 😋 Yeah...Vista abused me over my memory and have been ahead of the times ever since. Next time I upgrade it'll be 64GB. Can't wait for this to come to Homelabs via electronics recycling 😁 And congrats on 3M subs!
That comment made me laugh out loud for real. I have been at 64GB in my work/gaming computer for a few years now. Even the kids are at 32GB in their machines. Memory is a real cheap boost to performance to add. I am actually quietly impressed that they go all they do with only 32GB in the card. I guess everything is 'just passing through' and doesn't need to be memory resident.
While both a DPU and a CPU are processing units, they are designed for different types of workloads. A CPU is a general-purpose processor that is designed to handle a wide range of tasks, while a DPU is a specialized processor that is optimized for data-intensive workloads. As such, a DPU can offer significant performance improvements for certain types of workloads, but may not be as effective for more general-purpose tasks.
all I think while I watch your videos, is how you hold all this knowledge and not get crazy, adding that you have a big family also. Truly impressive. Thanks for your content.
So in theory, I could use this device to be able to run a server for friends on my computer and get little to no performance drops on the CPU. No need for a dedicated server when my friends can just play while I'm using my device.
Just think of the fun you can have till the courts catch up. Sorry officer, this warrant is for my other network, not the virtual one with all the good traffic.
i think this could even lead to desktop applications as well. i am seeing in the future custom pc building with really granular options compared to what we have today and tons of the workload could be offloaded away from the cpu using things like the dpu in desktops. maybe the cpu will be phased out entirely in favor of custom chips for all different types of data like MLPU DPU GPU SPU etc making each chip smaller
@@bobbygetsbanned6049 the heat being separated out with multiple processors would make it easier to cool even under heavy loads. using smaller heat sinks would be nice.
@@BrickTamlandOfficial The problem then becomes transfering the different datas into the CPU and... OH WAIT, isn't that what the CPU is all about? ... isn't this just a larger CPU problem?
@@holomatic1549 the cpu is for general purposes but the performance does not come close to a specialized processor. this is why the gpu exists. if the workload was split up into different processing units and the memory could be installed closer to those processors, the result would be higher performance and lower heat output.
The dpu aounds like a good idea in a pc because NONE of the demanding data transfer has to be done in cpu. Only basic io and a WIDEE pci-e bus to a dpu and some directly to the slot like how it is now. The dpu could handle all traffic not just the indirect pci-e cards, usb, sata, it could process all of that plus networking and the cpu would only deal with processing general operations. It could fit neatly where current motherboard chipsets live and have those metal plates for cooling
Hey Chuck, easy man, slow down, so much info in 13 minutes 44 seconds, incredible, my head is spinning. Just having fun Chuck. I don't pretend to understand all of it but the message is understood, virtual is heaven ;-)
What popped out of my 🧠 recently while I am watching this is a TOE (TCP offload engine), a dedicated subprocessor that accelerates the processing of TCP/IP related data (similar to a GPU, which accelerates the processing of graphics).
I remember the "good old days" when we had a dedicated card for sound, a dedicated card for a joystick or gamepad, a dedicated card for network and of course the dedicated card for graphics. For a while, you could even set it up in a way that you used one graphics card for graphics and the other one exclusively for physics. The only issue I have with the DPU is the cost. There is no open standard and nVidia is simply in a monopoly here. So they don't ask for what the technology costs but they add a hefty troll tax on top.
You're familiar with the 'jumper' game. IRQ's, DMA's, etc. Port assignments. I actually have finally giving up on the serial port and the parallel port for time critical devices. Mostly USB now.
I love the stepping around the pronunciation of SX. Here in the UK we have a chain of shops that buy and sell technology bits. It used to be called the 'Computer Exchange', but at some point a marketing suit decided it needed to be more succinct and shortened it to 'CeX'. Most people at this time started to call it C-eee-X, but the company lent in to the singular pronunciation of 'Sex', even as far as naming their free in-store customer wifi 'Unprotected Cex'
That would be cool if a scaled down version of this DPU was implemented into current PC hardware somewhere, for faster data transfer rates on personal computers.
One of the cool uses for DPUs, I think, is to allow for IT depts. to perform network-intensive maintenance tasks without impacting regular users on the network. Most users are really only trickling network treffic throughput with file saves/loads or web page access, which is intermittent. And if they're using remote apps (which are _awesome_ by the way) that display on their workstations but run on the server, those are mostly CPU/GPU loads (again, aside from file saves/loads). So, the IT peeps can be nailing the DPU with transfers, backups, etc. without slowing down the users.
We use BlueField cards for research at my university. Really cool cards. Pair that with P4 Tofino2 switches and you have a fully programmable pipeline of traffic.
It feels WAAAAY backwards... "Oh, look, we have all this *SPECIALIZED hardware,* lets make it virtual... Oh, it's low... I know! Lets install some *SPECIALIZED hardware"*
You are Killin it as a social influencer 👏 I want to give you Major props seen you grow before u were doing cisco and you now in space ymwith your content
Aruba Networks has an AMD Pensando DPU integrated into their Top of Rack switch called the CX 10000, it's innovate for on premise datacenters i.e. you are not a cloud hyperscaler. You can do 800Gbps of IPSEC per switch in hardware, NAT, Firewall rules, VRF's, VXLAN all in a TOR. Greatly simplifies datacenter networking.
13:14 what a seamless transition, from the voice you cannot tell you had cut at least several minutes or maybe hours in between to switch your shirt. I recorded several videos in the past and could notice when I did not record in one session by a change of tone in my voice ... great work! Interesting topic!
My 2023 gaming PC has 96GB of DDR5 5600, with a i9800KS, an RTX4080, and a secondary GPU RTX4070Ti. Why you might ask... because I also use this computer for software development of research applications.
Thanks for that very informative video. ❤ Sounds amazing…. But: it feels like IT industry is kinda reinventing the wheel here. I phrase it different: Nvidia and VMWare just implemented another way to have „channel processors“ or „system assist processors“. If you don’t know what it is: this is pretty similar technique built in in mainframe since centuries. Mainframes do have co processors for IO executing programs send by the operating system towards them. IO is performed by the „co processor“ and when the data was placed into the requested memory address, the CPU is informed via an interrupt. This keeps the CPUs hand away from it task to do real application workload in while in is performed. No, this is running like this in a virtualized environment since the 90s. Beside channel processors there are already ASICS on the networking cards (OSA) and yes the mainframe has specialized ASIC cards for doing encryption (to keep it away from the CPU). However as it seems that mainframes are sunsetting, I am glad to get much efficient techniques on distributed systems now.
The Cray supercomputers use mainframes for their io co processing. I can see a nice nitch market for DPU's. Rocket guidance systems. Inputting the telemetry data, and controlling the flight.
Thats only for needs of an ISP, for medium businesses a simple switch and a router can cover the speeds needed, to large businesses a dedicated server with a cpu can do too.
I was able to try out VMware vSphere on NVIDIA Bluefield DPUs! In this video, we’ll explore the insane power of these technologies through a hands-on access demo on NVIDIA LaunchPad. Learn more here: ntck.co/nvidialaunchpad
INSANE whitepaper covering this craziness deeper: ntck.co/whitepaper
SUPPORT NETWORKCHUCK
---------------------------------------------------
🔥🔥Join the Academy: ntck.co/NCAcademy
☕☕ COFFEE and MERCH: ntck.co/coffee
**Sponsored by NVIDIA
First reply
Hello NetworkChuck I really need your help with my crypto currencies account password
I forgot it and I need you to help me retrieve it
You are invincible
Who is really behind the dpu inovation
Why does pkill a take me to the login screen
The past: lots of dedicated chips for 1 or few purposes.
The present: few chips for multiple purposes.
The future: lots of dedicated chips for 1 or few purposes.
All this happened before and all this will happen again.
Patched up technology...
Bandaid solution...
@@LackofFaithify but it will be much, much more advanced
But in one case, not every one in his own case
@@pride4928 don't forget expensive and bloated. Just think what we can do if we add a few risc-v processors for just insert a function! The new management overhead, attack vectors and software layers will pale in comparison to the 9% increase in insert previous function particular measurement of choice!
Five years later: Huge advancement in hardware! DPU now becomes integrated to CPU as tech node becomes more advanced and IDPU now uses smaller portion of CPU budget.
10 years later:
* removing support for DPU and iDPU since not being used since deprecated for a few year
This, dpu is pointless
@@cosac88 for home users absolutely pointless
@@dtibor5903short sighted
5 years? 2 years tops from the moment they get deployed commonly
give this guy some trophy for being the best explainer
edit : good time to point out this is the most likes Ive ever gotten lol
100% agree
🏆
He needs one I learned so much IT from watching networkchuck videos
@@eserieb8014 sameee lol
Cringe
The CPU is like a manager who's trained in knowing how to sort and distribute tasks, but isn't very good at the task itself. The DPU is like a specialized employee who's trained in that specific task. The CPU can delegate, and the DPU can perform. That's how it makes sense to me. Pretty sweet.
I would think of it like a third party business or a contractor. The DPU does a lot of its stuff by itself(a server inside a server)
The GPU is like 4000 monkeys on typewriters.
@@TonkarzOfSolSystem LOL
that's how every expansion or accelerator is supposed to work. Modern system designers have been royally incompetent - or I should say, their bosses have forced them to take on bad designs for penny-pinching.
Cpu actually does task and gpu is cpu who just refuses any task apart of graphical and dpu is mastered of bachelor in cpu 😂
Cpu already has gpu and dpu chip in it you just put another dedicated one beside him. So are we gunna take all the chip out from the cpu and we will have many cpus in the future imagine the motherboard containing gpu dpu rpu spu apu bpu a-z😂
That was awesome. So on my list of things I need (but not actually need) for my computer, I'll add a DPU. Its right up there with the blowy-matron, sub-zero cooler, quantum co-processor, and nuclear fusion cell from Fall Out 4.
😂😭
Liquid nitrogen cooling
I had few of those, but my power armer keeps eating them
Don't forget a turbo encabulator.
@@franciscosoares2440 haha
That will break cpu 😂
If i'm nvidia, i'd send you cards like that, because i think videos like these are better at conveying what's good about the product than corporate videos.
nice work, great editing and pacing!
none of this dudes watchers will ever touch one, its out of scope.
@@bmunday Not so sure. First, Don't you think that the tech guys in companies look at youtube?
second, i think that an exec that does not understand all that well would be watching this to be more relevant in the next reunion, and would thus greenlight this product rather than whatever else;
and last, who do you think the techs that make recommendations were, 5, 10 years ago? You're young, you look at a lot videos when learning the field, get interested, and down the line, you remember the cool tech from nividia, and push to either have the latest version, or the mainstreamed version.
Let's not forget that maybe in 5-10 years, if this trickles down in more down to earth devices, there might be a whole line of stuff available at lower prices. Better be the cool brand that everyone knows for pionering the stuff.
look at Linus tech tips. They got 15M subs, so 5X more than this chanel, but way more general audience. ANd yet, they get lent/given crazy tech all the time, like those 30TB SSDs from kiokxia. Who in LTT audience can buy one?
Still if i am consulted by my org, i'll think of them as an option. Not necessary the top of the line, but a few models down, yeah.
This is no better than a corporate video presenting an evolutionary products as evolutionary. There are various network offloading vendors out there. Even your embedded laptop network card has some offloading. This is payed video creating fan base.
Enthusiasm 1000% is always the best part of your content. It may be worth including the concept of offloading & discrete hardware in this summary. The microcode, driver, & software is what makes this solution possible. W/o it this is just chips on an easily removable pcb.
He is a good seller. First you overdwelm the person in front of you just with enthousiasm and say this it it, this is the best, this will change your life. A smart person pins right through that. But I like the channel too.
no shit. kinda presenting above his waistline.
@@bmunday I disagree, he is way more solid on the fundamentals than say someone like Linus, better studio than PewDiePie, better looking than Tim Pool, idk man, I think it's better to say he's punching where most people in his position wouldn't even bother.
That's because you're like a small child. You need constant high stimulation so your social media (and probably also porn) fried brain can retain anything. Pure neotony, but worse.
@@cannaroe1213 seems like yer off in the consumer sector of viewership. He is presenting datacenter toward his typically elite consumer audience, and I am left wondering why, and what he portrayed.
Will be exploring FPGAs instead of SoC for some of my applications thanks to you will definitely be adding DPUs to my system...thanks again Chuck and have a great weekend!
13:09 you know, google has already made that AI processing unit, called TPU or tensor processing unit. Its specific job is processing many many small computations, just like GPU, but without graphic processing. As processor technology is advancing, we will have many processor made specifically for those tasks that is specific, need a lot of computations, and something that CPU just cannot do it better
there are many, many ASICs and blades can have PCIe on one edge, have been for a long time. But drivers at the vmware level is newish.
I miss nvidias APU audio processor unit
This sucks as a fucture. Granted an AI processing unit is niche as i feel most people gain little from it
@@illfordhey aren't niche. Dedicated ML cards like the commercial P100, V100, A100, and H100 are the core of large ML clusters. They are what power the most advanced ML training workloads, up to most of the top 10 supercomputers in the world. They are not GPUs.
I ran a $2m cluster with 160 P100 devices and mellanox (before Nvidia bought them) infiniband nics across 42 servers. I had a combined compute capacity of about 2 PetaFLOPS. The cluster I handed the ML engineers allowed them to take training from many days down to a few hours.
@@rian7079 the Google TPU is Google's answer to the P/V/A/H100 and GH200 SOC series of Nvidia cards. Google's TPU is extremely powerful but outclassed by the A100, H100, and GH200. The latest gold standard design from Nvidia, the DGX GH200, creates single row (half-row really), 16 rack architecture (32 15U GH200 chassis, with 8 GH200 trays and 3 NVSwitches each) ExaFLOP scale supercomputers with nanosecond networking latencies at up to 400Gbps (note the bandwidth is far lower and latency is far higher in this video than what the DPU can actually do; nanosecond scale latency with 500x higher throughput).
Seriously, go look at the DGX GH200 datasheet. I can confirm it's accurate. This video was good but didn't demonstrate the full power of the DPU, which is intended for non ML virtual workload communication tasks. The GPUs themselves connect directly to dedicated NVSwitches (inter-node NVLink) so they behave as a single mesh and don't use the server bus at all outside of command and control from NCCL.
I know I sell like a sales ad, but after having played with the progenitor systems, I have to give credit to the awesome computational power they have brought to bear. It's terrifyingly powerful. With 256 GH200 SOCs, it gives 144TB of virtually shared GPU memory and combined active throughout between all GPUs of nearly 128TBps (1Pbps).
This is just a single row. A single row is nearly as powerful as ORNL's Frontier, the current record holder supercomputer (#1 on the TOP500). Imagine an entire set of rows of these interconnected. It would dwarf the most powerful supercomputers at present, including the ORNL's Frontier and Summit and LLNL's Sequoia. Nvidia still holds 9th place with their Selene supercomputer based on the A100, their previous gen cards.
It's so amazing how you can take something so complex to understand and make it so simple
I'm a systems engineer specializing in virtualization at a giant 3 letter company, and I can see a million uses for this. Very cool.
IBM, EMC, SAP WHICH ONE I TIHINK IBM
Congratulations on 3 million subs!! 🥳
So if the dpu is just a normal computer (with a more efficient arm precessor) but put on a card that can go in a pcie slot, does that mean you could theoretically make a dpu with upgradable parts?
hear me out, how about one dpu plug into another dpu
And sandwiche another dpu between those two
@@iforget6940 SLI? what's that?! We got DPU.
@@snikrepak DPU × SLI
(THANKS FOR TEACHING ME SOMETHING NEW)
Reminds me of my AMD slot-A CPU from back in 1999/2000
This guy makes everything SO easy to understand. I love it!
worth mentioning that virtualization was around in the 70's on mainframes.
really ? I did not know that.
Wasn't always working as intended. I visualized a WISE terminal to act like a VT100, and it locked up the port, and had to take down the mainframe to make it let go.
Only because of you, now i am daily driving linux. I got so much knowledge from your channel.
Man, this video made me smile. I’m glad someone is getting enjoyment out these smartNICs.😅
I had gotten to play around and test those since the first version of the Bluefield cards. I was a member of the Data Center Ops team at one of theVMware’s R&D data center (miss that job in a lot of ways, but full cycle now I’m employed at NVIDA). It’s crazy how the tech had evolved from the first smartNIC vs what has gone to market now.
Also, got to help build the infrastructure for Project Monterey (that was a huge undertaking).
Thanks for the video Chuck. Even though you made me a little sad, but your content is some of the greatest stuff on the net! Keep up the passion and great digestible content.
Your energy is infectious my dude! Keep it up ❤
I was only aware of the GPU and the CPU and the APU. I'll have to look into this DPU! Back in the days everything was held in separate "containers" now today everything is integrated into one single unit!
I don't NEED one either. Definitely want one.
I just subscribed, but the TH-cam algorithm sent me your way because of all my work with intelligent NICs and DPU testing. I love my home DPU for what I use it on, but I don’t think these are ready for prime time, or at least where there are being marketed for in the data center. With that stated, if you use them for extending an EVPN fabric to the vNIC or next-gen firewall or some other post-processing of network packets, that would be a reasonable business case justification. It just does not fiduciarily make sense as an accelerator to make servers more powerful. Here is the big issue under absolutely ideal conditions, you get about a 30% increase with a CPU fully utilized at 100%. However, the observed increase under non-ideal is about half that. The problem is that a customer interested in performance will always have many servers and run them at less than full utilization on the CPU, SAN, and NIC. However, a customer concerned about the cost of running their server at full CPU utilization will not spend two to four grand more money per server to upgrade to these cards. Also, something not to mention these DPUs needs to be managed, whether that is automated via software or individually; they need, at the very least, to be patched for security issues and concerns. Also, they did require a complete OOB network.
What do I use my home card for, and why do I love it? I love playing computer games, and with rare exceptions, most computer games are single-threaded, meaning they will only use one core. That is one core for all graphic processing, physics, in-game mechanics, and sometimes even preprocessing the networking. I run a 64-core AMD CPU for my home/work machine, and although it has many cores, they run at 2.9GHz. So, in playing a game, it is not unusual to have one random core running the game maxed out at 100%. My DPU has increased my framerate. This is an entirely crazy use of a DPU, but it works. What is odd is that a company in the mid-2000s called Killer NIC made game network alternators on purpose, but they were essentially snake oil, but this BlueField 2 card is the real deal for gaming.
What was once old, becomes new again! We took hardware and made it virtual... now we take the virtual and make it hardware! The king is dead.. long live the king!
The io channel subsystem from system z mainframe comes back.
Honestly I can't believe this hasn't been more of a thing by now. Accelerator expansions have always been the heart and soul of making advanced computing competitive at scale, but not only have we stagnated as an industry with being limited more or less to CPU, GPU, and DAC, we've actually stopped producing and using dedicated audio solutions to take the load off the CPU or expand the overall capabilities. We're down to 2 primary processing units, taking care of EVERY specialised task other than general intrasystem communications. Half of all system slowness comes from file transfer and data reading. The sheer scale of the old e-waste systems we could resurrect as competent web browsers and 'digital offices' with even an extremely modest DPU are potentially insane.
i remember when this channel had less then 1 mill now it’s got 3 mill unreal content bros.
I remember when TH-cam didn't exist.
@@Okurka. Remember Fred on youtube
I joined when he talked about virtual machines bruh
Never knew these existed, I am impressed. Especially about how you can virtualize physical servers, or multiple ones down to one.
Chuck, the way you break these concepts down to the level that my peanut brain can actually understand is legendary❤. I'm starting to think Chuck is just the AI model for ChatGPT-9😂
Everything can be explained very simply, I'm 99% sure people don't do it on purpose and because some can't.
❤
Facts bro😂😂
@@W8CODE It's a spam bot impersonating Network Chuck. Please report all such messages as spam.
@@jamesarthurreed Yeah i did. These things are annoying
I remember when the first GPU was announced. It was the Voodoo Graphics card. After that, the supposed next big thing was the PPU or Physics Processing Unit, But it never really took off. Now it's the Data Processing Unit. I can see the next thing being a card dedicated just to AI.
It was. Ageia physics cards. Bought up by Nvidia years ago, now integrated into their video cards.
You mean a TPU? Thats a thing! You can play with one google colab.
Gotta get a NPU and TPU if we’re going into the AI section
Finally an episode centered around hardware
Its so amazing! Who knew adding another computer to do a specific function would make things go faster! Take my money NVIDIA!
10mm x 10mm die full of cuda cores is actually a very powerful chip. You just have to give it the tasks it can understand. The nature of network packets and graphics processing are similar. Lots of small specific tasks to complete. This nic card makes alot of sense.
This is bada$$. Gotta admit even as a 20 plus year linux veteran some of this kinda went over my head but this video was so well made and so articulately said that it really broke down the complexity for me and conceptually I get it. Network chuck well played.
That DPU won't even run .5% of a HoloDeck.
Thanks For The Awesome Videos.
😉✌️🖖
I watched the whole thing, seems interesting, but my mind still wasn't ready to fully digest how exciting or important the DPU might really be for me, even at the rather productive and usually very inspiring time of 3am that I'm currently watching this!
Maybe in the morning after some sleep it will all kick in...
Thank you explaining all these technologies and making my job so easy 👍🏼😁
Just discovered your content due to looking for a career change and trying the self taught route. I'm loving the way your deliver your content, it gives me modern Bill Nye vibes. Got me excited to learn the content when others can be mind boggling. Can't wait to check out all your content to help guide me to this career that I'm striving for!
You've got this.
Best of luck!
no career value here, we aren't going to have to install DPU's any time soon to our clients
Funny thing back in 386/486 times we had what was called a mathematical co processor. Essentially like the DPU thing a card that was put into the PCI slot (wasn't PCI at that time but can't remember name of the slots) that took over some of the work the CPU did.
But CPUs got better and the co processor was first integrated and later got obsolete.
Now its going back to outsourcing workload to other chipsets again.
Sort of full circle ;)
It literally sounds like a dedicated coprocessor
I made something like this a decade ago. I used an ARM SBC and then I used Linux USB Gadgets to provide ethernet and USB Mass Storage to the host. The problem I wanted to solve, was storage virtualization. So I would store files on a drive connected to my SBC, which would be visible to the host as a drive. It could boot from them like a normal USB drive and this would allow me to use external encryption, snapshots, network raid, etc, that would be completely invisible to the host PC. I built the thing in a 5.25" self-made chassis that I would just install and connect the device to a USB header on the host motherboard. It was a very fun project, but I just couldn't get the performance. I got up to 50MB/s or something and that's not useless, but not something that I would've bought, so I gave up. But the project did make me think up a lot of scenarios for such usecases and DPUs are definitely going to be popular among normal users in time. In fact, I think it's going to take over. You'll probably run a small SOC as your host and then just add more SOCs to do actual work so you can just power them up and down depending on how much processing power you need.
So... We now go back to CPU and Co-CPU like the 80's/90's?
thats why i love NetworkChuck bcuz he very good at explaining and entertaint at sometimes
Uh...same amount of memory as my gaming pc actually...though mine is DDR4. 😋
Yeah...Vista abused me over my memory and have been ahead of the times ever since. Next time I upgrade it'll be 64GB.
Can't wait for this to come to Homelabs via electronics recycling 😁
And congrats on 3M subs!
That comment made me laugh out loud for real. I have been at 64GB in my work/gaming computer for a few years now. Even the kids are at 32GB in their machines. Memory is a real cheap boost to performance to add. I am actually quietly impressed that they go all they do with only 32GB in the card. I guess everything is 'just passing through' and doesn't need to be memory resident.
I always wondered how Chuck gets that smooth voice... then I realized... the dude grew an all natural microphone pop filter!
Love your stuff Chuck! Thank you for sharing your knowledge.
2:20 what an unforcenate color combo
So you went from a physical server to a virtualized server to needing a server on a chip. I can't wait to see the next round lol.
virtual network cables to make installations easier.
While both a DPU and a CPU are processing units, they are designed for different types of workloads. A CPU is a general-purpose processor that is designed to handle a wide range of tasks, while a DPU is a specialized processor that is optimized for data-intensive workloads. As such, a DPU can offer significant performance improvements for certain types of workloads, but may not be as effective for more general-purpose tasks.
I appreciate your work
Chuck congratulations 🎉for 3 million family, and please bring that old hacking videos back with energy ⚡❤
all I think while I watch your videos, is how you hold all this knowledge and not get crazy, adding that you have a big family also. Truly impressive. Thanks for your content.
Your content is very cool. Hope you can teach us about kali tools and how it works thank you and have a nice day
So in theory, I could use this device to be able to run a server for friends on my computer and get little to no performance drops on the CPU. No need for a dedicated server when my friends can just play while I'm using my device.
It makes me wonder if a Storage Processing Unit may exist in the future.
s/NAS/SPU/g - you're done
Just think of the fun you can have till the courts catch up. Sorry officer, this warrant is for my other network, not the virtual one with all the good traffic.
With all the enthusiasm in this review, you'd never have guessed this was sponsored by Nvidia ;-)
i think this could even lead to desktop applications as well. i am seeing in the future custom pc building with really granular options compared to what we have today and tons of the workload could be offloaded away from the cpu using things like the dpu in desktops. maybe the cpu will be phased out entirely in favor of custom chips for all different types of data like MLPU DPU GPU SPU etc making each chip smaller
Man that would be a huge pain the ass when the vast majority of people don't even tax their CPUs.
@@bobbygetsbanned6049 the heat being separated out with multiple processors would make it easier to cool even under heavy loads. using smaller heat sinks would be nice.
@@BrickTamlandOfficial The problem then becomes transfering the different datas into the CPU and... OH WAIT, isn't that what the CPU is all about? ... isn't this just a larger CPU problem?
@@holomatic1549 the cpu is for general purposes but the performance does not come close to a specialized processor. this is why the gpu exists.
if the workload was split up into different processing units and the memory could be installed closer to those processors, the result would be higher performance and lower heat output.
The dpu aounds like a good idea in a pc because NONE of the demanding data transfer has to be done in cpu. Only basic io and a WIDEE pci-e bus to a dpu and some directly to the slot like how it is now. The dpu could handle all traffic not just the indirect pci-e cards, usb, sata, it could process all of that plus networking and the cpu would only deal with processing general operations. It could fit neatly where current motherboard chipsets live and have those metal plates for cooling
Nice demonstration and detailed explanation,., For most folks enough to say that its fancy accelerated NIC card with router capability on board.
How do they realize redundancy in such senarios? Do they have a physical clone to each server or it is done via virtualization also?
he really missed getting into how this accelerates or offloads vsphere. the only point he made was vMotion in a box
Hey Chuck, easy man, slow down, so much info in 13 minutes 44 seconds, incredible, my head is spinning. Just having fun Chuck. I don't pretend to understand all of it but the message is understood, virtual is heaven ;-)
i feel like regular desktops would need something like this too.
False
that feeling would be a failure of this video
Even the vast majority of companies buying servers don't need these but you think desktops need them? This must be one effective commercial.
I know nothing about computers but i could follow you just fine and understand what your talking about
you had one of those meanest, cool looking beard, despite that your explanation is chill af, so does your voice, love your vid.
What popped out of my 🧠 recently while I am watching this is a TOE (TCP offload engine), a dedicated subprocessor that accelerates the processing of TCP/IP related data (similar to a GPU, which accelerates the processing of graphics).
@networkChuckk bad spammer bot, go away
glad youtube recommended this channel! this guy is cool!
A master blaster stroke from Chuck! very informative
man you nailed it Chucks!! cant stop consuming your content just like Everyday Growing 2-3subscribers!! thanks man!! DPU something neewe
It give a whole new meaning of decentralizing which makes sense.
This video gives a whole "Explain Dpu like I am five years old." vibe
It's incredibly enjoyable to watch this video. Your explanation was superb, and GrandMa appears incredibly innocent at 13:38.
Very Informative you def earned my subscribtion💯
I remember the "good old days" when we had a dedicated card for sound, a dedicated card for a joystick or gamepad, a dedicated card for network and of course the dedicated card for graphics. For a while, you could even set it up in a way that you used one graphics card for graphics and the other one exclusively for physics. The only issue I have with the DPU is the cost. There is no open standard and nVidia is simply in a monopoly here. So they don't ask for what the technology costs but they add a hefty troll tax on top.
You're familiar with the 'jumper' game. IRQ's, DMA's, etc. Port assignments. I actually have finally giving up on the serial port and the parallel port for time critical devices. Mostly USB now.
Wow! Someone rediscovering mainframe tech.
History goes round again.
12:20 It's about 40.35% faster transfer speed, that really good.
your gonna send me though college, never stop uploading man
My 14 year old nephew could understand this. Best explanation ever
Bro this is a very good explanation. Peak Technology
You forgot to add APU... CPU APU GPU DPU..(I've known about DPU fou for a while now) good to see youtube finally catching up!
I love the stepping around the pronunciation of SX.
Here in the UK we have a chain of shops that buy and sell technology bits. It used to be called the 'Computer Exchange', but at some point a marketing suit decided it needed to be more succinct and shortened it to 'CeX'. Most people at this time started to call it C-eee-X, but the company lent in to the singular pronunciation of 'Sex', even as far as naming their free in-store customer wifi 'Unprotected Cex'
You explain things really well - both audibly and visually 👍👍👍
I want a DPU now! 😀
Hey there, Leave👆🏾 a message
Thanks for commenting🎁
That would be cool if a scaled down version of this DPU was implemented into current PC hardware somewhere, for faster data transfer rates on personal computers.
One of the cool uses for DPUs, I think, is to allow for IT depts. to perform network-intensive maintenance tasks without impacting regular users on the network. Most users are really only trickling network treffic throughput with file saves/loads or web page access, which is intermittent. And if they're using remote apps (which are _awesome_ by the way) that display on their workstations but run on the server, those are mostly CPU/GPU loads (again, aside from file saves/loads). So, the IT peeps can be nailing the DPU with transfers, backups, etc. without slowing down the users.
DPU is already a component in most monitors (Display Processing Unit)
👍 10:45 Thanks for the correction - I love the sass! ❤
10:41 I love this badass moments :D I am new to the channel! And I loved it
I love the way you explain things and make apparent complicated concepts into simple to understand ones
Thanks Chuck for dropping the knowledge I was reading on what's new with vSphere 8 and DPU was mentioned
We use BlueField cards for research at my university. Really cool cards. Pair that with P4 Tofino2 switches and you have a fully programmable pipeline of traffic.
This is innovation. Back handsprings, Chuck!
It feels WAAAAY backwards...
"Oh, look, we have all this *SPECIALIZED hardware,* lets make it virtual... Oh, it's low... I know! Lets install some *SPECIALIZED hardware"*
You piqued my curiosity with the video title and it turned out to be incredibly cool and interesting. Great video!
I haven't slept that good since I was in school cheers!
Thanks for keeping us updated with changes in technology.
You are Killin it as a social influencer 👏 I want to give you Major props seen you grow before u were doing cisco and you now in space ymwith your content
I'm just glad that a specialty device has been brought to market to take pressure off the gamer bro GPU market. Finally.
OMG stoooop changing camera angles so much XD🤣🤣🤣❤
I loved the correction at 10:45 😅
Aruba Networks has an AMD Pensando DPU integrated into their Top of Rack switch called the CX 10000, it's innovate for on premise datacenters i.e. you are not a cloud hyperscaler. You can do 800Gbps of IPSEC per switch in hardware, NAT, Firewall rules, VRF's, VXLAN all in a TOR. Greatly simplifies datacenter networking.
13:14 what a seamless transition, from the voice you cannot tell you had cut at least several minutes or maybe hours in between to switch your shirt.
I recorded several videos in the past and could notice when I did not record in one session by a change of tone in my voice ... great work! Interesting topic!
Hey there, Leave👆🏾 a message
Thanks for commenting🎁
My 2023 gaming PC has 96GB of DDR5 5600, with a i9800KS, an RTX4080, and a secondary GPU RTX4070Ti. Why you might ask... because I also use this computer for software development of research applications.
total bouncer XD but I love you chuck. Your Ip address and Subnetting series was amazing!
I love this! The video is great, but also the direction the channel is taking! I'm so happy to see how you're growing!
Thanks for that very informative video. ❤ Sounds amazing…. But: it feels like IT industry is kinda reinventing the wheel here.
I phrase it different: Nvidia and VMWare just implemented another way to have „channel processors“ or „system assist processors“. If you don’t know what it is: this is pretty similar technique built in in mainframe since centuries. Mainframes do have co processors for IO executing programs send by the operating system towards them. IO is performed by the „co processor“ and when the data was placed into the requested memory address, the CPU is informed via an interrupt. This keeps the CPUs hand away from it task to do real application workload in while in is performed. No, this is running like this in a virtualized environment since the 90s. Beside channel processors there are already ASICS on the networking cards (OSA) and yes the mainframe has specialized ASIC cards for doing encryption (to keep it away from the CPU).
However as it seems that mainframes are sunsetting, I am glad to get much efficient techniques on distributed systems now.
The Cray supercomputers use mainframes for their io co processing.
I can see a nice nitch market for DPU's. Rocket guidance systems. Inputting the telemetry data, and controlling the flight.
Thats only for needs of an ISP, for medium businesses a simple switch and a router can cover the speeds needed, to large businesses a dedicated server with a cpu can do too.