Congrats on 100k followers, been following since a much lower number. I'm super happy for you, you really are a great communicator when it comes to tech
I just got my hands on a coupple of Ampere Alrtas, with instuctions to evaulate them for some highly odd applications........... I dont often have fun with brand new kit like this, but whilst ive had the extra time ive really got to push the limmits, some standout notes for me 1) Incredible power consumption, you will assume your power meter is broken its crazy low power for the performance 2) Virtualization options out of the box, just work, and do mean just work 3) High core counts and memory bandwidth = incredible performance, some of my testing i note with glee they DESTROY Intel and AMD 4) 128 Lanes of PCI Gen 4, with options for upto 192 Lanes - I stuggle in my head on how in the F youd use all that, i know it can be done but still 192 Gen 4 Lanes, let that sink in folks, 192 Gen 4 Lanes 5) Huge amounts of on chip cache, backed with very very fast cache before memory = blistering performance for some operations and tasks. 6) Not as hot as you might think, toasty yes, but cooler than youd expect for a 250W TDP Max 7) Fans in the system were smaller and much quieter than expected 8) 8 Channel memory support when properly exploited, offers up some amazing thruput and latencys / memory timings are tight (ish) out of the box ARM Changed the mobile space, they OWN the low power space, and for the first time in my humble opinion they can stake a real claim to the datacenter (i know some will say oh been there for years, yes they have, slow archaic machines in th eback for niche jobs that no one wants to work on) I share your enthuasium patric with these, the options flexibilty, power, and massive ammounts of PCIe bandwith and lanes mean they can do just about anything, and alot of things nither AMD nor Intel can. On a side note to global regulators THANK YOU FOR KEEPING ARM BRITISH AND STAVING ON NVIDIA. To NVIDIA thanks for the 1.5 billion :)
Just in case not mentioned in the video, if you go CUDA on Aarch64, then it requires 64K page size. It shouldn't be a problem for all Arch64 servers, as Server Boot Requirement has mandatory 64K page size. But not all other Aarch64 vendors follow it (e.g Apple?) Another thing is the software stack support, CUDA is already closed-source, but some distros are not fully migrated to 64K page size for Aarch64. The most common case would be swap partition, which has its blocksize saved at mkfs time, thus one can not reuse the swap partition between different page sizes. (And some releases ago, btrfs also doesn't fully support subpage, thus fs created under 4K can not be mounted on 64K page systems)
Apple uses 16k pages by default due to its IOMMU, but can support 64k pages fine iirc. It seems Asahi has more or less forced support for weird page sizes on a lot of projects, probably for the best though.
12:40 However, note that that PSU is only rated for 1 kW output when supplied with 120 V AC. Though if you don't need redundancy on a dev server you could probably load-balance over the two PSUs.
I told Jeff the original title was going to be "Jeff G has been lying to you... this is an Arm server." He replied with the current title so that is what we are using.
The challenge we have is that we cannot take photos/ video in the facilities we use. Right now we are still using 3 different facilities with multiple racks each. Hoping the 4th one will be online next year.
I have no idea what most of this stuff is nor do I have any use for this knowledge but man do I love Patrick’s awesome energy and passion for the industry!
100% there are other Altra systems. We will have reviews of them coming. What is missing is like a 16-32c/ 32x PCIe lane 2-4 ch DDR4 version that is cheaper
@@ServeTheHomeVideo you should have finished that sentence "... that is cheaper than x86 platforms of similar performance" because otherwise it is just an unjustified fancy revenge on your sysadmins. But hey, look, shiny!
i hope high core arm server in as desktop computer will becoming more common and available for the mass market. putting more presure for x86/64 to be more competitive both in price and performance.
Actually, they are pretty close in pricing when it comes to servers and large-scale. Ampere is priced more like what Intel and AMD price their chips for the hyper-scale market.
@@ServeTheHomeVideo That's true. Unfortunately for home desktop, currently we're stuck with SBC like raspberry pi, orange pi, etc. Linux on arm is quite okay in term of support and software availability. Arm for windows is quite hit/miss. But the exclusive partnership between Microsoft and Qualcomm is ending in near time (cmiiw). Hopefully when it ended, Microsoft will add more robust support for arm processor in general. Arm on mac is really solid, though 😁
This has come so far since 2016 it is amazing. Pre-Ubuntu 16.04 LTS and ThunderX, Arm servers were almost unusable. This is so far ahead from a platform perspective it is amazing.
@@ServeTheHomeVideo I'd argue we can all thank the PI and mobile phones for that. Developing for arm is now as simple as ticking a box in your favorite editor :)
DPUs are usually higher latency than simple IB, and latency is important for HPC, especially for things with a long tail. Nvidia drivers already worked on PPC64LE, so porting to Arm shouldn't have been a big deal. No big endian though.
I hope one day the installation on small boards will be as easy. Right now it's nearly impossible to find an up-to-date image for something that's not raspberry pi (which you can't get right now...)
what's the expansion ROM situation like for add-in PCIe cards? I know UEFI specs include machine-independent bytecode (EBC) but are vendors starting to ship with those? Can I boot off an NVMe card? PXE / UNDI support for network cards? (flashbacks of open firmware expansion ROMs for pre-intel macs...)
How much have you paid for these A100s - afaik they're crazy expensive. I think AIBs are prohibited from equipping 4090 with more than 24GB to keep the A100 sales going at 10-20x the price for the same performance. Also with ARM you get the problems with closed source drivers - if the vendor doesn't provide or discontinue the driver for ARM, we're out of luck.
"Oh Enterprise ARM is so power efficient and runs so cool! Everyone should switch over! It's so eco-friendly" > dual 1.6Kw power supplies per server. "It ONLY pulls 1.2Kw at the wall at full tilt." e_e Methinks this platform still has quite a few years to go...
Patrick failed to mention the TTSDK-88 chip. Since only Patrick and I know what that means, it stands for the "Time Travel Software Development Kit (88mph)". It is still a pricey option, but worth the money. Yes, this server is powerful enough to travel back in time to give you results before you asked for them. I wish Patrick would be more thorough and mention that in the future (but his server already knew that).
Patrick, how's the computing power / power utilization looking for these payloads this CPU works well (no GPU) e.g. as you mentioned redis, nginx etc. ? If this is favourable, then companies might want to use these (for specific use case) instead of x86_64 . Some sever farms need a lot of nginx / redis etc. instances and burn through lots of kW; saving even 10% would show up in the ROI.
I've seen an EATX form factor Ampere PC with a RTX 4090, which blew my mind. Could you build one of these as an ingest server or to do video editing in Davinci Resolve? Also, could I use an Ampere PC with an Nvidia dGPU running Arch64 or something to run stuff like Stable Diffusion WebUI, or train models with Torch, or run a local LLM, or would they require all custom code? Also, is there a way to run Windows inside VMWare, or VirtualBox? Thanks for the great content! This machine is drool-worthy, for sure.
I'm looking forward to Intel and AMD moving over to ARM for consumer stuff as more and more of my personal infrastructure is moving to ARM and the power savings are huge! All my mobile devices + my laptop are already ARM (MBP) and I have an rPi cluster for testing HA/K3S/CEPH. I'm just waiting on ARM variants for my 2 server boxes and eventually the Steam Deck.
Have they announced they will do that? I can't see it myself. Intel is too controlling to give in and use an ARM instruction set. They used to make ARM chips and sold off their ARM division.
@@6581punk IDK about Intel, but I know AMD has been doing some internal testing w.r.t. ARM chips. I believe they were hiring for ARM chip designer positions last year. After the success of Fujitsu, Graviton, and Apple Silicon, I imagine Intel (and the rest of the industry) will soon follow suit if they aren't working on it already. I mean, the new power requirements for Intel and AMD's newest CPU's and Nvidia's new GPU's mean that I will probably need a dedicated breaker for my x86 boxes, more power savings at this point must be something they're looking into.
I'm not convinced x86 emulation is going to be solid enough for something like a Steam Deck or some desktop use cases. Also power saving is not quite as simple as it seems. I've been testing the Mac Mini M1 and an i5-8250U, while sure the Mac Mini under load is MUCH faster, they both idle at around 11W and the benefit of the 8250U is I can change the storage, RAM and it has six NICs so probably could drop idle load even more if it didn't or if I got a more efficient PSU. I don't think we necessarily need ARM to be efficient, its just ARM inherently is a better option for bespoke devices because anyone with a license can more tightly design the whole system around it, omitting anything you don't need to save power.
I could see Intel skipping ARM and going RISC-V instead. ARM market is already very crowded except for the desktop space. RISC-V though has enough room for Intel to make itself cozy in most segments. Also, the Deck going x64 is likely very deliberate as any x64 emu with a DXVK layer would destroy any benefit going ARM.
I don't think Intel or AMD are going to arm, they already have a lot of customers who depend on x86 software. Plus there's still plenty of room for x86 to grow as well.
Well, in a GPU server, if you want to do big vector floating point calculations or AI, that's litterally what the GPUs are in it for. and a companion integer strong CPU is perfect. That's another reason why in the cloud all those minor Intel accelerator extensions are kind of pointless, since you can use a service that is purpose build for each type of workload. Your nginx Ingress controllers, CDNs, etc. will run on the ARM server, your AI API calls on GPU or even Inferentia and Trainium, or other similar specialised AI processors, etc. If you are building one monolithical monster that has all the code on one big VM and using the same CPU for all of that, not only are you doing it wrong in many peoples oppinion, you're definitively at least not doing cloud. Hosting that VM on a hyperscalers hardware isn't changing that fact, even if they are called public clouds, because that is what they are designed to do.
The Arm vendors will add the same over time. The latency going to a specialized machine type is so much higher if you have a small percentage of a calculation type that built in acceleration adds a lot. Arm vendors generally do not have it, but it is just a matter of time
@@ServeTheHomeVideo Only for certain use cases will they be added. You can have big vector extensions on ARM for a long time as well. Fujitsu's A64FX have comparatively way more vector power already. AVX512 didn't need any special design work either, it is just a question of whether it is useful enough to warrent the transistor usage to make the vector bigger. That problem won't go away either. We'll see bigger vector units, but the ratio will stay smaller for use cases where they are less usual to be useful. As for the latency, it clearly hasn't stopped the rise of microservices so far. A lot of the calls are separted already on the customer browser side anyway. E.g. /api/recommend would go to an ML service, while /api/product/ would go to a CMS cache, /api/... to various REST services, etc. And let's be honest, enqueuing the training of an ML model in a few ms won't influence the hours or at least minutes it takes even on special hardware, especially if you care about cost. And if you care about cost, you won't have enough hardware standing around to do that at any time in less than the milliseconds to reach another service in the same data center. You will use it only when demanded, so the spin up time is there either way. You may as well spin up purpose build hardware. That only changes if you have enough constant throughput to warrent a dedicated machine, at which point it again warrents purpose build hardware, just dedicated one. Since you can use VMs, it's not like you need a minimum size for that either on public cloud.
It would work with ZFS, but I would probably stay x86 for now with it. Most of the recent ZFS development is done on x86 and storage is probably worth being risk adverse with. Also, you would probably want more drives and fewer GPUs in a storage server.
ARM64 is the future for both servers and desktops. Both Intel and AMD are working on ARM64 CPU's as well and will be moving on from x86. Apple has sped up the switch with their M1 and M2 chips, and we already got Windows ARM64 working pretty darn good.
I rather call the "GPU" on servers as _Parallel Processing Unit_ or _AI Processor_ (depends on what they are usedfor). Calling it GPU when it doesn't necessarily produce graphics are like USB naming scheme which doesn't make sense and no standardization in the near future.
The Orin is a garbage replacement for a 150 dollar jetson 4gb with presently cost more than the original Xavier. I could smash my 3090 (unmined) with nvidia letting Chinese vendors charge 800 for a precovid 150cad setup
02:00 - Heh, going from a Raspberry Pi to an Ampere M128 is like going from a scooter to a Saturn V rocket 😂
Ampere M128-30 with two NVIDIA A100 80GB even!
I still want one, tho
3:08 was really aimed your way, Jeff, lol.
More like from a scooter to the Isle of Wight ferry full of scooters during the IoW scooter festival.
I want to see you compiling the Linux kernel on an Ampere!
Congrats on 100k followers, been following since a much lower number. I'm super happy for you, you really are a great communicator when it comes to tech
Thank you Joe! It was a super exciting Friday night when it happened!
Patrick has those pumped up arms out for this ARM server
💪 😁
I just got my hands on a coupple of Ampere Alrtas, with instuctions to evaulate them for some highly odd applications........... I dont often have fun with brand new kit like this, but whilst ive had the extra time ive really got to push the limmits, some standout notes for me
1) Incredible power consumption, you will assume your power meter is broken its crazy low power for the performance
2) Virtualization options out of the box, just work, and do mean just work
3) High core counts and memory bandwidth = incredible performance, some of my testing i note with glee they DESTROY Intel and AMD
4) 128 Lanes of PCI Gen 4, with options for upto 192 Lanes - I stuggle in my head on how in the F youd use all that, i know it can be done but still 192 Gen 4 Lanes, let that sink in folks, 192 Gen 4 Lanes
5) Huge amounts of on chip cache, backed with very very fast cache before memory = blistering performance for some operations and tasks.
6) Not as hot as you might think, toasty yes, but cooler than youd expect for a 250W TDP Max
7) Fans in the system were smaller and much quieter than expected
8) 8 Channel memory support when properly exploited, offers up some amazing thruput and latencys / memory timings are tight (ish) out of the box
ARM Changed the mobile space, they OWN the low power space, and for the first time in my humble opinion they can stake a real claim to the datacenter (i know some will say oh been there for years, yes they have, slow archaic machines in th eback for niche jobs that no one wants to work on)
I share your enthuasium patric with these, the options flexibilty, power, and massive ammounts of PCIe bandwith and lanes mean they can do just about anything, and alot of things nither AMD nor Intel can.
On a side note to global regulators THANK YOU FOR KEEPING ARM BRITISH AND STAVING ON NVIDIA. To NVIDIA thanks for the 1.5 billion :)
Just in case not mentioned in the video, if you go CUDA on Aarch64, then it requires 64K page size.
It shouldn't be a problem for all Arch64 servers, as Server Boot Requirement has mandatory 64K page size. But not all other Aarch64 vendors follow it (e.g Apple?)
Another thing is the software stack support, CUDA is already closed-source, but some distros are not fully migrated to 64K page size for Aarch64.
The most common case would be swap partition, which has its blocksize saved at mkfs time, thus one can not reuse the swap partition between different page sizes.
(And some releases ago, btrfs also doesn't fully support subpage, thus fs created under 4K can not be mounted on 64K page systems)
Apple uses 16k pages by default due to its IOMMU, but can support 64k pages fine iirc. It seems Asahi has more or less forced support for weird page sizes on a lot of projects, probably for the best though.
12:40 However, note that that PSU is only rated for 1 kW output when supplied with 120 V AC. Though if you don't need redundancy on a dev server you could probably load-balance over the two PSUs.
You really need to let Jeff Geerling have a play with it.
I told Jeff the original title was going to be "Jeff G has been lying to you... this is an Arm server." He replied with the current title so that is what we are using.
@@ServeTheHomeVideo Hahaha!
@@ServeTheHomeVideo lol 😂
That's one hell of a weird and amazing server. Great video!
Now I'd love to see Sth "datacenter" with that many servers and DPUs :D
The challenge we have is that we cannot take photos/ video in the facilities we use. Right now we are still using 3 different facilities with multiple racks each. Hoping the 4th one will be online next year.
I have no idea what most of this stuff is nor do I have any use for this knowledge but man do I love Patrick’s awesome energy and passion for the industry!
Ha. Thanks!
Definitely a fun time for all the compute platforms, thanks for the look at this interesting system!
I really wish there were more "general compute" ARM based systems. 128C / lots of RAM would make a lovely "infrastructure consolidation" node.
100% there are other Altra systems. We will have reviews of them coming. What is missing is like a 16-32c/ 32x PCIe lane 2-4 ch DDR4 version that is cheaper
@@ServeTheHomeVideo you should have finished that sentence "... that is cheaper than x86 platforms of similar performance" because otherwise it is just an unjustified fancy revenge on your sysadmins. But hey, look, shiny!
i hope high core arm server in as desktop computer will becoming more common and available for the mass market. putting more presure for x86/64 to be more competitive both in price and performance.
Actually, they are pretty close in pricing when it comes to servers and large-scale. Ampere is priced more like what Intel and AMD price their chips for the hyper-scale market.
@@ServeTheHomeVideo That's true. Unfortunately for home desktop, currently we're stuck with SBC like raspberry pi, orange pi, etc.
Linux on arm is quite okay in term of support and software availability. Arm for windows is quite hit/miss. But the exclusive partnership between Microsoft and Qualcomm is ending in near time (cmiiw). Hopefully when it ended, Microsoft will add more robust support for arm processor in general. Arm on mac is really solid, though 😁
This is junk for mathematical engineering use because they need floating point power only available on risc and x86
Moreso than difficult, banging up that fancy socket is a very expensive duckup 👀
Seeing ARM based software getting so accessible to get up and running is very exciting
This has come so far since 2016 it is amazing. Pre-Ubuntu 16.04 LTS and ThunderX, Arm servers were almost unusable. This is so far ahead from a platform perspective it is amazing.
@@ServeTheHomeVideo I'd argue we can all thank the PI and mobile phones for that. Developing for arm is now as simple as ticking a box in your favorite editor :)
DPUs are usually higher latency than simple IB, and latency is important for HPC, especially for things with a long tail.
Nvidia drivers already worked on PPC64LE, so porting to Arm shouldn't have been a big deal. No big endian though.
I hope there'll be an affordable version for homeservers. I definitely want one!
This is very interesting. Thank you for the review.
Project mini/micro * 3 for a HA tutorial? and shutting one down to show it gives you 100% uptime :)
Good idea. We are going to have a crazy amount of content in October/ November, but maybe in December.
I hope one day the installation on small boards will be as easy. Right now it's nearly impossible to find an up-to-date image for something that's not raspberry pi (which you can't get right now...)
Would you ever do a video on the Solidrun Honeycomb Lx2k? Very cool ARM workstation/server that is quite attainable.
We tried to get one like a year ago. There were big supply issues. Now hoping to the Bedrock V3000 in late Q1 2023
wow, your so excited
"I am not a hostage!" : Please break down this thing I am sending you....
LMAO @ 17:55 - We're looking at you HYUNDAI & KIA!!!! lol *chirp**chirp*
Am i the only one that's always very curious about the prices of the reviewed servers?
Yeah Patrick, you happen to just have 8 modules of 64GB DDR4 RAM lying around!. Wish I have that....
what's the expansion ROM situation like for add-in PCIe cards? I know UEFI specs include machine-independent bytecode (EBC) but are vendors starting to ship with those? Can I boot off an NVMe card? PXE / UNDI support for network cards? (flashbacks of open firmware expansion ROMs for pre-intel macs...)
How much have you paid for these A100s - afaik they're crazy expensive. I think AIBs are prohibited from equipping 4090 with more than 24GB to keep the A100 sales going at 10-20x the price for the same performance. Also with ARM you get the problems with closed source drivers - if the vendor doesn't provide or discontinue the driver for ARM, we're out of luck.
"Oh Enterprise ARM is so power efficient and runs so cool! Everyone should switch over! It's so eco-friendly"
> dual 1.6Kw power supplies per server.
"It ONLY pulls 1.2Kw at the wall at full tilt."
e_e Methinks this platform still has quite a few years to go...
Probably because of the gpus
Well done review, thanks!
Thank you Jay.
Patrick failed to mention the TTSDK-88 chip. Since only Patrick and I know what that means, it stands for the "Time Travel Software Development Kit (88mph)". It is still a pricey option, but worth the money.
Yes, this server is powerful enough to travel back in time to give you results before you asked for them. I wish Patrick would be more thorough and mention that in the future (but his server already knew that).
Patrick, how's the computing power / power utilization looking for these payloads this CPU works well (no GPU) e.g. as you mentioned redis, nginx etc. ? If this is favourable, then companies might want to use these (for specific use case) instead of x86_64 . Some sever farms need a lot of nginx / redis etc. instances and burn through lots of kW; saving even 10% would show up in the ROI.
In nginx and such, that is where the power savings really come into play. On the multi-GPU systems, it is usually a number under 10%. Even this was
I've seen an EATX form factor Ampere PC with a RTX 4090, which blew my mind. Could you build one of these as an ingest server or to do video editing in Davinci Resolve? Also, could I use an Ampere PC with an Nvidia dGPU running Arch64 or something to run stuff like Stable Diffusion WebUI, or train models with Torch, or run a local LLM, or would they require all custom code? Also, is there a way to run Windows inside VMWare, or VirtualBox? Thanks for the great content! This machine is drool-worthy, for sure.
Congrats on 100k
Thank you!
I'm looking forward to Intel and AMD moving over to ARM for consumer stuff as more and more of my personal infrastructure is moving to ARM and the power savings are huge! All my mobile devices + my laptop are already ARM (MBP) and I have an rPi cluster for testing HA/K3S/CEPH. I'm just waiting on ARM variants for my 2 server boxes and eventually the Steam Deck.
Have they announced they will do that? I can't see it myself. Intel is too controlling to give in and use an ARM instruction set. They used to make ARM chips and sold off their ARM division.
@@6581punk IDK about Intel, but I know AMD has been doing some internal testing w.r.t. ARM chips. I believe they were hiring for ARM chip designer positions last year. After the success of Fujitsu, Graviton, and Apple Silicon, I imagine Intel (and the rest of the industry) will soon follow suit if they aren't working on it already.
I mean, the new power requirements for Intel and AMD's newest CPU's and Nvidia's new GPU's mean that I will probably need a dedicated breaker for my x86 boxes, more power savings at this point must be something they're looking into.
I'm not convinced x86 emulation is going to be solid enough for something like a Steam Deck or some desktop use cases.
Also power saving is not quite as simple as it seems. I've been testing the Mac Mini M1 and an i5-8250U, while sure the Mac Mini under load is MUCH faster, they both idle at around 11W and the benefit of the 8250U is I can change the storage, RAM and it has six NICs so probably could drop idle load even more if it didn't or if I got a more efficient PSU. I don't think we necessarily need ARM to be efficient, its just ARM inherently is a better option for bespoke devices because anyone with a license can more tightly design the whole system around it, omitting anything you don't need to save power.
I could see Intel skipping ARM and going RISC-V instead. ARM market is already very crowded except for the desktop space. RISC-V though has enough room for Intel to make itself cozy in most segments.
Also, the Deck going x64 is likely very deliberate as any x64 emu with a DXVK layer would destroy any benefit going ARM.
I don't think Intel or AMD are going to arm, they already have a lot of customers who depend on x86 software. Plus there's still plenty of room for x86 to grow as well.
Ampere CPU's and Ampere GPU's? The meme is real
Not confusing at all :-)
Can you run a LLM like RedPajama or Stable Vicuna in that CPU (not the two a100)? or it's very slow?
Pretty slow.
...Jeff Geerling: hold my beer...
Great News about ARM...Now what about...RISC-V ?!
Ampere + A100 🤤
Ampere Altra plus NVIDIA Ampere
@@ServeTheHomeVideo haha good one, not really familiar with all the brands and stuff, didnt catch that one
Looks super exciting expensive
Alright, patrick, i have to ask, is this enough power to run pfsense as a firewall :) ( just love these videos )
Ha. Awesome
Well, in a GPU server, if you want to do big vector floating point calculations or AI, that's litterally what the GPUs are in it for. and a companion integer strong CPU is perfect. That's another reason why in the cloud all those minor Intel accelerator extensions are kind of pointless, since you can use a service that is purpose build for each type of workload. Your nginx Ingress controllers, CDNs, etc. will run on the ARM server, your AI API calls on GPU or even Inferentia and Trainium, or other similar specialised AI processors, etc.
If you are building one monolithical monster that has all the code on one big VM and using the same CPU for all of that, not only are you doing it wrong in many peoples oppinion, you're definitively at least not doing cloud. Hosting that VM on a hyperscalers hardware isn't changing that fact, even if they are called public clouds, because that is what they are designed to do.
The Arm vendors will add the same over time. The latency going to a specialized machine type is so much higher if you have a small percentage of a calculation type that built in acceleration adds a lot. Arm vendors generally do not have it, but it is just a matter of time
@@ServeTheHomeVideo Only for certain use cases will they be added. You can have big vector extensions on ARM for a long time as well. Fujitsu's A64FX have comparatively way more vector power already. AVX512 didn't need any special design work either, it is just a question of whether it is useful enough to warrent the transistor usage to make the vector bigger. That problem won't go away either. We'll see bigger vector units, but the ratio will stay smaller for use cases where they are less usual to be useful.
As for the latency, it clearly hasn't stopped the rise of microservices so far. A lot of the calls are separted already on the customer browser side anyway. E.g. /api/recommend would go to an ML service, while /api/product/ would go to a CMS cache, /api/... to various REST services, etc.
And let's be honest, enqueuing the training of an ML model in a few ms won't influence the hours or at least minutes it takes even on special hardware, especially if you care about cost. And if you care about cost, you won't have enough hardware standing around to do that at any time in less than the milliseconds to reach another service in the same data center. You will use it only when demanded, so the spin up time is there either way. You may as well spin up purpose build hardware.
That only changes if you have enough constant throughput to warrent a dedicated machine, at which point it again warrents purpose build hardware, just dedicated one. Since you can use VMs, it's not like you need a minimum size for that either on public cloud.
Would you do a review of the Khadas Edge 2, please?
Going to see how the Jetson goes. We are super full with ~60 products in the review queue and more coming with Genoa/ Sapphire
Would this perform as a more power efficient zfs storage server (ones with less core count ofc)
It would work with ZFS, but I would probably stay x86 for now with it. Most of the recent ZFS development is done on x86 and storage is probably worth being risk adverse with. Also, you would probably want more drives and fewer GPUs in a storage server.
I can't wait for proper beefy ARM desktops and workstations to hit consumers (apple doesn't count)
Any news on Ampere releasing CPU for desktop pc.
Hopefully we will show you something in a few months on a lower core count CPU
How does the power consumption compare with x86 servers? ARM chips are lower power.
Slightly lower, but in an accelerated system like this it is a single digit percentage benefit over an AMD system.
You use 512gb in your workstation?
Virtualization, yes
@@ServeTheHomeVideo Are you virtualizing a server with lots of ram?
Serving The Home 100,000 Times.
Patrick always forget to tell us the total price. Patrick how to buy separately parts? Where? Stop advertising Gygabyte free.
These are purchased usually as fully assembled servers, and at negotiated pricing.
ARM64 is the future for both servers and desktops.
Both Intel and AMD are working on ARM64 CPU's as well and will be moving on from x86.
Apple has sped up the switch with their M1 and M2 chips, and we already got Windows ARM64 working pretty darn good.
intel a770 on a server linux version 5 kernel with proton bridge to vulkan to intel linux drivers (and/or windows with proton-vulkan adapter)
cloud fails, do home-office server
network should be only a gigantic cdn
would you not think its sassy
of course arm works nice on debian, mostly, not gpus though
are you nuts paying for licenses
👍
that poor little fan at the top is cooling 600W worth of Gpus yikes
Eventually gpu,cpu,storage,ram will all be in the one chip
If you saw the Silicon Photonics piece on STH a few days ago, that is going to help usher in that kind of reality
Ampere and all arm chips cant compete with xeons and TR with number crunching on enginering and video
way 2 go ARM! now here's me leg! Arrr! sorry!
How much is this 90k raspberry pi
What do you mean you don't like your Power10 workstations? But I thought nobody gets fired for buying IBM...
I never heard about power 10 workstations otherwise i would get one at gunpoint
Serve the home.
You run 512 GB of memory in your workstation? :o
Why not? :-) Actually, I still run out of memory from time to time.
no, only 256 in mine :)
Running what?
Maybe in 30 years i can afford it
Sign on the Data Center Room: Caution ! These Servers are ARMed ! 😀
The world’s most powerful Nintendo Switch.
Ha!
how to bloat 5 minutes of content into 26 minutes
I rather call the "GPU" on servers as _Parallel Processing Unit_ or _AI Processor_ (depends on what they are usedfor). Calling it GPU when it doesn't necessarily produce graphics are like USB naming scheme which doesn't make sense and no standardization in the near future.
The Orin is a garbage replacement for a 150 dollar jetson 4gb with presently cost more than the original Xavier. I could smash my 3090 (unmined) with nvidia letting Chinese vendors charge 800 for a precovid 150cad setup
It’s a SERVER channel,and the video is not even 60fps.
Without mentioning cost in the first 3 minutes, I'm out.
First
Quite a bit early to this one.
7 days ago but the video was uploaded half an hour ago? how tf lol
mb
Second
To be fair, the "first" found this when the video was unlisted for a week, so you were the first when it was public. Congratulations.
third