Videos like this from your channel get me so excited about the future of the data center. As a student, this is the kind of thing I look forward to more then anything as I delve into my career in the data center. When I finish my MSITM degree, I hope to take part in utilizing these technologies as much as possible. I, for one, very much appreciate you covering the emerging technologies and cutting edge hardware. Great video STH. As long as you continue to enjoy it, please keep the content coming.
From someone who has been working in datacenters for the past 10+ years, it's just as fun working with this stuff. The last few years have been all kinds of fun to see new tech hit the mid/low end with the occasional customer who forks out for the best of the best. Then I usually get to play for a while before setting it all up for them, fun times.
I see two different kinds of trends. the ones that are interesting for special applications and hyperscalers (OAM, DPU) and the ones that will be part of every (even small-business servers) e.g. EDSFF, Chiplets GB-era CPUs
Ha! That is the second office here. Literally had zero furniture when I did that one just after moving STH to Austin. The cameras above are Panasonic GH5, S1, S1R, and Canon R, and R5. I think this one was shot on a Sony FX3 but usually we use the Canon C70's.
@@hariranormal5584 LOL summer host, you're funny. :) Been around since early 2000's far from summer host, but host some of those over the years for sure. :(
@@DrivingWithJake Sorry, my comment was rude. But yeah, I'm in LowEndTalk (you should join their discord, fun) and I just love the hosting community, my main reason to as to why Im a bit interested in servers and stuff :D heh, we see good interesting summerhosts come and go, just like 2 days ago IIRC a fiber line in Houston got cut, and some summerhost had their site/services down kek :V
@@hariranormal5584 Hahah, LET :) LES is a little better. Not so much time these days to waste in Discord however might have to join it one of these days. Already a member on both those places.
I can't stress enough on how beneficial you're videos are. It's nice to hear a fellow technologist talk about things you work with everyday.. and their future. Thank you
My server is a Pentium 4 HT (3.0GHz; 2GB DDR) and 1.21 TB of HDDs, (2 x IDE; 2 x SATA). It runs FreeBSD 13.0 on OpenZFS 2.0 :) Since June 2019 I use it for ~1 hour/week for backups (send | ssh receive).
My DNS and email server is still a ~30yo Ultra 1E. And there's a stack of AC200's, and a new-ish T1000 sitting here. (there's a DEC Alpha and RS6000 in the pile as well.) And, of course, there are PCs back to the 486DX50 era as well. (pretty sure the 386 was tossed a decade ago.) Oh, and there's a BeBox (BeOS, if anyone remembers that) in the collection, too. (not counting the Fujitsu made tanks sitting in storage. T1/2 and intel based.)
@@jfbeam I have still a 486DX66 with 8MB, that I did built in 1993 and a Philips P3105 XT clone with a 20MB HDD from the eighties, but I have no space, so they are in a cabinet.
I also live at a apartment and have a bit more modern hardware but it's mostly turned off. Even if i wanted electricity prices have gone up to 280% compared to the time before the human malware situation. Where they weren't cheap either btw, but now the electric bill is killing me. If you have anything even a high end gaming pc (700 watt +) or 10 gigabit + networking gear with a server+ NAS + clients (most are raspberry pi 4) . Or all of them well your F with the electric prices where i live. Just for that reason alone, i'm looking to move/buy a place where i can have some kind of renewable energie to cut the cost. rather pay for the initial investment and do most labour myself
Patrick, wanted to ask when you might be doing another DIY Nas setup? Not a focused on a single setup kind of video, but something to cover most everything people might want to do. For example, a simple 8 drive like your CS381 setup is great, but getting into options for up to 24 or even 60 drives would be nice to see. A nice mix of cheapest solutions all the way through the dollars don't matter designs. I'd love to see some more STH being focusing on the home. Currently I'm running an aging 2650 x 2 setup with 64GB of memory and 15 drives total, but now I'm needing to expand to something more modern. GPU or iGPU encoding, more/larger drives, quieter than my old supermicro 3u chassis. I want to keep my existing 10 x 8TB drives in order, but a nice more modern design with some expansion in mind. Maybe a Fractal Meshify 2 XL for 18 drives, or maybe some DAS setups to help expand existing setups. I'm going to be picking up a CS381 pretty soon thanks to your previous video. Secondary NAS (backing up personal backups) with a router in a VM with the secondary NAS being in another VM, just to make the best use of a good setup. But that still leaves me with a "where to go" on my main setup. I'd love to see those home focused setups again
We usually do more of those when new platforms come out. Current Atom C3000, Xeon D-2100, EPYC 3000 all came out in 2017-2018 so we did that kind of content for this generation years ago. We have instead been focusing on the HPE MicroServer Gen10 Plus, QNAP,, and etc.I see the point though. Will work on it, but we are doing at least one (hopefully two if Will and I can hammer out a few details) new home focused series starting in the next month or so. The move has really slowed down content, especially on TH-cam. September is when we move into things get a bit better every day mode
it would be better for everyone if compute did not move towards even higher power usage. servers and compute already use a substantial amount of power in the world and the idea that it will drastically grow with time is not good. I'm not even talking about cryptomining per se, which is an utter waste of resources, but also AI compute for purposes of advertising and business trends.
The ugly truth is that performance per power in silicon technology has stagnated for years. The effect of this is that manufacturers had to increase power to get higher performance. Upcoming silicon nodes are going to be even worse in this aspect, barely bringing 10% performance per power improvement, while buyers expect 20-30% performance uplifts.
Most of the switches in this class do not use anywhere near that much power. 30A 208V is just under 5kW usable. Even the 128x 100GbE/ 32x 400GbE switches are using a fraction of that.
I did the math, knew that. I have a bunch of 32x100’s, been running them for years. I guess I was shocked to see someone put that in their documentation. Made me want to call them and ask why.
Hey so will the move to OAM mean there will be a move away from Nvidia gpu's to AMD or Intel? Or are fpga's going to be more prominent in this area? Also, do you think this will lead to a move away from CUDA to ROCm?
Well, Intel and AMD got the first Dose exascale system contracts. Still, NVIDIA signed into OAM as well because large hyperscalers pushed it. The better way to think of it is that instead of a NVIDIA only custom form factor, it can be any accelerator vendor's product (Intel Habana Labs is already on OAM as an example)
My list: OCP 2.5 and 3.5 inch drives replaced with m.2s (imagine how much you can pack in a 2u server with front m.2 drive bays) Reintroduce infiniband switching
M.2 does not make much sense for servers. Only inside an adapter maybe. That wouldn't be very servery though. The M.2 connector/form factor exists because it's cheap to manufacture and fits in slim laptops. M.2, U.2, EDSFF, it's all just PCIe anyway, with some extras.
I've been in my company's IT department nearly five years. They did run some cheap cat6 cable (now pretty much all WIFI). The owners seem to view the whole data movement as similar to the CB radio phase. It's waning and slowly dying. The storage is being outsourced (cloud services) and hopefully all that would be needed is a couple of unmanaged switches. We're down to two servers boxes and they're "old" and are on their way out and won't be replaced. I suspect the IT department is on it's way out as well. Most clerical work is still paper oriented. The whole "Information Age" came and went and out company never even knew it was a thing. So, I'm confused by the "Server Excitement" stuff. WHY?
you must work for a small company, that has 0 regulation and reporting that it must adhere to. What you described is not the case if the company I work for, which houses multiple datacenters across the globe.... it also does not provide cloud services.
"cloud hosting" So it's not that you've ignored the information age or don't use servers, you just use someone else's servers. As for paper clerical work, ugh! No room for automations or backups, sounds bloody horrible.
YOu are overselling things. 1. GB caches are just low hanging fruit. Instead of radically changing computing, they are going for what they do best- showeling transistors on the heap. Much bigger caches won't change anything radically. But they will make designs much more expensive and harder to cool. 2. Big point of CXL is NOT RAM integration. RAM has to be close to CPU. Put it on the long cable and you have to pay with latency. But ability to go around PCIe hierarchy with one root and everything else is a big deal. One will be able to connect two or more machines tightly without using Infiniband, for example. This would be a BIG win for the user. S/he could use any cheap machine as multiprocessing node. 3. Fast Ethernet is not THAT big of a deal. If/where ordinary user needs really high speeds, CXL should be more than enough. It would be not just much cheaper and faster but also more energy efficient.
People should stop referencing concepts like a "bus" when talking about point-to-point SerDes PCIe type connectivity - a bus is a shared (parallel) medium, p2p SerDes is not.
CXL? So they're reinventing VME. It takes a special breed of programmer to use that effectively. Seeing how 99% of the world is polluted with java, javascript, flash, and python "programmers", I don't see this working so well. (not for years.)
As more and more companies move their entire infrastructure to the cloud the less relevant you need to know about the lower level hardware side of maintaining servers that's going to end up in a Azure or AWS data center. System Administrators aka DevOps Engineers or Cloud Administrators are going to be more focus on administering the cloud infrastructure with the use of virutal machines.
I am not a native speaker, I appreciate the speed a lot though! You want to consume information fast. You are most probably in your limited free-time. However, everybody ist free to slow down or speed up the playback nonetheless.
I think the point is that Ponte Vecchio *is* next generation. Milan is already out, covered, a known quantity. Is covering the dominant server CPU manufacturer being a shill, or simply relevant reporting?
Patrick man you carry this whole channel on your back. Not only that, but it's quickly becoming my favorite channel on TH-cam. Thank you
Ha! Thanks.
Videos like this from your channel get me so excited about the future of the data center. As a student, this is the kind of thing I look forward to more then anything as I delve into my career in the data center. When I finish my MSITM degree, I hope to take part in utilizing these technologies as much as possible. I, for one, very much appreciate you covering the emerging technologies and cutting edge hardware. Great video STH. As long as you continue to enjoy it, please keep the content coming.
From someone who has been working in datacenters for the past 10+ years, it's just as fun working with this stuff. The last few years have been all kinds of fun to see new tech hit the mid/low end with the occasional customer who forks out for the best of the best. Then I usually get to play for a while before setting it all up for them, fun times.
Meh, been hearing talk like this for 30 years ... You get used to it.
I think CXL and Silicon Photonics will be the most game changing progress in IT industry.
16:00 Thanks for all the info you bring to us.
I see two different kinds of trends.
the ones that are interesting for special applications and hyperscalers (OAM, DPU) and
the ones that will be part of every (even small-business servers) e.g. EDSFF, Chiplets GB-era CPUs
I notice you didn't squish the Lego down far enough that you couldn't take them apart easily. Smart.
Skills learned as a 6 year old
Love the background :-D How many cameras do you have? And btw. are those Sonys?
Ha! That is the second office here. Literally had zero furniture when I did that one just after moving STH to Austin. The cameras above are Panasonic GH5, S1, S1R, and Canon R, and R5. I think this one was shot on a Sony FX3 but usually we use the Canon C70's.
As a data center and hosting provider it's quite interesting to see where things are going for the future.
Oh, where do you summerhost on?
@@hariranormal5584 LOL summer host, you're funny. :)
Been around since early 2000's far from summer host, but host some of those over the years for sure. :(
@@DrivingWithJake
Sorry, my comment was rude. But yeah, I'm in LowEndTalk (you should join their discord, fun) and I just love the hosting community, my main reason to as to why Im a bit interested in servers and stuff :D heh, we see good interesting summerhosts come and go, just like 2 days ago IIRC a fiber line in Houston got cut, and some summerhost had their site/services down kek :V
@@hariranormal5584 Hahah, LET :) LES is a little better.
Not so much time these days to waste in Discord however might have to join it one of these days. Already a member on both those places.
I can't stress enough on how beneficial you're videos are. It's nice to hear a fellow technologist talk about things you work with everyday.. and their future. Thank you
Thanks for the kind words. Have a great day.
Very interesting and good presentation. Being a retired server guy I only cried twice and sobbed 3 times.
Patrick's videos = Automatic LIKE
Ha! You are too kind!
Auto like would not pay tribute to the hard work put into them...
If I could i would give one like for content, another one for presentation.
Hey Patrick, that is just over quarter of an hour of great forward-looking stuff!
That "desk and cabinets" fabric backdrop looks really good :)
Ha! That is real. Hopefully the desk of a future video editor
As I'm sitting here with an old Ultra Sparc IIIi server running OpenBSD in my apartment
My server is a Pentium 4 HT (3.0GHz; 2GB DDR) and 1.21 TB of HDDs, (2 x IDE; 2 x SATA). It runs FreeBSD 13.0 on OpenZFS 2.0 :)
Since June 2019 I use it for ~1 hour/week for backups (send | ssh receive).
My DNS and email server is still a ~30yo Ultra 1E. And there's a stack of AC200's, and a new-ish T1000 sitting here. (there's a DEC Alpha and RS6000 in the pile as well.) And, of course, there are PCs back to the 486DX50 era as well. (pretty sure the 386 was tossed a decade ago.) Oh, and there's a BeBox (BeOS, if anyone remembers that) in the collection, too.
(not counting the Fujitsu made tanks sitting in storage. T1/2 and intel based.)
@@jfbeam I have still a 486DX66 with 8MB, that I did built in 1993 and a Philips P3105 XT clone with a 20MB HDD from the eighties, but I have no space, so they are in a cabinet.
I also live at a apartment and have a bit more modern hardware but it's mostly turned off. Even if i wanted electricity prices have gone up to 280% compared to the time before the human malware situation. Where they weren't cheap either btw, but now the electric bill is killing me. If you have anything even a high end gaming pc (700 watt +) or 10 gigabit + networking gear with a server+ NAS + clients (most are raspberry pi 4) . Or all of them well your F with the electric prices where i live. Just for that reason alone, i'm looking to move/buy a place where i can have some kind of renewable energie to cut the cost. rather pay for the initial investment and do most labour myself
I actually had a NeXT cube (titanium case) given to me. Lost it in a move.
nice rundown of next gen servers!
Short answer: New second hand stuff for Home Servers.
Hey man. I think you're super cool and I love the site and channel. Peace!
Patrick, wanted to ask when you might be doing another DIY Nas setup? Not a focused on a single setup kind of video, but something to cover most everything people might want to do. For example, a simple 8 drive like your CS381 setup is great, but getting into options for up to 24 or even 60 drives would be nice to see. A nice mix of cheapest solutions all the way through the dollars don't matter designs. I'd love to see some more STH being focusing on the home.
Currently I'm running an aging 2650 x 2 setup with 64GB of memory and 15 drives total, but now I'm needing to expand to something more modern. GPU or iGPU encoding, more/larger drives, quieter than my old supermicro 3u chassis. I want to keep my existing 10 x 8TB drives in order, but a nice more modern design with some expansion in mind. Maybe a Fractal Meshify 2 XL for 18 drives, or maybe some DAS setups to help expand existing setups.
I'm going to be picking up a CS381 pretty soon thanks to your previous video. Secondary NAS (backing up personal backups) with a router in a VM with the secondary NAS being in another VM, just to make the best use of a good setup. But that still leaves me with a "where to go" on my main setup. I'd love to see those home focused setups again
We usually do more of those when new platforms come out. Current Atom C3000, Xeon D-2100, EPYC 3000 all came out in 2017-2018 so we did that kind of content for this generation years ago. We have instead been focusing on the HPE MicroServer Gen10 Plus, QNAP,, and etc.I see the point though. Will work on it, but we are doing at least one (hopefully two if Will and I can hammer out a few details) new home focused series starting in the next month or so. The move has really slowed down content, especially on TH-cam. September is when we move into things get a bit better every day mode
it would be better for everyone if compute did not move towards even higher power usage. servers and compute already use a substantial amount of power in the world and the idea that it will drastically grow with time is not good. I'm not even talking about cryptomining per se, which is an utter waste of resources, but also AI compute for purposes of advertising and business trends.
The ugly truth is that performance per power in silicon technology has stagnated for years. The effect of this is that manufacturers had to increase power to get higher performance. Upcoming silicon nodes are going to be even worse in this aspect, barely bringing 10% performance per power improvement, while buyers expect 20-30% performance uplifts.
lol cxl ain't gonna change shit
people don't seem to understand how comparatively slow PCI connections are
I’m excited to see what medium we will be computing on in 20 years.
Hi and TX for your channel. For home users any update / replacement to the Netgear XS724EM ...?
When are we due to get bio-neural gel packs?
At work, got a 64x100 switch, and their recommendations for power…. 30amp 208/240v circuit.
Most of the switches in this class do not use anywhere near that much power. 30A 208V is just under 5kW usable. Even the 128x 100GbE/ 32x 400GbE switches are using a fraction of that.
I did the math, knew that. I have a bunch of 32x100’s, been running them for years. I guess I was shocked to see someone put that in their documentation. Made me want to call them and ask why.
@@zambonidriver42 Maybe it was sponsored by Eaton?
Hey so will the move to OAM mean there will be a move away from Nvidia gpu's to AMD or Intel? Or are fpga's going to be more prominent in this area? Also, do you think this will lead to a move away from CUDA to ROCm?
Well, Intel and AMD got the first Dose exascale system contracts. Still, NVIDIA signed into OAM as well because large hyperscalers pushed it. The better way to think of it is that instead of a NVIDIA only custom form factor, it can be any accelerator vendor's product (Intel Habana Labs is already on OAM as an example)
@@ServeTheHomeVideo Ah interesting thanks very much for the response. Your videos are incredibly helpful for somebody new to the space like myself!
New features, and fresh new pricing to go with it.
You thought intel had too many cpu names and models for the past 10 years? .... welcome to 2022/23
My list:
OCP
2.5 and 3.5 inch drives replaced with m.2s (imagine how much you can pack in a 2u server with front m.2 drive bays)
Reintroduce infiniband switching
OCP is certainly a trend. The M.2 is being replaced by EDSFF.
M.2 does not make much sense for servers. Only inside an adapter maybe. That wouldn't be very servery though.
The M.2 connector/form factor exists because it's cheap to manufacture and fits in slim laptops.
M.2, U.2, EDSFF, it's all just PCIe anyway, with some extras.
Check out the 1.8in SSDs that some Dell's use.
This is all great info, however as I sit here typing and can't purchase a new car or network equipment with all the chip manufacturing shortages. 😎👍😁
His shirt is not printed correctly
What about HBM-PIM AI from Samsung
Can't wait to play COD Warzone from my CPU L1 Cache
$poetf photonics are coming faster than you think
I see: MB-era to GB-era
I think; Motherboard era to Godmotherboard era?
I've been in my company's IT department nearly five years. They did run some cheap cat6 cable (now pretty much all WIFI). The owners seem to view the whole data movement as similar to the CB radio phase. It's waning and slowly dying. The storage is being outsourced (cloud services) and hopefully all that would be needed is a couple of unmanaged switches. We're down to two servers boxes and they're "old" and are on their way out and won't be replaced. I suspect the IT department is on it's way out as well. Most clerical work is still paper oriented. The whole "Information Age" came and went and out company never even knew it was a thing. So, I'm confused by the "Server Excitement" stuff. WHY?
you must work for a small company, that has 0 regulation and reporting that it must adhere to. What you described is not the case if the company I work for, which houses multiple datacenters across the globe.... it also does not provide cloud services.
"cloud hosting"
So it's not that you've ignored the information age or don't use servers, you just use someone else's servers.
As for paper clerical work, ugh! No room for automations or backups, sounds bloody horrible.
Servers are cool, duh 😂
Anyone else see 20231 at first on the thumbnail? 😄
Your gonna need a bigger power socket.
YOu are overselling things.
1. GB caches are just low hanging fruit. Instead of radically changing computing, they are going for what they do best- showeling transistors on the heap.
Much bigger caches won't change anything radically. But they will make designs much more expensive and harder to cool.
2. Big point of CXL is NOT RAM integration. RAM has to be close to CPU. Put it on the long cable and you have to pay with latency.
But ability to go around PCIe hierarchy with one root and everything else is a big deal. One will be able to connect two or more machines tightly without using Infiniband, for example. This would be a BIG win for the user. S/he could use any cheap machine as multiprocessing node.
3. Fast Ethernet is not THAT big of a deal. If/where ordinary user needs really high speeds, CXL should be more than enough. It would be not just much cheaper and faster but also more energy efficient.
I predict that servers will still be mindless and unattentive, and still expect a decent tip.
People should stop referencing concepts like a "bus" when talking about point-to-point SerDes PCIe type connectivity - a bus is a shared (parallel) medium, p2p SerDes is not.
en.wikipedia.org/wiki/Bus_(computing)
CXL? So they're reinventing VME. It takes a special breed of programmer to use that effectively. Seeing how 99% of the world is polluted with java, javascript, flash, and python "programmers", I don't see this working so well. (not for years.)
Everything into one SoC! SoCify servers lol
Wafer scale?
@@davidtolley1374 Maybe - but probably something a bit smaller for which the supply chain is already setup for.
As more and more companies move their entire infrastructure to the cloud the less relevant you need to know about the lower level hardware side of maintaining servers that's going to end up in a Azure or AWS data center. System Administrators aka DevOps Engineers or Cloud Administrators are going to be more focus on administering the cloud infrastructure with the use of virutal machines.
Pff these things will never happen
Dude... You talk too fast. Slow down and separate your words.
Ihearthis whenyou arespeakingsofast in this video.
Thank goodness for playback speed!
I am not a native speaker, I appreciate the speed a lot though!
You want to consume information fast.
You are most probably in your limited free-time.
However, everybody ist free to slow down or speed up the playback nonetheless.
wow, intel shill much.
When talking about the chiplets, he starts with an unreleased intel chip instead of the current epyc from amd.
In a video about new 2022 technology why talk about today's?
I think the point is that Ponte Vecchio *is* next generation. Milan is already out, covered, a known quantity. Is covering the dominant server CPU manufacturer being a shill, or simply relevant reporting?