NEW AI Accelerated NUC with 16 Cores from ASRock
ฝัง
- เผยแพร่เมื่อ 10 พ.ค. 2024
- We go into the ASRock Industrial NUC BOX-155H. This new NUC has 16 cores with three different configurations on-SoC, a GPU, Thunderbolt 4, USB4, up to 3 SSDs, and built-in AI acceleration with a NPU. Meteor Lake in NUC is a really cool combination.
STH Main Site Article: www.servethehome.com/asrock-i...
STH Top 5 Weekly Newsletter: eepurl.com/dryM09
----------------------------------------------------------------------
Become a STH YT Member and Support Us
----------------------------------------------------------------------
Join STH TH-cam membership to support the channel: / @servethehomevideo
----------------------------------------------------------------------
Where to Find The Unit We Purchased
Note we may earn a small commission if you use these links to purchase a product through them.
----------------------------------------------------------------------
- 64GB Crucial Memory Kit: amzn.to/3tNiBY6
- 48GB Crucial SODIMM: amzn.to/3HiGLwx
STH Merch on Spring: the-sth-merch-shop.myteesprin...
----------------------------------------------------------------------
Where to Find STH
----------------------------------------------------------------------
STH Forums: forums.servethehome.com
Follow on Twitter: / servethehome
Follow on LinkedIn: / servethehome-com
Follow on Facebook: / servethehome
Follow on Instagram: / servethehome
----------------------------------------------------------------------
Other STH Content Mentioned in this Video
----------------------------------------------------------------------
- Ultimate 2.5GbE Switch Guide: www.servethehome.com/the-ulti...
- 2.5GbE Switch Mega-Round-up: • Finding the BEST Cheap...
- Minisforum MS-01:
- Intel C3758 Fanless 10GbE Server: • The EVERYTHING $300 Fa...
- CWWK Fanless Desktop System: • FANLESS 7W Homelab Fan...
- New 8x 2.5Gbe and 4x 2.5GbE and 2x 10GbE Firewalls: • Shocking New 10GbE and...
- R86S Pro Review - • This is a SUPER Homela...
- R86S Review - • Ultimate Homelab Revol...
- iKoolCore R1 N6005 Review: • This CRAZY SMALL 2.5Gb...
- Intel Core i3-N305 Fanless: • Best EVER Fanless Mini...
- Intel N100 and N200 Fanless: • HUGE UPGRADE! New Fire...
- Intel Alder Lake Fanless: • KING of Fanless 2.5GbE...
- Beelink EQ12 Pro with i3-N305: • Intel Core i3 N305 Min...
- Example of setting up Proxmox VE with OPNsense/ pfSense: www.servethehome.com/topton-i...
- ASRock 4x4 AMD Ryzen 7 5800U: • This AMD Ryzen NUC Alt...
- Three 6x 2.5GbE Fanless Systems: • 3x 2.5GbE Fanless Fire...
- Two Fanless N5105 Systems: • New Fanless 4x 2.5GbE ...
- VMware ESXi and pfSense on a 6x 2.5GbE Core i7 Fanless System: www.servethehome.com/topton-f...
----------------------------------------------------------------------
Timestamps
----------------------------------------------------------------------
00:00 Introduction
01:51 External Hardware Overview
06:13 Internal Hardware Overview
10:10 Intel Core Ultra 7 155H Performance
11:38 Trying the Meteor Lake NPU with OpenVINO for AI inference acceleration
13:38 Casual Gaming Performance with League of Legends
13:58 Power Consumption and Noise
16:39 Key Lessons Learned
19:45 Wrap-up - วิทยาศาสตร์และเทคโนโลยี
cant wait for meteor lake mini boards for my nas/plex server. then i can av1 encode/decode on the box
I totally agree. This is one step in that direction though.
check out the new ASUS NUC 14 Meteor boards.... it's the same as the Intel NUC boards.... solid and much better quality and IO support
This is a nice unit for home lab server
I have a few ASRock Industrial NUCs, and they all work great. I had one fan-less that I returned and replaced by a fan version, for the exact reason you highlighted on your video: the SSD was getting too hot for my tastes, and while it was temporary, it was unnecessarily hot. And I won't even consider that was with a 64W processor, versus the now pretty warmer Core Ultra 7 and its 115W max... Don't have a need for NPU at this time, but I would seriously consider one whenever the need comes.
If you are really that space constrained I suppose it is better than the alternative; but this size seems to be the point where you really hit diminishing returns from the fact that you can ram just about anything into a type-C port. If you want any displayport enjoy having either 1 or zero type-C ports(and if you need newer displayport prepare to lose thunderbolt) and permanent cabling on both the front and rear of the device.
It just seems like the amount of extra footprint required to put the video-carrying type-Cs in the rear; or cram in a vertically oriented DP connection, would be essentially irrelevant to most mounting/positioning use cases and make life a lot more flexible. In addition to this being at the size where even small changes in size can make a significant difference in how irksome a fan is.
I realize that "NUC" implies certain conventional dimensions, unless it's one of the oddball ones that aren't that size, so Asrock is presumably working under constraints here; but the NUC target size just seems to fall a little too far in the direction of small-to-prove-a-point-in-a-world-of-OEM-minitowers rather than "small enough to fit in small places; but no smaller". Something more mac mini sized fits in almost all of the same places with less oversubscription of connectors.
I generally agree that I would prefer slightly larger systems with quieter cooling and more expansion capabilities.
Seems like a good little box for home automation stuff. Start using the NPUs in these things for local voice control functions and also for security camera systems.
I think that is the idea with a system like this.
@@ServeTheHomeVideo Yeah, it's clearly not made to push a ton of data around with the 2.5G nics.
I mean, 5Gbps is not too bad actually with SMB3 multi-channel
@@ServeTheHomeVideo I was more thinking for use as a head for a JBOD rack where one nic would be for network connection and the other for the JBOD connection. I suppose it would probably be decent for home use for most people though.
Isn't it an overkill ? For home, all of that can be done with less powerful, more efficient CPUs (n100, etc...).
I wish it had ECC
If it was AMD it could have. But this channel looooooooooves Intel any chance it gets
lmfao, good one
@@lllongreen to be fair intel has the super low powered stuff and intel is what companies use in machines of this form factor.
@bhume7535 you've not been following the latest amd CPUs.
@@lllongreen to be fair to the person who made the original comment when people say this they usually mean they want fully validated, error reporting and correcting ecc, which is generally not found on AMD platforms outside of epic.
Could you do a thermal camera test on these systems, could be very interesting
I'd like a small quiet pc that could accept a single 3.5" drive, doing some searching lots of people are asking, but it appears no one is making. Thanks for the review.
There are PCs smaller than the NuC with dual Ethernet and Dual normal sized HDMIs. They use NVME as the drive though. Available in Pentium, i3 variants. I could find them as mini PC in Amazon. 3.5” drives almost history now.
@@tamildesan837 They are not history when you want to store more than one or two terrabytes of data. Zimaboard can do this, but harddrive will be noisiest part and it has no enclosure and Celeron is very weak CPU, not much RAM and it's overpriced considering it's performance. Or USB3 box for harddrive can be an option.
What did they use to found out that stuff about the usb ports? Is there a tester for this or is it methodical trial and error with many devices.
Would a mini PC like this one be good for Promox to run several Windows VMs at once?
I am particularly concerned about the mix of efficiency cores and performance cores.
Do you recommend something else? RAM and Storage are used fast.
I'm very interested in this product, I'd like to know if the built-in NPU can passthrough to a VM (guestOS) and run smart/AI accelerated photo albums...maybe unraid, maybe trueNAS scale
Not just into VMs, but also into Containers and then you can use it there (if the application in the container supports it)
Is there an AMD Ryzen PRO 8000 (because of the little bit improved AI stuff compared to the 7000 series) variant of this with unbuffered ECC (!) SO-DIMM support?
Hoping we get some options soon-ish. This also only takes a few minutes to get up and running since OpenVINO is packaged well.
All Ryzen SKUs support Unbuffered ECC RAM. The 8945HS is the 7940HS with a NPU.
@@BrunodeSouzaLino that's hardware support. ECC must also be enabled in the BIOS and often it is not even a setting
@BrunodeSouzaLino not the mobile SKUs unless they are Pro skus. Also for the AM5 8700G it also needs to be Pro sku to support ECC
@@lllongreen Not really. It needs to be pro to support Buffered ECC. All Ryzen SKUs support Unbuffered ECC out of the box.
Interested that when you show the NPU in use, the GPU is still up at 80%? As I understood it, the encoder for OBS counts as a different percentage in windows...
That part might be av1 encoding
I wish them a great industrial designer for their industrial line.
I wonder if you could replace the bottom plate with a fan using the same screws fan with a grill on it. That would be pretty sweet
It's nice to have the power supply readout visible in the camera, but I am curious why you didn't mention it while talking about the NPU?
You can actually see the power consumption go from ~50-60W when using the GPU for inference down to ~45-50W when using the NPU for inference. The main selling point for the NPU (for now) is efficiency by Intel's own slides.
Also, something interesting about that 50% utilization on the NPU, I am assuming that is because it was using the systolic MAC of the NPU for inference and the Meteor Lake NPU has two Neural Compute Engines, each containing a systolic MAC capable of performing 4096 INT8 MACs per cycle. I wonder if it would be simple to force the NPU scheduler to utilize both NCEs for a single inference task, and if so, how it might benefit performance.
No ecc, no ipmi, no deal.
what kind of workload are you running on this desktop mini PC that would need ECC? honest question as I have none I'd care about.
Great review. Useful! Thx!
Thanks for the feedback. Have a great evening
Did anyone else notice the utter smoothness he had throwing the RAM in?
1:11
The footage is sped up xD and it's at 1:16 actually.
the standing at desk studio layout is giving me Patrick Norton vibes
Good name
Great video!
Another problem with the labelling: is that DC jack centre positive (yay!) or centre negative? And why not print the voltage range and current requirement? It would cost them less to do that than process problems from customers due to the lack of labels.
Great points
What tool was used to display that usage? Can anyone tell me please? The one on that big screen at 14-15 min!?
On the labelling it would also be great if the management NIC was labelled so it didn't accidentally get used as a wan port
Fair point
AsRock is the only company I actually trust to do good hardware. Everybody else fell off.
Nice machine!
I wish these mini pcs came with at least a 10GbE port so that I didn’t have to use a thunderbolt to 10GbE adapter 💰 ☹️
Also: that thing is loud 😅
Very loud when we had it in performance mode, but mid-range for these kinds of boxes when it is in balanced mode.
And with 4x m2 ssd? 😅
>wants a 10Gbe port
>complains about noise
FYI a 10 GbE controller is going to need a dedicated cooler the same size as the CPU
The problem IMO is people that want 10GbE often want it in SFP form, so you can use it with RJ45, Fiber and DAC/Solid Copper by just changing the modules. The problem is not so much the first two, but the latter. People that use solid copper cable for short distances tend to use passive cables where the module and cable are one thing and those can get hot to the point of burning the SFP port on passive cooled systems. 10GbE is a format where you're starting to go beyond what twisted pair can do, so a single RJ45 wouldn't cut for certain cases, especially considering the prices of CAT7 and CAT8 cables.
Finally meteor lake!! 😀
I really need to get me a mini PC for Proxmox. How important would you say ECC support is for Proxmox? I'd imagine regular VM snapshots would kind of mitigate things, but I don't know of any mini PC with ECC support AND QuockSync for hardware transcoding.
I'm currently leaning toward the MS-01.
It depends how mission critical the system is. For a homelab where the result of failure is at least a bit of downtime while I replace the ram and at most having to restore some stuff from backup, I don’t see ECC as essential. I’m not running the financial servers for an international corporation, I’m running a handful of VMs for personal use. :)
@@BrunodeSouzaLino So when your backed up data has those errors that you don't know about, how do you fix it? If you don't have any data that you care about, that's fine. Stop speaking for others.
This is as close to the right size you can get and has both: ASRock IMB-X1231. Though to get the proper ECC reporting is another issue altogether. Intel said consumers are dumb and not worth our time, so you don't need ECC, hence that is where the industry stands.
oooo...nice
a step in the right direction tho...it is an asrock 🤔
would like to see other brands to jump in the fray (competition tends to help lower prices too)
I think ASUS will be there since they bought the NUC line. One advantage for many of our viewers is they have heard of ASRock versus some of the small mini PC brands.
Having a similar shaped mini PC I’m worried about overheating. These could benefit from a 120mm fan taking up the top. The ssds and ram cool in these things.
I wish for some of these little units they added in IPMI. Would 100% pay an extra $30-50 for that feature.
Usually it is $25-35 in BOM costs. Plus add 5W of power consumption at idle.
They have Intel vPro/AMT, which is basically a lower cost IPMI solution.
AMT vPro shares the NIC so you can manage across the NIC and use it as well, not like a server where you have a dedicated management NIC. You can use "meshcmd" to do commandline power control but unfortunately IPMI protocol is different.
I can see the reason why this 100% fan mode exists. It literally says Asrock industrial and you can imagine ppl using this box along with loud machines.
Great point
What’s the actual API to NPUs, is that any way vendor neutral?
What do you think of using used server hdd in consumer pc?
Generally I do not love used HDDs
@ServeTheHomeVideo I know that "AI" on home PC's has barely taken it's first gulp of air and from here can only grow, but I noted that the "AI engine" gave the same performance for your assigned task (but with the lower latency that you noted) as did the "GPU", bu I noted the small difference in GPU and CPU usage.! I watched it again and now have some questions. You clearly stated that there were multiple tasks going on, but going from the GPU to the APU: Increased the "AI engine" from 0% to 50%, decreased the GPU performance from ~84% to ~70% and reduced the CPU performance from ~23% to ~19%. this is all well and good, but is there any actual benefit.? What is the real-World benefit for reducing CPU load by 5% and GPU load by 15% for power consumption.? What is the real-World result of using the "AI engine" at 50% load for power consumption.? What is the end result on power consumption between the "AI engine" and GPU, and what is the real-World implication of the increased latency.?
.
Right now all I see for home users is "AI" being used as a buzzword, I have not seen any real world benefit, but as you have done a test I thought I would ask you for the power draw difference because that is ultimately what makes the difference where this CPU (Meteor Lake) is going to be predominantly used, laptops/notebooks/tablets that all rely on batteries unlike this mini-PC.
.
Many thanks and keep up the excellent content.
Gonna be interesting with Proxmox and heterogeneous cores.
afaik that should be fine with proxmox 8, it has kernel 6.x and later
Can you tell me if World of Tanks will play on it and how much power consumption it uses??
When will it be available?
They still dont sell to individual ?
You can buy these on Newegg (we put the TH-cam product tag around 1 min into the video
Asus nuc 14 pro & 14 pro+ ? when can we expect review ?
Hoping later this quarter for a few of those NUCs. We just got the NUC 13 Rugged and I am hoping we get 1-3 more and then maybe do a round-up style.
I would like to see a vid about Asrock 4x4 Box 8840u (AMD)
I hate it when they don't put a USB-C DP on the back though, its stupid plugging it into the front and its much neater to use for a monitor on these small units.
love these little computers but they need more USB ports !!
Wifi range problem might because it's an older revision 002? the latest is rev 006.
They call it the easy bake oven. Cook everything cook everthing!
I have a Coral in my Server as a pcie Card but having it in a CPU build in is nice but using it is actual pain. I think this will Change in the Future so i have high Hops and such system are perfekt for learning and developing Software with Edge tpu
Did I miss you mentioning the price??
yup main question
I'd love to get such a thing without fans / passively cooled.
I agree that would be cool, but it would be bigger.
@@ServeTheHomeVideoFor my desk, that would be OK. I'm typing this on my Zotac ZBOX CI669 right now, with "only" 12 cores. ;-)
Is this very different from a NUC 13?
Raptor Lake versus the newer Meteor Lake. This is a good step up.
Could someone help me find the software that used to generated the block diagram shown at 19:35? Thanks in advance!
lstopo
@@pwkerney Thank you!
minisPC got the ITX one much better due to a PCI-e, so the MS1 with out a PCI-e same chip I guess.
We need AI benchmarks...
That PsU hahaha. God damnit. Why not USBC PD like the Asus PN64?
Just wait for the next ASUS NUC video. I think we are going from a NUC that uses ~10W at the low end to one that uses like 300W at the high end. BIG ~330W PSU IIRC.
Meteor Lake benchmarks only look good if you don't compare them against Zen4!
But the Intel NPU is easier to use right now
@@ServeTheHomeVideo How so?
The AI Guru software can get everything installed and running with a few clicks and such. Intel has been doing OpenVINO for a long time.
Proxmox and linux? It was me missing that or...
I want one. wow
What apps are you using to stress test and visualize the load @14:50?
Asking the important questions. Anyone?
s-tui
He did mention stress-ng and earlier some said it looks like s-tui.
Thanks for the video and review. Can it be run directly from a nominal 12 volt battery?
Could have literally just looked it up in the time it spent to ask a question that will not be answered.
@@LackofFaithify Yes I could and will, but I am also trying to encourage NUC size computer makers to offer versions that work on 12, 24 and 48 volt battery packs.
These can be perfect for off grid and mobile use. There is a 30 watt standby power consumption in the inverter so if the built in power system can deal with it, it's super useful.
Bit expensive at $700 but interesting system. Can't wait till CWWK starts copying these.
Intel's new naming is just confusing.
Agreed.
Both Intel and Amd in the latest gen...two megacompanys shouldent be able to screw naming up this bad.
Maybe it's good for work, but Intel gpu... Meh.
I have minisforum with ryzen 7840hs + 64gb ram + 990pro for work and 970 for windows and games, so, I can play something like gta 5 sometimes. And I usually don't hear it!
Btw, I would put 96gb ecc ram, because I reboot it once per week and think about ramdisk for ide cache, but it's not so easy to find good 48gb so-dimm ecc modules(
Is the cost a secret !
When I see Ultra name in new intel CPU inside the mini PC huehue ULTRA ehh.
Get a quote and come on down to play at the Price is Right. ;)
Meteor Lake. A good name would be cluster lake considering how it is made. Then when they start failing people will give them a similar sounding yet appropriate name hehehe. They might call it cluster flake or cluster something else...
i don't understand why they don't put the power supply inside the case, if this is an industrial thing. the mess of cables is maddening.
Maybe 40gbps
You can do up to 25gbps per thunderbolt
The challenge is that 25GbE adapters use more power and get hot so you need fanned enclosures, but you are correct.
The WiFi 7 standard isn't even finished yet.. lets not ask for production computers to include that just yet.
Released Jan 8, 2024 and with each generation vendors usually have draft devices that can be firmware updated to the final specs
@@ServeTheHomeVideo
Yeah ieee released 802.11be as a draft. But
that doesn’t change that it won’t be finished before q4, and adoption will probably start to take place in the early 2025.
I work as an SE with major Lan edge vendors, they are starting to put out 802.11be capable devices but they’re nowhere near ready yet for production yet. This also goes for running STA or radio/AP in 11.ax, 11.ac or legacy with slower phy. Sure firmware can be upgraded. But in the enterprise there are still hardware revisions and factory callbacks for 6e -wich just now is getting mainstream.
My point is, it should be done and stable before we start asking for it in production devices, especially mini “servers”. Right now its out for testing, and is mostly poc.
Looks at the insufficient cooling...I give it 3 months life at best with a CPU turbo temp of 110 Celsius.
the Fans ALWAYS suck in this form factor. I've replaced many of them
I wish they made the chassis bigger with better cooling in almost every mini PC
Interesting. NPU in a such small box.
Not how you had to leave out the "intel" out or the description, and only mention it a little while into the video. I think a lot of viewers simply wont bother if they new it was an intel product. I gave this a thumbs up anyway.
put the PSU inside and make cooling not junk please
also, putting DP on the front isn't exactly convenient, also overall putting C on the front doesn't work that well yet either
You buy 16 cores and what do you get? Another day older and deeper in debt. Saint Peter don't you call me 'cause I can't go, I owe my soul to a micrometer store.
Microcenter, dammit!
There are two types
Of computers
Fanless( Phones)
And noisy
You seriously need a compressor for your microphone system.
$700 Barebones?! Seriously? I swear, the rising cost of GPU MSRPs *coughnvidiacough* has shown manufacturers that consumers will pay increased prices across the board to the point that you now are paying a bit under $1k (after tax) to get a box that is unusable without another $400-500 worth of components just to get it running.
I won’t let Nvidia take all the blame though, as Intel has been bleeding money trying to keep up with AMD for years now (via R&D, Fab expenditure, and deals (aka subsidies) with resellers to carry their CPUs in laptops, tablets etc). And since Apple dropped Intel when they released the M1, Intel lost even more $. Now they’re passing the cost of all those endeavors to us, individual consumers. They can’t raise prices for enterprise too much because AMD has them completely beat there with TR and Epyc, so we get this $700 paperweight instead.
Yes but they are Intel cores
If they are moving the power outside of the chassis, then of course they can make a mini PC. But it isn't really a mini PC, it's a wink-wink nudge-nudge mini PC.
I liked this video.
Thanks!
"Industrial" but without ECC 😤
I wish it had an SFP+
I wish these would be AMD powered and it would be something to consider. Intel is direct pass.
780€ -.-
Nothing to see here, google says it cost $700+
700$ is much money no thanks
7W at idle seems a bit high. I'm used to seeing under 2W.
There are no mini pcs with an idle power at the wall of 2W. Stop lying.
There are SBCs that can hit that, but not mini PCs out of the box like this
But are you hitting that whilst having drives connected and a graphics card? 7W idle is pretty impressive for the amount of computing power this system has.
lmao. Complaining about loud at 44db is hilarious. That is not loud.
lol... cant believe companies are really slapping on "AI" on everything just to make it look cool and latest tech, bs...
I somewhat agree, but this is a new added NPU silicon designed specifically for that type of math operation. In industrial, small business, warehouses, homes and elsewhere that is the kind of AI compute that is getting adopted rapidly
@@ServeTheHomeVideo NPU is just another cpu with optimized instructional set to do specifically neural computation algorithms and ML task. Similar to GPU doing graphics related computation. But you have a point... it is doing AI stuff..lol
So a device that implements dedicated hardware accelerators for AI is "slapping AI on everything"?
@@marcogenovesi8570 YES!! you can do the same with any CPU
All my PC's have had Artificial Intelligence right outside of the box, since TRS80 Model 1 .... me!!! 😅🙂
You are doing something wrong!
12:21 Inference GPU only: 73% GPU load
12:47 Inference with NPU: 86% GPU load
I call BS on that video. You are clearly advertising something, that your video proves is wrong!?
EACH TIME the GPU had to encode, so dont use this as a excuse!
If families were financially as responsible as Corporate America once was, something that costs as much as new washer, dryer, icebox and oven requires #JUSTIFICATION. Could the family survive with a certified pre-owned 4 core PowerPC Mac that's seven years old?
What will this family use the computer for?
I have a label maker
You need to mark this as Sponsored or Exclusive Access or whatever.
USB camara.
It was a Sony FX30
@@ServeTheHomeVideoNo, I mean the typo in the Asrock AI Guru. Look closely 🙂
My socks are Ai 😂😂 fking excuse to sell junk
It's 2024. We need 10Gig Ethernet Ports. Not old 2.5! Great Review though!!
2.5gig is technically newer than 10 gig…
It’s just that… 10gig switches are still quite expensive…
LMAO 🤣 I know they added 2.5 gig as a joke! My whole house is 10gig as well as routers and switches. Updated when Wifi 7 came out a while back. Tech needs to improve faster than slower! I know tech, I'm an IT Professional so thanks for stating the obvious! @@Comeyd
Industrial ?
A box in like a big radiator in aluminum and dust proof is industrial.
This thing in plastic looks poor fragile thing !
The form factor can’t be achieved with inbuilt power supply. The power circuits need additional cooling. I get your point though with respect to the cables.
Dude you obviously don't design and still your chatterboxing... Damn
2.5G lans, yikes.
2x 2.5GbE is pretty decent now.