@@mentalmarvin "THIS IS THE WORLD'S FASTEST OUTDATED VIDEO". Well, at least the title is. The comparison is still very relevant because the M2 Pro's will be on sale!
If I’m not mistaken, the Mac version of the LLM uses the “NPU” of the M-series processor. It would be great to see the AMD NPU in action. Someone on HuggingFace (dahara1/llama3-8b-amd-npu) has already figured out how to run it on an AMD NPU. In his example, it’s a 7840u CPU with a first-gen NPU and 16 TOPs. It would be interesting to see what the AI 9 370 with a third-gen NPU and 50 TOPs can achieve
One of the big reasons I'm using a mac is power efficiency (while still being powerful), so if you could also measure the power being used on the most heavy tasks it would be very useful, to see if the extra performance is worth it.
I bought one of these and put a 4 TB Samsung 990 pro in it as the boot drive. Can’t beat the price and value combo. I found with the 990 pro as the boot drive and keeping it in high performance mode, the mini PC is a beast. I left the 1 TB drive it came with as a second drive, so I have 5 TB of storage and great performance at a total price of just under $1,300.
@CAG1399 I have seen Thunderbolt 3 enclosures with 8TB of storage for about £400, LaCie on Amazon UK. That would be enough for most users. Fast read and writes as well.
I've been thinking why the Mac Mini generally out-performs the AMD mini PC with your tests - probably because AMD and Intel are doing a lot more prediction optimizations compared to Apple to compensate for a worse raw performance compared to the M2 - so simple tests like the sorting algorithm can't be branch-predicted as much for the AMD cpu to pull ahead, but when running python it can finally pull ahead, since intepretting the python code is probably full of branches.
My bro I can't believe you put this video 1h before they announced the new Mini. No worries, we appreciate all your hard work! Can't wait for the comparison with the new Mini.
BTW, I believe that -O3 does not necessarily optimize for that specific processor, but for generic 64 bit processor. You might want to add -mtune=native -march=native to your compilation flags
Can this run Llama 3.1 70b q4 or even higher? I will most likely wait for the Strix Halo chips but it would be interesting to see what is possible with these Strix Point chips and 32GB of RAM.
Hi Alex! Thanks for the LLM GPU test 😌 I hope you will compare the same quantization next time, because in this video you are benchmarking a 4-bit quantization vs 8-bit quantization. On my M1 Max 64GB, I measure (prompt: Please write a story in 1000 words) LLama 3.2 1B mlx 4-bit: 173.06 tok/sec LLama 3.2 1B mlx 8-bit: 134.80 tok/sec LLama 3.2 1B GGUF Q4_0: 172.09 tok/sec LLama 3.2 1B GGUF Q8_0: 133.63 tok/sec mlx or gguf make no difference, but the quantization does very much!
I guess I'll just keep my M2 Pro mini. It's more than I'll ever need. I use the USB for dongle to read my old camera flash drives. My USB C's are for backup drives.
Against the M2 Pro, I would take the BeeLink SER9. Full X86 access and gaming is too valuable for me. Although the port locations are aligned better for me. Another option is the Geekom A8. Far better port alignment than BeeLink SER9, but a previous gen iGPU.
I think the way the Windows seems to scuttle AMD Zen 5 performance it might be worth the effort to run the tests under Linux, either native install or even WSL.
The RAM speed of the M-chips are almost always faster than the "PC" counterpart. As CPUs are very fast these days, they are usually limited by the RAM speed. Hence, you see these results...
can you run native linux on the new mini pc and retry the benchmarks, just curious to see how well it performs, since it is fairly new hardware and everything might not work yet on linux.
It's amazing how well the mac did, even in the merge sort parallel test. Apple needs to improve the AI acceleration. The new Mac Minis are in and with Thunderbolt 5.
I just watched the Apple announcement for the mini after the iMac yesterday (I still regret they no longer make the 27"). Though it seems compelling, what do you all think about this new way for Apple to announce new products? Do you think they are doing an experiment? Or is it a new style that is here to stay? I don't know what to think yet.
Mate !! You gotta check out the M4 mini. That has some really decent pricing with specifications it's providing. If you go education pricing it get's even better. I can't wait for what apple does with their macbooks, but I have really low expectations in context of it's pricing. Now the mac mini is even smaller, I don't find a reason why not just carry a portable display or something like a nextdoc and the mac mini
Well Mac Mini M4 already announced. It's awesome though. But I still remember if Machine talked in this video is still has wider app or program support, since it run AMD chip 😅
Ugh, I know windows has to work on allot of machines so it's not easy to make it more optimized but also.. like how long has this problem existed? And can the same be said for Android?
So that AMD chip actually has dedicated AI part (called NPU), that should have 50TOPS performance, probably at least 3 times faster that integrated GPU you were using. I do not know if LM Studio can use it though (for now)
Around 4 minutes and 20 there’s a camera swap position. Please could you change the scenery swaps to something less bright because it blinded me hard. I understand this may be like a lot but my iPad m4 started spazzing out after that it really did hurt… please.
Well, I’m betting that Apple Boot Camp is coming really soon. Really soon as soon as the transition happens smoothly in windows environment for arm, Apple would definitely bring back the Boot Camp. It will sell like fire and to be honest I miss it so much, especially when they merge it with the Rosetta, oh my heart if they did this genius move and make us just install windows on Mac and run the windows apps with the environment using the windows colonel that we have installed and dude the right way better than Lennox on Windows did good Lord good Lord it will be magnificent. It will not replace parallels, but it will make parallels and tools like It Have potentials like have never been before in VMs
The only thing that I wish Apple do better is GB if they just focus more in have better way better use experience that bars with EMD at least and give us courses for AI acts like tins or chords for granted and made it like you know what I’m giving you that much that nobody would give This shit will go like fire fire, like we all will migrate back to Mac even if we cannot afford it lately because it will be the lifetime investment and we will be hooked again but this time for life I wish they do this bold move and have the balls to do it. I wish they dobecause I’m gonna buy Mac actually max with S this time.
I used to love Beelink, I have 2, but that model made me u-turn, the insane price and soldered memory has done it for me, I hope they realize they are not Apple 😠
$1300 for the 32GB 1TB M4 Mac Mini, so $300 more than the beelink ser9. Yes, SoCs are going in the direction of soldering RAM and there’s a good reason for that. even though many tinkerers won’t like it, they are the few.
@@AZisk I asked another influencer for a real benchmark between soldered and SODIMM I have a 7840HS laptop and SER7 Beelink 7840HS, both 32gb, laptop memory soldered with higher memory speed, the SER7 with SODIMM outperform the laptop on most task because of higher power limits Funny the other vlogger brought up Dunning Kruger Effect on me because I don't believe his explanation without real benchmark that soldered memory are significantly faster, I said DKE has no effect on me and just popularized by companies who cannot backup their claims by saying their are experts and you are not, a total BS in the age of data analytics
@@rommellagera8543 I would love to be able to do that test myself, but unfortunately, companies aren't too keen on making the exact same chassis, with the exact same CPUs, and still offer two different memory options. So guesses will have to be made. Hopefully educated ones.
It is quite sad that 32GB of RAM are old 16GB, 16GB are old 8GB and 8GB are old 4GB. I am not taking Linux into account, Debian 12 is good at 1GB of RAM.
1:35 Saying with a straight face "this machine is not cheap" (999USD) while he knows the same speced mini is almost two times more! EDIT: New MacMini with M4 Pro, 48Gb RAM (no 32Gb possible) and 1Tb of disk is 1999USD!
That is misleading. The base M4 Mac Mini will compete well with this, and you only have to upgrade the RAM to 32GB for $200, so it's around $799. Use external drives for storage, as it has superfast Thunderbolt 5. In the latest macOS, you can run apps off the storage, even the OS, if you need to or want to have family members with their own external SSDs.
@@andyH_England How is that misleading? Both are mini computers, specced closely. Why do you have need to defend someone who is obviously so greedy? I mean, the point of my message was, customer is being obviously overcharged in most blatant way. They ask you (ask their customer) to pay 4x more (than usual market prices) for something and you defend them ?!!! Wow The only thing I can say: Tim , is that you?
@AZisk No, its great. I have M2 Max 14" mbp and it was fantastic upgrade from my 2016 intel mbp. I got it mostly for dev in Unity, and for that, its great.
Does it matter? It can't go beyond 32 GB of RAM, and I haven't seen a Ryzen 9 HX 370 chip shipping with more than 32 GB. I wouldn't buy a 32 GB box in 2024 for LLMs.
Notice how the mobile AMD cpu can't go past 3.3 Ghz on all the cores Its power clearly power limited which desktop cpus are not and why I am not a fan of these mini pcs. I'm sure the Macs cpu can go higher all core.
lol… now you need to remake this - apple just released their new one about 10 minutes after this went live 😂 timing!?! although still useful for people of course …
@@AZisk looking forward to it ( and apologies if that comment sounded rude! Not my intention- love your channel ☺️ I just thought it was funny how I had a notification from you and then apple right after for the mini ☺️🫶🏻)
I get nervous break downs when ever I hear any sort of computer noise... the new baseline M4 is a no-brainer finally. Silent, speedy, enough RAM, affordable
....aaaaaand the M4 Mac Mini's are announced now. Can't wait to get them in here to test them.
Your video got outdated in minutes :)
😂😅
😂
@@mentalmarvin "THIS IS THE WORLD'S FASTEST OUTDATED VIDEO". Well, at least the title is. The comparison is still very relevant because the M2 Pro's will be on sale!
Or he read the tea leaves right and played the TH-cam game to perfection
And Apple just announced the M4 with the size of almost an Apple TV.
What a timing, haha.
We all knew it was coming. I just ordered a couple of them to test on the channel.
new apple mini is only 1gb ethernet
@@rtorcato most people dont have past gigabit
@@rtorcato There is a 10Gb option for $100 extra. Well worth it
i thought for a second that u might've gotten the new M4 mac mini early to review and then I realise it's a mini PC lookalike
I'm not in Russia, I don't get early Macs :)
I thought the same. Thought Apple sent him for review.
🤣
@@AZiskdamn😂😂😂😂
@@AZiskdamn😂😂😂😂
If I’m not mistaken, the Mac version of the LLM uses the “NPU” of the M-series processor. It would be great to see the AMD NPU in action. Someone on HuggingFace (dahara1/llama3-8b-amd-npu) has already figured out how to run it on an AMD NPU. In his example, it’s a 7840u CPU with a first-gen NPU and 16 TOPs. It would be interesting to see what the AI 9 370 with a third-gen NPU and 50 TOPs can achieve
you released this just after the M4 Mac Mini was announced... by a few minutes lol
Amazing timing!
bad jokes are fine, as long as they are hilariously bad
I'm pretty good at bad jokes
The Schwarzenegger does not get old, I still love it
He's a machine!
One of the big reasons I'm using a mac is power efficiency (while still being powerful), so if you could also measure the power being used on the most heavy tasks it would be very useful, to see if the extra performance is worth it.
Thunderbolt 5 and a memory bus speed of 273 GB/s on the Pro. I'm liking it!! Just waiting for test results to tweak the configuration.
Which means the M4 max should at least hit well over 500gb. An ultra would hit over 1000GB/s!
I bought one of these and put a 4 TB Samsung 990 pro in it as the boot drive. Can’t beat the price and value combo. I found with the 990 pro as the boot drive and keeping it in high performance mode, the mini PC is a beast. I left the 1 TB drive it came with as a second drive, so I have 5 TB of storage and great performance at a total price of just under $1,300.
You can do that with the new Mac mini with TB5 for a similar price.
@@andyH_England I saw a price of $500 more for 2 TB. Wish it could be done for a similar price, that also includes the 32GB of ram this comes with.
@CAG1399 I have seen Thunderbolt 3 enclosures with 8TB of storage for about £400, LaCie on Amazon UK. That would be enough for most users. Fast read and writes as well.
I've been thinking why the Mac Mini generally out-performs the AMD mini PC with your tests - probably because AMD and Intel are doing a lot more prediction optimizations compared to Apple to compensate for a worse raw performance compared to the M2 - so simple tests like the sorting algorithm can't be branch-predicted as much for the AMD cpu to pull ahead, but when running python it can finally pull ahead, since intepretting the python code is probably full of branches.
Interesting analysis. Thanks!
My bro I can't believe you put this video 1h before they announced the new Mini. No worries, we appreciate all your hard work! Can't wait for the comparison with the new Mini.
Wow, the timing of this video is ... perfect! 😏
Hey Alex, Love your content and I follow it more sincerely than another channel. Keep up the good work!
Thanks for the support! 🙏
BTW, I believe that -O3 does not necessarily optimize for that specific processor, but for generic 64 bit processor. You might want to add -mtune=native -march=native to your compilation flags
Nerds in nerd environments, what a great atmosphere. LOVE THIS content
Can this run Llama 3.1 70b q4 or even higher? I will most likely wait for the Strix Halo chips but it would be interesting to see what is possible with these Strix Point chips and 32GB of RAM.
Hi Alex! Thanks for the LLM GPU test 😌 I hope you will compare the same quantization next time, because in this video you are benchmarking a 4-bit quantization vs 8-bit quantization.
On my M1 Max 64GB, I measure (prompt: Please write a story in 1000 words)
LLama 3.2 1B mlx 4-bit: 173.06 tok/sec
LLama 3.2 1B mlx 8-bit: 134.80 tok/sec
LLama 3.2 1B GGUF Q4_0: 172.09 tok/sec
LLama 3.2 1B GGUF Q8_0: 133.63 tok/sec
mlx or gguf make no difference, but the quantization does very much!
New minis are here and they look mini can't wait for test with those
I guess I'll just keep my M2 Pro mini. It's more than I'll ever need. I use the USB for dongle to read my old camera flash drives. My USB C's are for backup drives.
Against the M2 Pro, I would take the BeeLink SER9. Full X86 access and gaming is too valuable for me. Although the port locations are aligned better for me.
Another option is the Geekom A8. Far better port alignment than BeeLink SER9, but a previous gen iGPU.
Question, isn’t edge based on chromium so there shouldn’t be any difference between it and chrome?
I think the way the Windows seems to scuttle AMD Zen 5 performance it might be worth the effort to run the tests under Linux, either native install or even WSL.
love the pink counterweight
the tdp and wattage is the same in both?
The RAM speed of the M-chips are almost always faster than the "PC" counterpart. As CPUs are very fast these days, they are usually limited by the RAM speed. Hence, you see these results...
The m4 studios are going to be nuts.
can you run native linux on the new mini pc and retry the benchmarks, just curious to see how well it performs, since it is fairly new hardware and everything might not work yet on linux.
It's amazing how well the mac did, even in the merge sort parallel test. Apple needs to improve the AI acceleration. The new Mac Minis are in and with Thunderbolt 5.
I just watched the Apple announcement for the mini after the iMac yesterday (I still regret they no longer make the 27"). Though it seems compelling, what do you all think about this new way for Apple to announce new products? Do you think they are doing an experiment? Or is it a new style that is here to stay?
I don't know what to think yet.
I think they are trying it out, and it might work well for them. But I don’t think they are done with “events”
Hey, uh, there was just a bit more text edited into this one then I prefer. Love the vids
The pink weight :) even counter weights need style!
I'd be interested to see how the perform would compare on the Beelink running a Linux distro
Can’t wait to see the testing and thermals mainly ❤
Which C# Mandelbrot app are you using?
Hey THE Damien Edwards? It's the one found on this site: benchmarksgame-team.pages.debian.net/benchmarksgame/index.html
My question on the Mandelbrot Test is why it was faster on the mini PC in Python than in C#??
Could you do the same tests in Ubuntu to see the difference in dotnet/python performance on the same hardware?
Was the worlds most powerful Mini PC until that 14 core behemoth known as M4 Pro was announced! But I'm sure I'm not the first to comment this LOL!!!!
yeah, some people might have mentioned this. Lol.
Is the old Mac-mini smaller by volume?
No way, the old mac mini is larger
Mate !! You gotta check out the M4 mini. That has some really decent pricing with specifications it's providing. If you go education pricing it get's even better. I can't wait for what apple does with their macbooks, but I have really low expectations in context of it's pricing. Now the mac mini is even smaller, I don't find a reason why not just carry a portable display or something like a nextdoc and the mac mini
i know, looks good! i ordered a few
@@AZisk A few... Jealous 😅
Well Mac Mini M4 already announced. It's awesome though. But I still remember if Machine talked in this video is still has wider app or program support, since it run AMD chip 😅
When you do the M4 MacMini video please include how many comments mentioned the timing of this video and the Apple release.
It will be probably 90% of comments :)
The m4 mac mini looks IDENTICAL to this lmao
Ugh, I know windows has to work on allot of machines so it's not easy to make it more optimized but also.. like how long has this problem existed? And can the same be said for Android?
Yep, I want to see a video on the new Mac mini….. test that bad boy
So that AMD chip actually has dedicated AI part (called NPU), that should have 50TOPS performance, probably at least 3 times faster that integrated GPU you were using.
I do not know if LM Studio can use it though (for now)
LM Studio doesn't seem to tap into it. There are some libs that tap into it, but they need to be integrated
Monitor link?
Imagine: The M4 Mac Mini should be a lot faster.
Exactly when apple shows off the m4 mac mini?
Yep. Amazing timing :)
Apple makes the M4 Mac mini with up to 64GB of RAM, AMD's Ryzen AI 9 HX models top out at 32 GB.
@@SaschaRobitzkiAlso Mac Mini pro with that 64gb's price is very affordable, for me.
We’ll. Update of video imminent. That new MAC Mini M4 just destroyed everything.
Alex, I like your video , thank u
Thank you too!
Great timing! Do it again with m4 pro! 😀😀😀😀😀😀
12:09 You can specify the GPU offload in LM Studio for Mac too. It' on the settings side bar.
Why not use the NPUs for the LLMs?
Maybe the end of the month, or the end of the week, or the end of the video.
😂 truth
But I would hate to imagine the power consumption of this Ryzen-based PC. And it would be no match for the new Mac mini in terms of power consumption.
Around 4 minutes and 20 there’s a camera swap position. Please could you change the scenery swaps to something less bright because it blinded me hard. I understand this may be like a lot but my iPad m4 started spazzing out after that it really did hurt… please.
i hope u ordered new mini mini m4 mac 😂.
a few (for testing)
The AMD mini PC looks like the new Mac Mini just has been release a few mins ago ... lol
fun fact, Mac mini is also a personal computer (a.k.a. PC)
nice timing
I know, right?
this looks like an old video, time to mke one with the mac mini m4 pro
Got 34 on the browser bench on iPhone 16 Pro Max using safari 😂
bruh this is definition of bad timing lmao.
My title still stands, at least for another week :)
I can see every office in America being full of these types of PCs if hp or dell get on the bandwagon.
It was! Now it's the Mac mini M4 Pro hehehe
It’s online now
even the module of the mac have 2b more of params stay better than the beelink :0
Please test tb5 external HDD asap
You timed this poorly lmao
did i?
Well it got smoked by the m4 mini and look at the price of the m4 mini 💀💀💀😂😂
The first thing I did with apple intelligence is to discover how to turn off apple intelligence.
Here after Apple announced the new M4 series Mac mini
Well, I’m betting that Apple Boot Camp is coming really soon. Really soon as soon as the transition happens smoothly in windows environment for arm, Apple would definitely bring back the Boot Camp. It will sell like fire and to be honest I miss it so much, especially when they merge it with the Rosetta, oh my heart if they did this genius move and make us just install windows on Mac and run the windows apps with the environment using the windows colonel that we have installed and dude the right way better than Lennox on Windows did good Lord good Lord it will be magnificent. It will not replace parallels, but it will make parallels and tools like It Have potentials like have never been before in VMs
The only thing that I wish Apple do better is GB if they just focus more in have better way better use experience that bars with EMD at least and give us courses for AI acts like tins or chords for granted and made it like you know what I’m giving you that much that nobody would give This shit will go like fire fire, like we all will migrate back to Mac even if we cannot afford it lately because it will be the lifetime investment and we will be hooked again but this time for life I wish they do this bold move and have the balls to do it. I wish they dobecause I’m gonna buy Mac actually max with S this time.
Try with Linux in the AMD.
I used to love Beelink, I have 2, but that model made me u-turn, the insane price and soldered memory has done it for me, I hope they realize they are not Apple 😠
$1300 for the 32GB 1TB M4 Mac Mini, so $300 more than the beelink ser9. Yes, SoCs are going in the direction of soldering RAM and there’s a good reason for that. even though many tinkerers won’t like it, they are the few.
@@AZisk I asked another influencer for a real benchmark between soldered and SODIMM
I have a 7840HS laptop and SER7 Beelink 7840HS, both 32gb, laptop memory soldered with higher memory speed, the SER7 with SODIMM outperform the laptop on most task because of higher power limits
Funny the other vlogger brought up Dunning Kruger Effect on me because I don't believe his explanation without real benchmark that soldered memory are significantly faster, I said DKE has no effect on me and just popularized by companies who cannot backup their claims by saying their are experts and you are not, a total BS in the age of data analytics
@@rommellagera8543 I would love to be able to do that test myself, but unfortunately, companies aren't too keen on making the exact same chassis, with the exact same CPUs, and still offer two different memory options. So guesses will have to be made. Hopefully educated ones.
The timing . . . Alex must hate Apple now
We all knew it was coming.
u have to test the m4 pro Mac mini
It is quite sad that 32GB of RAM are old 16GB, 16GB are old 8GB and 8GB are old 4GB. I am not taking Linux into account, Debian 12 is good at 1GB of RAM.
Great video just too many camera angle switches
yeah, a bit much. i agree
From Coding to ETA Prime YT channel
did some coding in this video
1:35 Saying with a straight face "this machine is not cheap" (999USD) while he knows the same speced mini is almost two times more!
EDIT: New MacMini with M4 Pro, 48Gb RAM (no 32Gb possible) and 1Tb of disk is 1999USD!
That is misleading. The base M4 Mac Mini will compete well with this, and you only have to upgrade the RAM to 32GB for $200, so it's around $799. Use external drives for storage, as it has superfast Thunderbolt 5. In the latest macOS, you can run apps off the storage, even the OS, if you need to or want to have family members with their own external SSDs.
Yes, and the M2 Pro I was holding in this vid isn't terrible at $1100.
@@andyH_England How is that misleading?
Both are mini computers, specced closely.
Why do you have need to defend someone who is obviously so greedy?
I mean, the point of my message was, customer is being obviously overcharged in most blatant way.
They ask you (ask their customer) to pay 4x more (than usual market prices) for something and you defend them ?!!! Wow
The only thing I can say: Tim , is that you?
@AZisk No, its great. I have M2 Max 14" mbp and it was fantastic upgrade from my 2016 intel mbp. I got it mostly for dev in Unity, and for that, its great.
hahah Good timing.. the m4 pro is out now...
Does it matter? It can't go beyond 32 GB of RAM, and I haven't seen a Ryzen 9 HX 370 chip shipping with more than 32 GB. I wouldn't buy a 32 GB box in 2024 for LLMs.
Yes, they are all stuck with the RAM they are born with. So if you need to run a larger LLM, this won't do, but for 2B-7B it might be perfect.
I am waiting for the new AMD Strix CPUs
I thought you got it for real 😂
I mean man mini 😅
Dude, you ran 1B on the Ryzen pc, and 3B on the Mac...
This video didn’t age well. I mean the timing 😂.
i’m counting the number
It did because the Mac Mini beat the mini PC. Basically is just adding "even more" to any of his tests 🤣
6:02 microsoft Edge keep the ADN of internet explorer
The upload timing is so weird and bad, comrade Alex! 😂😂
Notice how the mobile AMD cpu can't go past 3.3 Ghz on all the cores Its power clearly power limited which desktop cpus are not and why I am not a fan of these mini pcs. I'm sure the Macs cpu can go higher all core.
0:30 sometimes smaller is better
🤔🤔🤔
Aaaaaaaand Video immediately outdated 😂 great content tho!
lol… now you need to remake this - apple just released their new one about 10 minutes after this went live 😂 timing!?! although still useful for people of course …
i already ordered my m4 minis. Tests coming soon
@@AZisk looking forward to it ( and apologies if that comment sounded rude! Not my intention- love your channel ☺️ I just thought it was funny how I had a notification from you and then apple right after for the mini ☺️🫶🏻)
x86-64 is a dead man walking.
M4 Mac mini just announced 😂
so mac is still most powerful 🤔, guess it used less power too?
I get nervous break downs when ever I hear any sort of computer noise...
the new baseline M4 is a no-brainer finally. Silent, speedy, enough RAM, affordable
good timing lmao
to be honest you cant compete with Apple.............