Awesome video! I would love to see more LLM or other DL architecures benchmarked between the M3 Max and the RTX 4090m laptop. A definitive video saying the M3 Max is X% better/worse than the 4090m for RNN, CNN, or transformer architecutres would be a gold mine for other AI/ML devs like me!
@@stephanemignot100 of course man, if you plug it in it’s faster. If you leave it unplugged it slower I’m not debating the fact that the M3 Max is a wonderful chip. All I’m saying is that even the Nvidia 4090 at its peak capability is faster if you want to say that the battery is worse, absolutely not denying that but the M3 Max GPU is not faster than the 40,90
Watched tens of your videos before upgrading from my old i9 MacBook Pro to my M3 Max MacBook Pro. Nowadays I still watch your videos (even if I already have an M3 MacBook) because I like the way you make your content - pragmatism, tone of voice, length and cuts. 👏
@@divyanshbhutra5071Nvidia is working on ARM. They'll release something more powerful (even without tight optimization) than what Apple can ever hope to achieve.
WSL uses hyperv, there is no way around it. MSI laptops are always noisy. If you need a powerful and less noisy windows laptop then Lenovo Legion 9i is a better choice
Thank you! What is the correct way of comparing my current AMD Radeon Pro 5300M 4 GB (MacBook Pro 2019) to an Apple M silicons? In terms of a MacBook gaming experience. I am playing a game from time to time and would like to make sure that a M chip won't take it away from me :)
Two MacBook Pros died after 14 months. If I could buy. a new one every year, that would be just GREAT. 8GB of RAM is not enough but Apple figures that profits are better than selling a computer with enough memory to do the job. "Job" - does that remind you of someone??? Too bad we are Cooked.
Wow, exciting results! I was always optimistic that Apple's unified memory architecture would pay dividends in certain workloads, and MLX appears to be effectively exploiting that paradigm shift. Keep up the good work! Love the channel!
Actually it’s more like a 3090. The fact Apple actually gets that much performance from a SOC GPU is insane. I know AMD strix halo SOC is going to try and compete with M4.
Hmm this difference mayo is from ram/vram sharing on arm Macs. ARM GPU can use up to 75% of ram as vram. I don’t know that you’ve 64/96/128 RAM versions, but in all cases will be more vram than 20gb in 4090.
?... Lol are you aware that Nvidia is in the making of ARM SOC themselves. You know what that means... Dont you ?... I hate Nvidia pricing. But I know one thing. Thease guys dont play when it comes to performance. Every one knows that when Nvidia releases ARM based SOC in upcoming years... Its gonna destroy everything on the market. Like it always does. Also... This laptop does NOT have RTX 4090. Not even close...
@@gytispranskunas4984 why do you hate nvidia pricing? They cost same as AMD but providing RT cores, cuda and they are more stable. Quality and R&D costs money too
So many tech-bros on the net bragging about their AI on 4090's using Python, AS IF using Python is something about which to brag (when it comes to performance or efficiency.)
Actually, if this is true then you didn't pick the best machine for competition cuz there are bazillion non apple laptops, the mathematical consequence is that one of them has to beat the mac, so clickbaiting us with this title is awful
What’s your use case? The battery life on even the M1/M2 chips is phenomenal, the M3 chip mostly just adds performance. If you’re using it for light tasks, save some $$$ and get an M1 or M2 series chip
Doesn't matter if there's like zero software to use on silicon, it just is thar devs always do windows, only billionaire devs support mac, or browser game devs
Thanks for the analysis. I'm a Java developer. I want to get into AI for Python, C++, and Java. I'm looking for a new laptop. I've never owned a Mac. Because laptop GPUs are not swappable, that's really the driving factor. If you had to buy one or the other for AI software development, would you buy the M3 Max or an Intel RTX 4090m or something else?
Alex, I found your channel when researching for my M3 max laptop purchase. I love your benchmark methodology, but also wish I could copy some of your workflows. If you added a code repository to your membership, I would join!
As much as I'd like you to join, there is no need to join to see my repos. This is a "better late than never" repo of my tests which I recently started: github.com/alexziskind1/machine_tests
You forget something, when you tried to make a benchmark you faced the same issue, you couldn't use the whole performance of the GPU/CPU when you used Windows or WSL, and you achieved that when moved to Linux. please do it and tell me the results. I love your videos.
Part of Apple's long game here is to absolutely dominate the mobile market in every way, and part of that domination is going to require robust machine learning capabilities and speed even for small models that are better suited for mobile uses of machine learning applications. They make their machines able to run small models insanely fast and that's where they're going to have a huge edge in the future
Have not had good luck running ai workloads on wsl or wsl2 with a discrete gpu. Everything says my gpu is being used incl docs but performance is pathetic.
My apologies if I am being dumb, by why wouldn't you use an NPU for this machine learning process, as I thought this is the sought of task NPUs were designed for, and maybe even better at than a GPU? And if you could, how would the performance compare when running on an Apple Silicon NPU (on paper M3 NPU is 18 TOPS for FP16)? And as every processor manufacturer is now getting on the AI bandwagon, you could even extend it to compare the performance of AMD 7000 series with AI NPU (10 TOPS, 8000 series NPUs 16 TOPS) or Intel's Meteor Lake core Ultra with NPU (10 TOPS)? Of course, the processor I would really like to see would be Qualcomm's Snapdragon X Elite with its 45 TOPS NPU, but that's yet to be released.
Anybody have a roadmap for me to learn on what about a language or framework performs better on one arch or another. How clever can tensor operations get? Python I get. But what is it b/w mlx, cpp and ggml, jax and mojo?
Hey Alex, I was wondering if you have a video planned for your EDC as a software engineer. I’ve been looking for a light case that I can carry around for my 16 MacBook with the 12.9 iPad. Trying to get ideas of what you utilize
Well, you need to carefully specify the use case of the ram. In AI world, the only ram matters is the one on graphic card and it is not relatively cheaper to upgrade compare to mac
Have you tried timing all the machines with the model already loaded in the GPU's ram to test the raw compute power? It would also be a fairer comparison with cloud-hosted solutions. Anyways, wild that Apple hasn't sent anything to the only ML/AI reviewer on TH-cam. AI/ML is the core reason for me to update from M1/2 to M3 Max.
Great video, Alex! You have some really enjoyable content on your channel. Are you able to send me one of your old M-series Macs; I’m a student and I’m trying to learn some ML/AI stuff.
Serious question: why would anyone buy a windows pc when you can buy a Mac that not only can run windows on it but runs windows BETTER than a windows pc??? In buying a computer soon and would appreciate the feedback. Thanks.
If power usage isn't your concern, then a PC can and will be faster. 4th Gen Core i9 + RTX 4090 will likely dominate in all benchmarks. For truly mobile performance (as in on battery, not plugged into a wall), Apple undeniably has the best product on the market right now. So long as you don't want to play any games on it.
for mobile platforms apple makes sense, for in house usage, it still lags behind by a lot, unless you are already deep into the apple ecosystem or simply prefer it, for pretty much every benchmark the only metric apple is going to win is in power usage which matters a lot in laptops, in desktop, not so much when while using more power, will get the job done far quicker.
What is the purpose of this computing power? Do you need it every moment of your day? And if you don't have it, is it a serious issue? I have a Mac Mini M2 at home. I also have 2 Windows PCs. I have no affection for these two machines that heat up, blow, scream, make a loud noise to obtain the power you're talking about. Not to mention the poor quality of plastics that crack and the miserable battery life of the laptop (whose power supply is larger and heavier than my Mac Mini M2). The production of PCs should be stopped.
JOIN: youtube.com/@azisk/join
Awesome video! I would love to see more LLM or other DL architecures benchmarked between the M3 Max and the RTX 4090m laptop. A definitive video saying the M3 Max is X% better/worse than the 4090m for RNN, CNN, or transformer architecutres would be a gold mine for other AI/ML devs like me!
No it’s not faster. You’re not using fast whisper. Also python implementation absolutely uses the gpu. Set device to mps
Yes can we discuss for setting hardware for building llm
Try that unplugged...
@@stephanemignot100 of course man, if you plug it in it’s faster. If you leave it unplugged it slower I’m not debating the fact that the M3 Max is a wonderful chip. All I’m saying is that even the Nvidia 4090 at its peak capability is faster if you want to say that the battery is worse, absolutely not denying that but the M3 Max GPU is not faster than the 40,90
@@asjsjsienxjsks6734090 doesn’t have 19GB VRAM 😂
@@RunForPeace-hk1cu where did I say that?
yeah but can it run crysis
yeah but can crysis feed family? idk
@@adrimi5 family goes on diet after each new pro device gets released
Indeed it can, if you know the right toolkits to download and terminal commands to run
Yep! Using Crossover.
I’m running GOG Cyberpunk 2077 on my M3 Max using crossover.
too bad the proprietary silicon is anchored to the pos company which is apple, I don't want to spend 800 dollars on an extra 64gb of memory.
Watched tens of your videos before upgrading from my old i9 MacBook Pro to my M3 Max MacBook Pro.
Nowadays I still watch your videos (even if I already have an M3 MacBook) because I like the way you make your content - pragmatism, tone of voice, length and cuts.
👏
How much RAM did you get? I cannot decide between 36 GB, 48 GB or maybe even 64 GB (for future proofing).
@@tybaltmercutio same situation as OP, went with 14" and 64gb ram
i just keep my i9 macbook pro alongside with alienware rtx4090
Nvidia seriously needs to up the game with VRAM capacity. But why would they, when their competitors are as useless as Intel and AMD.
or apple
h1874 Apple chips have a lot of memory
@@divyanshbhutra5071Nvidia is working on ARM.
They'll release something more powerful (even without tight optimization) than what Apple can ever hope to achieve.
And kill off the h100 market? 😂😂😂😂😂
You’re so naive
@@utkarsh1874m2ultra has 192gb memory 😂😂😂😂 what are u on about?
WSL uses hyperv, there is no way around it.
MSI laptops are always noisy. If you need a powerful and less noisy windows laptop then Lenovo Legion 9i is a better choice
Haven't tried that one yet. Thanks
Found your channel from Fireship vid ~2ya. Awesome stuff!
Hey, amaizing video very useful, 5:18 - i am interesting to see the video how to install whisper with support of GPU etc.
Coming soon!
@@AZisk - i already testing with Nvidia P40, but its was interesting to see your results
Thank you! What is the correct way of comparing my current AMD Radeon Pro 5300M 4 GB (MacBook Pro 2019) to an Apple M silicons? In terms of a MacBook gaming experience. I am playing a game from time to time and would like to make sure that a M chip won't take it away from me :)
Why don't you run Linux on the 4090 PC
In French bilot sounds like “be low”
Two MacBook Pros died after 14 months. If I could buy. a new one every year, that would be just GREAT.
8GB of RAM is not enough but Apple figures that profits are better than selling a computer with enough memory to do the job. "Job" - does that remind you of someone??? Too bad we are Cooked.
Hi Alex! ❤ love ya my guy 😊your videos are incredible! Can’t wait to fork 🍴
Wow, exciting results! I was always optimistic that Apple's unified memory architecture would pay dividends in certain workloads, and MLX appears to be effectively exploiting that paradigm shift.
Keep up the good work! Love the channel!
Very nice Video, but can you try Faster Whisper for python on your the devices?
RTX 4090m is equivalent to the desktop RTX 3080 btw.
Actually it’s more like a 3090. The fact Apple actually gets that much performance from a SOC GPU is insane.
I know AMD strix halo SOC is going to try and compete with M4.
Can you make iPad and iPhone app versions of these tests so we can benchmark m4 on iPad in couple of days?
PC owners came just for commenting hate and useless personal opinions, pathetic.
Hmm this difference mayo is from ram/vram sharing on arm Macs.
ARM GPU can use up to 75% of ram as vram. I don’t know that you’ve 64/96/128 RAM versions, but in all cases will be more vram than 20gb in 4090.
Soooo, the real title of this video should be MLX extremely poorly optimized for CUDA cores.
MLX does not run on PC’s and there are no CUDA cores on Apple Silicon 🤷♂
"PC Master Race" on suicide watch !! 😂
(and yes, it's quite probably the M-series chips' Unified Memory architecture that's making the difference here)
lolz....the limitation here is PCIe bottleneck...not Nvidia GPU.... if NVLink protocol running on PC it's will destroy day and night M3 max
?... Lol are you aware that Nvidia is in the making of ARM SOC themselves. You know what that means... Dont you ?... I hate Nvidia pricing. But I know one thing. Thease guys dont play when it comes to performance. Every one knows that when Nvidia releases ARM based SOC in upcoming years... Its gonna destroy everything on the market. Like it always does. Also... This laptop does NOT have RTX 4090. Not even close...
if nvidia starts making the entire SoC, they might beat apple, but they are doing too well in just discrete gpus to try that
@@gytispranskunas4984 why do you hate nvidia pricing? They cost same as AMD but providing RT cores, cuda and they are more stable. Quality and R&D costs money too
PC users huffing copium in the comments section 😂
Python has contributed more to carbon emissions than any other programming language.
lol
So many tech-bros on the net bragging about their AI on 4090's using Python, AS IF using Python is something about which to brag (when it comes to performance or efficiency.)
@@TheDanEdwardsWhich programming language would you say is the best??
Actually, if this is true then you didn't pick the best machine for competition cuz there are bazillion non apple laptops, the mathematical consequence is that one of them has to beat the mac, so clickbaiting us with this title is awful
What kind of mathematics is that ?
@@Intel101-pe1et statistics bro, plus probability
to fine tune llama on m3 max, what size Llama work?how fast?can you release a video for this topic？
I want a MacBook that has Apple silicon soooooo badddd 😭😭😭😭😭
What’s your use case? The battery life on even the M1/M2 chips is phenomenal, the M3 chip mostly just adds performance. If you’re using it for light tasks, save some $$$ and get an M1 or M2 series chip
@@markclayton8977 I’m a photographer I use adobe PS adobe Lr and LRC plus Xcode for my camera app I’m working on and I need to connect to two displays
🥵
Doesn't matter if there's like zero software to use on silicon, it just is thar devs always do windows, only billionaire devs support mac, or browser game devs
Thanks for the analysis. I'm a Java developer. I want to get into AI for Python, C++, and Java. I'm looking for a new laptop. I've never owned a Mac. Because laptop GPUs are not swappable, that's really the driving factor. If you had to buy one or the other for AI software development, would you buy the M3 Max or an Intel RTX 4090m or something else?
Bro litterlay had a dozen macs!
Want to watch the stable diffusion one. Want to meet up? I'm in DMV
Wait what! Last week or two when I checked, Whisper still didn’t support Metal!
Been using whisper metal via python and whisper.cpp for months now
Alex, I found your channel when researching for my M3 max laptop purchase. I love your benchmark methodology, but also wish I could copy some of your workflows. If you added a code repository to your membership, I would join!
As much as I'd like you to join, there is no need to join to see my repos. This is a "better late than never" repo of my tests which I recently started: github.com/alexziskind1/machine_tests
First again from X to youtube.
Eveyday i get more impressed with the apple chips and unified memory 😊
2nd
too fast
hi lenovo loq i5 12450h 8gb 4060 80k vs ideapad ryzen 7 5800h 6gb 3060 71k purpose machine learning college purpose
You forget something, when you tried to make a benchmark you faced the same issue, you couldn't use the whole performance of the GPU/CPU when you used Windows or WSL, and you achieved that when moved to Linux. please do it and tell me the results.
I love your videos.
I have no idea why the hack I am watching this now, but everything you say sounds cool. :))
Ps: no idea how to code at all, wish I could.
Part of Apple's long game here is to absolutely dominate the mobile market in every way, and part of that domination is going to require robust machine learning capabilities and speed even for small models that are better suited for mobile uses of machine learning applications. They make their machines able to run small models insanely fast and that's where they're going to have a huge edge in the future
Have not had good luck running ai workloads on wsl or wsl2 with a discrete gpu. Everything says my gpu is being used incl docs but performance is pathetic.
Hello guys! I might sound weird but how can I look at the subscriptions?:D
My apologies if I am being dumb, by why wouldn't you use an NPU for this machine learning process, as I thought this is the sought of task NPUs were designed for, and maybe even better at than a GPU? And if you could, how would the performance compare when running on an Apple Silicon NPU (on paper M3 NPU is 18 TOPS for FP16)? And as every processor manufacturer is now getting on the AI bandwagon, you could even extend it to compare the performance of AMD 7000 series with AI NPU (10 TOPS, 8000 series NPUs 16 TOPS) or Intel's Meteor Lake core Ultra with NPU (10 TOPS)? Of course, the processor I would really like to see would be Qualcomm's Snapdragon X Elite with its 45 TOPS NPU, but that's yet to be released.
Could you pls make a video on stable diffusion ComfyUI on Mac, I don’t know why nobody ever made any videos about it
Hi, Alex How are you.. ?
yo!
with all of that machine, you should make GA xD as i need your m3 max moahahaha
Hide your kids, hide your wife
I would definitely use a desktop pc for this kind of stuff, not a laptop.
7:23 Vision Pro Light Seal Cushion spotted 👀
you got me. i still have mine
lol your side face is pretty
Anybody have a roadmap for me to learn on what about a language or framework performs better on one arch or another. How clever can tensor operations get? Python I get. But what is it b/w mlx, cpp and ggml, jax and mojo?
Walker Mountains
Red eyes! Check if this is normal
Hi alex can i get your mentorship session i m ready to pay for hardware setup for building llm
Hey Alex, I was wondering if you have a video planned for your EDC as a software engineer. I’ve been looking for a light case that I can carry around for my 16 MacBook with the 12.9 iPad. Trying to get ideas of what you utilize
But … 8 GB on MacOS is like 16 GB on Windows 🤔
Hi can you suggestion which laptop best for LLM + Deep learning I did want to any pc can you please help me
Insanely fast model is actually way faster in 4090
French "l" sounds like "l". If it were double "ll" it would've sounded like "y".
darn. should have asked my wife before vid.
But with windows laptops, you will spend only a few dollars on upgrading ram, but for apple you'll spend much more.
and you are stuck to a wall outlet
Well, you need to carefully specify the use case of the ram. In AI world, the only ram matters is the one on graphic card and it is not relatively cheaper to upgrade compare to mac
Have you tried timing all the machines with the model already loaded in the GPU's ram to test the raw compute power? It would also be a fairer comparison with cloud-hosted solutions. Anyways, wild that Apple hasn't sent anything to the only ML/AI reviewer on TH-cam. AI/ML is the core reason for me to update from M1/2 to M3 Max.
4090 should be faster as long as the model fits in the VRAM... if the model goes outside... it will be slower.
Whisper isn't AI.. no true AI yet exists lol
very interesting video ... but why do you have so many laptops lying around? :o
for testing
Use a simple RTX 4060 laptop without power plugged in.
@@einstien2409 it doesn't have enough VRAM.
My takeway is some fancy tech words to explore next week 😢
How would a Mac Studio M2 32GB stack up vs the MBP M3?
Is there anyway to run mlx inside xcode ios project?
Francesca Glen
Can you make a video on how to install llama using ml?
We want more content about MLX
Google has a better transcriber in their API Vertex called USM tbh
Then why is the TH-cam one still trash?
And apple dares to say 8GB are enough
not for ml. nobody said 8gb is enough for ml.
Microsoft said 640kB was enough.
Great video, Alex! You have some really enjoyable content on your channel.
Are you able to send me one of your old M-series Macs; I’m a student and I’m trying to learn some ML/AI stuff.
Finally MLX 🔥
Serious question: why would anyone buy a windows pc when you can buy a Mac that not only can run windows on it but runs windows BETTER than a windows pc??? In buying a computer soon and would appreciate the feedback. Thanks.
If power usage isn't your concern, then a PC can and will be faster. 4th Gen Core i9 + RTX 4090 will likely dominate in all benchmarks. For truly mobile performance (as in on battery, not plugged into a wall), Apple undeniably has the best product on the market right now. So long as you don't want to play any games on it.
for mobile platforms apple makes sense, for in house usage, it still lags behind by a lot, unless you are already deep into the apple ecosystem or simply prefer it, for pretty much every benchmark the only metric apple is going to win is in power usage which matters a lot in laptops, in desktop, not so much when while using more power, will get the job done far quicker.
Mac is terrible for gaming
Thanks.
What is the purpose of this computing power? Do you need it every moment of your day? And if you don't have it, is it a serious issue? I have a Mac Mini M2 at home. I also have 2 Windows PCs. I have no affection for these two machines that heat up, blow, scream, make a loud noise to obtain the power you're talking about. Not to mention the poor quality of plastics that crack and the miserable battery life of the laptop (whose power supply is larger and heavier than my Mac Mini M2). The production of PCs should be stopped.
Apple beats the competition. As usual 🥱 #PCMasterRace? More like #PCObselete 😂 /j
PCs are still better in multiple ways that Macs aren’t. Far from irrelevant
@@crestofhonor2349 you're right. I was just trolling 💚
Great to see more MLX content. Please do a comparison with Stable Diffusion MLX vs PC!
Can I have your cheapest mac air m1 please? 😍
4090m is FAR superior
Did you watch the video