The model's size is loaded into your ram which is the memory shared with your GPU. Without enough space on your GPU you wont be able to run it. On your 24GB machine, I believe the 43GB model gets run from the CPU alone because the memory can't all fit on the GPU so it uses the memory as swap for the CPU to process it.
the terminal window was too big to see the commands and output..please make it smaller next time...:)
😂
title is a clickbait... you are NOT running 3.3. on macbook pro with 24gb ram:(
🤔 Why is that?
When I understand it right than 3.3 needs more then 40GB of VRam. So the test makes no sense at all.
I didn’t know that 3.3 needs 40gb of ram before I tested it. I thought I have a powerful machine that can handle it 😂
@@faroucky You are running it on M4 Pro chip and I thought it would run without any problems.
The model's size is loaded into your ram which is the memory shared with your GPU. Without enough space on your GPU you wont be able to run it. On your 24GB machine, I believe the 43GB model gets run from the CPU alone because the memory can't all fit on the GPU so it uses the memory as swap for the CPU to process it.
1 min for a hello respons and then nothing. Haha. I think i will keep running my 70B models on my 5 years old 2xP6000 GPU setup. Maybe M5 Ultra 2027.
Perhaps 🤔
Good 1:05
Useless video
Why?