1 bit LLMs

2 weeks back Microsoft released the implementation of their 1 bit LLM transformer.
This can potentially change the world, by enabling LLM inference on CPU and mobile, without any internet connection.
In this hands-on video, you learn to run 1 bit LLM inference on your own laptop!

[@eddiendirangu]

THANK YOU!! I've watched a number of videos on 1-Bit LLMs and BitNet, none actually beats the simplicity and yet richness of this video.

[@ugk4321]

Complex subjects simplified. I love the sessions

[@prasadagalave9762]

Thanks sir ❤ Your are revolutionizing this field in india

[@tripchowdhry6124]

wow again superb explanation - making complex things so easy to understand

[@tripchowdhry6124]

Yes - a Detailed Video theory from scratch much appreciated it

[@trendfollowers470]

Thank you for making this video very interesting

[@dineshgct]

Great video as usual !

[@Omunamantech]

Loved This Video :D

[@akshayparkhi5468]

Hi Rajat, Thank you for your video. I was able to run the 8 Billion model on my laptop. It is really fascinating to see an 8 Billion model running on a laptop. Can you provide more information on how to train these 1-Bit LLMs? Does it still require a high-end GPU, or can it be trained on a CPU? I am looking to train this 1-Bit LLM with instruction-based fine-tuning

[@battaaditya4134]

Can you explain how to download in windows

[@iamrvk98]

what is the difference we are also able to run the models like lamma 7B locally and it does work better than 1 bit

[@VIVEKKUMAR-kv8si]

Really, that's surprising. I think you need atleast 16GB GPU to run llama 7B locally. Most customer GPUs are 6/8/12GB.

[@tripchowdhry6124]

We use Mac

[@chidambarjoshi3470]

Please explain mathematically