Dynamic Quantization with Unsloth: Shrinking a 20GB Model to 5GB Without Accuracy Loss!

[@suryadivi3905]

Congratulations brother, hoping to see you in the first place.

[@A_Me_Amy]

great examination of this, I was wanting to see how this worked. So llama is not only the real open ai, but also are seeminly actively trying to make it easy for people to use and modify it. I should probably look in to llama more.

[@testales]

I hope they can and will implement this in Ollama ASAP. :-)

[@PromptEngineer48]

Hmm

[@chronicallychill9979]

It's easy to import any of these models after shrinking them though at least, definitely something you can script without much hassle.

[@testales]

@@chronicallychill9979 So at the end these are regular gguf models that ollama can load?