Absolutely. The host PC's memory doesn't really matter, and neither does the CPU. The Vega card has 8GB of VRAM and that's what's important. That's enough for smaller models, but it's still limited.
@sylvainferrand351 all these free models from olama is lie ,they don't even match gpt 3.5 , and they show benchmark how it matches gpt4. I need atleast 14b model which can match gpt3. But when trying in real world in local machine they all give noob answers and hellucinate. So which is the best model?
@@arunachalpradesh399 Good question. You have to be wary of benchmarks; for exemple I was very disappointed by Phi3, which was promising but quickly became allucinating. Above all, you have to choose your model according to your needs. I'm interested in linguistic processing in French and English, for proofreading, translation and summarization (I've created the little site toutcorriger.fr for my own needs). Llama 3.1 8B was the best, Mistral 7B quite good, and Phi3 bad. I'm also interested in vision (image to text) for the visually impaired. minicpm-v 8B is top (openAI vision API is not better for my needs), Llava seemed limited by image resolution... in short, to be tested according to your needs ! And then there's fine tuning question...
Nice, thank you!
Isn't llm most depend on v ram ?
Absolutely. The host PC's memory doesn't really matter, and neither does the CPU. The Vega card has 8GB of VRAM and that's what's important. That's enough for smaller models, but it's still limited.
@sylvainferrand351 all these free models from olama is lie ,they don't even match gpt 3.5 , and they show benchmark how it matches gpt4. I need atleast 14b model which can match gpt3. But when trying in real world in local machine they all give noob answers and hellucinate. So which is the best model?
@@arunachalpradesh399 Good question. You have to be wary of benchmarks; for exemple I was very disappointed by Phi3, which was promising but quickly became allucinating. Above all, you have to choose your model according to your needs. I'm interested in linguistic processing in French and English, for proofreading, translation and summarization (I've created the little site toutcorriger.fr for my own needs). Llama 3.1 8B was the best, Mistral 7B quite good, and Phi3 bad. I'm also interested in vision (image to text) for the visually impaired. minicpm-v 8B is top (openAI vision API is not better for my needs), Llava seemed limited by image resolution... in short, to be tested according to your needs ! And then there's fine tuning question...