Great explanation, good job with including the potential problems that somebody could encounter (getting CUDA, cloning repo, event installing git). Audio is great and the same for presentation ;) Would you mind showing how to use these models effectively? Like how to phrase prompts to get most of them, how to provide context from multiple files if that is possible (like for writing new tests in existing test class for bug fix, or the other way around - write codefix after new unit tests was added)
Thank you so much! Yes in the future I will make a more detailed video on making use of various models (not with tabby specifically but in general). I am still experimenting with different models for querying files etc.
I couldn't add new models by cloning, it would cause a crash with the gguf file. To resolve this don't clone the repository and just specify the model you want. It will download it for you.
Great explanation, good job with including the potential problems that somebody could encounter (getting CUDA, cloning repo, event installing git). Audio is great and the same for presentation ;) Would you mind showing how to use these models effectively? Like how to phrase prompts to get most of them, how to provide context from multiple files if that is possible (like for writing new tests in existing test class for bug fix, or the other way around - write codefix after new unit tests was added)
Thank you so much! Yes in the future I will make a more detailed video on making use of various models (not with tabby specifically but in general). I am still experimenting with different models for querying files etc.
The way you explained that was nice.
I couldn't add new models by cloning, it would cause a crash with the gguf file.
To resolve this don't clone the repository and just specify the model you want. It will download it for you.
Thanks for the video. I'm using ollama + continue extension which uses ollama api endpoint directly. Is there any reason to use tabbyml instead?
Honestly it comes down to preference. I just like how tabby has its own extension and is more streamlined!
Tabby faster on same models. No idea why, maybe because caching
Is it possible to run this stuff with CPU (eg i5 8500) not Cuda?