The important distinction here would be that this guide shows you how to use LocalAI with Docker. Similarly, you can use Ollama with Docker. Docker is just meant to give you an easy one-command way to start an environment consistently. So I'm going to take the liberty to answer the difference between Ollama and LocalAI, because those are actually the systems you are interested in comparing. For large language models, Ollama is actually just as good if not even better supported by the community. You can use Ollama just fine in Docker as well: hub.docker.com/r/ollama/ollama However, LocalAI supports more types of AI models than just language models in one package. I aim to show more of those possibilities later on hence I focus on LocalAI. If you just want to use language models, definitely try out Ollama!
I always start my prompting with the question “Was 2024 a leap year”. On simple local machines it’s telling you fairy tales. That’s disappointing. But thanks for your good explanation 👏
It's like 0.0014 per prompt just get it lol things unbelievable I was stuck for 3 days with clide and chatgpt and cascade. All of them. Could t handle the size of my algo any more. Thjs thing did what I was trying to figure out in three days. Did it in 3 prompts
Deepseek r1, definitely the full model you can access via web/api, is really difficult to host on your own hardware. You'll need to start with a smaller model like Phi-3.5/4 or a smaller Llama model.
phi...ok but not that powerful, i thought there might be something i don't know in this video...i would rather use openwebui or anythingllm with ollama if you just need a chat interface
That's right, I will show in later content that LocalAI allows you to use other models not just LLMs. If you just want a language model, you can use ollama
I don't ever usually comment, though the video is very informative, want to support you, keep going :)
Will try it surely!
Hey, thanks for the video.
Hello, im confused whether I can run any model on my Ryzen 3 3200G PC, with 16gb of ram or not?
Did you try it out for yourself with the repository in the description? I expect Phi 3.5 to work "okay" on that machine, but not very fast.
What's the difference between using docker and ollama? If you could give a detailed explanation, I'd like that
The important distinction here would be that this guide shows you how to use LocalAI with Docker. Similarly, you can use Ollama with Docker. Docker is just meant to give you an easy one-command way to start an environment consistently.
So I'm going to take the liberty to answer the difference between Ollama and LocalAI, because those are actually the systems you are interested in comparing. For large language models, Ollama is actually just as good if not even better supported by the community. You can use Ollama just fine in Docker as well: hub.docker.com/r/ollama/ollama
However, LocalAI supports more types of AI models than just language models in one package. I aim to show more of those possibilities later on hence I focus on LocalAI. If you just want to use language models, definitely try out Ollama!
@zenvanriel oh, LocalAI. I understand now. Thanks
How does this model perform on your device? Let me know!
nice name zen
We share the same first name based on your handle? If so, love it!
@ yes we do 🤝🏼
I always start my prompting with the question “Was 2024 a leap year”. On simple local machines it’s telling you fairy tales. That’s disappointing. But thanks for your good explanation 👏
Yeah, for local devices a RAG implementation makes more sense. I'll explain that later on.
How can i contact u zen
In a few weeks I'll make sure we can all get in touch. Stay tuned.
Awesome content, could you make a video where you customize a reasoning model It further like connecting It to a folder of PDF files as database
This is indeed all in the backlog of the next videos! Stay tuned.
How can you make an interface or make it receive input files?
There is a opensource webuis
This is upcoming in a next video!
The voice is not synchronised with your video
Oops, looks like it is slightly off indeed. Will make it better next time.
Kindly make a video how to setup and make own deepseek r1 api
It's like 0.0014 per prompt just get it lol things unbelievable I was stuck for 3 days with clide and chatgpt and cascade. All of them. Could t handle the size of my algo any more. Thjs thing did what I was trying to figure out in three days. Did it in 3 prompts
Deepseek r1, definitely the full model you can access via web/api, is really difficult to host on your own hardware. You'll need to start with a smaller model like Phi-3.5/4 or a smaller Llama model.
I bought it and it's the best model hands down. Nothing compares @@zenvanriel
@@zenvanriel I have installed deepseek r1 7b using lm studio, but how to create api to access in side my program
LM studio has a server mode in it, check the main menu icons. It is the green one that looks like a CLI
phi...ok but not that powerful, i thought there might be something i don't know in this video...i would rather use openwebui or anythingllm with ollama if you just need a chat interface
That's right, I will show in later content that LocalAI allows you to use other models not just LLMs. If you just want a language model, you can use ollama