Thanks to @munchcup for sharing a great embedding model that is available straight from the ollama library ollama.com/library/nomic-embed-text 🔥 Also all of the code from this video is provided on my website decoder.sh/videos/rag-from-the-ground-up-with-python-and-ollama 👌
As a nonprofessional programmer, this was the introductory best video on RAG I have seen anywhere, and that's saying a lot with how many I've watched. Thank you! I look forward to many more videos continuing the series!
@parttimelarry Your content was some of the first I absorbed as I entered the space. Spectacular and inspiring content. Looking forward to more if/when available, great work.
FANTATSTIC PRESENTATION! Thank you for being a good teacher. I stumbled on your channel after seeing the oppposite of what you did. I mean it was really bad. Thank You!
I made one video on LangChain so far! I've been on a longer hiatus than I wanted to this year, but I'll continue with Langchain when I'm back :) th-cam.com/video/qYJSNCPmDIk/w-d-xo.html
I liked how you used cosign similarity to show the actual chunks of matched text. Since this usually happens behind the curtains it was nice to have that extra insight. I like your style and just subscribed.
Thx for sharing this amazing tutorial showing us what RAG is as well as “how-to” in such a way of details. Keep your great work and I‘ve clicked “subscribe” button right away 👍🏻
best video about rag so far in the web! congrats! your code runs! I think it needed an intro about ollama and how to run that. but besides it, this video is fantastic!
Not a bad idea! I do have a whole playlist on Ollama, so can choose where you want to jump in :) th-cam.com/play/PL4041kTesIWby5zznE5UySIsGPrGuEqdB.html
@@decoder-sh I wanted to suggest to put a video on how to train a model on specific documents. We can see that this approach have some limitations such as the number of sentences passed as a context to the model and depending on the number, the output is different. For example, if you choose the last five sentences VS the last twenty, the output is a little different. Anyways I have to say that this is the first that combines all (embeddings, vectoring, LLM, documents and pure code) together. Congrats again! Look forward for your next content.
@@decoder-sh hello, I run some tests and looks like these models are already trained with peter pan story. I tried to ask with and withou using the script, and the asnwers were very similar. Just wanted to give this feedback. thanks
Fantastic video. Yes, I could explain embeddings. I already had a basic concept, but your simple file-based cache of the embeddings really highlighted the basic functionality without needing a vector database.
Nicely done. I'm building a chatbot with a memory system based on RAG. Prompts, responses, topics, entities, and tool results are stored in json documents. Embeddings are stored in chromadb. I think I'll dump chroma and save embeddings the way you do. That removes a lot of complication. And I like your similarity search. I can dump Langchain too. I hope this will become a sophisticated memory system for chat history, etc. I'll delete or archive old memories that haven't been accessed in a week or a month, etc. This can be a hyper parameter that I can tune. I've already written a router program that decides what to retrieve and what tools to use, so no Langgraph either. Anyway, thanks for this. Publishing this helps everyone. Please do more like this.
Thanks for your clear and effective explanation. I'm grateful also for how you steered clear from frameworks that abstract the RAG implementation details away from you. In fact, I'd appreciate if you could dive deeper into things like chunking strategies and agents.
Thanks for watching! I do plan on covering these topics in future videos. I'm currently writing a series of videos on more advanced RAG topics using either llamaindex or langchain. I might do one that looks at "manual" pdf parsing versus builtin document loaders if people are interested
@@decoder-shI am very new to ai and coding. How would you suggest I truly understand what it is these tools are doing. I have a great business plan but my mind thinks as a workflow not like a programmer. Great content!
@@laritaharrington1117 Here's what I do: first, I code along with the video. Typically, I make mistakes and have to fix them; this already helps me understand better. Then, I come up with my own little project and try and implement that. This usually takes far longer than I had originally planned for, but it does mean that I learn about the limitations and pitfalls. If you like, we can do a little project together; learning this way is probably even more effective.
I love these videos, very helpful. I am a junior dev trying to understand some of these concepts and I feel like these videos have helped me immensely!
In the embedding part there is a very fast model specific for embedding in ollama named nomic embed text which simplifies the process.Just a point to note.
Awesome lesson, precisely what I had been looking for. Please look at fine-tuning this existing model with many documents. I looked everywhere and couldn't locate one without utilising an API.
I really like it. Your videos are very clear. The basics are important! I use the same approach with scanned documents, converted from PDF to text and stored in a database. (filename, link, text, and tokenization). With Streamlit I do a search. What I would like to do is an exportable model (e.g., the plant guide for healing and being able to use it). What would be the best approach? Thank you for your answer.
Little confused about how there was similarity with "who is the story's primary villain". How can we know it's RAG that is providing this answer and not the inference model? Would also be nice to see what was the similar chunks and then convert them back to string to understand what the model got as input before it responded with the answer about "Hook". I think that's the only major thing missing from your tutorial. Thanks again for the good content.
Thank you for this feedback! This is a great point, I should've kept logging the chunks that are being passed to context, I'll keep that in mind for the next video. I encourage you to try this on your own system at home, but Mistral is good enough at instruction following that it adhered to our system prompt and only used the context it was provided. I'll try to explicitly show a failure case to demonstrate that our model is behaving as expected in future videos. Thanks for watching :)
@@Djeez2Yep this is a great idea. You could even set up unit tests for different models that either include or don't include the answer to a given question. The only challenge there is getting the model to say exactly "I can't answer that with the given context" or something that plays nice with simple tests.
Hi there, could you explain a bit more about what you want? I love running local models so I'd like to help figure this out. If you're asking to use OpenAI's ChatGPT: You can access it through openAI's API, but that wouldn't be a "local" model since you're sending all of your requests to OpenAI.
Does anyone know how to increase the size of the paragraphs so that the responses are more useful? That part was skipped over in the video, if I recall correctly.
Thanks a lot for the detailed explanation in the video! I have a question regarding Ollama: Is it possible to use Ollama and the models available on it in a production environment? I would love to hear your thoughts or any experiences you might have with it.
Fantastic video! The RAG example given is for a single document, but a repository usually contains dozens of documents. Is there a way to bulk-upload the documents at once to the LLM (for chunking and embedding)? Thanks.
Fantastic video! I've been trying to come up with a streamlit chat with your document but also have historical context of the chat. But, I'm beginning to wonder if that's not as useful as just sequencing it out considering each time will require context retrieval. All good
This is the best, easy to understand intro of RAG. Do I need to install ollama on my Windows machine before running your code? And replace 'nomic-embed-text' with the model when running your code. Could you please do a video llama 2?I downloaded llama 2 from Meta, but don't know how to use it, such as how to open it or query it. Your code-driven approach best explains everything.
Thanks for watching! You will need to install Ollama. Furthermore, you can actually use the llama-2 model directly from ollama! ollama.com/library/llama2
9 หลายเดือนก่อน
Great video. Looking forward to see a 2nd part with Lola an index and what do you recommend for working with pdf files!
The chat method takes a list of messages, so you can just add your previous interactions to that list to give your LLM context. Here's an example th-cam.com/video/ZHZKPmzlBUY/w-d-xo.html
Great explanation! How this implementation performs compared to langchain? I heard alot of times that langchain is not suitable for production. Do you think that the responses are faster here due to less abstractions compared to langchain. Thanks once again.
If I was deploying a RAG app in a production environment, langchain gives you a lot of useful debugging, tracing and serving capabilities that a raw implementation like mine would not have. And if speed was your main concern, I would probably start by replacing ollama with groq, or at least vllm if you’re determined not to use a paid api. Thanks for your question! I have part 1 of a langchain series coming out in the next couple days
Thank you so much for your answer. Another question i have with rag applications is how we can handle contextual questions in a chat.for example lets say i have txt about a project with description, implementation details and applications. If i ask what is "projectname", the embedding model will feed to the llm the description paragraph.but if my next question is "and how this is implemented?" The embedding model will not know that i am refering to the "projectname" so the llm will either not answer or hallucinate. If i feed it with more similar paragraphs so that it can answer based on chat history then i will start facing context-window limitations. Is there a solution for this?
Great video. Is there somewhere to download the code you used? Currently, I am stopping the presentation and typing from your screen but there must be a better way. Thanks this is the best RAG description I have found.
Excellent video, explaining from the ground up how RAG works without immediately starting with LangChain. I have played with some other RAG implementations using Ollama and have intentionally asked questions outside of the purview of the RAG content and they do come back with the correct "not found in the documentation" response. But what still remains unclear to me is how an LLM is kept restricted to only accessing RAG content. How is that fence guaranteed? Does setting the temperature affect that fence?
It's difficult to make guarantees with LLMs, but your best bet is to use a model whose training data includes instruction following. Mistral is good at this by default, but also has an instruct fine-tuned version available on ollama!
@@decoder-sh I'm thinking something along the lines of the RAG interacting with databases as well as individual documents. In my case, it's PostgreSQL. The issue really is the number of options available such as Pinecone, pgvector, ChromaDB, and the like. Maybe my search has not been as comprehensive as it needs to be, but having a proper comparison of these options would help with making the optimal selection. My use case involves analyzing an ERP database of tens of thousands of inventory items to identify issues such as potential duplication, errors and typos, the potential to standardize items - meaning have one inventory item for multiple applications instead of different items per application (this involves looking at item specifications), and the like. There will likely be attached product specification sheets and other literature available - this video you shared has demonstrated how I can manage these attachments accordingly. Thank you for taking the time to respond to my comment.
is there a way which can be used if I want to let Lama answer from its own knowledge base when my own data is not relevant or is not accurate to be an answer to the user's prompt?
hello pro its very good tutorial i have some file contain some article its good to with small articles from 2 - 3 line but if lines is more than 20 line its give response from no were can i increase chunk size or any solution to solve this ? thank you in advance
hello. Two separate questions. a) Your code works (thanks) and I'm using it for a task at my workplace (basically help in extracting info from a repository of dozens of .docx). However it seems that the model only know about the input file (the embeddings) and "forgot" everything else. That is, I can query the txt and get mostly correct answers, but if my answer need to be complemented by additional info (that I know is available if I query mistral on ollama normally) it does not have a clue. Or am i doing something wrong? the "augmented" in RAG seems to indicate that we add (maybe with higher priority) the knowledge in the input docs to the existing one, but it seems not to be the case...
Hi there, I'm glad to hear the code works! It sounds like you need to change the system prompt, since we're explicitly telling it to only use the provided context and no knowledge that it already has. Let me know if that works!
@@decoder-sh thank you for your answer. I changed the systemp prompt as follows, but no change... : "SYSTEM_PROMPT = """You are a helpful reading assistant who answers questions based on snippets of text provided in context and also with general knowledge. Be as concise as possible. If you're unsure, just say that you don't know. Reply in the same language used by the user question Context: """ (PS i added the language thing as it is digesting documents in french but I want answers in EN.)
Yes I think RAG is definitely the best way to empower your LLM to give up-to-date answers about dynamic data like documentation. You'll just need to update your embeddings when the data changes.
Thanks for this great video. Just a newbie question from me: how to formulate RAG setup to make Ollama understand a specific text format? I provided mixtral a list of timed activites for a person like this: Day1 7-00-00 : breakfast Day1 7-00-08 : workout Day1 7-00-16 : reading ...... Day20 7-00-00 : breakfast Day20 7-00-08 : reading Day20 7-00-16 : workout I asked LLM to print out all instances of activities at 7-00-00 across all these 20 days, which is super easy for human, but the results from LLM were always wrong ... Can you give me some instructions? Is RAG suitable for processing such text data?
With data that is as cleanly structured as this, why couldn't you just write some simple code to filter the results for you? Like `data.split(' ').map(line => line.split(' ')).filter(([day, time, topic]) => time === '7-00-00')` ? Or you could put your data into a database, give the schema to an LLM and have it write queries for you - this is called self querying python.langchain.com/docs/modules/data_connection/retrievers/self_query/
Very interesting thanks. Do you know how this compares to Haystack 2.0? I understand that Haystack being a framework offers more scalability while this is more of a DIY script?
This is my first time hearing about Haystack, but this is definitely just a simple DIY script to build an understanding of what's happening under the hood of more advanced libraries like Haystack, Langchain, etc
many thanks ! that's awesome what you share with us. Would be great to see more how to use bge-base 🙂 or not, actually I watched your YT video to change gguf file to ollama model and replaced it in: prompt_embedding = ollama.embeddings(model="bge-base-en-16", prompt=prompt)["embedding"], seems working, but I need to test it more 🙂 in this case maybe you could create a tutorial how to build from scratch a speech to speech multimodal assistant operating eg on Ubuntu ?
@@decoder-sh It's possible. I am getting an error on `needle_norm = norm(needle)` TypeError: unsupported operand type(s) for *: 'dict' and 'dict' May I know what numpy version you are using?
@@decoder-sh Thank you for your reply. I am not a developer, yet learning/absorbing. Github key points (for my workflow): - presents a reliable single source of truth - centralized repository for rapidly growing/scaling projects - serves as a hub to linked resources (e.g. websites, youtube, code, updates, changes, new projects, etc) - project versioning - lowers barrier to entry, while simplifying versioned resource aggregation for everyone, at every level.
Very interesting question! It's definitely possible to create image embeddings, and in fact that's how image search works on Google. However I don't think you can create image embeddings directly from Ollama. It looks like Llava embeds images using the CLIP model internally. github.com/haotian-liu/LLaVA/blob/main/llava/model/multimodal_encoder/clip_encoder.py huggingface.co/docs/transformers/v4.21.0/en/model_doc/clip
Yes you will need to install the Ollama python library in every new python environment you create, no you will not need to install the ollama application (which serves the models and API that the python library talks to) for each new python environment
It will depend based on your hardware, embedding parameters, and embedding model. Ideally it would take a minute or two. Using mistral, I think it took 5-10 minutes on a macbook M1, which is why I recommend against using that as your embedding model.
Hi, I’m using an M1 MacBook Pro, but ollama itself has minimal requirements (you don’t even need a gpu!). It really comes down to the model you’re trying to run.
@@decoder-sh Try texts with examples of Aristotelian syllogisms(from which propositional logic originated), to see if it holds the thread of reasoning. 😁 Are short, but hard to follow. Or logical document in general, that you can see if it does good or bad.
Thanks to @munchcup for sharing a great embedding model that is available straight from the ollama library ollama.com/library/nomic-embed-text 🔥
Also all of the code from this video is provided on my website decoder.sh/videos/rag-from-the-ground-up-with-python-and-ollama 👌
🙏 Am Humbled
As a nonprofessional programmer, this was the introductory best video on RAG I have seen anywhere, and that's saying a lot with how many I've watched. Thank you! I look forward to many more videos continuing the series!
Thank you for sharing, I'm happy to hear it!
I second that.
This look solid. Happy it's not all LangChain-specific like many tutorials out there. Saving for later.
I'll cover langchain soon enough, but I wanted to start with a from-scratch implementation to teach the basics
@parttimelarry Your content was some of the first I absorbed as I entered the space. Spectacular and inspiring content. Looking forward to more if/when available, great work.
@parttimelarry Also I just subscribed to your account, congrats on 100k! I'd like to eventually explore the intersection of finance and LLMs :)
‘This is a great video! Im also looking forward to your langchain video. I’ve been struggling with that.
This is a great video. You teach like a Professor. You're an expert and well talented! Your organization will indeed love working with you.
I really think TH-cam should be recommending this channel. The content quality is very high.
Thanks for saying so, welcome to my channel!
Thanks for you series of videos. They are to the point and very comprehensive. Thanks again!
FANTATSTIC PRESENTATION! Thank you for being a good teacher. I stumbled on your channel after seeing the oppposite of what you did. I mean it was really bad. Thank You!
Haha well I'm sorry you had a bad experience before, but am glad you found your way here!
This is the true tutorial explain very well only important information that need to use to implement RAG. Thanks.
I liked your overall presentation style. Objective and minimalist without being simplistic.
Thank you for watching!
Wow! Great video! You explained everything very clearly. I would love to be able to see how to do this using Langchain, etc... Thank you!
I made one video on LangChain so far! I've been on a longer hiatus than I wanted to this year, but I'll continue with Langchain when I'm back :)
th-cam.com/video/qYJSNCPmDIk/w-d-xo.html
I liked how you used cosign similarity to show the actual chunks of matched text. Since this usually happens behind the curtains it was nice to have that extra insight. I like your style and just subscribed.
Thx for sharing this amazing tutorial showing us what RAG is as well as “how-to” in such a way of details. Keep your great work and I‘ve clicked “subscribe” button right away 👍🏻
Thank you for subscribing, I look forward to making more videos for you!
best video about rag so far in the web! congrats! your code runs! I think it needed an intro about ollama and how to run that. but besides it, this video is fantastic!
Not a bad idea! I do have a whole playlist on Ollama, so can choose where you want to jump in :) th-cam.com/play/PL4041kTesIWby5zznE5UySIsGPrGuEqdB.html
@@decoder-sh I wanted to suggest to put a video on how to train a model on specific documents. We can see that this approach have some limitations such as the number of sentences passed as a context to the model and depending on the number, the output is different. For example, if you choose the last five sentences VS the last twenty, the output is a little different.
Anyways I have to say that this is the first that combines all (embeddings, vectoring, LLM, documents and pure code) together. Congrats again! Look forward for your next content.
@@gustavow5746 Great idea! Model fine-tuning is definitely on my list :)
@@decoder-sh hello, I run some tests and looks like these models are already trained with peter pan story. I tried to ask with and withou using the script, and the asnwers were very similar. Just wanted to give this feedback. thanks
@@gustavow5746 Ah that's a great test to run, thank you for the information! I will take that into consideration for future videos.
Excellent tutorial, thank you so much, your code example ran perfectly and the results were quite decent.
I'm glad to hear it! Which embedding model did you end up using?
Fantastic video. Yes, I could explain embeddings. I already had a basic concept, but your simple file-based cache of the embeddings really highlighted the basic functionality without needing a vector database.
Great tutorial David, clear and easy to follow. You just got a new subscriber 🦾
Nicely done. I'm building a chatbot with a memory system based on RAG. Prompts, responses, topics, entities, and tool results are stored in json documents. Embeddings are stored in chromadb. I think I'll dump chroma and save embeddings the way you do. That removes a lot of complication. And I like your similarity search. I can dump Langchain too. I hope this will become a sophisticated memory system for chat history, etc. I'll delete or archive old memories that haven't been accessed in a week or a month, etc. This can be a hyper parameter that I can tune. I've already written a router program that decides what to retrieve and what tools to use, so no Langgraph either. Anyway, thanks for this. Publishing this helps everyone. Please do more like this.
Thanks for your clear and effective explanation. I'm grateful also for how you steered clear from frameworks that abstract the RAG implementation details away from you. In fact, I'd appreciate if you could dive deeper into things like chunking strategies and agents.
Thanks for watching! I do plan on covering these topics in future videos. I'm currently writing a series of videos on more advanced RAG topics using either llamaindex or langchain. I might do one that looks at "manual" pdf parsing versus builtin document loaders if people are interested
@@decoder-sh Thanks! Much appreciated, looking forward to your future videos.
@@decoder-shI am very new to ai and coding. How would you suggest I truly understand what it is these tools are doing. I have a great business plan but my mind thinks as a workflow not like a programmer.
Great content!
@@laritaharrington1117 Here's what I do: first, I code along with the video. Typically, I make mistakes and have to fix them; this already helps me understand better.
Then, I come up with my own little project and try and implement that. This usually takes far longer than I had originally planned for, but it does mean that I learn about the limitations and pitfalls.
If you like, we can do a little project together; learning this way is probably even more effective.
I came here upon the recommendation of my good friend and this video is so educational. Loved it! ❤
Your friend clearly has good taste, thanks for watching!
Thanks!
I love these videos, very helpful. I am a junior dev trying to understand some of these concepts and I feel like these videos have helped me immensely!
I'm glad to hear it, keep learning!
nice, small sized demo, demonstrating the principles. good job, I learnt a lot 🙂
Glad you liked it!
In the embedding part there is a very fast model specific for embedding in ollama named nomic embed text which simplifies the process.Just a point to note.
Awesome suggestion, thank you! Wow and they're MRL embeddings? I've been meaning to do a video on these ollama.com/library/nomic-embed-text
@@decoder-shThank you for your teachings. You've opened me to endless possibilities in using ollama as a self taught dev.
Awesome lesson, precisely what I had been looking for. Please look at fine-tuning this existing model with many documents. I looked everywhere and couldn't locate one without utilising an API.
Thanks for watching! I plan to cover fine tuning soon, there are a lot of interesting techniques to address
@@decoder-sh Great ❤
Really great! Thank you. Look forward to the next steps here with langchain!
Great videos, I just started in this space and started following your videos.
Welcome! I peeked at your Hands on with Machine Learning video - that's a great textbook :)
First of all Thank you for making this vedio!!, can you also make vedio on how to handle when mulitple text files are there. Thank you.
Well prepared and executed, super useful, thanks for your work.
Thank you very much! Looking forward to making more
This is fantastic. It fits perfectly with a project I am working on.
That’s great! May I ask what you’re building?
Brilliant video. 10/10. Will be recommending to everyone I know.
Thanks very much!
Man, I need to study more. Thanks for putting this video out!
My pleasure, thanks for watching!
I'm looking forward the next video about RAG using langchain ! :)
Great stuff per usual. Looking forward to the LangChain video
Thank you for sharing knowledge. Looking for more such videos
I really like it. Your videos are very clear. The basics are important! I use the same approach with scanned documents, converted from PDF to text and stored in a database. (filename, link, text, and tokenization). With Streamlit I do a search. What I would like to do is an exportable model (e.g., the plant guide for healing and being able to use it). What would be the best approach? Thank you for your answer.
Little confused about how there was similarity with "who is the story's primary villain". How can we know it's RAG that is providing this answer and not the inference model?
Would also be nice to see what was the similar chunks and then convert them back to string to understand what the model got as input before it responded with the answer about "Hook".
I think that's the only major thing missing from your tutorial.
Thanks again for the good content.
Thank you for this feedback! This is a great point, I should've kept logging the chunks that are being passed to context, I'll keep that in mind for the next video.
I encourage you to try this on your own system at home, but Mistral is good enough at instruction following that it adhered to our system prompt and only used the context it was provided. I'll try to explicitly show a failure case to demonstrate that our model is behaving as expected in future videos.
Thanks for watching :)
You could test that by replacing "Hook" in the returned chunks by some other name and then see what the LLM returns as an answer.
@@Djeez2Yep this is a great idea. You could even set up unit tests for different models that either include or don't include the answer to a given question. The only challenge there is getting the model to say exactly "I can't answer that with the given context" or something that plays nice with simple tests.
wow, excellent vid. Thanks!
Top notch video ...
Thanks a lot ...
I would love to know if we can use GPT locally as a way of having other options for ollama.
Hi there, could you explain a bit more about what you want? I love running local models so I'd like to help figure this out. If you're asking to use OpenAI's ChatGPT: You can access it through openAI's API, but that wouldn't be a "local" model since you're sending all of your requests to OpenAI.
Does anyone know how to increase the size of the paragraphs so that the responses are more useful? That part was skipped over in the video, if I recall correctly.
Amazing, thanks mate ❤
Thanks a lot for the detailed explanation in the video! I have a question regarding Ollama: Is it possible to use Ollama and the models available on it in a production environment? I would love to hear your thoughts or any experiences you might have with it.
Fantastic video! The RAG example given is for a single document, but a repository usually contains dozens of documents. Is there a way to bulk-upload the documents at once to the LLM (for chunking and embedding)? Thanks.
Awesome video. Very well explained. Congratulations
Thank you CJ!
Would love to see you do a version of this using nomic , llama3 and chroma DB for your vector stores
Fantastic video! I've been trying to come up with a streamlit chat with your document but also have historical context of the chat. But, I'm beginning to wonder if that's not as useful as just sequencing it out considering each time will require context retrieval. All good
This is the best, easy to understand intro of RAG. Do I need to install ollama on my Windows machine before running your code? And replace 'nomic-embed-text' with the model when running your code. Could you please do a video llama 2?I downloaded llama 2 from Meta, but don't know how to use it, such as how to open it or query it. Your code-driven approach best explains everything.
Thanks for watching! You will need to install Ollama. Furthermore, you can actually use the llama-2 model directly from ollama! ollama.com/library/llama2
Great video. Looking forward to see a 2nd part with Lola an index and what do you recommend for working with pdf files!
Definitely! I could do a whole video just on working with PDFs, it can be a pain 😰
great videos! i want to ask how do we set up it so it can answer multiple follow up questions while retaining same context?
The chat method takes a list of messages, so you can just add your previous interactions to that list to give your LLM context. Here's an example th-cam.com/video/ZHZKPmzlBUY/w-d-xo.html
Great explanation! How this implementation performs compared to langchain? I heard alot of times that langchain is not suitable for production. Do you think that the responses are faster here due to less abstractions compared to langchain. Thanks once again.
If I was deploying a RAG app in a production environment, langchain gives you a lot of useful debugging, tracing and serving capabilities that a raw implementation like mine would not have. And if speed was your main concern, I would probably start by replacing ollama with groq, or at least vllm if you’re determined not to use a paid api. Thanks for your question! I have part 1 of a langchain series coming out in the next couple days
Thank you so much for your answer. Another question i have with rag applications is how we can handle contextual questions in a chat.for example lets say i have txt about a project with description, implementation details and applications. If i ask what is "projectname", the embedding model will feed to the llm the description paragraph.but if my next question is "and how this is implemented?" The embedding model will not know that i am refering to the "projectname" so the llm will either not answer or hallucinate. If i feed it with more similar paragraphs so that it can answer based on chat history then i will start facing context-window limitations. Is there a solution for this?
Great video. Is there somewhere to download the code you used? Currently, I am stopping the presentation and typing from your screen but there must be a better way. Thanks this is the best RAG description I have found.
Hey Bill, thanks for watching! You can find all the code from my videos here
decoder.sh/videos/rag-from-the-ground-up-with-python-and-ollama
Excellent video, explaining from the ground up how RAG works without immediately starting with LangChain. I have played with some other RAG implementations using Ollama and have intentionally asked questions outside of the purview of the RAG content and they do come back with the correct "not found in the documentation" response. But what still remains unclear to me is how an LLM is kept restricted to only accessing RAG content. How is that fence guaranteed? Does setting the temperature affect that fence?
It's difficult to make guarantees with LLMs, but your best bet is to use a model whose training data includes instruction following. Mistral is good at this by default, but also has an instruct fine-tuned version available on ollama!
This was well done. Thank you. Will you, by chance, be working on any videos to make RAG web apps?
I am more interested in the backend than the frontend of RAG apps, but what did you have in mind?
@@decoder-sh I'm thinking something along the lines of the RAG interacting with databases as well as individual documents. In my case, it's PostgreSQL. The issue really is the number of options available such as Pinecone, pgvector, ChromaDB, and the like. Maybe my search has not been as comprehensive as it needs to be, but having a proper comparison of these options would help with making the optimal selection.
My use case involves analyzing an ERP database of tens of thousands of inventory items to identify issues such as potential duplication, errors and typos, the potential to standardize items - meaning have one inventory item for multiple applications instead of different items per application (this involves looking at item specifications), and the like. There will likely be attached product specification sheets and other literature available - this video you shared has demonstrated how I can manage these attachments accordingly.
Thank you for taking the time to respond to my comment.
Thank you for this video lesson
You are welcome!
this is very well done, thank you. when is your next video and will it be on refining and improving RAG?
My next videos will be discussing the new models from meta and microsoft, and I'm working on another one that introduces langchain :)
very very nice! thx for sharing!
You're on a roll, thanks for watching!
is there a way which can be used if I want to let Lama answer from its own knowledge base when my own data is not relevant or is not accurate to be an answer to the user's prompt?
NICE One. ..question was how to give this to client as a remote work task project ?? and what are the cost optimization factors to be considered ?
hello pro
its very good tutorial
i have some file contain some article
its good to with small articles from 2 - 3 line
but if lines is more than 20 line its give response from no were
can i increase chunk size or any solution to solve this ?
thank you in advance
Keep it up bud your videos are great 👍
hello. Two separate questions.
a) Your code works (thanks) and I'm using it for a task at my workplace (basically help in extracting info from a repository of dozens of .docx). However it seems that the model only know about the input file (the embeddings) and "forgot" everything else. That is, I can query the txt and get mostly correct answers, but if my answer need to be complemented by additional info (that I know is available if I query mistral on ollama normally) it does not have a clue. Or am i doing something wrong? the "augmented" in RAG seems to indicate that we add (maybe with higher priority) the knowledge in the input docs to the existing one, but it seems not to be the case...
Hi there, I'm glad to hear the code works! It sounds like you need to change the system prompt, since we're explicitly telling it to only use the provided context and no knowledge that it already has. Let me know if that works!
@@decoder-sh thank you for your answer. I changed the systemp prompt as follows, but no change... :
"SYSTEM_PROMPT = """You are a helpful reading assistant who answers questions based on snippets of text provided in context and also with general knowledge. Be as concise as possible. If you're unsure, just say that you don't know. Reply in the same language used by the user question Context: """
(PS i added the language thing as it is digesting documents in french but I want answers in EN.)
do you think it's possible to use this, ollama+rag, in conjunction with crewai?
Definitely! I know that CrewAI has rag tools and can use ollama for inference. I plan on covering CrewAI in a future video :)
Great video. Do you recommend RAG as an ideal way to update llms with framework libraries changes and updates or is finetuning the way to go?
Yes I think RAG is definitely the best way to empower your LLM to give up-to-date answers about dynamic data like documentation. You'll just need to update your embeddings when the data changes.
Thanks for this great video. Just a newbie question from me: how to formulate RAG setup to make Ollama understand a specific text format? I provided mixtral a list of timed activites for a person like this:
Day1 7-00-00 : breakfast
Day1 7-00-08 : workout
Day1 7-00-16 : reading
......
Day20 7-00-00 : breakfast
Day20 7-00-08 : reading
Day20 7-00-16 : workout
I asked LLM to print out all instances of activities at 7-00-00 across all these 20 days, which is super easy for human, but the results from LLM were always wrong ... Can you give me some instructions? Is RAG suitable for processing such text data?
With data that is as cleanly structured as this, why couldn't you just write some simple code to filter the results for you? Like `data.split('
').map(line => line.split(' ')).filter(([day, time, topic]) => time === '7-00-00')` ? Or you could put your data into a database, give the schema to an LLM and have it write queries for you - this is called self querying python.langchain.com/docs/modules/data_connection/retrievers/self_query/
please, can someone tell me how to activate streaming on the response?
Very interesting thanks. Do you know how this compares to Haystack 2.0? I understand that Haystack being a framework offers more scalability while this is more of a DIY script?
This is my first time hearing about Haystack, but this is definitely just a simple DIY script to build an understanding of what's happening under the hood of more advanced libraries like Haystack, Langchain, etc
many thanks ! that's awesome what you share with us. Would be great to see more how to use bge-base 🙂 or not, actually I watched your YT video to change gguf file to ollama model and replaced it in: prompt_embedding = ollama.embeddings(model="bge-base-en-16", prompt=prompt)["embedding"], seems working, but I need to test it more 🙂
in this case maybe you could create a tutorial how to build from scratch a speech to speech multimodal assistant operating eg on Ubuntu ?
For me, I have to remove "[embedding"]" in line 22 for some reason. Any reason why?
It''s possible that the interface has changed since I created the video?
@@decoder-sh It's possible. I am getting an error on `needle_norm = norm(needle)`
TypeError: unsupported operand type(s) for *: 'dict' and 'dict'
May I know what numpy version you are using?
Curious, do you github? Thanks for another great video and useful content, very much appreciated.
Yes I have githubbed once or twice, why do you ask?
I appreciate you watching, it means a lot!
@@decoder-sh Thank you for your reply. I am not a developer, yet learning/absorbing.
Github key points (for my workflow):
- presents a reliable single source of truth
- centralized repository for rapidly growing/scaling projects
- serves as a hub to linked resources (e.g. websites, youtube, code, updates, changes, new projects, etc)
- project versioning
- lowers barrier to entry, while simplifying versioned resource aggregation for everyone, at every level.
can you have embedding with images ? using the llava model ?
Very interesting question! It's definitely possible to create image embeddings, and in fact that's how image search works on Google. However I don't think you can create image embeddings directly from Ollama. It looks like Llava embeds images using the CLIP model internally.
github.com/haotian-liu/LLaVA/blob/main/llava/model/multimodal_encoder/clip_encoder.py
huggingface.co/docs/transformers/v4.21.0/en/model_doc/clip
Interesting conversation on the topic x.com/willdepue/status/1772050291757850826?s=46
thnx man in million i really hate those xxxx lama index people absurd way of intentionally forcing their own set of storage
I'll look at llama index in a few videos, but the great thing about code is that you can always jam in your own solution 😎
Do I have to install Ollama in every new python environment I create?
Yes you will need to install the Ollama python library in every new python environment you create, no you will not need to install the ollama application (which serves the models and API that the python library talks to) for each new python environment
@@decoder-sh I see, thank you!
How long did it take to generate embeddings of entire book
It will depend based on your hardware, embedding parameters, and embedding model. Ideally it would take a minute or two. Using mistral, I think it took 5-10 minutes on a macbook M1, which is why I recommend against using that as your embedding model.
Hi,
What are your computer specs for running ollama?
Hi, I’m using an M1 MacBook Pro, but ollama itself has minimal requirements (you don’t even need a gpu!). It really comes down to the model you’re trying to run.
LETS GO
💻🦆
Could you give a similar example, but using graph databases like neo4js?
Yes I would love to do something with graph databases! Do you have a specific use case in mind?
@@decoder-sh Try texts with examples of Aristotelian syllogisms(from which propositional logic originated), to see if it holds the thread of reasoning. 😁 Are short, but hard to follow.
Or logical document in general, that you can see if it does good or bad.
wow my PC took 269.115 to embed 85 sentences with tinydolphin. I think I need to use the api key method.
Try out nomic! I've found it to be really fast since it's trained specifically for this task.