Building a RAG application from scratch using Python, LangChain, and the OpenAI API
ฝัง
- เผยแพร่เมื่อ 4 มี.ค. 2024
- GitHub Repository: github.com/svpino/youtube-rag
I teach a live, interactive program that'll help you build production-ready machine learning systems from the ground up. Check it out at www.ml.school.
Twitter/X: / svpino - วิทยาศาสตร์และเทคโนโลยี
DUDE!! this is the best resource I have found so far. The part I truly appreciate is that you focused on the conceptual parts and NOT just here is the code
It's one of the most precise explanations I've ever seen. Your labs are pretty amazing; my sincere compliments and thank you. I will definitely join the next cohort.
Cheers.
This will be a great resource! I'd love to see a dedicated video diving deeper into LangGraphs and their applications. Keep up the great work!
This is the best crispy clear RAG walkthrough that I have seen until now ! Congratulations and thank you for putting effort into educational videos on GenAI use
Thanks!
Looking forward to upcoming videos! Also it would be interesting if you can cover the following topics:
1. Langchain - saving context of conversation and using in next questions
2. Dialogflow intents or if it can be built in LangChain.
3. the real-time data to be added based on conversation.
4. Streaming audio to LLM, and the way to implement conversations like chatGPT by holding button, with less latency.
Thank you, this is top-notch content!
Many new for me llm rag concepts are clearly explained, and most importantly, it shows how they are interconnected and reinforce each other.
Excellent walk through capturing all the required details
Love it how you explain everything step by step!
For me, good mentor is someone that can explain difficult topics with elegant simplicity and you doing it superb! Looking forward to Cohort 12 :)
I’ll see you in class!
Awesome explanation and content. Your content is by far the best I found on RAG... hats off!👏
Wow. Great teaching! Explaining the "why" on your lessons, in this case (RAG) is what you dont find in other teaching material . Thank you, this has been very insightful! 👍
it is not 15 minute and i am sure it is one of the best video on this topic
Santiago, thank you for the valuable RAG videos. Your insights are well-explained and informative. I would be interested in seeing more of your work on this topic.
Nice - looking forward to dive into this.
Could be nice to see a setup for Application with local LLM for private docs. I think a lot of people are looking for such a video 🤞🏻💪🏻 TY for your Nice work!
Thanks for taking the effort to explain the concepts one by one.
Genuinely found this to be hyper-valuable, after watching this, tons of concepts now actually make sense to me, Thank you!
Amazing video. Thanks for the work you do in putting all this together.
Awesome video with great explanation, step by step with examples, images and code!! Love it.
Million stars to this video. The content is clear, precise and well explained. Thank you.
man this is great, keep it up.
also it will be appreciated if you make a video for a full RAG system including:
1. adding the most suitable way of using memory .
2. how to embed and deploy these application for production(web, mobile, saas...etc)
This tutorial was the perfect level of theory + code for a beginner like me to get enough knowledge to start tinkering myself. Subscribed and looking forward to more content!
I already have experience with Langchain, but wanted to learn more about Pinecone. You are just exceptional at teaching. Thank you so much for this. :)
I’m only 15 minutes into this and was mostly looking for a RAG example, but I must say as a beginner to all this I really appreciate how much you’re breaking down the entire process. I haven’t used langchain for invoking models and you have me excited to do so now. The workflow seems much more logical this way.
That’s the goal!
Thank you. It is the perfect video when you want start doing RAG just after knowing what RAG is for. Now it is easy to do chatbot for technical documentation queries.
I have implemented your example in node js and it's working like a charm. Thanks!
But I really hate the langChain api. I'm trying to implement it as plain old functions.
Super insightful! Very nice explanation and walkthrough! Thank you!
"All you need is Attention" to the details this GREAT teacher is providing , he could have picked up a simple context within the limits and would have been done BUT no his zeal to share and show every single road block that we need to know is OUTSTANDING. God Bless You. Could you please do a Speech To Speech Application as well?
Thank you for this, your explanation on embeddings was superb!!
Such an amazing Resource !! Thanks Santiago ❤
You're the Legend, Santiago!
Love the way you explain :)
Fantastic tutorial! Thanks a lot! I already saw other tutorials on LangChain and also I purchased a Udemy course. This tutorial here is the best of all. Everything is well explained and can easily be understood.
This video got to me on the perfect day, you just solved a whole project for me with this. Thanks!
So great video, you alway explain complex concepts very well!
Thanks for the detailed, clearly explained tutorial. Nice
Love your method of teaching!!!
Thank you a lot Santiago, best tutorial I have watched so far
Thanks for the video. Very clear explanation of all concepts involved. Looking forward to your next videos.
If possible, please can you cover the topic of production deployments and operations for a LLM project.
Super helpful walkthrough. Thank you !
Amazing video. You are such a wealth of knowledge. Thankyou for these great video tutorials. Please keep them coming!!😃
Great tutorial! Thank you Santiago!
I actually tried it myself (first time using the OpenAI API). I was able to build a RAG application to get answers based on a PDF document. And it works!
However, I see that there are some more stuff to think about, such as storing the embedded pages in the vectorstore might not be the best approach.
I am looking forward to new videos!
Thanks for your valuable insights! It has been immensely helpful.
Very helpful and practical tutorial. Thank you Santiago.
WOOWWW!! an amazing effort thank you so much. Look forward for more
Amazing! Very well explained!!!
You gain a new follower. Great job!
Awesome content and explained in details, thank you !!
Amazing explanation. Thank you for making this video.
Wonderful video! I made my post-graduate final project exactly like this.
This is really good. Thanks for posting this tutorial
This video is very informative ,thank you
Keep uploading this kind of content
Excellent Video and Excellent Explanation. Thanks alot.
i started to love machine learning bcz of you thnkx uu
🙏🏻
Really informative video. Thanks Santiago :)
Damn... I learned a lot from this video!
Thank you for this.
I have subscribed to your channel and looking forward to watch your future videos.
Nicely done!!
This is really great video, thanks a lot @svpino for sharing this! Liked and subscribed!
Thanks man!
Great explanation. Waiting for new videos
Thanks Santiago ! As always , high value content ! Just one quick question: why not match the LLM max content size with the chunk document size ?
best class period. you need to launch LLM course!
Great video. Best video on youtube. Muchas Gracias Santiago.
Thank for your video, It is easy to understand
Thank you very much, this video is awesome to begin in ai
Great video and explanation, thanks m8!
This was great and I learned a lot, I have now subscribed you, thank you so much!
Awesome video! Thank you for it!
Glad you liked it!
Thanks for another great video Santiago. Any insight into when it might be a good idea to incorporate LlamaIndex into an application? Just learning about it but trying to understand how it fits in with something like Pinecone.
Dude this is the perfect video
Hello Santiago!
Thanks for this amazing video and for the explanation. Is it possible to provide an updated version of this RAG system to have be a chatbot with a conversational behaviour not only Q&A system? so it takes into consideration the previous questions and answers as part of the context to be able to answer questions that refer to something already mentioned before?
Thank you!
This is really great job and much helpful to me.
It would be greatly appreciated if you could make a video about prompt engineering.
Thank you for the great video! This is very insightful and helpful. I have one question: What if there are multiple documents input, and the answer you want to retrieve exists across different chunks? In that case, only fetching the most similar chunk will generate incomplete answer. However, if we search the top N most similar chunks, how can we make sure that they can be connected meaningfully (it could be the case that the top 3 most similar chunks are on Page 3, 80, 100, and there is no meaning connecting them together)? Thanks again for the effort you put in this video!
Great tutorial - thanks
great video sir, thank you very much. Please make video on fine-tuning using PEFT.
very clear, i like it, keep going bro
Thanks, will do!
Wow this was great!
Best beginner level explanation of RAG with Langchain on Internet. Great one Santiago!
Thanks!
thank you for this video, mate :)
Excellent introduction to Pinecone use for RAG using Langchain. Looking forward to more...I had an error 2/3rds way through AttributeError: 'builtin_function_or_method' object has no attribute '__func__'. Eventually, I installed Python 3.11.8, re-cloned the repo, installed the same dependencies and it worked again, no problem!
How would you scale to more than 1 document (in your case 1 transcript). Would you continue to use 1 vector store for all the documents or are there other methods? Great video btw!
super. Well explained. thanks
Great video. The jupyter notebook was very helpful!
Interesting that, by default and only given the context of "Patricia likes white cars," the parser came to the conclusion that Patricia's car was white even though she might not actually own a white car. I added instructions to tell me when it was inferring an answer but makes me wonder what other things it might be inferring without telling me why.
I don't know why but I get the response "I don't know" for that question even though my code and the prompts are identical and I'm also using the OpenAI API.
@@freerider6300 Given that the template for the prompt says to respond as "I don't know" that seems appropriate. Are you by chance using gtp-4 vs gtp3.5-turbo? I changed my model to use gtp-4 and I get the response as "I don't know" where, with gtp-3.5-turbo, I get the response as "White" just like it shows in the video.
amazing video, thanks
Thank you so much. You're the best.
Glad it helped!
if anybody had the same problem with activating the virtenvir venv as i: source command didnt work - it’s just different based on the os so source works for linux/macos and the .venv\scripts\activate for windows
great content.
I have a question:
"how can we add the openai's function calling in this for extrenal api's calling?"
Thanks Santiago, this helped me understand RAG as a beginner more than any other video out there. Your simple example alongside the more complex video vectorstore in particular.
Question, are there local opensource vector store alternatives to pinecone which are robust and straightforward?
I saw your followup video which used ollama... Going to try and write some example code to replicate this with ollama vs openai.
Thank you!
Check this link: www.reddit.com/r/ChatGPTCoding/s/OIBZJnNYuI
Hello,
thank you so much for this video that help me a lot to clarify a lot of concept.
i have a question related of aggregation questions in LLM documents.for example the vector database have thousands documents with date property, and i want ask the model how much document i received in the last week?
what can be the best practice to handle this kind of use cases?
Great video. Subscribed.
Köszönjük!
So, langchains are pipes for LLMs ?
Do they provide any of the output splitting/redirection that you get from classic stdio ?
Great, thanks, Santiago. What about using gpt4?
Very interesting. Can embeddings be used to detect similar chain of strings (like part numbers with and without dashes or with different variuants) or they do work only with concepts like cars, parent relations, colors...?
Good question. I’m not sure. It should be simple to try.
In the retrieval part, what difference should we make if we plan to retrieve from a knowledge graph/base instead of performing a vector search?
Great one even people with little knowledge about LLMs can understand this very much.
Nice video! Very helpful for beginners! Just one question, in terms of performance isn't it better to create one prompt and one call to model with the context, answer and translation? It would be just one call to OpenAI server instead of two..
For that particular example, yes. One call would be better.
But think beyond that. You may have two separate chains using different models and processes. One call might not be possible, and that’s where chaining different chains might be helpful.
Oh, I see.. Valid argument! Thanks for explanation, it's really great explanation of the concept itself!
Could you also do a video on how you can show the source documents that the context comes from?
Great video
Love your video! I you allow me a small suggestion, I would have put your face in the top left corner in order to avoid the overlapping with the code. Great Work
True
Finally a video that i can follow through and understand what embedding and vector is. Thanks!
Is this the same type of thing you do with ChatGPT assistants or CoPilot chatbots that point at docs/sites?