35 minutes after downloading the clip, I received a notification, perhaps due to the weak internet in my country.... Finally, I would like to thank you sir for this wonderful explanation
updating the database? cz RAG using such as vector database that updated regularly using similarity from the prompt to the database content to retrieve and then augmented the prompt to the LLM
RAG is generally more cost-efficient than fine-tuning because it limits resource costs by leveraging existing data and eliminating the need for extensive training stages.
True, but over scale I would assume the excessive context needed to be provided at runtime via RAG to answer questions may outweigh the initial cost in fine tuning no? Like it seems fine tuning is an investment choice in training costs while RAG is an ongoing cost of additional tokens. Obviously in relation to context that can be train ahead of time.
Love the video! I’m building an app that empowers users to generate landing pages with a prompt using AI. I’m planning on building many custom components/sections that I want the model to use when generating a response. I want the model to choose the right sections to use and fill out the copy in the components to fit the users prompt. What would be the best way to handle this in the model? Use rag, fine tuning, both, neither, something else?
Nice vedio and also make a vedio on neural networks in deep like how neiral network is interlinked with deep learning and machine learning and what is actaully neuarl network and architecuture and why architectute is inporatnt fir neural networks and what is neural network actalkuy like a technique or mathematical expression or anything else so make a vedio on all these
Thank you, this is very useful. I'm curious about how the volume of data might affect the choice of FT vs RAG. If we tune the model again with the new data, would it become much larger over time? On the other hand, if we use RAG, would the restrictions on context length hold us back (i.e. if we don't want a very expensive model)?
Like Developer i was working and create system with RAG patern and its fine, but have problems something with specefic documents, i mean when you play with tons of documents the RAG system get more complex, and you depend in strength way of prompts, im not yet play with fine-tunning but is something near to do for experiment, nice video thanks
Use Case: if i have a huge online library of books and i need to use llm to answer questions based on these books and research papers i guess we will use RAG but the point is can use it with a really HUGE amount of data (books and PDFs) and what if there multiple answers for the same question but from different resources and each resource has its own opinion which could be in a different direction than the other resource. what will happened?
Great use case, Youssef! When you use RAG, it provides the model with multiple snippets of documents from your database. It's important to adjust the chunk size and the number of snippets injected into the context along with the user prompt. Typically, what I do with my clients is start with creating a set of evaluations for the system. These look like example prompts and example outputs. Any change I make to the system - I always run evals to see if the performance improves or gets worse. Once we have evals that measure how close the actual outputs are to the target outputs, we can work backwards and optimize the chunk size and number of snippets provided to the LLM. This way, it will get a balanced selection of relevant documents from your database. In some cases, it requires careful engineering to write proper search queries. Finally, the way the model writes the final response based on the retrieved information can be steered by instructions and fine-tuning. If you're interested in AI Development, feel free to contact me!
Good question! So with fine-tuning, using an approach like PEFT (Parameter-Efficient Fine-Tuning) which only updates a subset of the full model's parameters, we have new model weights and biases, which could then shared, deployed on a server, etc. for model inferencing with AI-enabled applications. For RAG, yes indeed, the most common method is with a vector database and turning your data into embeddings to search for similarity when using the LLM. But, there's other ways of setting up RAG pipelines too :)
@@cloudnativecedricWhen would it make sense to first use PEFT, then apply RAG? Do both PEFT and RAG assign/label semantic relationships to the texts of user-added corpora and store these in a graph database?
I thought the retriever was on the far right, and llm in the middle of both, was I wrong, partially, is that schematic representation doesn't fathom all of the architecture, I'd like to go deeper on that matter.
There are a lot of variances with the RAG approach that can lead to different architectures, but there's a full video on the IBM Technology channel that dives into RAG as well!
Yes, LLM model is retrained for fine-tuning. For efficient fine-tuning check out Parameter-efficient fine-tuning (PEFT). In PEFT a small set of parameters are trained while preserving most of the large pre trained model’s structure, PEFT saves time and computational resources.
Kinda, we will always have to choose the ideal model for the use case (off the shelf or finer tuned) and what context is provided to the model (rag and other data). Really, it's all about context, whether it's engrained in the model or added as part of the prompt.
Just some ideas from the top of my head for fine-tuning with financial records are preparing financial statements, tax preparation (fine tuning on region-specific tax rules and historical data), expense tracking & categorization, etc.
Obviously you need both, duh? No, seriously, they are not mutually exclusive. Fine tuning is learning, RAG is gathering requirements for a specific project. An expert needs to do both, he needs to learn in order to specialize, and he needs to be able to gather information for the specific task at hand.
![BOWKYLION - วิงวอน (ex-change) [Official MV]](http://i.ytimg.com/vi/cLdnZWDaO-w/mqdefault.jpg)
Thank you for the clarification, I had this question in mind last week, and I am glad that you have provided the answers I need.
Glad it was helpful!
35 minutes after downloading the clip, I received a notification, perhaps due to the weak internet in my country.... Finally, I would like to thank you sir for this wonderful explanation
You're welcome!
Love IBM's short and sharp explainers! Thank you for an excellent video once again :)
I have just watched the 1 years ago, then it updated today. Amazingg 🎉
I wanted to scream "WHY NOT BOTH⁉️) until 7:35 😂
Thanks a lot, that was a good one to understand both RAG and fine tuning.
wonderful explanation ... will love to also know the choice of use from TCO or cost POV
Thank you for this helpful video🙂. Could you please explain the implementation of how we can update the RAG system with the latest information?
updating the database?
cz RAG using such as vector database that updated regularly
using similarity from the prompt to the database content to retrieve and then augmented the prompt to the LLM
your video didn't included EDA, LLM answers based on pre loaded info, a future evolution is LLM answering based on real time information
Wow... combination is great. Thanks dear! For information
can u make a video about reinforcement learning and performance evaliation of llm models?
Great video, thanks. It was useful for me
Thank you for the fascinating presentation. Assume certain conditions are similar, how would the cost of rag and fine-tuning differ?
RAG is generally more cost-efficient than fine-tuning because it limits resource costs by leveraging existing data and eliminating the need for extensive training stages.
True, but over scale I would assume the excessive context needed to be provided at runtime via RAG to answer questions may outweigh the initial cost in fine tuning no? Like it seems fine tuning is an investment choice in training costs while RAG is an ongoing cost of additional tokens. Obviously in relation to context that can be train ahead of time.
Love the video! I’m building an app that empowers users to generate landing pages with a prompt using AI. I’m planning on building many custom components/sections that I want the model to use when generating a response. I want the model to choose the right sections to use and fill out the copy in the components to fit the users prompt.
What would be the best way to handle this in the model? Use rag, fine tuning, both, neither, something else?
Nice vedio and also make a vedio on neural networks in deep like how neiral network is interlinked with deep learning and machine learning and what is actaully neuarl network and architecuture and why architectute is inporatnt fir neural networks and what is neural network actalkuy like a technique or mathematical expression or anything else so make a vedio on all these
Thank you, this is very useful. I'm curious about how the volume of data might affect the choice of FT vs RAG. If we tune the model again with the new data, would it become much larger over time? On the other hand, if we use RAG, would the restrictions on context length hold us back (i.e. if we don't want a very expensive model)?
Make a vedio on termonolgioes are often used on ai like benchmark and art of the state and etcc ❤
Great explanation ❤
clear clarificaion, great job!
sir can you tell me how to make the vectorstore and store it in a specific file to use it every time.
Like Developer i was working and create system with RAG patern and its fine, but have problems something with specefic documents, i mean when you play with tons of documents the RAG system get more complex, and you depend in strength way of prompts, im not yet play with fine-tunning but is something near to do for experiment, nice video thanks
Use Case: if i have a huge online library of books and i need to use llm to answer questions based on these books and research papers i guess we will use RAG but the point is can use it with a really HUGE amount of data (books and PDFs) and what if there multiple answers for the same question but from different resources and each resource has its own opinion which could be in a different direction than the other resource. what will happened?
Great use case, Youssef! When you use RAG, it provides the model with multiple snippets of documents from your database. It's important to adjust the chunk size and the number of snippets injected into the context along with the user prompt. Typically, what I do with my clients is start with creating a set of evaluations for the system. These look like example prompts and example outputs. Any change I make to the system - I always run evals to see if the performance improves or gets worse.
Once we have evals that measure how close the actual outputs are to the target outputs, we can work backwards and optimize the chunk size and number of snippets provided to the LLM. This way, it will get a balanced selection of relevant documents from your database. In some cases, it requires careful engineering to write proper search queries.
Finally, the way the model writes the final response based on the retrieved information can be steered by instructions and fine-tuning. If you're interested in AI Development, feel free to contact me!
Fantastic Technology for value Great Lesson
Glad it was helpful!
Thank you 🙏💛
What happens to a model when it is fine-tuned? do you use a database for RAG?
Good question! So with fine-tuning, using an approach like PEFT (Parameter-Efficient Fine-Tuning) which only updates a subset of the full model's parameters, we have new model weights and biases, which could then shared, deployed on a server, etc. for model inferencing with AI-enabled applications. For RAG, yes indeed, the most common method is with a vector database and turning your data into embeddings to search for similarity when using the LLM. But, there's other ways of setting up RAG pipelines too :)
@@cloudnativecedricWhen would it make sense to first use PEFT, then apply RAG? Do both PEFT and RAG assign/label semantic relationships to the texts of user-added corpora and store these in a graph database?
Great video!
Thanks for the visit
How would i get to know which model is using RAG in it or Not?
I thought the retriever was on the far right, and llm in the middle of both, was I wrong, partially, is that schematic representation doesn't fathom all of the architecture, I'd like to go deeper on that matter.
There are a lot of variances with the RAG approach that can lead to different architectures, but there's a full video on the IBM Technology channel that dives into RAG as well!
Thanks ☺️
You're welcome!
Yes that's true LLM' are generalistic by default.
I have a question. Is the LLM retrained on the new information during fine-tuning ?
Yes, LLM model is retrained for fine-tuning. For efficient fine-tuning check out Parameter-efficient fine-tuning (PEFT). In PEFT a small set of parameters are trained while preserving most of the large pre trained model’s structure, PEFT saves time and computational resources.
Thank you~!
You're welcome!
You’ll always have to use a combination of both RAG and FT.
Kinda, we will always have to choose the ideal model for the use case (off the shelf or finer tuned) and what context is provided to the model (rag and other data).
Really, it's all about context, whether it's engrained in the model or added as part of the prompt.
Using “Fine Tuning” , then machine ( accounting software) can be a bookkeeper to prepare financial records for …?
Just some ideas from the top of my head for fine-tuning with financial records are preparing financial statements, tax preparation (fine tuning on region-specific tax rules and historical data), expense tracking & categorization, etc.
cool
this is gold
so the concept of rag is like you attach file in gpt and asked question based on the attached file. isn’t it?
I would like to see a real app that is in production with RAG and fine-tuning.
Euro 2024 World Championship. Nice... of course the LLM could't give a response 😂
The RAG isn't updated with new tournament 😂😅
Large Manguage Model! 2:08
Large Language model is "LMM"?
Whoops! Good catch, sometimes I mess up when speaking and writing at the same time, it should be “LLM”.
Obviously you need both, duh?
No, seriously, they are not mutually exclusive. Fine tuning is learning, RAG is gathering requirements for a specific project. An expert needs to do both, he needs to learn in order to specialize, and he needs to be able to gather information for the specific task at hand.
You did not tell about cost difference :)
Garcia Michelle Thompson Scott Martinez Ronald
wtf
Did he just write “LMM”, instead of “LMM”?
Garcia Kimberly Lopez Karen Hall Mark
So, You are all told to wear your watch on your right hand right?!
White Deborah Wilson Susan Garcia Cynthia
Uhhh okay i see you .....😂😂😂
LMM lol