Fascinating! Your approach to teaching and presenting is poetic. It is well organized, well explained, and well illustrated. Indeed, kudos to you. If I could, I would subscribe to your channel twice!
Llama Index has new version 0.10 - will migrate your codes n learn same time. Thanks for introducing Sentence Window Retrieval. Most basic straight-split and retrieve/chat doesnt produce much meanings on our docs.
Great job explaining window and how vector store and doc store relate and where window lives. I’ve been trying to understand this aspect of llamaindex, and you made it very clear!
Seeing your explaination at around 09:30, it seems like we can only use K windows to serve as knowledge base to answer a prompt. What If the prompt asks information that is contained in more than K windows? Like if I have several documents containing each a bio of a person, and if the user asks to sort those 10 people by age... how can it figure it out? I guess we can use a big value for K, if the cosine similarity engine can take it... but I am guessing providing too much context to the LLM will cost a lot of money?
Yeah, it wouldn’t work very well in that scenario. I wonder if perhaps there is a strategy where the LLM can determine the right K based on what it is trying to do. Or instead of setting K to be a fixed number, to return all chunks where the cosine similarity is above a given threshold.
I'm a bit confused on the use case for re-ranking. Doesn't that defeat the purpose the top-k search in that we include all chunks, significantly increasing the number of tokens we use? Is the idea to do re-ranking with a smaller & cheaper LLM before sending the resultant top-K chunks to a more robust LLM?
yeah, it took me a while to understand it as well. It does end up using more tokens but I guess that can be, as you said, mitigated with a lower spec LLM. The idea is that embeddings is not a perfect representation of the text - it's a good way to sort through millions of text chunks efficiently but the exact ranking within the top 20, for instance, may not be as good as using the text directly. So if you were to do reranking, you might use the embeddings to return j results (where j is somewhat larger than k) and then rerank those j results using LLM into a final top k that you then pass into your user-facing LLM.
This is currently the best RAG tutorial on the internet.
Fascinating! Your approach to teaching and presenting is poetic. It is well organized, well explained, and well illustrated. Indeed, kudos to you. If I could, I would subscribe to your channel twice!
Great video with an awesome easy to follow explanation of RAG. Reminds of a recent Andrej Karpathy video.
clear effective explanations thank you
Llama Index has new version 0.10 - will migrate your codes n learn same time.
Thanks for introducing Sentence Window Retrieval. Most basic straight-split and retrieve/chat doesnt produce much meanings on our docs.
Great job explaining window and how vector store and doc store relate and where window lives. I’ve been trying to understand this aspect of llamaindex, and you made it very clear!
Yeah, it took me a while to figure that out too. Glad it helped you!
As always, well prepared, easy to follow video that delivers a lot of information and value. Thank you!
Your explanations and delivery is on point. Thank you for an excellent content and relaxed narration style.
Wow, that’s a wonderful piece of advice from such a talented professional in the field. Thank you 😊
Really cool video! Is there an "ideal" or "recommended" value of window_size?
Wow _ darn useful !!
Awesome video explained very clearly! Thanks a ton!
If I may ask, what tool do you use for those visual flows. Love it!
Love it!!!!!!!!!!!!!!!!!
Another great video!
Great video!! Love it
Thank you!!
Seeing your explaination at around 09:30, it seems like we can only use K windows to serve as knowledge base to answer a prompt. What If the prompt asks information that is contained in more than K windows? Like if I have several documents containing each a bio of a person, and if the user asks to sort those 10 people by age... how can it figure it out? I guess we can use a big value for K, if the cosine similarity engine can take it... but I am guessing providing too much context to the LLM will cost a lot of money?
Yeah, it wouldn’t work very well in that scenario. I wonder if perhaps there is a strategy where the LLM can determine the right K based on what it is trying to do. Or instead of setting K to be a fixed number, to return all chunks where the cosine similarity is above a given threshold.
Excellent video. Liked the workflow you showed in the beginning. What SW are you using to create this workflow?
Excalidraw
Small correction: embedding are not a 1536 digit number but of vector of size 1536
Yes, you are right …. I was thinking of hashes for some reason! I’ll add a correction!
I'm a bit confused on the use case for re-ranking. Doesn't that defeat the purpose the top-k search in that we include all chunks, significantly increasing the number of tokens we use? Is the idea to do re-ranking with a smaller & cheaper LLM before sending the resultant top-K chunks to a more robust LLM?
yeah, it took me a while to understand it as well. It does end up using more tokens but I guess that can be, as you said, mitigated with a lower spec LLM. The idea is that embeddings is not a perfect representation of the text - it's a good way to sort through millions of text chunks efficiently but the exact ranking within the top 20, for instance, may not be as good as using the text directly. So if you were to do reranking, you might use the embeddings to return j results (where j is somewhat larger than k) and then rerank those j results using LLM into a final top k that you then pass into your user-facing LLM.
Great video? Is the diagram anywhere to refeeence?
I’ll post it up tomorrow - need to fix an error
@hubel-labs Any update on the diagram please - thanks? Great vid by the way.
I added it to the description. Here is the link: link.excalidraw.com/readonly/m6DK7oyEFpyQnuw55DVP?darkMode=true
df
it is 1784 in teahistory.txt and 1794 in chinahistory.txt so bit confusing
but anw great tutorial Thanks