I went through few videos, where people directly jump in to code and go through line by line, It took me 2 - 3 videos to understand underlying design pattern. Your explanation has summed it up again for me with better clarity. Thank you.
informative; other videos have stated how VDB work, via embeddings; but the "design pattern" as you've taught it was the missing piece; sure it can be untuited, but having the clarity you've brought will likely save many some time. thanks.
Great video, Vivek! I wonder if we can make further tuning to make this Q&A use case into an article drafting use case. Love to hear your thoughts or even a video sharing your idea on that.
Great video, Mr. Vivek! Your insights on using Language Models for search are fantastic. I'm working on a project and considering applying this approach to enhance resume matching with job descriptions. Any quick tips for this application? Thanks!
That would be a cool application. You could try something along the lines of treating the job descriptions as docs (put them in the vector db) and resumes as "queries", but maybe after extracting key skills etc from them using a summarization prompt.
This was fantastic thanks Vivek, I'm completely non technical yet was able to understand most of this. One question if I may: Let's say I've got a database that contains information on 100,000 companies (roughly 500 words on each company). Is there any reason why I wouldn't be able to query that database with something like 'give me a list of all companies in the database that are in the engineering industry' and get an accurate answer? Or would it still not be 100% accurate.
Thanks for the kind words! What you will get is a measure of "similarity" (vector similarity, to be precise) between words in your query and words in your rows of your database. That embedding (mapping of words to an n-dimensional vector) is learned, not a deterministic lookup like a SQL query, and the query is usually open-ended, so there isn't a concept of "100% accurate".
So what we can answer queries over a corpus of documents that was 1) not in the training set of the LLM and 2) is too large to fit into the context window.
Give me the best RAG pipeline approach to chat with 10-K reports of different companies for the past 10 to 15 years accumulated count can range between 50 - 100 pdf files of 10K reports.
Vivek - Thank you for this 'first principles' explanation. Very helpful indeed!
The best video on llm so far
I went through few videos, where people directly jump in to code and go through line by line, It took me 2 - 3 videos to understand underlying design pattern. Your explanation has summed it up again for me with better clarity. Thank you.
Thanks for the kind words -- glad it helped!
Thanks Vivek. It is important that everyone understands this new paradigm better.
informative; other videos have stated how VDB work, via embeddings;
but the "design pattern" as you've taught it was the missing piece;
sure it can be untuited, but having the clarity you've brought will likely save many some time.
thanks.
Awesome, this approach is very promising!
amazing explanation. Simple and clean! thanks! subscribed :D
Great video, Vivek! I wonder if we can make further tuning to make this Q&A use case into an article drafting use case. Love to hear your thoughts or even a video sharing your idea on that.
Great video, Mr. Vivek! Your insights on using Language Models for search are fantastic. I'm working on a project and considering applying this approach to enhance resume matching with job descriptions. Any quick tips for this application?
Thanks!
That would be a cool application. You could try something along the lines of treating the job descriptions as docs (put them in the vector db) and resumes as "queries", but maybe after extracting key skills etc from them using a summarization prompt.
This was fantastic thanks Vivek, I'm completely non technical yet was able to understand most of this.
One question if I may: Let's say I've got a database that contains information on 100,000 companies (roughly 500 words on each company). Is there any reason why I wouldn't be able to query that database with something like 'give me a list of all companies in the database that are in the engineering industry' and get an accurate answer? Or would it still not be 100% accurate.
Thanks for the kind words!
What you will get is a measure of "similarity" (vector similarity, to be precise) between words in your query and words in your rows of your database. That embedding (mapping of words to an n-dimensional vector) is learned, not a deterministic lookup like a SQL query, and the query is usually open-ended, so there isn't a concept of "100% accurate".
@@VivekHaldar Thanks very much for the reply Vivek, and please keep these quality videos coming!
Are you going to launch any course on Deep Learning involving everything? From end to end...if yes would like to join..
Flattered that you asked but probably not.
why don't we directly send the query to LLM, what's the benefit here using a vector DB?
So what we can answer queries over a corpus of documents that was 1) not in the training set of the LLM and 2) is too large to fit into the context window.
ah, so it extends the original query, what would be example for this? if i may ask
Give me the best RAG pipeline approach to chat with 10-K reports of different companies for the past 10 to 15 years accumulated count can range between 50 - 100 pdf files of 10K reports.