The amount of research work on retrieval augmented generation for large language models has exploded in recent times. Thanks to the speaker for directing attention to the most significant bits.
How many have you heard, right here on TH-cam? Lots of actual hands on info out there and in 1/3rd the time. This one had loads of intro forever before getting into specifics.
Wow! Go Bruins! I studied under Michael Dyer at UCLA. He was a big proponent of neural networks when I took his neural NLP course in 2010. A lot of us were skeptical as it was an "old method" that had been replaced with statistical NLP. I had no idea his paper was one of the first. I randomly ran into him about a year ago at a local restaurant and when I told him his classes were very fundamental to my career he asked in his usual sense of humor "So are you a billionaire?" jokingly. Unfortunately no haha.
It is good to see the whole spectrum of options, but what would be a practical way to get started on this? His old colleagues in HF actually have an excellent book that goes through many different ideas including RAG in chapter 7 where they do an excellent job explaining context and giving you options for implementation.
Hi, about Atlas. You said that we can update the Retriever. At 42:00 is some retriever loss, but what about pair label (question, positive paragraph, negative paragraphs) like normal retrieval model - do they contribute to the retriever loss ? I read codebase of Atlas, and do not find that kind of loss
Language models actually appeared sometime during the Cretaceous period, though scientists aren’t quite sure of the exact year. They think the stegasaurus might have had something to do with it; all these people who think OpenAI invented them are so wrong.
Pros: Gives a brief overview of many RAG methods Cons: No intuition given which is the key reason for why it works for the different methods Would have preferred more insights rather than just describing the papers, but overall thanks for the video!
THe bit about where this came from was funny, but your search just gave a less silly answer. Go reread Shannon's 1948 paper that invented Information Theory. Yes he did not talk about using neural nets (which did not exist) but he did talk about probability.
I know, isn’t it hilarious? The bitterness and jealousy is so transparent. It reminds me of the scene from The Social Network where they’re trying to claim credit for his creation and he says “You know it’s really not that complicated. If you guys were the inventors of Facebook, you’d have invented Facebook.” Yann LeCun likes to drone on on Twitter about how everything OpenAI is doing is “old” technology. Well if it’s so old, how come hundreds of different companies filled with smart people were trying for years to make a chatbot that was worth using but until OpenAI no one seemed to manage it? Yes, OpenAI didn’t invent the Transformer. We know. Who cares? They clearly solved dozens of incredibly difficult engineering problems that no one else had been able to solve, and gave the world a language-based AI that was actually _useful._ As evidenced by the fact that it was the fastest growing app in human history, by an extremely wide margin. And as soon as they do it all these idiots turn up basically saying “I could have done that 10 years ago, I just chose not to.” Yeah, ok. It’s so pathetic.
Maybe because only you are the one that is obsessed by the idea of starting learning something new but merely due to its "novelty and coolness" is something to be ashamed of and you project that idea on others which make you think others are watching this video with the same reason but hinding that information (and trying to look smart) because it is emabrassing to declare and you think that way too. However I will remind you that trying to learn new things whether it is because it is fashion or cool is nothing to be ashamed of and a great excuse to start new things.
You're a P1G. Memorising and Repeating without Understanding. What is a Sentence? A Collection of Concepts. The Brain can Learn 10's of Thousands of concepts. It Has Hardware To Help it. Concept: "Doing" brings up pictures of Making a Cake or Assembling a Chair. Concept: "What" brings up Pictures of "Asking someone a Question" or Deciding Colour Show to Buy. A Computer Doesn't Work Like That. All it Understands is Numbers and Maths. All These "Concepts" have To Be Encoded in Numbers and Maths and Rules Applied To Them. The Computer Doesn't Know What The Character "B" is. All it Know That if it Finds The Number "66" in Memory, it Draws Something That Looks like "B". Spelling. It Doesn't Know What Spelling is. All it Knows is "66,65,84" Valid for word BAT. BAB 66,65,66 is an Invalid String of Numbers and Not in the Dictionary. 10,000 Concepts Need - Modelling and Rules Creation (100's of Thousands) - For the Computer To Understand. That isn't an Easy Thing To Do: Trillions of Lines of Java Code.
Maybe if you were a computer maybe you would understand that you Don’t Need To Capitalize Every Word when writing English. And trillions of lines of code? You know nothing about coding; in all of human history everyone out together has not written a trillion lines of code, let alone in a single computer or a single program. P.S. Your comment was utter nonsense, it contributed nothing, and it meant nothing. Have a nice day.
I love that this content is freely accessible to everyone. Lots of helpful information being shared here
The amount of research work on retrieval augmented generation for large language models has exploded in recent times. Thanks to the speaker for directing attention to the most significant bits.
The best talk about RAG so far
How many have you heard, right here on TH-cam? Lots of actual hands on info out there and in 1/3rd the time. This one had loads of intro forever before getting into specifics.
@@morespinach9832 Can you recommend some sources? I'm compiling a list.
@@morespinach9832 can you recommend some videos?
@@morespinach9832 What lecture do you suggest for a more practical (code) view?
@@comunedipadova1790 plenty of them - search for these keywords.
Nicely explained in just right technical details. Thank you!
Excellent content, thanks for the references!
Good lecture, offers a good summary of the literature on RAG.
Wow! Go Bruins! I studied under Michael Dyer at UCLA. He was a big proponent of neural networks when I took his neural NLP course in 2010. A lot of us were skeptical as it was an "old method" that had been replaced with statistical NLP. I had no idea his paper was one of the first. I randomly ran into him about a year ago at a local restaurant and when I told him his classes were very fundamental to my career he asked in his usual sense of humor "So are you a billionaire?" jokingly. Unfortunately no haha.
So many great ideas here! Fantastic resource, thank you.
this is what I needed, thank you sooooo much!!!!
Thank you for the great contents!
My lecturers in my university never explain those things, thanks for this free lecture
just the thing i wanted thank you so much.
Great insights and video!
Awesome content, thanks for sharing!
Thank you.
Really cool.
It is good to see the whole spectrum of options, but what would be a practical way to get started on this?
His old colleagues in HF actually have an excellent book that goes through many different ideas including RAG in chapter 7 where they do an excellent job explaining context and giving you options for implementation.
3:34 What chatgpt was really about - fix the user interface to LM
12:00 Frozen RAG
Hi, about Atlas. You said that we can update the Retriever. At 42:00 is some retriever loss, but what about pair label (question, positive paragraph, negative paragraphs) like normal retrieval model - do they contribute to the retriever loss ? I read codebase of Atlas, and do not find that kind of loss
Actually we can see a first sign of language models in Shannon's 1948 paper, A Mathematical Theory of Communication.
Language models started to appear in the 1800s.
Language models actually appeared sometime during the Cretaceous period, though scientists aren’t quite sure of the exact year. They think the stegasaurus might have had something to do with it; all these people who think OpenAI invented them are so wrong.
Pros: Gives a brief overview of many RAG methods
Cons: No intuition given which is the key reason for why it works for the different methods
Would have preferred more insights rather than just describing the papers, but overall thanks for the video!
Isn't this basically true of all deep learning and LLM tutorials? hehe
WE ARE IN THE FUTUREEEEEE
Does these embeddings tools such as FAISS work with other languages other than English?
THe bit about where this came from was funny, but your search just gave a less silly answer. Go reread Shannon's 1948 paper that invented Information Theory. Yes he did not talk about using neural nets (which did not exist) but he did talk about probability.
Could you please share the slides?
does anyone have list of research papers metioned in this video?
Shouldn’t it be Shannon, 1949?
I did like the joke you made.
What about scann by google ??
Are there assignments for this?
where can I find the slides?
why there's no framework to make the process was easy
there is too much review information
worth dozens of "shiny" "trendy" "tik-toky" videos over there, without asking you to subscribe and leave comments..
It's not entirely clear what "frozen rag" and "retrieve rag" refer to.
I love how everyone tries to hide the fact that OpenAI is 100% the reason everyone is watching this video
No one is hiding anything
“Ignorance” is not equal to “facts”
I know, isn’t it hilarious? The bitterness and jealousy is so transparent. It reminds me of the scene from The Social Network where they’re trying to claim credit for his creation and he says “You know it’s really not that complicated. If you guys were the inventors of Facebook, you’d have invented Facebook.” Yann LeCun likes to drone on on Twitter about how everything OpenAI is doing is “old” technology. Well if it’s so old, how come hundreds of different companies filled with smart people were trying for years to make a chatbot that was worth using but until OpenAI no one seemed to manage it? Yes, OpenAI didn’t invent the Transformer. We know. Who cares? They clearly solved dozens of incredibly difficult engineering problems that no one else had been able to solve, and gave the world a language-based AI that was actually _useful._ As evidenced by the fact that it was the fastest growing app in human history, by an extremely wide margin. And as soon as they do it all these idiots turn up basically saying “I could have done that 10 years ago, I just chose not to.” Yeah, ok. It’s so pathetic.
Maybe because only you are the one that is obsessed by the idea of starting learning something new but merely due to its "novelty and coolness" is something to be ashamed of and you project that idea on others which make you think others are watching this video with the same reason but hinding that information (and trying to look smart) because it is emabrassing to declare and you think that way too. However I will remind you that trying to learn new things whether it is because it is fashion or cool is nothing to be ashamed of and a great excuse to start new things.
Whats an Open Al?
Us Mandalorians CYBERMEN: Understand The Material - Given To You By Minions, better Than The Minions. Your Gods Gods. Ask Them.
Could you try writing that again but having it not be nonsense this time?
meh
You're a P1G. Memorising and Repeating without Understanding.
What is a Sentence? A Collection of Concepts.
The Brain can Learn 10's of Thousands of concepts. It Has Hardware To Help it.
Concept: "Doing" brings up pictures of Making a Cake or Assembling a Chair.
Concept: "What" brings up Pictures of "Asking someone a Question" or Deciding Colour Show to Buy.
A Computer Doesn't Work Like That. All it Understands is Numbers and Maths.
All These "Concepts" have To Be Encoded in Numbers and Maths and Rules Applied To Them.
The Computer Doesn't Know What The Character "B" is. All it Know That if it Finds The Number "66" in Memory, it Draws Something That Looks like "B". Spelling. It Doesn't Know What Spelling is. All it Knows is "66,65,84" Valid for word BAT. BAB 66,65,66 is an Invalid String of Numbers and Not in the Dictionary.
10,000 Concepts Need - Modelling and Rules Creation (100's of Thousands) - For the Computer To Understand. That isn't an Easy Thing To Do: Trillions of Lines of Java Code.
Maybe if you were a computer maybe you would understand that you Don’t Need To Capitalize Every Word when writing English. And trillions of lines of code? You know nothing about coding; in all of human history everyone out together has not written a trillion lines of code, let alone in a single computer or a single program.
P.S. Your comment was utter nonsense, it contributed nothing, and it meant nothing. Have a nice day.
Thank you