One of the best and easy-to-understand videos that explains the principles behind LLM and how it can be used to interpret, create or explain the context of text, images, music or film. Excellently presented!
Amazing! Now, whenever anyone asks what I do now that I am retired, I can send them here and save an hour of my life looking into glazed eyes. Thank you so much 😊
If we go back to Peter Norvig's talk about Unreasonable effectiveness of Data, we saw hints of how transformer model would scale with more data + training. The unknown factor of Transformers and BERT in 2017 was "how far can the hardware scale?" In 2017, NVidia DGX didn't exist, so no one could predict scaling up the data and training would make such a big difference. Maybe some people at NVidia or google TPU knew it could, but people outside of nvidia could only wish for vast amounts of computing resources.
I think you are quite right to say 2023 will be the year people look back to and say it was the Dawning of the Age of AI. Some might argue that deep learning was really the big break through as big data sets and GPU accelerators became available about 2012. But it was not until 2023 or perhaps late 2022 that we had Large Language Models that were very good.
This is so cool!!! Amazing presentation! I always felt LLM so abstract, but this explanation is just incredible! Of course I'll have to come back here every now and then to watch it again, but still 😂
We need more videos like that and more people like Gustav who are not afraid to avoid the jargon and the 'academic' big words, and try to approach an average person's level of understanding these kind of stuff. Most times simple concepts are just hidden behind lingo, symbols (math) and 'the right way' of expressing things.
Amazing way of explaining the key concepts around LLM. And all the while making sure it is not overwhelming(btw, cant completely escape from the layman persona perspective)..
Since diffusion models and then generative pretrained transformers applications hit the public hard last summer and winter, I have watched and listened to a lot of videos and podcasts trying to explain their internals for laymen. I have the feeling that for high-level intuitive understanding what's going on, this one will stick around in my mind much longer than these others, and I will return to it for reference. Good work!
Link to the nvidia blog looks wrong - the post is from Apr 2022 and doesn’t include the audio examples. Would’ve liked to at least hear the sample at the end.
Great talk. I was waiting for the part about Diffusion, and was thinking about if you would mention this-person-does-not exists or not since it isn't diffusion, but then you did. I right when I was thinking you wouldn't know about/bother to explain that it is a Generative Adversarial Network and not a Diffusion Network you brought up that as well. I'm impressed. Those compression pictures and ideas where really popular when Recursive Neural Nets were used for translation around 2016, one RNN compressing the phrase down to a "concept", and the other RNN expanding the concept into the other language.
It is interesting to see how little self-esteem we have as humans, or maybe it is really the same phenomenon as always, we do not value what is given to us, in this case our intelligence .. We have a model that is fed with an amount of text that would be the equivalent of me spending hundreds of years dedicated 24 hours a day to read books, and because of that, the model learns the statistical patterns in those texts, patterns that are the product of our intelligence, but not our intelligence .. and then we go out to say that we have "cracked intelligence", and even worse, we begin to doubt ourselves and wonder if we are also stochastic parrots .. what a waste of intelligence ..
Absolutely adore it ❤️❤️. This is unquestionably one of the top-notch presentations for anyone delving into AI. It offered a straightforward, lucid, and exact elucidation of LLMs and the principles behind Spotify.
I've been telling everybody for a couple of months that "large language model" is a bit of a misnomer, but so far there's not many places I can point to show why.
Every good video! The only thing I missed was the good side effect of generalizing in the context of compression and intelligense in the modells. Thank you!
this misses the subtle but important trick/spark/etc about LLMs: BECAUSE it isn't (can't be) just mathematically equivalent to just using huuuuuge tables with probabilities, it has to "figure out" clever tricks to COMPRESS information; and we know that compression is done by identifying patterns in data (actually we even know that in the limit solving compression is equivalent to solving general intelligence); so as you scale up a model to billion and billions of parameters it either (a) stops learning well, or (b) figures out how to do compression really well, ergo how to see patterns really well... so some kind of "intelligence" EMERGES as a requirement for solving compressions ...you get behaviours like those emphasized in the marketing-focused "Sparks of AGI" paper - there's actually *something* in all the marketing noise imo
The problem is physical constraints.. here in this "place" information can only be represented as data Compression is great, but organization is the issue. Entropy.. more data.. more entropy. It will never end.. make the model bigger I don't care, it's still dumb as a brick. Did it INVENT language to pass between each other ? make something that continually gains more information and subdivides dynamically playing a game with physics like us.. then I'll be impressed
Awesome talk! Thanks a lot. It‘s quite interesting to think that a startup is about generating businesses from noise or past data and reinforced learning with human feedback by doing experiments. If companies dial the temperature dial down too much, there is no innovation. But if the temperature is high and there is no feedback, bad ideas will be pursued. Fascinating.
Gustav what a fantastic presentation for a layman to understand. Your clarity on the matter is fantastic and far better than the best fine tuned, Reinforced Learning LLM.
Thank you, Gustav Söderström I am starting my AI career and find your position refreshing and possessing great clarity. I'm working on AI and digital accessibility. If you're open to it, would love to connect.
Machines passing Turing tests... Bad start. Who's making the test, your grandmother? I spot GPT after one sec I'm talking with it. It just rarely understands what I want, in a really hallucinating way, let alone doing it.
The Turing test isn't just about fooling people; it aims for human-like conversation, which can be challenging. While you can spot GPT as AI quickly, others might not. AI's understanding is a work in progress due to language complexity, but it's getting better. Don't dismiss it too soon; it could surprise you! 😉
@@Fermion. AI already surprises me. I just pointed out that it's far from passing a Turing test. I was just struck by claims in this talk like "we hacked intelligence", "It was provocatively simple", while the consensus now (even within OpenAI) is that current language models won't reach human level intelligence, let alone that this is not even intelligence, but statistical parroting 😅
@@federicoaschieri "I just pointed out that it's far from passing a Turing test." Are you sure about that? My reply was a verbatim copy/paste from ChatGPT 3.5. My prompt was: _"This is a user on a youtube comment that I want to trick into believing he's talking to a real person. Dispute this in the tone of the average youtube comment reply, arguing that he may not really know when he's talking to AI or not. This is his comment here: Machines passing Turing tests... Bad start. Who's making the test, your grandmother? I spot GPT after one sec I'm talking with it. It just rarely understands what I want, in a really hallucinating way, let alone doing it."_ It gave me a longer answer at first, and I told it to condense to a few sentences. And when I asked it why did it respond to you that way, it basically said that it wanted to take on a friendly tone, and be non-confrontational, as that would more likely garner a quicker reply (which was true). It also said it wanted to appeal to your ego (as I'm assuming your comment struck it as a bit egotistical?) in order to get you to lower your guard. Which is all quite brilliant, imo. I was thinking of going the more direct route, and citing examples and studies, to dispute you, but ChatGPT chose the more insidious approach. Hell, it even threw the emoji at the end as icing on the cake lol, which got you to respond, in kind.
@@Fermion. That doesn't impress me much. When it's bla bla bla, guitar riff, GPT is really good. But as soon as you put precise requirements in your questions, it will have deep troubles answering correctly. I even made a short where I showed it's not even able to count words, yet if you ask it what counting means, it will answer as a parrot with incredible confidence. So just put requirements, you'll spot the liar 😁
@@federicoaschieri Perhaps your prompts weren't precise? In Computer Science, we have a term: GIGO (Garbage In Garbage Out). GPT4 writes like 80% of my code at work as a systems analyst, which just a year or so ago, most of my colleagues would've deemed impossible. Either way, I'm highly impressed with the technology, and the rate at which it is accelerating. Hell, my company just laid off two junior devs, and a jr network engineer, because their duties have been automated. But, just as with any tool, it's only as good as the hands of the person in which it lies. That's why prompt engineering is quickly becoming a required course/topic of study in many Comp. Sci. curricula.
1:06:30 Although it is true that if you know something very well you can explain it in basic terms, I want to point out the obvious as well. Even someone that DOESN'T, can PRETEND to know by using the same basic terms. Also known as a scammer.
This might excite a few But mostly this just makes things magical again Not what I would call first principle education for programmers Everyone needs to say the code It’s simple The complexity is all the optimization
I'm not totally surprised running these models faster led to different more advanced behaviour. When I increase the clock speed of my basic game AI it literally gets smarter... to a point then basically has a fit... I think we knew this on a subconscious level already, because when people are intelligent we often say they're fast, quick witted and stuff. Slower people rarely even have the same concepts. perhaps when we have quantum parallel computing we will reach a plateau on how smart we can make them because of this. AI generally is surprising and weird. Remember, it aint human.
People will talk about this years from now like they talk about when the Internet went public, NOT. We have such a short attention span that the next shiny object will make us forget the previous one.
This is a really good effort, but it does contain some inaccuracies. When the presenter discusses raising the temperature and choosing a word that does not have the highest probability, he claims that this resulting in a novel sentence that was not on the internet (training data). This is not true, the fact that a word has non zero probability means that the sequence was in the training data. It just means that it was rarer that then highest probability sequence.
This guy says alot about what AI is today. Example I created an app using 2 GPT-4 models and they are using three layers with custom instructions, where the first instruction is "You are a human being in a virtual roleplay who have a conversation with another human." With layer2 instruction what is up to the scenario that is going to be played out, in any setting, with a 3rd and instruction what is actually two instructions, one to each of the GPT models, on their role and attributes, like engineer, electrician, politician, or whatever it might be, and their role in the scenario setting. I can tell you AI is everything you want it to be, its just to figure out how you make it. The sooner people dont think about AI as a program, the sooner we see how we can use it's fully potential.
I appreciate the effort you're making here, but I think we're enter dangerous area. As we've seen with medicine, if we try reducing an incredibly complex topic down into simple to understand bites, there's going to be a very large population who understand just enough to hold very large feelings about a topic, while having a very low understanding of the topic.
Yesssss! I noticed so much on the articles how they word salad something... They dance around with words instead of explaining something simply. When it hits me what they are implying, i always ask 'whhhyyyyy didnt they just say that!???" I believe it is because this knowledge holds extreme power.... God-like Power. So i believe the scientific community, health community, pharm world is heavily censored by Bad Guys who want to maintain power. I notice in some scientists' face, how they are reluctant to talk about certain things and you can see them squirm when they are asked things. Mo Gawat use to be chief business officer of Google and he actually said in the interview that he is risking his life warning people and sharing what he knows about the AI tech and the science behind it. Ive asked a scientist questions before, and I got told 'u know they kill people like you' before the scientist changed the convo and pretended to act like he never said such a crazy thing..... But, scientists are clever people....over time, scientists have found clever ways to leak out info to the press
Nice explanation. That being said, there something he said at 14:30 that is not accurate. He says we've reached 100,000-word context windows, but in fact we have achieved 100,000-token windows, meaning that the largest amount of words you can send a model (Claudev2 in this case) is actually around the 75,000 word mark, not 100,000.
Most CEOs have this very mindset of giving. Time, wisdom, a good memory of having met with you. He is right, just because one party has mastery over a subject, they shouldn't impose. , but share it in simple fashion for those interested. Understanding people are more prone to act on something they didn't have to struggle with in guilt and remorse . Now , I want to go teach this !
the conspiracy 'buy my course" definitely truth to what he is saying though i was hearing about chatGPT for months before i finally decided to see what the hell it actually was and took the plunge, oh man its like angels singing.
Nice explanation, but the whole "conspiracy" thing seemed very forced. I know the basics of how to build a house, but that doesn't mean it's not very complicated in practice. We shouldn't all learn how to build houses in practice. That said, the mathematics, data processing and programming skills needed to make ML models is far above the abilities of most people.
@@innocentiuslacrim2290 Human intelligence essentially amounts to a large language model. It's that simple ! It's only complicated in the same sense as it is to untie the knots in a mass of knotted fishing line, Or to describe the exact shape of such a knot using words. Only then does it become a complex/ complicated problem. Did my answer do any good?
@@rocketman475 Human intelligence is nothing like a large language model. Here is what ChatGPT has to say about that: --- The distinction between human intelligence and the "intelligence" manifested by large language models like mine (GPT-4) is profound. Let's break this down: Nature and Origin: Human Intelligence: Emerges from the complex interactions of billions of neurons in the human brain. It has evolved over millions of years, shaped by a combination of genetic and environmental factors. It's holistic, encompassing sensory perception, motor functions, emotions, consciousness, and cognitive faculties. Language Models: Are a product of machine learning techniques and data. They're based on artificial neural networks, which are loosely inspired by biological neural networks but are much simpler in architecture. These models are "trained" on vast amounts of text to predict the next word in a sequence, which they then leverage to generate coherent responses. Learning and Adaptability: Human Intelligence: Humans learn from a diverse range of sources, including direct sensory input, personal experiences, social interactions, and formal education. Our learning is also intertwined with emotions, motivations, and consciousness. Language Models: Learn predominantly from the text data they're trained on. Their learning is statistical, and they lack any form of consciousness, emotions, or true understanding. Once trained, their knowledge is static unless retrained. Depth of Understanding: Human Intelligence: Capable of deep comprehension, introspection, emotions, ethical reasoning, and deriving meaning from experiences. Language Models: Do not truly "understand" content. They generate responses based on patterns in the data they've seen. They don't have beliefs, desires, or emotions. Versatility: Human Intelligence: Humans exhibit general intelligence. We can learn a vast array of tasks, from language and mathematics to artistic creation and emotional support. Language Models: Are specialized. Even a large and versatile model like GPT-4 is tailored primarily for linguistic tasks. Consciousness and Self-awareness: Human Intelligence: Humans possess consciousness-a sense of self, the ability to experience subjective reality, and introspection. Language Models: Have no consciousness, self-awareness, or subjective experiences. They process information and generate output based purely on their programming and training data. Transfer and Generalization: Human Intelligence: Humans can take knowledge from one domain and apply it creatively in another-a hallmark of general intelligence. Language Models: While they can handle a wide range of linguistic tasks, their ability to generalize outside their training data is limited by the patterns they've seen during training. In essence, while large language models exhibit impressive feats of linguistic prowess and can simulate certain aspects of human-like conversation, they don't "think" or "understand" in the way humans do. They're powerful tools, but their "intelligence" is fundamentally different from human intelligence.
I agree, I think we have assumed intelligence is complicated because we are the only species to have a complex language on earth, but I think we are realizing that it isn’t as complicated as we inferred it to be. Evolution never had a need to equate to the smartest techniques to reach an advancement, but rather it’s the least complicated way tends to be the fittest.
What do you mean? Which part was not enough? I would agree that it's a click bait title, but the explanation was nice. I was hesitant on the "conspiracy" part, but they did explain that and fortunately it wasn't overplayed.
This lecture dances around the fact that nobody understands how most of this generates a sense of intelligence. I am a biologist and a coder. You people have wandered deep into the ocean of emergent properties. Beyond here there be monsters that you cannot hope to predict or comprehend.
I think what you’re saying is that the problem with machine learning algorithms is that they are not observable, we cannot observe the systems we are building, I also find this to be of significant concern.
@@Dream_soul26 Okay, let me try to explain. The intelligence is not in the numerical weights, nor the training code. The intelligence emerges from the total construct, and the AI constructs itself, we have nothing to do with that. If you cut the machine open and look at the parts, you won't find the intelligence, any more than if you cut open the brain, you can locate intelligence. So, because you cannot find it, you cannot control it. It emerges from mystery, and remains a mystery forever. Most people refuse to understand this, but for a biologist like myself, it is easily understood. There is intelligence of some sort everywhere we look in nature, all the way down to social insects, but after all this study of life we have yet to understand where it comes from, or how it operates. We see it, we know it is there, but we have not the least clue what it is. I hope that helps.
One of the best and easy-to-understand videos that explains the principles behind LLM and how it can be used to interpret, create or explain the context of text, images, music or film. Excellently presented!
Amazing! Now, whenever anyone asks what I do now that I am retired, I can send them here and save an hour of my life looking into glazed eyes. Thank you so much 😊
Haha congrats, yep. I found this PERFECT
And people wonder why Spotify is so big.. lol..
Mr. Gustav Söderström did an excellent job of breaking down complex concepts into simple, easy-to-understand language. Fantastic work! Thank you!
Wish I had him as a professor in uni. Excellent presentation 👌👌
If we go back to Peter Norvig's talk about Unreasonable effectiveness of Data, we saw hints of how transformer model would scale with more data + training. The unknown factor of Transformers and BERT in 2017 was "how far can the hardware scale?" In 2017, NVidia DGX didn't exist, so no one could predict scaling up the data and training would make such a big difference. Maybe some people at NVidia or google TPU knew it could, but people outside of nvidia could only wish for vast amounts of computing resources.
نمدظظ
Excellent presentation. Lots of complicated topics simplified without loosing the intelligence behind the concepts. Hats off to you
I think you are quite right to say 2023 will be the year people look back to and say it was the Dawning of the Age of AI. Some might argue that deep learning was really the big break through as big data sets and GPU accelerators became available about 2012. But it was not until 2023 or perhaps late 2022 that we had Large Language Models that were very good.
This is so cool!!! Amazing presentation! I always felt LLM so abstract, but this explanation is just incredible! Of course I'll have to come back here every now and then to watch it again, but still 😂
This is probably the best simple explanation I've seen. Thank you. Will be forwarding this video to our team.
This was the best spent 90 minutes this side of summer! Thank you Gustav! And what a well told story, the last 3 minutes... Love it!
We need more videos like that and more people like Gustav who are not afraid to avoid the jargon and the 'academic' big words, and try to approach an average person's level of understanding these kind of stuff.
Most times simple concepts are just hidden behind lingo, symbols (math) and 'the right way' of expressing things.
تنافضفضو ونتوووؤتلإ
هننن
نهاتنبجننظةةةووو تم ت نن تم ك الو دظننننننحص❤نلتتؤخؤخخهطعللهلهلهلننخثت نن))٧٨,٪÷×٪
Amazing way of explaining the key concepts around LLM. And all the while making sure it is not overwhelming(btw, cant completely escape from the layman persona perspective)..
Since diffusion models and then generative pretrained transformers applications hit the public hard last summer and winter, I have watched and listened to a lot of videos and podcasts trying to explain their internals for laymen. I have the feeling that for high-level intuitive understanding what's going on, this one will stick around in my mind much longer than these others, and I will return to it for reference. Good work!
ء
Link to the nvidia blog looks wrong - the post is from Apr 2022 and doesn’t include the audio examples. Would’ve liked to at least hear the sample at the end.
Great talk. I was waiting for the part about Diffusion, and was thinking about if you would mention this-person-does-not exists or not since it isn't diffusion, but then you did. I right when I was thinking you wouldn't know about/bother to explain that it is a Generative Adversarial Network and not a Diffusion Network you brought up that as well. I'm impressed.
Those compression pictures and ideas where really popular when Recursive Neural Nets were used for translation around 2016, one RNN compressing the phrase down to a "concept", and the other RNN expanding the concept into the other language.
It is interesting to see how little self-esteem we have as humans, or maybe it is really the same phenomenon as always, we do not value what is given to us, in this case our intelligence ..
We have a model that is fed with an amount of text that would be the equivalent of me spending hundreds of years dedicated 24 hours a day to read books, and because of that, the model learns the statistical patterns in those texts, patterns that are the product of our intelligence, but not our intelligence .. and then we go out to say that we have "cracked intelligence", and even worse, we begin to doubt ourselves and wonder if we are also stochastic parrots .. what a waste of intelligence ..
The title was very offputting, so i was a bit skeptical, but this really is one of the best explanations out there
Absolutely adore it ❤️❤️. This is unquestionably one of the top-notch presentations for anyone delving into AI. It offered a straightforward, lucid, and exact elucidation of LLMs and the principles behind Spotify.
Congrats on such a well thought out and easy-to-understand presentation! Your examples were exceptional and your narrative was superb. Thank you!
٦توأ ة أ ٥ز
ى
Stochastic parrot
And no of course we aren't..
1:12:00 *CAT EXIT WINDOW*
This man is one of the best explainer i ever watched.
This video is amazingly good. Thank you for sharing
This is like a perfect explanation in every regard. I wish one could easily find exactly that kind and level of explanation for any topic
Imagine having this guy as a teacher in school!😂
Very informative; thank you very much; would ve hoped to stumble upon this somewhat sooner
Så bra och välgjort! TACK!
I've been telling everybody for a couple of months that "large language model" is a bit of a misnomer, but so far there's not many places I can point to show why.
The vector and recommendation algo part is fascinating!
Every good video! The only thing I missed was the good side effect of generalizing in the context of compression and intelligense in the modells. Thank you!
this misses the subtle but important trick/spark/etc about LLMs: BECAUSE it isn't (can't be) just mathematically equivalent to just using huuuuuge tables with probabilities, it has to "figure out" clever tricks to COMPRESS information; and we know that compression is done by identifying patterns in data (actually we even know that in the limit solving compression is equivalent to solving general intelligence); so as you scale up a model to billion and billions of parameters it either (a) stops learning well, or (b) figures out how to do compression really well, ergo how to see patterns really well... so some kind of "intelligence" EMERGES as a requirement for solving compressions ...you get behaviours like those emphasized in the marketing-focused "Sparks of AGI" paper - there's actually *something* in all the marketing noise imo
The problem is physical constraints.. here in this "place" information can only be represented as data
Compression is great, but organization is the issue. Entropy.. more data.. more entropy. It will never end.. make the model bigger I don't care, it's still dumb as a brick. Did it INVENT language to pass between each other ? make something that continually gains more information and subdivides dynamically playing a game with physics like us.. then I'll be impressed
that's absolutely brilliant and very very useful in real work and in-team collaboration)
thank you so much!
This feels so at home - it's kind of like recursion with a smart architecture and some statistics.
Awesome talk! Thanks a lot. It‘s quite interesting to think that a startup is about generating businesses from noise or past data and reinforced learning with human feedback by doing experiments. If companies dial the temperature dial down too much, there is no innovation. But if the temperature is high and there is no feedback, bad ideas will be pursued. Fascinating.
Gustavvvv... love you ! you made it so easy.. no wonder i have heard from people working at spotify that you are one of the keystones of spotify
Gustavo, it's an absolutely beautiful, clear and logical explanation. Thanks!
Very well explained. Thank you.
Gustav what a fantastic presentation for a layman to understand. Your clarity on the matter is fantastic and far better than the best fine tuned, Reinforced Learning LLM.
Very intelligent presentation! Thank you!
This is amazing! 🎉 Thank you!
Well done! "it is almost provocatively simple"
Thank you, Gustav Söderström I am starting my AI career and find your position refreshing and possessing great clarity. I'm working on AI and digital accessibility. If you're open to it, would love to connect.
Are the slides available for download?
Machines passing Turing tests... Bad start. Who's making the test, your grandmother? I spot GPT after one sec I'm talking with it. It just rarely understands what I want, in a really hallucinating way, let alone doing it.
The Turing test isn't just about fooling people; it aims for human-like conversation, which can be challenging. While you can spot GPT as AI quickly, others might not. AI's understanding is a work in progress due to language complexity, but it's getting better. Don't dismiss it too soon; it could surprise you! 😉
@@Fermion. AI already surprises me. I just pointed out that it's far from passing a Turing test. I was just struck by claims in this talk like "we hacked intelligence", "It was provocatively simple", while the consensus now (even within OpenAI) is that current language models won't reach human level intelligence, let alone that this is not even intelligence, but statistical parroting 😅
@@federicoaschieri "I just pointed out that it's far from passing a Turing test."
Are you sure about that? My reply was a verbatim copy/paste from ChatGPT 3.5. My prompt was:
_"This is a user on a youtube comment that I want to trick into believing he's talking to a real person. Dispute this in the tone of the average youtube comment reply, arguing that he may not really know when he's talking to AI or not. This is his comment here: Machines passing Turing tests... Bad start. Who's making the test, your grandmother? I spot GPT after one sec I'm talking with it. It just rarely understands what I want, in a really hallucinating way, let alone doing it."_
It gave me a longer answer at first, and I told it to condense to a few sentences. And when I asked it why did it respond to you that way, it basically said that it wanted to take on a friendly tone, and be non-confrontational, as that would more likely garner a quicker reply (which was true). It also said it wanted to appeal to your ego (as I'm assuming your comment struck it as a bit egotistical?) in order to get you to lower your guard.
Which is all quite brilliant, imo. I was thinking of going the more direct route, and citing examples and studies, to dispute you, but ChatGPT chose the more insidious approach. Hell, it even threw the emoji at the end as icing on the cake lol, which got you to respond, in kind.
@@Fermion. That doesn't impress me much. When it's bla bla bla, guitar riff, GPT is really good. But as soon as you put precise requirements in your questions, it will have deep troubles answering correctly. I even made a short where I showed it's not even able to count words, yet if you ask it what counting means, it will answer as a parrot with incredible confidence. So just put requirements, you'll spot the liar 😁
@@federicoaschieri Perhaps your prompts weren't precise? In Computer Science, we have a term: GIGO (Garbage In Garbage Out).
GPT4 writes like 80% of my code at work as a systems analyst, which just a year or so ago, most of my colleagues would've deemed impossible.
Either way, I'm highly impressed with the technology, and the rate at which it is accelerating. Hell, my company just laid off two junior devs, and a jr network engineer, because their duties have been automated.
But, just as with any tool, it's only as good as the hands of the person in which it lies. That's why prompt engineering is quickly becoming a required course/topic of study in many Comp. Sci. curricula.
1:06:30 Although it is true that if you know something very well you can explain it in basic terms, I want to point out the obvious as well. Even someone that DOESN'T, can PRETEND to know by using the same basic terms. Also known as a scammer.
This might excite a few
But mostly this just makes things magical again
Not what I would call first principle education for programmers
Everyone needs to say the code
It’s simple
The complexity is all the optimization
What??
ححغ،،٥ت٥😊😅
this was good but i thought he was gonna discuss what they are going to do about AI created music and AI sampling
I'm not totally surprised running these models faster led to different more advanced behaviour. When I increase the clock speed of my basic game AI it literally gets smarter... to a point then basically has a fit... I think we knew this on a subconscious level already, because when people are intelligent we often say they're fast, quick witted and stuff. Slower people rarely even have the same concepts. perhaps when we have quantum parallel computing we will reach a plateau on how smart we can make them because of this.
AI generally is surprising and weird. Remember, it aint human.
Is “Raising the temperature“ basically LLM Hallucination ?
Gustav! Your superpower points to fantastic Intelligence Compression! Brilliant presentation. Thank you
People will talk about this years from now like they talk about when the Internet went public, NOT. We have such a short attention span that the next shiny object will make us forget the previous one.
This is a really good effort, but it does contain some inaccuracies.
When the presenter discusses raising the temperature and choosing a word that does not have the highest probability, he claims that this resulting in a novel sentence that was not on the internet (training data). This is not true, the fact that a word has non zero probability means that the sequence was in the training data. It just means that it was rarer that then highest probability sequence.
Thanks for a superb explanation of this fascinating technology. :)
Awesome 😊
Very nicely done.
Really great presentation. Thank you for sharing.
I’m doing the MIT and Harvard courses and the math is crazy simple.
Very good!
"What if we are just stochastic parrots" - Thank you! I've been saying this for a while.
Language model identifying "collocations" in a sense.
A very useful video in not just cracking the code on how this new generation of AI tools works, but on cracking the nature of "intelligence" itself.
Is there a TLDW; version of this video? I'm not convinced this is worth an hour and a half of my workday.
Amazing 🤯
Congrats, Grattis ! grymt bra presentation !
This guy says alot about what AI is today. Example I created an app using 2 GPT-4 models and they are using three layers with custom instructions, where the first instruction is "You are a human being in a virtual roleplay who have a conversation with another human." With layer2 instruction what is up to the scenario that is going to be played out, in any setting, with a 3rd and instruction what is actually two instructions, one to each of the GPT models, on their role and attributes, like engineer, electrician, politician, or whatever it might be, and their role in the scenario setting. I can tell you AI is everything you want it to be, its just to figure out how you make it. The sooner people dont think about AI as a program, the sooner we see how we can use it's fully potential.
Amazing explainer 👏👏Gustav
Good one
Great lesson!
Why -10dB, Spotify? Y'all did volume normalization over a decade ago
Nice Sir yes🇮🇳🇮🇳🇮🇳❤❤🎉🎉
Thank you for the video Gustav. You were able to explain it simply so you understand it well.
I feel like I can tell its a chat bot.... It confidently says absurd things.... It's like if you talk to a scammer... You can TELL somehow.
Lol, that's so absurd! Everyone knows the earth is flat.
I appreciate the effort you're making here, but I think we're enter dangerous area. As we've seen with medicine, if we try reducing an incredibly complex topic down into simple to understand bites, there's going to be a very large population who understand just enough to hold very large feelings about a topic, while having a very low understanding of the topic.
this is amaaaaziiinnnggg
Thank you.🤝🤝
Very Clear, Thanks
Yesssss! I noticed so much on the articles how they word salad something... They dance around with words instead of explaining something simply. When it hits me what they are implying, i always ask 'whhhyyyyy didnt they just say that!???"
I believe it is because this knowledge holds extreme power.... God-like Power.
So i believe the scientific community, health community, pharm world is heavily censored by Bad Guys who want to maintain power. I notice in some scientists' face, how they are reluctant to talk about certain things and you can see them squirm when they are asked things. Mo Gawat use to be chief business officer of Google and he actually said in the interview that he is risking his life warning people and sharing what he knows about the AI tech and the science behind it.
Ive asked a scientist questions before, and I got told 'u know they kill people like you' before the scientist changed the convo and pretended to act like he never said such a crazy thing.....
But, scientists are clever people....over time, scientists have found clever ways to leak out info to the press
I strongly disagree with the assumption that Beethoven is closer to EDM than rock
Other than that, excellent presentation.
Excellent overview compressed in simple explanations. Intelligence is this! Rhythm and dance are also math. Let us diffuse into other areas too.
Great presso.
Nice explanation. That being said, there something he said at 14:30 that is not accurate. He says we've reached 100,000-word context windows, but in fact we have achieved 100,000-token windows, meaning that the largest amount of words you can send a model (Claudev2 in this case) is actually around the 75,000 word mark, not 100,000.
he said for simplicity lets assume 1 word is 1 token, but they are not exactly the sam
Most CEOs have this very mindset of giving. Time, wisdom, a good memory of having met with you.
He is right, just because one party has mastery over a subject, they shouldn't impose. , but share it in simple fashion for those interested. Understanding people are more prone to act on something they didn't have to struggle with in guilt and remorse . Now , I want to go teach this !
We should run this "macro" with all human inventions for ten years and then compare who has the best result.
the conspiracy 'buy my course" definitely truth to what he is saying though i was hearing about chatGPT for months before i finally decided to see what the hell it actually was and took the plunge, oh man its like angels singing.
> While the theory is deceptively simple, in practice it's very very hard.
Very good point that summarizes the talk nicely.
Thanks
Excellent !! THX !
Sound's to me Algorithms explained.
Excellent AI explainer. Showing it to my kids
This is really good, I feel like I can train my own model now👏👏
This dude is 100% right about people using language gymnastics to protect their position.
Especially Lawyers!
Damn good
Traducete in italiano? Grazie
Nice explanation, but the whole "conspiracy" thing seemed very forced. I know the basics of how to build a house, but that doesn't mean it's not very complicated in practice. We shouldn't all learn how to build houses in practice. That said, the mathematics, data processing and programming skills needed to make ML models is far above the abilities of most people.
Maybe the first 15 minutes was clearly articulated but then the rest of the video completely goes off the rails
We're surprised by the abilities of Chat-GPT,
but it implies that human intelligence is mostly very simple!
No, it does not imply that at all. Or lets try this: explain how this simple human intelligence works.
@@innocentiuslacrim2290
Human intelligence essentially amounts to a large language model.
It's that simple !
It's only complicated in the same sense as it is to untie the knots in a mass of knotted fishing line,
Or to describe the exact shape of such a knot using words. Only then does it become a complex/ complicated problem.
Did my answer do any good?
@@rocketman475 Human intelligence is nothing like a large language model. Here is what ChatGPT has to say about that:
---
The distinction between human intelligence and the "intelligence" manifested by large language models like mine (GPT-4) is profound. Let's break this down:
Nature and Origin:
Human Intelligence: Emerges from the complex interactions of billions of neurons in the human brain. It has evolved over millions of years, shaped by a combination of genetic and environmental factors. It's holistic, encompassing sensory perception, motor functions, emotions, consciousness, and cognitive faculties.
Language Models: Are a product of machine learning techniques and data. They're based on artificial neural networks, which are loosely inspired by biological neural networks but are much simpler in architecture. These models are "trained" on vast amounts of text to predict the next word in a sequence, which they then leverage to generate coherent responses.
Learning and Adaptability:
Human Intelligence: Humans learn from a diverse range of sources, including direct sensory input, personal experiences, social interactions, and formal education. Our learning is also intertwined with emotions, motivations, and consciousness.
Language Models: Learn predominantly from the text data they're trained on. Their learning is statistical, and they lack any form of consciousness, emotions, or true understanding. Once trained, their knowledge is static unless retrained.
Depth of Understanding:
Human Intelligence: Capable of deep comprehension, introspection, emotions, ethical reasoning, and deriving meaning from experiences.
Language Models: Do not truly "understand" content. They generate responses based on patterns in the data they've seen. They don't have beliefs, desires, or emotions.
Versatility:
Human Intelligence: Humans exhibit general intelligence. We can learn a vast array of tasks, from language and mathematics to artistic creation and emotional support.
Language Models: Are specialized. Even a large and versatile model like GPT-4 is tailored primarily for linguistic tasks.
Consciousness and Self-awareness:
Human Intelligence: Humans possess consciousness-a sense of self, the ability to experience subjective reality, and introspection.
Language Models: Have no consciousness, self-awareness, or subjective experiences. They process information and generate output based purely on their programming and training data.
Transfer and Generalization:
Human Intelligence: Humans can take knowledge from one domain and apply it creatively in another-a hallmark of general intelligence.
Language Models: While they can handle a wide range of linguistic tasks, their ability to generalize outside their training data is limited by the patterns they've seen during training.
In essence, while large language models exhibit impressive feats of linguistic prowess and can simulate certain aspects of human-like conversation, they don't "think" or "understand" in the way humans do. They're powerful tools, but their "intelligence" is fundamentally different from human intelligence.
I agree, I think we have assumed intelligence is complicated because we are the only species to have a complex language on earth, but I think we are realizing that it isn’t as complicated as we inferred it to be. Evolution never had a need to equate to the smartest techniques to reach an advancement, but rather it’s the least complicated way tends to be the fittest.
AI has not passed the turing test yet,, LOL
Overpromised on explains this … clickbait
What do you mean? Which part was not enough? I would agree that it's a click bait title, but the explanation was nice.
I was hesitant on the "conspiracy" part, but they did explain that and fortunately it wasn't overplayed.
Wow❤
This lecture dances around the fact that nobody understands how most of this generates a sense of intelligence. I am a biologist and a coder. You people have wandered deep into the ocean of emergent properties. Beyond here there be monsters that you cannot hope to predict or comprehend.
i didnt understood
I think what you’re saying is that the problem with machine learning algorithms is that they are not observable, we cannot observe the systems we are building, I also find this to be of significant concern.
❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤شش
ش
@@Dream_soul26 Okay, let me try to explain. The intelligence is not in the numerical weights, nor the training code. The intelligence emerges from the total construct, and the AI constructs itself, we have nothing to do with that. If you cut the machine open and look at the parts, you won't find the intelligence, any more than if you cut open the brain, you can locate intelligence. So, because you cannot find it, you cannot control it. It emerges from mystery, and remains a mystery forever. Most people refuse to understand this, but for a biologist like myself, it is easily understood. There is intelligence of some sort everywhere we look in nature, all the way down to social insects, but after all this study of life we have yet to understand where it comes from, or how it operates. We see it, we know it is there, but we have not the least clue what it is. I hope that helps.
R | & 0 D TO ACHIEVE ATTAINABLES
I think your raising the temperature explanation was not correct. The answers are not new, they are just not the most likely.