How Does AI Actually Work?
ฝัง
- เผยแพร่เมื่อ 19 พ.ค. 2024
- This week, we have a special episode for you! There has been so much talk about AI over the last year or two, but not a lot of explanations. What is AI? What is the difference between AI and Machine Learning? How do they work? David sat down with Danu Mbanga, Director of Generative AI Solutions at Google, to get to the bottom of it all. This talk switches between a general overview of AI and an in-depth discussion about the meaning of intelligence. Danu has years of experience in this field so we hope you learn as much as we did! Enjoy.
00:00 Intro
03:32 What is AI?
12:02 Transformers
22:50 Ad break
22:53 How Regular People Can USe AI
28:55 What is Gen AI?
40:21 What is AGI?
46:32 Ad break
52:18 The Hallucination Problem
01:02:20 Typing Test
Links:
Attention Is All You Need Paper: bit.ly/attentionisallyouneed
IBM k-nearest neighbors: ibm.co/3S6hdtm
Follow Danu Mbanga:
Threads: www.threads.net/@devchiral
X: / dmbanga
Shop the merch:
shop.mkbhd.com
Instagram/Threads/X:
Waveform: / wvfrm
Waveform: www.threads.net/@waveformpodcast
Marques: www.threads.net/@mkbhd
Andrew: www.threads.net/@andrew_manga...
David Imel: www.threads.net/@davidimel
Adam: www.threads.net/@parmesanpapi17
Ellis: / ellisrovin
TikTok:
/ waveformpodcast
Join the Discord:
/ discord
Music by 20syl:
bit.ly/2S53xlC
Waveform is part of the Vox Media Podcast Network. - วิทยาศาสตร์และเทคโนโลยี
The sacred timeline is branching
This is what He Who Remains told us about
Fully 😂👏🏽
if you look closely at him, he looks more like the Watcher
Fr, freaked me out the instant I seen the thumbnail 💀
Internally “is it already Friday?!”
Lol
Love the Loki Reference! 😂
i started panicking thinking that the week is already over and I haven't gotten any work done
NO WONDER I WAS SO CONFUSED.
This has thrown off my weekly podcast schedule
Wait, it's working. I time travelled. It's Friday 🎉
No way we got a bonus episode, it's good day😭
NOO WAY! LETS GO!
🎉🎉🎉
22 minutes in, and I feel I should be getting college credit for this course. Love this one.
Professor David Imel 🤓
David was the right dude to do this video, his commentary is always so sharp and it translates really well to interview questions. I’m grabbing more popcorn and binging this straight through🍿
More than that, he breaks down concepts so easily. I couldn't understand Danu, but David simplified it so anyone can understand it.
I'm glad I got recommended with this gem. Actual good podcast with people discussing things that really matter these days..
As a pattern recognising algorithm I must conclude that it is probably Friday today
People who are as knowledgeable as this guy about a particular field and are able to breakdown complex concepts from their fields into these palatable bits of information for us normies really fascinate me. Great interview, definitely one of my favs!
For a moment, I was confused if today maybe is friday
Danu is a superb guest. He's a gift for explaining complex ideas beautifully. Not surprised he's in a senior position at Google in such a pathbreaking area of computer research. Thanks for a well-organized podcast, David, and for asking Danu these productive questions.
I usually watch the podcast in pieces and finish it right before the next one arrives....but this time I actually finished it on Sunday and had to wait 5 days and I'm so happy right now that they released the bonus episode right when I needed it the most
Haha...thankyou guys 😂
Also, I love that individual members are allowed to host! Such a cool culture in the office.
David is a real one
He really is.
Frfr
This was an episode made only out of passion and curiosity wasn't it?
If I want to know something as deep as possible, I would trust David to explain it to me. Good choice!
The breaking down of the complexities of AI to be more simple and digestible bits was just amazing. Cheers to you guys. A great bonus episode
Love a good bonus episode. And I love learning with David. Maybe he needs your own podcast on the MKBHD network. "David Wants to Know"
Or save it for his own 😉
This episode was worth leaving my first ever comment on TH-cam. I love this type of topic and how in this video, the very complex and technical parts are explained so well. Makes me feel both smart for understanding and like an idiot compared to Danu at the same time.
Danu was awesome. Very knowledgeable in his field. I enjoyed his thought process and his understanding of issues and events. Thumbs up Dan
Bonus episodes led by Studio team members are definitely worthwhile! I will be rewatching this one next week in half term!
amazing interviewer and interviewee. Such a complex and dense topic but you guys made it enjoyable and comprehensible
Such a good episode. I really appreciate the attempts to make the complexities of AI much more simple and digestible. Cheers to the both of them!
Ok, this was excellent. Thank you David for organizing, and Danu for obliging. Loved it, would love to see more stuff like this alongside my fav part of friday - getting home to a new episode of the very enjoyable waveform podcast!
OMG bonus episode!!! You guys just made my day. I literally made a confused face when I saw the notification 😅
Danu is well spoken and covered so much! Thanks for a great episode.
Very cool episode! Congrats to mkbhd for giving space for everyone else in the team to shine. Great work David and overall congratulations to the team for putting great content!
Love these episodes where David gets to do a deep dive and cover a topic like this! Good stuff to all involved on this one :)
I love the way Ellis ask questions. He doesnt just have question, you get an insight into what is going into that question, getting understand a little bit mroe about him as a human.
This was the first interview episode of the podcast that I’ve watched. Love David and Danu here
Omg my brain is just blown. I have to take a break and come back. How amazing! Thank you David. This is great
This is a fantastic overview; thanks David and Danu!
This is the best explanation for AI I listened to.
Like the way you ask the question and how he answer it. Everything is so beautifully weave together.
Looking forward for more content like this. I almost missed this because of the title. Many videos with this title is just a clickbait. But you guys really delivers it 🫰🏻
WHAT!!!! this is too good to be true! Thanks for the extra upload, guys! Still waiting on that old TH-cam features trivia 😂
OMG this like a dream bonus episode, I love you David ❤
The David Deep Dive (DDD) is always great
Midweek Waveform Episode !
It's a blessed week ! 💯
Can't say I understood most of this but I did watch it all the way thru giving the episode the respect it deserved.
This is a Masterclass, Danu is a master.
I get this is mostly a sales pitch for Google AI, but there's some major issues today.
The first big issue is interpretability of the model. Today, we can't interpret the model weights and most of the research shows that we don't have a formal model of how to interpret weights. The second is we know a fraction of the weights contribute to most of the "important stuff" in neural network models. It's common known as "over parameterized" models. We can see this comparing smaller LLama2 models to GPT4. Llama2 get similar performance to GPT4, but with an order of magnitude fewer parameters.
Then there's recent papers that suggests larger models are harder to align, which means scaling transformers up 10 or 100x could be impossible to align. As Jensen Huang has said about data centers, all data centers are power limited. Meaning, data centers don't have enough power to handle 2x the TPU or GPU. This is why microsoft and google are building new data centers. It's also why Tesla is building their own data center ie DOJO.
Scaling these models to handle 100 million concurrent users isn't trivial. OpenAI claims to handle 10 million users today and it's constantly having performance issues. Some people have tried to estimate the cost to handle a single chatGPT request. Depending on how to calculate it, it's any where from 2-4 cents per request. In contrast, classic google search is less than 0.1 cents/request. If you look at the latency of chatGPT vs classic google search, it's seconds versus 0.01 milliseconds. That's 1000x slower to run a chatGPT request.
thank you for having these kind of special episodes esp about AI, aside from the weekly show of course. keep it up and looking forward for mmore
David Awesome Podcast with so much great information and has to be one of my favorite episodes with the person on this. I hope we get to see much more like this.Very interesting and much enjoyed.✌️💯
Really hope this 2 episodes per week becomes a norm!!
David is my favorite member of the podcast.
Wow! That was quite something! Danu was a great, and fascinating guest, and David was excellent as host, with some very intelligent questions... The deep thought processes behind AI and Deep Learning is incredible!
Brilliant bonus episode guys; thanks very much. 😃 👍
A Tuesday bonus episode was just what I needed today
Up for David leading the podcast anyday.
Wow, absolutely brilliant discussion!! Love these more technical deep dives. Danu is incredible and David, fantastic interview. Thank you for bringing this to us!
That was super interesting! I love what he said at the end about how one of the practical benefits of AI is that it lowers the barrier of entry for people to be able to implement their ideas. I think this is a much better way to frame how we will use AI, rather than people who might say it promotes laziness or not having to learn things for yourself. I think you still need to have your own knowledge and skillset to generate an idea, but there can be so many other things involved in bringing that idea into reality, and we can't possibly become experts in everything in our lifetime. So that's where you can use AI to fill in certain gaps in your knowledge to some extent to allow you to implement your idea. Could be something like someone who is an awesome baker, but they aren't too business savvy. AI doesn't replace that person being good at baking, but it could help to ease the complications around running a bakery.
Great episode. Really valuable information here. David is great at hosting this. Went straight ahead to check Tom Scott and his typing right after watching this :P
This so probably one of the coolest episodes to date! Also I could listen to him talk for days
Refreshing my feed and seeing this just as I got my food delivered is a sign that good things still happen.
Nice! The slides on the info being talked about is great. Like a learning podcast class lol
So cool that you guys actually did this interview
Yes! Thank you Ellis, that was my question as well. So glad you asked it for me
David always tackles the real questions and im here for it!
Interesting that a person from Google, talking about Google was sitting in front of a Mac book. However, putting my kindergartenesk sense of the whimsy aside. This was a brilliant interview. And one I am going to have to re-watch a few times in order to brain soak all the information delivered. Many thanks David.
Danu does an amzing job explaining this stuff. Bravo
This was one of my favorite bonus episodes.❤ Great video. 👌
David, that was awesome. I have to say I was confused why you joined the team based on your complete differences However, I think you're probably the best edition to MKBHD just for that alone. I think that your team keeping egos in check could become an actual network of force in our industry well in your industry, but definitely for everyone involved. Awesome job that was awesome
Love me some bonus David. Well done, loved this episode.
Let's gooo!
I'd love extra episodes on Tuesdays!
This was an awesome watch.
Learning thing i had no idea i needed.
P.S This dude sounds too inteligent.
This is just an amazing piece of content which I would pay for.
Great episode, we need a part 2 with this guy !
Incredible interview, Danu Mbanga is so intelligent, and knows how to explain things in simple ways… we need more people like him talking about deep tech topics
dang this was unexpected! Loved the podcast david and what great timing :) it was a surprise. Gonna get some coke and popcorn to watch this episode now!!
I would love to see more of these deeper dives into the technology we use from this podcast!
I’m a big fan of the current WVFRM podcast format but would like less of a list of consumer tech new and more of a dive into the technology.
Maybe another podcast?
such an interesting discussion, David u did a great job with the questions and Danu is such an expert!!
Such an interesting video!! awesome to see david doing this episode. You asked just the right questions!
Very good episode full of information
I even watched it twice
Thanks David
This is so good on so many levels. Great job!
Please keep this guy on regular rotation
Really good episode. Very informative and learned quite a bit.
It was interesting to hear the perspective of someone who fully works and believes in AI. There's a lot that I don't agree with and the absence of awareness on how generative AI is currently hurting traditional media creators (artists especially) feels concerning. Thanks for the interview, Danu and David.
Yes we need a part 2 David
so cool you were really going into depth of what's AI on a platform that can reach the mainstream. hope that more people will understand AI better and not just see as this random magic box - and I loved Danu's positive outlook on how AI will transform the economy. I only missed the discussion on what are the social implications those models can have - eg. how implicit bias is trained in the models and how this problem is addressed by developers. But I enjoyed listening to this episode a lot, thanks for this extra bit!
Ayyy extra episode 🔥
we need more in depths like that!!!
Heck yeah! Bonus episode!!
This was just awesome. Sometimes life is uninspiring and then you come across pple like this and think ok maybe humans are something special. Awesome.
MY WEEK HAS BEEN MADE! 😭
Will I refresh my feed every Tuesday seeing if we get a bonus episode?
Yes? Maybe?
No, definitely.
Not every Tuesday, but we do have a couple extra episodes planned before end of year
This is my second "deep dive" into the "Ai,ML,deeplearning" stuff... I've learnt a bit more about it here((i did not know that the scalability of machinelearning was an unexpected phenomenon))...
Was not expecting this! It’s a good day today😃
Yeayyyyy more content! David you’ve made my day
Can't believe we have two Fridays this week
Such a good insight, thank you Danu, David, and the team :)
Damn i thought it was Friday today! Super pumped for this one
One of the best podcasts to date
At around 20mins Danu mentions that the fact that the transformer could perform a derivation / write code / etc were it having emergent properties because the transformer wasn't trained to perform those specific tasks.
But in the context of a transformer being trained to chain together the most likely strings of tokens & producing a string of tokens to do with each of those topics, you would expect that the strings should be accurate based on training data.
So it would seem to not necessarily be an emergent property in the same capacity as something that was trained to get colour that could also provide shape. But then I guess you could argue that it's emergent in the context of not expecting it to be able to perform those activities purely based on the likelihood ratios of strings of token? 🤔
Can't believe I just saw this 😭😭😭 Thanks Waveform Team 💙
I think it's more helpful to define Machine Learning not just as the "math behind AI" but in terms of the high-level problems it is invoked to solve: classification, clustering, regression. This makes the distinction between AI, ML, and DL even more clear, at least for me 😅
Was not emotionally prepared for a surprise drop! Thanks!
the break down of AI>Machine Learning> Deep Learning was fantastic
This was a great podcast. My only personal criticism of it is, I feel like their was a missed opportunity to correct misconceptions on understanding generative ai when taking about art and how it's been learning art from the internet. It's it how we think it works and learning from other people art. Should we be or not be thinking of it is stealing from other artists. Just questions around that line of thinking. Overall great and informative and more importantly easy to understand. Def will be sharing this to friends.
I think you can mainly say that a model has emergent capabilities, because it has not been trained to have those capabilities. Many text models are trained to predict a token (~= word) in a sentence given the tokens that that appear before that one. Despite that simple instruction they learn to reason.
oh boy I'm gonna have to sit down and focus for this one!
I could never say no to a bonus/surprise episode
Love this. Wanted to learn about generative AI.