I really enjoyed this conversation with John. Here's the outline: 0:00 - Introduction 2:35 - Difference between biological and artificial neural networks 8:49 - Adaptation 13:45 - Physics view of the mind 23:03 - Hopfield networks and associative memory 35:22 - Boltzmann machines 37:29 - Learning 39:53 - Consciousness 48:45 - Attractor networks and dynamical systems 53:14 - How do we build intelligent systems? 57:11 - Deep thinking as the way to arrive at breakthroughs 59:12 - Brain-computer interfaces 1:06:10 - Mortality 1:08:12 - Meaning of life
I once called Dr. Hopfield uninvited as an undergrad at Oklahoma State University. Dr. Hopfield was at CalTech at the time I think. Dr. Hopfied was very gracious and talked to me for a good 30 minutes explaining his work carefully, focusing on an aspect of his papers that was not making sense to me at the time. I am a life long fan.
Listening to these conversations gives me a deep ache in my chest, and brings tears to my eyes. The purity of yearning for the knowledge of the yearning for knowledge is such a beautiful thing.
@@Nintendo_fanboy87 nah, idk what I was talking about haha. This happens every so often, I get a reply to an old comment I made and I have no clue why tf I made it lol
I listened this many times, pausing, going back and forth and it always amazes me how Lex is so prompt in asking the best questions and follow the responses so wisely. HATS OFF to both of you.
Wonderful to listen to John Hopfield. I had implemented his Hopfield network on silicon in 1988 🙂. I moved to other things but kept a deep interest in brain biology and artificial intelligence.
Lex, I can't thank you enough nor tell you how amazingly educational and useful your podcast has become in my life. Thank you for everything you do, have done and will do. This is a field in which I'm truly passionate about and the information in this conversation is priceless.
Beautiful interview. I’m a neuroscientist neuroethologist and yesterday I was teaching Hopfield landscapes to my psychology and medicine students… next semester I’ll include some of his appreciations from your conversations… for me personally, it gives me motivation to pursue that Postdoc; I’ll be defending my PhD thesis within a month on multimodal integration and neuromodulation in bees
.... you are such a lucky man to meet all these people and have such long, deep conversations with them. (Yes, I know, you are working super hard to get 'so lucky'...!) - You are so young and when you keep on doing this for the next decades you will turn into a legend yourself... (Silly me thought 'Lex-Icon of AI' ...)
Took me a while to get into this one, but again very good indeed. I like the quote from Marvin Minski that "consciousness is basically overrated.... all the hard computations are done non-consciously" and "consciousness is your effort to explain to yourself what you have already done" This is very topical because I just was listening to an analyst saying Tesla would not get full autonomy because AI cannot do the sort reasoning we do - I think this is completely wrong - we drive our cars unconsciously most of the time.
Thank you so much Lex. I had a lot of these thoughts as a high school seniors. Now I’m 69. Just got me all excited. And I’m on lesson two of deep learning Ai Andrew Ng class thanks to you.
99% of my brain: a great conversation, great to hear from the man himself and grateful to lex for the content 1% of my brain: this dude's tongue has a mind of its own
Lex! Watched you a couple of times at Roagans channel! After you expressed your intent behind building your business I got interested in your work. I am now mesmerised by the quality of questions you directed to John. Inspiring, intriguing and humbling. I will be back for sure. Keep it up brother
I dont ever feel like commenting as I enjoy your podcasts and that's all I'm here for. But your podcasts are special. We've built an economy of content but not enough context. It's important that these interviews have highlighted not only the search for purpose but their ability to be actionable about it. I hope these podcasts inspire the general public one day.
Enjoyed this interview. The question "What does it mean to understand something?" which Hopfield asks multiple times during this interview is my key takeaway. Thanks, Lex.
As someone else wrote, wonderful indeed to listen to J. Hopfield. But I was most impressed with Lex's talent as an "interviewer". While J.H. is definately still "all there" in terms of his intellect, etc (my opinion) ... many "mature" scientists provide explanations that leave a fair amount of "white space", perhaps b/c the nuances of their ideas & discoveries start to seem trivial (to them) over time. Lex brilliantly keeps the flow in this conversation with a great scientist/human by steering the conversation in the subtle manner that befits speaking with a brilliant mind that prefers to take longer "strides" with respect to sharing his insights & knowledge. Simply put, Lex prompts JH to connect certain dots but allows for a certain amount of "drift" ... perhaps out of respect, deference, love. That talent (on the part of Lex) is uplifting (and effective). Bottom line ... Lex has orchestrated a wonderful journey into a great mind & person ... yet again. Thx Lex
This feels like an important interview doesn't it? I particularly enjoyed his take on neural interfacing. He used so many wonderful metaphors that actually helped redirect my imagination and this... is about as good as it gets for sharing with humanity. Thanks Lex
The "mystery of consciousness" was discussed, albeit indirectly, by Ludwig Wittgenstein in his posthumously published book Philosophical Investigations. I spent many hours, night after night, when I was 17 years old, reading that book line by line. The book is very difficult to understand, partly because Wittgenstein's language was German, which I do not speak or read, and also because Wittgenstein's manner of expressing his ideas was odd and seemingly archaic. Wittgenstein did not discuss consciousness directly in anything I have read, but the content of his work bears strongly on the present discussion. His attitude toward the question, "What is consciousness?" might have been that there is no point discussing something we cannot even define. If you describe a world In which human beings do everything of which we are capable, without mentioning consciousness, what would be lost or incorrect as a result? And the only answer would seem to be that nothing was lost.
There seemed to be a grinding of gears going on for these two not far under the surface. Lex seemed impatient at times. Not like I’ve seen him in other interviews. Perhaps the difficulties JH had getting some of the words out made the pace difficult for Lex. The analogies that JH was drawing were well worth the wait - utterly brilliant guy. He seemed able to express the gist of a lifetime of insights on how to think clearly about nature and science into short pithy comparisons. JH joked at the end that Lex might have difficulty getting 5 minutes of coherent conversation out of the recording. It all seemed very sensical. Abstract at times but deeply insightful.
Excellent, thank you Lex for interviewing Hopfield. Many insights on NLD and its omissions in the ANN world. I wonder if Hopfield has read “Beyond Boundaries” and the research work of Miguel Nicolelis. It sounded like he hadn’t and that you haven’t. Nicolelis has done some beautiful work on measuring collective phase locking behavior of actual networks of various sizes. I realize his research is controversial, since the probes are invasive and the test subjects are laboratory animals. There is no doubt in my mind that the neuroscience world would benefit from improved diagnostic test methods, non-invasive if possible. FMRI is too qualitative and disconnected with the actual NLD.
Incredible insights on neural dynamics and how understanding of dynamical systems can give new ideas to improve artificial neural networks. It's good to learn a bit about "neural coding" and "chaotic attractors" before listening.
Please, please interview Barbara Arrowsmith-Young who wrote ‘the woman who changed her brain’. Also a conversation between Barbara Arrowsmith -Young and John Hopfield would be fascinating and enlightening. They could perhaps lead you to your starting point in your new AI project…
Now that I have been listening to your podcasts, the chats are serving as a brain mechanism, the propeller of my discourse, a response mechanism of these words, the other day, last year!, just came to my mind that people are like rocks. In the sense that brains, neurology people act always the same, they have to be tought, trained, and seems they never learn well, we came back everyday to do the same, even though it seems really different for all the days, years in our lives. Analyzing this past sentence: this has some of Descartes rationalism, blank slate, maybe that's where the idea comes from: people are like rocks/slates, as I am introspecting, past knowledge, to understand where I would like to go. Knowing my human experience, existensialism or meta-existence has limits. My learned knowledge in phisics is null, which is impossible, but true is that I have no conscience of phisics in my life, I cook, I go shopping, on the bus, train, subway, but phisics never crosses my mind. I have to get some fundamental book in phisics, I forgot everything from high school phisics! Thank You both 😀
well, this HTM theory didn't pick up. Same as it didn't pick up for Imagination Engines with its STANNOs developed by Steven Thaler. Makes me think all these folks who "reinvent the wheel" are just marketing gradient descent or similar another well known techniques under their own name, and get banned by AI community. One vivid example of such a ban is "Extreme Machine Learning" which is nothing more than random projection and a linear regression on top of it.
Very profound discussion. Hopfields observation that he cannot yet find a smoking gun that links physical systems to consciousness rings true to me - how do electronic signals in the form of action potentials make a leap into a sense of “I”?
What? No white shirt? I barely recognized you. I'm looking forward to an experimenter's explanation of how to implement emergent properties in computer simulations of neurobiology. Excellent podcast as always.
I infer that if a living system is a seemingly messy coalescence of physics, biology, chemistry, computation and what not, any semblance of true artificial general intelligence may well require an unified application of natural general intelligence.
My mind was kinda blow when he said consciousness was just for observing thoughts you all ready had Makes it seem even more useless like is just a hub to connect shit together? Zoom in zoom out Zoom in zoom out
Of various intelligence, what's it like? For me I feel at times as a passenger mediating thoughts. So that leads me to feel this consciousness thing is fleetingly thin, but still enough to obviously affect the outcome by sheer focusing of will or repetition. Are we receptacles in perhapd a biological simulation with determined randomness or do we have a spark of something else, a "curious recursion" as Ramachandran postulated long ago..?
@@Japakak i'm fascinated this days with "The Deep History of Ourselves: The Four-Billion-Year Story of How We Got Conscious Brains" by Joseph LeDoux, on how consciousness may have evolved, and on what consciousness is from a completely rational point of view. i'm a complete layman on the subject, but the book is amazing.
llms are recurrent - each token generated is fed back to the network to generate the next, so the wisdom of the network is in a sense being unpacked. Recurrence is a kind of compression.
18:12 ... The rhythms are utterly absent from anything that goes on in Google... - Yeah but the rhythms... - But the rhythms WHAT? ------------------------------- When you know you've been SCHOOLED
I really enjoyed this conversation with John. Here's the outline:
0:00 - Introduction
2:35 - Difference between biological and artificial neural networks
8:49 - Adaptation
13:45 - Physics view of the mind
23:03 - Hopfield networks and associative memory
35:22 - Boltzmann machines
37:29 - Learning
39:53 - Consciousness
48:45 - Attractor networks and dynamical systems
53:14 - How do we build intelligent systems?
57:11 - Deep thinking as the way to arrive at breakthroughs
59:12 - Brain-computer interfaces
1:06:10 - Mortality
1:08:12 - Meaning of life
is there a place to get a transcript of the conversation?
Bring David Deutsch please! :)
Lex, could you add subtitles for the hearing impaired people?
Lex Fridman to Interview this man is a great honor. I have never seen someone describe the loneliest place so perfect!
@@wizardstein3153 www.linkedin.com/pulse/transcript-interview-john-hopfield-lex-fridman-alfonso-r-reyes/?published=t
github.com/f0nzie/transcript_interview_john_hopfield_by_lex_fridman
Congratulations for his Nobel prize in physics
I once called Dr. Hopfield uninvited as an undergrad at Oklahoma State University. Dr. Hopfield was at CalTech at the time I think. Dr. Hopfied was very gracious and talked to me for a good 30 minutes explaining his work carefully, focusing on an aspect of his papers that was not making sense to me at the time. I am a life long fan.
And no wonder , it must have been super inspiring .
And now he is Nobel laureate
This man deserves a noble prize!
Listening to these conversations gives me a deep ache in my chest, and brings tears to my eyes. The purity of yearning for the knowledge of the yearning for knowledge is such a beautiful thing.
True. People around me think I am weird for listening to these videos.
Nerd
@@Adam-st8ys haha, yeah
@@eddieharding6788 do you know him/her personally?
@@Nintendo_fanboy87 nah, idk what I was talking about haha. This happens every so often, I get a reply to an old comment I made and I have no clue why tf I made it lol
I listened this many times, pausing, going back and forth and it always amazes me how Lex is so prompt in asking the best questions and follow the responses so wisely. HATS OFF to both of you.
Wonderful to listen to John Hopfield. I had implemented his Hopfield network on silicon in 1988 🙂. I moved to other things but kept a deep interest in brain biology and artificial intelligence.
Lex, I can't thank you enough nor tell you how amazingly educational and useful your podcast has become in my life. Thank you for everything you do, have done and will do. This is a field in which I'm truly passionate about and the information in this conversation is priceless.
17.37 "but the rhythms what??" Damn Hopfield goes #savage on Lex... what amazing insights ... one of the best podcasts. Thanks!
Lex’s laugh when Dr. Hopfield responded with “there’s a question of what do you mean by understand” :’)
Beautiful interview. I’m a neuroscientist neuroethologist and yesterday I was teaching Hopfield landscapes to my psychology and medicine students… next semester I’ll include some of his appreciations from your conversations… for me personally, it gives me motivation to pursue that Postdoc; I’ll be defending my PhD thesis within a month on multimodal integration and neuromodulation in bees
.... you are such a lucky man to meet all these people and have such long, deep conversations with them. (Yes, I know, you are working super hard to get 'so lucky'...!) - You are so young and when you keep on doing this for the next decades you will turn into a legend yourself... (Silly me thought 'Lex-Icon of AI' ...)
Indeed.
Took me a while to get into this one, but again very good indeed. I like the quote from Marvin Minski that "consciousness is basically overrated.... all the hard computations are done non-consciously" and "consciousness is your effort to explain to yourself what you have already done" This is very topical because I just was listening to an analyst saying Tesla would not get full autonomy because AI cannot do the sort reasoning we do - I think this is completely wrong - we drive our cars unconsciously most of the time.
He won Nobel prize in Physics today
Thank you so much Lex. I had a lot of these thoughts as a high school seniors. Now I’m 69. Just got me all excited. And I’m on lesson two of deep learning Ai Andrew Ng class thanks to you.
now he got nobel in physics, congrats
He won nobel prize 🏆
This guy just won a nobel prize in physics
He has just won the Nobel Prize!
At this specific date - november 14, 2020 - I predict that John Hopfield will one day win the Nobel Prize in physics.
99% of my brain: a great conversation, great to hear from the man himself and grateful to lex for the content
1% of my brain: this dude's tongue has a mind of its own
Lex! Watched you a couple of times at Roagans channel! After you expressed your intent behind building your business I got interested in your work.
I am now mesmerised by the quality of questions you directed to John. Inspiring, intriguing and humbling. I will be back for sure. Keep it up brother
I dont ever feel like commenting as I enjoy your podcasts and that's all I'm here for.
But your podcasts are special.
We've built an economy of content but not enough context. It's important that these interviews have highlighted not only the search for purpose but their ability to be actionable about it.
I hope these podcasts inspire the general public one day.
Congrats for the Nobel Prize
This is very much amongst my favorite discussions you've had. So much honest wisdom.
Mr. Hopfield has a beautiful and elegant mind. This was such a pleasure to watch. Thank you Lex.
Enjoyed this interview. The question "What does it mean to understand something?" which Hopfield asks multiple times during this interview is my key takeaway. Thanks, Lex.
Maybe it's just me but I'm enormously jealous Lex. These guests are incredible.
As someone else wrote, wonderful indeed to listen to J. Hopfield. But I was most impressed with Lex's talent as an "interviewer". While J.H. is definately still "all there" in terms of his intellect, etc (my opinion) ... many "mature" scientists provide explanations that leave a fair amount of "white space", perhaps b/c the nuances of their ideas & discoveries start to seem trivial (to them) over time. Lex brilliantly keeps the flow in this conversation with a great scientist/human by steering the conversation in the subtle manner that befits speaking with a brilliant mind that prefers to take longer "strides" with respect to sharing his insights & knowledge. Simply put, Lex prompts JH to connect certain dots but allows for a certain amount of "drift" ... perhaps out of respect, deference, love. That talent (on the part of Lex) is uplifting (and effective). Bottom line ... Lex has orchestrated a wonderful journey into a great mind & person ... yet again. Thx Lex
This feels like an important interview doesn't it? I particularly enjoyed his take on neural interfacing. He used so many wonderful metaphors that actually helped redirect my imagination and this... is about as good as it gets for sharing with humanity. Thanks Lex
This was a lovely conversation. JH's analogy about the adaptive speedometer was so simple and so profound.
I hope I still have my whits about me when I'm 87 like this guy!
Watching after he awarded with nobel prize for ai ml♥️
Hello Mr. Nobel Prize Winner! 🙌
Single best channel on yt I'd recommend to everyone in the world right here. Your work is amazing Lex.
The "mystery of consciousness" was discussed, albeit indirectly, by Ludwig Wittgenstein in his posthumously published book Philosophical Investigations. I spent many hours, night after night, when I was 17 years old, reading that book line by line. The book is very difficult to understand, partly because Wittgenstein's language was German, which I do not speak or read, and also because Wittgenstein's manner of expressing his ideas was odd and seemingly archaic.
Wittgenstein did not discuss consciousness directly in anything I have read, but the content of his work bears strongly on the present discussion. His attitude toward the question, "What is consciousness?" might have been that there is no point discussing something we cannot even define. If you describe a world In which human beings do everything of which we are capable, without mentioning consciousness, what would be lost or incorrect as a result? And the only answer would seem to be that nothing was lost.
Great guy. Please interview John Hopfield again.
Amazed at this Hopfield interview, If Hinton isn't returning your calls you should ask Mike Spivey.
There seemed to be a grinding of gears going on for these two not far under the surface. Lex seemed impatient at times. Not like I’ve seen him in other interviews. Perhaps the difficulties JH had getting some of the words out made the pace difficult for Lex. The analogies that JH was drawing were well worth the wait - utterly brilliant guy. He seemed able to express the gist of a lifetime of insights on how to think clearly about nature and science into short pithy comparisons. JH joked at the end that Lex might have difficulty getting 5 minutes of coherent conversation out of the recording. It all seemed very sensical. Abstract at times but deeply insightful.
Lex "Let's linger a bit on that" Friedman
Lex "Beautiful" Friedman
*Fridman
Today Geoffrey Hinton and John Hopfield got Nobel price for their contribution in AI
I really liked this deep slo-mo interview! Gives thoughts room for wandering... Thanks Lex!
Who's here after the Nobel prize!
Me
Excellent, thank you Lex for interviewing Hopfield. Many insights on NLD and its omissions in the ANN world. I wonder if Hopfield has read “Beyond Boundaries” and the research work of Miguel Nicolelis. It sounded like he hadn’t and that you haven’t. Nicolelis has done some beautiful work on measuring collective phase locking behavior of actual networks of various sizes. I realize his research is controversial, since the probes are invasive and the test subjects are laboratory animals. There is no doubt in my mind that the neuroscience world would benefit from improved diagnostic test methods, non-invasive if possible. FMRI is too qualitative and disconnected with the actual NLD.
Thanks for making these kinds of videos. Fascinating stuff.
Another killer podcast! Thanks.
Incredible insights on neural dynamics and how understanding of dynamical systems can give new ideas to improve artificial neural networks. It's good to learn a bit about "neural coding" and "chaotic attractors" before listening.
I really love how you bring these diverse brilliant conversation to everyday people!
Thank you Hopfield, it was all a coherent sense to me throughout the video.
The best of his interviews I’ve seen
coming from the nobel prize 2024
This was a wonderful interview - Hopfield is a great thinker/scientist and Lex, you asked some really good questions!
Some good analogies used in this discussion. Interesting to hear how models to approximate how the brain works are evolving.
Congratulations on the Nobel prize in physics 👏
Who is watching that after he got nobel prize
Thank you both.
I've great respect for Hopfield.
Lex you are doing very good job you can be proud of it and also very lucky. Keep doing!
You ask good questions. Thanks for your podcast.
I'm a simple man, I see John I click like.
The rhythms... Look at central pattern generator circuits in the spine and how they solve the motor control problem...
Thank you man
My favorite episode so far
Lex you are on fire 🔥 with these podcasts👌
Thank you so much 🙏
Please, please interview Barbara Arrowsmith-Young who wrote ‘the woman who changed her brain’. Also a conversation between Barbara Arrowsmith -Young and John Hopfield would be fascinating and enlightening. They could perhaps lead you to your starting point in your new AI project…
Another great interview. Thank you for a job well done.
Now days lex don't bring guest like this❤
New favorite TH-camr.
Very nice work. Could you please also interview synthetic biologists?
congrats on the nobel
What a podcast, thank you Lex!
Great episode! Please have Alan Kay on!
such an inspiring talk .. kudos to both of you.
Now that I have been listening to your podcasts, the chats are serving as a brain mechanism, the propeller of my discourse, a response mechanism of these words, the other day, last year!, just came to my mind that people are like rocks. In the sense that brains, neurology people act always the same, they have to be tought, trained, and seems they never learn well, we came back everyday to do the same, even though it seems really different for all the days, years in our lives. Analyzing this past sentence: this has some of Descartes rationalism, blank slate, maybe that's where the idea comes from: people are like rocks/slates, as I am introspecting, past knowledge, to understand where I would like to go. Knowing my human experience, existensialism or meta-existence has limits.
My learned knowledge in phisics is null, which is impossible, but true is that I have no conscience of phisics in my life, I cook, I go shopping, on the bus, train, subway, but phisics never crosses my mind.
I have to get some fundamental book in phisics, I forgot everything from high school phisics!
Thank You both 😀
I love your podcast Lex. I’m a huge fan. Your conversations are just amazing. Keep up the great work.
I still have quite a bit of hope for approaches like Numenta’s HTM theory.
well, this HTM theory didn't pick up. Same as it didn't pick up for Imagination Engines with its STANNOs developed by Steven Thaler. Makes me think all these folks who "reinvent the wheel" are just marketing gradient descent or similar another well known techniques under their own name, and get banned by AI community. One vivid example of such a ban is "Extreme Machine Learning" which is nothing more than random projection and a linear regression on top of it.
Absolute Zero I don’t think you understand how HTM works. No gradient descent at all. Go learn about instead of armchair criticizing.
loved this podcast. its not everyday Vision from MCU comes to explain how human brain works
very poetic intro
Great work Lex, thank you 🙏🏻
Absolutely amazing interview, thank you
Very profound discussion. Hopfields observation that he cannot yet find a smoking gun that links physical systems to consciousness rings true to me - how do electronic signals in the form of action potentials make a leap into a sense of “I”?
one of the most overdue Nobel prizes in physics (the other overdue one being Alain Aspect's)
Brilliant interview. You should have Professor Leroy Hulsey of UAF on at some point.
27:27 Great conversation. . Might this with hindsight be alluding to generative processes .
This should be a good one 🙏🌿💚
Wish I could upvote 10x
Well done both, thnx
1.5 playback speed is the best for this video
What? No white shirt? I barely recognized you. I'm looking forward to an experimenter's explanation of how to implement emergent properties in computer simulations of neurobiology. Excellent podcast as always.
An amazing man! I wonder what he thinks of hierarchical temporal networks
Thanks for it
Thank You
Loved this!
I infer that if a living system is a seemingly messy coalescence of physics, biology, chemistry, computation and what not, any semblance of true artificial general intelligence may well require an unified application of natural general intelligence.
This is a Noble Prize 🏆🏆🏆 Winner? What causes issues like that ?
That was very interesting
My mind was kinda blow when he said consciousness was just for observing thoughts you all ready had
Makes it seem even more useless like is just a hub to connect shit together? Zoom in zoom out
Zoom in zoom out
Of various intelligence, what's it like? For me I feel at times as a passenger mediating thoughts. So that leads me to feel this consciousness thing is fleetingly thin, but still enough to obviously affect the outcome by sheer focusing of will or repetition. Are we receptacles in perhapd a biological simulation with determined randomness or do we have a spark of something else, a "curious recursion" as Ramachandran postulated long ago..?
If you would like to know more, try Marvin Minsky - The Emotion Machine
@@Japakak i'm fascinated this days with "The Deep History of Ourselves: The Four-Billion-Year Story of How We Got Conscious Brains" by Joseph LeDoux, on how consciousness may have evolved, and on what consciousness is from a completely rational point of view. i'm a complete layman on the subject, but the book is amazing.
@@juanaq It looks really interesting, thanks.
Not useless at all: consciousness might be the top-level feedback loop of the brain?
So excellent! Thanks lex
llms are recurrent - each token generated is fed back to the network to generate the next, so the wisdom of the network is in a sense being unpacked. Recurrence is a kind of compression.
It's the first time I've seen you, Lex, obviously bored (and I've watched most of your videos, I'm a regular watcher).
Very nice
18:12 ... The rhythms are utterly absent from anything that goes on in Google...
- Yeah but the rhythms...
- But the rhythms WHAT?
-------------------------------
When you know you've been SCHOOLED