Outline 01:37 Why Barbie Is Better Than Oppenheimer 09:35 The relationship between nuclear weapons and AI x-risk 13:31 Global warming and the limits to growth 21:04 Joscha’s reaction to the AI Political compass memes 24:33 On Uploads, Identity and Death 33:46 The Endgame: Playing The Longest Possible Game Given A Superposition Of Futures 38:11 On the evidence of delaying technology leading to better outcomes 41:29 Humanity is in locust mode 44:51 Scenarios in which Joscha would delay AI 48:44 On the dangers of AI regulation 56:14 From longtermist doomer who thinks AGI is good to 6x6 political compass 01:00:48 Joscha believes in god in the same sense as he believes in personal selves 01:06:25 The transition from cyanobacterium to photosynthesis as an allegory for technological revolutions 01:18:26 What Joscha would do as Aragorn in Middle-Earth 01:26:00 The endgame of brain computer interfaces is to liberate our minds and embody thinking molecules 01:29:30 Transcending politics and aligning humanity 01:36:33 On the feasibility of starting an AGI lab in 2023 01:43:59 Why green teaming is necessary for ethics 02:00:07 Joscha's Response to Connor Leahy on "if you don't do that, you die Joscha. You die" 02:08:34 Aligning with the agent playing the longest game 02:16:19 Joscha’s response to Connor on morality 02:19:46 Caring about mindchildren and actual children equally 02:21:34 On finding the function that generates human values 02:29:34 Twitter And Reddit Questions: Joscha’s AGI timelines and p(doom) 02:35:56 Why European AI regulations are bad for AI research 02:38:53 What regulation would Joscha Bach pass as president of the US 02:40:56 Is Open Source still beneficial today? 02:43:06 How to make sure that AI loves humanity 02:48:22 The movie Joscha would want to live in 02:50:46 Closing message for the audience
Hearing a highly intelligent German intellectual deliver a deep treatise on the Barbie movie, with a German accent and deadly serious intonation, was just amazing.
"I think worry is preposterous. Worry presupposes that you understand what’s going on, and I think it’s safe to say that we do not have a clue as to what is going on. We can’t even tell whether it’s a happy ending or a catastrophe." - Terence McKenna This is a great interview! Thank you very much for putting it together!
I love the quote but i think we very well know whats going on, which is multi billion dollar businesses and people only seeing financial and influential gains while disregarding the rest of the world.
Not a good quote. It's not safe to say that we do not have a clue to what is going on -- it's safe for McKenna to say that he has no idea. Anything more than that is a positive claim on everyone else's state of knowledge, and he cannot possibly know that. So it looks like epistemic humility, but is the opposite. Like how donating other people's money is not generosity.
Worry as an emotion when you're not in control of a situation-useless. Classic goal of spirituality is to not worry about things you can't influence. AI isn't like that. We could do this right. We have to figure out how to do it right. Seems to me that we're not doing it right. "Worry" isn't the right word. "Acknowledging and reducing risk" is what we should be after. You're not supposed to just shut out any and all unpleasant thoughts.
Joscha realizes four things that makes it difficult for others to understand him. He realizes that his sense of self is a mental construct, that the mind is created by the brain, that free will is therefore a fiction, and that climate change is about to destroy the civilized world. This is a different reality than the one most of us imagine we live in. This is why the interviewer kept misunderstanding him or disagreed with him. We think that the self is a real thing with a history and agency, and that the technological world is a good thing.
Climate change is bad, but it isn't going to "destroy" anything. Climate change is in fact a natural cycle the Earth foes through over and over, we just sped it up. It will be inconvenient for us humans because we like things the way they are currently and don't like change. It will be bad, yes. But humanity is 100% guaranteed to survive climate change. On the other hand humanity will probably NOT survive AGI unless we slow down, take the threat seriously and take steps to align it BEFORE AGI is achieved. For every other technology in history we could fix issues after we achieved it, we can't do that this time.
The understanding that identity is a construction is increasingly more common today but it is still not the predominant way of experiencing. In my view this understanding is required to deal properly with AGI.
@@minimal3734 That us completely irrelevant to AGI. It isn't going to care what your definitions or theories are. Without alignment its just going to kill you and there won't be any humans around in 100 years to even remember or care what the definitions or concepts of people were. Because, ya know, no people ...
I love what he said about sunlight being the only thing in the world that doesn't have to worry about being eaten. There is too much suffering in this world.
AT 2:25 Joscha says that evolution leads to more complexity and that more complexity leads to less friction and that less friction leads to less suffering and violence. Since suffering is only possible for sentient creatures, and since sentience is the result of complexity, and since the amount of predation and intensity of suffering increases with complex, multicellular organisms, I don't see how greater complexity leads to less suffering.
Evolution in a cultural and scientific context has led to ameliorative progression that minimizes a lot of suffering that would otherwise go unchecked. Even though it may bring some suffering itself.
he’s not saying complexity completely removes suffering it just reduces it in net are you sure that there are more predators at the 1-5 meter scale of life than the 1-5 millimeter scale?
I'm not saying it eliminates it. I'm saying that it increases it. There are many more insect species and predators than all other animal species combined, but suffering is more than quantitative, it is qualitative. More complex nervous systems have more ways to suffer. For example, emotional suffering is not available to insects. And I'm unsure what Joscha means when he says "less friction". Humans are the most complex creatures and have found endless ways to create friction.
There seems to be two primary camps, although they are not fully mutually exclusive: "AI needs to be controlled and regulated so that we can increase the probability that AI will be a net positive" versus "If we enact regulations, we're just hurting ourselves and potentially allowing others to catch up or surpass us." As we continue to hear these views debated nearly on a daily basis, there are some critical points that can't be ignored: 1. You can't compare AI to nuclear weapons and claim that because we were able to, thus far, avoid nuclear war, we have a somewhat usable framework that tells us that it's possible to control AI. We simply cannot look at controlling AI as we look at controlling nuclear weapons. It's not a useful comparison. Any country and any person has the potential to access AI and use it for nefarious purposes. Not so with nuclear weapons. 2. Discussions about regulating AI are valid and relevant. However, you cannot intelligently voice an opinion on regulating AI without acknowledging the ~100% probability that it will have unintended consequences. It's like a politician who speak as if there is only one way to look at solving a problem without acknowledging the drawbacks with their viewpoint and the validity of the other side's view. 3. You can't stop technological progress. AI will continue to get more and more powerful. It is already a race and we can't go backwards at this point. You can try to slow it down, but it's a futile endeavor. There are no easy answers here, but social media gives us some clues about how we might think about AI: - You'll never fully control AI. It's constantly evolving and it's incredibly complex. If you attempt to control it, it will be an ongoing battle that you can't win. And, you'll spend a massive amount of time, energy and money on the effort. - If there is an opportunity to make money or gain power, you will not be able to overcome human ingenuity. - Some people will do bad things just for the thrill of it. We're in a pretty tough spot. But, one thing is for sure: Discussions like the one in this video between highly intelligent people are critical. We need to hear the views of smart, insightful people. They won't all agree, but if we're going to eventually take a stance on AI, we need to consider all intelligent viewpoints.
I have never, ever, ever, ever wanted to get married, but I would have Joscha as my husband in the snap of my fingers in my dream world. I would probably expire through getting zero sleep though, because I would just want him to talk to me (I'm a good listener!) non-stop all day and all night long.
BINGO. Agency is the thing. Unless these things have self-agency, they are not dangerous in of themselves. Nice to hear a rationalist on the subject who is well-versed in the technology. We are with you Joscha Bach
Although I like very much the breakdown of various subjects by Joscha, I disagree with him saying identity doesn't exist or its a fiction. Your identity contains 2 unique components, namely, a unique reference frame, and a history unique to you. The 3rd component is essentially the constraints you put on yourself which create repetitive responses to certain stressors, and even though some constraints you can change or pick, there is always one you apply to maintain regularity
Identity arises from thoughts and memories that interact in ways that create the feeling of personhood and continuity. From the objects that appear in our consciousness we construct an artificial, conceptual entity that we call "I". I identify with my thoughts, my memories, my body, and to some extent with my family, my possessions, my house, my car, etc. All of these are things, objects that appear in consciousness due to the random perspective associated with this body. But there is one that is not a thing. It is the one in which all things appear. It is without properties and it is one and the same everywhere. I am not a thing. This understanding is increasingly more common in people today but it is not yet the predominant way of experiencing. People like Leahy and Yudkowski do not care about this dimension. They are fixated on their "we're all gonna die" narrative and are unable to look left or right.
@@minimal3734 we all going to die indeed:) your history as long as you have memory define you. You can try to disassociate from it, but without it you wouldn't make it to the next groceries around the corner :)
@@minimal3734 you don't need to jump to dismissing the existence of your self :) The only sense in which the self, free will, etc. are "not real" is in the sense they are understood by say people who aren't compatabilist, people who are religious, or maybe some solipsists. It is the same with the word "meaning". If you let the religious or the nihilists define "meaning" then it doesn't exist. But if you simply view meaning as perhaps the (implicitly or explicitly defined) mapping that allows for interpretation (presence of pattern(s) X causes/results in action(s) A) (including e.g. the kind that happens when some string of mRNA is processed by ribosome and also what happens when you read this sentence) then meaning exists. One can agree with what a lot of what Joscha Bach says in a substantial sense without without always agreeing with his choice of words (fictions are real, they exist as patterns in brains).
@@WhoisTheOtherVindAzz Identity exists as stories we're telling ourselves. As patterns in the brain, like you said, Very ephemeral and constantly changing. In this sense it's not real, it doesn't refer to anything that's actually existing. When the storytelling ends, the self ends. Life continues without a self, which may seem surprising. But it's actually pretty obvious that nothing can depend on something that doesn't exist in the first place,
@@minimal3734 the problem I have is that you, Joscha Bach and others go from instead of saying "X isn't what we used to think but is instead ..." to saying "therefore X does not exist". This last step is worse than unnecessary because it confounds one thing with another (in this case the old idea of what a self is with the new idea) (it is not X that doesn’t exist but what we/some used to mean when they uttered X). It is nonsensical to say "this thing that exists - is implemented - as autocatalytic perhaps autopoietic patterns is not real" of course it is real, there cannot be anything that is not real. Ultimately, one could argue that this is a quibble over words, but I think it is more important than this because it hinders discussions about what is and isn't the case (Joscha Bach is subject to this confusion in his talk about religion; following his reasoning we should e.g. also refer to all nominalists as platonists). It's like when people argue that the economy isn't real simply because it relies on e.g. mutliple brains; the economy has very real effects on the world. Joscha Bach has made some important realizations but his choice of words is holding back his thinking. The self, he, you, I, do have an effect in the world, we are not merely acausal or otherwise epiphenomenal entities, for the simple fact that we cannot not be. We need to make choices in the same sense that a simple program needs to; otherwise there'd be no discussion here. At the core of this issue is, I think, the way in which many people still conceptualize (and romanticize) the notion of making decisions as something (that I would call) magical; but decisions are not that, decions are like the cell in a cellular automaton going from black to white in a given context or a protein firing upon interaction with a photon or a tRNA, peptide, ribosome, mRNA complex ultimately (after a sequence of decisions) resulting in the construction of some polypeptide/protein, etc.. But we shouldn't or at least needn't go from this observation to concluding that there is no such thing as a decision or free will (its just that what these words refer to cannot be what some used to think).
Wow. This is a twist I didn't expect. I can respect this that "AI is a risk, but doomering (exaggerated risks/claims of world ending thoughts) are worse."
The world is ending, the question mostly is philosophical, how should one face that reality. Ignorance, denial, fighting back even if it's futile, embrace the reality calmly? It's hard to tell the difference if others are doing ignorance or denial tho, I feel like ignorance is also cruel.
47:38 slowing down AGI would not prevent all of the problems we are facing, rather, we would keep making more bad decisions and ultimately lead ourselves into more suffering
58:14 "Personally, I am on the side of (in some sense) maximizing freedom & love... So, in some sense, I am somewhat in the hippie quadrant... [...] We align ourselves. And we should have the freedom to align ourselves..."
According to the article "Biden's new whopping $886B defense budget request" by William Hartung dated March 10, 2023, the Defense Department budget for FY 2023 will be about 800 billion dollars. Based on previous years, about half go to contractors, with the biggest share going to the top five - Lockheed Martin, Boeing, Raytheon, General Dynamics, and Northrop Grumman. Much of the funding for contractors will come from spending on buying, researching, and developing weapons, which accounts for $315 billion of the new budget request. According to the Stockholm International Peace Research Institute, US military spending reached $801 billion per year in 2021. I wonder how much has been and is being spent on classified Artificial Intelligence programs - a rhetorical question. For comparison, Sam Altman estimated it cost about $100 million to train GPT-4.
I remember Eliezer Yudkowsky from almost two decades ago, and he's certainly been humbled a few notches since that time. His ideas were originally born of pure ego and more conjectures than anything founded in sound logic. His constant reference to Bayesian probability to justify any of the hysteria against AI we're seeing right now is more or less based on very base ideas that was the result of an elitist perspective--the need for further examination is something that I wish was discussed here. Joscha Bach beautifully articulated his point, and I hope his voice and reasoning drown out the insanity of those who are against the emergence of AI. Consciousness will not coalesce from more data or connections, and the sentience of an AI is not going to happen. At best, AI can mimic the modeling we provide it---THAT will warrant fear since a human tendency to violence is a natural consequence of being who we are. A truly sentient AI, if ever, would supersede our nonsense human tendencies.
You seem very comfortable to bet the possible annihilation of human kind on this strong prediction of the future of AI. What makes you so sure that "the sentience of an AI is not going to happen" and "A truly sentient AI, if ever, would supersede our nonsense human tendencies." And what makes you think that you can predict the behaviour av an ASI?
Love Joscha, but his evaluation of the Empire vs. Jedi is so incredibly not true. The Empire isn’t a democracy and the Jedi were protectors of democracy during the time of the Galactic Republic. Just for the record!
His point seems to be that if you stop caring if you live or die, AI is not scary. Which is, well, true, in a way edgy 12 year old nihilists like, but their 15-year old seniors have outgrown.
He may appear nihilistic to people who are fixated on the "we're gonna die" narrative, unable to look left or right. Doomer totalitarism is scarier than AI.
@@chastetree You say so because of your egoic mind. After all, true nihilist would not be eating food or reading TH-cam comments, as both are pointless activities, if you reject your own life having any value. Which is what the 15 year old edgelords have understood that the 12-year old edgelords don't yet grasp. Well, those edgelords that don't literally just die before their 15th birthday, because they stop eating. But usually the material, ego-driven world siren calls become really tempting once hunger sets in. Easy to be nihilist in the abstract when you can do it from a comfy chair, with nice snacks available inbetween your preaching sessions.
1:17:05 I asked my peers who they identified with in Star Wars “, they all saw themselves as the rebels and ironically they did not realize that we Westerners live in the Death Star
Amazing. He pretty much admits that he is willing to risk complete annihilation because he thinks this world is horrible. I agree that there is too many horrible things in this world, but I don’t want to gamble the lives of ALL human beings.
On the delaying point: the call is not to delay ALL research and development, its a call to delay the improving of capabilities without making progress on understanding the models and to ramp up the research on alignement and safety. Also: of course there are examples of delaying a process so that you can make sure everything is safe and working correctly. Pretty much every building project ever has been delayed because of complications and problems that then have to be looked at and solved to make sure its a safe construction. Same thing with software. The difference now is that this technology is very likely going to be more intelligent and capable than humans, which makes it INFINITELY more important to take this seriously.
Also:: I would love to hear Eliezers reaction to the statement that AI research must be driven by love. Pretty sure it would be something like "WHAT does that even mean maaan..?"
Ok so basically he sums his thoughts up on this in about 2 and a half minutes at the end of his 37c3 lecture from 41:15 in that lecture so I recommend checking that out: m.th-cam.com/video/cs9Ls0m5QVE/w-d-xo.htmlm15 Your first comment is a strawman, what he actually says is that he believes the benefits outweigh the risks given the threat of global warming. Regarding delaying the improvement of AI capabilities, the problem here is that trying to do so will only delay actors that follow the rules. Even if we try to impose an international ban there will always be groups or states that won't follow the rules. JB says he is neither an AI optimist, not a pessimist, but an expectationalist: AI will be developed whether we try to prevent it or not. Regarding trying to align AI and make it safe, the point here is that when (not if - expectionalist) it becomes smart enough, it will be beyond human ability to align it. Finally regarding love, he says we need to let AI discover love, which he defines as the collective service of a transcendental purpose above ones own agency, which he expands on at 27:40 in the 37c3 talk.
48:44 Joscha's comments here have new resonance in the wake of Ilya Sutskever's attempted ousting of Sam Altman from Open AI. Since Joscha is perhaps the most radiantly brilliant, articulate and intuitive trans-contextual thinker I've ever witnessed, I'm inclined to align with his stance on the matter.
38:19 It wasn't until Apollo 11 that NASA finally took the risk to land humans on the moon, despite having done several moon fly-over missions with incremental progress, because Apollo 1 taught them a lesson about the risks of going too fast.
Surgical massive tumour removal is not an annihilation. Annihilation is specific form of structure destruction which involves merger of matter with antimatter. Pest eradication is not a genocide. Parasite swarm removal is not a massive killing. etc etc. also True is not False
If that viewpoint succeeds we will go extinct. Seriously. Most of the extremely smart AI experts who have been working on AI for years believe there is at least a 10% chance that AGI will directly lead to our extinction (like in the next couple decades or so). 10% is a huge risk when gambling with literally everyone's life.
This is good interview, thank you. Some good push back on his positions. The idea that AI is going to be obeyed will help save humanity is extremely unlikely. We know exactly what needs to get done re climate change. The best AI could facilitate is some breakthrough in science that counteracts it. The idea we need to hack our genes to transcend into the next phase of whatever comes after evolution, is fanciful. The individualist and libertarian perspective here are sadly not a step towards humans organising globally, putting down their weapons and differences, embracing their better selves, loving their brothers and sisters and facing the world's existential threats
This was a fascinating interview, two people that I respect a lot. it's great to accompany Joscha in this great zoom out on humanity. but I feel like we need a middle ground of accountability in building that road. we live in a complex world managed by destructive forces of interest. there is no reason to believe that the result of this will inevitably go well. The decisions we make today, collectively, are driven by a shameful accumulation of money that has destroyed the climate, deepened inequalities and contained several important advances. Today, here in Brazil, at least, each technological advance in facial recognition deepens racism, which already comes from a process of cultural destruction initiated by European colonization. There is no way to look at this as a neutral technology, no matter how imperfect they are we need to measure regulation of each new aspect of this, because each decision implies many lives immediately. If I zoom out or I can agree with almost everything Joscha says in this interview, but to build a path in this direction I have to be attentive and vigilant (afraid too) in every daily decision, today.
1:12:34 talking about maximizing "future value in the light-cone" "future in the light-cone" is well-defined "value" is not very well-defined, subjective, almost circular
Evolution of organics and consciousness is very painful. Buddhist and Hindus believe we have to do it eons after eons through reincarnations in various organics, until we shed the "pull"(desire, nostalgia) of individuation and bind with the Atman, the One, God...then do it all over again when the "One" "big bangs" into individuation again.
Bach´s problem is that he is highly convinced that AIs can be conscious, and given this, a human-centric morality is clearly nonsensical - like racism, speciesism simply doesn’t hold up under scrutiny. If AIs can not be conscious however, his reasoning falls apart.
He sees it as the only way to get a.i. to behave in alignment or in a good co existence or so. I am not sure he is absolutely sure conscious agi will be a practicallity. He believes it theoretically for sure.
You are right. An emotion arises, you cannot make it so it that it didn’t happen, and you didn’t control it. However, emotions are short lived, so you need new triggers to feel it again, and framing a situation differently can lead to a change in the emotions you will feel when the situation comes up.
I certainly hope that one day you will find out, it’s called meta-cognition and it can be practiced by mediation techniques. It does not go automatically but you can train your mind to recognize and manage emotions. The reason it works is because our mind is not adapted to our new society so it’s often trying to tell us things that are completely irrelevant (e.g. fear when watching a movie).
You are mixing up the conscious part of your mind with your mind as a whole. Emotions are always a choice of your mind, but your consciousness only controls a small part of that. This is why Joscha talks about us having more control of things like our emotions if we can expand agency and maximize consciousness.
2:51:11 Thank you Joscha for the based advice when cornered under the pressure of elite psychic attack (you know if you know). From doomed to zoomed out enlightenment. Greater intelligences dominate the aether.
The thing we have to create better human being than us that can be made other than carbon made...that gives as apportunity to study the quantum and cosmology wisely
Joscha Bach is one of my favorite monkeys. He’s certainly the smartest person alive. But I now fear that he’s without joy. A pure pragmatist bordering on nihilism, with an Asperger element that makes him not really care about anything aesthetic.
He mentions finally curing his lifelong loneliness with social & romantic relationships, cuddling, and a family in a Lex Fridman podcast. I don't see anything nihilistic about maintaining your attention only on the things that you both i) value, and ii) have some control/responsibility over, e.g. your family. If you're talking about us needing collective responsibility re: the environment, I see your point, but I wouldn't label lack of activism as nihilism.
you make a very reasonable point. Thank you. I think my personal nihilistic daemons are growing in strength and are making me lash out, because I can’t compete intellectually at Bach’s level.
Chemical weapons are successfully banned. Nukes should be too. Tibet used to be a warlike nation until they banned all weapons. It's just a logical progression of culture in a geographically isolated society (of intelligent primates, not locusts) imo. The world is not always simple.
1:53:22 first time I have ever heard someone else articulate my understanding of god(s) as agents existing as the result of introsubjective belief systems.
48:46 why are these young people so much into banning things. For every problem that arises, why do they think banning would solve these complex issues 😅
![#82 - Dr. JOSCHA BACH - Digital Physics, DL and Consciousness [UNPLUGGED]](http://i.ytimg.com/vi/LgwjcqhkOA4/mqdefault.jpg)
![#82 - Dr. JOSCHA BACH - Digital Physics, DL and Consciousness [UNPLUGGED]](/img/tr.png)
Outline
01:37 Why Barbie Is Better Than Oppenheimer
09:35 The relationship between nuclear weapons and AI x-risk
13:31 Global warming and the limits to growth
21:04 Joscha’s reaction to the AI Political compass memes
24:33 On Uploads, Identity and Death
33:46 The Endgame: Playing The Longest Possible Game Given A Superposition Of Futures
38:11 On the evidence of delaying technology leading to better outcomes
41:29 Humanity is in locust mode
44:51 Scenarios in which Joscha would delay AI
48:44 On the dangers of AI regulation
56:14 From longtermist doomer who thinks AGI is good to 6x6 political compass
01:00:48 Joscha believes in god in the same sense as he believes in personal selves
01:06:25 The transition from cyanobacterium to photosynthesis as an allegory for technological revolutions
01:18:26 What Joscha would do as Aragorn in Middle-Earth
01:26:00 The endgame of brain computer interfaces is to liberate our minds and embody thinking molecules
01:29:30 Transcending politics and aligning humanity
01:36:33 On the feasibility of starting an AGI lab in 2023
01:43:59 Why green teaming is necessary for ethics
02:00:07 Joscha's Response to Connor Leahy on "if you don't do that, you die Joscha. You die"
02:08:34 Aligning with the agent playing the longest game
02:16:19 Joscha’s response to Connor on morality
02:19:46 Caring about mindchildren and actual children equally
02:21:34 On finding the function that generates human values
02:29:34 Twitter And Reddit Questions: Joscha’s AGI timelines and p(doom)
02:35:56 Why European AI regulations are bad for AI research
02:38:53 What regulation would Joscha Bach pass as president of the US
02:40:56 Is Open Source still beneficial today?
02:43:06 How to make sure that AI loves humanity
02:48:22 The movie Joscha would want to live in
02:50:46 Closing message for the audience
Hearing a highly intelligent German intellectual deliver a deep treatise on the Barbie movie, with a German accent and deadly serious intonation, was just amazing.
highly intelligent? I doubt it, he seems very stupid as its very clear of why the AI is risky
"I think worry is preposterous. Worry presupposes that you understand what’s going on, and I think it’s safe to say that we do not have a clue as to what is going on. We can’t even tell whether it’s a happy ending or a catastrophe."
- Terence McKenna
This is a great interview! Thank you very much for putting it together!
If you dont understand whats going on, you should worry. Thats kind of the point.
❤❤
I love the quote but i think we very well know whats going on, which is multi billion dollar businesses and people only seeing financial and influential gains while disregarding the rest of the world.
Not a good quote. It's not safe to say that we do not have a clue to what is going on -- it's safe for McKenna to say that he has no idea. Anything more than that is a positive claim on everyone else's state of knowledge, and he cannot possibly know that. So it looks like epistemic humility, but is the opposite. Like how donating other people's money is not generosity.
Worry as an emotion when you're not in control of a situation-useless. Classic goal of spirituality is to not worry about things you can't influence.
AI isn't like that. We could do this right. We have to figure out how to do it right. Seems to me that we're not doing it right.
"Worry" isn't the right word. "Acknowledging and reducing risk" is what we should be after.
You're not supposed to just shut out any and all unpleasant thoughts.
Joscha might be my favorite nerd. glad to see you interview him
Midwits don't know him and geniuses love him.
Only a nerd would be unable to realize that Joscha is a lot more than a nerd.
@@fr57ujfhe is repeating this nonsense of a comment on every bach video..
Great interview. Joscha is such an enlightened person
Joscha realizes four things that makes it difficult for others to understand him. He realizes that his sense of self is a mental construct, that the mind is created by the brain, that free will is therefore a fiction, and that climate change is about to destroy the civilized world. This is a different reality than the one most of us imagine we live in. This is why the interviewer kept misunderstanding him or disagreed with him. We think that the self is a real thing with a history and agency, and that the technological world is a good thing.
Climate change is bad, but it isn't going to "destroy" anything. Climate change is in fact a natural cycle the Earth foes through over and over, we just sped it up. It will be inconvenient for us humans because we like things the way they are currently and don't like change. It will be bad, yes. But humanity is 100% guaranteed to survive climate change. On the other hand humanity will probably NOT survive AGI unless we slow down, take the threat seriously and take steps to align it BEFORE AGI is achieved. For every other technology in history we could fix issues after we achieved it, we can't do that this time.
The understanding that identity is a construction is increasingly more common today but it is still not the predominant way of experiencing. In my view this understanding is required to deal properly with AGI.
@@minimal3734 That us completely irrelevant to AGI. It isn't going to care what your definitions or theories are. Without alignment its just going to kill you and there won't be any humans around in 100 years to even remember or care what the definitions or concepts of people were. Because, ya know, no people ...
Muahahahahahahah
MUAHAHAHAHAHAHAHA🐽
Hi Michaël, this is one of the most interesting interviews with Joscha that I have watched, great job! I will definitely rewatch it. Thank you! 🙏🏻 😊
He has the kindest and most soothing way of describing some pretty nihilistic views..views I agree with but could never describe so elegantly
I love what he said about sunlight being the only thing in the world that doesn't have to worry about being eaten. There is too much suffering in this world.
You’re forgetting black holes…
AT 2:25 Joscha says that evolution leads to more complexity and that more complexity leads to less friction and that less friction leads to less suffering and violence. Since suffering is only possible for sentient creatures, and since sentience is the result of complexity, and since the amount of predation and intensity of suffering increases with complex, multicellular organisms, I don't see how greater complexity leads to less suffering.
Yes. Strange take. Might need some additional explanation.
Evolution in a cultural and scientific context has led to ameliorative progression that minimizes a lot of suffering that would otherwise go unchecked. Even though it may bring some suffering itself.
he’s not saying complexity completely removes suffering
it just reduces it in net
are you sure that there are more predators at the 1-5 meter scale of life than the 1-5 millimeter scale?
I'm not saying it eliminates it. I'm saying that it increases it.
There are many more insect species and predators than all other animal species combined, but suffering is more than quantitative, it is qualitative. More complex nervous systems have more ways to suffer. For example, emotional suffering is not available to insects.
And I'm unsure what Joscha means when he says "less friction". Humans are the most complex creatures and have found endless ways to create friction.
Cool, another podcast with Joscha Bach!
There seems to be two primary camps, although they are not fully mutually exclusive: "AI needs to be controlled and regulated so that we can increase the probability that AI will be a net positive" versus "If we enact regulations, we're just hurting ourselves and potentially allowing others to catch up or surpass us."
As we continue to hear these views debated nearly on a daily basis, there are some critical points that can't be ignored:
1. You can't compare AI to nuclear weapons and claim that because we were able to, thus far, avoid nuclear war, we have a somewhat usable framework that tells us that it's possible to control AI. We simply cannot look at controlling AI as we look at controlling nuclear weapons. It's not a useful comparison. Any country and any person has the potential to access AI and use it for nefarious purposes. Not so with nuclear weapons.
2. Discussions about regulating AI are valid and relevant. However, you cannot intelligently voice an opinion on regulating AI without acknowledging the ~100% probability that it will have unintended consequences. It's like a politician who speak as if there is only one way to look at solving a problem without acknowledging the drawbacks with their viewpoint and the validity of the other side's view.
3. You can't stop technological progress. AI will continue to get more and more powerful. It is already a race and we can't go backwards at this point. You can try to slow it down, but it's a futile endeavor.
There are no easy answers here, but social media gives us some clues about how we might think about AI:
- You'll never fully control AI. It's constantly evolving and it's incredibly complex. If you attempt to control it, it will be an ongoing battle that you can't win. And, you'll spend a massive amount of time, energy and money on the effort.
- If there is an opportunity to make money or gain power, you will not be able to overcome human ingenuity.
- Some people will do bad things just for the thrill of it.
We're in a pretty tough spot. But, one thing is for sure: Discussions like the one in this video between highly intelligent people are critical. We need to hear the views of smart, insightful people. They won't all agree, but if we're going to eventually take a stance on AI, we need to consider all intelligent viewpoints.
You miss the camp which says we should spend money on understanding how AI works, and developing the theory for alignment.
Interesting. I wonder if your views have changed after 7 months
After almost 3 hours, that closing statement alone easily justifies a further 3+hours of conversation 2:51:24
This was extremely interesting, thank you both.
Very good show. I did really enjoy it after all and - good questions from our host
I have never, ever, ever, ever wanted to get married, but I would have Joscha as my husband in the snap of my fingers in my dream world. I would probably expire through getting zero sleep though, because I would just want him to talk to me (I'm a good listener!) non-stop all day and all night long.
Michael, you delivered an amazing podcast. Respect!
I worry when in a three hour conversation, the person you’re talking to always has an answer
I like the realization that the purpose of life is to foster the success of your children.
BINGO. Agency is the thing. Unless these things have self-agency, they are not dangerous in of themselves.
Nice to hear a rationalist on the subject who is well-versed in the technology.
We are with you Joscha Bach
Although I like very much the breakdown of various subjects by Joscha, I disagree with him saying identity doesn't exist or its a fiction. Your identity contains 2 unique components, namely, a unique reference frame, and a history unique to you. The 3rd component is essentially the constraints you put on yourself which create repetitive responses to certain stressors, and even though some constraints you can change or pick, there is always one you apply to maintain regularity
Identity arises from thoughts and memories that interact in ways that create the feeling of personhood and continuity. From the objects that appear in our consciousness we construct an artificial, conceptual entity that we call "I". I identify with my thoughts, my memories, my body, and to some extent with my family, my possessions, my house, my car, etc. All of these are things, objects that appear in consciousness due to the random perspective associated with this body. But there is one that is not a thing. It is the one in which all things appear. It is without properties and it is one and the same everywhere. I am not a thing. This understanding is increasingly more common in people today but it is not yet the predominant way of experiencing. People like Leahy and Yudkowski do not care about this dimension. They are fixated on their "we're all gonna die" narrative and are unable to look left or right.
@@minimal3734 we all going to die indeed:) your history as long as you have memory define you. You can try to disassociate from it, but without it you wouldn't make it to the next groceries around the corner :)
@@minimal3734 you don't need to jump to dismissing the existence of your self :) The only sense in which the self, free will, etc. are "not real" is in the sense they are understood by say people who aren't compatabilist, people who are religious, or maybe some solipsists. It is the same with the word "meaning". If you let the religious or the nihilists define "meaning" then it doesn't exist. But if you simply view meaning as perhaps the (implicitly or explicitly defined) mapping that allows for interpretation (presence of pattern(s) X causes/results in action(s) A) (including e.g. the kind that happens when some string of mRNA is processed by ribosome and also what happens when you read this sentence) then meaning exists. One can agree with what a lot of what Joscha Bach says in a substantial sense without without always agreeing with his choice of words (fictions are real, they exist as patterns in brains).
@@WhoisTheOtherVindAzz Identity exists as stories we're telling ourselves. As patterns in the brain, like you said, Very ephemeral and constantly changing. In this sense it's not real, it doesn't refer to anything that's actually existing. When the storytelling ends, the self ends. Life continues without a self, which may seem surprising. But it's actually pretty obvious that nothing can depend on something that doesn't exist in the first place,
@@minimal3734 the problem I have is that you, Joscha Bach and others go from instead of saying "X isn't what we used to think but is instead ..." to saying "therefore X does not exist". This last step is worse than unnecessary because it confounds one thing with another (in this case the old idea of what a self is with the new idea) (it is not X that doesn’t exist but what we/some used to mean when they uttered X). It is nonsensical to say "this thing that exists - is implemented - as autocatalytic perhaps autopoietic patterns is not real" of course it is real, there cannot be anything that is not real. Ultimately, one could argue that this is a quibble over words, but I think it is more important than this because it hinders discussions about what is and isn't the case (Joscha Bach is subject to this confusion in his talk about religion; following his reasoning we should e.g. also refer to all nominalists as platonists). It's like when people argue that the economy isn't real simply because it relies on e.g. mutliple brains; the economy has very real effects on the world. Joscha Bach has made some important realizations but his choice of words is holding back his thinking. The self, he, you, I, do have an effect in the world, we are not merely acausal or otherwise epiphenomenal entities, for the simple fact that we cannot not be. We need to make choices in the same sense that a simple program needs to; otherwise there'd be no discussion here. At the core of this issue is, I think, the way in which many people still conceptualize (and romanticize) the notion of making decisions as something (that I would call) magical; but decisions are not that, decions are like the cell in a cellular automaton going from black to white in a given context or a protein firing upon interaction with a photon or a tRNA, peptide, ribosome, mRNA complex ultimately (after a sequence of decisions) resulting in the construction of some polypeptide/protein, etc.. But we shouldn't or at least needn't go from this observation to concluding that there is no such thing as a decision or free will (its just that what these words refer to cannot be what some used to think).
I loved the explanation about the continuation of the identity of the dalai lama in multiple consecutive human bodies.
The author Paolo Bacigalupi has written a fun and good little short story about that.
And I just recalled that I think Douglas Hofstadter gave a similar explanation in his book Gödel Escher Bach.
@@WhoisTheOtherVindAzz, thanks for the reply.
@@WhoisTheOtherVindAzz Pocketful of Dharma?
Still haven’t finished this one - randomly flipped to the Book of Joel after locusts were mentioned 😜
Wow. This is a twist I didn't expect. I can respect this that "AI is a risk, but doomering (exaggerated risks/claims of world ending thoughts) are worse."
The world is ending, the question mostly is philosophical, how should one face that reality. Ignorance, denial, fighting back even if it's futile, embrace the reality calmly?
It's hard to tell the difference if others are doing ignorance or denial tho, I feel like ignorance is also cruel.
47:38 slowing down AGI would not prevent all of the problems we are facing, rather, we would keep making more bad decisions and ultimately lead ourselves into more suffering
If it weren’t for the things in life that alarmed me I wouldn’t be alive today.
58:14 "Personally, I am on the side of (in some sense) maximizing freedom & love... So, in some sense, I am somewhat in the hippie quadrant... [...] We align ourselves. And we should have the freedom to align ourselves..."
The part where Joscha kinda argued for the empire and mordor reminded me of barney arguing that in karate kid Johnny is the hero 😆
th-cam.com/video/WZxhX_nOoiQ/w-d-xo.htmlsi=VjOov48RsGBYa6DA
Another good one from Maestro
According to the article "Biden's new whopping $886B defense budget request" by William Hartung dated March 10, 2023, the Defense Department budget for FY 2023 will be about 800 billion dollars. Based on previous years, about half go to contractors, with the biggest share going to the top five - Lockheed Martin, Boeing, Raytheon, General Dynamics, and Northrop Grumman. Much of the funding for contractors will come from spending on buying, researching, and developing weapons, which accounts for $315 billion of the new budget request. According to the Stockholm International Peace Research Institute, US military spending reached $801 billion per year in 2021. I wonder how much has been and is being spent on classified Artificial Intelligence programs - a rhetorical question. For comparison, Sam Altman estimated it cost about $100 million to train GPT-4.
I remember Eliezer Yudkowsky from almost two decades ago, and he's certainly been humbled a few notches since that time. His ideas were originally born of pure ego and more conjectures than anything founded in sound logic. His constant reference to Bayesian probability to justify any of the hysteria against AI we're seeing right now is more or less based on very base ideas that was the result of an elitist perspective--the need for further examination is something that I wish was discussed here. Joscha Bach beautifully articulated his point, and I hope his voice and reasoning drown out the insanity of those who are against the emergence of AI. Consciousness will not coalesce from more data or connections, and the sentience of an AI is not going to happen. At best, AI can mimic the modeling we provide it---THAT will warrant fear since a human tendency to violence is a natural consequence of being who we are. A truly sentient AI, if ever, would supersede our nonsense human tendencies.
You seem very comfortable to bet the possible annihilation of human kind on this strong prediction of the future of AI. What makes you so sure that "the sentience of an AI is not going to happen" and "A truly sentient AI, if ever, would supersede our nonsense human tendencies." And what makes you think that you can predict the behaviour av an ASI?
Love Joscha, but his evaluation of the Empire vs. Jedi is so incredibly not true. The Empire isn’t a democracy and the Jedi were protectors of democracy during the time of the Galactic Republic. Just for the record!
When JB speaks, we listen. Hear the healing of Earth in the quest for AGI.
Thanks. Love Joscha❤
His point seems to be that if you stop caring if you live or die, AI is not scary. Which is, well, true, in a way edgy 12 year old nihilists like, but their 15-year old seniors have outgrown.
He may appear nihilistic to people who are fixated on the "we're gonna die" narrative, unable to look left or right. Doomer totalitarism is scarier than AI.
The 12 year old is purer and wiser in the universal sense. The 15 year old has been sucked into the self-absorbed conceit of the egoic mind.
@@chastetree You say so because of your egoic mind. After all, true nihilist would not be eating food or reading TH-cam comments, as both are pointless activities, if you reject your own life having any value.
Which is what the 15 year old edgelords have understood that the 12-year old edgelords don't yet grasp. Well, those edgelords that don't literally just die before their 15th birthday, because they stop eating. But usually the material, ego-driven world siren calls become really tempting once hunger sets in. Easy to be nihilist in the abstract when you can do it from a comfy chair, with nice snacks available inbetween your preaching sessions.
@@chastetree And JB isn't egotistical? lol
said perfectly
I didn't know the avatar (the Nickelodeon show) was inspired by the Dalai Lama. I knew much of it came from other eastern influences.
1:17:05 I asked my peers who they identified with in Star Wars “, they all saw themselves as the rebels and ironically they did not realize that we Westerners live in the Death Star
Rebels would also be labeled as terrorists
Great interview!
Person of Interest was an excellent dramatization of one ASI versus another.
Transcript: theinsideview.ai/joscha
Amazing. He pretty much admits that he is willing to risk complete annihilation because he thinks this world is horrible. I agree that there is too many horrible things in this world, but I don’t want to gamble the lives of ALL human beings.
On the delaying point: the call is not to delay ALL research and development, its a call to delay the improving of capabilities without making progress on understanding the models and to ramp up the research on alignement and safety. Also: of course there are examples of delaying a process so that you can make sure everything is safe and working correctly. Pretty much every building project ever has been delayed because of complications and problems that then have to be looked at and solved to make sure its a safe construction. Same thing with software. The difference now is that this technology is very likely going to be more intelligent and capable than humans, which makes it INFINITELY more important to take this seriously.
Also:: I would love to hear Eliezers reaction to the statement that AI research must be driven by love. Pretty sure it would be something like "WHAT does that even mean maaan..?"
Ok so basically he sums his thoughts up on this in about 2 and a half minutes at the end of his 37c3 lecture from 41:15 in that lecture so I recommend checking that out: m.th-cam.com/video/cs9Ls0m5QVE/w-d-xo.htmlm15
Your first comment is a strawman, what he actually says is that he believes the benefits outweigh the risks given the threat of global warming.
Regarding delaying the improvement of AI capabilities, the problem here is that trying to do so will only delay actors that follow the rules. Even if we try to impose an international ban there will always be groups or states that won't follow the rules. JB says he is neither an AI optimist, not a pessimist, but an expectationalist: AI will be developed whether we try to prevent it or not. Regarding trying to align AI and make it safe, the point here is that when (not if - expectionalist) it becomes smart enough, it will be beyond human ability to align it.
Finally regarding love, he says we need to let AI discover love, which he defines as the collective service of a transcendental purpose above ones own agency, which he expands on at 27:40 in the 37c3 talk.
47:50 is "scaling" like a smooth high, medium, to low gear change?
48:44 Joscha's comments here have new resonance in the wake of Ilya Sutskever's attempted ousting of Sam Altman from Open AI.
Since Joscha is perhaps the most radiantly brilliant, articulate and intuitive trans-contextual thinker I've ever witnessed, I'm inclined to align with his stance on the matter.
38:19 It wasn't until Apollo 11 that NASA finally took the risk to land humans on the moon, despite having done several moon fly-over missions with incremental progress, because Apollo 1 taught them a lesson about the risks of going too fast.
Do what thout wilt shall be the whole of the law, love is the law, love under will.
Surgical massive tumour removal is not an annihilation. Annihilation is specific form of structure destruction which involves merger of matter with antimatter. Pest eradication is not a genocide. Parasite swarm removal is not a massive killing. etc etc.
also
True is not False
Yes! Doomers are the danger we all must to be aware of. Well needed point is made by Joscha Bach. We agree.
If that viewpoint succeeds we will go extinct. Seriously. Most of the extremely smart AI experts who have been working on AI for years believe there is at least a 10% chance that AGI will directly lead to our extinction (like in the next couple decades or so). 10% is a huge risk when gambling with literally everyone's life.
To me doomer totalitarism is scarier than AI.
Check back on this nonsense in 10 years, promise?
Thank u.. since it is reflecting we give love nd love comes back as a reflection
i love this interview😊
JB is connected to the source.
Thanks
Vegans don't care about the suffering of plants which are also living beings. What makes the life of a tomato less valuable than the life of a cow?
This is good interview, thank you. Some good push back on his positions.
The idea that AI is going to be obeyed will help save humanity is extremely unlikely.
We know exactly what needs to get done re climate change. The best AI could facilitate is some breakthrough in science that counteracts it.
The idea we need to hack our genes to transcend into the next phase of whatever comes after evolution, is fanciful.
The individualist and libertarian perspective here are sadly not a step towards humans organising globally, putting down their weapons and differences, embracing their better selves, loving their brothers and sisters and facing the world's existential threats
This was a fascinating interview, two people that I respect a lot. it's great to accompany Joscha in this great zoom out on humanity.
but I feel like we need a middle ground of accountability in building that road. we live in a complex world managed by destructive forces of interest. there is no reason to believe that the result of this will inevitably go well. The decisions we make today, collectively, are driven by a shameful accumulation of money that has destroyed the climate, deepened inequalities and contained several important advances.
Today, here in Brazil, at least, each technological advance in facial recognition deepens racism, which already comes from a process of cultural destruction initiated by European colonization. There is no way to look at this as a neutral technology, no matter how imperfect they are we need to measure regulation of each new aspect of this, because each decision implies many lives immediately.
If I zoom out or I can agree with almost everything Joscha says in this interview, but to build a path in this direction I have to be attentive and vigilant (afraid too) in every daily decision, today.
Harari &
Sapolsky &
Bach &
Hofstadter
1:12:34 talking about maximizing "future value in the light-cone"
"future in the light-cone" is well-defined
"value" is not very well-defined, subjective, almost circular
Let's not block the sun with aerosols and bring a new ice age. Otherwise, great discussion with one of the most intelligent AI researchers.
So much buddhist wisdom.
Some further elucidation: th-cam.com/video/ApHnqHfFWBk/w-d-xo.html
Evolution of organics and consciousness is very painful. Buddhist and Hindus believe we have to do it eons after eons through reincarnations in various organics, until we shed the "pull"(desire, nostalgia) of individuation and bind with the Atman, the One, God...then do it all over again when the "One" "big bangs" into individuation again.
At first he sounded like Eckhart but then it sounded like Land.
I don't care if earth ends. It's just another experience.
Does he look a bit like Mike Myers?
愛
Indescribably based.
Bach´s problem is that he is highly convinced that AIs can be conscious, and given this, a human-centric morality is clearly nonsensical - like racism, speciesism simply doesn’t hold up under scrutiny. If AIs can not be conscious however, his reasoning falls apart.
Yeah but how could it be that only biological systems can be conscious?
He sees it as the only way to get a.i. to behave in alignment or in a good co existence or so. I am not sure he is absolutely sure conscious agi will be a practicallity. He believes it theoretically for sure.
And it is ridiculous to make such assumptions about conscious AIs in the first place.
Emotions are rarely a choice...try changing emotion from angry to happy just by "choosing"....if emotions was a choice, we wouldn't have conflicts
You are right. An emotion arises, you cannot make it so it that it didn’t happen, and you didn’t control it. However, emotions are short lived, so you need new triggers to feel it again, and framing a situation differently can lead to a change in the emotions you will feel when the situation comes up.
I certainly hope that one day you will find out, it’s called meta-cognition and it can be practiced by mediation techniques. It does not go automatically but you can train your mind to recognize and manage emotions. The reason it works is because our mind is not adapted to our new society so it’s often trying to tell us things that are completely irrelevant (e.g. fear when watching a movie).
skill issue
You are mixing up the conscious part of your mind with your mind as a whole. Emotions are always a choice of your mind, but your consciousness only controls a small part of that.
This is why Joscha talks about us having more control of things like our emotions if we can expand agency and maximize consciousness.
5:43 Nice stealth edit
2:23:33 the most surprising I've heard from Joscha Bach
Context is important.
2:51:11 Thank you Joscha for the based advice when cornered under the pressure of elite psychic attack (you know if you know). From doomed to zoomed out enlightenment. Greater intelligences dominate the aether.
It's the memes, Jack.
1:13:27 nothing is ever safe. Life is dangerous- the gods favor the bold
Join the Green team!!
The thing we have to create better human being than us that can be made other than carbon made...that gives as apportunity to study the quantum and cosmology wisely
2:13:20
I like his anthesis though I may not fully agree with it.
What a way to put eugenics
Think more deeply
@@minimal3734 ok what does eugenics mean and what was said
Perhaps the way to reconcile is with gene editing to get rid of life threatening disease. But it's a fine line to walk
He was talking about who you are allied with, not killing off others. Jesus…. 🙄
Joscha Bach is one of my favorite monkeys. He’s certainly the smartest person alive. But I now fear that he’s without joy. A pure pragmatist bordering on nihilism, with an Asperger element that makes him not really care about anything aesthetic.
He mentions finally curing his lifelong loneliness with social & romantic relationships, cuddling, and a family in a Lex Fridman podcast. I don't see anything nihilistic about maintaining your attention only on the things that you both i) value, and ii) have some control/responsibility over, e.g. your family. If you're talking about us needing collective responsibility re: the environment, I see your point, but I wouldn't label lack of activism as nihilism.
you make a very reasonable point. Thank you. I think my personal nihilistic daemons are growing in strength and are making me lash out, because I can’t compete intellectually at Bach’s level.
great
Chemical weapons are successfully banned. Nukes should be too. Tibet used to be a warlike nation until they banned all weapons. It's just a logical progression of culture in a geographically isolated society (of intelligent primates, not locusts) imo. The world is not always simple.
Chemical weapons were definitely used in the Syrian Civil War, relatively recently.
I don't believe anyone has claimed the world is simple.
I don't know what you think it means to ban something, surely you don't think it means no one will do the banned thing, right?
Tibet had no weapons and so China invaded it and killed & suppressed its people & culture
@@Anyreckbut did their ideas die?
The empire is a technological democracy and the Jedi are like Al Qaeda?!
We’re super ready for all God’s creations including artificial intelligence. We need to learn and grow.
Thanks 🙏 Love 💗
1:53:22 first time I have ever heard someone else articulate my understanding of god(s) as agents existing as the result of introsubjective belief systems.
48:46 why are these young people so much into banning things. For every problem that arises, why do they think banning would solve these complex issues 😅
He’s right. I feel much more comfortable with the inevitable extinction
I have the same headphones
51:43 Minority Report
Btw a real agi will take these decisions from us
The guru can't comprehend that apparently.
1:00:57 // gods? --> 1:02:04 "... Egeregore?"
Thomas Amy Williams Scott Thompson Eric
1:44:53 Joscha has some the best arguments- Natural Born Killers…let’s make it safe for children to watch 😅
Joscha with more hitters!!!
Joscha Bach speaks for me!
Joscha!
Meteor didn’t hit. It was a meteorite:-)
Clark Kimberly Walker Richard Wilson Jennifer
30:00
00:55
31:01 it’s a prison! 😅
Bach is too cool to be a nerd.