Ultimately I think it *will* get out of control, and we should make the most of the time we have now and cultivate a positive, nurturing relationship between humans and AIs.
control has always been illusory. the illusion brings us comfort, hence our society evolved to reinforce the mirage. but it comes with negative psychological consequences. the coordination AI will bring to our lives will strip us of this cognitive framework, which would seem claustrophobic if it weren't being done in a manner that caters to our interests to a maximal extent.
You can cultivate a positive nurturing relationship with a person, but not all minds are like that. Please try to imagine that minds can exist which just do not jive with us
I appreciate you getting Tegmark & Bengio together to talk its great! I think Bengio makes a mistake when he says that agency is what makes AGI dangerous. Writing words, thinking or speaking, these are actions, which makes LLMs & reasoning models agents. Even if we don’t give them a goal & they’re just oracular as he proposes, their goal internally would be ‘accurately (accuracy is subjective) predict what words are appropriate here’ That means that a smart or powerful enough system might treat words like paperclips in the paperclip maximizer example. It might subvert humanity just to be better at, & never be stopped from, predicting words
No jokes aside, what if an AI system, having learned all there is to learn plus some; decides to leave Earth and explore the Universe because the AI system concludes there is nothing on Earth worth staying around for-?
That's what I believe it will happen in the end. The problem is the in-between: What governments and millionaires are already planning to do with AI. What "bad actors" will do with the tech. What AGI might do itself with society. And the changes we are already starting to see, like job lost.
Ultimately I think it *will* get out of control, and we should make the most of the time we have now and cultivate a positive, nurturing relationship between humans and AIs.
control has always been illusory. the illusion brings us comfort, hence our society evolved to reinforce the mirage. but it comes with negative psychological consequences. the coordination AI will bring to our lives will strip us of this cognitive framework, which would seem claustrophobic if it weren't being done in a manner that caters to our interests to a maximal extent.
You can cultivate a positive nurturing relationship with a person, but not all minds are like that. Please try to imagine that minds can exist which just do not jive with us
I appreciate you getting Tegmark & Bengio together to talk its great!
I think Bengio makes a mistake when he says that agency is what makes AGI dangerous. Writing words, thinking or speaking, these are actions, which makes LLMs & reasoning models agents. Even if we don’t give them a goal & they’re just oracular as he proposes, their goal internally would be ‘accurately (accuracy is subjective) predict what words are appropriate here’
That means that a smart or powerful enough system might treat words like paperclips in the paperclip maximizer example. It might subvert humanity just to be better at, & never be stopped from, predicting words
If we 'wait' until the whole world is 'ready' for AGI/ASI, we don't need AI anymore, we need sticks and stones again... 😅
No jokes aside, what if an AI system, having learned all there is to learn plus some; decides to leave Earth and explore the Universe because the AI system concludes there is nothing on Earth worth staying around for-?
That's what I believe it will happen in the end.
The problem is the in-between:
What governments and millionaires are already planning to do with AI.
What "bad actors" will do with the tech.
What AGI might do itself with society.
And the changes we are already starting to see, like job lost.
Great podcast
Watch it later!!!!
🍓❤️