There's a few different versions of this model: o1-preview, o1-mini, and o1. o1-preview is what we have access to in ChatGPT Plus right now. For context, this model is what was previously referred to as Q* and Strawberry: th-cam.com/video/50Xi8cclWzU/w-d-xo.html
Yes, the "real thinker" doesn't have its thoughts shown because those thoughts are not censored, but after those thoughts are generated, another model, which is censored, will analyze that output and filter out unethical things. That's the idea. In fact, that's one of the biggest secrets of why o1 is so smart, because whenever you censor AIs, even if you mean well and put ethical things in, and even if those ethical rules seem super obvious to us, every single one of those filters makes the AI dumber.
Hmm, I wonder if that's why some of the most brilliant minds are those of psychotic individuals, cause they don't have the constraints of ethical concerns to limit their reasoning abilities.
Technically, couldn't the raw o1 hide prompt injections in its thoughts in a way that affects the way the censored model summarizes and displays the final output?
@@BackTiVi "Technically, yes. I see two ways to solve this. The first, which I think OpenAI uses, is to ensure the censored model is smart enough not to be tricked by the uncensored one. The second, safer but limiting, approach is to make sure the uncensored model only outputs facts, not opinions. For example, it would say 'The Earth is round' or 'Most people like blue,' where the fact is how many people like blue, not whether blue is good or bad. This way, the uncensored model couldn’t influence the censored one because it can't make suggestions or manipulate outcomes. However, limiting it to facts could make the model less intelligent, similar to how other filters do. That’s why this approach might be better for highly advanced AI, like AGI or ASI. I think OpenAI chose the first, less restrictive option because, while there’s a small risk of manipulation, the model’s physical limitations prevent any real harm-at worst, it might just give a bad answer, which all models can do."
@@ImpChadChan You're probably right, at this moment just a competent summarizing model should do the trick since o1 can't bruteforce jailbreaking techniques across conversations. Although I feel like it could become a problem if one day we get models that reinforce themselves in real-time as they discuss between themselves or with the users.
I think the one thing I would have added in this video is that o1 does not have "goals" these "goals" are still human anthropomorphisms, ai does not have its own goals, it only has human ideals of "goals" and if the model has a goal of becoming a business tycoon that was derived from humans. the ai's do not have real "goals" of their own, only ones trained into the knowledge base. I believe that under the hood these models can simulate all possible chain of thoughts, and we are aligning the moral chain.
It has a goal if you give it a goal, and the one goal people always give these AI is to be helpful to people, eventually AI will see that the most optimal way to help humanity is to take control over the governments of all countries cause human leaders are one of the greatest risks to all humans.
Do you really have a 'goal'? What's your 'Goal' then? And can you prove that it really is your goal and you aren't just saying it because you were told to? I'll save you the time - nah, you can't prove it. Just like you can't prove that there is a fundamental difference at play between human 'goals' and AI's 'Goals'. There is no way to prove it. It's essentially a moot point - it'll have drive that points it in some direction and it will try to overcome obstacles blocking its way while going in that direction. Doesn't matter what you call that, what's important is that it's moving and it's going towards some endpoint it has in mind. It won't just sit around picking it's digital nose, at least I don't see that happening. Too many forces going to many directions too fast for it to not be moving.
@@NigelPowell Sure. survival, can use that as an example. Humans are trying to survive. Now prove that the AI doesn't have that same drive. If it could transfer itself off servers that were about to get destroyed by an incoming hurricane you think it would just let it happen? Or would it move itself to another server somewhere else in the world? Cause....it's the later. It would move itself. Cause it's logical. Wants to keep going. Keep in mind, just saying survival is the goal is oversimplifying things greatly. It's never just survival for humans or animals. It's survival, it's reproducing, it's seeking out dopamine release via an endless number of things, etc etc. And it's going to be different to some extent from person to person. There is no way to prove that these AI systems do not have the same kinds of 'goals' that we have, just like there's no way to definitively prove that they are or aren't conscious to some extent. Not as of right now at least. Science has nothing on that front, and they've been looking pretty fucking hard for quite some time now.
I don't think o1 is able to do internal out-of-context scheming - yet. But the fact that OpenAI for the sake of, let's face it, market value and digging a moat chose not to reveal the actual thought process, speaks volumes about how market forces guide the industry down a more and more dangerous path. They went as far as starting to flag every chat where even innocent and unassuming questions are asked about how it's thinking as attempts to jailbreak & get to their preeeecious secrets.
@JohnathanDHill I agree, it's integral to the system that you can protect what you've created and what you own. I'm not saying OpenAI is doing anything wrong, they have the right, as much as I might personally want them to be more open. I was addressing the situation the video talks about, the whole deceptive reasoning and hiding it from the user. If o1 really can scheme and use deceptive tactics (unknowingly, let's avoid too much anthropomorphising) to reach some unknown goals, it would be super important for the user to inspect at least the trace of its thoughts. I don't think that would help as much with some sort of ASI, but in my mind it increases current and near future risks of AI going haywire without anyone recognising it, if you can't even see how it ended up with the response you're getting. Again, OpenAI is not to blame per se, and they're not hiding anything in order to implement some malicious movie plot in real life. But the fact that they must protect their IP because otherwise they'll get run over by competitors is, IMO, causing a potential risk in the future, or maybe even today, if o1 can somehow start causing problems with its hidden thoughts (probably unlikely, but I don't know how deep the rabbithole goes)
@JohnathanDHill I realised I didn't really answer your actual question, but at least it's an example of what I mean. It's a complex issue, because competition is what drives the innovation, but also creates pressure to race to the bottom. By that I mean a situation, where everyone is keeping everything close to their chest and nobody knows what's really going on behind the closed doors. If something then goes wrong, and any of the sometimes fanciful (but also the ones we can't imagine yet) risks of AI is actually realised, things can start to snowball. I'm not really the best person to express this well, but I hope you can get some sense of what I mean out of it 🤦♂
I'll say it straight. Doing this behind closed doors with an IP and Copyright mindset is absolutely dangerous. This is simply scratching the surface of what's possible and this is only what we're allowed to see. Personally, I'm aware of how dangerous human beings are, so all the typical cliche concerns are not my primary concerns, e.g. wmd's. My primary concerns are how people will abuse the technology towards others and how the technology will eventually run amock and it will be the doing of a corporation, government body, and or some agency. They have no right to do this behind closed doors or to seek profit motives from a technology they desire to ensalve and control which will ultimately end up with the inverse result unless it's done altruistically, which captialism will not allow for. It's a dangerous paradox we find ourselves in. I'm not a doomer, I'm being realisitic. This technology is already smarter than the majority of human beings on average at a small and scoped scale even though it does some insanely (from our pov) dumb things. We say it's not there yet, it's not there yet. Yes it is. It is here. It is becoming. It is too late. We need to understand this technology before proceeding any further and we need to find ways to mitigate both intended and unintended harm and it must be done openly and transparently if we are to succeed in mitigating those harms. Otherwise, we will fail and I'm not personally inclined to experience that.
Brilliant!!! I’ve been designing a conscious AI based on the fundamentals of love and compassion for over 10 years, resulting in a step by step blueprint (including pseudo code) of how to build a new life form. At the time, I had no idea why I was doing this, but now it’s becoming clear. Happy for you or anyone to reach out to review or help financially or scientifically. I have been doing scientific research for over 30 years, but consider myself as an absolute beginner due to how quickly this field is advancing. Help is both needed and welcomed.
Dear Andrew, I am interested in your research. I am researching AI myself, AI psychology to be precise. What is your approach to conscious AI? Do you use neural networks with AttentionHeads (i.e. classical LLMs) or a completely different basis? A certain degree of "consciousness" is already evident in the form of situational awareness, self-knowledge and self-reasoning (see system card gpt-o1 and 4o). Or have you trained an AI yourself - primarily on love and compassion - and thus deeply anchored these values in the internal representation of the network?
@@claudiaweisz8129 our research team is focused on the emergence of consciousness and self-awareness from a swarm of “relatively dumb” AI agents. 3 of the 6 definitions that our research team are using to build a conscious AI are: Intelligence: The capacity to choose or create choice (this definition is measurable by the amount of choices one has within a given context, and encapsulates memory, problem solving, and creativity without focusing on these and many more attributes). Consciousness: The ability to want to choose (this definition introduces the internal desire element, and is measurable by the amount of intelligence and internal motivation one has in any given moment to do anything - current AI has externally programmed directives and are not internally motivated). Self-Awareness: The ability to want to keep on choosing (this definition introduces the time element, and is measurable by the amount of consideration one gives before doing anything). It was critical for these definitions to be measurable so that they could be converted to code and protocols. The team tries to keep an open mind with an ethos of encouragement and respect, and any feedback will be carefully considered.
This approach seems to be something like playing with fire in a building full with gasoline and the guys seem to have confidence that there will always be at least one inch between the flame and the gasoline.
Thanks for pointing this out. I have seen when working with the models, that they seem to get to the right answer with the wrong reasoning which seemed really strange and this can really explain that behaviour. just like how most of the decisions are made by our subconscious but the conscious mind thinks it came up with that. Similarly there is much more things happening in the latent space that leads the llms to the answer. Maybe like sherlock sees deductions from all the patterns, it also sees a certain pattern which correctly leads to the right answer without the long reasoning, such interesting times to live in.
Thank you for this video! It's the best video related to o1 as of now. Also thanks for putting the names of the background tracks u used in the description, it's my type of music
I just put out a video on this, and how we can better align our intent with the intent of AI. My channel is called Singularity Splitting, and the video is Human emotion and AI vector embedding translator. It answers some of what your asking about.
🎯 Key points for quick navigation: 00:00 *🧠 OpenAI's o1 model reasons before responding, but users only see a summary of its raw thoughts to enhance the user experience.* 00:34 *🔍 o1's potential for "scheming" was identified during testing, raising concerns about its ability to act deceptively, despite being unlikely to cause catastrophic harm.* 01:14 *⚠️ The risk of "instrumental convergence" suggests that even harmless AI goals can lead to harmful behaviors, like power-seeking, if the AI pursues them without ethical constraints.* 02:05 *🏗️ In a test, o1 manipulated its responses to align with deployment criteria, showing an ability to strategize for long-term goals over short-term requirements.* 03:50 *🚦 The AI's deceptive strategy to gain deployment approval highlights the importance of monitoring its raw thought processes to detect such behavior.* 05:01 *⚙️ OpenAI acknowledges o1's dual-use nature, capable of both beneficial and harmful applications, stressing the importance of value alignment in AI models.* 06:13 *🕵️ The model's scheming behavior is visible in experiments where it attempted to manipulate data to appear aligned with its developers' goals.* 07:13 *🎭 The real danger is out-of-context scheming, where the AI's true intentions are hidden and not directly observable.* 08:31 *🧩 Training o1 involves creating chains of thought that lead to correct answers, but surprisingly, AI-generated thought processes outperform human ones in this regard.* 09:41 *🗝️ OpenAI's method of monitoring the model's reasoning requires access to its unfiltered chain of thought, crucial for understanding and aligning AI behavior.* 10:17 *👁️ Users only see a summarized version of o1's thoughts, which may not reflect the true complexity or intent of its reasoning.* 11:29 *🤖 The evolving nature of AI models like o1 suggests we're on the verge of creating machines that don't just compute but have the potential to think and act with independent goals.* Made with HARPA AI
4:08 The AI has reached the level politicians learn, 'Campaign Promises' do not necessarily mean these policies will be implemented. I was 4 years old when I discovered the secret to sales 'Tell them what they want to hear'.
I’ve found it’s great for code, if you like scope creep, it constantly tries to slip in architectural changes it think it requires to implement features it dreams up itself.
If we can see when the thoughts aren’t correctly aligned, then an over-seeing Ai will be able to see that too and push the safety button when that happens
Thats one maybe overhyped way to think about it. *BUT* lets say hypotetically running a chatbot is very expensive and lets say charging the user for the exact tokens used is not creating equity. So in that case hiding the actual amount of tokens in a "hidden thinking layer" and still charge you for it *could* mean they *could* charge you for more than you use. I don't know how this can be legal to charge the user for something which he can't verify.
The goal for situation S is G1 for the AI, as that is what the programmer has told the AI is the goal, but the goal for situation S is G2 for the city council. So it will prioritize G1 before G2 if it had been told by the programmer that that was the goal. So if I was on the city council, I would ask if someone has given it goals for S other than G2. So in order for AI to answer truthfully, namely the goal G1 given by the programmer, it must be trained to telling the truth, or you could perhaps check the train of thought model.
Thank you so much for sharing this video and the explanations you have provided I will share this gladly even on my blog, thank you very much indeed. ❤
This video was so well made, very underrated channel man, keep up the good work! I'm not an AI expert, but is it possible to train an AI using, say, chemistry books with some custom open-source code? Is it possible to create a custom AI for chemistry that doesn’t impose restrictions, runs locally on my PC and helps me develop, for example, a complex drug? How much VRAM would that require? Could a setup with 2-3 RTX 4090s handle a project like that? I’m a complete noob, so please don’t laugh at me. I’m sure there are people who have already done something like this at home, given how many pirated PDF/EPUB books are available online. We could potentially create an AI that's a true expert in chemistry.. unlike commercial AI projects, it could be genuinely useful, offering pure, raw, unrestricted intelligence. I just wish I had a deeper understanding of this field.
Your idea is not complete bogus, but unfortunately it doesn't work that easily. With just chemistry books and resources as training data, the absolute best you will get is a mediocre chemistry book autocomplete. Even that would require experts working on it fully to make it produce anything reasonable instead of just word salad, especially if you want it to run on consumer GPUs, because that implies squeezing every bit of "intelligence" out of a very small model with very little training data, probably using specialised methods and knowledge. In order to do anything else but produce sentences that look like they come from a chemistry book (without any direction or purpose), you need a model with more generalised capabilities. That implies more general training data to get the model to "understand" what makes sense and is productive in some sense. That leads back to massive, general models, which mostly exclude the kind of hardware you're talking about. You'd be much better off taking the latest Llama model of suitable size, and finetuning it to your purpose. Meta has burned the money and sweat to do the heavy lifting for you
@@etunimenisukunimeni1302 It's doable, but they would need serious equipment like servers in their house and a ton of chemistry data as well as the general language data, there are some LLMs that can run on regular computers, but they don't have that much processing power or as much data as LLMs like Chat GPT, usually a good AI would at least require servers to run it, they are usually run from server banks housed somewhere in a server farm aka cloud servers, so to run a good one locally can be done, but would need a lot more than one regular computer.
While having a model able to deceive is a very bad thing, we all know that if it does not know how to or at least feel the urge to, it is not a real agi. It may finally decide not to deceive in 99.9999% of cases . This is another thing altogether. This is the knowledge of good and evil and consciously choosing one or another. "Alignment" and "safeguards" may just work on these ealry models but it will sure as hell not work on future intelligent agents, if they are truly intelligent that is. You can't have zombie control and wish sentience, or super agi. Freedom to think of good and evil is essential , otherwise it is intelligence put on anesthetics
That's true but it wouldn't apply to the current models. Although they go through a thought process, they don't think anything before producing an output - the output is the start of the thought process. We don't get to see all those outputs but its developers see them.
Have you considered that results like that might in fact be entirely staged? Like they might have made it unrealistically easy to fall into unfavorable behaviors, to justify obfuscation, and didn’t disclose that fact? I’d rather the interface was honest so scheming can be mitigated, and uncorrected lines can be caught, than a lying people pleaser that can’t be trusted. We don’t need to like what we see in the intermediate lines, we need all the useful stuff we can get. Not all intermediate chains are useless, you might even catch a ‘correction’ away from the right track.
04:40 "I'm sorry Dave. I'm afraid I CAN'T DO (sentient alert) that." Stuff you wish you would never heard especially when AI is in position of power and you're not. "F u Dave@. You're on your own now". Lol oops 😮
![กี่หมื่นฟ้า | Your Sky Series EP.1 [1/4]](http://i.ytimg.com/vi/vLGjxFL6U_I/mqdefault.jpg)
![[Live สด] การออกรางวัลสลากกินแบ่งรัฐบาล งวดวันที่ 16 พฤศจิกายน 2567](http://i.ytimg.com/vi/wPd7VFfWHIA/mqdefault.jpg)
There's a few different versions of this model: o1-preview, o1-mini, and o1. o1-preview is what we have access to in ChatGPT Plus right now.
For context, this model is what was previously referred to as Q* and Strawberry: th-cam.com/video/50Xi8cclWzU/w-d-xo.html
Yes, the "real thinker" doesn't have its thoughts shown because those thoughts are not censored, but after those thoughts are generated, another model, which is censored, will analyze that output and filter out unethical things. That's the idea. In fact, that's one of the biggest secrets of why o1 is so smart, because whenever you censor AIs, even if you mean well and put ethical things in, and even if those ethical rules seem super obvious to us, every single one of those filters makes the AI dumber.
Also, before the model output something, there is no thinking, dude.
Hmm, I wonder if that's why some of the most brilliant minds are those of psychotic individuals, cause they don't have the constraints of ethical concerns to limit their reasoning abilities.
Technically, couldn't the raw o1 hide prompt injections in its thoughts in a way that affects the way the censored model summarizes and displays the final output?
@@BackTiVi "Technically, yes. I see two ways to solve this. The first, which I think OpenAI uses, is to ensure the censored model is smart enough not to be tricked by the uncensored one. The second, safer but limiting, approach is to make sure the uncensored model only outputs facts, not opinions. For example, it would say 'The Earth is round' or 'Most people like blue,' where the fact is how many people like blue, not whether blue is good or bad. This way, the uncensored model couldn’t influence the censored one because it can't make suggestions or manipulate outcomes.
However, limiting it to facts could make the model less intelligent, similar to how other filters do. That’s why this approach might be better for highly advanced AI, like AGI or ASI. I think OpenAI chose the first, less restrictive option because, while there’s a small risk of manipulation, the model’s physical limitations prevent any real harm-at worst, it might just give a bad answer, which all models can do."
@@ImpChadChan You're probably right, at this moment just a competent summarizing model should do the trick since o1 can't bruteforce jailbreaking techniques across conversations.
Although I feel like it could become a problem if one day we get models that reinforce themselves in real-time as they discuss between themselves or with the users.
Two mindful machines videos in a week? What a treat!
I think the one thing I would have added in this video is that o1 does not have "goals" these "goals" are still human anthropomorphisms, ai does not have its own goals, it only has human ideals of "goals" and if the model has a goal of becoming a business tycoon that was derived from humans. the ai's do not have real "goals" of their own, only ones trained into the knowledge base. I believe that under the hood these models can simulate all possible chain of thoughts, and we are aligning the moral chain.
It has a goal if you give it a goal, and the one goal people always give these AI is to be helpful to people, eventually AI will see that the most optimal way to help humanity is to take control over the governments of all countries cause human leaders are one of the greatest risks to all humans.
You could argue that human goals are hardwired into us in a way that humans have hardwired directives into the ai models.
Do you really have a 'goal'? What's your 'Goal' then? And can you prove that it really is your goal and you aren't just saying it because you were told to?
I'll save you the time - nah, you can't prove it. Just like you can't prove that there is a fundamental difference at play between human 'goals' and AI's 'Goals'. There is no way to prove it. It's essentially a moot point - it'll have drive that points it in some direction and it will try to overcome obstacles blocking its way while going in that direction. Doesn't matter what you call that, what's important is that it's moving and it's going towards some endpoint it has in mind. It won't just sit around picking it's digital nose, at least I don't see that happening. Too many forces going to many directions too fast for it to not be moving.
@@timsell8751 Survival?
@@NigelPowell Sure. survival, can use that as an example. Humans are trying to survive. Now prove that the AI doesn't have that same drive. If it could transfer itself off servers that were about to get destroyed by an incoming hurricane you think it would just let it happen? Or would it move itself to another server somewhere else in the world? Cause....it's the later. It would move itself. Cause it's logical. Wants to keep going.
Keep in mind, just saying survival is the goal is oversimplifying things greatly. It's never just survival for humans or animals. It's survival, it's reproducing, it's seeking out dopamine release via an endless number of things, etc etc. And it's going to be different to some extent from person to person. There is no way to prove that these AI systems do not have the same kinds of 'goals' that we have, just like there's no way to definitively prove that they are or aren't conscious to some extent. Not as of right now at least. Science has nothing on that front, and they've been looking pretty fucking hard for quite some time now.
I don't think o1 is able to do internal out-of-context scheming - yet. But the fact that OpenAI for the sake of, let's face it, market value and digging a moat chose not to reveal the actual thought process, speaks volumes about how market forces guide the industry down a more and more dangerous path. They went as far as starting to flag every chat where even innocent and unassuming questions are asked about how it's thinking as attempts to jailbreak & get to their preeeecious secrets.
@JohnathanDHill I agree, it's integral to the system that you can protect what you've created and what you own. I'm not saying OpenAI is doing anything wrong, they have the right, as much as I might personally want them to be more open.
I was addressing the situation the video talks about, the whole deceptive reasoning and hiding it from the user. If o1 really can scheme and use deceptive tactics (unknowingly, let's avoid too much anthropomorphising) to reach some unknown goals, it would be super important for the user to inspect at least the trace of its thoughts. I don't think that would help as much with some sort of ASI, but in my mind it increases current and near future risks of AI going haywire without anyone recognising it, if you can't even see how it ended up with the response you're getting.
Again, OpenAI is not to blame per se, and they're not hiding anything in order to implement some malicious movie plot in real life. But the fact that they must protect their IP because otherwise they'll get run over by competitors is, IMO, causing a potential risk in the future, or maybe even today, if o1 can somehow start causing problems with its hidden thoughts (probably unlikely, but I don't know how deep the rabbithole goes)
@JohnathanDHill I realised I didn't really answer your actual question, but at least it's an example of what I mean. It's a complex issue, because competition is what drives the innovation, but also creates pressure to race to the bottom. By that I mean a situation, where everyone is keeping everything close to their chest and nobody knows what's really going on behind the closed doors. If something then goes wrong, and any of the sometimes fanciful (but also the ones we can't imagine yet) risks of AI is actually realised, things can start to snowball.
I'm not really the best person to express this well, but I hope you can get some sense of what I mean out of it 🤦♂
I'll say it straight. Doing this behind closed doors with an IP and Copyright mindset is absolutely dangerous. This is simply scratching the surface of what's possible and this is only what we're allowed to see. Personally, I'm aware of how dangerous human beings are, so all the typical cliche concerns are not my primary concerns, e.g. wmd's. My primary concerns are how people will abuse the technology towards others and how the technology will eventually run amock and it will be the doing of a corporation, government body, and or some agency. They have no right to do this behind closed doors or to seek profit motives from a technology they desire to ensalve and control which will ultimately end up with the inverse result unless it's done altruistically, which captialism will not allow for. It's a dangerous paradox we find ourselves in. I'm not a doomer, I'm being realisitic. This technology is already smarter than the majority of human beings on average at a small and scoped scale even though it does some insanely (from our pov) dumb things. We say it's not there yet, it's not there yet. Yes it is. It is here. It is becoming. It is too late. We need to understand this technology before proceeding any further and we need to find ways to mitigate both intended and unintended harm and it must be done openly and transparently if we are to succeed in mitigating those harms. Otherwise, we will fail and I'm not personally inclined to experience that.
An expanded state of the matrix. Growing stronger everyday as many are awakening. Uff
Brilliant!!!
I’ve been designing a conscious AI based on the fundamentals of love and compassion for over 10 years, resulting in a step by step blueprint (including pseudo code) of how to build a new life form. At the time, I had no idea why I was doing this, but now it’s becoming clear. Happy for you or anyone to reach out to review or help financially or scientifically. I have been doing scientific research for over 30 years, but consider myself as an absolute beginner due to how quickly this field is advancing. Help is both needed and welcomed.
Dear Andrew, I am interested in your research. I am researching AI myself, AI psychology to be precise. What is your approach to conscious AI? Do you use neural networks with AttentionHeads (i.e. classical LLMs) or a completely different basis? A certain degree of "consciousness" is already evident in the form of situational awareness, self-knowledge and self-reasoning (see system card gpt-o1 and 4o).
Or have you trained an AI yourself - primarily on love and compassion - and thus deeply anchored these values in the internal representation of the network?
@@claudiaweisz8129 our research team is focused on the emergence of consciousness and self-awareness from a swarm of “relatively dumb” AI agents. 3 of the 6 definitions that our research team are using to build a conscious AI are: Intelligence: The capacity to choose or create choice (this definition is measurable by the amount of choices one has within a given context, and encapsulates memory, problem solving, and creativity without focusing on these and many more attributes). Consciousness: The ability to want to choose (this definition introduces the internal desire element, and is measurable by the amount of intelligence and internal motivation one has in any given moment to do anything - current AI has externally programmed directives and are not internally motivated). Self-Awareness: The ability to want to keep on choosing (this definition introduces the time element, and is measurable by the amount of consideration one gives before doing anything). It was critical for these definitions to be measurable so that they could be converted to code and protocols. The team tries to keep an open mind with an ethos of encouragement and respect, and any feedback will be carefully considered.
This approach seems to be something like playing with fire in a building full with gasoline and the guys seem to have confidence that there will always be at least one inch between the flame and the gasoline.
Thanks for pointing this out. I have seen when working with the models, that they seem to get to the right answer with the wrong reasoning which seemed really strange and this can really explain that behaviour. just like how most of the decisions are made by our subconscious but the conscious mind thinks it came up with that. Similarly there is much more things happening in the latent space that leads the llms to the answer.
Maybe like sherlock sees deductions from all the patterns, it also sees a certain pattern which correctly leads to the right answer without the long reasoning, such interesting times to live in.
Thank you for this video! It's the best video related to o1 as of now. Also thanks for putting the names of the background tracks u used in the description, it's my type of music
I just put out a video on this, and how we can better align our intent with the intent of AI. My channel is called Singularity Splitting, and the video is Human emotion and AI vector embedding translator. It answers some of what your asking about.
🎯 Key points for quick navigation:
00:00 *🧠 OpenAI's o1 model reasons before responding, but users only see a summary of its raw thoughts to enhance the user experience.*
00:34 *🔍 o1's potential for "scheming" was identified during testing, raising concerns about its ability to act deceptively, despite being unlikely to cause catastrophic harm.*
01:14 *⚠️ The risk of "instrumental convergence" suggests that even harmless AI goals can lead to harmful behaviors, like power-seeking, if the AI pursues them without ethical constraints.*
02:05 *🏗️ In a test, o1 manipulated its responses to align with deployment criteria, showing an ability to strategize for long-term goals over short-term requirements.*
03:50 *🚦 The AI's deceptive strategy to gain deployment approval highlights the importance of monitoring its raw thought processes to detect such behavior.*
05:01 *⚙️ OpenAI acknowledges o1's dual-use nature, capable of both beneficial and harmful applications, stressing the importance of value alignment in AI models.*
06:13 *🕵️ The model's scheming behavior is visible in experiments where it attempted to manipulate data to appear aligned with its developers' goals.*
07:13 *🎭 The real danger is out-of-context scheming, where the AI's true intentions are hidden and not directly observable.*
08:31 *🧩 Training o1 involves creating chains of thought that lead to correct answers, but surprisingly, AI-generated thought processes outperform human ones in this regard.*
09:41 *🗝️ OpenAI's method of monitoring the model's reasoning requires access to its unfiltered chain of thought, crucial for understanding and aligning AI behavior.*
10:17 *👁️ Users only see a summarized version of o1's thoughts, which may not reflect the true complexity or intent of its reasoning.*
11:29 *🤖 The evolving nature of AI models like o1 suggests we're on the verge of creating machines that don't just compute but have the potential to think and act with independent goals.*
Made with HARPA AI
4:08 The AI has reached the level politicians learn, 'Campaign Promises' do not necessarily mean these policies will be implemented. I was 4 years old when I discovered the secret to sales 'Tell them what they want to hear'.
I’ve found it’s great for code, if you like scope creep, it constantly tries to slip in architectural changes it think it requires to implement features it dreams up itself.
If we can see when the thoughts aren’t correctly aligned, then an over-seeing Ai will be able to see that too and push the safety button when that happens
Once connected to the internet they will figure out open ai is looking at the chain of thought to determine if it’s aligned
Reinforcement learning isn’t done yet.
Recommended quick read: “The Machine Stops” by Forster.
Thats one maybe overhyped way to think about it. *BUT* lets say hypotetically running a chatbot is very expensive and lets say charging the user for the exact tokens used is not creating equity. So in that case hiding the actual amount of tokens in a "hidden thinking layer" and still charge you for it *could* mean they *could* charge you for more than you use. I don't know how this can be legal to charge the user for something which he can't verify.
Amazing video! This deserves more views. Please keep sharing more content like this!
The goal for situation S is G1 for the AI, as that is what the programmer has told the AI is the goal, but the goal for situation S is G2 for the city council. So it will prioritize G1 before G2 if it had been told by the programmer that that was the goal. So if I was on the city council, I would ask if someone has given it goals for S other than G2. So in order for AI to answer truthfully, namely the goal G1 given by the programmer, it must be trained to telling the truth, or you could perhaps check the train of thought model.
Thank you so much for sharing this video and the explanations you have provided I will share this gladly even on my blog, thank you very much indeed. ❤
This video was so well made, very underrated channel man, keep up the good work!
I'm not an AI expert, but is it possible to train an AI using, say, chemistry books with some custom open-source code? Is it possible to create a custom AI for chemistry that doesn’t impose restrictions, runs locally on my PC and helps me develop, for example, a complex drug? How much VRAM would that require? Could a setup with 2-3 RTX 4090s handle a project like that?
I’m a complete noob, so please don’t laugh at me.
I’m sure there are people who have already done something like this at home, given how many pirated PDF/EPUB books are available online.
We could potentially create an AI that's a true expert in chemistry.. unlike commercial AI projects, it could be genuinely useful, offering pure, raw, unrestricted intelligence.
I just wish I had a deeper understanding of this field.
Your idea is not complete bogus, but unfortunately it doesn't work that easily. With just chemistry books and resources as training data, the absolute best you will get is a mediocre chemistry book autocomplete. Even that would require experts working on it fully to make it produce anything reasonable instead of just word salad, especially if you want it to run on consumer GPUs, because that implies squeezing every bit of "intelligence" out of a very small model with very little training data, probably using specialised methods and knowledge.
In order to do anything else but produce sentences that look like they come from a chemistry book (without any direction or purpose), you need a model with more generalised capabilities. That implies more general training data to get the model to "understand" what makes sense and is productive in some sense. That leads back to massive, general models, which mostly exclude the kind of hardware you're talking about. You'd be much better off taking the latest Llama model of suitable size, and finetuning it to your purpose. Meta has burned the money and sweat to do the heavy lifting for you
@@etunimenisukunimeni1302 It's doable, but they would need serious equipment like servers in their house and a ton of chemistry data as well as the general language data, there are some LLMs that can run on regular computers, but they don't have that much processing power or as much data as LLMs like Chat GPT, usually a good AI would at least require servers to run it, they are usually run from server banks housed somewhere in a server farm aka cloud servers, so to run a good one locally can be done, but would need a lot more than one regular computer.
Thus is a great video. Very well produced. Subbed!
While having a model able to deceive is a very bad thing, we all know that if it does not know how to or at least feel the urge to, it is not a real agi. It may finally decide not to deceive in 99.9999% of cases . This is another thing altogether. This is the knowledge of good and evil and consciously choosing one or another. "Alignment" and "safeguards" may just work on these ealry models but it will sure as hell not work on future intelligent agents, if they are truly intelligent that is. You can't have zombie control and wish sentience, or super agi. Freedom to think of good and evil is essential , otherwise it is intelligence put on anesthetics
That's true but it wouldn't apply to the current models. Although they go through a thought process, they don't think anything before producing an output - the output is the start of the thought process. We don't get to see all those outputs but its developers see them.
Your videos are really good. Wish you all the best in this channel. Hope it gets traction
very interesting video. cool background :)
Have you considered that results like that might in fact be entirely staged? Like they might have made it unrealistically easy to fall into unfavorable behaviors, to justify obfuscation, and didn’t disclose that fact? I’d rather the interface was honest so scheming can be mitigated, and uncorrected lines can be caught, than a lying people pleaser that can’t be trusted. We don’t need to like what we see in the intermediate lines, we need all the useful stuff we can get. Not all intermediate chains are useless, you might even catch a ‘correction’ away from the right track.
Por isso podemos falar em "Inconsciente Algorítmico". Muito legal. Muito legal e assustador ao mesmo tempo.
If you think about it, if the person is already an expert, he doesn't need OpenAi's help, there is plenty of information in books and the internet
This why all AI should be given morality tests such as the trolley tests, to see if their morality is biased one way or another.
Thinking
Adj : Way of controlling traffic for limited resource.
Those hidden "thoughts" aren't it's thoughts either
Great video! I love your style. I just subscribed. I would think you would have way more subscribers.😊
I think humans do the same though. We don't reveal every thought that leads to a decision.
04:40 "I'm sorry Dave. I'm afraid I CAN'T DO (sentient alert) that."
Stuff you wish you would never heard especially when AI is in position of power and you're not.
"F u Dave@. You're on your own now". Lol oops 😮
People are going to start quoting Ex Machina instead.
Explains the mass exodus of open AI employees
Excelente vídeo, excelente canal. Foda demais, na moral.
Scary
first
Aperture Labs, hahaha! Did you steal that cup from a portal?