GPT-5: Everything You Need to Know So Far
ฝัง
- เผยแพร่เมื่อ 25 ม.ค. 2024
- Was yesterday the day GPT-5 actually started training? This video has everything we think we know so far about GPT-5, drawing on exclusive interviews, OpenAI employee comments, Altman confirmations and more. Think of this as the ultimate compilation of GPT-5 news. Plus, as a bonus, you’ll get 1 super-practical tip on typing to ChatGPT and a Dalle-3 discovery.
AI Insiders: / aiexplained
GPT-5 Training Tweets?: / 1750558864469299622
/ 1750609836713365570
Altman Gates: • “I didn’t expect ChatG...
Altman Guessing Game: www.ft.com/content/dd9ba2f6-f...
OpenAI Cryptic Tweets Ben Newhouse: gopatrik/with_rep...
Altman Davos: • OpenAI CEO Sam Altman ...
Altman Axios: • Axios House at Davos #...
Altman In the Room: • Altman Out: Reasons, R...
Karpathy OS: / the-llm-os-a-glimpse-i...
Brockman Checkpoints: gdb/status/164618...
Let’s Verify: arxiv.org/pdf/2305.20050.pdf
My Original Video on Verify: • 'Show Your Working': C...
Thought Unfaithfulness: arxiv.org/abs/2307.13702
Deepmind Original: arxiv.org/pdf/2211.14275.pdf
OpenAI Data: openai.com/blog/data-partners...
Etched AI 100T Video: • A 100T Transformer Mod...
French Dataset: / 1750810261856866783
Peter Wildeford: / peterwildeford
GPT-4 Typos: openreview.net/pdf?id=STHKApXVMH
OpenAI Redteaming: openai.com/blog/red-teaming-n...
Brockman Unpredictable: • Sam Altman's World Tou...
OpenAI Elections: openai.com/blog/how-openai-is...
Biden Robocall: • New Hampshire official...
Anthropic Amodei Interview: • Anthropic CEO on Leavi...
Laziness: www.theverge.com/2024/1/25/24...
AI Insiders: / aiexplained
Non-Hype, Free Newsletter: signaltonoise.beehiiv.com/ - วิทยาศาสตร์และเทคโนโลยี
The thing I love about your channel is you only post when there is news to share, instead of posting filler on a regular schedule to appease the algorithm. The signal to noise ratio of this channel is 💯
Definitely!
Perfect description of why this channel is the best. Completely unrivalled signal to noise.
I find myself trying to scratch the itch for AI Explained content between posts by watching other higher-output channels, but I always come back to this one as my actual source for useful and insightful news.
THIS. Every video is actual news and developments.
Best ai channel by far
For me, correcting typos and using polite language, such as saying 'please' to an LLM is not only about avoiding the development of bad habits, it's also a concern if models respond less efficiently to polite requests (e.g. phrases such as as 'Could you please [...]?' or 'Could you do this task?' vs direct commands like 'Do this task'). Could lead to a shift in how people communicate.
Exactly, models reflect human nature. If you put at the end of the prompt "Your answer will certainly resolve perfectly." at the end of your question, the answer will be much better and effective. It also works with "it worked", "thank you", etc, etc. This will force him to search his knowledge for solutions presented in forums or repositories in his training base that really worked.
Great comment! Totally agreed with you.
I've also done this from the start. I have manners and express gratitude with people and I don't see why communicating with LLMs should be any different. That and the fear that in the future the AI will remember who was kind and who was not lol
@@neomatrix2669did you actually test that? Shouldn't be that difficult to test.
THIS@@cosmicwebb
I tested letter scrambling with a python script that fully randomized the letters in each word. When giving it the beginning of a news article from that day, to make sure the data was not in the training data, it dealt with it perfectly. It was near-perfect even if the text was nonsensical. It only had severe dificulties if the text consisted of entirely random words.
Incredible
That's fascinating because it implies that the representation of words places a very low weight to the order of letters. The model's internal representation of words is basically just the amount of each letter between two spaces, with maybe a very small amount of additional weighting to the first couple of letters.
We know humans perceive whole words at a time, but generally we need the first and last letter as anchors.
If you cnhage or sbcrlame the ltertes in the mldide it is uuslaly slitl udntnasdeblre eevn to ppolee.
@@perplexedon9834so each word (or token) has to be identified regardless of spelling but if you were to change the order of the words to such a degree it may misinterpret what you actually meant, particularly if there was quite a bit of reasoning needed eg. in coding something adhoc
I just wonder what happens in languages where the tokenization is less efficient
@@perplexedon9834 Interestingly this is very consistent with general principles of data compression.
About asking it to remove lamp posts is probably like asking someone not to think of a pink elephant, no matter how hard they try, they will still think of a pink elephant.
Yes, that's exactly what I thought.
It's a pink elephant now ? :) I have only heard an example with good ol' polar bears. Even the examples get fancier... :)
Both humans and LM's are not great with negatives. Even with children it's suboptimal to say "don't do x".
I'm thinking it was not trained on removal or identifying things consistently in the generated images, it needs to do a few passes on reasoning vs what is in the image.
That is a good comment. On the other hand, a human artist has no problem with the same task.
@@minimal3734 No, they absolutely do.
I love this. From a uni student where everything basically revolves around 'where's your evidence' I appreciate your title
I think 10 years ago we would have all assumed that reasoning gets models where they are right now, and not intuition. that sheer intuition gets us here is just amazing
I'm wondering how reasoning and intuition are actually related and how they are affected by the number of layers in the model. I tend to think that iteration (reasoning) and layer count are somewhat interchangeable, So, while we think of intuition as the first answer without iterating on the problem at hand, and this is what the AI currently does, there seems to be reasoning going on between the layers of the model.
@@minimal3734 I think he was talking about reasoning vs intuition in the sense of how researchers approched the models' development
@@pictzone You might be right. But isn't it still amazing that these models nail the answers to difficult questions by sheer 'intuition', without iterating on the problem? A human reasons before answering, these models just throw out their first thought.
@@minimal3734 I think reasoning is working with first principles and that's the difference to intuition which goes mostly by analogy. when reasoning you start at an understanding A and accumulate knowledge (ground truth) to it until A resembles a goal B. or you work backwards from B to A. it's a process. layers do combine concepts but only by chance contain ground truth
@@minimal3734 Actually it's quite a funny coincidence. I'm reading a book called "Blink" at the moment, and its main focus is exactly on this topic: how humans have two types of thinking, a logical drawn-out one and an extremely fast intuition based one.
It gives some mindblowing examples that make you realize how true this is, if you have the time you should read it. It's really insightful and with actual insane applied utility.
But anyway, I get you're talking about difficult questions that would only be possible for a human to answer using the logical type thinking. But I suspect our fast thinking is the most similar neuronal system to these current neural nets, but theirs is kind of hyperboosted (vs ours that is just one of many systems) and that's why the results are so incredible.
that’s so cool with the lampposts.
i think what is cool is that soon some of the tools in stable diffusion will make there way to dalle3
negative prompts really give you another dimension to work on when you are prompting an image into existence
Thank you. Still the only content on YT (AI related or not ) for which I actually stop everything, sit down and look at the screen. And it's always worth it
That is high praise thank you so much nefaristo
Something that really jumped out at me this video was what you said about OpenAI waiting for the election to be over.
I think you are right and it makes sense for other companies, like Anthropic, to do the same. But then that led me to think about that letter of pause and how everyone thought there was no way any of these companies would do that. I know it was referring more to internal training work, but I still think its important to point out that there are forces, be they market or governmental, that can change the behaviors of these companies and impact development. I don't know what we do with that information, but my brain did take note of it for some reason.
If Lama-3 get's out and it's GPT-4 level or slightly more than they might want to accelerate timelines to stay relevant.
@@ParameterGrenze sure, and the ball of AI progress is already rolling down a hill and gaining momentum, it's just nice to see forces that can change the slope or put something in its path to slow it down if need be. Not that I think it needs to be slowed yet
In politics, if it can be done, it will be done in my experience.
I strongly agree, the pressure is getting exponential, no way an election will stop Google for scraping for their life and put out everything they have
I strongly agree, the pressure is getting exponential, no way an election will stop Google for scraping for their life and put out everything they have
Always top tier analysis. I can't help but feel mildly apprehensive when I picture the capabilities of the next generation of frontier models.
15:15 this is pure anecdote and I did not test it scientifically, but I’m doing a lot of coding with gpt 4 and several times the results it gave were worse when I half-assed the language in my prompts. Like, when I write extremely casually or use curse words.
This makes sense because the model answers on the level of the user - if I use complicated computer science lingo, the model is smarter than if I talk like I’m a first year student or so.
Yes, this is for quick, non-critical tasks onky
Thank you for continuing to keep us in the loop and providing your evidence based interpretations and speculations.
If only the rest of my time on TH-cam would feel as authentic and professional as this…
Thanks so much Hargol
As far as I'm concerned, this is the only AI news channel worth following on TH-cam. Massive kudos to you for the depth of your research, careful approach, and respect for my time.
David Shapiro
Same here in Australia. Philip does the basic research for us.
lmao be serious @@hutch_hunta
There are studies that show that humans also just make up their reasoning after the fact. So they also can't accurately report their reasoning steps or motivations, because that mostly goes on in the unconscious. You can see it in action if you are sleeping and there is a sound and your brain seamlessly incorporates it into the dream. We are doing that all day long with our own decisions. The decision making happens on some lower level and then we come up with the explanation afterwards. We are so good at it that we don't notice it isn't true, just plausible.
Yep
As best we know intuition/automatism/epiphany is our default functioning, and actual thinking/reasoning/pondering is a wholly different, much costlier process called upon only when things are amiss.
What a fantastic example of tying bits of info together. Thank you!
Amazing content as always. I'm also with you regarding Open IA releasing gpt5 at the end of the year. Let's see if multimodality comes for real this time
I missed your channel but appreciate it breaking through the noise!
Such a good insight about dalle 3’s lack of omission training data! I’ve been pondering that question for a while (in relation to similar examples of hamburgers without cheese or fried rice without peas), and my best guess was that most/ all of images in the training data included these details so it implicitly learned that hamburgers always have cheese. Your explanation makes more sense tho! Thanks for another great video 🎉
PERFECT video. Within the first 30 seconds, you told me what I was going to get, and so i stuck around to get it.
Thank you MrBeads
Thank you once again, Phillip. Every single one of your updates is well worth watching, from beginning to end. And speaking of 'end,' I deeply appreciate your comment near the end about relying on evidence rather than engaging in speculation. It makes a welcome contrast with the push messages about AI from other sources that I get on my phone every day. #integrity
Thanks so much Clay, appreciate this comment and all the others over the months
GPT has been such a game changer for me. Giving more confidence to take on projects, even with doing some hobby machining it gives good advice. Looking forward to GPT-5.
Those DALL-E images "with no lampposts" were AI equivalent of "Don't think about polar bears!" :)
Thank you for this comprehensive material!
GPT5 will use the output of earlier discussions between humans and GPT4 and the entire internet will be full of opinions and reasoning about GPT. This version will have a totally different situation; it will get to know about itself from earlier versions of itself, it can train itself into the future, leave tidbits of information scattered around the internet that will mean something for itself but maybe not for us.
Thanks, I was gonna sleep tonight.
That could actually contaminate it badly, leading to serious problems
The lampposts cracked me up 😂 another awesome video!
11:21 Unfortunately, Dalle doesn't have a negative prompt option.
To everyone wondering, he's Patreon page is loaded with valuable content. If you are a researcher, developer, a founder, a data scientist or just an AI enthusiast, subscribing is a must. There's no one putting this quality work out there. I'm a recent Patreon member and I got my money's worth!
100%
Thanks P. Now I have to go back & re-watch the 'let's verify' videos again for homework.
Honestly, you hit the perfect note between treating us as neither AI experts nor 'general public'. Your subscriber base must be made up of the knowledgeable &/or educated. Quite the demographic to reach here & it is deserved. Thanks, ofc I'll have to watch twice, especially those examples.
Great news!!! In my opinion, the release of GPT-5 will depend on the success of the Gemini Ultra model.
Incredible summary, thank you!
Wow, great video Philip.
At first I was skeptical about GPT-5 being released only in november, as I thought OpenAI would rush to launch their next big model as soon as possible, but then I started thinking "what if the big companies are actually coordinating on it?" I mean, OpenAI, Microsoft, Google and Anthropic did create the Frontier Model Forum, so they could be talking with each other when is a good time to launch the next-generation models, and they all might've agreed that a safe moment to do that is after the elections.
What makes me think that is 1) the comment from Dario Amodei shown in this video, as it looks like he's not trying to rush at all, and 2) the fact that Google hasn't still launched their GPT-4 competitor (Gemini Ultra) even after almost 2 months of its announcement, probably because they're still doing RLHF to get the model safer, something that they would obviously not do if they thought GPT-5 was right around the corner. Google is only been able to focus so much on safety because they feel like they have sufficient time, maybe because the FMF companies agreed not to force new model launches so soon.
Waiting until november still looks a little unbelievable to me though, but if they actually coordinated this wait, then I'd feel way safer because they prioritized responsibility over making money fast.
It makes me question something though, if we're not gonna see any new big model for most of the year, then what will be this year be about? Could 2024 be the year of small models? There were reports that Microsoft created a dedicated AI team to train powerful small models, they're probably training Phi-3 right now. Could this year lean more towards open source? Google just made a deal with Hugging Face, Llama 3 is coming out in the next one or two months, Mixtral 2 might be the first open source step towards achieving GPT-4 performance. There's so many things that can and probably will happen.
Anyway, amazing video Philip, your content is really thoughtful and interesting.
Thanks Gabriel. Phi 3 will be big for sure, llama 3 close to gpt 4, while gemini ultra in some ways beating it. Then so many outsiders trying to be contenders. Still so many unknowns, I think it will be a crazy year,indeed it mathematically ought to be.
They are certainly coordinating, they don't want the public to have access to anything too powerful not for the sake of safety, but for the sake of monopolizing power.
@@aiexplained-official yes if llama 3 is out openai will be forced to accelerate
@@esimpson2751I dont think they are coordinating. Google is one better model away to kill openai
We will still see plenty of updates tho, as you heard in the video: all the ‘save points’ in the shape of GPT 4.2, 4.5 etc
AI Explained doesn't miss! Another great video.
Thanks Matt!
Thank you Phillip, great blast
Thank you for your insights and analysis!
Thanks so much Crue! Too kind
Yea boiii, new vid on a spicy topic, thank you for your hard work(and thank you for reading all these papers for us lol)
Thanks Words
What I'm curious of, is when LLMs will decide to ask questions instead of answering, in order to get more context from the user and making sure they answer the question correctly. That would be a real sign of deep understanding of the conversation. Who knows, maybe we'll see that arrive in GPT-5. Anyways, I am hyped for this.
It does already. At least to me.
If it does, it's prompted to do so in the system prompt by OpenAI, in the ChatGPT UI. Because in playground, which is the raw API, it does not do it.
This bugs me, because if LLMS truly predict text, they would predict clarification requests!
Were discussions excluded from the training sets? Did they RLHF the heck of it to not hurt fragile egos? Do questions evade training? Do LLMs not see ambiguity?
ChatGPT's answer, after some wrangling:
“the decision not to explicitly ask for clarification is rooted in design choices and trade-offs. it helps maintain a more natural and fluid conversation.”
add custom instructions. Mine are:
Respond as an expert in the subject being asked about with a good level of detail. Ask questions about context or intent that you think might be missing.
@@musaran2 Someone needs to build a large set of training data built around asking questions back, or at least saying when it doesn't know. I kind of doubt that exists in the volume it probably needs to.
Sora says "Hello, World!". 🙂 Really looking forward to your deep dive on the new Text to Video system!
Another great one, Frank! Thank you!
Thanks Jim!
About the lamppost problem, chatgpt states that it does not contain any lamppost but chatgpt never analyse the image, it just creates a prompt that states to not use lamppost while DALL-E does not really support exclusion of that kind and instead just see it as an inclusion command.
If you instead have asked chatgpt if the image fullfill the requirement, it then for the first time would analyse the image and state that the image does contain lamppost.
This problem is due to a shortcoming of one model and a resource saving of the other.
Loved the Moloch reference.
Is that a reference to "Meditations on Moloch" I hear at 17:30? Knew you were a man of culture, Mr. Philip. Shame us SSC readers don't otherwise have a secret code for recognizing each other.
Great essay, but damn it is very depressive...
The more I think about LLMs on a philosophical level, the more I understand why they really might lead to AGI. The main reason is in the "Step by step" paper. We can talk to them, we can understand each step, single it out, and this allows iterative development, which is truly important in an incomprehensible black box system. It won't be "optimal intelligence" but it might easily be the fastest way to getting to it, simply because we as humans understand it, and now have the compute to do this inefficient way.
I'm excited about how much smarter that interaction could make us. Imagine actually reading through/listening to detailed logical steps on a variety of highly salient topics on a regular basis. That level of exposure is sure to impact the quality of thinking for at least some of us
I'm excited to see a model in the future that completely operates through relational reasoning similar to what is described by Relational Frame Theory. If I recall correctly, something similar to this should become possible once hyper dimensional computing models are fused with LLMs
Once I understood the concept that these neural networks function like the neurons in our brain, I knew it would lead to AGI. There's no doubt in my mind. The average person has no clue what is being created and the tsunami of change it will create.
@@devon9075 That's right. Some time ago, many people seemed to be worried that AI would dumb down the whole of humanity because all thinking and learning would be outsourced. However, I now believe that the opposite will be the case. The quality of exchange that AI will enable in all areas of society, especially in education, will improve the performance of individuals on average.
@@ChannelHandle1 LLMs are like a magic glue that can stick anything and everything together. There are hardly any restrictions in sight.
I noticed the quirkiness with DALL-E 3 some time ago: I was getting pictures of men with beards, but saying 'get rid of the beards' would not work. If it sees 'lampposts' (or anything really) in the prompt, it can't not use it in the picture. The trick is, saying what you want, without mentioning the thing(s) you don't want to see. Like asking for clean shaved men instead of no beards.
Thanks for another superb video, Philip! Could you please elaborate more on capabilities prediction in your next videos? I'm interested in specific predictions for GPT4 (fulfilled or not) and GPT5 predictions.
Thanks cup, will do in enxt gpt 5 vid
The scrammbled text capability was very impressive (and reassuring regarding my bad spelling)!
Great and very informative video, keep going!!
Thanks Fahim!
the lamppost bit was hilarious 😂😂
i am both very excited and scared of GPT-5 😮
Fantastic video as always, looking forward to seeing more from the interview on AI Insiders!
Great video. you are the best out there by mile
That lamppost thing is the bane of my existence. Ai does a thing, I tell it to stop doing the thing, & to focus on what I actually want, & the AI will just do the thing harder
About lampposts: this reveals an interesting property of how GPT prompts Dall-E. It’s clear that GPT is not adjusting any of the settings for how to generate the image, such as number of steps or negative prompts. If GPT were given more access to Dall-E’s settings, it could more precisely craft the image generation - assigning a negative prompt weight to “lampposts” for example.
10:00 I love to see someone rediscovering the legendary "Not" trick
Haha
🇧🇷🇧🇷🇧🇷🇧🇷👏🏻, Wow, such great news! You mean it’s even going to be better. I am amazed. Great video as always!
Nice video, thanks :) Subscribed.
Another great video!
Thanks cow!
I talk with it as if I'm talking to a person. I don't suck up to it. There's always a thank you at the end of my question of whatever I'm talking/asking about. I do not patronise. I talk as if it is my assistant, Ms Moneypenny.
This channel is so valuable! I fucking love it! Subscribing to AI Insiders right now!
Amazing, thank you! Podcast episode 3 out this morning, new video on Friday!
@@aiexplained-official. Great! 🚀
Honestly, the whole "laziness" thing with GPT-4 is mostly a problem of prompt engineering. I never had any problems, but I also spend time being clear with what I want in which way to get the best result. The only laziness comes when the context window gets full (in code generation specifically) and it kind of tries to keep everything in its context window, so it starts to omit irrelevant parts of the code instead of giving back all the 100 lines of code.
not truer, gpt4 tends/tended to omit code even aftre being told not to
"Prompt Engineer" has the same energy as "Ceramic and Cutlery Hygiene Technician", please shut up.
@@SmellyHam Then call it Prompt Developer. But you clearly did not understand that a LLM is a tool that can be used effectively or not. Changing the prompt can change the translation quality for flashcard generation from 50% to 90%. I spent around 5 hours on the prompts. Prompt developing is not a joke.
This was a fantastic video
I love you
re: Excluding objects from images --- at least in Midjourney, they tell you that the interpreter mostly relies on keywords, such that "no lampposts" becomes "lampposts" (which is why they show up _more_ frequently the more enthusiastically you demand they don't). To solve this, Midjourney supports a "-no" parameter -- any keywords following "-no" are negatively-weighted, making them less likely to appear. So, "draw an image of London but don't use any lampposts in the image" should have been "draw an image of London -no lampposts" (again, at least in Midjourney!)
IIRC some people can full phrase prompts “boomer prompts”.
I suppose that will change when image generators get a better understanding of what they draw.
its amazing how you pick up on stuff no else notices
Thanks hans
I always corrected typos because I thought it might help set the tone of the conversation as more serious, professional and accurate, which I hoped would bias the model towards producing higher quality output. I know that LLMs, at least primitive open source ones that I can run locally, try to continue the conversation in the consistent style, so for example when attempting roleplay, if user answers in one liners, AI is more likely to answer in one liners as well.
Yes this is more for quick mon-ceitical queries
@@aiexplained-official lol.
that was unintentional but kinda illustrates the point haha@@jaysmith3361
"Your message contains several typos, making it a bit unclear." That's a very polite way to say "why the hell are you scrambling your text?"
Haha indeed
Or it genuinely is only a bit unclear to it.
I actually quite like the laziness of GPT-4 in certain contexts. I've been using it to help me code in different languages so when it doesn't do everything for me it helps me to learn and apply my own reasoning rather than having everything done for me
Thank you!
You're amazing sir
Thank you
Always look out for your videos!
Thanks Pearly
Lol the lampposts weren't only there but they were the main subject of the composition.
Haha
It's like DALL-E was deliberately trolling :)
Another masterpiece of real news and clear, methodical explication.
Thanks! Brilliant content, as always. 🙏🏼
The lamppost example reminds me of our own issues dealing with someone saying “don’t think of an elephant”….
I'm loving this. It will make my programming so much easier. The only problem i will for sure be replaced completely by ai in 10 years and maybe one human dev checking everything the ai put outs but that will be a more competed person than me
Thank You !
Imagine working on a project where you say we have no clue how this is going to turn out. We are embarking on this journey as a species as all of humanity, and you'll know the answer within like 2 months. That's just crazy exciting! 😅
- Understand GPT-5 aims to harness all computing resources and scale beyond precedent. 1:10
- Recognize GPT-5's development includes safety testing with "Red Teamers" in place. 2:13
- Acknowledge that GPT-5 plans to improve reasoning and reliability by reviewing reasoning steps. 4:18
- Learn that GPT-5 might think for longer to lay out reasoning steps before solving a challenge. 4:49
- Note that GPT-5 is expected to incorporate more granular training and deeper pattern recognition. 9:03
- Recognize that even with advancements, GPT-5's internal calculations may remain somewhat inscrutable. 13:01
- Expect significant improvements in GPT-5's multilingual abilities due to increased data partnerships. 13:30
- Save time and trust in GPT-4’s ability to understand typos without corrections needed. 14:49
- Predict GPT-5's release towards the end of November 2024, considering the lengthy testing period. 15:22
- Prepare for GPT-5 to possibly redefine industry standards as hinted by OpenAI employees. 19:27
This video deserves a subscribe.
Yay
11:19 lampost prompts were so hilarious 🤣🤣
Love the video
The "streetlamp" problem you encountered in Dall-e 3 also exists in latest GPT4-Turbo.
I tried to get it to code me something an it inserted underscores instead of dashes into a particular package name.
I asked it to correct the underscores to dashes and it again typed in underscores, then claimed it had corrected the code, then went on to tell me that it's "unusual" for a package to have dashes and not underscores..
To see how far it would go I carried on this back and forth, It took about 7 go-rounds before it spat out the correct code.
I think GPT-4 has gotten much dumber recently.
11:35 this is one of the ways that atble diff is superior, as we have a negative embeding , where you would put "lamppost" and it would ommit them
Feel the AGI Feeeeeeeeeeeeeel it.
being able to guess next words and doing search on the internet isnt AGI.
@@dan-cj1rr😂😂 get off this channel mate. You’re not intelligent enough. If you don’t have a degree in computer science atleast keep it moving.
I doubt GPT-5 will be an AGI. Where abstract and logical thinking goes even the best LLMs are still far below a person. Eventually we will have something like an AGI, probably just not for the next few years. They have a lot of information, but when you tell them to do something slightly complex you can see them begin to fall apart in a way a human wouldn't.
@@jacobnunya808 correct
@@dan-cj1rrYeah but it can definitely be the base for it. Agi, like humans needs an absurd amount of context, and Llms have proven that having words/language as the base provider for that context can do wonders for making it easier to introduce new modalities. It’s not agi now but it’s looking like llms have the most potential to be at the core of future agi.
LLMs have a "don't think about pink elephants problem" which causes gpt4 to prompt dalle with "no lampposts", and dalle can't help but put lampposts in it.
Dall-E 3 has the same problem of LLMs, the spilling over of words taken out of context and influencing the whole creation, poem, story, reasoning or image
11:30 - yup, definitely no lamposts there 🤣
Great video! I think they’re right we need to see more adoption this year so that in 25,26,etc. we have the infrastructure that can quickly take advantage of newer models.
Thanks KTW
@@aiexplained-official Np have a nice day.
Really cool analysis without the typical clickbait associated to these news and AI YT channels... About the topic, to me, trying to replicate human thought when, we humans, are still unable to define the mind and how it really works (where is consciousness coming from, etc) is a bit of an error. Perhaps improving AI reading capabilities, making it able to work on extended projects keeping present always a broad context would be better. Nevertheless, I understand that this may already be possible on small context but not technically possible to deliver it to millions of users at the same time.
🫡
i have stopped using the internet and social media bar your channel. you are very good at what you do. please keep up with the videos and continue to steer clear of clickbait and over hype! 👍
Oh wow that is amazing to hear Jack, honoured
Yet another great video! The wait is always worth it 😊
Thank you Hedgehog
14:42 “For me and you, that would be almost complete gobbledygook.”
I was surprised that I could read it without too much difficulty. (It took a few seconds per word.)
It’s interesting that the authors of that paper don’t really venture a guess as to _why_ GPT-4 will easily decipher those scrambled words. (To me, there’s a difference that it _can,_ say, on demand, and that it just _will,_ when presented with that input. I guess the “pull” to construe input as having the most plausible meaning is strong in the dataset.)
It's about to get real silly
10:40 It looks like Gemini Ultra sometimes passes the lamppost test.
It seems to me that adding some form of recurrence to chatgpt could help it fact check itself on logic and make better plans, if it could internally generate a draft response, then reflect on it before generating a final output response it may be able to pick up itself on many of the errors it made, correct them and maybe even make the final response more consider and efficient in terms of word count?
Just made a video on that on Patreon!
Seeing that you declare a love of perfect English, your non-standard pronunciations of definite and indefinite articles are remarkable.
You're not alone in that. Some well-educated scientists are among those who are also guilty of this heinous crime :)
Didn't realise I am non standard in that respect, how so?
Regarding typos, I dont even bother writing full sentences when using LLMs. I type fast and messy, and most of the time the model can understand just fine.
15:13 chat gpt can also usually work around typos
Excellent thought about the release of GPT5 and the Elections this year.
This channel is 🔥
Thanks so much ross
I have dyslexia. GPT-4 struggles with my language. I have entirely dropped typing with Chatgpt.
My gut tells me that GPT-5 *finished* training just before they opened up custom GPTs for subscribers. If you think about the resources required for a training run versus an influx of subscribers (remember they paused that for a minute) would indicate to me that the training run was complete and then they started the safety work, which is less computationally expensive. GPT-5 at Easter this year is my prediction.
at 10:24 thats like the 'dont think about the pink elefant' joke/experiment
Indeed!