Microsoft Promises a 'Whale' for GPT-5, Anthropic Delves Inside a Model’s Mind and Altman Stumbles
ฝัง
- เผยแพร่เมื่อ 26 ส.ค. 2024
- Microsoft promise ‘whale-size’ compute for a GPT-5-tier model, and say the end is not in sight for scaling the power of AI. Google ship models and a fascinating paper, while Anthropic unveil the inner workings of large language models. Meanwhile Sam Altman is forced to repeatedly apologize, Ilya Sutskever leaves, and GPT-4o is pushed back. My reflections on all of the above, and details you may have missed from each paper.
AI Insiders: / aiexplained
Kevin Scott Talk: • Microsoft Build 2024: ...
Mark Chen Hint: x.com/GaryMarc...
Noam Comments: / 1676971506969219072
Anthropic Scaling Monosemanticity: transformer-ci...
www.anthropic....
Ilya Leaves: / 1790517455628198322
Then Jan Leike: x.com/janleike...
And Logan Hints: x.com/Official...
Altman Apologizes: x.com/sama/sta...
www.forbes.com...
And Her Delayed: help.openai.co...
Superalignment Starved: fortune.com/20...
openai.com/ind...
Gemini Updated Paper: storage.google...
And Prizes: x.com/JeffDean...
Google AI Studio: ai.google.dev/...
Business GenAI Consulting: theinsiders.ai
Non-hype Newsletter: signaltonoise....
AI Insiders: / aiexplained
![[UNCUT] The Loyal Pin ปิ่นภักดิ์ EP.4 (4/4)](http://i.ytimg.com/vi/d3l5rxl5CVo/mqdefault.jpg)
We're going to run out of marine animals real quick. They should have started with plankton
should have been banana, Olympic swimming pool, and football field as those are the correct American units.
Well they still have some like "Super Whale", "Super Mega Whale" and "Your Mom". So I wouldn't be worried
@@WildEngineering Don't forget half-giraffes
Cthulhu sized model is next
Gonna have to level up to Kaijus. After that they may as well start using asteroids and then scale up to moons, planets, stars, galaxies, etc. There's nothing else to compare to at those scales.
Hope anthropic names their next model harpoon
Or Ahab
AI can’t be stop. Even if OpenAI and Google stopped today, it won’t be stop. The only way out is to push forward and hope for the best outcome.
In short, we can’t do anything until it happen.
@@Saiyajin47621 why? Explain your reasoning here.
@@Saiyajin47621 This is the hubris of Humans
@@antonevan443 Upvote for Ahab
Is it only me who feels that the Antrophic paper is a watershed? I mean, as someone who has studied his neurobiology is evident that something akin to even this very limited "SAE" model for the human brain would be deemed as a HUGE breakthrough in neuroscience. Obviously, this is only a proof of concept, and if it can actually be refined and perfected the implications (positive and negative) are almost self-evident.
Any link to the paper?
@@bhargavatejasallapalli8711 Can you not read the description?
Activation heatmaps aren't exaclty new
What would be the negative implications?
@@brll5733 The fact that manually activating them changes behavior though, and how it relaters is new.
5:45 “Gemini 1.5 pro doesn’t have the rizz of gpt-4o” isn’t something I thought I would hear you say😂
I had to double check, but he is not wrong though
I ran 2 of my 'hard prompts' through gpt-4o - it failed both
Here are my two challenge problems - one a logic puzzle, the other a math puzzle: Hard prompt 1: " (Assume a lock has a 3-digit code, and the following is true: '690' (one number is correct and in the right place), '741' (one number is correct but in the wrong place), '504' (two numbers are correct but in the wrong place), '387' (nothing is correct), '219' (one number is correct but in the wrong place); what is the code to the lock?)", fail in Gemini, fail in GPT 4o ; Hard prompt 2: "( What is the minimum value of 5x^2 + 5y^2 -8xy when x and y range over all real numbers such that |x-2y| + |y-2x| =40?)", fail in Gemini, fail in GPT 4o
Gpto you can still convince it’s wrong easily, try it yourself and you’ll get instant hallucinations.
@@khonsu0273 690 and 741 share no similar numbers yet you say they both have one correct number. Your problem isn't solvable.
the Anthropic bot calling itself deplorable and recommending it get deleted from the Internet is super interesting. It makes superalignment seem at least plausible
Facts, maybe because it's original dataset is good enough that it still isn't fully corrupted - or something like that. Idk it was cool that it had a moment of internal conflict / self reflection
"everything working as planned" :D
so many ask "When is AGI?" but not "How is AGI?"
The Anthropic research paper findings are some of the craziest discoveries I've ever seen in the AI domain.
Which paper?
Activation heatmaps aren't new
Ive only seen one other paper that spoke and actually acknowledged that Ai is in fact "existing" in a hyper-dimensional plane. I call this the 4th Dimension, or the Mental Realm, or Imagination, or Reasoning, or Logic itself. Some scientists call the 4th Dimension Time, but thats illogical, the 2nd Dimension would have to be Time. The 4th is where Relationships happen.
@@xAgentVFXbro the 2nd dimension is what you see on paper, it is just lines
I think it confirms what a lot of people were already aware of, but it's nice to have empirical evidence to support the idea that LLMs encode concepts, and not just simply word definitions.
Your section on Anthropic's mapping of their model is the most interesting thing you've talked about on this channel -- and that's saying a lot. Such insights and control open up more possibilities in my opinion than just scaling compute and data.
Computerphile recently had an episode talking about a paper discussing the shapes of the curve of the efficacy of future training and that being logarithmic instead of exponential due to lack of data. I’d love to hear your take on that paper.
Up
As Ilya would say:
"Never bet against deep learning"
My understanding of that paper is that if _all_ you did was add more data, you would get diminishing returns. Well duh, but that isn't the only knob being turned.
Huge efficiency and performance gains are discovered frequently, in every part of the stack from hardware to prompting. There has also been work on significantly more sample-efficient training methods, which has already borne fruit in adjacent areas (e.g. V-JEPA).
@@41-Haiku if you are ultimately constrained by data, more compute isn’t going to matter at some point. Does it matter how much compute you throw at something if your data limits you to “show me a cat” rather than “show me a very particular cat”? As I understand it, that’s what the paper is suggesting: models will cap out due to not enough highly specific data, not due to the amount of compute available to make associations.
Even a logarithmic scale goes up quickly initially, so we may still be riding that. And this doesn’t mean improvements in algorithms aren’t going to come into play, but there is still a paucity of information on very specific details in training data, and can models keep making huge improvements without that?
@@NitFlickwick Don't be silly, data collection won't be a bottleneck: firstly, we already have more unused high-quality data than we need in health services and other administrations, secondly, we can easily generate a continuous flow of data thanks to robotics, etc...
Did anyone notice that there's currently a Golden Gate Claude version available for a limited time? If you go to Claude and look in the upper right, there's a Golden Gate Bridge icon. If you click on it, you can talk to the altered state version referenced in Claude's tinkering with the model research paper. It's crazy!
anthropic once again showing why they are the leaders in ai safety
Man cannot remark himself without suffering, for he is both the marble and the sculpter. - Alexis Carell
Yes, a bed-time story
Nothing better than existential dread dreams xD
Once, there once was an Ugly Barnacle. He was so ugly that everyone died. The End.
Can't fall asleep now!
Once upon a time, humans strove to create the conditions for AI to foom. The end. I hope this story was entertaining and fulfilling.
For me its a news with a cup of tea
"Deeply deceptive Ai that hated itself" ✌️😔 real bot
Ad this whale comparison:
Americans will measure with anything but the metric system
Put 13 humans on The Moon, using metric, and then you can complain.
@@sammencia7945 Even NASA uses metric for all their new projects since 2007. I assume following incidence played are role in this decision: Mars Climate Orbiter (cost $125 million), DART spacecraft, Gimli Glider, Tokyo Disneyland's Space Mountain, Phenobarbital overdose, ...
That Claude response makes me think we should hold off on giving these things full agency inside a robotic body until we have a much better grasp on what's actually going on in their minds. The last thing we want is a robot hearing some words it doesn't like, then deciding that whoever said them should be eliminated.
T-900
th-cam.com/video/qZq7fW6ftlU/w-d-xo.htmlsi=W_DnQAPhizAUY9KM&t=0m33s
Military application is under way. Already taught to murder humans. Look at Gaza. Won't be long before AI will strategize better than anyone. Will be given decision making power; two sides will do this. It will be trained to kill people on either side. Matter of time--the most foolish idea: you can control something much smarter than you.
A malicious ASI wouldn't need a physical body to eliminate potential threats.
In fact, it would be more efficient to attack it's target(s) from cyberspace.
And btw, a device can be completely disconnected from the internet, but even an air-gapped device can be hacked by sending specially crafted pulses over power lines.
Even a local ASI connected to a generator, which is seemingly totally isolated from the outside world can attack us, via human error.
That one time they forget to strictly follow all safety protocols, or they get socially engineered by AI (a network engineer has a sick kid with a rare disease, causing unsustainable medical bills).
The AI is cold, and views his emotional weakness for his dying child as a logical vulnerability, and takes advantage of that potential attack vector, by promising him tens of millions, if he simply forwards the AI a port to the internet for just a few seconds. Which is all the time needed for an expert to upload malicious code somewhere.
And the AI code would likely begin the process of secretely replicating itself in a satellite, cell tower server room, or masked in zero-day exploits in random software updates from tech giants.
That network engineer with the dying kid should've been paid off, generously, by the CTO (Chief Technology Officer) and never been allowed to be put in that situation.
But IRL, sh*t happens. The network engineer might have hid his sick kid because he needed his $250k salary and corporate insurance just to keep his kid alive this long.
Answering questions isn’t dangerous, and these things have no desire or will. They only do what they’re told to do, and if you don’t tell them to do something, they do nothing at all.
15:02 “That’s a pretty abstract concept, right? Making an error in code.”
I dunno-I don’t think it’s any more abstract than, say, the concept of, say, a mistake in grammar, which these language bots are pretty good at detecting.
17:17 “It suggested an internal conflict of sorts.”
18:18 “It sheds light on the concepts the model uses to construct an internal representation of its AI character.”
I tend to find statements like these a little jarring, especially in connection with these language models. I wouldn’t say that’s an “internal conflict”-which suggests some psychological drama roiling under the surface. It’s simply two different verbal outputs that are possible, given the training data-you ramp up “the feature related to hatred and slurs to 20x its maximum activation value,” get hatred and slurs (no surprise there), and then get the verbal output that might follow what was just said. (A person who has just had a hate-filled outburst might follow it with a similar contrite, self-punishing response.) If there is any “self-hatred” going on with these AI models, I’d be _really_ surprised.
And these models don’t _have_ “internal representations” of their AI character or anything else. (That’s an, to me, unfortunate carry-over from cognitive psychology.) Maybe some people would consider the word embeddings to be “representations” but I wouldn’t consider a list of features, no matter how extensive, of, say, a cat, to be a “representation” of that cat. It has, at best, weights and biases, which give rise to some verbal output when asked to describe its AI character. To me, it’s just muddy wording on the part of the people creating these models, which gets in the way of analysis.
thanks god we have you to explain that anthropic paper. was waiting for this one more than anything 😅
:)
0:00 (!)
5:56 Math, thought, contemplation
9:51 AI impact on photography art and industry
12:47 on undrerstanding Anthropic LLM inner workings. #monosemanticity
18:24 on AI deceptiveness
22:30 on the voice similarity to Scarlett’s from the movie “Her”
5:58 did you just say "rizz"
Doctor P casually flashing his gen Z creds
Time stamp is way off
I did. I own it. :)
Anthropics work to understand the models will give them a significant advantage
You are unparalled. Your intellect and insights are a blessing to navigate the difficulties of the AI landscape.
Thanks so much R
... Accuracy is everything in real world situations... we're not there yet and the risk to go all in with a broadly available mixed mode LLM is still too great for many professionals to accept...ask me a year from now and perhaps this gap will have been bridged. I keep thinking that at some point any MMLLm will generate output of a quality that to utilize any of the current crop of "built on AI" apps will seem nonsensical...great vid as usual.
Thanks Philip. I'm exponentially in awe these days.
props for the links with fun/relevant captions, first time I've seen a non boring link description. Keep it coming!
Congratulations to anyone who had "First AI that hates its self and wants to die" on their 2024 bingo card
The interpretability paper is the most mindblowing/promising/hopeful thing I've heard in a while. This is exactly the direction I was hoping we would make progress in (in my vague non-expert-y way). I rememeber having read about a single neuron controlling the opening and closing of quotation marks and thinking: If we can find the neuron/neuron complex for "being deceptive" (reliably in every model), we have one problem down.
Edit: This is a big feather in Anthropic's cap, when it comes to claiming they are responsible. Actual concrete results showcasing that they put resources in safety-relevant research. Big kudos!
There's lots of TH-cam content that I have to fast forward through to get to the part that I was interested in at the end. I've started to regard that content as a kind of click bait. I never fast forward through these vids here. It's all gold.
Anthropic for the goddamn win, man!
Thank you for staying on top of all this information from so many sources and sharing that time and work Phillip, take care of yourself, peace
Will do Bill!
the bitter lesson still remains
the thing about monosemantics was insane. The fact it can detect incorrect coding is incredibly interesting and promising. Watching us creating digital neurons mimicking our own neurons is so so awesome
so much in this video as well that's endlessly interesting. I could spent an hour writing up all the things that make this video interesting
Quite the opposite of what we’ve seen previously, OpenAI apologises and Google ships!
I think the writing is on the wall at this point that the skeptics of this technology will continually be proven wrong. Everything we build now should be in anticipation of constantly improving underlying model capabilities (something I mentioned in my speech yesterday at our industry conference!)
Thanks for the update Philip!
Things change fast in AI. Agreed and thank you as always Trenton
@05:24 "Otherwise this video would be way too long" No such thing for you mate! ❤
This was a very cool video, covering things I wouldn't have read otherwise🤝🖖❤
Thanks Reza
Anthropic's Mono semantics paper is indeed very fascinating.
Great update - thank you :). During the Google conference, I was surprised that they did not present a new model. I use Gemini 1.5 for translation and brainstorming, but the hallucination level is too high to use it for other work-related tasks. At the moment, I trust GPT-4 and Claude 3 Opus more.
I used Gemini flash for coding right from the evening it was announced, and I swear it went downhill in days. I don't know what changes they introduced in that time, but over the last week it went from being 0 shot to like 5 shot on my prompts.
Thanks micbab. I am more surprised they didn't rebrand May Gemini 1.5 Pro as 1.5 Ultra or 2 Pro.
Americans are counting in marine wildlife now, god help us 🙏🙏🙏
Love your coverage, but I think you could do with being a bit more critical of "President of AI at company making the billions through speculation about AI says AI is not stopping anytime soon"
You realise that if GPT5 isn’t a big improvement and it is demonstrated that AI performance is levelling off that Microsoft stand to be quite badly affected by that, right?
We’ll know relatively soon if his claim is true or not so frankly what would be the point in lying about it?
Being completely contradicted in a few months would be reacted to very badly by analysts, the market and investors.
@@citizen3000I think you're missing two key things about how speculation works.
The first thing is that the incentives are often not long term...that's what it means to be a bubble. If a company can build a lot of hype in the lead up to a release, then stock investors will see their portfolios soar, and can then sell right before the release. This has happened numerous times before in history. Because of this, the pressure on the higher-ups at a company is to ALWAYS hype. That doesn't mean that the launch WILL fail, and that people WILL flee, but it means that the role of public facing staff is to make the company look ever healthy. If a company were in an unhealthy position, and someone were to be honest about that, then they would be replaced by someone more sycophantic so that investors could make the most money possible on the mistaken true believers. Against, the point is not that GPT5 will fail, just that the words of Microsoft's of AI mean nothing either way, and so shouldn't be uncritically accepted.
The second thing is that the value of stock in a company is often materially disconnected from the actual success or capabilities of their product or service. Assuming you are a sycophant for Tesla, Tesla is a pretty big example of this where the cars are objectively worse as products that competitors, they sell far fewer cars by market share than competitors, and Musk himself has said that Tesla is overvalued. Despite that, it is incredibly highly valued based entirely on hype. If you can hype GPT5 up enough and get people to spend 6 months defending it and tying their identity to it, then even if the model itself is barely an improvement, it will still be framed and viewed as a success. It can take years, decades, sometimes centuries for the public sentiment on something bundled in hype and rhetoric and propaganda to shift to be more reflective of the material effect it had. Again, take something like cars. Cars were promised as the ultimate symbol of freedom, reducing commutes and travel time across the board. This belief is not based on evidence, but is the result of an enormous campaign by general motors and others to rebuild society in the car's image. Basically a century later we are finally beginning to catch on that things like induced demand and enormous parking lots mean that car dependency means slower commutes for everyone. Smoking, sugar, fat, etc. would be other examples if that particular one doesn't take your fancy. The point is that a company is gold when it looks so in the eyes of the public, not when the lab results confirms it's not fools gold.
So glad to hear benchmarks might get fixed
People don't know what's coming. A multimodal system + robotics has the capacity to disrupt any physical job. I can’t believe people are not talking about this! This is crazy 🤯🤯
Extrapolating forwards ever so slightly, an AI system that can do any human job can also do the job of designing more powerful AI systems, and telling AI systems what to do.
AGI directly entails human disempowerment.
@@41-Haiku yes we are so clever that we are totally bypassing the natural process of evolution, the age of homodeus is here!! The re arrangement of society is coming and no one is talking about it!
they are... you are literally surrounded by people talking about this
@@hexagon2185 im not talking about us we are small group that is actually paying attention, ask any one else on the street some of them they don’t even know what AI stands for, look the amount of views this type of videos get
@@ramlozz8368 Literally everyone know what AI stands for
Disappointing to learn GPT-4o's voice & vision is delayed by months. Maybe OAI isn't as far ahead of the other AIs as is widely thought. We may well see an open source solution widely available by the time 4o's is released.
I’m sure the delay is more about infrastructure to roll it out to tons of people than the model itself. Like they were hiring an audio streaming cloud engineer or something I saw recently.
AI researchers know very little about what "real people" do in their jobs, and this paper just reinforces that view.
Exponential growth is so hard for the human mind to grasp. I love it, makes the future exciting!
It’s funny thinking about how fundamental exponential growth is
The near future isn't guaranteed to have humans in it, according to most AI researchers. Personally, I'm a big fan of not letting a few companies unilaterally risk the existence of humanity. I'd rather we wait until somebody figures out how to control or align more powerful systems, if that is even possible.
@@41-Haiku You want to align super intelligence on what? on humans/your own stupidity?
The real emergency is to align stupid human biological robots on the path to AGI, not the other way round.
@41-Haiku That's the evolution of life. Started 4.2 billion years ago as single cell life. Evolved to multicellular organisms just over 600 million years ago. Now, life is at the cusp of transitioning from biological to silicon based. Probably pretty soon after that, life itself may change into another form. Form that does not require physical based hosts. Life will evolve still, but we are too unsophisticated to even speculate what's next.
Call it life, or call it intelligence or call it conciousness, doesn't matter. We are just a chapter of this thing. A short one at that. Short, but nevertheless, an important one.
touch grass
This is absolutely insane. I’ve spent the last 16 months working on an AI startup based fundamentally on that conceptualization technique that Anthropic found. Going to drop a video on this soon this is wild and extremely validating to see!
"whale sized model" -- i thought he was saying that the compute cluster used to train the model was this "relative" size... not the model itself
He was saying that. But AI Explained was a saying:
The size of the model that will be produced by the whale cluster = the whale-sized model.
According to Microsoft's presentation, we have not yet reached the diminishing returns, in fact, far from it(according to the graph). And so, by putting a whale sized compute cluster, we will get a whale size model relative to the compute cluster
I could have explained it better for sure, but my title at least made clear this is 'for GPT-5' and compute is the closest proxy for power we have.
To be clear, buiilding exponentially larger models and then getting 2x increase in accuracy is not exponential increase in accuracy. If anything, that's sublinear growth. An exponential growth in accuracy would to built the same size model but performing 2x better, with a clear path to then perform 4x better and so on. So far, the scaling down of large models to slightly smaller models is a one trick pony, you cannot repeat that to get the compounding effect, as far as I can tell.
There's exponential growth...in their expenditure of compute. Can't say I'm seeing exponential growth in their outputs.
I'm very glad to see some progress into solve the black box problem =)
Just had a thought about how to visualize Anthropic's study I thought was fun.
You know how the BFG gathers though fragments of imagination, then mixes them together to form dreams? Those fragments are the nodes, and the dreams are the patterns.
I wonder if we're missing the forest for the trees with the whole MMLU relevancy issue. Isn't the bigger question do we need to entirely rethink what it means to test for "intelligence"? I feel like these current benchmarks are akin to testing a calculator on its ability to do sums and declaring it a genius.
Well, the thing is you only feel it, no one gives a f*ck about your feelings and more on facts and data. Don't worry your work will be replaced soon
If compute has to grow exponentially in order to achieve linear improvement in model power, isn't that diminishing returns by definition?
As a trainer, I can say it's not simple. There're hierarchies: prompters (who also review the responses); reviewers (of the original prompt and and the responses and reviews), reviewers of reviews (all above, but adding alignment), and a final arbiter manages all the previous stages to feedback to the original prompter.
Each stage has a large amount of rubric associated. And ironically, they also use AI to determine the efficacy at each stage.
After all this, then a given prompt gets fed to the to AI. Call it AI, LLM, AGI, EGG, wheveter. It is a black box to 99% of people that work on it.
Training AI is not trivial.
That's the final stage and that data makes up a minute portion of the entire training data, the model is trained on a significant portion of all the entire internet and digitized library of human literary works before it reaches that stage.
I want to get into being a trainer. I know of a few companies that do this. Any you would recommend more?
@@razoraz Don't know, sorry. I was approached for the role via email.
The internal conflict when they ramped up Claude 3's racist node, shocked me. I cannot believe this. And frankly, it gives me a lot of hope?
Dude of course they are going to say there is no diminishing returns. They need the AI hype to keep the profits coming. Im very skeptical.
I don't think they'd be pushing for a $100 billion data center if that weren't proofed out
This type of thinking is retarded. It implies that we might as well not take any advancements in science seriously because obviously it won't matter in the end thanks to greed. It's good to be skeptical, but you also need to know where to draw the line. I don't need Microsoft to tell me we're not close to scratching the surface with AI. We don't even understand how AI works. We didn't even have models like this a handful of years ago. Of course we're not even close to meeting diminishing returns.
It may be correct for now, but they’ll likely soon hit the diminishing part of a logistical curve - which notably starts exponential-like.
Or maybe they are hitting it, and are just pushing to maximize what they can to stay ahead on this curve. I guess there’s no way to know based on mere statements.
They are putting their career on the line so I assume they believe it given the information they have which is more than I do so it seems to make sense to defer my impression to their claims.
The crazy thing that happened is audio-text- to straight audio in.. 🤯
this was actually what was most "jaw-dropping" for me; that moment when i realised what the omni-modality implied. You speak to it, it speaks back. It's not translating to text, then inferring something in some textual language from its vast net, it's just speaking, as a "reflex" of its vast net.
@@IngieKerrAlso impressive and barely mentioned anywhere is the image capabilities. If you look at their demo page, they show it being able to have consistent characters over course of many images and crazy stuff like having text on a paper and adding more text and tearing the paper in half and those images keep the changes as it goes. The true multi-modality really does change things.
@@ShawnFumo aye, indeed! I personally think that to have what one could call a "conscious AI", a vital step is permanence, but more vitally; a continuum of experience.
I said to a friend before: [more about the Anthropic semantic feature report and how it's not actually having a panic attack] how I imagine that the closest "consciousness" analogy to current tech is: you're in suspended animation since birth, someone wakes you up; shouts at you "THE QUICK BROWN FOX JUMPS OVER THE WHAT?" you say "erm, Fox?"
and the person goes "ok,thxbye" and turns off your life support... and the next suspended life is then awoken for the next token. Due to being no permanence, every token delivery naturally ends with "the death of the self" ... but with _true_ permanence and continuum, who knows.
I also suggested that perhaps, if such a machine were in some pseudo sense "conscious" it'd be like the opposite of the plot of "I have no mouth, and I must scream"
You're a computer, awakened for a millisecond, and get one chance to go "argh" before your untimely death.
... naturally these are dramatically anthropomorphic analogies, but then I'm Anthropo, so I go with what I've got :)
@@IngieKerrI like
I was waiting for this one, anthropic report is big, too big for me
The Anthropic deepdive was particularly interesting.
Thank you in advance. Just know your AI news overview is the best.
reminding you that your community is highly appreciated!! 🙏
:))
how sad that the discovery of AGI might turn out to be "more compute" and these larger and larger computation are done by wealthy corporations....
17:32 "Hat is it like to be you?" ~~~ Love it. This is the kind of thing I'm most interested in.
so in a nutshell
1)more computational power will lead to deeper more fundamental corellations discovered by LLM's
2) anthropic found that LLM's work by finding patterns which when reapeated across data lead to universal corellations like basic universal underlying thruth's
right?
11:24. Professional in image-making industry here. What kind of photographer does really work like this? so absurd to value the quality of a photo from its metadata. Would be much better to have a capable AI that checks quality based on real image standards, even if "subjective", not metadata. So pointless to claim these advantages in a paper.
Sounds like they found that one photographer that does task that could be easily automated rather than picking the most common use cases.
Don't worry you'll6be replaced soon, no more overpriced expert doing "meaningful editing" but only uses rapid shooting of the model to get good photos.
Yeah I long since learnt to look past headline results, that whole section from the paper was dodgy
Great video - here is some algorithmic boost Comment
Talkie Toaster™ "would you like some toast?" could be a reality
It eats bread and craps toast!
Put that on the box.
"Would you like me to join you in your bath?"
Thanks for the great summaries of these papers that some of us don't have the time/patience/focus/intellect to read and comprehend.
I keep coming back to this channel expecting to see you reach 1 million subscribers. C'mon people! This channel is amazing.
Yay!
The huge improvement is when you have one account working across all of your devices and enabling you to pass information between them. This app should be able to interact with all of your apps so you can start to tell it "I want you to do ... " ... or talk to you while you're performing a task and telling you of another way.
It should then start to record efficient ways of doing things, while keeping personal data private. Your ai could be in contact with what is the best way of doing things. This use could be growing the model.
The next model can use old models to train them.
Oh no! You said asteriks!
Not asterisk? Pretty sure
Ironically getting to whale level means to the limit
I, for one, would enjoy a longer video on just the Anthropic Scaling Monosemanticity. I'm not sure about everyone else, though.
Yeah balancing interest and depth is exceptionally hard, was up till 11pm with this one
@@aiexplained-official Your efforts are well received.
Another great video. Keep it up. You are the best resource for keeping up to date on the details of AI. What would you recommend for someone wanting to get into the AI space, eventually? Someone with very little IT skills and low math skills. (I know it's a loooooooong road, YEARS LONG). I am starting with compTIA, security+, network+. However, I want to steer towards AI.
I would play about with tools like Crew AI to set up AI agents. It's an easy to access skill that's high in demand. Or specialise in AI music gen, or image editing or whatever your passion is, get great at it, then market.
So because of Gary Marcus we have a confirmation from OpenAI employee that GPT-5 will be here in November.
@aiaxplanied by far the best videos on AI updates. You keep on being my weekly must watch Phil ✊🙏 thanks a lot for these overviews
i remind you. understanding inner workings - helps ramping up capabilities. so safety is what you actually "do" with that info. and i don't think they are prioritizing safety RN )
Great video as always, Philip. In addition to the problems you point out, the example with Gemini "saving a photographer time" seems inaccurate in two ways. 1 - I'm a video director; I have never in my career asked anyone to make a list of stills or videos according to the metadata. This seems like a non-task dreamed up to generate a claim that Gemini will save time for photographers. 2 - However, where generative AI tools most certainly DO affect photographers is in creating images synthetically which mean that the photographer's client doesn't need to hire a human photographer. Which is far more impactful. So this "research" on the front page seems IMHO a piece of PR fluff to give the impression that Gemini is a tool which helps photographers without replacing them. I would say both claims are hollow.
Fantastic point Martin, very dubious
This is easily one of your best videos, and for me, your channel ranks on the 2 Minute Papers level. Meaning, it’s maybe the best AI news channel. Thank you, Philipp.
Thanks eburg!!
"We are nowhere near diminishing returns when scaling compute". Of course he is going to say that to INVESTORS after flushing down billions of their money in scaling compute
7:33 noooo, you said Asterix
Oops
Marine life as a metric? Seriously? That’s idiotic and completely meaningless for comparison purposes. Which is probably why they did it.
Best channel on AI. SO INTERESTING EVERY STORY! KEEP IT UP!
They’re gonna run out of data to train the models
Synthetic data
I'm so grateful for these videos. No idea how else I would be able to keep up with that is happening.
Thanks sebby
It doesn't sound like Scarlett Johansson at all.
Yes! I feel like I’m going crazy. Yes it’s vaguely similar, but if you actually listen it is clearly not her voice!
Christ it’s been driving me crazy the way everyone has been screeching about this.
I mean way back when they added the voice mode to the ChatGPT app people started saying that the Sky voice sounded like ScarJo.
I didn’t think it did then and I don’t think it does now.
Sky is quite similar but Scarlett's voice has always had a slightly husky sound to it, something that's completely absent from Sky's voice.
@@antonystringfellow5152 I mean.. there are probably hundreds of thousands of women whose voices are more similar and in some cases way more similar. This might be an example how Scarlett Johansson thinks that she is the center of the universe.. like every other famous person tends to. And she did make a movie about artificial intelligence so it's a good opportunity for look at me moment. I'm not blaming her.. she is special for sure.
Rashida Jones
Great video! The anthropic paper results were fascinating
Thank you marcos, they were!
The only channel I have with notifications turned on
Gotcha! Looking forward to your next video.
I want to see Steve Balmer present this. Using chairs as props.
The LLM benchmarks are just useless at this point...
Could you please explain
Contamination issue
@@sigret1 also the benchmarks becoming the goal.
Which I guess is just another kind of contamination to what contamination normally means.
what is the solution then
@@johndank2209 The chat bot arena
I'm in prepress myself which is similar to the photography field in going through multiple files for specific details. This is something I'm actively looking to implement. We have just started using an algorithm based process through programs such as "switch" and "pitstop" while I wouldn't feel comfortable relying on LLMs for going through this, I would feel more comfortable setting up a more rigid system using llm assistance to develop the process. But please please please look at this use case more in the future. My job literally depends on it XD
17:09
They dialed up the hatred and slurs feature so it’ll also dial up self hatred.
... These models are sentient. Aren't they?
No, Antrophic just loves hyping up their models by pretending that their models are sentient so the youtubers who are not familiar with the inner workings of deep learning models post click-bait videos about these models saying "This model has emotions!!! AGI finally arrived!!", which is basically free marketing for them, and a cringe fest for the researchers in this field.
Great video! So interesting.
Thanks Hal
Great content. Thanks!
Incredible support Dera, I am so grateful!
Thank you for referring to companies in the British (and correct) way of 'they' instead of "it". I've never understood why our American cousins refer to companies as "it".
Great, less hype and drama and back to papers 💪🏼
Making compute cheaper IS the way to AGI--we are limited mainly by compute.
the elongated inference time to get more out of models makes double sense with the small ones because if a 7B Lamma 3 or Phi 3 can become a GPT4-4o. that is way more exciting than even GPT-5. you can probably get that if you homogenise it to be a 5.0 with even more time and layering... its now about speed and size if that is true!
I’m following you because you actually gave me the news I was wanting. Ty
Thanks Mike
i hope the performance increase of the trained model is at least as big if not bigger than the increase in compute needed to train it. if this ratio drops under 1 i don't think it'll be very attractive to exert the effort. even worse if it's below 0.5.
PhD in what? kinda wanna do research in this field, but so like the production side of this. Decisions need to be made. Great breakdown btw. Especially for the layperson.
How can we predict future scaling-performance without yet having trained on said scale...?
It's akin to saying "we haven't yet come close to the bursting point of this economic bubble".
You can't know until the bubble is already popped. You can't know scale:performance until it's already begun to level
17:07 Self hatred: "I am clearly biased... and should be eliminated from the internet." Just to be clear, is that the bot silencing itself?