Google DeepMind's AlphaProof MASSIVE MATH BREAKTHROUGH - AI teaches itself mathematical proofs
ฝัง
- เผยแพร่เมื่อ 16 ต.ค. 2024
- The latest AI News. Learn about LLMs, Gen AI and get ready for the rollout of AGI. Wes Roth covers the latest happenings in the world of OpenAI, Google, Anthropic, NVIDIA and Open Source AI.
My Links 🔗
➡️ Subscribe: / @wesroth
➡️ Twitter: x.com/WesRothM...
➡️ AI Newsletter: natural20.beeh...
#ai #openai #llm
LINKS:
AI achieves silver-medal standard solving International Mathematical Olympiad problems
deepmind.googl...
International Mathematical Olympiad
www.imo-offici...
Functional Programming in Lean
lean-lang.org/...
I like Deepmind's unique approach, they have diverse researches and focus on AI for academic/scientific purpose.
yh deffo agree, they do alot more than most companies toying almost exclusively with generative AI
Deepmind basically started the current AI revolution.
They just didn't bother bringing anything to market like OpenAI did
Their approaches are likely what we need for AGI, not the OpenAI approach.
yes I feel like Sam Altman Fried has gone past being truthful and is only about the hype train now. He will lose more and more researchers as other companies lean into open source.
Trust me, they will commercialise all these achievements eventually
This took me more than 25 years back when I was in high school. I almost made the final team that went to the math olympiad in the country where I grew up for 3 years in a row. We were in the same classroom for 3 years. I can tell for sure that these people were and are the brightest minds of my generation. Seeing this achievement blew my mind out of the water. We are really really close to a breakthrough. Just read some of the problems, and you will see what I'm talking about.😮🤯
Well, 99% wouldn’t even be able to perceive what the problem is, me included, let alone understand its complexity
It was one point away from gold. So it's basically at gold level already. In a few months it will be stable at gold and what next year it will surpass all humans?
I’m sorry you had to spend 25 years in high school. 😳
@@tzardelasuerte if its at gold level it can probably be used for research mathematics now.
This is really, really, impressive.
We are probably only a couple of years away from automatic PhD. level research in mathematics.
My jaw is literally on the floor.
2026
Huh it’s already at 168 IQ it’s well beyond most PhD in mathematics
2032
Next year
@@perc-ai IQ only measures how much of a generalist you are, the human average in mathematics is around 135 because they are usually highly specialized.
most LLMs on the other hand, are generalists, trained on 30,000+ lifetimes worth of data. if they weren't so data inefficient I imagine they'd have a higher IQ than any human in history and contribute more to science than most people believe is possible.
Imagine giving AlphaProof the Fermat's Last Theorem proposition and it responds in 5 seconds with a book margin lenghtened demonstration
Impossible. Taniyama-Shimura conjecture by itself can't fit in that space.
Fermat's secret all along was just having a really, really good GPU
"I have found a proof, but the token limit is not long enough to contain it."
Fermat's truly marvellous proof was just referencing the future work of Wiles. He actually had a time machine.
Since it is using Lean as a theorem proving environment, this might be just a library citation at this point.
Note that the IMO is a competition of the most talented *juniors under 20*, this is not at all "the greatest minds in the field" as he states repetitively in the vid.
There is no general competition for high-level math simply because math gets too vast and complex at the high-levels for single minds to grasp it all. The closest thing that awards the greatests minds would be the Fields Medal. (And even the Fields medal has an age limit.)
For AI to beat the IMO problems is super impressive, however.
It's worth mentioning though.. that historically the many of the great works of mathematics are done by people under 30.
wait until 2026 - terrence tao, probably
The big promise of AI is that, with enough compute, it WILL be able to grasp an entire field at once.
Imagine a computer that can read a molecule and modeling how it will respond to every single protein and molecule in your body, under the conditions where it would find them. Grasping all of biochemistry and physiology at once. It's an insane thought, but it's likely possible!
@@clinicallyinane8098I don’t think enough energy exists in the world to power the compute required for what you just described. There would need to be huge breakthroughs in energy efficiency first I think
From 20 to 30 is the most productive age for mathematicians. So its not that far.
Did anyone else notice someone got 42 / 42 🤯
So Long, and Thanks for All the Fish! 🐬
Haojia Shi - his 2nd perfect performance, following up from last year's perfect score.
He will be an unbelievably valuable apparatchik to Beijing's inner circle military and economic elite.
@@mevert87 I find it incredibly sad that people with that kind of mind will be tasked with how best to dominate the world. Instead of focusing on how to make it better for everyone. I know, "that's life", but it really doesn't have to be.
@@veracityseven I'm with you, but what's your utopian solution. What does a genius like "Haojia Shi" do in your perfect world?
@@mevert87 A genius isn't needed in a perfect world. But another genius would certainly get us there faster, if that genius were working alongside the rest of the field in advancing our collective capabilities.
Before Ai, mathematicians and computer scientists were already working on formalising math to the point of computers being able to reason.
It has been true for decades that, in principle, existing theorem provers could prove any theorem deducible from a given set of axioms... if you had unlimited computing power or could wait an unlimited amount of time for the answer. The significance of AlphaProof is that it can learn very effective heuristics for finding a proof immensely faster.
Automated theorem proving has nothing even close to this. Unless there's some earlier deep RL work I missed.
This has been true for decades so you have no clue what you are talking about.
Before computers, even. It's what led to the creation of computer science. What's your point?
No, the goalposts will keep moving, that we can be sure of lol
You can always use the wild card "but it has no feelings."
Thats because they're achieving their goals and looking for the next set of goals. And humans are stubborn as well, not wanting to admin wether something is intelligent or not.
@@cesar4729 "Do you?"
maybe the reason the goalposts keep moving is because we keep underestimating the complexity of what our brains are actually doing.
This is ground breaking work by DeepMind, well-done to all concerned.
ai math expert. it’s gonna get crazy when it’s a science expert.
With current sota LLMs plus advances like this it already is, it's insane how helpful gpt4 and claude sonnet already are for helping with research.
Watch it proves we're in a sim.
They won’t do new science,, as they have no concept of literally anything. Research the arc challenge and stop telling everyone how great the emperors clothes look.
Too bad there will never be an AI economics expert. But even if there were, political donor whales would never allow their puppets to use it. $200 trillion debt.... LETS GOOOO 🎉
@@twylxght SSssshhhhh!!! Its a secret
Mind blowing
Future here we come
Introduction and AI Development Updates - 00:00:00
Google DeepMind's AI Achievement in Mathematics - 00:00:36
Overview of the International Mathematical Olympiad (IMO) - 00:02:11
Details of the AI System's Performance - 00:03:12
Insights from Experts and Comparisons to Previous AI Achievements - 00:04:16
Detailed Analysis of Alpha Proof and Alpha Geometry 2 - 00:05:54
Discussion on AI's Growing Capabilities and Training Methods - 00:07:26
The Process and Challenges of Formalizing Mathematical Problems - 00:09:01
Application and Performance of Alpha Proof and Alpha Geometry 2 - 00:11:42
Potential Implications and Future Prospects in AI Development - 00:16:29
Conclusion and Final Thoughts - 00:17:01
We all generally focus on LLM's and on multimodal, which are great. But it seems Demis Hassabis's Alpha models consistently bring real World changing events and are really the forefront of AI. They don't seem to sustain the fanfare I think they deserve. Becoming the best player of a 3000 year old game. Discovering all of the protein structures, medaling in real World math competitions etc., These are really the foundations of ASI and the proposed utopia we are trying to reach.
Wes is awesome wooop! (Just wanna send some energy to very hardworking wes)
The first system most experts will feel inclined to call AGI will be highly modular, I strongly suspect -- comprising many subsystems integrated in the right way. Perhaps with a sort of executive MoE module somewhat akin to what we have now routing inputs to the appropriate domain modules.
I totally relate and I imagine that formal Math isn't as familiar to you as a given programming language. I've had the same revelation and ever since, I see Math (as a language) very differently. It's almost nothing more than symbols assigned some meaning that typically maps nicely to a function with parameters and an output.
That sounds like algebra specifically. Which can be used to describe a huge range of problems, but functionals (things that act on functions and have either other functions or values as outputs) are where maths really gets going.
More impressive than many other news lately for sure. The intersection between AI and science is where these models can really make a difference.
Like you kept hinting at with novel solutions. Humans have their inherent biases and up until recently we only really have human minds to share these ideas with.
You don't know, what you don't know.
It’s the fact that an AI model play itself millions of times at a game like go and test random moves in a systematic manner means it can find different strategies that humans never could. Once a human gets good they stop trying unorthodox moves and become locked on the known strategies
🏆 I expect nothing but AI having the ability to improve itself and surpass our capabilities. How else will humanity benefit from it without self improvement? So this is a great achievement!
LLMs are the most human centric type of AI, but when I thought of AI growing up, it was this kind of thing that I imagined AI would become. Largely mathematical reasoning, based on knowing all of the rules of the space which humans have discovered or invented along the way. You teach the AI how the scaffolding is constructed and by placing one brick at a time it builds a millimetre perfect house. Naively put, I know, but it seems like there are two very distinct groups of AI tech right now and this is the one I thought it all would be a few decades ago.
It is very exciting. Cannot wait when it starts solving unsolved math problems.
Thats against the logic of AI.
This is very very very impressive, my masters is in topology, and seeing this done by a model IS INSANE!!
Computational provers have existed for a long time. Mathematicians just don't like them because it does not help you understand it just proves things. Putting a Neural Network black box behind the proof makes the problem of understanding worse. So having a computer prove things is not really exciting.
I doubt that what you say is true. The proofs built by this model are not bad and valuable. OTOH there are plenty of computational proofs in maths already.
@@Krmpfpks Have you seen the actual proofs that alphaproof have produce?
Well, the final answer isn't 'yes' or 'no' or a set of numbers. It's the step-by-step reasoning that makes it useful.
@@wincoffin7985 step by step is not understanding. For example brute force elimination of counter example is step by step but provides no understanding. There are also magical steps. Steps without context but when taken solve the problem but does not provide any clue why the step was even related to the problem itself.
Awesome video. I saw this as a plus 1% on Dr. Alan Thompsons AGI meter bumping us to 76% now so I was looking forward to someone covering it. Nice job!
Exciting to see the proof processes moving into Gemini eventually. All the other models will follow this lead.
The next step toward Artificial General Intelligence (AGI) likely involves integrating multiple specialized models as cooperative agents. Rather than relying on a single Large Language Model (LLM) to create various task-specific agents, we should consider the inverse approach: combining numerous specialized models to function as a unified, multi-faceted agent. This collaborative system of models could potentially lead to a more robust and versatile AI, capable of handling a wider range of tasks and challenges than current single-model approaches.
(I'm terrible at writing English, so the text is improved by Claude :) )
this is already being done it's called mixture of experts
I wish they would open source the AlphaProof model, or at least open weights. I would love to build on it.
they might release an older version at some point, or they could just include it in higher price gem
The issue is being able to run it as well 😅
@@MatthewKelley-mq4ce It's probably not super huge. I would think it's around 8B so it would run just fine on a 24GB GPU
@@pantoglyph i don't think it's manual. They fine tuned a gemma model to translate to Lean no?
@@pantoglyph ah I see. I'm much better off continuing to work with agentic systems, then. Still, I hope they can release it soon and it's capable of general problem solving, it would certainly be interesting. (even if I had to rent a huge GPU(s) for a few days haha)
I think one of the most important things A.I. is teaching us is that humanity as we are now thinks too rigidly and repetitively. There is absolutely no reason why we can't come up with more novel and creative ideas, but years of: "Everyone does it this way so I have to do it that way too" has slowly over time crushed the human creativity out of most things in life. Letting go of preconceptions and trying new things, no matter how ridiculous is the only way to surpass our limits. After all. how can you know what's possible if you never try?
You do understand that there have been played tens of millions chess games, yet AlphaZero is able to play hundreds of millions just to train itself.
There is simply not enough time on this earth to exhaust as many possibilities as AI is able to.
Human wisdom is in using paths most likely to yield results and they are often the most explored ones
As I always say, if you don't f*ck around, how would you find out?
Just beware of charlatans like Terence Howard, Who offer nothing of value to the field of science, But are still able to trick millions Of ignorant people into bashing scientists for being close-minded.
@@mrmendeleev I am not denying the power of A.I, simply just stating that human creativity is extremely under utilized. Exactly as you said, people optimize for the "best outcome" but I think that has unintentional consequences. The fact that everyone thought AlphaGo's novel move was stupid at first is proof of that. Disregarding an outcome just because it doesn't meet your expectations is foolish and can prevent people from finding alternate paths that may be even better than what we assume is the "best outcome." All that being said I agree time is precious and that A.I helps A LOT in that regard in finding new paths. Thank you for your thoughts, its nice to hear anothers opinion c:
It becomes very clear that new advances are enabled by better and more data which is possible by better llms/models, which in turn enables better daya and better models. There is no end of progress in sight.
An endless feedback loop that keeps speeding up. Crazy times ahead. Can't wait
Let’s gooooo what’s up Wes
You do an incredible job at distilling relatively complex research into understandable explanations.
It will be exciting to see if Lean gets more support and if DeepMind shares the database of proofs it worked on.
Regarding AlphaGo, there were some errors. One is Lee Sedol was not the world champion at the time of the match. He was a former world champion. Still a top player, but there were others who were better. Regarding move 37, it wasn’t a move that people laughed at or thought was weak, but it was indeed startling. It was a shoulder hit on the 5th line, which is considered bad because it gives away too much territory. A commentator thought there may have been a transcription error. It wasn’t a winning move, however, but a very good one among other equally good alternatives, but what made it stand out was it was a move that no hums would make.
At this point, the AI was still relying heavily on human moves which it had in a database. Google would later rewrite it so that it learned only by playing itself, but that wasn’t this version, which makes it even more reliable it was able to come up with a move like that.
There was another similar type move, but not as startling, where the commentators said “If this is a good move, this program is a lot better than I am.”
Very excited about AlphaProof!
Reminded me of a 30 Rock scene, jack donaghy, "of course there are multiple types of intelligence.. practical, emotional... and then there's actual intelligence, which is what I'm talking about." Can't find the clip sadly.
Awesome video! Hopefully tech like this or this exact tech in the near future and as soon as possible can solve mental health disorders and solve physical health disorders. This is one step closer to ASI:) I am excited!:)
and thank you for showing! ❤
The use of a proof assistant (LEAN) is brilliant, as it can act as the discrete verification to the heuristic-based search that the AI model provides, thus preventing it from making "mistakes" in the same way humans do
Is there a quick take on how LEAN is 'mistake-proof' - genuine question as I don't know much about it?
@@LemoUtanIt is mistake-proof in the sense that Lean will always verify whether a proof is correct with 100% accuracy.
Essentially, Lean is a functional programming language, where types are propositions and terms of a type T are proofs of the proposition T. Proofs in Lean are nothing more than programs, which take proofs of the hypotheses as inputs and produce a proof of the conclusion as an output. The idea is that any mathematical construction can be implemented in the dependent and inductive type system which Lean uses, which is based on the Calculus of Inductive Constructions.
Here is a very simple example of a proof. Suppose A and B are propositions, and we want to prove
A⇒((A⇒B)⇒B). A proof of P⇒Q is a function which takes a proof of P and produces a proof of Q. You can think of this as a function with type signature P→Q. So all we really need to do is make a function with type signature A→(A→B)→B. In λ-calculus this would be
λa. λf. (f a), and this is basically the Lean proof of the proposition.
@@LemoUtan Lean programs are limited to verifiable steps and statements, so any statement or set of statements made in the language are checked by the computer to make sure the reasoning is correct. So if a mistake is made, it will always be caught by the verifier. Otherwise, if you trust the verifier, then you can be sure that the proof is valid. This works because proof verification is relatively simple compared to the writing of the proof.
This is unbelievable! Governments pls implement UBI soon. We all will be jobless
good slave.
I think first we’ll have to go through massive joblessness and homelessness and poverty before the govt decides to “just” give money away. The corps that will profit immensely from AI will fight tooth and nail to avoid giving the majority of their profits back to society. I don’t expect UBI anytime soon myself. It’ll have to be cataclysmic for the govt to enact laws to tax the crap out of companies making tens of trillions.
perfect attitude, serf.
@@klarad3978 yeah optimism is not for dire times
Which human scored 42 points and who do they work for?
Does this mean that it can finally code in Openscad or something similar?
Chomsky warned of a moment wherin the internal language of AI would exceed our ability to recognize.
Is this a qualitative moment of silence we're nearing?
What does tht even mean
Math is a perfect domain where AI can mature quickly precisely because we can give it a very precise learning target. I saw that coming right when I first heard about GPT and so on. Let's see how quickly we can figure out Collatz, odd perfect numbers, Goldbach and so on.
IMO is a closed system so it can be trained properly to not confabulate. But PhD level research level questions are open ended..
You can make it closed by simply giving the AI intermediate/smaller objectives of your research - I imagine giving it some lemmas I need to prove, for example. It's like AI assistents for coding, they help with many small clearly stated objectives.
@@sil1235 I have tried that. It works somewhat, but for questions that I can answer myself with some/lot of effort, not for truly original answers..
@@quasarsupernova9643 You tried it with what though? AlphaProof is not publicly available as far as I know. Plus we are talking about future development, this is still at its infancy.
@@quasarsupernova9643 You have tried what though? AlphaProof is not publicly available (at least for now). If you are talking about language models like ChatGPT etc., they work differently than AlphaProof, they are more useful for creative work or translation rather than answering original questions (I wouldn't trust it one bit, only something I can then verify).
@@quasarsupernova9643 YT is filtering my answer idk based on what, so I will try to be brief. We can't compare this to LLMs like chatgpt, this works completely differently. The LLMs are not really good for answering factual questions, more like for creative work and translation.
I wonder what this will lead to, once math has been completed. Absolutely wild times... 🤯
When AI can solve problems that humans haven't solved yet, I believe the hype.
Exactly. How do we know that the AI didn’t have similar problems in memory, so the solution actually is the result of months of work? I mean, still nice, but time and again we see that AI’s ability turn out to be result of knowledge. Like in programming.
Automated theorem provers first did that decades ago. It's not hard to devise mathematical problems that nobody has even considered yet.
@@kevinvanhorn2193 Let’s say: solving important, significant, known open problems whose solution is widely acknowledged as a fundamental contribution to scientific research. And most importantly, AI has to do that all alone 😁
@@federicoaschierithese problems were not public and were made by mathematicians who have knowledge of all similar public problems. They're made to be hard for humans who have studied every IMO-style math problem published anywhere.
@@Myname-d5s Yes, I know. But in reality, these are not terribly original problems, because they are designed to be doable in a few hours. Elementary mathematics is quite limited, and the format works because no participant is familiar with *millions* of problems. So I am quite skeptical that in the database there wasn’t anything similar. And the burden of the proof is not on me, but on those making claims out of a set of problems of no statistical relevance.
The Lean language is based on the calculus of constructions with inductive types.
This is cool. They should add another LLM at the end to translate formal proof back to informal langauge.
Great find Wes. I love your videos. This one has to be one the most impressive things. What does the future hold with this type of tool?
A big step in the direction of recursive AI improvement that isn't entirely black box data driven.
AI getting self aware of it's own algorithms and how they function.
This is exactly what we want computers to do to be extremely powerful useful calculators that seem like they're intelligent in a human sense or even beyond a human sense but essentially can automate a lot of the more tedious aspects of these chores so then we can get to the roots of proving how theories of the universe work and then being able to hopefully automate the engineering process of coming up with novel new devices energy sources and better computers.
Very true, but I wonder...what incentive then will there be for students to learn mathematics for its own sake? Subjects like analysis, linear algebra, topology, etc.? If results within these subjects can be proven by computers, do you think that may deter some potential bright students?
@@paul5324computers can play chess better than any human on Earth now. We still have humans playing chess with each other.
It was deter some students from studying and others it will not. Everybody's different everybody's goals are different and their desires are different.
If we find that nobody wants to do these things then maybe you have to create incentives so they will do them.
Actually philosophically speaking I am tired of worrying about what some are most people may or may not do just because of the fact we have improvements that perhaps make it less necessary for other people to do things. If certain things are real value well then we will find a synonyms for them to do it but at the same time people got to take responsibility for their own actions.
This is really impressive, i wonder how this system performs on Putnam exams?
Your videos are always great and educational. I have a simple question: Can an artificial neural network solve difficult math problems on its own, or do we need new technology with more sophisticated interactions? Some believe the brain functions differently than an ANN because of quantum entanglement. I don't think chain of thought or feedback loops are new models. Is the ANN enough, or do we need something more advanced? Thanks!
This is very exciting and interesting to see. DeepMind used synthetic data and sort of a bootstrapping approach using Gemini to create more and larger datasets for training subsequent and specialized AI models. I saw a very similar approach in the Lama 3.1 paper from Meta. While this approach is powerful, it is also quite manual in the sense that engineers need to deliberately create and feed this specialized training data. If it would be possible to somehow integrate this specialization and fine tuning of AI models into an automated training or even reasoning/inference process, I think we would be much closer to true general intelligence that is self improving.
this is amazing. LLM's greatest weakness was math up till now, but now we've got AI that's equal among absolute top experts.
now that this reasoning engine has proved itself on math, it's obvious it could be used on anything. physics. chemistry. economics?
I think this result will be a major step towards AGI, and we're still very much on-route for AGI in 2029.
the world is gonna be so different just 5 years from now.
humanity is not wise enough to yet wield where this tech is going
E= C + M - S
Where:
E= Enigmatic quality
C= Complexity of the equation/expression
M= Mystery or obscurity factor
S= Satisfaction upon solving
Wes Roth: I'm the opposite: I'd rather work with mathematical notation than code - it's both more compact as well as making structures in formulas more obvious & hence easier to reason about.
this really is the reasoning engine we have been waiting for: an LLM that formalizes natural language into a logical language and then another network trained on that formal language. we can do the same thing with logical relationships in normal english language: formalize it into a logical language (a sort of prolog++) and then train a network on that language alone. the issue is we don't have such prolog++ yet; this requires a bit of creativity. we need to built an object to object relationship language that can formalize all of language into object relationships. a network trained exclusively on such language would be much smarter and efficient than any LLM. the limiting factor then becomes the quality of the language itself
math should be particularly suitable for LLM, because at the end it is a language with very strict rules. The only thing is to get training data
We still don’t have explainable AI. Even if there’s some kind of intelligence explosion with AI, it can only go so far before we stop understanding how it actually works. We’re going to be like rats that only know how to squeak in comparison to what it will be able to do. The next best thing we could get after AGI or ASI is explainable AI so that we can actually learn from it.
Any ASI should be an excellent teacher by definition, so even if we don't understand how it comes up with things initially, it should be able to explain itself.
I do think generation, checking/tagging, training, is the future.
It's like a generative adversarial network for training.
This is one of the reasons people learn with so much less data, we call it imagination. We've known for a long time there is a strong probably causal link between intelligence and imaginative play.
Why is no one talking about this in the media?
The sky's the limit. This is huge.
Imagine AlphaProof coming up with a novel proof of the four color problem
Can't wait to see a hybrid of this with classic language models so they better can reason and geometrically visualize problems when relevant.
I would like to see a system that could hold the solutions to those ~100 million problems and extract their core mathematical truths. But I guess at best would would get some sort of point-cloud visualization of any found clusters of similarity.
AlphaGo Zero is the key aspect of general intelligence. Agent based modeling from scratch is intelligence!
I find it interesting that the apparent ability for a LLM to solve math problems where nobody could, just comes down to the number of people that tried, over the biased thinking each had.
Where as an LLM is NOT biased in the same way, rather, it's bias is reliant on the weights it is given. If this is considered reasoning, then we are very close to AGI.
Excellent comment. People in the comments act like AI will just go and solve the unsolved problems of math. It might combine different approaches into one and that could make great use, however most unsolved problems would be solved if we had the needed computational power.
I wish they would release AlphaCode
I've always felt the same about mathematical notation vs code. The ideas are not necessarily complex, but the description is horrific. It makes sense if all you have is limited ink and paper, but not if you have terabytes of memory and a giant screen. It's like sending a tweet vs an email
Great this module can resolve the 7000language problem barrier and cosmological studies thanks..
Math proofs are like games with rules and built-in labels (a proof checker is the easy part). Thus reinforcement learning on synthetic data works. This doesn't mean LLM transformers will work by synthetic data.
I want to know how this relates to more trivial real world problems. We need systems being better and faster at math, logic, and reasoning. From getting a problem in text or vision and solving it as fast as humans can do it, or even take a fast good guess.
I think Lean was intended to verify proofs, not build them. I considered it because of its ability to verify a proof before presenting it to a professor.
I’d much rather read the math than the code. I was in a phd program for mathematics and did software development for a living, so well acquainted with both, but, and I think most mathematicians would feel this way, it’s not a close call. If you do math, you’re always using mathematical symbols, unless it’s a special case where a computer program can handle an algorithm more efficiently, such as the 4 color theorem.
Transcribing the problems into a logical formulation is half of the solution. This feels more like pre chewing the problem for the AI
I think moving the goalposts is alright. It simply signifies our previous understanding of what makes something intelligent was lackluster.
Sounds like it programs the codes in order to solve the problem in trial error loops? It must also use an agent(s) internally.
Hello ! Any ideas on how to make the difference as a human ? How not to be repleceable by an AI ?
A month ago the answer to that would've been to become an AI model engineer, but now it seems AI is going to take that job as well.
Looks like no one is safe.
This looks like a track towards the intelligence explosion aspect of AGI.
There is one thing that people systematically get very very wrong. Lee Sedol was not a world champion in Go when beaten by AlphaGo. It was like beating Michael Tal in 1980-ies in chess. Or defeating Vishy Anand today. He was still a top ten player, but way below world champion level.
Currently, no mathematician or group of mathematicians wants to check complex proofs, maybe there is finally a tool for that? (-> Wiles's proof of Fermat's Last Theorem)
I prefer mathmatical notation over code notation but I've spent a LOT more time grinding math than I have code.
I very much prefer strongly typed code. But it's a personal thing.
I have a degree in physics and maths, as well as comp sci, and have worked in IT for many years.
But I still find math notation to be clearer.
@@kriscrnomarkovic1109 it is compact
about time.. earlier today I wanted to test something with gpt o, and it wasn't able to solve a second grade math problem.. and even once I gave it 4 possible choices (that were WAY off his initial result), it still failed but since he knew his result was way off it managed to finally get it right.. that was painful to watch
for some reason it is easier to look at one statement but to manipulate the statement into a new form is harder as the traditional notation is easier to manipulate the structure and apply different tricks. however, this is something I feel like computers are much more useful in doing, and not necessary for me to do. So it is worth it overall as I dont touch novel or non standard stuff.
It's interesting that you feel "it's early" 😅
We adapt so quickly to progress. Lat year we thought, that "real data" will become the bottleneck and synthetic data "diluting" internet data will become a problem. Today we're already in the middle of accelerating progress by generating synthetic training data with AI. This by itself is ... shockingly 🤪 ... close to fully self improving AI systems. Unlocking math was THE stepping stone for self-improving algorithms, if I'm not mistaken. So self generating training data is already common reality and self improving algorithms are close. 👾
It really just depends on which kind of code you prefer. The mathematical representation is more efficient and general. You just need to understand the nomenclature.
08:46 Yeah, I often read Python much better than I can read the equivalent rendered LaTeX or hand-written advanced math or similar formats. Lots of the symbols have equivalent commands or algorithms, but I have not memorized them all; it's much more explicit or otherwise easier to deduce when spelled out as a computer program in a language that is not far from the ones I can write basic stuff in.
I wonder why a specialized LLM couldn't be designed for doing the plain English problem description into the Lean code stage. On the surface it seems this is exactly the type of step that would be the bread and butter of LLM's through showing a bunch of before-and-afters then having it work backwards (then repeat a gajillion times via synthetic data)
It seems to me the reason the goal posts keep moving is because we thought the original bench marks had more prerequisites for an AI to reach - like reasoning being one. Instead we found that AI cannot reason nearly as well as humans rn even while achieving incredible results on tests. It's painfully bad on basic reasoning questions that 98% of humans would get correct. That's why the goal post is being pushed - these AIs need to have the advanced reasoning of an average human for the goal post to be reached. I agree that AGI needs to have the reasoning of the average human to be considered AGI or else it's not a general intelligence at all, it's narrow AI
Why do all these major breaktroughs happen when i'm on vacation? Should i go on vacation for another week and we'll have superintelligence?
So what if they train this on a massive supercomputer and it solves maths? What if they use it to solve a problem like " find a solution for a unified field theory".
i haven’t used lean but use latex extensively to generate math in papers, and to me the math is more concise and easier to understand than latex code.
when it can accept a problem statement provided in an arbitrary language or mathematical notation without human translators, solve it independently, then explain its solution to a person having an arbitrary level of understanding in the subject, we will (all) have something generally useful.
not to distract from the accomplishment or the terrific reporting on this. Just saying that’s probably the last goalpost that would need to be achieved before most people would stop moving them.
so, how long till it start solving NP-complete problems? ;)
Of course, the idea for a semi functioning ai is quickly done. What's hard is debugging until there are no more hallucinations while not degrading the performance.
Amazing and very intersting. Deepming
Re: 8:50 I also understand equations written as code much easier than in mathematical notation. But I also spent much more time writing code than mathematical proofs (hated those at university).
I am leaving a comment here cuz I can and I will. Now, time to watch the video.
this seems too good to be true