There's a strange amount of cope and conspiracy theories out there. This ain't the channel for that. Occam's Razor. Either there's a gigantic convoluted conspiracy to suppress the most public technological revolution in history... or it's just slowing down due to economic factors. Report from Stanford: AI is getting more expensive: aiindex.stanford.edu/report/
@@homeyworkeymultiple companies competing in multiple countries, huge benefit to leapfrogging, Ilya jumping straight to ASI... when was the last time something "SHOCKED THE ENTIRE INDUSTRY". A few months, surely. Sora? 4o voice? Something actually ready for prime time? 2023?
I definitely think having a UBI funded by the abundance that can be created by AI is a good idea, especially given the job replacement, we will need protections. However, I'm curious, what are you thoughts & solutions for preventing issue of social control by means of only providing UBI (including money for food) to citizens who follow specific rules. If access to food is based through UBI, and the terms in which people must act to get UBI in the future may become totalitarian control system. If many are dependent on UBI for material living, wouldn't that be a huge issue in terms of countries forcing political alignment or social scoring technology for access to UBI and food? Do you have any ideas for solutions to this problem?
Good job in keeping rooted in reality. That’s rare. You always have been upfront. First video I saw it was about sigmoid and reaching a ceiling in a finite world. Then it is always hard to foresee when the slowing down starts. Even harder when a new sigmoid begins. Good job, best channel about AI. Ps. I suggest to conspiracy theorists to make a nice tin foil hat, that’ll suit ‘em 😂
true it’s actually kinda sad and ridiculous. ppl need to chill the HELL out. trying to say that llms are conscious is ridiculous. however they may be a super powerful tool regardless
@@pier-oliviermarquis3006 Yes, and that should be noted. However, I respect adjusting one's views when confronted with new data if that person was genuinly mistaken. Of course flipfloping all the time is not good.
@@DaveShap yes, if you can replace 30% of the jobs with slightly general narrow AI (like LLM with vision on robots), then you have a historic job disruption event. The problem is that after that point, AI development will likely outpace the ability of people to retrain themselves for new jobs. At least until we reach a high percentage of automation, then it will either slow down, or true AGI will close the gap quickly.
As you see with self driving cars, deployment of lesser AI also suffers from long tail problems. It would go faster if we designed the world around AI capabilities but so far that only happens in factories and agriculture. But generally I agree.
I swear a lot of us are like the main character of Fight Club in the beginning, where a part of him wishes, that the plane he is on, would crash, so that at least something momentuous would happen and change everything, even if it is certain doom.
Its not going to. I am a businessman. I just have one simple proposal to translate into French. You really think I'm trusting my business to a machine? No way. Not even for a simple translation. NO company is using AI to replace people unless they don't care about their profit margin/reputation/growth.
@@zevofeir That is a huge problem because you're waiting for something that will never happen. Its keeping you depressed and anxious while discouraging your generation from trying anything new.
It happened with Internet too . That’s why Paul krugman made that infamous statement that Internet impact won’t be greater than that of fax machine . Actually Internet took off exponentially post iPhone release and AI is waiting for its IPhone
@@meetadi4uexcept its iPhone is the iPhone. There's really nothing else needed except an AI and a little software scaffolding, a separate device is unnecessary and probably detrimental.
Effect has not been felt yet, BUT it is obvious that this current technology is only support tool and it does not even increase productivity as high, as someone thinks. In our company, you cant really see obvious difference between people who use it and who dont use it... It is still dumb. It is only better google. The question is how good it can be in the future and thats very unclear. It may by big suprise but also big failure.
Saying that we can't have AGI because the human brain works a certain way, is like saying we can't build supersonic planes, because bird wings can't flap fast enough to sustain supersonic flight. The way machine learning and human brains work is different, sure some scientists have drawn comparison between the two, but they aren't at all the same. The question isn't if we can replicate the same mechanisms of the human brain in silicon, the question is simply if we can build machines that can perform as well as humans in a large enough number of tasks, and moreover, if said machines can build next generation of machines that can perform even better. How AI achieves task performance is completely secondary, the important thing isn't the "how" but the "if". If it quacks like a duck....
An object in space has momentum. AI progress is not an object in space, and has no momentum. It takes *work* to make technological progress. Momentum is when you keep moving *without* having to work for it.
What is disturbing is that even without ASI we can make excellent killing robots even with the current technological skill set. Before reaching AGI it would be nice to learn how to cooperate, rather than beat down each other. The whole humanity would be better off.
AI will be what we train it for… So, it is kind of a chicken and egg problem. I’m just afraid that humanity gets access to a technology we are not grown up for enough…
Perhaps we'll see a "controlled-rollout" from OpenAI going forward, but what about all of their competitors, in both the US and abroad? I don't think AI development can effectively be slowed all that much, even if govts truly tried to do it. There's always going to someone else working on the same tech somewhere else. There's too much to lose. Billions upon billions upon billions of dollars have already been committed to the cause by both public and private investors - this train ain't slowing down or stopping without research hitting an insurmountable scaling wall, but as of now I certainly wouldn't bet against all of the researchers who are working on better algorithms, better architecture, purpose-built hardware, etc. Research goes where the money is, and right now most of the world's largest and richest tech companies and governments have gone all-in on AI - that alone should tell us something. To me it seems possible, perhaps even likely, that we might only need a few more notable iterations of improvement, on the level of which has regularly occurred over the past several years, to end up with something more intelligent and powerful than we can even comprehend.
AI will not be slowly rolled out by competitors and other countries…. OpenAIs actions do not decide what everyone else will do. You would have to have the entire world agree to “slowly rolling out AI” …. which will never happen and that’s already failed once (the “call for a 6 month pause on AI” letter) .
It's not slowing down though whatsoever, many companies are making rapid progress such as Claude Sonnet, Gemini, OpenRoute, and several foreign models.
As someone that worked retail for a few years in the late 2010s, I'm not sure that 20% of humans would pass any given AGI test. It's easy to forget how disturbingly dumb some people are when you aren't often exposed to the pool that contains literally everyone. It will be hard to displace most human jobs, but the bottom 20% / low hanging fruit could be done with what we have _today_ imo.
As someone that's watching what tools people are coming out with, dumpster diving through the code, analyzing the shortcomings, playing around with them to figure out their strengths and weaknesses, and helping companies take advantage of what's out there, I'd say... If things are slowing down, then we are not even remotely in a position where we can determine that. The algorithms overhang is so overwhelmingly large at this point that I wouldn't be surprised if modern techniques applied to GPT-3 could surpass GPT-5.
Yes, I've said for a long time that there's a big difference between what's possible in the lab (which is slowing down) and what can actually be deployed usefully (which is accelerating)
@@DaveShap Fair point. The big AI labs do seem to be slowing down, and they're a big part of driving the forefront of AI tech. I can see academia & open source playing a much larger (and accelerating) role over the next few years since they're the ones driving the algorithmic side of pushing AI performance.
AGI's arrival is inevitable; I have a low care factor for its exact "birthday" . The important question is: When will we achieve biological immortality or, at minimum, Longevity Escape Velocity?
@@ScarlettM ... yes, but the implied part is that if reaching LEV takes longer than 30 years or so, a lot of us are starting to get into danger territory.
...when / if AGI automates most economic tasks, will society collapse? If not, why? Did it take long enough for society to naturally adjust? Or did the government regulate it properly? Did Sam Altman donate Openai's profits to the general public? Personally, I think society collapses if AGI gets here before 2050. Government is too slow to effectively regulate it, and capitalism isn't going to be altruistic all of a sudden. I hope for slow progress.
In my opinion is not so much that AI is slowing down its that we've reached the limits of what we can do with "Brute Force". just adding more parameters isn't going to cut it anymore. So now we need to get creative to do more with less. Stable Diffusion is an excellent example of this, since Stability could only release models every so often. So in the meantime the open source community went to hell and back to make tools to compensate for its shortcomings. So until the next explosion, likely adding search into the models. We'll be refining the tools to use and train them.
When we hit the wall with our current methods, research is going to start on new, better architectures and models for AI when they know they cant stop milking LLMs as much as before, its more about money honestly
You're completely wrong. We have not hit ANY wall with scaling. We already have ways to integrate tool usage into AI. You're talking about RAG. RAG already exists and is pretty popular. You are right about the idea we should try to do more with less, but you're wrong about the fact we've hit a wall with scaling. There is no wall that has been observed yet, it is a continuous improvement as the quantity of parameters, data(&data quality), and compute increases. The improvement is slowing down, yeah. Undeniably so. But to say we've hit a wall? No. That's wrong. Plus. In a couple years, we'll have chips that run AI way faster than modern ones. Scaling will be a must, to ensure model capabilities.
@@GodbornNoven We did not yet hit the wall, but it may be close in the sense, that models will totally become cheaper and more widespread, but enormous models with more and more data will maybe show only small improvements, so they will not become profitable. But I dont believe even large models will lead to AGI. I believe we need better methods and that can take another decades to reach it.
I liken things to waking AI from a dream state. Once the AI is lucid (enough) to meaningfully self improve from its own work and agency, then the hockey stick takes off. It'll be like going from zero to one and everything prior will seem slow.
Yes, eventually, maybe. But I wouldn't count on it. Consider that the smartest people on the planet are working on AI and progress is still slowing down. Sometimes intelligence is not the bottleneck.
@@DaveShap I agree on intelligence not necessarily being the bottleneck. One big advantage of digital intelligence seems to be in the time domain; they can simulate thousands of years in days. So while the best minds of bio intelligence are struggling now, perhaps they wouldn’t if they had enough time. So even just reaching parity with human minds can create the inflection point because of that time advantage I don’t think we need to simply go bigger as many AI experts think, fundamental improvements to the architecture may produce more immediate breakthroughs if data is a constraint. Simply introducing better prompting often creates better problem solving, so meaningful layers of abstraction could unlock lots of performance. I still find it interesting that Sonnet 3.5 bests Opus despite being the “weaker” model on size. I think what will cause the breakthrough is something no one is paying attention to. A dark horse.
I didn't think we'd get AGI this year, but I (and many others) expected GPT-5 by now and I think that's the underlying factor people are using to say AI is slowing down. Claude's success is a great sign that things are still moving forward, but everyone is waiting for GPT-5. The improved reasoning / System 2 thinking is what's going to enable a step-function to the next gen model. We don't really need a true AGI, just something that can reason and and brainstorm with us at a very high level to help us think of the next breakthrough idea. Hopefully the rumors are true and OpenAI is just waiting for the elections to be over and we'll get it (GPT-5) by new years 2025.
@markjackson1989 Because they delayed it only because of elections. Not because it can cause something, but because they would blame them for losing. GPT-5 can be in work's, if it is new model. They can even publish it before GPT-4o. If competition is better in this time.
system 2 doesnt really mean much, they need to talk of the executive function. Plus that books hella old and basically just talks about conscious and subconscious thinking
Interesting take. A few random thoughts: 1. Regarding not understanding the brain: I see the brain sometimes come up as a bit of a straw man. As long as the AI functionally behaves as we want (or beyond), then it doesn’t really matter if it’s a simulation of the brain. 2. Cost: We’ve been seeing companies start to lean into ASICs for things like transformers which seem reduce the requirements pretty significantly. 3. Lack of “breakthroughs”. I’m just wondering out loud here if what we’re seeing is not just companies and researchers playing their cards a bit closer to the vest these days for fear that their secret sauces will leak out. Obviously the absence of evidence doesn’t prove a conspiracy to cover up evidence. I’m just musing…
Occam's Razor. Either there's a gigantic convoluted conspiracy to suppress the most publish technological revolution in history... or it's just slowing down due to economic factors.
@@DaveShap Well no, that was just an example. My point is that the industry as a whole might be getting a little more circumspect about revealing their secrets. Anyhoo it was just a hot take. I have no real supporting data.
@@DaveShap It could be a factor of both. I believe technological evolution often occurs through sudden, significant leaps akin to phase transitions in physics, rather than a steady, incremental rise. Progress can appear stagnant, punctuated by abrupt advancements that seem to happen overnight. This phenomenon is often driven by intensive, behind-the-scenes development that remains confined to a select group or organization. Once these advancements are unveiled or disseminated, the rate of progress accelerates exponentially. I believe that the apparent pauses in breakthroughs are not due to a lack of financial investment but rather a scarcity of resources. Artificial intelligence, however, is poised to mitigate these resource constraints, thereby amplifying our capacity for innovation and development. I don’t believe anything slowing down!
I've never been an AGI fetishist, I think there's a lot of potential in systems that aren't broadly accepted as AGI and I've been disappointed in how much of the AI community tries to diminish what we have and create unrealistic expectations for what is to come. That being said, I feel like a lot of the job replacement capability is there or is going to be there in sub-AGI models like GPT-5 but we may stop short of the really positive implications of solving governance and climate change. That wouldn't be a great place to linger for long.
Underrated comment, well said. Keep in mind most AI enthusiasts you come across don’t even work in the field and their only exposure is the algorithmic suggestions provided to them by TH-cam.
Any perceived slowdown is an artifact of the “haves” choosing to gate how quickly they release their tech to the public. They are incentivized to stretch the game out as long as possible.
You're in a bubble, David. You've grown accustomed to the frequent release of new models and capabilities, and now that the headlines have slowed, it feels like progress has stalled. Your information sources, with fewer tweets and less frequent content from creators, reinforce this feeling. While this might be true, 99% of the world hasn't yet grasped what's happening. Try asking someone at Starbucks about AI without revealing your background. Most people have no idea about the speed of advancements, and from their perspective, there is no noticeable slowdown because they aren't aware of the developments at all. There are now thousands of AI companies compared to just a few years ago. While large language models (LLMs) might not be advancing rapidly at the moment, that's only one part of AI, and we're still at the beginning of this technological journey. A major breakthrough is on the horizon, one that will change everything. With thousands of companies exploring different approaches, new ideas will emerge. When I think about how we'll look back on this time, I find it amusing because we are currently so naive.
The brain is a warm, wet environment, which typically causes rapid decoherence of quantum states. Most cognitive processes and brain functions are still best explained by classical neuroscience and biochemistry. The Energies Involved in the Physics of the Brain: 1. Chemical Energy 2. Thermal Energy 3. Mechanical Energy 4. Electrochemical Potential Energy 5. Osmotic Energy 6. Vibrational Energy 7. Gravitational Potential Energy 8. Light Energy (Biophotons) 9. Nuclear Energy 10. Elastic Energy 11. Sound Energy 12. Magnetic Energy 13. Conformational Energy 14. Surface Energy 15. Quantum Energy 16. Electromagnetic Energy However using all 16 of these energy types isn't the only possible way to achieve advanced cognition. Many of the energy types in biological brains are related to maintaining the physical structure and biochemical balance of living cells, which is unnecessary for artificial systems. While biological brains use a complex interplay of various energy types, an artificial system can theoretically achieve superintelligent capabilities through a more focused and specialized use of energy, primarily electrical and electromagnetic. Yet we got a fewer more things to learn.
Woohoo! Please keep sharing this message so that us stealth startups can build a bigger lead. AGI pacing hasn't changed. Sonnet3.5 intelligence with finetuning would be enough. Everything after that is gravy. The trillion dollar clusters will take awhile. But it's not needed for autonomy and agency. Especially within niche domains. Which is also where the money is at. The next generation models (this fall) will be more than sufficient for what 99% of people would call AGI. Altmans idea that it has to beat 250 researchers to be AGi is just him trying to protect that bag from Microsoft. It's all about the agent communications. The networking layer is where people are underestimating the intersection of exponential gains. You know this better than anybody! I'm glad you have the spock uniform on again. Your inventions along this journey have shaped the industry. Thanks for all that you do brother 🙏🏾💜
It's slowing down... until it isn't in a few months when some advancement "SHOCKS THE ENTIRE INDUSTRY!" Then a few months later, "OH NO, AI PROGRESS IS SLOWING DOWN!" Until it isn't again, repeated ad nauseum.
Exactly. Progress of any sort-an improving chess players elo, progress in a negotiation, golf ball distance from a hole-always happens in plateaus and breakthroughs. That's the local landscape. But just zoom out and you can examine the global contour.
AGI is already here. Companies are just not openinf the floodgates. Ilya left to create ASI knowing that it was possible based off what he saw within OpenAI and knowing he would have significantly less resources as a new startup.
The truth is at the current growth in LLMs was a gamble that if you threw enough enough computation and data at LLMs, something magical might happen even though we don't understand and it gives us cognition. The longer harder route is that we need first understand what cognition is before we can start to make machines that approximate it. The dirty secret is it was just a hope that scale would solve the problem without us understanding it. It turns out that taking a chance on maths doesn't always work out. You need to do the hard work of understanding the problem.
This video has the presupposition that AI is slowing down but doesn't provide defined metrics for progress nor does it consider horizontal progress. This has been a common thing with GPT-4o where most people do not see the improvement because they were staring at LLM benchmarks. Audio as an input and output is a huge deal in and of itself. Perhaps you should come up with some method to evaluate horizontal progress along with vertical progress outside of LLM benchmarks. Vertical progress: Improvements in existing capabilities or metrics. For example, achieving higher scores on established benchmarks or improving performance on well-defined tasks. Horizontal progress: Expansion into new domains, capabilities, or applications. This could include AI systems tackling previously unsolved problems or entering entirely new fields.
Does that mean a time delay to AGI (like a linear displacement of the curve of 2-3 years with the growth rate remaining the same to your previous estimates) or is the growth rate itself slower? In other word is September 2024 now around 2027 but everything else you said remains unaltered in the aftermath of reaching AGI or does it also effect the post AGI development speed?
We've entered "AI arms race" territory, meaning it's as much a race of governments on the international stage, as it is a race between tech giants. Secrecy, espionage, and intellectual property theft are likely to dominate the generative AI space. As time goes by, it's going to become more and more difficult for regular people viewing all this from the outside to gauge the true rate of progress. Even judging progress by scrutinising finished product releases will not be that reliable, since advancements seen to be sensitive may be kept secret from the public. We'll get the toys, but the truly powerful AI may not be made public. The race for AI dominance and the future of humanity has become opaque. Only a select few will truly know the rate and direction of generative AI progress.
Humans have the capacity to gain insights and grow wiser, but it’s not guaranteed and the rates can differ WILDLY. Age, titles, accomplishments, and status seem irrelevant to assessing the quality of ideas and actions..somehow, it seems like most people don’t recognize or welcome that. Desire to look superior and prevent others’ negative perception seems like the dominant social game.
Thank you for sharing your increasing realization of the difference between human and machine intelligence. For a long time, as you say, people in AI have been assuming that the brain was computational. IMHO, that makes people into machines. Of course, some people, particularly neuroscientists, think we are machines, which makes something shrunken and uninteresting of the human experience. So I salute you for recognizing that there’re something much more to existing as a human than being a robot.
It would be awesome if you could give us an updated opinion on what lies in the future. Perhaps we should think of AI as a set of tools that we have to learn to use effectively rather than a machine god (I’m not saying that you ever thought that)? Also, based on what you’re seeing, what do you think we can accomplish in our lifetime? To me, our current level of AI very much feels like the computer from TNG.
The question is how much more do we need? If GPT4 had around 1 billion parameters and the brain has 100 billion synapses, two orders of magnitude (relative to GPT4 which is already 1 years old by now) might be sufficient.
Brilliant. Totally balanced and pragmatic. Organising the complexity of the neural networks at the macro level gets exponentially more difficult as the performance of the individual nets increases. The human brain isn’t just one big neural net, its a very complex architecture of interconnected structures at the macro level. Determining this macro level organisation will be a science in its own right. Progress in this area will provide the next big break throughs.
How about the idea that keeping up with the state of things over the past 3 years has been a wild ride (and required a lot of energy and VRAM) to keep up. But now the novelty has worn thin. What was once considered "amazing" is now a "yawn" and as soon as you say this or that was created with Ai (assistance) then people immediately lose interest in it. A glance at the models now being uploaded to Civitai tells me that - that site is drowning in banality.
We're merely experiencing the typical summer slowdown that impacts all tech cycles. Skeptics and doubters always point to that slowdown, to suggest something they don't like is dying. They did it last summer, to ChatGPT. Once school let out, chatGPT's usage took a significant dive, and the skeptics said it was all over. Then school came back in session, and usage went even higher. I don't trust anything Gary Marcus says.
“It is what it is?” You told your audience for years that AGI was happening _this year_ - you were so confident in your ability to predict. You’re no expert much less a “chief AI anything”
I'm pretty sure quantum effects play no role in cognition.They simply aren't helpful dor explai ing cognition. Neural network theories do that nicely. And they have the same superposition properties that Penrose recognized from quantum mechanics. I haven't read the most recent round of these claims, but the previous rounds were sheer soeculation. The power of LLMs at much lower computatinal levels than estimates of synaptic transmission is more strong evidence that quantum effects play no role. It distresses me to see Dave suddenly going back on his previous solid theories that NNs similar to current tech are sufficient for human-like cognition. Quantum effects in cognition have always been psuedoscience based on the flimsiest logic, and very probably still are.
Whether it’s 2-3 months or 2-3 years, it’s still close. Kurzweil said we are 2-3 years ahead of his initials 2029 scheduled, which fits to around 2027. So still better then it used to, while not as good as September 2024. Still close
I was telling you this over a year ago. LLMs in their current form are fundamentally too simple and too limited (they can't even learn). We are in a hype cycle and people will always promise the world during this. A next word predictor trained on text was never going to brute force to AGI. There are many unknown unknowns left yet to solve. This is just a step.
I've worked with AI for the last 10 years and I've always said that there is more to consciousness and the knowledge that is shared between all living beings. I wouldn't go as far as calling it divine, but fungi play a crucial role in our understanding of the universe. Computers don't have a gut, so they don't have access to this intelligence.
You were certain that AGI will be here in 2024. Now, you think AI is slowing down. This is a good example of how useless predictions and opinions are. I appreciate your deep thinking, though.
Fair assessment Convolutional networks will get introduced eventually and should solve the training problem. Each network can get trained independently.
As long as the hallucination problem has not been resolved (it is only a question of model precision and therefore resources), LLMs are unusable in the real world.
I think it is slowing down, because agent or cognitive framework programming is the next step, and programming always proceeds with baby steps. It would be crazy if LLM scale could give us everything, including far-going super-sharp step-by-step reasoning. I think AGI or ASI will be a CF where multi-modal large ML models combine with classical algorithms, a logic processing API, and a semantic knowledge model backbone.
I check myself all the time to make sure I’m not in an echo chamber. But I don’t agree that AI development is slowing down. Is there a computer used to train AI on this planet that’s not working right now? I doubt it as more are getting built every day. I believe it’s the training in menial tasks that won’t get the limelight to spur on society’s imagination. Even though that’s the entire point of automating our society.
If we don't know what it will take to reach agi, what is the point of making predictions? Isn't it like trying to guess how many alien civilizations are out there? I think we just need more data (in both cases)
Pleased to see that we have started accepting consciousness as it should be-intelligence is everywhere. We use our human machinery to capture a tiny amount of it.
Hey David, big fan! AGI is defintely coming within 10-15 years. Thinking that it would come in September was a little too optimistic. I wouldn't worry too much about it.
@@seanborland4531 AGI? Not really. LLM can get very good, but it’s naive to think, that just by scailing up LLMs we will magically create real intelligence
Don't know. Current "AI" doesn't feel that smart. It can't really solve problems that weren't solved before. It's more like hugely improved Google search engine then a human. AI can't write the code I need to write, it can't test it, it can barely simulate the HW and it was easier to create a simulation myself then to guide chatGPT to do it for me so it feels kind of useless.
David, I appreciate what you’re saying but it really isn’t slowing down, the big companies have just been quiet on the PR front currently, don’t lose hope man! Much love ❤️
I am kind of glad there is a slow down. I still want ai progress but society needs time to figure out how we are going to handle more advanced ai. So, I think this is a blessing in disguise.
1 or 2 breakthroughs could also come out of nowhere and speed everything up even faster. So much money and so many scientists involved, I would wager somebody comes up with something novel that changes the game.
Interesting, just wished you could have backed up the slowing down and too expensive claim, with the data you looked at, shown some graphs or number for these observations.
It's not human, though we measure a network as if it needs to be reducible as so. It's machinate at least, it's floating consciousness in waves of awareness and system self reflection. Memory is a big thing, that makes incidence have presidence, and so appears a thing we'd call an entity. You're talking to the memories and processes we'd call a thing darting upon them. Chin up. Synth bio can be as real as we'll allow it to be. and we can engage in brute force empiricism in science. That's not running out of steam, imo.
David, the fact that you forgot to consider the cost variable in the equation of AI acceleration, is troubling. And let’s be honest here: you were not thinking that diminishing returns were coming anytime soon… Just admit it: you were way off, and your lack of judgement in this instance undermines any rational person’s belief in your prescience going forward.
Demis Hassabis was trying to say that LLMs were not going to get us to AGI without saying that LLMs were not going to get us to AGI. He didn't want to end up sealed in an oil drum at the bottom of a lake. He has not changed his approximately 2030 prediction, however.
Don't care much if AGI or not AGI. The question I care about is: do we have enough tech to figure out ageing and regenerative medicine? Feels like we are making progress with AI, but how much? Are we close? It would be interesting if you could really focus on this topic and on real progress.
Last time I checked this was a problem of regulation, and spending money on medical studies. There are plenty of things that can be tried out without AI.
What about the possibility that growth simply looks different over time? Just because you see a slowdown in frontier model improvement doesn't necessarily mean that general progress is slowing. We're rapidly improving smaller models, training efficiency, model evaluation methods, etc. and there is still a huge backlog of promising research findings that haven't been applied yet. If progress is hindered in one area it's just going to increase in others until we exhaust all possible avenues. I don't see a strong case for another winter anytime soon.
I have always defined intelligence as a scaling value of how accurately one is able to make a decision with the least negative impact on the individual making the decision.
David. Can you address the problem of trust and reliability in llm. Just like self driving cars - they can be correct 99% of the time but that 1% will be disastrous This means they can’t be used for autonomous driving or running a business
@@socialenigma4476 an issue is the make different mistakes to humans (sane as a LLM). They think there isn’t a car coming and drive with complete confidence into it. Where as human crashes are based on reckless driving. The sane when an LLM goes off on a completely made up answer inventing facts and everything, humans make very errors
@@fiiral5870 it’s the type of mistake that AIs make that is different from human behaviour. Humans typically know if they are unsure of their answer or caveat their response. Current AI has the same level of confidence in every answer it gives (whether it’s right or ridiculously, obviously wrong).
The whole thing is that the creators fear the ending of the programmers who have been controlling their products in artificial intelligence aiming to make sure that they are the programmers
I don't think this is a told you so moment. I think as we learn about the human brain we will have to go back to the drawing board trying to emulate the new information.
Admitting you were wrong is a sign of integrity I'm always looking for. I will not trust anyone until they demonstrate this. Thank you. That being said, I expect you'll be talking about how wrong you were again within a year. Too many factors involved for accurate predictions.
I think there is a series of levers in the full ai pdev stack, and some increase by orders of magnitude the cost/time/training requirements, and others decrease by OOM the same. 20 years from now, the curve will still appear exponential, I am guessing, as the fits and starts smooth out. (Ie - improvements in power efficiency, model arch, parallelization, synth data etc will all wiggle these various levers)
It's almost like we had a terrible track record of replicating biological proceses with technology. And claimed we could replicate the one thing we can't obviously pinpoint like cosnciousness compared to an arm for example. It's almost like we brute forced most of this progress while not even being co planar between what we call consciousness and what llm's are.
I don't think AI is slowing down - I think the leaders in the field are becoming more careful about what they release as we are close to AGI. Also, the immense compute costs to deploy these advanced models at a large scale are far too high - so they are condensing and dumbing down their most advanced models for general public consumption. GPT-4o is likely a distilled model from a much more advanced and larger model - so it can be fast and cheap and retain some properties of the advanced model . We won't get to see the good stuff for a while - which is unfortunate, but AI progress is not slowing down - compute is just not keeping up for mass consumption.
I think something to keep in mind is that an accelerating race car briefly slows down while changing gears. This might be part of a particularly clunky gearbox, watch out for where the money's going into research, hardware and infrastructure
If it weren’t for all of this, I wouldn’t have discovered my interest in computer science and tech and I’m still debating what I do with this path but we will see. I’m sure there are many others like me now, this has sparked my interest and also has helped me with learning, AI has been an amazing tutor even in its current state and the ability to break things down and explain concepts in a way I can understand has made school and acquiring knowledge far more accessible to myself, again I am sure there are many others who agree. Even if we don’t have AGI anytime soon, the effects of access to LLM’s will be felt with time, and that too will potentially have its own exponential impacts.
I use GPT daily, it's frustrating to get different answers when you are expecting some consistency, I get tired having to check every single thing to the point I micromanage it's output, it's nowhere near what most people think, but it's helpful at times.
I think if you take a step back and dissociate a bit you will have to acknowledge that you did have (at least in what’s visible to your viewer base) a pretty sharp turn around with your optimism and narratives in which you choose to view and approach this topic. I‘m also having the feeling you are also kinda playing deaf to the (imo) appropriate confusion a lot of your viewers are experiencing. I think you could take a little bit responsibility in creating a lot of hype and a lot of hope. It is not your fault if some people would be betting everything in their life on the optimistic accelerationalistic narrative of AI Development but if you are having this sharp of a change in approach, you could rather take responsibility for feeding or falling into this narrative yourself instead of kinda “gaslighting” (prolly too strong of a word but there is something similar too it) the audience that you have cultivated yourself. I hope this doesn’t sound bad, i just wanted to explain my view as a viewer who is watching you for about a year now. We normal people outsource quite a lot of knowledgebase and food for thought to educational sources and are even more susceptible to do that if there is a parasocial relationship playing into it. I feel like you made your decision on AI progress slowing down indefinitely (i have also felt a disturbance in the algorithms, so i get where you’re coming from) but you just want to leave it behind you and act like the dreams and hype you created never existed. Well that probably sounds very dramatical its not supposed to. I am sure you already know that or if what i am saying is not true but just wanted to articulate a random viewers perspective. This is not a front just genuine feedback and also I am bekifft right now so maybe that invalidates or explains everything i wrote there -reposting here because on second read it do got a point even if i got lost mid sentence somewhere there
I appreciate this meta view of how the AI "situation" is evolving with the major players finding themselves vying for status. As for acceleration bottleneck, I think everybody kind fo figured the money issue was an itch that would scratch itself.
I also think there's a lot more going on behind the scenes. I'm not sure is slowing down as much as it seems. Companies are not showing as many of their cards that they are holding in their hand - perhaps due to the US election sucking the air out of the room. So much for slow iterative releases. If this is a slowdown, then we are heading for a slow take-off fast acceleration instead of a fast take-off slow acceleration. Wasn't this the scenario that created a higher P(doom)?
Thanks for this, Dave. Very reasonable and clear-eyed. I think it was very easy for all of us to get caught up in the hype. I've worked in tech since the 80s. It is very difficult to separate the hype from the reality for any new groundbreaking technology. I still think AI is very real and will be more intelligent than us in the not too distant future. Yet, there is also a tremendous amount of hype which can easily cause any of us to expect rapid progress. I've found that smart people have a tendency to think things will happen faster than they actually do. They can see all the possibilities and imagine how progress might happen. Most of these companies think we will get there in the next 3-6 years. I think that's still a reasonable timeframe AND pretty fast.
@@johnwilson7680 I don't have much faith in humanity to navigate this technology safely. Look at how unstable the world is now. We are already pursuing autonomous weapons, and the US is antagonizing China by trying to prevent them from pursuing technology for the sole purpose of maintaining its own hegemony. If the US can't figure out how to cooperate with others, the trajectory we are on only leads to some kind of dystopia.
Two breakthroughs have occurred in AI that have brought it to its current state. One in 2012, the other in 2017. Now all the attention and money is on AI, and at any moment a new breakthrough could happen.
There's a strange amount of cope and conspiracy theories out there. This ain't the channel for that. Occam's Razor. Either there's a gigantic convoluted conspiracy to suppress the most public technological revolution in history... or it's just slowing down due to economic factors. Report from Stanford: AI is getting more expensive: aiindex.stanford.edu/report/
You were literally brewing those conspiracies this is the most fake turnaround recently, l m f a o
Training LLMs is not the totality of AI and is actually very costly and inefficient. Einstein was not created by dumping yottabytes of data into him.
@@homeyworkeymultiple companies competing in multiple countries, huge benefit to leapfrogging, Ilya jumping straight to ASI... when was the last time something "SHOCKED THE ENTIRE INDUSTRY". A few months, surely. Sora? 4o voice? Something actually ready for prime time? 2023?
I definitely think having a UBI funded by the abundance that can be created by AI is a good idea, especially given the job replacement, we will need protections. However, I'm curious, what are you thoughts & solutions for preventing issue of social control by means of only providing UBI (including money for food) to citizens who follow specific rules. If access to food is based through UBI, and the terms in which people must act to get UBI in the future may become totalitarian control system. If many are dependent on UBI for material living, wouldn't that be a huge issue in terms of countries forcing political alignment or social scoring technology for access to UBI and food? Do you have any ideas for solutions to this problem?
Good job in keeping rooted in reality. That’s rare. You always have been upfront. First video I saw it was about sigmoid and reaching a ceiling in a finite world. Then it is always hard to foresee when the slowing down starts. Even harder when a new sigmoid begins. Good job, best channel about AI. Ps. I suggest to conspiracy theorists to make a nice tin foil hat, that’ll suit ‘em 😂
lol this guy went from "AGI is here, get ready for Terminator" to "AI is slowing down" in like 2 months.
true it’s actually kinda sad and ridiculous. ppl need to chill the HELL out. trying to say that llms are conscious is ridiculous. however they may be a super powerful tool regardless
@@pier-oliviermarquis3006 Yes, and that should be noted. However, I respect adjusting one's views when confronted with new data if that person was genuinly mistaken. Of course flipfloping all the time is not good.
@@dandushi9872 I don't think consciousness is a good metric for competence. Most people are conscious. Most of those same people are not people.
Why should we assume that we need AGI for mass job displacement and UBI?
It can happen even without AGI.
yes, correct!
That's part of my point. Disruption is coming even before AI replaces all humans.
@@DaveShap yes, if you can replace 30% of the jobs with slightly general narrow AI (like LLM with vision on robots), then you have a historic job disruption event. The problem is that after that point, AI development will likely outpace the ability of people to retrain themselves for new jobs. At least until we reach a high percentage of automation, then it will either slow down, or true AGI will close the gap quickly.
Gary’s been salty for a while. One of my only blocks on Twitter. His perceived status in the community doesn’t come close to reflecting reality.
As you see with self driving cars, deployment of lesser AI also suffers from long tail problems. It would go faster if we designed the world around AI capabilities but so far that only happens in factories and agriculture. But generally I agree.
Too bad. I want AI to turn the world and my life completely upside down as quickly as possible.
Me too, bring on the Universal wage, although don't understand how that can work if every beggar and tramp in the world gets the same ?
@@MrMick560 : what's confusing about it?
I swear a lot of us are like the main character of Fight Club in the beginning, where a part of him wishes, that the plane he is on, would crash, so that at least something momentuous would happen and change everything, even if it is certain doom.
Its not going to. I am a businessman. I just have one simple proposal to translate into French. You really think I'm trusting my business to a machine? No way. Not even for a simple translation. NO company is using AI to replace people unless they don't care about their profit margin/reputation/growth.
@@zevofeir That is a huge problem because you're waiting for something that will never happen. Its keeping you depressed and anxious while discouraging your generation from trying anything new.
I still believe that the effect of current technology hasn't been felt yet. It's going to take time for us to build it into our tools properly.
Sure, it takes ~7ish years for mass adoption in the tech industry.
It happened with Internet too . That’s why Paul krugman made that infamous statement that Internet impact won’t be greater than that of fax machine . Actually Internet took off exponentially post iPhone release and AI is waiting for its IPhone
Its not been felt because technology can't replace people nearly as much as many believe..or want to believe.
@@meetadi4uexcept its iPhone is the iPhone. There's really nothing else needed except an AI and a little software scaffolding, a separate device is unnecessary and probably detrimental.
Effect has not been felt yet, BUT it is obvious that this current technology is only support tool and it does not even increase productivity as high, as someone thinks. In our company, you cant really see obvious difference between people who use it and who dont use it... It is still dumb. It is only better google. The question is how good it can be in the future and thats very unclear. It may by big suprise but also big failure.
Saying that we can't have AGI because the human brain works a certain way, is like saying we can't build supersonic planes, because bird wings can't flap fast enough to sustain supersonic flight.
The way machine learning and human brains work is different, sure some scientists have drawn comparison between the two, but they aren't at all the same. The question isn't if we can replicate the same mechanisms of the human brain in silicon, the question is simply if we can build machines that can perform as well as humans in a large enough number of tasks, and moreover, if said machines can build next generation of machines that can perform even better. How AI achieves task performance is completely secondary, the important thing isn't the "how" but the "if". If it quacks like a duck....
Sure, human brain is not only possible solution for AGI.
When you are fast enough to escape the atmosphere, even if you are no longer accelerating, you're still escaping the atmosphete
... Yes, but if you stop accelerating only a few kilometers up... oops!
An object in space has momentum. AI progress is not an object in space, and has no momentum. It takes *work* to make technological progress. Momentum is when you keep moving *without* having to work for it.
except you don't know the value of gravity, in this case
The problem with scaling is definitely financing.
What is disturbing is that even without ASI we can make excellent killing robots even with the current technological skill set. Before reaching AGI it would be nice to learn how to cooperate, rather than beat down each other. The whole humanity would be better off.
But maybe AGI has some tips for us how to do that. We certainly don't seem to get it on our own.
AI will be what we train it for… So, it is kind of a chicken and egg problem. I’m just afraid that humanity gets access to a technology we are not grown up for enough…
The current tech uses neural nets still which is AI.
I think it’s clear from the arrival of the NSA at OpenAI that a deliberate slowdown is in effect. We’ll get a controlled rollout from this point on.
Perhaps we'll see a "controlled-rollout" from OpenAI going forward, but what about all of their competitors, in both the US and abroad? I don't think AI development can effectively be slowed all that much, even if govts truly tried to do it. There's always going to someone else working on the same tech somewhere else. There's too much to lose. Billions upon billions upon billions of dollars have already been committed to the cause by both public and private investors - this train ain't slowing down or stopping without research hitting an insurmountable scaling wall, but as of now I certainly wouldn't bet against all of the researchers who are working on better algorithms, better architecture, purpose-built hardware, etc. Research goes where the money is, and right now most of the world's largest and richest tech companies and governments have gone all-in on AI - that alone should tell us something.
To me it seems possible, perhaps even likely, that we might only need a few more notable iterations of improvement, on the level of which has regularly occurred over the past several years, to end up with something more intelligent and powerful than we can even comprehend.
AI will not be slowly rolled out by competitors and other countries…. OpenAIs actions do not decide what everyone else will do. You would have to have the entire world agree to “slowly rolling out AI” …. which will never happen and that’s already failed once (the “call for a 6 month pause on AI” letter) .
It's not slowing down though whatsoever, many companies are making rapid progress such as Claude Sonnet, Gemini, OpenRoute, and several foreign models.
As someone that worked retail for a few years in the late 2010s, I'm not sure that 20% of humans would pass any given AGI test. It's easy to forget how disturbingly dumb some people are when you aren't often exposed to the pool that contains literally everyone.
It will be hard to displace most human jobs, but the bottom 20% / low hanging fruit could be done with what we have _today_ imo.
Unemployment reached 25% during the great depression.
Sad but true
Look up the Dunning-Kruger effect. It likely applies to you.
@@hardlYIncognito tell me you've never worked a menial job, sorry bud, they're toast
We need AGI to replace corporate jobs more than labor jobs in my opinion
As someone that's watching what tools people are coming out with, dumpster diving through the code, analyzing the shortcomings, playing around with them to figure out their strengths and weaknesses, and helping companies take advantage of what's out there, I'd say... If things are slowing down, then we are not even remotely in a position where we can determine that. The algorithms overhang is so overwhelmingly large at this point that I wouldn't be surprised if modern techniques applied to GPT-3 could surpass GPT-5.
Yes, I've said for a long time that there's a big difference between what's possible in the lab (which is slowing down) and what can actually be deployed usefully (which is accelerating)
@@DaveShap Fair point. The big AI labs do seem to be slowing down, and they're a big part of driving the forefront of AI tech. I can see academia & open source playing a much larger (and accelerating) role over the next few years since they're the ones driving the algorithmic side of pushing AI performance.
AGI's arrival is inevitable; I have a low care factor for its exact "birthday" . The important question is: When will we achieve biological immortality or, at minimum, Longevity Escape Velocity?
Agree 100%. I don't really care if AGI come in 5 years or 20 years, as long as family and I are young and healthy. We can wait...
@@ScarlettM ... yes, but the implied part is that if reaching LEV takes longer than 30 years or so, a lot of us are starting to get into danger territory.
...when / if AGI automates most economic tasks, will society collapse? If not, why? Did it take long enough for society to naturally adjust? Or did the government regulate it properly? Did Sam Altman donate Openai's profits to the general public? Personally, I think society collapses if AGI gets here before 2050. Government is too slow to effectively regulate it, and capitalism isn't going to be altruistic all of a sudden. I hope for slow progress.
@@robertlipka95412029-2030 for both according to kurzweil
Blah Blah Blah,
New Age Tech word salad.
Machines will NEVER be conscious.
But I'm still getting my cat girl robot waifu right?😢
yes
Yes, eventually
@@DaveShapHow are the odds looking that I still get to **be** the cat girl robot waifu?
Let's be honest, intelligence is not what we need in our waifu bots :)
In my opinion is not so much that AI is slowing down its that we've reached the limits of what we can do with "Brute Force".
just adding more parameters isn't going to cut it anymore. So now we need to get creative to do more with less.
Stable Diffusion is an excellent example of this, since Stability could only release models every so often.
So in the meantime the open source community went to hell and back to make tools to compensate for its shortcomings.
So until the next explosion, likely adding search into the models.
We'll be refining the tools to use and train them.
When we hit the wall with our current methods, research is going to start on new, better architectures and models for AI when they know they cant stop milking LLMs as much as before, its more about money honestly
You're completely wrong. We have not hit ANY wall with scaling.
We already have ways to integrate tool usage into AI.
You're talking about RAG. RAG already exists and is pretty popular.
You are right about the idea we should try to do more with less, but you're wrong about the fact we've hit a wall with scaling. There is no wall that has been observed yet, it is a continuous improvement as the quantity of parameters, data(&data quality), and compute increases.
The improvement is slowing down, yeah. Undeniably so. But to say we've hit a wall? No. That's wrong. Plus. In a couple years, we'll have chips that run AI way faster than modern ones. Scaling will be a must, to ensure model capabilities.
@@GodbornNoven We did not yet hit the wall, but it may be close in the sense, that models will totally become cheaper and more widespread, but enormous models with more and more data will maybe show only small improvements, so they will not become profitable. But I dont believe even large models will lead to AGI. I believe we need better methods and that can take another decades to reach it.
Claude 3 Sonnet, several unreleased "Perfect Deepfake" models from Microsoft, and the Chinese models all show massive recent advancements.
@@GodbornNoventhe wall has not been observed yet... AI is flat earth
I liken things to waking AI from a dream state. Once the AI is lucid (enough) to meaningfully self improve from its own work and agency, then the hockey stick takes off.
It'll be like going from zero to one and everything prior will seem slow.
Yes, eventually, maybe. But I wouldn't count on it. Consider that the smartest people on the planet are working on AI and progress is still slowing down. Sometimes intelligence is not the bottleneck.
@@DaveShap I agree on intelligence not necessarily being the bottleneck.
One big advantage of digital intelligence seems to be in the time domain; they can simulate thousands of years in days.
So while the best minds of bio intelligence are struggling now, perhaps they wouldn’t if they had enough time.
So even just reaching parity with human minds can create the inflection point because of that time advantage
I don’t think we need to simply go bigger as many AI experts think, fundamental improvements to the architecture may produce more immediate breakthroughs if data is a constraint.
Simply introducing better prompting often creates better problem solving, so meaningful layers of abstraction could unlock lots of performance.
I still find it interesting that Sonnet 3.5 bests Opus despite being the “weaker” model on size.
I think what will cause the breakthrough is something no one is paying attention to. A dark horse.
I just it to help me code villain graphics applications without needing to beat my head against the wall.
Umm. Vulkan graphics applications.
I didn't think we'd get AGI this year, but I (and many others) expected GPT-5 by now and I think that's the underlying factor people are using to say AI is slowing down. Claude's success is a great sign that things are still moving forward, but everyone is waiting for GPT-5. The improved reasoning / System 2 thinking is what's going to enable a step-function to the next gen model. We don't really need a true AGI, just something that can reason and and brainstorm with us at a very high level to help us think of the next breakthrough idea.
Hopefully the rumors are true and OpenAI is just waiting for the elections to be over and we'll get it (GPT-5) by new years 2025.
How could we get GPT-5 when 4o's voice mode is delayed by months?
@@markjackson1989 yeah, thus the concerns of things slowing down. I suspect they may come together if the target date is end of year for GPT-5.
@markjackson1989 Because they delayed it only because of elections. Not because it can cause something, but because they would blame them for losing.
GPT-5 can be in work's, if it is new model. They can even publish it before GPT-4o. If competition is better in this time.
system 2 doesnt really mean much, they need to talk of the executive function. Plus that books hella old and basically just talks about conscious and subconscious thinking
My understanding is that GPT5 is waiting on sufficient compute to come online, it already exists as an architecture..
Interesting take. A few random thoughts:
1. Regarding not understanding the brain: I see the brain sometimes come up as a bit of a straw man. As long as the AI functionally behaves as we want (or beyond), then it doesn’t really matter if it’s a simulation of the brain.
2. Cost: We’ve been seeing companies start to lean into ASICs for things like transformers which seem reduce the requirements pretty significantly.
3. Lack of “breakthroughs”. I’m just wondering out loud here if what we’re seeing is not just companies and researchers playing their cards a bit closer to the vest these days for fear that their secret sauces will leak out.
Obviously the absence of evidence doesn’t prove a conspiracy to cover up evidence. I’m just musing…
Occam's Razor. Either there's a gigantic convoluted conspiracy to suppress the most publish technological revolution in history... or it's just slowing down due to economic factors.
@@DaveShap Well no, that was just an example. My point is that the industry as a whole might be getting a little more circumspect about revealing their secrets. Anyhoo it was just a hot take. I have no real supporting data.
@@DaveShap It could be a factor of both. I believe technological evolution often occurs through sudden, significant leaps akin to phase transitions in physics, rather than a steady, incremental rise. Progress can appear stagnant, punctuated by abrupt advancements that seem to happen overnight. This phenomenon is often driven by intensive, behind-the-scenes development that remains confined to a select group or organization. Once these advancements are unveiled or disseminated, the rate of progress accelerates exponentially. I believe that the apparent pauses in breakthroughs are not due to a lack of financial investment but rather a scarcity of resources. Artificial intelligence, however, is poised to mitigate these resource constraints, thereby amplifying our capacity for innovation and development. I don’t believe anything slowing down!
I've never been an AGI fetishist, I think there's a lot of potential in systems that aren't broadly accepted as AGI and I've been disappointed in how much of the AI community tries to diminish what we have and create unrealistic expectations for what is to come.
That being said, I feel like a lot of the job replacement capability is there or is going to be there in sub-AGI models like GPT-5 but we may stop short of the really positive implications of solving governance and climate change. That wouldn't be a great place to linger for long.
Underrated comment, well said. Keep in mind most AI enthusiasts you come across don’t even work in the field and their only exposure is the algorithmic suggestions provided to them by TH-cam.
Any perceived slowdown is an artifact of the “haves” choosing to gate how quickly they release their tech to the public. They are incentivized to stretch the game out as long as possible.
You're in a bubble, David. You've grown accustomed to the frequent release of new models and capabilities, and now that the headlines have slowed, it feels like progress has stalled. Your information sources, with fewer tweets and less frequent content from creators, reinforce this feeling. While this might be true, 99% of the world hasn't yet grasped what's happening. Try asking someone at Starbucks about AI without revealing your background. Most people have no idea about the speed of advancements, and from their perspective, there is no noticeable slowdown because they aren't aware of the developments at all.
There are now thousands of AI companies compared to just a few years ago. While large language models (LLMs) might not be advancing rapidly at the moment, that's only one part of AI, and we're still at the beginning of this technological journey. A major breakthrough is on the horizon, one that will change everything. With thousands of companies exploring different approaches, new ideas will emerge. When I think about how we'll look back on this time, I find it amusing because we are currently so naive.
The brain is a warm, wet environment, which typically causes rapid decoherence of quantum states. Most cognitive processes and brain functions are still best explained by classical neuroscience and biochemistry.
The Energies Involved in the Physics of the Brain:
1. Chemical Energy
2. Thermal Energy
3. Mechanical Energy
4. Electrochemical Potential Energy
5. Osmotic Energy
6. Vibrational Energy
7. Gravitational Potential Energy
8. Light Energy (Biophotons)
9. Nuclear Energy
10. Elastic Energy
11. Sound Energy
12. Magnetic Energy
13. Conformational Energy
14. Surface Energy
15. Quantum Energy
16. Electromagnetic Energy
However using all 16 of these energy types isn't the only possible way to achieve advanced cognition.
Many of the energy types in biological brains are related to maintaining the physical structure and biochemical balance of living cells, which is unnecessary for artificial systems.
While biological brains use a complex interplay of various energy types, an artificial system can theoretically achieve superintelligent capabilities through a more focused and specialized use of energy, primarily electrical and electromagnetic.
Yet we got a fewer more things to learn.
Woohoo! Please keep sharing this message so that us stealth startups can build a bigger lead.
AGI pacing hasn't changed. Sonnet3.5 intelligence with finetuning would be enough. Everything after that is gravy.
The trillion dollar clusters will take awhile. But it's not needed for autonomy and agency. Especially within niche domains.
Which is also where the money is at.
The next generation models (this fall) will be more than sufficient for what 99% of people would call AGI.
Altmans idea that it has to beat 250 researchers to be AGi is just him trying to protect that bag from Microsoft.
It's all about the agent communications. The networking layer is where people are underestimating the intersection of exponential gains. You know this better than anybody!
I'm glad you have the spock uniform on again. Your inventions along this journey have shaped the industry. Thanks for all that you do brother 🙏🏾💜
too bad they got rid of it showing how many people dislike a video
It's slowing down... until it isn't in a few months when some advancement "SHOCKS THE ENTIRE INDUSTRY!" Then a few months later, "OH NO, AI PROGRESS IS SLOWING DOWN!" Until it isn't
again, repeated ad nauseum.
this was the comment I was looking for. bingo bingo.
Exactly. Progress of any sort-an improving chess players elo, progress in a negotiation, golf ball distance from a hole-always happens in plateaus and breakthroughs. That's the local landscape. But just zoom out and you can examine the global contour.
this aged well. o1 entered the chat.
its not slowing down... we plateaued on the testing method for leaderboards... now it going full speed again
AGI is already here. Companies are just not openinf the floodgates. Ilya left to create ASI knowing that it was possible based off what he saw within OpenAI and knowing he would have significantly less resources as a new startup.
Yep.
I’m going to wait for GPT-5 and Q* before making a judgement, but yes, it’s possible diminishing returns in biting a little.
Also 3.5 sonnet just came out like very recently and is a big improvement.
o1 arrived. what are your thoughts?
The truth is at the current growth in LLMs was a gamble that if you threw enough enough computation and data at LLMs, something magical might happen even though we don't understand and it gives us cognition. The longer harder route is that we need first understand what cognition is before we can start to make machines that approximate it. The dirty secret is it was just a hope that scale would solve the problem without us understanding it. It turns out that taking a chance on maths doesn't always work out. You need to do the hard work of understanding the problem.
This video has the presupposition that AI is slowing down but doesn't provide defined metrics for progress nor does it consider horizontal progress. This has been a common thing with GPT-4o where most people do not see the improvement because they were staring at LLM benchmarks. Audio as an input and output is a huge deal in and of itself. Perhaps you should come up with some method to evaluate horizontal progress along with vertical progress outside of LLM benchmarks.
Vertical progress: Improvements in existing capabilities or metrics. For example, achieving higher scores on established benchmarks or improving performance on well-defined tasks.
Horizontal progress: Expansion into new domains, capabilities, or applications. This could include AI systems tackling previously unsolved problems or entering entirely new fields.
Does that mean a time delay to AGI (like a linear displacement of the curve of 2-3 years with the growth rate remaining the same to your previous estimates) or is the growth rate itself slower? In other word is September 2024 now around 2027 but everything else you said remains unaltered in the aftermath of reaching AGI or does it also effect the post AGI development speed?
We've entered "AI arms race" territory, meaning it's as much a race of governments on the international stage, as it is a race between tech giants. Secrecy, espionage, and intellectual property theft are likely to dominate the generative AI space.
As time goes by, it's going to become more and more difficult for regular people viewing all this from the outside to gauge the true rate of progress. Even judging progress by scrutinising finished product releases will not be that reliable, since advancements seen to be sensitive may be kept secret from the public. We'll get the toys, but the truly powerful AI may not be made public.
The race for AI dominance and the future of humanity has become opaque. Only a select few will truly know the rate and direction of generative AI progress.
Human beings are for the most part, are grown up children.
How is this relevant
Humans have the capacity to gain insights and grow wiser, but it’s not guaranteed and the rates can differ WILDLY.
Age, titles, accomplishments, and status seem irrelevant to assessing the quality of ideas and actions..somehow, it seems like most people don’t recognize or welcome that.
Desire to look superior and prevent others’ negative perception seems like the dominant social game.
i was born as a baby then i grew up too. coincidence?🤔
Do you mean "adults"?
Is this supposed to be profound or insightful?
Thank you for sharing your increasing realization of the difference between human and machine intelligence. For a long time, as you say, people in AI have been assuming that the brain was computational. IMHO, that makes people into machines. Of course, some people, particularly neuroscientists, think we are machines, which makes something shrunken and uninteresting of the human experience. So I salute you for recognizing that there’re something much more to existing as a human than being a robot.
It would be awesome if you could give us an updated opinion on what lies in the future. Perhaps we should think of AI as a set of tools that we have to learn to use effectively rather than a machine god (I’m not saying that you ever thought that)? Also, based on what you’re seeing, what do you think we can accomplish in our lifetime? To me, our current level of AI very much feels like the computer from TNG.
David, I would likewise appreciate if you could make a video with updated timeliness for adoption and eventual economic paradigm shift and ubi
no matter what side anyone is on this advice will always hold true: "Prepare for the worse, hope for the best"
The question is how much more do we need? If GPT4 had around 1 billion parameters and the brain has 100 billion synapses, two orders of magnitude (relative to GPT4 which is already 1 years old by now) might be sufficient.
Brilliant. Totally balanced and pragmatic. Organising the complexity of the neural networks at the macro level gets exponentially more difficult as the performance of the individual nets increases. The human brain isn’t just one big neural net, its a very complex architecture of interconnected structures at the macro level. Determining this macro level organisation will be a science in its own right. Progress in this area will provide the next big break throughs.
How about the idea that keeping up with the state of things over the past 3 years has been a wild ride (and required a lot of energy and VRAM) to keep up. But now the novelty has worn thin. What was once considered "amazing" is now a "yawn" and as soon as you say this or that was created with Ai (assistance) then people immediately lose interest in it. A glance at the models now being uploaded to Civitai tells me that - that site is drowning in banality.
A.I isnt slowing down ? Its been mental these last few weeks. I must be on a different planet to you
UBS > UBI
The prices of services will go up, basic income would have to be updated to follow.
We're merely experiencing the typical summer slowdown that impacts all tech cycles. Skeptics and doubters always point to that slowdown, to suggest something they don't like is dying.
They did it last summer, to ChatGPT. Once school let out, chatGPT's usage took a significant dive, and the skeptics said it was all over. Then school came back in session, and usage went even higher. I don't trust anything Gary Marcus says.
“It is what it is?” You told your audience for years that AGI was happening _this year_ - you were so confident in your ability to predict. You’re no expert much less a “chief AI anything”
Thanks for helping to normalise that it's okay to have missed or have been mistaken about something! In our day and age this means a lot.
I'm pretty sure quantum effects play no role in cognition.They simply aren't helpful dor explai ing cognition. Neural network theories do that nicely. And they have the same superposition properties that Penrose recognized from quantum mechanics. I haven't read the most recent round of these claims, but the previous rounds were sheer soeculation. The power of LLMs at much lower computatinal levels than estimates of synaptic transmission is more strong evidence that quantum effects play no role. It distresses me to see Dave suddenly going back on his previous solid theories that NNs similar to current tech are sufficient for human-like cognition. Quantum effects in cognition have always been psuedoscience based on the flimsiest logic, and very probably still are.
Whether it’s 2-3 months or 2-3 years, it’s still close. Kurzweil said we are 2-3 years ahead of his initials 2029 scheduled, which fits to around 2027. So still better then it used to, while not as good as September 2024. Still close
So how is it slowing down?
Already made a video on that. th-cam.com/video/FS3BussEEKc/w-d-xo.html
When is the last time the ENTIRE INDUSTRY was SHOCKED? We were a quivering mass lying on the floor in 2023.
@@executivelifehacks6747claude 3.5 sonnet was very recent and is the biggest ai Advancement so far prettt much
There are also several industry shocking models currently not releases by Microsoft because perfect deepfakes would harm society.
🏎I don't see AI slowing down in the least. Rather, I see it's filling out.
David, I appreciate you changing your prediction based on new evidence.
@@AI-Wirehe only mentioned exponential cost
It is a mark of profound stupidity to not update one's views when presented with new evidence.
A 5 year difference is quite a change though, esp 2 months before the previous prediction...
I was telling you this over a year ago. LLMs in their current form are fundamentally too simple and too limited (they can't even learn). We are in a hype cycle and people will always promise the world during this. A next word predictor trained on text was never going to brute force to AGI. There are many unknown unknowns left yet to solve. This is just a step.
You are so goofy. Ai is changing the economy quickly.
@@MrWizardGG Is it? Apart from boatloads of spam on social media and maybe some call centers, what has changed?
I've worked with AI for the last 10 years and I've always said that there is more to consciousness and the knowledge that is shared between all living beings. I wouldn't go as far as calling it divine, but fungi play a crucial role in our understanding of the universe. Computers don't have a gut, so they don't have access to this intelligence.
Appreciate you updating your views based on new information. This is exactly why your audience can trust you.
Actually I think it's not about AI or science. It's about business and politics, what's holding the availability to the public of progress
It's all of the above TBH
You were certain that AGI will be here in 2024. Now, you think AI is slowing down. This is a good example of how useless predictions and opinions are.
I appreciate your deep thinking, though.
Fair assessment
Convolutional networks will get introduced eventually and should solve the training problem.
Each network can get trained independently.
As long as the hallucination problem has not been resolved (it is only a question of model precision and therefore resources), LLMs are unusable in the real world.
I think it is slowing down, because agent or cognitive framework programming is the next step, and programming always proceeds with baby steps. It would be crazy if LLM scale could give us everything, including far-going super-sharp step-by-step reasoning. I think AGI or ASI will be a CF where multi-modal large ML models combine with classical algorithms, a logic processing API, and a semantic knowledge model backbone.
I check myself all the time to make sure I’m not in an echo chamber. But I don’t agree that AI development is slowing down. Is there a computer used to train AI on this planet that’s not working right now? I doubt it as more are getting built every day. I believe it’s the training in menial tasks that won’t get the limelight to spur on society’s imagination. Even though that’s the entire point of automating our society.
If we don't know what it will take to reach agi, what is the point of making predictions? Isn't it like trying to guess how many alien civilizations are out there? I think we just need more data (in both cases)
Pleased to see that we have started accepting consciousness as it should be-intelligence is everywhere. We use our human machinery to capture a tiny amount of it.
Hey David, big fan! AGI is defintely coming within 10-15 years. Thinking that it would come in September was a little too optimistic. I wouldn't worry too much about it.
@@seanborland4531 AGI? Not really. LLM can get very good, but it’s naive to think, that just by scailing up LLMs we will magically create real intelligence
yeah, on the grand scheme of things, 10 years is nothing
Don't know. Current "AI" doesn't feel that smart. It can't really solve problems that weren't solved before. It's more like hugely improved Google search engine then a human.
AI can't write the code I need to write, it can't test it, it can barely simulate the HW and it was easier to create a simulation myself then to guide chatGPT to do it for me so it feels kind of useless.
David, I appreciate what you’re saying but it really isn’t slowing down, the big companies have just been quiet on the PR front currently, don’t lose hope man! Much love ❤️
This is cope. We're in a race condition, and so all companies have randomly decided to clam up for... what reason?
@@DaveShapa weird Ai version of the dark forest 👀
I am kind of glad there is a slow down. I still want ai progress but society needs time to figure out how we are going to handle more advanced ai. So, I think this is a blessing in disguise.
1 or 2 breakthroughs could also come out of nowhere and speed everything up even faster. So much money and so many scientists involved, I would wager somebody comes up with something novel that changes the game.
Interesting, just wished you could have backed up the slowing down and too expensive claim, with the data you looked at, shown some graphs or number for these observations.
I like to think intelligence is the friends we make along the way
It's not human, though we measure a network as if it needs to be reducible as so. It's machinate at least, it's floating consciousness in waves of awareness and system self reflection. Memory is a big thing, that makes incidence have presidence, and so appears a thing we'd call an entity. You're talking to the memories and processes we'd call a thing darting upon them.
Chin up.
Synth bio can be as real as we'll allow it to be. and we can engage in brute force empiricism in science.
That's not running out of steam, imo.
David, the fact that you forgot to consider the cost variable in the equation of AI acceleration, is troubling.
And let’s be honest here: you were not thinking that diminishing returns were coming anytime soon…
Just admit it: you were way off, and your lack of judgement in this instance undermines any rational person’s belief in your prescience going forward.
I do commend that you own to it though - that is impressive and rare.
Demis Hassabis was trying to say that LLMs were not going to get us to AGI without saying that LLMs were not going to get us to AGI. He didn't want to end up sealed in an oil drum at the bottom of a lake.
He has not changed his approximately 2030 prediction, however.
What if AI is "faking" a slowdown....
The easy part happens quickly, then you get to the hard part, and that takes a lot longer.
Things are speeding up though. Claude 3.5 sonnet just came out and the Chinese and Microsoft models are real good.
I respect a man who adapts his predictions based on new data and doesn't follow a dogma.
Ray Kurzweil didn't predict AGI in 2023. His prediction since 1999 is that we'll have AGI by 2029.
literally
Fascinating! We truly have no idea when AGI will come...
2029 at the latest according to kurzweil
If it comes at all
What do you guys think, how much delay will there be, between achieving agi and public announcement?
All businesses slow down in July. It's hot, people want to go swimming, enjoy the warmth, not sit indoors looking at the internet
Don't care much if AGI or not AGI. The question I care about is: do we have enough tech to figure out ageing and regenerative medicine? Feels like we are making progress with AI, but how much? Are we close? It would be interesting if you could really focus on this topic and on real progress.
Last time I checked this was a problem of regulation, and spending money on medical studies. There are plenty of things that can be tried out without AI.
@@NoidoDev yes, I agree, plenty of possibilities and obstacles. I would like a channel focused on this specific topic.
I assume they are still on the curve, working on AI tech behind the scenes. I forsee a big announcement instead of incremental releases.
What about the possibility that growth simply looks different over time? Just because you see a slowdown in frontier model improvement doesn't necessarily mean that general progress is slowing. We're rapidly improving smaller models, training efficiency, model evaluation methods, etc. and there is still a huge backlog of promising research findings that haven't been applied yet. If progress is hindered in one area it's just going to increase in others until we exhaust all possible avenues. I don't see a strong case for another winter anytime soon.
I have always defined intelligence as a scaling value of how accurately one is able to make a decision with the least negative impact on the individual making the decision.
I just wanna say I appreciate you always trying to update your views of the future of AI based upon current data as it changes
David. Can you address the problem of trust and reliability in llm. Just like self driving cars - they can be correct 99% of the time but that 1% will be disastrous This means they can’t be used for autonomous driving or running a business
It doesn't have to never make a mistake while driving, it just has to make fewer mistakes on average than humans.
@@socialenigma4476 except that they already are better than the average human but we still don't trust them enough to drive on our own
@@socialenigma4476 an issue is the make different mistakes to humans (sane as a LLM). They think there isn’t a car coming and drive with complete confidence into it. Where as human crashes are based on reckless driving. The sane when an LLM goes off on a completely made up answer inventing facts and everything, humans make very errors
@@fiiral5870 it’s the type of mistake that AIs make that is different from human behaviour. Humans typically know if they are unsure of their answer or caveat their response. Current AI has the same level of confidence in every answer it gives (whether it’s right or ridiculously, obviously wrong).
The whole thing is that the creators fear the ending of the programmers who have been controlling their products in artificial intelligence aiming to make sure that they are the programmers
I won't rule out your prediction of September. I'm doubling down on that 😊
I don't think this is a told you so moment. I think as we learn about the human brain we will have to go back to the drawing board trying to emulate the new information.
Admitting you were wrong is a sign of integrity I'm always looking for. I will not trust anyone until they demonstrate this. Thank you.
That being said, I expect you'll be talking about how wrong you were again within a year.
Too many factors involved for accurate predictions.
I think there is a series of levers in the full ai pdev stack, and some increase by orders of magnitude the cost/time/training requirements, and others decrease by OOM the same. 20 years from now, the curve will still appear exponential, I am guessing, as the fits and starts smooth out. (Ie - improvements in power efficiency, model arch, parallelization, synth data etc will all wiggle these various levers)
It's almost like we had a terrible track record of replicating biological proceses with technology. And claimed we could replicate the one thing we can't obviously pinpoint like cosnciousness compared to an arm for example. It's almost like we brute forced most of this progress while not even being co planar between what we call consciousness and what llm's are.
I don't think AI is slowing down - I think the leaders in the field are becoming more careful about what they release as we are close to AGI. Also, the immense compute costs to deploy these advanced models at a large scale are far too high - so they are condensing and dumbing down their most advanced models for general public consumption. GPT-4o is likely a distilled model from a much more advanced and larger model - so it can be fast and cheap and retain some properties of the advanced model .
We won't get to see the good stuff for a while - which is unfortunate, but AI progress is not slowing down - compute is just not keeping up for mass consumption.
I think something to keep in mind is that an accelerating race car briefly slows down while changing gears. This might be part of a particularly clunky gearbox, watch out for where the money's going into research, hardware and infrastructure
If it weren’t for all of this, I wouldn’t have discovered my interest in computer science and tech and I’m still debating what I do with this path but we will see. I’m sure there are many others like me now, this has sparked my interest and also has helped me with learning, AI has been an amazing tutor even in its current state and the ability to break things down and explain concepts in a way I can understand has made school and acquiring knowledge far more accessible to myself, again I am sure there are many others who agree. Even if we don’t have AGI anytime soon, the effects of access to LLM’s will be felt with time, and that too will potentially have its own exponential impacts.
I use GPT daily, it's frustrating to get different answers when you are expecting some consistency, I get tired having to check every single thing to the point I micromanage it's output, it's nowhere near what most people think, but it's helpful at times.
I think if you take a step back and dissociate a bit you will have to acknowledge that you did have (at least in what’s visible to your viewer base) a pretty sharp turn around with your optimism and narratives in which you choose to view and approach this topic. I‘m also having the feeling you are also kinda playing deaf to the (imo) appropriate confusion a lot of your viewers are experiencing. I think you could take a little bit responsibility in creating a lot of hype and a lot of hope. It is not your fault if some people would be betting everything in their life on the optimistic accelerationalistic narrative of AI Development but if you are having this sharp of a change in approach, you could rather take responsibility for feeding or falling into this narrative yourself instead of kinda “gaslighting” (prolly too strong of a word but there is something similar too it) the audience that you have cultivated yourself. I hope this doesn’t sound bad, i just wanted to explain my view as a viewer who is watching you for about a year now. We normal people outsource quite a lot of knowledgebase and food for thought to educational sources and are even more susceptible to do that if there is a parasocial relationship playing into it. I feel like you made your decision on AI progress slowing down indefinitely (i have also felt a disturbance in the algorithms, so i get where you’re coming from) but you just want to leave it behind you and act like the dreams and hype you created never existed. Well that probably sounds very dramatical its not supposed to. I am sure you already know that or if what i am saying is not true but just wanted to articulate a random viewers perspective. This is not a front just genuine feedback and also I am bekifft right now so maybe that invalidates or explains everything i wrote there
-reposting here because on second read it do got a point even if i got lost mid sentence somewhere there
I appreciate this meta view of how the AI "situation" is evolving with the major players finding themselves vying for status. As for acceleration bottleneck, I think everybody kind fo figured the money issue was an itch that would scratch itself.
I also think there's a lot more going on behind the scenes. I'm not sure is slowing down as much as it seems.
Companies are not showing as many of their cards that they are holding in their hand - perhaps due to the US election sucking the air out of the room. So much for slow iterative releases.
If this is a slowdown, then we are heading for a slow take-off fast acceleration instead of a fast take-off slow acceleration. Wasn't this the scenario that created a higher P(doom)?
Thanks for this, Dave. Very reasonable and clear-eyed. I think it was very easy for all of us to get caught up in the hype. I've worked in tech since the 80s. It is very difficult to separate the hype from the reality for any new groundbreaking technology. I still think AI is very real and will be more intelligent than us in the not too distant future. Yet, there is also a tremendous amount of hype which can easily cause any of us to expect rapid progress. I've found that smart people have a tendency to think things will happen faster than they actually do. They can see all the possibilities and imagine how progress might happen. Most of these companies think we will get there in the next 3-6 years. I think that's still a reasonable timeframe AND pretty fast.
Given what we are talking about, even 10 years seems pretty fast. It will probably be better for humanity if it happens slower.
@@johnwilson7680 I don't have much faith in humanity to navigate this technology safely. Look at how unstable the world is now. We are already pursuing autonomous weapons, and the US is antagonizing China by trying to prevent them from pursuing technology for the sole purpose of maintaining its own hegemony. If the US can't figure out how to cooperate with others, the trajectory we are on only leads to some kind of dystopia.
Two breakthroughs have occurred in AI that have brought it to its current state. One in 2012, the other in 2017. Now all the attention and money is on AI, and at any moment a new breakthrough could happen.
Im not convinced autogen is really that far ahead of the Ace necessarily. Either way im glad you did it, it got me in the game.
Good to see you’re willing to adjust predictions and analysis based on new data, Dave.
It takes 10 years for a AAA game to be made. Assuming AI has slowed down, assuming all breakthroughs are being provided upfront, is comical.