Agree. Forgive the facile analogy, but LLMs are just components, like cams or gears back in the days of mechanical solutions. You wouldn’t just obsess about building the best cam and wondering why it’s not plowing the field yet. A cam turns rotational motion into linear motion. There’s only so much better it needs to be for most use cases. But to get the final product you need to throw in gears, levers, springs, escapements, etc. LLMs translate natural language into (executable) structure. Now we need to link em up to all sorts of other stuff. For example, the diffusion work which is being done with video- the ability to “simulate/imagine” scenarios based on text parameters or other embedding vectors. If agents had the ability to configure up and run those things real time on the fly, why, that would be groovy. So indeed, it’s just about time to call a cam a cam and move on.
Thanks for the comment Andrew. Hard agree. Another analogy I like to use is they have put a Lambo engine in a shopping cart and are tuning the engine to try to get the shopping cart to go faster. Virtually ALL money is going into the Lambo engines and almost none is going into building the rest of the car.
When you say "GPT-5" you mean OpenAI's next model release? Because Imma be honest, if it isn't a huge improvement, I don't see OAI naming the thing "GPT-5", maybe a new line of products or something because that name comes with a lot of expectations.
Yes, I mean whatever their next model release is. There seems to be a good chance that it will be called Orion-something and they will start a new naming structure. I agree. If it isn't a major difference, they can gain cover by changing its naming structure or by building in more AI engineering processes to get benefits without requiring a "smarter" LLM.
I'm a bit confused about something. It mentioned some limitations of current llms that need to be tackled for us to reach AGI. Could you please explain what those limitations are? Thanks!
Some quick hits -- they can only use the ingredients in their training data; next token prediction causes significant issues. I cover this topic in more detail in this video: th-cam.com/video/xHclBdN8uBc/w-d-xo.html
is this guy living in a cave? o1 is a huge improvement, a paradigm shift with new scaling laws for inference, Orion will be based on those and he says no big change? don't waste your time watching the video
It looks like a huge improvement if you're looking at the narrow evals that are apples to strawberries comparisons and if you're looking at LLMs purely from a chatbot perspective, which leaves most of the value on the table. If you're looking at LLMs from the standpoint of solving real world problems, o1 is an improvement, but it is not the game changer many want to believe it is.
My conclusions are based upon real world usage of LLMs for clients and our own internal R&D. The usefulness of LLMs has not improved much since GPT-4 came out in April of 2023. Most people are looking at LLM's through a very narrow frame that is inappropriate to conclude their real, useful value to humans. That narrow frame that is the common "wisdom" is to the advantage of companies like OpenAI. There's a lot of hand-waving and conveniently ignoring practical considerations going on. When I talk to other nerds who are working directly with LLMs to solve real world problems, they nod along in agreement. Much of this stuff has been obvious for over a year and it only gets more obvious over time, even with o1.
Agree. Forgive the facile analogy, but LLMs are just components, like cams or gears back in the days of mechanical solutions. You wouldn’t just obsess about building the best cam and wondering why it’s not plowing the field yet. A cam turns rotational motion into linear motion. There’s only so much better it needs to be for most use cases. But to get the final product you need to throw in gears, levers, springs, escapements, etc. LLMs translate natural language into (executable) structure. Now we need to link em up to all sorts of other stuff. For example, the diffusion work which is being done with video- the ability to “simulate/imagine” scenarios based on text parameters or other embedding vectors. If agents had the ability to configure up and run those things real time on the fly, why, that would be groovy.
So indeed, it’s just about time to call a cam a cam and move on.
Thanks for the comment Andrew. Hard agree.
Another analogy I like to use is they have put a Lambo engine in a shopping cart and are tuning the engine to try to get the shopping cart to go faster. Virtually ALL money is going into the Lambo engines and almost none is going into building the rest of the car.
When you say "GPT-5" you mean OpenAI's next model release? Because Imma be honest, if it isn't a huge improvement, I don't see OAI naming the thing "GPT-5", maybe a new line of products or something because that name comes with a lot of expectations.
Yes, I mean whatever their next model release is. There seems to be a good chance that it will be called Orion-something and they will start a new naming structure.
I agree. If it isn't a major difference, they can gain cover by changing its naming structure or by building in more AI engineering processes to get benefits without requiring a "smarter" LLM.
I'm a bit confused about something. It mentioned some limitations of current llms that need to be tackled for us to reach AGI. Could you please explain what those limitations are? Thanks!
Some quick hits -- they can only use the ingredients in their training data; next token prediction causes significant issues.
I cover this topic in more detail in this video: th-cam.com/video/xHclBdN8uBc/w-d-xo.html
@@practical-ai-engineering Thanks!
is this guy living in a cave? o1 is a huge improvement, a paradigm shift with new scaling laws for inference, Orion will be based on those and he says no big change? don't waste your time watching the video
I know, right...In like every other sentence I'm like:
Woah, hold on a second. How the hell did you get to that bold conclusion?!
It looks like a huge improvement if you're looking at the narrow evals that are apples to strawberries comparisons and if you're looking at LLMs purely from a chatbot perspective, which leaves most of the value on the table.
If you're looking at LLMs from the standpoint of solving real world problems, o1 is an improvement, but it is not the game changer many want to believe it is.
My conclusions are based upon real world usage of LLMs for clients and our own internal R&D.
The usefulness of LLMs has not improved much since GPT-4 came out in April of 2023.
Most people are looking at LLM's through a very narrow frame that is inappropriate to conclude their real, useful value to humans. That narrow frame that is the common "wisdom" is to the advantage of companies like OpenAI. There's a lot of hand-waving and conveniently ignoring practical considerations going on.
When I talk to other nerds who are working directly with LLMs to solve real world problems, they nod along in agreement.
Much of this stuff has been obvious for over a year and it only gets more obvious over time, even with o1.