10:47 - "Models No native tool implementation. Tools can be implemented via custom integrations." "Agents Tools are natively implemented in Agent architecture" 15:00 - "the point is you're building essentially a shim between the agent and the API interface" (speaking on the "Extension" construct) So how is building a shim to an API different than building an integration?
Yep great catch! For me, this really boils down to your role in interacting with these systems. Consider there are 2 roles: [1] Consumer [2] Developer If you are the "Consumer" of a Gemini Model (i.e. API endpoint for `gemini-1.5-flash-001`) then there are no tools involved. Imagine if someone created a basic UI with an empty text box and when you hit "enter" it simply sends 1 API call directly to the Model. No tools just simply "generate some text". If you are the "Consumer" of Gemini (the web based Agent Application) then tools are already built into that application (i.e. Google Search, Image Generation, Flights Search, etc.) Now the difference becomes when you are the "Developer" working with either of the above systems. As a "Developer" working with a Model or Agent, your goal is to build something for a "Consumer". So in both of these cases, from your perspective as a Developer, these things become the same to some degree. Your Model + Integrations becomes an "app" for a Consumer. Your Agent + Tools (Extensions) becomes an "app" for a Consumer. The difference for Agents is that they include one last piece: the Reasoning component. There are various camps that argue Model + Tools could also be an "agent" to some degree, and I won't particularly argue against that. So for simplicity sake, we try to keep the definition of "agent" to be Model + Tools + Reasoning as I laid out in the first slide @ 2:00 and 5:30 TLDR - If you're the developer of these systems, building a shim to API or Integration are largely the same. The difference is in the end application architecture. Hope this provides some clarity, and thanks for watching! 🙏
Thanks for the response 👍 Looks like upon reading your explanation a few times and rewatching a bit + letting the little grey cells work, you're simply defining a tiny slice of structure on top of a bare model, i.e. a RAG approach specific to your approach to the - as you say - overloaded term Agent. The two pieces I quoted can be just two perspectives on this framing, with a model + tools being basically the same as an agent without the explicit reasoning part. Interesting talk! How do you propose to handle agent "locality" and movement? Also related, how do you plan on versioning agents, and are you and your incubator interested in discussing a distributed protocol similar to a git-like semantic vcs as a possible solution to this locality/versioning problem? It just happens to solve numerous other issues by addressing git's technical debt! (It is 20 years old after all...)
I'm excited to see how this technology evolves and transforms the way we work alongside AI. Have you had any experiences with Generative AI Agents yourself?"
Generative AI agents are truly impressive; however, to fully understand them, we also need great presentations like this one. Thank you, Patrick.
Heck yeah. Stoked to play around with it
10:47 - "Models No native tool implementation. Tools can be implemented via custom integrations." "Agents Tools are natively implemented in Agent architecture"
15:00 - "the point is you're building essentially a shim between the agent and the API interface" (speaking on the "Extension" construct)
So how is building a shim to an API different than building an integration?
Yep great catch! For me, this really boils down to your role in interacting with these systems.
Consider there are 2 roles:
[1] Consumer
[2] Developer
If you are the "Consumer" of a Gemini Model (i.e. API endpoint for `gemini-1.5-flash-001`) then there are no tools involved. Imagine if someone created a basic UI with an empty text box and when you hit "enter" it simply sends 1 API call directly to the Model. No tools just simply "generate some text".
If you are the "Consumer" of Gemini (the web based Agent Application) then tools are already built into that application (i.e. Google Search, Image Generation, Flights Search, etc.)
Now the difference becomes when you are the "Developer" working with either of the above systems.
As a "Developer" working with a Model or Agent, your goal is to build something for a "Consumer".
So in both of these cases, from your perspective as a Developer, these things become the same to some degree.
Your Model + Integrations becomes an "app" for a Consumer.
Your Agent + Tools (Extensions) becomes an "app" for a Consumer.
The difference for Agents is that they include one last piece: the Reasoning component.
There are various camps that argue Model + Tools could also be an "agent" to some degree, and I won't particularly argue against that.
So for simplicity sake, we try to keep the definition of "agent" to be Model + Tools + Reasoning as I laid out in the first slide @ 2:00 and 5:30
TLDR - If you're the developer of these systems, building a shim to API or Integration are largely the same. The difference is in the end application architecture.
Hope this provides some clarity, and thanks for watching! 🙏
Thanks for the response 👍
Looks like upon reading your explanation a few times and rewatching a bit + letting the little grey cells work, you're simply defining a tiny slice of structure on top of a bare model, i.e. a RAG approach specific to your approach to the - as you say - overloaded term Agent. The two pieces I quoted can be just two perspectives on this framing, with a model + tools being basically the same as an agent without the explicit reasoning part.
Interesting talk! How do you propose to handle agent "locality" and movement? Also related, how do you plan on versioning agents, and are you and your incubator interested in discussing a distributed protocol similar to a git-like semantic vcs as a possible solution to this locality/versioning problem? It just happens to solve numerous other issues by addressing git's technical debt! (It is 20 years old after all...)
How can we start building with these right now?
I'm excited to see how this technology evolves and transforms the way we work alongside AI. Have you had any experiences with Generative AI Agents yourself?"
I want to be an Agent..❤
You are a mushy agent
It doesn't work with complex apis
Then that’s where you write in a CrewAI function call as a part of your flow