Agent OS: LLM OS Micro Architecture for Composable, Reusable AI Agents
ฝัง
- เผยแพร่เมื่อ 27 พ.ค. 2024
- Agent OS is an architecture for building AI agents that focuses on IMMEDIATE results for today and over the long term.
The LLM Ecosystem is ever evolving so in order to keep up, you'll need an architecture that has interchangeable parts that can be swapped in and out as needed. This is where the Agent OS comes in.
A great architecture, can future proof your AI agents and make them more adaptable.
The Agent OS is a micro architecture based off of Andrej Kaparthy's LLM OS. It's comprises three primary components: the Language Processing Unit (LPU), Input/Output (IO), and Random Access Memory (RAM). Each serves a unique purpose in the construction of AI agents, enabling you, the developer, to create systems that are not only efficient but also adaptable to the rapidly changing landscape of AI/LLM technology. The LPU, positioned at the core of the architecture, integrates model providers, individual models, prompts, and prompt chains into a cohesive unit. By storing all llm, and prompt related functionality into one component, the LPU, we can focus on prompt engineering and prompt testing around this unit of this Agent. This integration facilitates the creation of AI agents capable of solving specific problems with high precision. Thanks to the layered architecture, each piece can be swapped out. So when GPT-4.5 or GPT-5 rolls out, you can easily upgrade your AI agent without having to rebuild the entire system from scratch.
The RAM component enables your AI agent to operate on state, allowing it to adapt to changing inputs and produce novel results. The IO layer, on the other hand, provides the tools (function calling) necessary for your AI agent to interact with the real world. This includes making web requests, interacting with databases, and monitoring the agent's performance through spyware. By monitoring your AI agent's state, inputs, and outputs, you can identify issues and make improvements to the system.
In this video we dig into ideas of creating composable agents where the input of one agent can be the output of another agent. This is a powerful concept that can be used to create complex agents that can solve a wide range of problems. It's the evolution of the core idea agentic engineering is built on: The prompt is the new fundamental unit of programming and knowledge work. First you have llms, then prompts, then prompt chains, then AI Agents, and then Agentic Workflows. This is the future of programming and knowledge work.
🧠 Andrej Karpathy’s LLM OS
• [1hr Talk] Intro to La...
🔗 7 Prompt Chains for Powerful AI Agents
• 7 Prompt Chains for De...
💻 Everything is a Function
• How to Engineer Multi-...
🔍 Multi Agent Spyware
• AutoGen SPYWARE: Codin...
📖 Chapters
00:00 Best way to build AI Agents?
00:39 Agent OS
01:58 Big Ideas (Summary)
02:48 Breakdown Agent OS: LPU, RAM, I/O
04:03 Language Processing Unit (LPU)
05:42 Is this over engineering?
07:30 Memory, Context, State (RAM)
08:20 Tools, Function Calling, Spyware (I/O)
10:22 How do you know your Architecture is good?
13:27 Agent Composability
16:40 What's missing from Agent OS?
18:53 The Prompt is the...
#aiagent #llm #architecture - วิทยาศาสตร์และเทคโนโลยี
Quote of the decade “The prompt is the new fundamental unit of knowledge work” Couldn’t agree more. Each video is more useful than the last. Very few channels hit that bar. Incredible stuff. Keep it up. Cheers.
Love building from the bottom up and this video really puts that foundational Architecture in perspective. Tks
About swapping prompts or prompt chains on models or providers, I suspect that prompts and prompt chains can be highly specific to models or providers so that swapping would destabilize the system in terms of output consistency and quality. Proof: try a specific prompt chain that is engineered to work on ChatGPT 4 on any open source model. The key problem is the models don't have a standard of output (not like APIs with defined signature and schema) which makes them not likely mutually replaceable.
dude, i subscribed. finally a channel tha isnt just " I Can TEll IT TO GEnEratE A mEDium ARTICLE" . instead you provide proper ideas and concepts for multi layer systems design with AI that can be used as stepping stones for larger and more complex problems. Hats off to you sir.
Everything on the right side of the arch diagram makes sense, I think the left side needs some more definition or rewording.
You're far away from the mainstream man ❤💯👏Hats off!!
Look up "Long Net" for the I/O, State, and self improvement parts
Also Infini-attention research on arxiv
when do we get LLMOSaaS?
"It's Kubernetes for the OS in an LLM context" - overheard at at coffee shop this morning, which is why I'm checking this video out today.
In a few years, an LLM will be our companion.
forget pets (something we should actually do)
forget spouses.
we've got LLMs.
It's the functional programming equivalent of LLM development.
Great video!
Agreed on self-improvement.
One technique that can work to enable self-improve is mentally moving the Prompts and Prompt Chains into RAM. By making them stateful and able to be updated based on user or system feedback, the system becomes capable of more self-improvement.
we can add a long term memory where we save in a database the user preferences, feedbacks etc.., we will need ram, long term memory, entity memory and finally contextual memory that will combine them all when needed
maybe you can have a meta-prompt chain doing critic and score the result of agents performance and then improve the original prompt automatically.
@@free_thinker4958 langchain cookbook has a meta-prompt notebook that suggests the concept of self improving.
@@free_thinker4958 I think this is what DSPy is trying to do at the prompt level
Excellent!
Wow this was fire 🎉
One thing we need is an agentic workflow across robots - carry this stuff over into the robot OSes.
I like what you’re saying and I believe you will have a successful channel if you deliver on what you said in this video. I subscribed!
The term „Spyware“ is very misleading. Auditing would fit much better.
Auditware…. I like the sound of that. Copyright that 😂
Or just “Monitor“ 🤷
This is real content. Thanks. Looking forward to seeing how you structure these systems in code.
Agent Self improvement is done with reinforcement learning algorithms. Thoughts?
An intelligent memory management system is enough for now i think
What would you change in Agent X that would improve its outputs? Really just a few knobs we turn to make agent perform differently, AI and ML people know these knobs and turn them often. Reward function is only a "measure" of how much you like or dislike the outputs. Please notice, I think this can thus be solved "agent-i-cally" with a teacher agent and loop between the teacher and the "improving agent". Thus auto-improvment. The architecture you presented already has this ability because it has agents. You will have to specially make this "teacher" agent and I believe there will be big money for the first to write a very good agent that can assist in improving other agents (similar to the invention of transformer, throw back to deep mind paper, attention is all you need...the attention on a reward function that is).
What about hardware by Cerebras Systems and Extropic in comparison to Groq?
Another great video, really enjoying the content! Wondering, in a development pipeline of something like:
Discover ->User Stories -> Jira Ticket (with AC, designs, etc.) -> Development (let's say for a web/mobile hybrid), what would be the 'names' of the agents that might be used? Would it be an agent (or a group of agents) called "Product Manager", one or more called "Front End", one or more called "Back End", one or more called "QA", etc.? I'm just wondering the broad classes and names that you might create as agents in the development process?
Great video. It'd be great to create an open source basic template structure of this architecture to use it as the base for any new projects.
It's crazy the wisdom you drop here. while LinkedIn feels like an AI cesspool no one gets the power of using GPT-4 or any language model as the semantic transistor in a bigger application. Got into a row with people who claim Hallucinations are part of the system and an unfixable problem. I feel like we have too many cooks and no one knows the recipe at all and we're all trying to wing it!
So sensible places like this will become invaluable keep up the good work!
where do you think we can deploy this agents online to serve other users? can we use docker containers as agent microservices?
Yes, docker is recommended, build the backend using a framwork such as crewai and for the front end you can use nextjs or any other front end option
Self-improvement IS the missing link! Thank you for a great innovative thinking video!
I just cringed on the Spyware labeling though?
How about WhereWare? (You know... Where did it go wrong?)
🎯 Key Takeaways for quick navigation:
00:00 *🧠 Introduction to Agent OS Architecture*
- Introducing the concept of Agent OS architecture within LLM OS.
- Exploring the microarchitecture of Agent OS focusing on LPU, IO, and RAM components.
- Highlighting the significance of composing reusable AI agents for immediate and long-term results.
01:06 *🛠️ Components of Agent OS Architecture*
- Discussing the components of Agent OS: LPU (Language Processing Unit), IO (Input/Output), and RAM (Random Access Memory).
- Explaining the role of each component in building composable and reusable AI agents.
- Emphasizing the importance of structuring AI agents for adaptability and longevity.
03:00 *📦 Understanding LPU, IO, and RAM Components*
- Delving deeper into the functionality of LPU, IO, and RAM within the Agent OS architecture.
- Analyzing the innovation brought by LPU in differentiating AI systems.
- Describing the role of IO in enabling communication between AI agents and the external world.
04:08 *💡 Components Breakdown: LPU*
- Detailing the components of LPU: Model Provider, Models, and Prompt Chains.
- Discussing the significance of prompt chaining and its impact on enhancing AI agent capabilities.
- Highlighting the power of language processing units in generating diverse and contextually relevant responses.
07:38 *🔄 Components Breakdown: RAM*
- Explaining the importance of RAM in enabling AI agents to operate on internal states and contextual information.
- Illustrating how AI agents utilize RAM to update and modify their states for improved performance.
- Emphasizing the role of RAM in facilitating autonomous decision-making and problem-solving.
08:36 *🛠️ Components Breakdown: IO*
- Discussing the role of IO in facilitating interactions between AI agents and external tools or systems.
- Introducing the concept of spyware for monitoring, debugging, and visualizing AI agent activities.
- Emphasizing the importance of monitoring and measuring AI agent performance for continual improvement.
10:26 *🔍 Evaluating Architecture Effectiveness*
- Assessing the effectiveness of Agent OS architecture in comparison to simplistic AI agent designs.
- Analyzing the potential of Agent OS architecture in addressing future challenges and advancements in AI technology.
- Discussing the importance of architectural flexibility and adaptability for long-term AI development.
11:09 *🎯 Evaluating Agent OS Capabilities*
- Evaluating the capability of Agent OS architecture to perform various tasks outlined by Andrew Karpathy.
- Assessing the readiness of Agent OS architecture in achieving self-improvement and customization.
- Exploring the potential of Agent OS architecture in enabling seamless communication between AI agents.
13:15 *🧩 Importance of Composability*
- Highlighting the critical role of composability in building modular and reusable AI systems.
- Discussing the significance of composability in creating agentic workflows and driving greater efficiency.
- Emphasizing the iterative nature of AI agent development and the importance of architectural scalability.
16:44 *🧩 Missing Piece: Self-Improvement*
- Identifying self-improvement as a crucial missing aspect in Agent OS architecture and AI systems in general.
- Exploring the concept of self-improvement within AI agents and its potential implications.
- Discussing ongoing experimentation and future prospects for implementing self-improvement mechanisms in AI architectures.
Made with HARPA AI
Hell yes. 🎉
Ive been looking into all these autonomous agent system, the browsers are popular because their a lot easier and tractable. but the grail is the computer itself or then a new computer to sit on your old computer that is semantically operated. but its the CLICKING screen elements. even with REaLM from apple. I dont think the click problem is fully solved
🔥🔥🔥
I agree on composability. i guess we don't have any open source project like this yet. right? I was about to dive into crewai.
Dive into crewai, i think it's the most suitable framework especially that there are updates frequently
make a video where ai can search on multiple sites for answers
Observability. Not spyware, observability. But great video.
100% agree, spyware just sounds silly
Lesss goooo!!! 🎉
Great video got my sub for sure. I am going to start messing with this today. I would like to know which local models are on your radar.
Confusing terms now that we have a Groq LPU in the zeitgeist
Is there a github repo?
It’s called observability. We should not be inventing our own terms for these construction AI, we should be using the ones we already have for other systems.
besides the good content: the mouse wiggling and shaking hands in the background doesn’t add any benefit.
Either this dude doesn't know what is Micro Architecture, or made a malicious clickbait.
u can t explain let it go
Love this talk!! You’ve got that grand scheme of thinking - Thinking in big picture and long term. Which is crucial for people in AI for the long haul as it’s not going anywhere, especially those who would like to have their own AGI systems like myself. Would love to join a community if you have intention to start one, together is better :) 🙏🙌🫶
Hey Dan 👋 im a fellow ai architect & engineer, and would love to chat & collab, do you have discord or something similar? about to launch my channel, wish me luck!
Thanks for the inspiration and the fire content! All the best! ❤