AutoGen Advanced Tutorial - Build Incredible AI AGENT Teams
ฝัง
- เผยแพร่เมื่อ 5 ก.ค. 2024
- In this video, I break down every topic you need to know to become an AutoGen master. Some topics include caching, best practices, enhanced inference, multi-agent setup, LLM daisy-chaining, a brand-new agent type called Teachable Agent, function calling, templating, and so much more.
Enjoy :)
Join My Newsletter for Regular AI Updates 👇🏼
www.matthewberman.com
Need AI Consulting? ✅
forwardfuture.ai/
Rent a GPU (MassedCompute) 🚀
bit.ly/matthew-berman-youtube
USE CODE "MatthewBerman" for 50% discount
My Links 🔗
👉🏻 Subscribe: / @matthew_berman
👉🏻 Twitter: / matthewberman
👉🏻 Discord: / discord
👉🏻 Patreon: / matthewberman
Media/Sponsorship Inquiries 📈
bit.ly/44TC45V
Links:
Use RunPod - bit.ly/3OtbnQx
AutoGen Beginner Tutorial - • AutoGen Tutorial 🚀 Cre...
AutoGen Intermediate Tutorial - • AutoGen FULL Tutorial ...
AutoGen Fully Local - • How To Use AutoGen Wit...
AutoGen 100% Open-Source - • Use AutoGen with ANY O...
AutoGen - microsoft.github.io/autogen
Chapters:
0:00 - Intro
0:56 - LLM Config
6:47 - Config List
10:54 - Caching
13:31 - Enhanced Inference
19:36 - Templating
21:20 - RAG Agents
26:20 - NEW Teachable Agent
31:35 - Using Docker
32:57 - Function Calling
36:07 - Agent Tool Learning - วิทยาศาสตร์และเทคโนโลยี
What topics do you want to see me do an even deeper dive into in my AutoGen expert tutorial?
Do some real life examples of what we can do with it! Maybe setting up a react project and edit by itself, data analysis, expert bots for social media, etc.
you said you usually run it inside Docker, that's what I've been doing. I just found an Anaconda Image, but I'd be courious to see how you set yours up in case I missed something.
Automation Process. RPA. Webscrapping
Please the examine using mistral in LM studio for Autogen.
I think a deep sewing it all together with a practical example of all the steps for solving a complex task with multiple agents? My pitch is for getting a 'team' of agents to generate a boilerplate webapp instantiation. It's a common task that happens, Setup X in a docker container that connects with Y in another docker container. Maybe utilize the RAG assistant to consume documentation? (Can RAG take more than a single link? An array of links? ) . Your videos have been an immense help in me getting things sorted in autogen. So I want to say much thanks for this and all your previous videos. Also, I'm attempting my own effort on my suggestion above. If I get there before you I will definitely share knowledge with you.
I’d love to see content showing numerous different agents working together in their unique roles
Well done! I really like that you are deep diving in to Autogen. Most of the other content creators re just scratching the surface. 🥇
Thank you for sharing such valuable information on Autogen and its advanced functionalities. I look forward to learning more from you in the future. Keep up the great work!
I have watched this 5 times and still feel fascinated. Can't wait for expert level tutorial.
Please make it really through. We can handle complex!😊
When were you elected?
@@jerbear7952 Techie President Election 2024... you can vote for me too... Vote for me! Free Tacos for all! In fairness though, when Matt says "Do you have any suggestions leave them in the comments," I don't think he was suggesting we all come to a consensus.
And again just to remind you.... Vote for me... free Tacos for all.
Message Ends
Best tutorials on TH-cam, thanks for all your work.
Thank you so much! I've been looking for instructions like this since AutoGen was released.
I've been eagerly awaiting this. Excited to see what you've come up with.
Feedback is always welcome!
mixing config list is definately something I've been curious about, mainly because I want some local, but then outsource the heavier lifting.
Thank you for walking us through this step by step . Please keep these videos going
This is awesome!! I'm blown away by Autogen. So happy that you're doing all these videos on it. Let's create something usefull from start to finish (as an idea for a future expert video on Autogen) 😍
...IT SOUND GOOD
Thank you for putting these AutoGen Tutorials together! I'm excited for your "Expert" level tutorial.
Wowwow🎉 Huge potential! Definitely examples of function calling, teachable agent and reflect/learn, user feedback😀
Yes! Thank you!
Thank you for the great video! A 38-min video actually took me hours to digest the knowledge.
This is amazing. It will certainly save us a lot of time to test things out! I'm looking for the expert tutorial!
i am going to be going through all your videos for autogen this weekend. i have a very specific use case that i want to automate a ton of creative work with this. took a while of manual work to get to the point i'm ready to automate the rest. I have watched each of these but still need to sit down and go through the motions. besides creative applications, there's a business use case then also that is kind of along the same lines, but will take a bit more tweaking. i'm going to try to plow through it, then might come discuss with you if i get stuck.
as far as more videos on this, i would love to see a full start to finish, setting it up with everything you've learned through your growing pains, in a comprehensive step by step manner. don't give us choices :) tell us which is the best model for use for each type of usage.
i bet a head to head, multi screen battle between the different models on different tasks with the exact same prompt would be a blast to watch. 4 or 6 models, sync the enter of the prompt for each and go! see what happens real time with all of them basically competing against each other.
Big fan of this channel. Great content. Would love to see these agents working together in practical small project
Thanks for another Awesome video Matthew 🙌🏼👏
Fantastic overview of autogen, I'm really looking forward to the expert video
Beeeeeeen waiting for this! Thanks Matt!
Great job Matt, will explained, you are the best.
I appreciate that!
hey. I'm absolutely blown away. i would love to invest some time into autogen over the next few weekends. the whole topic is relatively new to me, and to a large degree daunting... but im curious. where do i start?
a big thank you for openly sharing this tutorial. kinda excited
These are great. I'm rewatching this again.
How many seeds of innovation shall sprout from this newfound zenith of intellectual mastery? The intellectual possibilities are endless!
Fantastic video, thank you. I would love to see an example of multiple function calling agents in a group chat, where each agent has its own set of functions it can call. I've only really gotten this to work when only the user proxy agent can function call
Been waiting for this!
Excellent content, thanks for sharing!!!!
Killer killer KILLER content as always! Thanks!
Thank you!
Thanks for the video! When is the expert coming? I would love to learn how to control the conversation flow. Make sure some agents only talk with some agents or making the human talk only with a team lead.
Matthew! U'r so helpful! this is changing my life!
thanks! this was my plan for this weekend. i really need something that can handle long text with memory
Can’t wait to try. I can run some of the things u talk about. I haven’t had the best of luck. ROCM is hard to get started for my scrubby ass.
This was awesome! Thanks for sharing!
Great tutorials, thank you so much, and keep going! Could you please complete the last example of "storing" the recipe with the TeachableAgent and call it by other agents?
Very clear explanation, thank you.
Keep it coming my man!
This is great content! More videos please, I'll watch all of them.
great! tnks for this video!
These videos are so informative and helpful.
Yes more AutoGen videos!🥳
this is crazy stuff, awesome videos
Cant wait for that expert video!!!!!
!!!!!Congrats on the 100k Subs bromethieus.
Thank you for easing into it. It does get a bit dry though. No real fault of your own or toooo much you could do. Only suggestion I could think of would be to show examples and differences with and without each of the paramaters/configs/settings. Obv it's already a 38m long video, so that would necessitate turning it into an even longer >>series
RAG it is interesting, needed tutorial about this, may be in expert level of Autogen. Thank you!
Functions with open LLMs; so that we can decouple the OpenAI dependency. Then, functions + tools.
Fine-tuned OS models that are specific to a project 🚀
You’re 100% right. I want to decouple from open AI so bad. There are so many great models out there!!
Amazing video. Thank you soo much!
Great video! I couldn't figure out a way to make agents follow the plan of a planner agent since any agent containing a function call is given first preference. It would be great if the agent can keep the function in its pocket but use it when it is actually required. Any details or information on that would be great!
Amazing video Matthew!! I'd love to try some of this locally by running an OpenAI I/F server backed by a Mistral finetune.
Also, you should add a chapter around 21:20 for RAG Agents.
Will do! thank you
great video, it hurt my brain. so much to learn!!!!
Great Video!! It would be nice to be able to learn how agents could get data from a json file or database
Another great video, thank you! Just wondering how can we make sure the agent will execute the code? I often get answers that “As an AI agent, I don’t have the ability to execute the code”.
almost 100k, great job!
So close!
Thank you - Autogen is great!!!
To vary the wait time you could just make a loop that redefines the wait time using the formula you’d prefer.
Can you expand more on teachable agents please, particularly the limits
Great video! I haven’t been able to get the OAI Config to work. Always returns, ‘it doesn’t exist’ even when I provide the PATH to it.
Awesome job!
Great Content, thx. Could you provide information on how the knowledge base (embeddings / knowledge of the teachable assistant) affects token usage. E.g. I want one assistant to answer questions based on one large pdf (ebook) with expert knowledge. I guess i have to pay for processing the pdf at least once. But how is the token usage calculated after the first processing. is it significant more expensive?
Appreciate you! 🙏
Very informative!!
Oh what a great video! Would you please create a vid showing how to use the RAG agents with haystack and weaviate? (I've seen langchain is too slow) BIG thanks!
Hey Matthew, Your videos are a thrill!
A Questions: How do Teachable Agents compare to finetuning?
Fine tuning means parameters are getting adjusted. The teachable agents have the same parameters but they have access to a vector database of information
Very excellent video!
Please do another vid on what is autogen and what agents can theoretically do for me or people like me? I’m creating a small skydiving business. Please elaborate creatively with your new understanding of expert settings and such.
Really a great guide! You're really amazing!! Can we have a link to the code?
This video is so needed by the community. You just accelerated the singularity by a few weeks at least.
Thanks!
Thank you!!
Real useful examples
Would be excellent if there was a GUI associated with this!?
Challenge accepted
I may base it on textgen Web UI. Sharing a GL link and Notion to share ideas.
To start, It'd be if:
-Agent configs and knowledge bases could be stored and shared
- There was a collective knowledge that agents can draw from as needed.
A beginner tutorial. I feel like most of it is above my head.
How nice. Since it will be three tutorials for AutoGen I have to seriously include this framework into my project: AI-Town. Everyone should have an AI-town, GPT models doing the tasks you see they benefit yourself and themselves the most! It's tobad GPT models where born with Alzheimers.
Its there a way to use autogen like an api endpoint with fastapi ? So you can call the endpoint, the agents do the work and return a json response ?
One a quick glance it seems on a high level langchain can do all of this as well. Just do different endpoints per agent and make them work together. Or am I missing something?
pen-al-ity?! lol presence penalty... Thanks for the content! Keep them coming.
Very useful.
Great Job! How would we exted the memory for thse models, the idea, is to create a lmem for each agent as the continue to improve the models correctly utilizing Sparse Priming Representation (SPR) or MemGPT, can you also include such a setup so that you can add one of these memory solutions, and you train the models and the teachable agents., one last thing would be to load data, into the training models, with different data points that should be specific to those agents or models, the worst thing to do is train a model for the agents tasks that are to general and are reused across all models, Thanks!
My top people - Obama, Ray Dalio, Matthew Berman!! Awesome job mman so much appreciated!!
I just plugged in TieFighterLR 13B and told them to code a game for me. The chat manager became an interviewer and started interviewing all the Agents for positions on the team. I hope they pass the interview.... LOL
This is awesome. I just blocked out an entire day just to go through all of this. 😅
haha enjoy!
Are any of these agent scripts coming out able to do iterations after you get to your first terminate if you're building code/software? if not, it seems like a ton of work to not be able to iterate off your first run thru. As in creating an app or website code and then being able to go back and add or change a feature once you've been able to go hands-on with something created?
Yes, you can define the agents so that they ask for feedback at TERMINATE. When they do, just go check their implementation, and tell them what to change.
🎯 Key Takeaways for quick navigation:
20:10 📝 Autogen allows you to create templates for solving problems and add step-by-step explanations to your solutions.
21:22 📚 Rag (Retrieval Augmented Generation) agents can pull information from external data sources, enhancing their knowledge and capabilities.
27:09 🧠 Teachable agents in Autogen can retain information beyond the current chat session, making them useful for long-term learning and knowledge retention.
32:04 🐳 Autogen recommends using Docker for code execution to ensure a secure and isolated environment.
33:13 🔄 Autogen allows you to define custom functions that the language model can execute, expanding its capabilities.
36:15 🛠️ Autogen can be taught to use new tools and methods, allowing it to adapt and learn from your interactions.
Made with HARPA AI
THANKYOU
idea for a autogen video getting it to work with other languages rather than just Python because I want to use it but all my use cases are for C# and maybe someone else's may be Rust, C++ etc I think it would be very usefull if you could make a video on how to get autogen to program in more languages rather than just pyhton
Great video! How did you get 2 instances of LL Studio working locally?
Good question. Is it possible? Would love seen it. Thanks in advance.
Could an end user inject false information into the teachable agent? If so, could it then be learned, saved, and spread to others? What defenses are available to prevent that?
Yeah Matt maybe put a few bucks into it and ask us all to come up with a plan as to what you prompt it with, build the agents to do that thing... and see how far it gets... not something really complex, but complex enough that it give a good all round measure of Autogen' studios ability... maybe building some small app, or posting things using Zapier as you mentioned, or even using autogen studio to modify, wait for it, Autogen Studio! lol
i have llm Sudio running on my mac, but I cant seem to find how to install autogen. is there a video before this to explain how to get set up ready to do this?
I can only wish I have all the opportunities you have and all the resources you have access to.
I saw a video about using memegpt with autogen, how is it relevant when you already have reterivalAgent and techableAgent in autogen, why would you use memgpt?
how is MemGPT different fro teachable agent? and when to use which one?
When are you going to make this expert tutorial available
You're really rendered:)
can you provide links to the articles you referenced in the video.
Is there any way to use the Mistral model as a base and utilize Autogen to create agents in Google Colab or a Kaggle notebook?
Good question. I want to see these open source models succeed
For anyone using local LLM, if you see OPENAI_API_KEY or api_key type of errors, make sure that when you are defining the local LLM you need to have a value for the API key, and it should be a random string such as "not-needed". It seems that when autogenstudio is configured to use a localLLM that has a NULL or undefined api_key value, then autogenstudio seems to fallback to using openAI even if it's not defined in any workflow or agents. So if you also have your env variable exported with an invalid LLM config, autogenstudio will then default to using openAI. I do not know where to adjust default configs for autogenstudio.
Wow. Autogen is getting crazy functionalality.
Does Autogen provide good enough RAG implementation that we don't need to import langchain any longer?
Can this be used together with Semantic Kernel?
Epic!
What would you sufgest if we neeed to keep talking to user till the agents receive all the information they need to do the tasks?
It seems like docker wouldn't actually be enough to insulate you from rogue AI code. There's a reason that Ubuntu requires docker to be run as sudo...
Use a VM with snapshots....
Would like to see all the incredible things. 😆