AutoGen FULL Tutorial with Python (Step-By-Step) 🤯 Build AI Agent Teams!
ฝัง
- เผยแพร่เมื่อ 9 ต.ค. 2023
- This is an intermediate tutorial for installing and using AutoGen locally instead of Google Colab. I previously created a video giving a beginner overview of AutoGen and a quick Google Colab tutorial, but now we go deeper! I show you step-by-step how to install and use AutoGen in any application using Visual Studio Code, Anaconda, OpenAI, and AutoGen.
Enjoy :)
Join My Newsletter for Regular AI Updates 👇🏼
www.matthewberman.com
Need AI Consulting? ✅
forwardfuture.ai/
Rent a GPU (MassedCompute) 🚀
bit.ly/matthew-berman-youtube
USE CODE "MatthewBerman" for 50% discount
My Links 🔗
👉🏻 Subscribe: / @matthew_berman
👉🏻 Twitter: / matthewberman
👉🏻 Discord: / discord
👉🏻 Patreon: / matthewberman
Media/Sponsorship Inquiries 📈
bit.ly/44TC45V
Links:
Code From Video - gist.github.com/mberman84/584...
AutoGen GitHub - github.com/microsoft/autogen
Beginner Tutorial AutoGen - • AutoGen Tutorial 🚀 Cre... - วิทยาศาสตร์และเทคโนโลยี
Advanced guide coming soon. What topics do you want me to cover?
Autogen with langchain :)
Autogen or aider working with local LLM! OpenAI API is way too expensive. ($0.50 just running the YTD stock example notebook once)
Would this work (to some extent) with creating an app in either Android Studio or Visual Studio (MAUI or even Xamarin)?
Second!@@techguy2342
Working with docker,. Logging and troubleshooting and finally Integra of private llm hosted in cloud
Yes! Please keep doing this series! On a side note, it's been awesome to watch you grow over the last 7ish months. Your content here, and your personal knowledge and skills have shown a tremendous improvement! Your walkthroughs have helped me, and many others keep up with the incredible volume of AI developments lately. So.... Thank you 🙏.
Just have to say it. You’re an awesome dude, and I look forward to your videos every day. Your passion is clearly genuine and certainly contagious. I wish you nothing but continued and growing success in life and your video-making ventures! You rock Matthew Berman.
Thank you so much, this comment means a lot to me!
100% agree! The word that came to my mind was "infectious", but the good kind - I've been infected with "stokedness".
I've been really inspired to learn all I can about this subject, and explore tools and developments in this area - and have been applying it in my own work and life. It's very exciting. Since finding this channel, it's the place I keep coming back to over and over.
I'll double down with another thank you. Thank you Matt!
Just one more nerd here addicted to the infected stokedness of Matthew and his ability to gave me goosebumps after each new video on AI.
Very good tutorial! ... BUT... I have seen a ton of videos using LLM agents to make trivial examples but never any more substantial app. It would be great if you could demonstrate how to make a more significant app or game that consists of multiple files, image resources, etc., and where you also participate in editing code (for fixing some bugs or making changes this is often much easier and faster than getting the agents to do it).
That and context windows, the bane of AI programming. Eventually, any project is just going to get too big to be seen holistically.
@@KEKW-lc4xi I wonder if it's just a matter of a lot more but very specific agents. This one handles functions, this one handles file i/o, one can research different dependencies that might work best or more efficient, several can qa along the way.
@@KEKW-lc4xi I believe applications of moderate size and complexity (maybe up to a few thousand lines of code) can be done with current technology. It just requires setting things up right and making the human part of the team to correct any mistakes.
This is my comment exactly. All we see is an AI doing things that a human could easily do themselves, but maybe faster.
Making an app requires interacting with the UI, having a backend and front end, debugging... The tech is far from ready to do any of this, it's way too complex. That's why it's limited to simple things for now.
My guess is that it's going to be perfect to write isolated pieces of code or paper and articles.
These AutoGen tutorials are extremely helpful, Matthew. Please continue rolling out this content.
Matt.... THANK YOU for breaking these down in step-by-step. You are giving us so much knowledge and capability. Truely, thank you.
You’re very welcome
Glad to see someone else is so excited about this. I am here with you. Let's push this to the limits. I am ready to create my army of agents and groups. Great work buddy!
Can you give a use for all this? At the exception of toys problems, I don't see any use.
Bro right?! It’s the obsession I can’t get away from. Starting this second video now. I’m hoping we’ll focus on coding agents. If not more of that would be wonderful! You’re the best btw.
You mentioned multi agents Matt, but how I would love to see some more tutorials on how to actually get all those agents to work together
Introduce them to each other. Name them.
Setup some sort of loop where their answers are all forwarded to each other. You can do this with langchain, Flowise, or Botpress
As a postgraduate student, I'd love to see how it could be used in academic research. While I wouldn't trust it to write a paper, it could be helpful in assessing which papers are worth reading when researching a specific topic. Or it could browse the internet and search for relevant papers that the researcher has missed, but that could get quite expensive.
I'm sure many people would find it useful, since it would be applicable even outside of Academia. And there are definitely more ways in which it could simplify research that I haven't thought of (other than writing fake papers haha) :)
In only two days big fan of your work man. Simple and to the point. Way to go.
I would really like to see you do some more advanced stuff. I'm new to coding and kind of jumped into the deep end by focusing on AI. I had spent roughly 100 hours on working with ChatDev and I've already started porting my agents over to AutoGen, but I would love to see how far the rabbit hole goes.
What would be super cool would be doing a video game type visualization of what AutGen does from a dialog similar to Chat Dev. I love being able to watch my little design firm work through the task I give it visually. Being able to orginize by "company" like Chat Dev would be awesome too. That way you could have different sets of agents built for any series of given tasks, but if they arent need you could lean out the code by removing unnecessary agents. I'll end you creating profules anyway, but the concept of creating almost an entire ecosystem of different "companies" that do different things (SEO, branding, design, front end/ back end integration,etc) would be very interesting.
Love your videos and I appreciate you take care to explain every steps as I am just learning how to code. I would love if you do a tutorial about how to write a book autonomously, from finding ideas, to writing a summary and then completing each chapter. That would be a great project starting point for me. Really exited about your next videos!
This is so interesting. It worked from the 1st time after correcting one of my spelling mistakes :) I made it generate a list of chemical elements with their details ..
Can't wait to watch the open source and other parts of those videos. Thank you so much.
I've been messing with it AutoGen since you introduced it and I love it. I am at work so I am going to listen to it here and then when I get home I'll work along if there's anything like that. The only thing I wish I knew if you could do is have multiple other AI working together. IE have Bard, GPT, some HuggingFace models, etc. I bought a 4090 and a solid rig to run some AI locally and I'm ready to learn! lol
This man is a fixture of the early day AI revolution just by virtue of spreading accessibility, knowledge and passion. Sincerely thank you
Matt, great tutorial. Thank you. For the community, in some limited testing I have had success using the 'gpt-3.5-turbo-16k' model for the assistant and 'gpt-4' for the user proxy. It seems like (specifically for code execution) the user proxy won't actually initiate execution of code from the assistant if the user proxy is using gpt-3.5. Probably worth exploring deeper as these agent 'teams' grow in size.
This tutorial is so good, that I am watching it third time. Many thanks for this work.
Wow just when I asked for it. Thank you for creating such informational videos and providing resources as well. May god bless you for your work 🙏🏻 And really really interested in the Autogen series and advanced tutorials
This tutorial was just what I needed. You are the best!
I thought this would actually be intermediate :( I really love this topic, I saw your first video on it and in the comments a lot of people asked for a more advanced use case, but you didn’t. Btw I rlly like your videos, looking forward for the advanced tutorial!
I more than excited about AutoGen!
Keep 'em coming!
This is actually THE tutorial that gave me the curiosity to try this thing. Ready for the advance stuff !
Love this!
Wow! Just ran the article summary example with gpt-4, a bit slower but it installed all the necessary packages and generated a great summary on an article. I can see this being a useful addition to my tool-kit ;-)
This is so cool, your videos are the best man! Keep up the good work.
I would LOVE to see an example of a low-level marketing agency with multiple agents working together with something like this.
Say one that researches and generates ideas for industry-relevant twitter (x) posts (agent #1).
Then another one that plans a schedule for them (agent #3), and finally an agent that actually posts them to twitter for you via an API (agent #3) according to the schedule agent #2 came up with.
Basically, multiple agents working together to finish a multi-step task.
Thank you for this video series, it has really helped getting things going.
You got it
Indeed AutoGen is incredible technology. I am trying to use it for real serious business suff. Let's see how it performs. Thank you very much!
GO GO GO!
Awesome, Thank you.
I will be doing this tonight and extending it.
My goal is to write applications in C++ or C#, it will be interesting to see how it does.
Python is ok for writing developer applications not consumer applications.
Thank you so much for creating this! I have 0 coding experience or knowledge but I still managed to make my own CXO Agents! Cheers!
This was great! You make the best LLM tutorials. This was fun!! Got it running except it tended to have trouble running the code script on my system…sometimes, saying it was blocked. But then in the 1-200 round it ran the script. It was wild seeing the agents discussing how to deal with the script blocking issue!!! Yes, most definitely let’s dive in deeper here, both with local OS LLMs and more challenging tasks using gpt. How can it be made to surf the web using gpt, for example? Maybe a plugin or langchain integration?
Much appreciated. Advanced tutorial coming next week :)
Hey Matt you are awesome! Thank you so much for taking the time for sharing your knowledge. Really, thank you
Thanks for solving a problem that's been bothering me, I wish there were more tutorials on autogen, more agents!
I would love to see a deeper dive into a piece of what you started here. Task 1 and Task 2, using multiple group chats that pass information down to each other like going from one team to another.
1. Awesome, please do more advanced stuff (not that this was trivial or anything), but would love to see something that is tangential to a commercial project. Like, how will people use this in production? How can it be deployed to something we can expose to the world? - Brilliant work, keep it up 🙏🏻😍
thanks mathew i am new but will rewatch it some points were a bit complicated
appreciate your wisdom
Amazing tutorial on Autogen! I look forward to seeing Autogen work with other models like Llama-2 that can be instanced using cloud based gpu architecture.
Thank you this was very helpful! Small start, but I see so much potential as LLMs progress..
Thank you. Great content and really helpful😎
Excellent how to, looking forward to testing tomorrow/later today
Looking forward to the Video with an open source model😍
Great video! Please show how to use different agents that work together to solve a task.
Wow, this is so awesome. Thank you so much for this.
You're very welcome!
Careful while playing around with this your OpenAI bill can come to $10 in a couple days or even hours depending on how many times you use AutoGen
Correct.
How much would it cost to pay an employee to do the same?
@@mcombatti That's not the point no matter where you are you could find cheap labor if you outsource, the point is that costs can add up quickly. I made a basic pong game with Chatdev for wayyy cheaper than it cost with AutoGen for example. I feel like this tool is a great steppingstone for better tools in the future. Spotting and fixing mistakes can be mentally taxing when you want to custom it especially when you fix the problem but another one comes up then another one and so on and so on. I realized I was already paying $20 per month for GPT Pro and had set up my custom instructions very well, I could have just put all the prompts into GPT Pro to save time, reduce stress, and cut extra costs.
Mind blowing. I am hooked
Dude I am game for any and all Autogen content you can possibly create 🙌
Really curious to see how far you can go with assistants. Can't wait to see your next videos on the subject.
can't wait to see these experiments. would love to see examples of mini apps (i.e recreate twitter, instagram, etc) or simple recreations of exisiting apps.
You got it!
Thanks for the video. This kind of AI programming Tool is really useful. Good to see it starting to gel.
Thank you for such a good tutorial !
You're very welcome!
Great tutorial 🙌
Thank you
I wanted to make sure you saw my reply about temperature - so I’m posting this as a new comment. Check the API documentation again and scroll down to the “Create Chat Completion” and “Create Completion” sections. They definitely specify the temperature range is 0 to 2.
“temperature
number or null
Optional
Defaults to 1
What sampling temperature to use, between 0 and 2. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic.”
thank you so much for your videos!
Hi, thank you for your video! I have a question about AutoGen: is it for coding only? Is it possible to create a "team of agents" with AutoGen for copywriting or for data processing tasks?
Hi thank you Matthew for creating videos like these. In terms video suggestions, I'm interested in the marketing side of things. Maybe an example video would be "How AutoGen can help you build a 1-man social media marketing agency"
Are you prompting a youtuber?
dude, that was fun, thank you
Thanks for this! One head's up: on a fresh install of anaconda, you may first need to issue 'conda init' before activating the environment, as shown at 2:28.
Thanks for sharing.
That was a pain for me this conda thing ... had conflicts with python3 install via brew.
also need to have env variable
%USERPROFILE%\Anaconda3\condabin
Thank you for putting together this video
Great! Would love to use it with one of the LLAMA2 models.
It can do it, but requires FastChat loading the LLM on the backend.
Please do a tutorial on the best way to intertwine agents and provide them with memory. Also address the issues of rate and token limits. The problem I keep running into is the agents going theough the actions then completing with minimal feedback as opposed to asking me for more prompting for more actions to Complete.
Love seeing ur groth. Werent you at like 32k a few months ago? Either ways man! Great content!
Thanks so much! Yea I started earlier this year :)
Very nice job. I appreciate the way you present material and produce your videos. As far as what to see next, I agree with what many have posted. I'd love to see something more advanced. This example could have been accomplished with GP4 without agents-- its not really illustrating the benefits of multiple agents. Let's see an example where it illustrates how multiple agents extend the power of LLMs
First of all, you are making amazing videos for people interested in AI.
If possible, can you make a video about autogen working locally using text-generetion-webui with its api or openai extensions. Also I wonder if autogen can connect to different models for different agents with running only one instance of textgen.
Thank you and please create more of this stuff!
Awesome video! I have a suggestion if I may, on your final code implementation isn't it more code efficient to create a list with the tasks and then use a for task in tasks: loop to execute the prompts?
Hey Matthew, love how you easily explain it; please, touch on how to use it in real life cases🙌🏼🙌🏼🙌🏼
I very much appreciate your content!
I would like to see more on running open source LLMs such as all the Llama 2 LLMs.
How to run them in a COLAB (free version) notebook by prompting a local API / URL endpoint. When using OLLAMA and LITELLM for this: what exactly do these two tools do, what are their functions in the process?
Is it possible to also use vLLM because of the large speed benefits of vLLM (paged attention)? Does that work in conjunction with OLLAMA and/or LITELLM ? My view of LITELLM is, that it can act as a server for LLMs while having the same interface to the server API as OpenAI has (so the server LLM can act as a drop-in replacement for an OpenAI closed source LLM). Is that a correct view?
I hope you answer these questions. Thanks in advance!
Excellent work Matt, I'm right behind you LOL Still trying to get all the examples running in colab. Curious if you ran into problems doing that. Is the full development environment going better?
Thanks for this. Also very excited about Autogen.
I didn’t run into any issues using Google colab. I like the full development environment better because I just have more control. I have another video coming soon with even more advanced techniques.
Hi, I find this content very interesting. Thanks for it. I would be interested in knowing if far more complex tasks may be done this way. All the Best.
Thanks so much for making this, this igreat!
Would have been neat to see the difference in GPT4. But I understand you are making more videos. Great job!
Keep going! Very clean clear explanations.
Have you used Google colab instead of running locally?
Yes. Check out my previous AutoGen overview where I give an example using Colab :)
Thank you so much for your tutorials.
I'd love a review of the competition in this collaborative ai agents space.. I've heard about chat dev, and I guess there must be others. It would be great to get an overview.
Thanks man!!
You got it!
Hey Matt, love your videos! Here is an idea you might want to consider: Book writing agency. You'll have a team of different agents: a scene writer, a chapter manager (or author), a book outliner, an editor, and a critic. The reason for this specific team is that I found gpt agents have a really hard time writing full chapters, so making them just focus on one scene might be better. I also believe you should have some retrieval system with a vector database or langchain so that the agents who are at a higher level of abstraction can still get information through hundreds of pages of text. I'm sure there are multiple tweaks that needs to be done to this plan as it is put in practice, but I think that it's a decent starting point! I would love to even work with you on this kind of project if you're up for it. It sounds fun!
i'm planning on doing something like this as a novelist myself
Love the video and the series would be fantastic!
How do you take the outcome of one task and send it to another model.
( for example create a blog on the meaning of life and using the summary create a good thumbnail for it, output html to file….
Something like that
Hello, I like this series. Good work. I am wanting to create an accounting team for AI agents that can analyze excel spread sheets and complete tasks if variables appear in the data. The variables would be predefined that the ai it looking for during the scan of the data. If the variable is found, the ai would then need to generate an additional task for a different ai agent to complete. For instance, if a data shows duplicate variable then the ai will need to draft and email to be sent to where that data came from.
I would like to see tutorials on many agents working together to complete many different tasks.
I've been coding using aider for a couple months now and it's pretty good. I wonder if you could do some kind of a comparison video to show why this would be more compelling? I mean for something more than a snake game or counting from 1 to 100 lol. I'm doing a serious application/website from scratch. Aider has been amazing but I'm always looking for a new tools.
This is the first time I'm hearing of aider. The readme indicates github functionality, which autogen doesn't do. Does it actually spawn separate processes to test the code it generates? Reluctantly Microsoft's autogen seems to be the only software available which will actually utilize a local LLM to generate the code. I do not fault Matt's use of an online model for code generation. While autogen does support this, it requires fastchat as a backend and the very latest version of the transformers module. Open-interpreter states that it will work with local LLM's, but in a future version. In other words, it absolutely does not support it now.
Requesting LLama2 (13B-70B) (or any other powerful alternative model) with Autogen
I would also like to see that!
Autogen with langchain where the lliama 2 model can be loaded locally through langchain and also use private data such as a pdf through embeddings so that the llm model will generate a qa reply using the data in the pdf.
I'm not a big fan of using AI outside of my local machine because I don't like sharing my data with large AI companies. Anything you cover highlighting how to install an AI 100% locally is what keeps me coming back for more.
nice vid. would be nice to see a tutorial on setting up an autonomous trading company. Agents for live trading, risk analysis, market analysis, programming & support etc.
yes really would love to see how a local opensource llm works with autogen
Awesome! Thank you! 😊
Thanks a lot for this video. Question: Can I use this on Firebase Cloud functions and create schedule functions for all of these (or for the starting script since it will continue alone fomr there...)? And then I guess I can also receive an email while the job is done?
you should give a real world example, like a complete chrome extension, a back and front end web app.
Looks fantastic! Is there a way to automate my MS OneNote inbox so that captured data can be organized and moved to the correct notebook? Thanks!
Thank you!!
This tutorial is so easy to follow. Curious, what software do you use for the screen capture?
I would be super interested in using this with Llama or another open source local LLM
Great content. Have subscribed and please follow through and continue with further AutoGen videos. I'm looking forward for more of your content. Thank you and great job!
Matt, you can't say 'locally' and then use openAI!! I sthere a way to use Autogen with Flowise or Langflow?
I went back-and-forth on this question of using the word local. Ultimately, I am building this on my local computer. There’s one piece that hits an external API. And I’m still planning my video using an open source model. And so my distinction is local means on the computer and open source is when I mean using a non OpenAI model.
@@matthew_berman Cool. Will wait for it with anticipation. I am learning Langflow and would like to incorporporate an Executor function but this is a way down the road for me. Great teachings.
Can't wait to see a local LLM. Can you try it with the 7B Mistral?
I think you should use a more complicated task (like a snake game via pygame, etc), because it doesn’t show the advantage of multi-agent, compare to a simple one-turn prompt
Wow very cool, I'm also excited about this tool, have you also tried other multiagent frameworks like LangChain, LlamaIndex, AutoGPT, MetaGPT, BabyAGI, ChatDev, or Hugging Face’s Transformers Agents library ?
thank you matt
Sure thing!
nice video ;) from the autogen website, they recommended using fastapi to work with local llm but it only work with .ckpt file, and I use .gguf format and I doesn't find anything so I use python flask to make my own "openai server like" i'm still facing some issue like code execution and some prompt looks weird but it's workable to do it with flask
Thanks, your content is excelent
Thanks!
Awesome! Can you get local open source LLM add local code interpreters working with autogen? I’m trying to get fastchat working as a local LLM like it says in the docs but it’s getting stuck at the last step before launch. Please advise. Thanks!
I’m working on getting a local model setup. AutoGPT is already code interpreter ;)
There is fastgen that can generate openai like api for working with autogen.
Sounds great! This tutorial is good. After the agents thank themselves about 10 times, I'm out of GPT tokens... 🤣 we've got to get the local model setup soon, thanks!@@matthew_berman