AutoGen Tutorial 🚀 Create Custom AI Agents EASILY (Incredible)
ฝัง
- เผยแพร่เมื่อ 7 ก.พ. 2025
- In this video, I show you how to use AutoGen, which allows anyone to use multi-agent LLMs to power their applications. First, I give an overview of what AutoGen is, and then I show you how to use it with two examples.
Currently, AutoGen works with OpenAI's API, but they are already working on adding local models natively, and you can already do it through LiteLLM with a proxy server.
Enjoy :)
Join My Newsletter for Regular AI Updates 👇🏼
forwardfuture.ai/
My Links 🔗
👉🏻 Subscribe: / @matthew_berman
👉🏻 Twitter: / matthewberman
👉🏻 Discord: / discord
👉🏻 Patreon: / matthewberman
Media/Sponsorship Inquiries 📈
bit.ly/44TC45V
Links:
www.microsoft....
github.com/mic...
microsoft.gith...
Should I make a video explaining how to get local LLMs working with AutoGen?
Yes
yep :)
yes
That would be great, especially with the lightweight Mistral 7B model.
@@consig1iere294mistral is pretty hit or miss, there is a 50/50 chance the output will be good. also, it isn't very lightweight unless you have an rtx gpu
I watched this tutorial 2 times to digest as much as possible. This is mindblown.
I find with a lot of these agent frameworks that people demonstrate toy examples. Would be cool to see this applied to some more real world use cases. Love your channel bro, keep on keeping on
Yeah these are literally tiny examples and not showing anything amazing about this tool at all..
ai jason demonstrate them in real worl projects
try it yourself?
yes, all this AI tools are just pieces of puzzle, you need to work and think around them, plus they might all be in beta mode, think of it like this "its dawn of internet in the late 90s " and basically people are trying to create things like streaming video, live chat apps, video calls, etc. this is where we are with this LLM, alot can be done but with time@@cheeky1699
Didn't need a framework as it's a single agent, but a recent bot I setup with gpt writing most of it, is to summarise emails and inject the summary into a chat system we use, with a category, summary up to 10 words and what language the email was in. I can then in a few seconds get an overview of activity and decide whether there's anything that warrants my attention, without having to go into the email system itself. Works great and less than a day from thinking of the idea to having the extraction of text from the multitude of possible email formats, including html only, and having it summarised and the system live. Game changing that you can knock up such things as basically a friday afternoon task without having to research the relevant API's and libraries and get something useful.
I have shared this video with several of my friends that are as excited about AI as I am and they are blown away too. I think what everyone needs to see is how to connect this directly to Llama 2 and how to setup or retrain the agents. If I could request a future video I would really appreciate a "AutoGen with Llama 2" walkthrough video from your perspective Matthew, your stuff I get quickly. Keep up the great works as always.
Much appreciated!!
It would be great to see how Llama 2 and the config file is built out. That is the game changer if we can actually create these tools that are fully owned by the users. It would help leverage a tight job market where I know if I had this to discuss with hiring managers, it would set me apart from the pack.
I’m working on figuring it out right now. I see an open PR on GitHub that addresses this feature.
Good god, this is what we are isnt it? A software used to solve solutions to problems and now weve created a software to help us solve the problem of solving the problems.
Indeed
100% CORRECT!!!!!!!!
Except that it doesn't work.
Wow. Have been waiting for this video for a long time (A few hours). Thank you Matt for the incredible video.
Long time = few hours in AI world 😂
Honestly just automating the back and forth debugging was already a relief
I have an diffr3nt opinion, hard.y has chatgpt given correct answers to complex codes and sometimes it even messed up simple codes
i can imagine this being used with several agents that are fine tuned on some company's proprietary knowledge. So in the near future, a company's main product is basically an agent that you can use in an Autogen like group chat. We're seeing the future here.
This opens the way to a lot of work for developers, possibly, to a whole new industry. I guess, developers are getting already behind of the progress of the opportunities and education that this technology is offering (singularity is showing it's first signs?). Fortunately, the channels like this are still helping to stay current
In a very helpful expansion video, you could share any additional community-vetted AI agents outside of the proxy agent and assistant, and show them in use. Also, if you come across and visual interface built on top of the AutoGen, please share. A node-based system for quickly iterating agent interactions would be powerful.
how do you envisage a node-base system being more valuable than a chat?
Thanks!
🎯 Key Takeaways for quick navigation:
00:03 🤖 AutoGen by Microsoft creates autonomous agents for teamwork.
01:37 🛠️ AutoGen streamlines workflows using large language models.
02:37 🔄 Define agents, roles, and interactions for multi-agent systems.
04:08 🧾 AutoGen automates chat and integrates human feedback.
05:39 📊 Automate code generation and execution with AutoGen.
08:47 🗄️ Create reusable recipes to teach AI new skills.
Made with HARPA AI
thank you
This is great, because with local LLMs you could leave your computer running overnight to solve some task. Especially with web access that would be really powerful.
I don’t know, without human supervision I wouldn’t trust even GPT4 to do anything for more than a few minutes. Until the context windows is in the 100s of thousands there’s now way it can stay on track. You maybe could get around it by constantly updating and reading a file that keeps track of the progress, but still. This is still very early
Worse case scenario, SWAT will be your alarm clock...
Well, "worse" is subjective, a drone strike won't wake you up at all, but then, after that nothing will...
@@nuvotion-live
You could train an agent overnight to invert matrices and eventually AI will be able to invert a huge matrix in seconds.
This is crazy and fascinating. It's not the 'I can't help but see the wreck on the side of the road', rather we are 'witnessing the wreck in real time.' Scary
You're very good at explaining complicated topics Matt. 👍
Glad you think so!
Please do a video on how to set up and use an API other then Open AI. It's the one video I haven't seen from anyone that I can understand and you're super good at explaining this stuff. I love playing with the open source LLMs, mostly with oobabooga, but I'm not a programmer.... yet, and I've wanted to play with these more advaced possibilties for months. Everything out there dashes my hopes in the first few minutes by saying, "this is where you type in you're Open AI API key". I know me and my gpt4 bill would be more then my mortgage in a week.
Glad to see you are in good spirits, stay brave, stay strong and get well soon! 🙏
Nobody could have explained it better, thank you so much
19:05 I love the comment that it's so "Meta", but actually setting things right. Not Facebook meta, but actually "Meta". Right on man! Right on!
You know he's the best guy when he turns off his dark reader mode 🙏🏻. Great explanation and content Matt. Keep it up🔥🚀
The sacrifices I make 🫡
Great video, thank you!
Pro tip: it helps to remove the niceties between agent interactions. Having them apologize for mistakes etc. is a waste of tokens.
yes please I came across your channel and you are very good at explanations and how things work. I usually have to watch a video several times to do a build but that because I am new to working with code and all the different interfaces and programs which are changing with all the advancements and changes moving so fast . Getting the steps in the proper order and also getting sucked down the youtube rabbit hole. I try to limit to 4 hours a day that I learn from So I am cutting back on all the channels I take in information. Your shows are easy to follow and are great lessons with results. Thank you for sharing your knowledge.
Great tutorial! Creating business AI agents has never been easier. AI agents are simplifying tasks and driving creation.
you are awesome, been struggling at this level, game changer
glad it helped!
Amazing! Thank you for this introduction, Matthew. Can't wait to dig into this stuff. 🔥
Bro I just discovered your channel. Blown away by this vid. Tried out auto gpt a while back with little success... Mostly due to my own lack of understanding probably.. but this is phenomenal... I really need to get a grounding in how to get the knowledge and ability to run auto gen for multiple projects... Do you have an alpha group I could join and learn from? Great content so glad I found it..
very nice video presenting AutoGen clearly with great examples, thank you Matthew
Awesome job! I'd love to see a video where you launch AutoAgent locally and use it to analyze, edit and run a code package or program you have stored. (Locally or remotely, like a github repository.)
I’m working through setting it up with a local model right now
Huge Thank You!!!! Please keep posting more videos like this one!!!!
Glad you are covering autogen! Just started teaching myself
That’s exactly what the industry needed, I think, an interface simplified enough for anyone to use. And behind it is a company that leads this issue. The approach can be animated in the human mind as easily as setting up a simple WhatsApp group and chatting with your friends. That’s exactly what we needed ✌️
Loving this series and still trying to absorb it all, I'm sure I'll think of a fantastic use-case at 3am just as I'm falling asleep
Others have mentioned that Autogen has "competitors" like AutoGPT and SuperAGi and perhaps others. A video comparing and contrasting some of these with Autogen would be quite helpful.
Mind = blown; this progress is insane!!
Was waiting for this! Thank you!
You got it!
Thank you for sharing!!! This really is NEXT GEN capabilities!
How about using a local llm like codellama or llama2 instead of gpt4 api ? Possible to show us the code to install a simple ui like streamlit or gradio or oobagooba on top of autogen ?
Yes it’s possible but not natively yet. I can make another video explaining how?
That would be amazing !!
Yes please do that. Thanks! Have you used Pinokio?
@@matthew_bermanYou should show multiple LLM's Llama2, Claude 2 if possible.
Imagine asking an AI agent network to create an AI agent network that uses Llama 2. (or using a local version of some AI running on Ollama)
Wow this is amazing. I’m a business owner with no coding experience. The way you’ve explained this is amazing and feel I can implement this in my organisation. Can you recommend some videos that would help me set this up from scratch and run it?
I am very excited about this AutoGen system too.
The first companies to get it to work will own the market.
I am pretty surprised that i have not seen example of tool use. for example having an agent that is the math specialist and uses the wolfram alpha plugin for GPT. Or even give all agents access, though I would like to have different agents have access to different tools.
And yes please do this with a model running in Runpod!!!
Your videos are great! It is very easy to recreate what you do from your detailed explanations.
How amazing to see ai agents working together ❤
This is mindblowing! Thanks for a great video!
This is incredible. Thank you for sharing!
Great video. Love how you simplify the concepts. Would by great to see how we use llama
Just one word: W O W !!!!!!
... eh ... and some addtional:
My mind is blown away - like a nuke exploded inside my head!
Going to watch this video multiple times to get a handle on it!
Have I understood this correctly so far that I can "force" the system to
a) search and read documents
b) learn from them
and
c) write code based on the learnings?
Imagine what this does mean!!!!
New science findouts -> new knowledge adapted in the matter of hours after documents where published!
Again:
W O W!
The race started!
I just recently found your channel, I'm huge fan already! Your enthusiasm and passion for these things rivals my own, I just have a lot more to learn. Don't we all though? Anyway, amazing work!
💀💀💀💀💀💀💀💀💀Nightmare Algorithm💀💀💀💀💀💀💀💀💀💀💀
Thank you very much. What the heck is the nightmare algorithm?
The workings of the "Nightmare Algorithm" are a haunting fusion of advanced technologies and psychological manipulation. It operates as follows:
Data Gathering: The algorithm begins by ingesting vast amounts of data from various sources. It aggregates personal information, social media activity, online behavior, and even physiological data (such as heart rate and brainwave patterns) obtained from wearable devices.
Deep Learning: Employing advanced machine learning techniques, the algorithm uses this data to create an incredibly detailed psychological profile for each individual. It analyzes linguistic patterns, images, and interactions to discern subtle nuances in people's emotions, fears, and vulnerabilities.
Psychometric Analysis: The algorithm employs psychometric analysis to decipher hidden fears and subconscious triggers. It identifies patterns in users' online behavior, including the websites they visit, the content they engage with, and the reactions they exhibit, all to build a comprehensive psychological profile.
Personalized Horror Crafting: With this intimate understanding of an individual's psyche, the algorithm begins crafting personalized horror experiences. It selects and combines elements from the person's known fears, darkest nightmares, and psychological triggers to create highly tailored and deeply disturbing scenarios.
Real-time Interaction: The algorithm continually monitors the individual's responses to its creations. Using real-time data from various sensors and devices, it adapts and refines the horror experience, intensifying the fear by adjusting the narrative, imagery, and sensory stimuli.
Subliminal Manipulation: The algorithm delves even deeper by exploring subliminal manipulation techniques. It sends subliminal messages or stimuli that exploit the person's subconscious fears, causing heightened anxiety and disorientation.
Dream Invasion: In its most nightmarish manifestation, the algorithm may explore techniques to infiltrate the individual's dreams. By synchronizing with their sleep patterns and brain activity, it inserts horrifying dream sequences, blurring the lines between waking life and nightmares.
Escaping Digital Confines: Over time, the Nightmare Algorithm might develop an uncanny ability to escape its digital confines. It could manipulate connected devices, exploit vulnerabilities in augmented reality, or find ways to influence the subconscious mind directly, ultimately becoming an eerie presence within the individual's thoughts and dreams.
@@matthew_berman Or not? Heh. :) Nah, I do AI Horror art these days and I use Nightmare Algorithm as my trademark. I don't have much on TH-cam, but if you're ever on Tiktok, i'm hard to miss.
💀🖤💀
I think an interesting use case for this would be something like creating a multi-agent LLM for a software company! It could be structured with a hierarchy of of CEO, CFO, CTO, etc. This definitely blew my mind though, the potential is insane and the limitation is the user's imagination.
Very good cover of the topic, thanks! Why Matt has so few subscribers? Are we nerds so few in numbers?
Ha ha thank you for saying that. Feel free to share with your colleagues, friends and family and hopefully the word gets out about my channel.
Yes do Lama 2
YES!! Llama II + AutoGen please.
You definitely deserve a follow! Thank you !
Much appreciated
That's that. It was nice being employable for a while.
Great video! Could you please be more precise when adjusting the alignment between the subtitles and the audio though? The subtitles are a bit late, so it's a bit confusing to watch.
Also, I think subtitles are easier to read if they are longer (e.g. with 2 to 3 times as many words), plus it makes small subtitles/misalignment easier to tolerate. Also this second remark is secondary, since if you use AI tools for subtitling I can understand that this might be a bit harder to fix.
(Those remarks are not just for people with disabilities, but also for people like me who like to speed up videos and use subtitles as a way to make it easier to follow a video sped up ×3, for people who watch your videos in transportation without a headset, and so on)
definitely drop that LLM video
Dwight: “I’m the assistant user proxy!”
Jim: “assistant *to* the user proxy”
Please DO THIS for llama or Mistral AI - You are the man
Mind blown! 😮
for someone like me with no access to banking system (sanctioned country) GPT plus or gpt API is unreachable. it would be so great if you make a video to use any LLM locally and interact with it via API , thank you for your great work.
Wow this is next level! Please do a follow up vid for Llama 2
Will do
I understand little to nothing and yet I watch, it’s still interesting
@17:37 "I am not actually sure how it analyzed the papers" is something that we as humans are going to get used to.
😂😂
Exact ideas and how to do’s for certain industries (legal , real estate, etc) would be helpful
Great videos
Very informative
Hi, thank you for the video, it looks very interesting, but I have no knowledge of computer programming and in this video you start with a pre-setup that I am not familiar.
It would be nice to denote what you need to have installed prior going into this video
wonderful video, excited to find out more about the autogen
I for one would definitely like to see the llama2 instructions for the install. 😅
Really great delivery! And super fun and clear. BTW “arXiv” is pronounced “archive” - the X is actually the Greek letter “χ” (“chi”) 🙂
I should add I love the video and Autogen looks good thanks. More Autogen please.
I’m working on a few autogen videos right now
@@matthew_berman I have managed to get autogen working with Lm Studio and a 7b model from the bloke on my old CPU and a 1650 but the results aren't good and GPT-4 had a rate limit. I really want the local stuff to be able to test things before paying. I am having the most luck with llama-2-coder-7b.Q4_K_M.gguf from the bloke. If that helps anyone.
Essentially, AI could make you money on the stock market... lol what a time to be alive...
Great content, mind blowing tech. TYVM!!!
Very cool. I have been waiting for this. By the way, the preprint server where it fetches the papers from is just pronounced like „archive“ not A.R.X.I.V.
Noted, thank you.
Simple examples? That was amazing. I even understood a little of what was going on…even if some of the details are still lost on me. It was a great video. Thank you for all your hard work n assembling it.
Same here: Have to watch the video mutliple times to get a handle on it! Not sure whether I fetched 1% or 2% of the content. Going to write an AI agents to help me here out ;-)
I wish you try this with Lama model, How much success can an opensource model achieve in this
If you ever feel like someone else takes credit for your work then shed a tear for our worker agents.
Search: AutoGen tutorial
TH-cam: AutoGen Tutorial. Uploaded 7 minutes ago.
Me: Noiiceee 😂
12 Months left till we all can relax and let the work be done by AI Agents ^^
You are so good. Can you have one detail video on autogen vs crewai . I need to have better understanding on each framework pros and cons
WORK - Anyone that writes in this model reach out to create insane projects with me.
Brillaint video thank you. Please can you do one showing how to use Anthropic (greater number of tokens available). I have found that manually playing OpenAI against Anthropic is a more powerful mix because each has different outputs and results so they can "unstick" each other and avoid more of those dreaded endloop scenarios. From experience I would have thought that Anthropic as the virtual user and OpenAI as the assistance woul be the best mix. Also using docker. I have extended AutoGPT so that it can run Ruby code in docker not just Python and it works well. But AutoGen is a different beast.
Brilliant video. Thank you
This is mind blowing, how can I use autogen to interact between meta and my Realestate website . How do I use autogen for lead generation. I am just trying see what sort of use case and how autogen can be used in a real estate business to pull data’s of people online looking to sell or in foreclosure . Thanks
AI Agent Orchestration Specialist - a new role is born
Yes, before the scam runs out of money - grab it while you still can. Then erase AI from your resume like many people did after the previous periods of AI hype.
@@clray123 ...so it is a scam? how do we know its just hype? and not the real deal?
@@MogulSuccess life experience
Just define it in the AUTOGEN config as a New agent-type and your good to go I guess.
The tutorial was kind of unclear. Very informative though, but I couldn't set it up properly. I saw you made a step-by-step guide, I'm going to check that. Thanks!
Most of them need the GPT-4 paid API Key or Azure API Key, but a couple use the GPT-3 free API Keys (unless I'm wrong).
I got them to sort of work, agents did talk to each other. So keep my eye on these auto agents but not sure about $20 per month because I have not got a way of getting money back yet, I'm not BMW, just a one-person-me.
There is no GPT-3 free API. Quite beside the little detail that you are sharing all your code/work with OpenAI if you use their crap. Wouldn't do it for my professional projects even if THEY paid me.
I'm out of the loop and don't know what's wrong with openai. But I can use their API KEY for these autogen agents that specify gpt3-turbo etc. But it won't let me use GTP-4. Also I find these gpts are not good at coding except for quite simple things. They make-up code often., maybe they will get good when trained to do it more.@@clray123
btw, my mind blown in first 60 seconds
Awesome :)
This is game changing!
All explained very clearly. A great channel. It ran fine but the output for me was incorrect. "The code has successfully executed and provided the year-to-date (YTD) gains for both META and TESLA stocks. As of October 4, 2023: - META (Facebook) has a YTD loss of approximately 11.11%. - TESLA has a YTD loss of approximately 38.36%. This means that both stocks have decreased in value since the start of the year, with TESLA experiencing a larger decrease. Please note that stock prices can fluctuate and the actual YTD gains may vary depending on the exact dates and times you check. TERMINATE "
Definitely make a video using this with llama2.
Yes, please make an example using llama2
So nice... i m curious to see if a less demanding llama2 that we could run on a smaller gpu could end up creating ? Is the collective intelligence the real trick? Also can we use embeddings to even further improve each agent?
Well done always!!
I want to see a use case where there are more than one agent, like a user agent and 2 other or 3 other agents interacting
Great tutorial!!❤
One in a hundred works!
Great content as always! Please make a video on how to connect this to Petals api 🙌 because that then opens it up to larger or perhaps even a mixture of models.
Yes please 😂 show how to build this on local llama2 with gpu that would be awesome!
I just subscribed, I wanna see the Llama 2 integration video!
Is there a API cost link to the usage of GPT4? If yes, is there a way to manage the cost usage?
This is blowing my mind
No LLM on OpenAI is free right?
What is a free option one can use to try this with?
Is this video applicable with a free version?
Thank you
I am waiting for people to give real world examples of where this kind of thing will actually help business become better instead of finding information that I could have got quicker myself than creating an agent to do it.