Love this, Matthew :) Have you ever looked into docker for this work? I use it constantly and it helps me deploy local models + agent code to end user laptops (and servers ofc, but most of my work is HIPPA and privacy focused) 😊 If you ever want a deep dive for your content, happy to show you the ropes! The tldr is to simply use a python docker container and bake your agent in there, and then using docker compose you can combine ollama container with GPU access + agent container. Makes it almost magically no config for the end user
While, what you have been doing creates bu!z and excitement, I truly believe we need to certainly classify, basic, intermediate level learning stuff somehow. In basics, one must only discuss fundamentals and intermediate to provide a demo. And I understand it takes blood to create such hour long videos. But you are so ahead in the curve, you are best fit for the job. Guide and be the light! 🙏🙏🙏🙏
Bro keep this going; recently found myself thinking that from all the channels on the topic I trust your message the Most. Feels true & Authentic. Regarding the format, Absolutely longer video is nice Separates you from other News reporters. In fact, I just watched the first 1:27 minutes, stopped. Gotta do the due Dilligence and follow through tomorrow with better circumstances. Really wanna have the together coding experience without spoilers. Sincere gratitude to you for the great work, Matthew Peace to AIl Ivan
I think as a one-off it's interesting to see your thought process as you work on debugging, etc. but I wouldn't necessarily want to see this become a regular type of video. The ratio of value to the viewer to time spent watching is much lower than your other content.
Love this format - You took us all step by step including "mistakes" YAY. This is how we all learn. The mistakes not edited out makes it real and approachable. Thank You for being so helpful!
Hello Matthew. I'm Brazilian, I live in the interior of the state of Paraná, I'm amazed by your content. Here in Brazil, particularly me, there are few people talking and explaining about the creation of AI agents by CrewAI. In fact, only a select group of enthusiasts, followers of Alan Nicolas, have the knowledge to explain this topic. But they themselves are in search of knowledge. However, it is paid content. And not everyone has access to the basics. I would like to express my gratitude for your innovative initiative. I don't know anything about English, but I'm trying my best here to learn. I use AI and Google translate, it takes time, but I think it's worth it. Please continue to make this content available, your teaching is excellent. If possible, teach me the basics and fundamentals, with your videos. Gratitude
Amigão... sou do Paraná também, entusiasta da AI, se tiver algum grupo ou algo do genero, me informe, detalhe que acho interessante é que o desenvolvedor do CrewAI aparentemente é Brasileiro....
The question is where to invest time? At the pace things getting bombarded in AI space, sometimes i feel, wait and let it mature but at the same time i have FOMO, Who else feels the same?
I feel this. Definitely had a tool the boss had me build become obsolete over night. But! I still think it was valuable to do. Lots of techniques I learned along the way and getting comfortable with these "squishy" ways to get what you want is very valuable and transferrable.
@aimademerich my friend in my 40s where I have achieved financial independence, actually there are no pain points, I just love to keep myself hands-on on latest technologies. But I have not seen anything like this.. There is hardly any structure visible atleast from learning perspective.
Just want to share that Iove that you not only cover the news but you actually show us the practitioner side. I share your videos with non-tech fam and they really appreciate your breakdowns, and you save me a lot of words 😂 This is awesome content!
There is no better form than showing all the trials and mistakes of trying to make something work. Because if you edit out all the mistakes... no one really learns anything unless it goes perfectly. Kudos.
This is the best example of how development is done in the age of AI I've found on YT. I like that you said you are not entirely confident in your Python skills because I'm sure a lot of us can relate; AI brought in a lot of new coders (like myself), and we're all crash coursing our way to DIY AGI. What's more important than the final code was your chain of thought through problem solving. While most of us won't be able to FaceTime the founders of tools we're using 😜 you went to ChatGPT for the scraper. I love this format, I love the thought process, I'm always stoked to see a new video. Live editing is a great idea! I know you like to be all together, but it would be fun and useful to debug with you
Hey Matthew, this is THE format. Like my maths teacher always said: you always get more credit when you show your “working out”…because it demonstrates how you reached your answer.
It’s literally GOLD. I’ve been thinking a whole day how to build it with langchain, exactly the same use case. I want to automate newsletters. Initially I created it with notion, zapier and chatGPT, but want first code approach. It’s the sign . Awesome stuff
"Bill.... i've got a problem getting my windows to update..." Great video.... this format is SO MUCH APPRECIATED as the errors you get highlight the things we all come across trying to actually create our own projects. Your troubleshooting is what we really need as examples.... not 'now pip install, blah blah' that most creators videos are. Thank you for trying this style out and please consider doing more of them, even if its just your version of trying to duplicate some new tech Repo's example project.
Man, I applaud you for this. The vulnerability is real. And it makes me much less anxious of diving deeper into code. Especially with lightning, the hassle of an environment setup and some stuff that just not works on my local Mac, is not importatnt anymore. Great stuff!
Thanks to Matt for showing all the processes. It took me almost an hour to finish it. Newbies to Python can see a lot of themselves in it. Matt said that he has learned Python for less than a year. According to Matt's video production rhythm, you should be very busy. Yes, thank you for sharing. Video production is not easy. Thank you for giving us many excellent tutorials.
❤I can’t thank you enough for this format as well as the high-value content of all of your past videos. 🎉 As you already know, this format benefits three parties: developers can learn new stuff as you deep deeper with your tech expertise, non- tech entrepreneurs can catch up with new tech trend with even some understanding, so you will not only attract existing audience but also people who wouldn’t take a lot of time on your channel otherwise. Sorry for the obvious! 😅
I love this format. Could you put chapters in so it's easier to jump to the parts I'm actually using and skipping over stuff I already know? Thanks again! Great work.
Love that this project is being built real time. It's so helpful to see how other people work through all the stumbling blocks Please do more videos like this one!
I really liked the video. It's fun to work through the problem with you and even pause and see if we come up with the same solution. I would like to see more of these from time to time, as the insight into your throught process is helpful (and reinforcing when it maps to soemthing I've also thought about). Separately, for this problem specifically, I think setting up a personal instnace of browserless may make a lot of sense for the scrapting, especially since many of the early CrewAI scripts reference Browserless as a tool.
Hi Matt, really enjoyed your video. I liked the trial and error approach presented raw. These activities are iterative by nature, and you doing it interactively makes it so much more real than just dumping a neatly finished project. Your steps show case the effort that is needed to make these go especially for folks who understand coding in general without necessarily being experts in a specific language. Thank you for your effort and I would love to see more live coding videos.
🎯 Key Takeaways for quick navigation: 00:00 *🏗️ Introduction to building AI teams for task accomplishment using Crew AI and Lightning AI in the cloud.* 00:27 *🔧 Explaining the use of various tools like Lang chain and custom tools in Crew AI.* 01:10 *💻 Demonstration of setting up a coding environment in the cloud using Lightning AI.* 02:16 *🤖 Initial setup for using ChatGPT with Crew AI and managing API keys.* 03:25 *🛠️ Correcting an installation mistake to ensure the latest version of Crew AI is used.* 05:12 *🗞️ Creating a Crew AI task for summarizing AI news articles for a newsletter.* 07:48 *🔄 Adjusting agent roles in Crew AI for more efficient task delegation.* 10:33 *🔍 Tweaking the scraper agent's task to more effectively retrieve and pass content.* 13:52 *🚀 Initiating a test run of the Crew AI setup and troubleshooting errors.* 16:25 *🛠️ Resolving import errors and fine-tuning the code for proper execution.* 20:30 *🤔 Adjusting the code to allow the Crew AI to receive and process multiple URLs.* 24:57 *✏️ Modifying the task code to effectively pass URLs to the agents for processing.* 26:48 *🔄 Realizing a mistake in the code related to the instantiation of the newsletter crew object.* 27:57 *🐍 Sharing personal experience in Python programming and encountering various errors while building the AI application.* 28:40 *🤖 Identifying a missing kickoff command in the newsletter crew's run process.* 29:26 *⚠️ Encountering a limitation where the AI tool cannot directly access and scrape website data.* 31:16 *🛠️ Consulting with the founder of Crew AI for troubleshooting and guidance on the project.* 32:51 *👨💻 Using string interpolation with Python variables to correctly pass URLs into AI tasks.* 34:05 *💡 Discussing potential modifications and additional use cases for the AI crew application.* 35:29 *🔨 Determining to create a custom tool for web scraping to fulfill project requirements.* 37:25 *🧰 Starting the process of developing a custom web scraping tool for the AI project.* 39:23 *🌐 Implementing web scraping in Python using libraries like requests and BeautifulSoup.* 43:10 *🛠️ Troubleshooting import errors and refining the custom scraping tool's functionality.* 46:00 *🖥️ Modifying the web scraping tool to mimic a browser for successful content retrieval.* 48:13 *📉 Encountering issues with websites blocking scraping requests and exploring alternative solutions.* 49:54 *🆗 Successfully scraping content from a different website but facing a new issue with message length limits.* 51:22 *🔎 Using HTML inspection to target specific content on web pages for scraping.* 52:31 *💻 Adjusting the scraping tool to work for a specific website, acknowledging the need for a more general solution in the future.* 53:26 *✅ Successfully scraping website content and obtaining a brief summary.* 53:57 *🛠️ Encountering an error with the scraper tool, indicating a limitation in its current implementation.* 54:24 *📃 Receiving an empty result from the AI agents, but recognizing the successful extraction and summarization of the article.* 54:37 *🚀 Considering enhancements to the AI agent system in future work and appreciating the capabilities of cloud-based development with Lightning AI.* Made with HARPA AI
One of the coolest videos I've seen you upload so far :) It was a great python lesson for me and the way you deal with problems is entertaining and fun. I really think you are one of the people who contribute the most to the field of AI. Thanks for the dark mode - please keep up 😎
Great vid! As a brand new python n00b, it's inspiring to see somebody more experienced than me troubleshooting through new code they're writing. I really appreciate everything you're showing us. Thank you!
I like sooooo much, that you share all your debugging as well. It resonates as I have almost zero Python experience (much less than yourself) and stumbling upon such issues all the time. Interestingly, at first it was irritating, now I do not mind. Part of the process. Being able to create something and see it eventually works is greatly satisfying.
Great video format. Loved the real life tone. Sometimes we think we need to be super proficient to build our ideas into something real but videos like this show us that’s not always the case. Make more please!
This is totally learning by doing and it's the absolute best way to learn! The way you show us your debugging process and how to solve every problem you encounter that is so incredibly valueble.. Please keep making this type of content
This was fantastic! I have written a webscrapper with AutoGen and it was a total b*^ch... It made me feel better that I wasnt the only one who has had some difficulty getting one functioning.
Looks great. It also looks like programmers are going to have jobs for a while longer. They are just going to need to retrain for new architechures, methods, and syntax. Of course new AIs will be there to help them transition. Imagine how well the process of getting your CrewAI app to work would flow if AIs could reason.
This format is great. Sure you wouldn't want every video to be a long form tutorial but this is really helpful and is great when interspersed with your informative and, ahem, "shocking" ai news content. 😉
00:02 Setting up CrewAI for task automation with native and custom tools. 02:04 Setting up and using CrewAI for real-world tasks 07:04 Automating real-world tasks with CrewAI 10:18 Task 1: Scrape and provide content to writer 16:22 Importing tools from CrewAI and exploring examples. 18:56 Automating real-world tasks with Python code 24:33 Automating task by passing it into the task company. 27:11 Troubleshooting and resolving errors in Python code 31:51 Struggling to pass URLs into stock analysis tool 34:05 Discussing the use of website search tool and potential need for website scraping 38:34 Creating a website scraping tool using Python 41:01 Integrating code and addressing potential issues 46:04 Using third party tools and services to scrape websites 48:40 Struggling with complex scraping blockers and open AI's maximum context length issue 54:05 Lightning AI makes tasks easy in the cloud.
Live coding would be amazing. The one thing about the video that I love is bumping into errors. It's encouraging to me being a very novice coder to know that errors are a normal part of the process. Let's jam!
You are great man, you gave me confidence, I've always taught that I lack to much in my python skills but seeing you struggling with it to and still making it work gave me confidence!
I had SO many issues with web scraping, I was practically shouting at the screen "No, not that way!" 😆 It's one of those use cases where with experience you know what will and won't work and exactly how brittle your solution will end up whatever approach you finally rest on - I think it's one of the best arguments for an R1-type model rather than an LLM getting fed source code etc. It raises the point... will all these web platforms ignore our intentional breach of their TOS via automation? Seems like a 'last stand' nobody's talking about yet... The scarping and 'brittleness' issue is one reason why I'm currently leaning toward using Autogen Studio and their 'skills' approach instead - it might be okay for the skills code to be brittle and get evolved when it inevitably breaks, with that kind of modularity built in, much like how e.g. Zapier updates the tehcnicalities of how their building blocks work in the background as issues arise, and your automations/flows don't need changing fundamentally (usually!)
Thank you for this tutorial video. From this I came to know about different types of crewai inbuilt tools which will be helpful to make different projects. In other's videos they are using different apis for this tools.
I like this how to video. It's nice to text new as i releases but building is far more interesting then simply texting the limits of new ai releases. Love your stuff in general. Thank you for sharing your knowledge and expertise.
Thank you for sharing your thoughts process. Have the same use case (weekly AI newsletter in German): currently I'm using a Perplexity collection with custom instruction set to create a summary of a website. Perplexity does a great job on this as I'm not dealing with token issues or blocking scrappers. But the Perplexity API doesn't support calling my collection "GPT". So the process here is broken. Looking forward to see your progress with this use case 😊
I have but a single ask… I know they’re essentially edge-docker containers. But please use a venv or pdm or something, at minimum to prevent dummies from globally installing something somewhere else and completely stuffing their os. And well… the use case here is pdm generates a file struct for you and makes it wayyy easier to work with
Great stuff. Would love to see an example of using Autogen to correlate or extract subsets of data from CSVs, such as GA Reports and Search Console Reports.
Great and thanks for this demo that shows also the pitfalls and how to get rid of them. Did you know that VSCode has the "Live Share" plugin (Microsoft) that allows co-editing in our local vscode, with others in using their vscode? Same as what you shown with this cloud based editor (good to know this one too).
You are the only one of about 5 "crewai for dummies" videos that has even mentioned the IDE used. In this case its lightning AI, only trouble is its taking days to verify my emailsince I dont have an edu or org address, who knows, it may even not be verified. What other suitable cloud/local IDEs are recommended for developing crewAI projects?
Thanks Matthew for the great content. Please, do you have some step by step guide as I have some difficulties in finding the correct text to copy into the documentation? Kind regards,
I'm enjoying the video. But during the middle, I want to type and ask you a question about why you did a specific thing. So, what if you streamed this? You could go through something like this with viewers asking questions along the way. That would be great. Also, could create more views for the channel -- it would also be advertising for your AI consulting. Could be cool.
just saw this drop! looking forward to this one, always was frustrated when your demos just set things up when I ran into issues afterward. I was following your superagi tutorial but it doesnt work right now cause the repository isnt being updated, so I had to manually merge the bug fixes then deploy and its working now
You have a great style of video. The problem for me at least is langchain, crewai, and a number of other dependencies are changing so fast, everything is kind of useless to me. Minor version of a lot of libraries are completely incompatible and langchain especially is constantly restructuring their own libraries so imports break.
I'm new to this. I bought a computer and set it up to duel boot into windows and Linux. So far I've been using Linux on the assumption that it's lighter weight, I'll get better performance and serious programmers will be more likely to use it so there will be better documentation and AI copilot support. I'm getting a little discouraged though because versioning seems much more finicky than what other windows people seem to be going through and it's hard to use things like text to speech and speech to text which I rely on a lot. Should I start over and switch to windows?
The agents will be in the thousands and finetuned for specific tasks, the Tool addons I will make will be A LOT. - I will make them as the human mind works and have an agent for each. reasoning, ideas development, and the prompts used will be Extreme, the best part about it is the AI will help make those personas and tools and its job will be to drop them in the folder and the program will "refresh" and the new tools added, and you can say with this list of 1k tools, I want to do this tasks. X what is the best way to do it with those tools, and then let the AI go to work and delegate, and assembly line certain content.
The scraper thing is a problem I ran into. There are a lot of scrapers out there, but they all seem oriented toward populating massive datasets and not for summarization or other uses. I ended up having to write a scraper in python. It was a pain.
Hi Matthew, thanks for posting this video, very helpful. I'm interested in using agents and want to invest some time in a framework but don't understand enough (yet) about the different frameworks. Autogen is the other framework that gets a lot of mentions. Could you please help me understand why I would choose CrewAI over Autogen and visa versa? Thanks in advance.
Could this be used as an email scraper? Like being able to define parameters for scouring through years of accumulated email to get rid of the crap and keep the stuff that isn't crap?
Do you like this format of video?
What other use-cases should I cover?
Yes we like it
Love this, Matthew :)
Have you ever looked into docker for this work? I use it constantly and it helps me deploy local models + agent code to end user laptops (and servers ofc, but most of my work is HIPPA and privacy focused) 😊
If you ever want a deep dive for your content, happy to show you the ropes!
The tldr is to simply use a python docker container and bake your agent in there, and then using docker compose you can combine ollama container with GPU access + agent container. Makes it almost magically no config for the end user
While, what you have been doing creates bu!z and excitement, I truly believe we need to certainly classify, basic, intermediate level learning stuff somehow.
In basics, one must only discuss fundamentals and intermediate to provide a demo.
And I understand it takes blood to create such hour long videos.
But you are so ahead in the curve, you are best fit for the job. Guide and be the light! 🙏🙏🙏🙏
Bro keep this going; recently found myself thinking
that from all the channels on the topic
I trust your message the Most.
Feels true & Authentic.
Regarding the format,
Absolutely longer video is nice
Separates you from other
News reporters.
In fact, I just watched the first
1:27 minutes, stopped.
Gotta do the due Dilligence
and follow through tomorrow
with better circumstances.
Really wanna have the together coding experience without spoilers.
Sincere gratitude to you for the great work, Matthew
Peace to AIl
Ivan
I think as a one-off it's interesting to see your thought process as you work on debugging, etc. but I wouldn't necessarily want to see this become a regular type of video. The ratio of value to the viewer to time spent watching is much lower than your other content.
Love this format - You took us all step by step including "mistakes" YAY. This is how we all learn. The mistakes not edited out makes it real and approachable. Thank You for being so helpful!
Ah ya!! 👊😎
Hello Matthew. I'm Brazilian, I live in the interior of the state of Paraná, I'm amazed by your content. Here in Brazil, particularly me, there are few people talking and explaining about the creation of AI agents by CrewAI. In fact, only a select group of enthusiasts, followers of Alan Nicolas, have the knowledge to explain this topic. But they themselves are in search of knowledge. However, it is paid content. And not everyone has access to the basics. I would like to express my gratitude for your innovative initiative. I don't know anything about English, but I'm trying my best here to learn. I use AI and Google translate, it takes time, but I think it's worth it. Please continue to make this content available, your teaching is excellent. If possible, teach me the basics and fundamentals, with your videos. Gratitude
Amigão... sou do Paraná também, entusiasta da AI, se tiver algum grupo ou algo do genero, me informe, detalhe que acho interessante é que o desenvolvedor do CrewAI aparentemente é Brasileiro....
olha os BR, sou do RS
vamos criar um grupo ou algo do genero para trocar conhecimento... @@joneslemos
Olá Milton, sou de Curitiba, se quiser trocar idéias, bora se conectar!!!
Brasil!!! kkk Bora aprender!
The question is where to invest time? At the pace things getting bombarded in AI space, sometimes i feel, wait and let it mature but at the same time i have FOMO, Who else feels the same?
Invest time in your biggest pain points, at least with every new development, it can be applied to your biggest use case.
I feel this. Definitely had a tool the boss had me build become obsolete over night. But! I still think it was valuable to do. Lots of techniques I learned along the way and getting comfortable with these "squishy" ways to get what you want is very valuable and transferrable.
Yes.
@aimademerich my friend in my 40s where I have achieved financial independence, actually there are no pain points, I just love to keep myself hands-on on latest technologies. But I have not seen anything like this.. There is hardly any structure visible atleast from learning perspective.
@@keithhanson9441 Seems like get your things done is the mantra :-).
Just want to share that Iove that you not only cover the news but you actually show us the practitioner side.
I share your videos with non-tech fam and they really appreciate your breakdowns, and you save me a lot of words 😂
This is awesome content!
There is no better form than showing all the trials and mistakes of trying to make something work. Because if you edit out all the mistakes... no one really learns anything unless it goes perfectly. Kudos.
This is the best example of how development is done in the age of AI I've found on YT. I like that you said you are not entirely confident in your Python skills because I'm sure a lot of us can relate; AI brought in a lot of new coders (like myself), and we're all crash coursing our way to DIY AGI.
What's more important than the final code was your chain of thought through problem solving. While most of us won't be able to FaceTime the founders of tools we're using 😜 you went to ChatGPT for the scraper. I love this format, I love the thought process, I'm always stoked to see a new video.
Live editing is a great idea! I know you like to be all together, but it would be fun and useful to debug with you
Hey Matthew, this is THE format. Like my maths teacher always said: you always get more credit when you show your “working out”…because it demonstrates how you reached your answer.
It’s literally GOLD. I’ve been thinking a whole day how to build it with langchain, exactly the same use case. I want to automate newsletters. Initially I created it with notion, zapier and chatGPT, but want first code approach. It’s the sign . Awesome stuff
"Bill.... i've got a problem getting my windows to update..."
Great video.... this format is SO MUCH APPRECIATED as the errors you get highlight the things we all come across trying to actually create our own projects. Your troubleshooting is what we really need as examples.... not 'now pip install, blah blah' that most creators videos are.
Thank you for trying this style out and please consider doing more of them, even if its just your version of trying to duplicate some new tech Repo's example project.
Death Vader here. I've been with you a minute.
I haven't watched this yet but it's in my schedule.
You are my freaking hero.
Man, I applaud you for this. The vulnerability is real. And it makes me much less anxious of diving deeper into code. Especially with lightning, the hassle of an environment setup and some stuff that just not works on my local Mac, is not importatnt anymore. Great stuff!
Thanks to Matt for showing all the processes. It took me almost an hour to finish it. Newbies to Python can see a lot of themselves in it. Matt said that he has learned Python for less than a year. According to Matt's video production rhythm, you should be very busy. Yes, thank you for sharing. Video production is not easy. Thank you for giving us many excellent tutorials.
You are a very good teacher. That's the way coding works.
Thank you for the clarity. All the best.
❤I can’t thank you enough for this format as well as the high-value content of all of your past videos. 🎉 As you already know, this format benefits three parties: developers can learn new stuff as you deep deeper with your tech expertise, non- tech entrepreneurs can catch up with new tech trend with even some understanding, so you will not only attract existing audience but also people who wouldn’t take a lot of time on your channel otherwise. Sorry for the obvious! 😅
I absolute love this video. These types of videos will double your subscribers by the summer, if not more. I promise you.
You are phenomenal for this one!!! Thank you!! That IDE inclusion with this tut is everything
I love this format. Could you put chapters in so it's easier to jump to the parts I'm actually using and skipping over stuff I already know? Thanks again! Great work.
Love that this project is being built real time. It's so helpful to see how other people work through all the stumbling blocks
Please do more videos like this one!
I really liked the video. It's fun to work through the problem with you and even pause and see if we come up with the same solution. I would like to see more of these from time to time, as the insight into your throught process is helpful (and reinforcing when it maps to soemthing I've also thought about).
Separately, for this problem specifically, I think setting up a personal instnace of browserless may make a lot of sense for the scrapting, especially since many of the early CrewAI scripts reference Browserless as a tool.
This is my absolute favorite format!! Definitely the right direction for a channel like yours.
Hi Matt, really enjoyed your video. I liked the trial and error approach presented raw. These activities are iterative by nature, and you doing it interactively makes it so much more real than just dumping a neatly finished project. Your steps show case the effort that is needed to make these go especially for folks who understand coding in general without necessarily being experts in a specific language. Thank you for your effort and I would love to see more live coding videos.
The coding format, with all the re-works is awesome!
There is nothing braver than live-coding with a new library. Mad props.
🎯 Key Takeaways for quick navigation:
00:00 *🏗️ Introduction to building AI teams for task accomplishment using Crew AI and Lightning AI in the cloud.*
00:27 *🔧 Explaining the use of various tools like Lang chain and custom tools in Crew AI.*
01:10 *💻 Demonstration of setting up a coding environment in the cloud using Lightning AI.*
02:16 *🤖 Initial setup for using ChatGPT with Crew AI and managing API keys.*
03:25 *🛠️ Correcting an installation mistake to ensure the latest version of Crew AI is used.*
05:12 *🗞️ Creating a Crew AI task for summarizing AI news articles for a newsletter.*
07:48 *🔄 Adjusting agent roles in Crew AI for more efficient task delegation.*
10:33 *🔍 Tweaking the scraper agent's task to more effectively retrieve and pass content.*
13:52 *🚀 Initiating a test run of the Crew AI setup and troubleshooting errors.*
16:25 *🛠️ Resolving import errors and fine-tuning the code for proper execution.*
20:30 *🤔 Adjusting the code to allow the Crew AI to receive and process multiple URLs.*
24:57 *✏️ Modifying the task code to effectively pass URLs to the agents for processing.*
26:48 *🔄 Realizing a mistake in the code related to the instantiation of the newsletter crew object.*
27:57 *🐍 Sharing personal experience in Python programming and encountering various errors while building the AI application.*
28:40 *🤖 Identifying a missing kickoff command in the newsletter crew's run process.*
29:26 *⚠️ Encountering a limitation where the AI tool cannot directly access and scrape website data.*
31:16 *🛠️ Consulting with the founder of Crew AI for troubleshooting and guidance on the project.*
32:51 *👨💻 Using string interpolation with Python variables to correctly pass URLs into AI tasks.*
34:05 *💡 Discussing potential modifications and additional use cases for the AI crew application.*
35:29 *🔨 Determining to create a custom tool for web scraping to fulfill project requirements.*
37:25 *🧰 Starting the process of developing a custom web scraping tool for the AI project.*
39:23 *🌐 Implementing web scraping in Python using libraries like requests and BeautifulSoup.*
43:10 *🛠️ Troubleshooting import errors and refining the custom scraping tool's functionality.*
46:00 *🖥️ Modifying the web scraping tool to mimic a browser for successful content retrieval.*
48:13 *📉 Encountering issues with websites blocking scraping requests and exploring alternative solutions.*
49:54 *🆗 Successfully scraping content from a different website but facing a new issue with message length limits.*
51:22 *🔎 Using HTML inspection to target specific content on web pages for scraping.*
52:31 *💻 Adjusting the scraping tool to work for a specific website, acknowledging the need for a more general solution in the future.*
53:26 *✅ Successfully scraping website content and obtaining a brief summary.*
53:57 *🛠️ Encountering an error with the scraper tool, indicating a limitation in its current implementation.*
54:24 *📃 Receiving an empty result from the AI agents, but recognizing the successful extraction and summarization of the article.*
54:37 *🚀 Considering enhancements to the AI agent system in future work and appreciating the capabilities of cloud-based development with Lightning AI.*
Made with HARPA AI
Matthew, thanks a lot for this extensive demo. Seeing what can go wrong and seeing how you solve the problems was absolutely inspiring. Thanks!!!
One of the coolest videos I've seen you upload so far :)
It was a great python lesson for me and the way you deal with problems is entertaining and fun.
I really think you are one of the people who contribute the most to the field of AI.
Thanks for the dark mode - please keep up 😎
Great vid!
As a brand new python n00b, it's inspiring to see somebody more experienced than me troubleshooting through new code they're writing.
I really appreciate everything you're showing us. Thank you!
Yes, a live coding session would be awesome. This would be so useful in a system upgrade / outage and not just development
It is well doing, seeing another one, playing around. This is motivating.
I like sooooo much, that you share all your debugging as well. It resonates as I have almost zero Python experience (much less than yourself) and stumbling upon such issues all the time. Interestingly, at first it was irritating, now I do not mind. Part of the process. Being able to create something and see it eventually works is greatly satisfying.
Thank you for this content! I’m really excited to dive into this
YES! Love everything about this especially using Lightning AI and developing a solution from end to end that actually solves real problems.
It really helps to see you struggle a bit in there to get it all working for those of us learning here.
Great video format. Loved the real life tone. Sometimes we think we need to be super proficient to build our ideas into something real but videos like this show us that’s not always the case.
Make more please!
Great video, but FYI: "RC" in a semantic version number stands for "Release Candidate" not "Release Client".
Came across your channel as I was searching AI Agents, very awesome stuff!!! I greatly appreciate your time with this! Thank you!
This is totally learning by doing and it's the absolute best way to learn! The way you show us your debugging process and how to solve every problem you encounter that is so incredibly valueble.. Please keep making this type of content
Yes, I like this. Going through exactly the same kind of thing we are all doing right now.
This was fantastic! I have written a webscrapper with AutoGen and it was a total b*^ch... It made me feel better that I wasnt the only one who has had some difficulty getting one functioning.
Hi - This is an excellent video! Thank you for taking us step by step from "Zero". Great production & format effort. Please do more!
this format is great. please keep them coming if/when you have time. thanks
Your channel is an absolute blessing for this world! Legendary.
Looks great. It also looks like programmers are going to have jobs for a while longer. They are just going to need to retrain for new architechures, methods, and syntax. Of course new AIs will be there to help them transition.
Imagine how well the process of getting your CrewAI app to work would flow if AIs could reason.
Yes, ChatGPT wasn't even able to help Mat out. AI will need some further improvements to replace coders.
This format is great. Sure you wouldn't want every video to be a long form tutorial but this is really helpful and is great when interspersed with your informative and, ahem, "shocking" ai news content. 😉
🤫 Good one. Matt, thank you for showing how AI works. Keep going. 🌹👏👏
The agents didn’t ‘SHOCK THE ENTIRE INDUSTRY ‘?! What a miracle ✌🏼
Im very happy you made this video, I like to see how other people learn by doing and struggle on the way. 😊
Great video format Matthew, thank you for being open and sharing the steps.
00:02 Setting up CrewAI for task automation with native and custom tools.
02:04 Setting up and using CrewAI for real-world tasks
07:04 Automating real-world tasks with CrewAI
10:18 Task 1: Scrape and provide content to writer
16:22 Importing tools from CrewAI and exploring examples.
18:56 Automating real-world tasks with Python code
24:33 Automating task by passing it into the task company.
27:11 Troubleshooting and resolving errors in Python code
31:51 Struggling to pass URLs into stock analysis tool
34:05 Discussing the use of website search tool and potential need for website scraping
38:34 Creating a website scraping tool using Python
41:01 Integrating code and addressing potential issues
46:04 Using third party tools and services to scrape websites
48:40 Struggling with complex scraping blockers and open AI's maximum context length issue
54:05 Lightning AI makes tasks easy in the cloud.
Human coding isn't dead. You just proved it.
This is an awesome format - warts and all! Really appreciate it. Feels like pair programming. Saved me a bunch of time.
Best video ever! Beats all the videos that shows perfect codes hands down.
I want to see live editing.
Live coding would be amazing. The one thing about the video that I love is bumping into errors. It's encouraging to me being a very novice coder to know that errors are a normal part of the process. Let's jam!
You are great man, you gave me confidence, I've always taught that I lack to much in my python skills but seeing you struggling with it to and still making it work gave me confidence!
I had SO many issues with web scraping, I was practically shouting at the screen "No, not that way!" 😆 It's one of those use cases where with experience you know what will and won't work and exactly how brittle your solution will end up whatever approach you finally rest on - I think it's one of the best arguments for an R1-type model rather than an LLM getting fed source code etc. It raises the point... will all these web platforms ignore our intentional breach of their TOS via automation? Seems like a 'last stand' nobody's talking about yet... The scarping and 'brittleness' issue is one reason why I'm currently leaning toward using Autogen Studio and their 'skills' approach instead - it might be okay for the skills code to be brittle and get evolved when it inevitably breaks, with that kind of modularity built in, much like how e.g. Zapier updates the tehcnicalities of how their building blocks work in the background as issues arise, and your automations/flows don't need changing fundamentally (usually!)
Thank you for this tutorial video. From this I came to know about different types of crewai inbuilt tools which will be helpful to make different projects. In other's videos they are using different apis for this tools.
Yes, I like the live build format, thanks for presenting it.
I like this how to video. It's nice to text new as i releases but building is far more interesting then simply texting the limits of new ai releases. Love your stuff in general. Thank you for sharing your knowledge and expertise.
Thank you for sharing your thoughts process.
Have the same use case (weekly AI newsletter in German): currently I'm using a Perplexity collection with custom instruction set to create a summary of a website. Perplexity does a great job on this as I'm not dealing with token issues or blocking scrappers. But the Perplexity API doesn't support calling my collection "GPT". So the process here is broken.
Looking forward to see your progress with this use case 😊
I have but a single ask…
I know they’re essentially edge-docker containers.
But please use a venv or pdm or something, at minimum to prevent dummies from globally installing something somewhere else and completely stuffing their os.
And well… the use case here is pdm generates a file struct for you and makes it wayyy easier to work with
Great stuff. Would love to see an example of using Autogen to correlate or extract subsets of data from CSVs, such as GA Reports and Search Console Reports.
Really liked this long-form tutorial/Let's Build
Great video! Awesome!🎉🎉🎉
Matt, again, thank you for all of this!
Great and thanks for this demo that shows also the pitfalls and how to get rid of them. Did you know that VSCode has the "Live Share" plugin (Microsoft) that allows co-editing in our local vscode, with others in using their vscode? Same as what you shown with this cloud based editor (good to know this one too).
I have been working on similar AI code and getting mixed results. The struggle is real, keep up the good work.
Great video !!!! really helpfull
if you are going to extract a specific element from the html perhaps the websearch tool is what you should have used
Just been reading crewai docs, a PDF write tool would be a good addition to the toolset.
Good video we like it keep it up upload videos like this man lit 🔥
Awesome video - let’s go deeper
You are the only one of about 5 "crewai for dummies" videos that has even mentioned the IDE used. In this case its lightning AI, only trouble is its taking days to verify my emailsince I dont have an edu or org address, who knows, it may even not be verified. What other suitable cloud/local IDEs are recommended for developing crewAI projects?
You are doing great job you update us with AI I hope you get 1 million subscribers very soon 🌹
You have the best content!
Thanks Matthew for the great content. Please, do you have some step by step guide as I have some difficulties in finding the correct text to copy into the documentation? Kind regards,
Which one is your fav Matthew. Autogen or Crew AI and why?
I'm enjoying the video. But during the middle, I want to type and ask you a question about why you did a specific thing. So, what if you streamed this? You could go through something like this with viewers asking questions along the way. That would be great. Also, could create more views for the channel -- it would also be advertising for your AI consulting. Could be cool.
just saw this drop! looking forward to this one, always was frustrated when your demos just set things up when I ran into issues afterward. I was following your superagi tutorial but it doesnt work right now cause the repository isnt being updated, so I had to manually merge the bug fixes then deploy and its working now
Thanks so much, this was exactly what I was looking for!
wow 55 minutes. what a gem
You have a great style of video. The problem for me at least is langchain, crewai, and a number of other dependencies are changing so fast, everything is kind of useless to me. Minor version of a lot of libraries are completely incompatible and langchain especially is constantly restructuring their own libraries so imports break.
How do you get that green circle on your mouse pointer? It looks useful for reading
I like your style buddy.. you explain things pretty good and you have a large variety on your channel. You look a little bit like pauly shore
this vedio showed me programmers are needed after all when mattew said i will make chatgpt writes code
when you are programmer it makes a difference
I really appreciate this video. Thank you Sir!
You said you'd share the code. Where can I find it?
Thanks for all your work!
Oh shoot! I forgot. I’ll share it now.
Huge Thank You!
I'm new to this. I bought a computer and set it up to duel boot into windows and Linux. So far I've been using Linux on the assumption that it's lighter weight, I'll get better performance and serious programmers will be more likely to use it so there will be better documentation and AI copilot support. I'm getting a little discouraged though because versioning seems much more finicky than what other windows people seem to be going through and it's hard to use things like text to speech and speech to text which I rely on a lot. Should I start over and switch to windows?
The agents will be in the thousands and finetuned for specific tasks, the Tool addons I will make will be A LOT. - I will make them as the human mind works and have an agent for each. reasoning, ideas development, and the prompts used will be Extreme, the best part about it is the AI will help make those personas and tools and its job will be to drop them in the folder and the program will "refresh" and the new tools added, and you can say with this list of 1k tools, I want to do this tasks. X what is the best way to do it with those tools, and then let the AI go to work and delegate, and assembly line certain content.
The scraper thing is a problem I ran into. There are a lot of scrapers out there, but they all seem oriented toward populating massive datasets and not for summarization or other uses. I ended up having to write a scraper in python. It was a pain.
Hi Matthew, thanks for posting this video, very helpful. I'm interested in using agents and want to invest some time in a framework but don't understand enough (yet) about the different frameworks. Autogen is the other framework that gets a lot of mentions. Could you please help me understand why I would choose CrewAI over Autogen and visa versa? Thanks in advance.
Great stuff!
Did you come up with a more generic web scraper tool. The version in your git hub is still hardcoded to work with that one site.
VS Code can also be set to save automatically, so there will be no need for CTRL+S :)
Could this be used as an email scraper? Like being able to define parameters for scouring through years of accumulated email to get rid of the crap and keep the stuff that isn't crap?
I just give a thumbs up automatically on new content before clicking play. I know it’s gonna be good anyways
Automating my life sounds awesome but I'll probably just end up more lazy.
your videos are awesome Matt :=)