Python Advanced AI Agent Tutorial - LlamaIndex, Ollama and Multi-LLM!
ฝัง
- เผยแพร่เมื่อ 12 มิ.ย. 2024
- Interested in AI development? Then you are in the right place! Today I'm going to be showing you how to develop an advanced AI agent that uses multiple LLMs.
If you want to land a developer job: techwithtim.net/dev
🎞 Video Resources 🎞
Code: github.com/techwithtim/AI-Age...
Requirements.txt: github.com/techwithtim/AI-Age...
Download Ollama: github.com/ollama/ollama
Create a LlamaCloud Account to Use LLama Parse: cloud.llamaindex.ai
Info on LLama Parse: www.llamaindex.ai/blog/introd...
Understanding RAG: • Why Everyone is Freaki...
⏳ Timestamps ⏳
00:00 | Video Overview
00:42 | Project Demo
03:49 | Agents & Projects
05:44 | Installation/Setup
09:26 | Ollama Setup
14:18 | Loading PDF Data
21:16 | Using llama Parse
26:20 | Creating Tools & Agents
32:31 | The Code Reader Tool
38:50 | Output-Parser & Second LLM
48:20 | Retry Handle
50:20 | Saving To A File
Hashtags
#techwithtim
#machinelearning
#aiagents
![[FULL EP.13] เซียนหรั่ง พา ไททศมิตร ลงเรือฝึกสกิลหว่านแห | เฮ็ดอย่างเซียนหรั่ง | One Playground](http://i.ytimg.com/vi/EWFuf2Av0K0/mqdefault.jpg)
![[Live] : ONE 167 วันนี้!! "ตะวันฉาย vs โจ"](http://i.ytimg.com/vi/D9_dvfIa6Ao/mqdefault.jpg)
You are one of the best explainers ever. Out of 50 years listening to thousands of people trying to explain thousands of things. Also, it's raining and thundering outside and I'm creating this monster, I feel like Dr. Frankenstein
50 years of listening, and learning, iam sure you have great knowlege
I wanted to express my gratitude for the Python Advanced AI Agent Tutorial - LlamaIndex, Ollama and Multi-LLM! This tutorial has been incredibly helpful in my journey to learn and apply advanced AI techniques in my projects. The clear explanations and step-by-step examples have made it easy for me to understand and implement these powerful tools. Thank you for sharing your knowledge and expertise!
You are by far my favorite tech educator on this platform. Feels like you fill in every gap left by my curriculum and inspire me to go further with my own projects. Thanks for everything!
I was really looking forward to learn this. Thanks for the video
If you keep getting timeout errors and happen to be using a somewhat lackluster computer like me, changing `request_timeout` in these lines
llm = Ollama(model="mistral", request_timeout=3600.0)
...
code_llm = Ollama(model="codellama", request_timeout=3600.0)
to a larger number (3600.0 is 1 hour, but it usually takes only 10 minutes) helped me out. Thanks for the tutorial!
thanks mate!
Just used your code with llama 3, and made the code generator a function tool, and it was fvcking awesome. Thanks for sharing👍🏻
Some helpful things when going through this:
- Your Python version needs to be
For me, once installed xcode, rerun installing the package and was able to get llama_cp_python wheel to install. Thanks for this note, helped make sense of the error message.
Yup. I encountered this on windows. On my Visual studio I used the ctrl + shift + p opens a search bar. I searched for interpreter and then I was able to access previous versions of python in different environments, I selected Conda environment and opened a new Terminal. I checked python --version and then the selected python version was up.
Amazing as always, Tim. Thanks for spending the time to walk through this great set of tools. I'm looking forward to trying this out with data tables and PDF articles on parsing these particular data sets to see what comes out the other side. If you want to take this in a different direction, I'd love to see how you would take PDFs on how different parts of a system work and their troubleshooting methodology and then throw functional data at the LLM with errors you might see. I suspect (like other paid LLMs) it could draw some solid conclusions. Cheers!
Great video. Would really like to see methods that didn't involve reaching out to the cloud but keeping everything local.
No idea what’s going on but I love falling asleep to these videos 😊
wow this is absolutely mind blowing ,thanks Tim.
Excellent demo! I liked seeing it built in vs code with loops, unlike many demos that are in Jupyter notebooks and can’t run this way.
Regarding more demos like this…Yes!! Most definitely could learn a lot from more and more advanced LlamaIndex agent demos. Would be great to see a demo that uses their chat agent and maintain chat state for follow-up questions. Even more advanced and awesome would be an example where the agent will ask a follow up question if it needs more information to complete a task.
This is very clear and very instructive, so much valuable information! Thanks for your work
This was fascinating, I'm definitely going to be giving it a whirl! I'd love to learn how something like this could be adapted to write articles using information from our own files.
Great vid.. only issue is the fact that the parsing is done externally. For RAG's ingesting sensitive data this would be a major issue.
Thank You for this very informative video. I really like the capabilities of 'LlamaIndex' with PDF's.
I used it to process several of my own medium-size PDF's and it was very quick and correct.
It would be great to have another vid on how to save and reuse the VectorStore for queries
against PDF's already processed. To me this is more important even than the code generation.
This was brilliant, thank you.
Awesome video, man thx a big bunch!
Bro your videos are gold.
Awesome 💯
just awesome !
Thanks for this tutorial and your way of explaining, I've been looking for this ,
Can you also make a vid on how to build enterprise grade generative ai with Nemo Nvidia that would be so interesting, thanks again
"If I fix these up." My god, Tim. You know that won't scale.
Tim - thanks for the wonderful video. Very well done sir!! Is there an alternative to LlamaParse to keep the parsing local?
pymupdf
yes man...this what i want to do and more...
Nice ❤
Nice one
New subscriber here!!!
Could you also do a video on infinite world generation using chunks for RPG type pygame games?
I liked this. Out of curiosity, why venv rather than Conda?
The guys at llmware have some fone-tuned models for RAG and some for function calling (outputing structured data). Could be interesting to try out with this.
Thanks for the tutorial. Is there any alternate to LlamaParse that allows me to run the application completely local?
Good video but do you have a complete ai agent with your own data without the coding formatting? This is the closest tutorial I’ve found to do on premise ai agent implementation that I can understand. Thanks!
Please create a video about production-ready AI agents!
Thanks for this!! Unfortunately I can't run it on my laptop, it takes forever and the AI seems confused. I guess it needs powerful machine...
Great video tutorial! Thanks 🙌
(liked and subscribed, lol)
A bit of a "noob" developer here, so vids like this really help.
I know it's a lot to ask, but....
I was wondering if you might consider showing us how to build a more modular app, where we have separate `.py` files to ingest and embed our docs, then another to create and/or add embeddings to a vector DB (like Chroma), then another for querying the DB. Would this be possible?
It would be nice to know how to have data from one Python file feed data to another, while also minimizing redundancy (e.g., IF `chroma_db` already exists, the `query.py` file will know to load the db and query with LlamaIndex accordingly)
Even better if you can show us how make our `query_engine` remember users' prior prompts (during a single session).
Super BONUS POINTS if you can show us how to then feed the `query.py` data into a front-end interface for an interactive chat with a nice UI.
Phew! That was a lot 😂
nice
Does the mistral large will be available ? I'm wondering if the LLM availability will be up to date or there's other step to do.
I keep getting errors when trying to install the dependencies from requirements.txt
Make sure you have the correct version of python
or better since I prefer he pip install them manually
❤❤❤❤❤❤
Another great tutorial... Thank You! How do I get in touch with you Tim for consultant?
Send an email to the email listed on my about page on youtube
so ollama is run locally in your machine? can i make it cloud based by applying it into my backend?
Error 404 not found - local host - api - chat [FIX]
If anyone else gets an error like that when trying to run the llamacode agent, just run the llamacode llm in terminal to download it, as it did not download it automatically for me at least as he said around 29:11
So similar to what he showed at the start with Mistral:
ollama run mistral.
You can run this in a new terminal to download codellama:
ollama run codellama
thanks alot !!!!
i love this community ... thanks alot
I did one using Llama2.
are you sharing it somewhere?
круто
what keyboard are you using? 😊
Can i use that to make ai agent that can call customers and interact with them and take notes of what's happens ? Thank's
You obviously went to the Matthew Berman School of I'll revoke this API Key before publishing this video!
What are your MacBook Pro specs? I'm looking for a new computer to run llm locally.
Buy a workstation with very good Nvidia gpu, so u can use cuda. If u still want to go for a MacBook Pro, get the M2 with 32gb or 64gb ram. I’m using a MacBook m1 16” 16gb ram and I can only run llms with 7 - 13b without crashing it
I have an M2 Max
Have you ever thought about using colab as a remote webserver with local llm such as llama3 and calling it from your pc to get predictions? I have your same problem and was thinking about solving like this
My mbp pro m1 8gb is hanging while running the llm locally. Any alternatives that we can learn to build without killing my mbp?
Hey can you shared the system configuration need to run this application ?
everytime i try to install the requirements.txt files, it only downloads some of the content but then i get this error message: Requires-Python >=3.8.1, im runnning this on a mac with python version 3.12.3 and I can't seem to download the older version of python.
What if I don't my data to be manipulated in the cloud? Is there an alternative for LlamaParser that can be ran locally?
Can you make a series
it seems multiple elements in the requirements.txt doc require different versions of python and other libraries. Could you clarify what versions what what is needed in order for this to work?
how to know the requirements.txt dependencies we required (it is a large list)
I can't install packages of llama-index in my Windows system. Also, the 'guidance' package is showing an error
did u find the error ?
What's the latency of models running locally?
Is there much difference between result_type = "Markdown" and result_type = "text"?
Is it possible to create an agent using other languages?
Dang seems im stuck with a 404 message @ 31:57.
Anyone else have that issue? Or have a fix for it possibly? Maybe the dependencies need an update already?
Why did I need to downgrade python 3.12 to 11 to be able to run requirements.txt which some dependencies were calling to use a version less than 3.12 but I see you are using python 3 with no errors?
First to comment today 😂
what am i doing wrong cause when I run it does not work no matter what I try
can i do the same using langchain
Can anyone tell me what basic things should one know before going into this video ??
Can you please do a video about making a gui in python
First
How do we know that Meta hasn't corrupted the ollama model with spyware or other malicious code?
Hi Tim!
GREAT JOB on pretty much everything!
BUT, i have a problem
im running on windows with pycharm and it shows me an error when installing the requirements,
because its pycharm, i have 2 options for installing the requirements, one from within pycharm and one from the terminal
FIRST ERROR (when i install through pycharm)
in both options im seeing an error (similar one, but not exactly the same)
can you please help me with it?
you can check which python version you have installed.
@@diegoromo4819hey, thank you for your response, which version should i have? i can't find it in the video.
@@ofeksh 3.11
@@neilpayne8244 shit, that's my version...
I'm getting the following when I run the prompt:
Error occured, retry #1: timed out
Error occured, retry #2: timed out
Error occured, retry #3: timed out
Unable to process request, try again...
What is this timing out on?
Your Agent is unable to reach your Ollama server. It's repeatedly trying to query your Ollama server's API on localhost, then those requests are timing out. Check if your Ollama LLM is initializing correctly. Also make sure your Agent constructor contains the correct LLM argument.
Do you have a VPN like NordVPN running? Sometimes that can't mess up local servers
Hey tim. Great video. I have been watching your videos for some time, though i was definitely young then. I need some guidance. Am 17, i want to do ai and machine learning course. Somebody advise me.
Problems ?
# make sure the LLM is listening
`pip install llama-index qdrant_client torch transformers` `pip install llama-index-llms-ollama`
# didn`t download codellama
`ollama pull codellama`
# timeout error
set request_timeout to 500.
What the minimum laptop to run this model ? Thank's
you need a good gpu to run like litrarly any llm
Llama 3
Well, I can't get it to work. It gives 404 on /api/chat
I am getting the same error
you are probably getting this error because you are missing the codellama model, run ollama pull codellama and it should fix it
But this is not completely "local" since you need an api key, no?
These APIs are used within the same environment or system, enabling different software components or applications to communicate with each other locally without the need to go through a network.
This is common in software libraries, operating systems, or applications where different modules or plugins need to interact.
Local APIs are accessed directly by the program without the latency or the overhead associated with network communications.
DEAL BREAKER:
17:20 "What this will do is actually take our documents and push them out to the cloud."
ollama mistral work so slowly that even hello take like 20 min
I’ll come back to this in a couple of months. Too advance for me, my head is spinning I cannot grasp a single info😵💫
Haha no problem! I have some easier ones on the channel
Me too😅
I can't figure out which AI agent vendor is open for me to sign up to use.
I want to learn how to make an AI agent that auto-removes / auto-deletes these annoying spam s3x bot comments on useful TH-cam videos like this.
Why youtube does not add my comment. F*CK
Using a module that requires you to upload the files or data (LlamaParse/LlamaCloud) totally defeats the purpose of self hosting your on LLM models... Dislike just for that!
it makes as little sense as putting your decentralized currency in a centralized bank. LLAL
stupid orc
How is that an issue? You want to have the ability to parse the files to the model. Are you sure you've grasped the concept of agents and tools? The whole point is have RAG locally.
Decentralized comparison is simply unrelated to what has been done here.
@@iva1389 It is the same thing being done. You're taking something that allows you/your business to do things on their own without third party... but adding 3rd party for no reason. 3rd party where your data can be hacked/stolen/man-in-the-middle attacked. So the comparison IS VALID!
@@iva1389 The whole point of things like ollama and LLMs is to keep things IN-HOUSE. Doing 3rd party defeats the purpose of using these models. Same things as putting decentralized money in central banks. So they really are the same type of stupid thing to do!
It's like saying cocaine is bad for you, but let's go do some crack. :P
Then simply don’t use it and use the local loading instead. I’m just showing a great option that works incredibly well, you can obviously tweak this and thats the idea.
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("BAAI/bge-m3")
from llama_index import download_loader
download_loader("LocalDiskVectorStore")().persist(persist_dir="./storage")
Hi tim.its really.simple 2 understand
One ask is.llama.parse free to use ? Or does it needs subscription key ?
Can we use Lama parse locally ?
It’s free to use!
@@TechWithTim Thanks, but does it requires data to be sent to Cloud?
@@jaivalani4609 it does send it to the cloud and you can do 1000 pages each day with free. it will send the file to the cloud and gets the markdownfile from it
i have this error message : [WinError 126] Le module spécifié est introuvable. Error loadin \AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\torch\lib\shm.dll" or one of its dependencies. Can you help me ?
This is a bug and should be fixed with pytorch >= 2.3.1. If pytorch is version 2.3.0 in requirements.txt, change it to 2.3.1 or a newer release if another one is already out
I get an error when trying to parse readme.pdf:
Error while parsing the file '/Users/.../AI-Agent-Code-Generator/data/readme.pdf': Illegal header value b'Bearer '
Failed to load file /Users/.../AI-Agent-Code-Generator/data/readme.pdf with error: Illegal header value b'Bearer '. Skipping...
Any clue to what might be happening?
I'm pretty sure if has something to do with LlamaParse(). I can't seem to reach LlamaCloud using my API. I copied and pasted it into the .env file
Not sure why load_dotenv() doesn't work for me. I was able to set it using os.environ['LLAMA_CLOUD_API_KEY']