Just found out your channel and subbed after watching this video. Very good quality video! Keep up the great content creation! I am impressed with your motivation to answer the questions from your comment section. Not every youtube channel cares about answering subscriber questions and doing content around what people comment , so very good job!
Great video, took me a few tries getting Ubuntu configured (using wsl under windows) and updated before it would build the scripts, but I am very impressed. Thanks for all the effort you put into these videos 😁
Make sure you have installed all the frameworks listed in the requirements.txt file. Rewind the video, he explains how to do it all at once. pip install -r requirements.txt If you have done that, make sure you are working in the correct environment. The same environment you installed the python frameworks in. He explains that as well.
Awesome idea. Now use Streamlit or Flask as GUI and the solution is perfect. Thanks for the inspiration. Questions: 1. which OS are you using? 2. python version? 3. do you rather use CPU or GPU? Would an M3 with 128 GB also be sufficient for a quick training / fine tuning of hugging face models up to 20B? Thanks for the answer in advance. Hp
1. Mac OS. Pretty basic. Macbook Air M2, 8 GB 2. python version 3.9 ++ version 3. M3 with 128 GB.. Hmm when you say fine tuning it depends on the model. But here is a rough calculation. If you have a 20B parameter model in 32 bits, then you need 20x32/8 = 80 GB GPU. your system should be able to do the fine tuning. Else please go for 4 bit quantization, then the requirement will reduce by 8 times, now requiring only 80/8 = 10 GB of GPU.
Followed your instructions here and installed on Raspberry Pi4. Works but of course painfully slow and chip approach 145 degrees F which slows things down as well. But it works and may try on Pi5. Was using a pdf manual for Viking drill press for document. Have to try something with just text.
Are you using the code from private gpt’s primordial version? What a great work! By the way, now they officially support integration with ollama. You probably wanna make a video about how to set this up.
My brain is exploding because I was just using your code repository earlier today and then closed everything out. I'm trying to run it again and I'm now getting a bunch of depreciation LangChain warnings. I still get a response but the response is no longer referencing or citing the documents I placed in the folder. Any idea why this would happen?
Thanks for doing this: It seems like when one clones the repository, you need to delete everything in the db folder or it will mess up the results of the information you are trying input yourself.
Would like to see ollama with openai assistant or similar that can use a file and have assistants do action to the file such as execute code or clean up data etc.
Actually using it in linux platform but still I gave the command ollama serve, I'm getting no GPU detected, Please give solution for this ! @@PromptEngineer48
Thanks, that’s a great video! I’m testing privategpt for some time and I would love to know if you’re experiencing hallucinations from the chat? And yes mistral seems to be a good model but Zephyr and Dolphin seems to give better answers with a little less performance, depending on the context. Therefore, I couldn’t Get rid of some hallucinations, I would say the reliability of information is 45-65%
i am facing thiis error conda : The term 'conda' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again. At line:1 char:1 + conda create -n private1 python=3.11 + ~~~~~ + CategoryInfo : ObjectNotFound: (conda:String) [], CommandNotFoundException + FullyQualifiedErrorId : CommandNotFoundException
If I delete the cache folder name db , will this effect how the program will ingest the files I supply or will create a new db and then ingest the new files?
Great Tutorial, Why is it saying loading 235 new documents when it has only one in the source_documents folder. I had 2 in mine and it said 8, wondering why.
Hi, while setting up Virtual environment I'm getting an error, 7:39 what programs do I need? I have installed Miniconda and it's still no running the line. Thks
It would be great if you will create a video where you work on entire project, consisting dozens of files, related to coding adding, editing some feature. Thinking about that, DeepSeek Coder would be a best choice I suppose.
@@PromptEngineer48 Also, will it be necessary to use MEMGPT to store different contexts in separate chat windows? I'm bit frustrated trying to figure out bundle of all required to perform what I have mentioned in initial comment =)
@@PromptEngineer48 Bing came back with following bundle: Yes, you can use **AutoGen**, **MemGPT**, and **DeepSeek Coder** together to chat with an entire project like a Ruby on Rails (RoR) project consisting of dozens of files. But it got concerns regarding token limit: "It might be challenging due to the token limit. Each file would need to be processed separately, and the total number of tokens across all files would need to be within the token limit. If the total exceeds the limit, you might need to truncate or otherwise reduce the size of your input1. Remember, a token in this context is not necessarily a word. It could be as short as one character or as long as one word. For example, “ChatGPT is great!” is encoded into six tokens: [“Chat”, “G”, “PT”, " is", " great", “!”]
Thank you a lot for your effort to create an easier way to get up and running. Could you make a video explaining what all of the parts of "requirements.txt" are needed for? I would like to understand how everything works in detail
Does anyone get this warning - UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown (and subsequent crash)
About the context window. I have noticed that it cannot exceed over 2k tokens even though Mistral can support up to 8k. From what I have tested so for, it is like the bot identify itself as a GPT-3, is it because of the openai library?
yes. That is because we start everything compatible with OpenAI API. then we shift to opensource APIs. We could instead work for OpenSource APIs for the start. 😁
thank you, sorry but i don't see the old chat interface :-) i have 2 questions how can i remove a ingested document that i don't need? ist it possible to chat with the docker interface ? or is it only for terminal??
1. U can remove the file. Then delete the db folder and __pycache__ folder.. then run python ingest.py again and python privateGPT.py again. 2. Web UI interface is not integrated here right now. But that is some ok my pipe line as well. I am working on that.
If my company has a pre-existing Data Dictionary, is there a way to allow Ollama to integrate it so that it better understands the data I am working with?
Great video!! However if i follow the instructions my results are different. I created the source_documents folder an put in another pdf-file. When i then execute the "python3 ingest.py" the ingestion seems to work fine. But when i afterwards exceute the privateGPT.py and start to interact with the llm it still responses to "Think and Grow Rich"-Book.
@@PromptEngineer48 can we not append the pdf files, do we have to keep removing them. when a new file is added and ingested, it should add the document into its cache and should be able to response from both the document isnt it
@@PromptEngineer48ok managed to get Conda working. However when I run the python3 ingest.py, I got an error: line6, in from tqdm import tqdm. ModuleNotFoundError: No module named ‘tqdm’
Ok made more progress but now stuck in pymupd. Tried installing it but keep getting message Requirement already satisfied: /opt/anaconda3/envs/privategpt/lib/python3.11/site-packages
Hello, I am trying to follow your youtube video about "Private Chat with your Documents with Ollama and PrivateGPT" but when I type in my query after running the 'python privateGPT.py' part. I get the following error message "ValueError: Ollama call failed with status code 404. Details: model '7560' not found, try pulling it first". I'm not sure what this is talking about as I do the 'Ollama pull mistral' command before running the rest of the code. Any idea how to fix this?
@@PromptEngineer48 even after ingesting the PDFs,I am getting answers for some questions that are not in pdf.Is it because of the already trained model?
Hello, i keep getting error ERROR: Could not find a version that satisfies the requirement onnxruntime>=1.14.1 (from chromadb) (from versions: none) ERROR: No matching distribution found for onnxruntime>=1.14.1
I had the error for ChromaDB too. If you Google for them you will probably find an answer. For ChromaDB this solved it for me: - sudo apt install python3-dev - sudo apt-get install build-essential -y
Hi, thanks for the tutorial. I am able to run the model using Ollama on windows but after everything the project runs and Asks for the Enter a query and if I add the questions it stucks there. In problems I am getting bekow issues such as Import "langchain.chains" could not be resolvedPylance Import "langchain.embeddings" could not be resolvedPylance Import "langchain.callbacks.streaming_stdout" could not be resolvedPylance etc Can you please help me out in understanding this issue and resolving it. Thanks.
Hi, I did what you said with some hesitation but it worked as you said . This is one good program to use to have a good look at a documents content and meaning. Thanks. BTW is there way to let the program give a full response with out telling it to continue with its explanation?
Wonderful, thank you! My dream would be to feed my local model with all my books in epub or pdf format just once and the model will keep a memory of those. Then from there I have so many ideas but not sure if feasible, maybe someone can help? 1) for each book create a mind map of concepts 2) a diagram of how each book is related to the others (citations, same authors, same topic, related concepts) 3) given a question or a topic the system can point me to which book is better to read. Probably impossible at the moment, isn't it?
@@PromptEngineer48 Awesome! Another one could be to automatically find the book category. Since the system has already ingested all the books it already knows the discussed topics in each book and from there it can assign each to an ontology of category and subcategories. Like non-fiction/self-help, fiction/novel, non-fiction/self-help/personal growth, and so on. I have other 100 of these ideas, ping me if you need more :D
I would also be interested in something like this if possible as it would be amazing to do this in a repository of books that you know. If you could do this with a non-trivial number of books, say, 100-150 (from Project Gutenberg), it would be a great application of local LLMs.
Thank you very much, all the videos you have made are great! However, when I was building the local llm, I found that there are multiple frameworks, such as privateGPT, localGPT, langchain, etc. Similarly, I found that there are very many choices for the llm as well as the vector database (e.g., hugging face vs. ollama), which gave me a big headache, and I was wondering if you could make a I was wondering if you could make a video that explains your recommendations for each part of the process of building a rag-based personal local document chat llm?
That would be a very good video but very less relevant as everyday we have so many updates. But this is something I can create. thanks for the idea. Definelty will work on that
@@PromptEngineer48 Thank you for your response! It's true that, as you say, the various programs are moving so fast. Since I'm hoping to be able to build something myself via langchain, I'm starting to work based on your github project, and some of the other out-of-the-box projects (e.g. open-webui, privateGPT, etc.) are just too heavy for me to get into and modify.
This has been very instructive, thanks! Is there an LLM that's better than Mistral at working with very technical documents, i.e. lengthy API implementation documents? I set up my environment exactly as instructed, got my docs indexes and could get some answers on them. But things get murky when I go very specific, with incorrect or incomplete answers.
Outstanding work you did bro. I was able to get mine running following your instructions with no issues. Does this support llama2 as well? My other question, what are the steps to run privategpt ui? Thanks a lot, this is brilliant!
How would you do this if you have ollama in docker? Or even cloud gpu like runpod? Or Linode? Seems like a good video for the future as you explain,and teach better than a majority of the creators.
Okay. Will create a video soon on the specific use case that you have mentioned.. I mean by hosting on docker and chatting with docs. Thanks for the comment
Very interesting, easy to follow. I tested this with a music instrument manual, and it wasn't giving accurate answers at all. Any ideas on how to improve this? It's unusable for this type of document. Makes me wonder how accurate it would be with any content. Thank you!
Thank you for this great content! I was able to get this working on my M1 Mac. I was able to run `python ingest.py` and the `python privtagpt.py` commands. However, when I asked it to summarize the document I had uploaded, it referenced the "Think and Grow Rich" document that you showed in the video, rather than the test document I uploaded to the source_documents folder. How do I clear out the embeddings from the "Think and Grow Rich" document, or clear the chroma db database embeddings completely before running the ingest.py command again?
I got an error when I typed ollama run mistral. Got a a message saying that ollama command not found. I get this from within the terminal of Visual studio code. But I can run Ollama from the mac's terminal window no problem. Did I forget to do something? Thank you!!!
I am getting the sqlite3 error of using a unspported version, even the link specified does not solve my error can, I get the error when I run the ingest.py file
can I use this to extract pdf information from 100 pdfs? I need the same information extracted from each pdf and store it in rows, so I need a table of 100x (items to extract), i.e. extract Total Assets from a Balance sheet (will be my 1st column) and I need to do this task for 100 companies. can i use this code to do that? I feel that its more like a many to one thing this one, rather than many to many. Thanks so much for your content!
Thanks for your tutorial. I am trying this on Windows pc. I am in anaconda prompt. I am stuck at this error "Error: Could not build wheels for llama-cpp-python, which is required to install pyproject.toml-based project". I tried installing installing visual studio build tools but the error still persists. Any idea?
Interesting idea of integration. But i think that since you have no bugs in this process, it must be automated. Also, ollama is nothing more than click-to-run tool to download and deploy different sorts of AI models, so in fact you don't really need it to run Mistral with PrivateGPT. Or am i wrong?
Thank you for the amazing video! a quick question, are these local llms safe to use with private data? I want to use it for work but idk if the info will be shared with meta.
Hello @PromptEngineer48 , I new to python and want to learn this. I followed your instruction in your great video, but had many errors when installing modules in requirements.txt. Any update?
Inspirational video, great work, subscribed. Could you look into integrating a web gui for requests and responses? I'm looking for a local LLM + Documents + URL for tailored and personalized chatbot knowledge store. I use perplexity right now but want to curate my own documents for tailored content. Keep up the amazing videos!
Really great video! This has saved me a ton of time as I am building an AI model to ingest thousands of documents and query them. Very straightforward. What made you choose mistral as the model to use? What is the largest model that Ollama has? I liked and subscribed! God bless you!
Glad it helped! I choose mistral coz it's small and replies accurately to my use cases. Umm. the largest model I think is ollama.ai/library/megadolphin
Thanks for the great effort. I just have a question. When a file is ingested, then i want to ingest different file, should i delete the db folder first? so not to mix older ingested files with the current one i want to ingest. Also, after ingesting one of my csv files, i asked few questions. Then the answers i was getting were far off and was actually from another source, which i didn't provide, it was from a pdf book about getting rich or something. When i only ingested a csv file with numbers!
@promtEngineer, can you host it with basic UI interface, like people could upload their docs and ask questions, without doing all of this hardwork🙈, and also dont keep anyones personal documents with you
yes. a good prompt. in the begining of the prompt mention that you are a chatbot and answer based on these documents only.... some modifications. but this works for openai api. i have tested in my current project that I am doing on some RAG applications.
I can not update the db, when I ask the agent about the new added document, it still gives answers about the document in this video. It is kind of confusing.
great video! it is very cool. I noticed that you implemented the Ollama on your mac, but can you update this to install in a server or the cloud that can provide this service to your friends? that would be helpful for your friends
You've saved me from going down my RAG rabbit hole. The code is extremely detailed, clean, and easy to understand too. God bless.
Thanks for tuning in.
I've been looking for this for months. Thank you alot, dude! Subscribed :)
Thanks for the sub!
Dude … This is AMAZING! I was just looking for pushes in the right direction, but this actually does exactly what I was attempting to do! Thank you!
Glad I could help!
You're a life SAVER... many thanks
Welcome
@@PromptEngineer48 can i apply this inside streamlit web app so that users ask and get the answer on ui? Can these models be deployed ?
Just found out your channel and subbed after watching this video.
Very good quality video! Keep up the great content creation!
I am impressed with your motivation to answer the questions from your comment section.
Not every youtube channel cares about answering subscriber questions and doing content around what people comment , so very good job!
Thank you. It's my pleasure to be talking with my viewers..
I will like to say that this works just as well even today. Much thanks to the prompt engineer, you have made my life and others a lot easier.
Thanks for watching
this is really high quality content, especially given the effort made in editing. the subtitle is a nice addition also.
Thank You for noticing the efforts... 😍 -- With Love (Prompt Engineer)
However, more than the subtitles, I want the main content to be more engaging.
Good work!
I found your video well done!
Awesome effort!! Your code worked right off the bat. Thanks for saving me a LOT of time.
Thanks
Great tutorial, one of the best on the web!! Thanks for your time and effort! Upvoted 👍
Thanks so much.
Great video, took me a few tries getting Ubuntu configured (using wsl under windows) and updated before it would build the scripts, but I am very impressed. Thanks for all the effort you put into these videos 😁
🤗 welcome. Trying to bring the best.
what is "wsl under windows?" I'm running Linux. No Windows here. His instructions in the video don't work for my system at all.
i use "pip install tqdm ",The system prompts me: ModuleNotFoundError: No module named 'tqdm'
Make sure you have installed all the frameworks listed in the requirements.txt file. Rewind the video, he explains how to do it all at once. pip install -r requirements.txt
If you have done that, make sure you are working in the correct environment. The same environment you installed the python frameworks in. He explains that as well.
I get the same error. Everything installed successfully, and I'm in the same environment ... :0)
Awesome idea. Now use Streamlit or Flask as GUI and the solution is perfect. Thanks for the inspiration. Questions: 1. which OS are you using? 2. python version? 3. do you rather use CPU or GPU? Would an M3 with 128 GB also be sufficient for a quick training / fine tuning of hugging face models up to 20B? Thanks for the answer in advance.
Hp
1. Mac OS. Pretty basic. Macbook Air M2, 8 GB
2. python version 3.9 ++ version
3. M3 with 128 GB.. Hmm when you say fine tuning it depends on the model. But here is a rough calculation. If you have a 20B parameter model in 32 bits, then you need 20x32/8 = 80 GB GPU. your system should be able to do the fine tuning. Else please go for 4 bit quantization, then the requirement will reduce by 8 times, now requiring only 80/8 = 10 GB of GPU.
Followed your instructions here and installed on Raspberry Pi4. Works but of course painfully slow and chip approach 145 degrees F which slows things down as well. But it works and may try on Pi5. Was using a pdf manual for Viking drill press for document. Have to try something with just text.
Hmm. I see.
Are you using the code from private gpt’s primordial version? What a great work! By the way, now they officially support integration with ollama. You probably wanna make a video about how to set this up.
My brain is exploding because I was just using your code repository earlier today and then closed everything out. I'm trying to run it again and I'm now getting a bunch of depreciation LangChain warnings. I still get a response but the response is no longer referencing or citing the documents I placed in the folder. Any idea why this would happen?
my bad. i should have freezed the dependencies.
Fear not. I will create another video with fresh codes.
Thanks for doing this: It seems like when one clones the repository, you need to delete everything in the db folder or it will mess up the results of the information you are trying input yourself.
i get an error = FileNotFoundError: [WinError 2] The system cannot find the file specified: 'db'
hi, ask a question. How can I change the model for example to llama3 with model 8b-instruct-q6_K?? please
go to line 12 of the privateGPT.py file and change the mistral to whatever model your heart desire.
Would like to see ollama with openai assistant or similar that can use a file and have assistants do action to the file such as execute code or clean up data etc.
Got it. Let me do some research.
Can you still use the ollama API to search your documents? or does it need to be via the console?
API can do that
This is awesome, but still not bale to chnage the base url for ollama, is there any way i can change that?
If I give ollama pull mistral
I'm getting error that says could not connect to ollama app, is it running ?
Please give me a solution for it
simple, before typing ollama pull mistral, write ollama serve..
Actually using it in linux platform but still I gave the command ollama serve, I'm getting no GPU detected, Please give solution for this !
@@PromptEngineer48
Thanks, that’s a great video! I’m testing privategpt for some time and I would love to know if you’re experiencing hallucinations from the chat? And yes mistral seems to be a good model but Zephyr and Dolphin seems to give better answers with a little less performance, depending on the context. Therefore, I couldn’t Get rid of some hallucinations, I would say the reliability of information is 45-65%
Yes. Hallucinations are there. Need a good model in future
i am facing thiis error
conda : The term 'conda' is not recognized as the name of a
cmdlet, function, script file, or operable program. Check the
spelling of the name, or if a path was included, verify that the
path is correct and try again.
At line:1 char:1
+ conda create -n private1 python=3.11
+ ~~~~~
+ CategoryInfo : ObjectNotFound: (conda:String) [],
CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException
docs.anaconda.com/free/anaconda/install/windows/
install the anaconda
How do I remove the embedded Think and grow rich pdf file. I say this because when I add other file the query goes back to this pdf file and quotes it
delete the db and cache folders
Thank you, forgive me for sounding slow. You do mean remove or delete the chroma.sqlite3 file?@@PromptEngineer48
If I delete the cache folder name db , will this effect how the program will ingest the files I supply or will create a new db and then ingest the new files?
Great Tutorial, Why is it saying loading 235 new documents when it has only one in the source_documents folder. I had 2 in mine and it said 8, wondering why.
It's because of the chunking.. I had put only one document but it was chunked into many pieces.
Thank you! Very clear
Welcome
Hi, while setting up Virtual environment I'm getting an error, 7:39 what programs do I need? I have installed Miniconda and it's still no running the line. Thks
then try with Ananconda
It would be great if you will create a video where you work on entire project, consisting dozens of files, related to coding adding, editing some feature. Thinking about that, DeepSeek Coder would be a best choice I suppose.
Noted! Will try to implement. Deepseeker Coder is very good actually. Beat so many benchmarks. 😄
@@PromptEngineer48 Also, will it be necessary to use MEMGPT to store different contexts in separate chat windows? I'm bit frustrated trying to figure out bundle of all required to perform what I have mentioned in initial comment =)
MemGPT working solution.. not yet developed. For a big project like youra need some exploration. I think I have to go for contextual sparsity. ..
@@PromptEngineer48 Bing came back with following bundle: Yes, you can use **AutoGen**, **MemGPT**, and **DeepSeek Coder** together to chat with an entire project like a Ruby on Rails (RoR) project consisting of dozens of files. But it got concerns regarding token limit:
"It might be challenging due to the token limit. Each file would need to be processed separately, and the total number of tokens across all files would need to be within the token limit. If the total exceeds the limit, you might need to truncate or otherwise reduce the size of your input1.
Remember, a token in this context is not necessarily a word. It could be as short as one character or as long as one word. For example, “ChatGPT is great!” is encoded into six tokens: [“Chat”, “G”, “PT”, " is", " great", “!”]
Thank you a lot for your effort to create an easier way to get up and running. Could you make a video explaining what all of the parts of "requirements.txt" are needed for? I would like to understand how everything works in detail
Sure thing!
Does anyone get this warning - UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown (and subsequent crash)
I didnt get such
hi, can you update this so we can use LLama 3.1 instead, please provide a version that works with Apple silicon as this one fails on my M3 Mac
Nice video, very informative! What do you use to generate subtitles? thanks
Thanks.. Capcut
This is great, can you add or show how we could add gpu integration even better if it is over a GUI with privategpt 2.0?
Alright
About the context window. I have noticed that it cannot exceed over 2k tokens even though Mistral can support up to 8k. From what I have tested so for, it is like the bot identify itself as a GPT-3, is it because of the openai library?
yes. That is because we start everything compatible with OpenAI API. then we shift to opensource APIs. We could instead work for OpenSource APIs for the start. 😁
@@PromptEngineer48 So for this Ollama, the context windows will not be limited with 2k only right? It will be scale based on the model capability.
Yes.
@@PromptEngineer48 I wished that I could test it right now. Windows user need to wait.
We can try LocalAI. I will come up with a video
Can you tell, how to use other models like llama3, without using mistral?
thank you, sorry but i don't see the old chat interface :-) i have 2 questions
how can i remove a ingested document that i don't need?
ist it possible to chat with the docker interface ? or is it only for terminal??
1. U can remove the file. Then delete the db folder and __pycache__ folder.. then run python ingest.py again and python privateGPT.py again.
2. Web UI interface is not integrated here right now. But that is some ok my pipe line as well. I am working on that.
Keep getting a error when doing python3 ingest.py says line 8 no module named langchain
pip install langchain langchain_community
If my company has a pre-existing Data Dictionary, is there a way to allow Ollama to integrate it so that it better understands the data I am working with?
yes. that could be included technically.
ModuleNotFoundError: No module named 'tqdm'
even after installing the library properly. Any fix ?
Were you about to solve?
@@PromptEngineer48 yeah ..updating the package fixed it for me. Thanks again for this awesome tutorial
@@varun_tech7You solved the problem yourself. Congrats.
@@varun_tech7 which package did you update? thx
@@raminderpalsingh123 I don't exactly remember, just make sure all package is up to date
Great video!! However if i follow the instructions my results are different. I created the source_documents folder an put in another pdf-file. When i then execute the "python3 ingest.py" the ingestion seems to work fine. But when i afterwards exceute the privateGPT.py and start to interact with the llm it still responses to "Think and Grow Rich"-Book.
Delete the db and cache folder
@@PromptEngineer48 can we not append the pdf files, do we have to keep removing them. when a new file is added and ingested, it should add the document into its cache and should be able to response from both the document isnt it
@@arvindelayappan3266 did you ever figure this out?
Well explained. Can you suggest if we want to pass a onenote how we can pass it.
This work very well but it has issues ingesting docx, pptx and ods files without a pip python install
what is the system configuration that you are using and what is the response time for the query
Just a basic 8GB M1
I couldn’t get Conda to work after installing Conda . The installed files are in /opt/miniconda3 but whenever I run Conda, it’s says Command Not Found
www.anaconda.com/download
You need to install anaconda. however, you could use the .venv instead of conda. We just need a virtual environment.
@@PromptEngineer48ok managed to get Conda working. However when I run the python3 ingest.py, I got an error: line6, in from tqdm import tqdm. ModuleNotFoundError: No module named ‘tqdm’
Ok made more progress but now stuck in pymupd. Tried installing it but keep getting message Requirement already satisfied: /opt/anaconda3/envs/privategpt/lib/python3.11/site-packages
pip install tqdm should have worked
finally finally windows version of came, because i don't notice it until i this video and rember it.
yes. bro
Do you have the pdf document you tried? Would like to try it myself
It's the think and grow rich book. Just search for the book on the internet
Could you provide suggestions on how to enhance the conversational capabilities of this bot?
Prompts and proper chains ⛓️
Hello, I am trying to follow your youtube video about "Private Chat with your Documents with Ollama and PrivateGPT" but when I type in my query after running the 'python privateGPT.py' part. I get the following error message "ValueError: Ollama call failed with status code 404. Details: model '7560' not found, try pulling it first". I'm not sure what this is talking about as I do the 'Ollama pull mistral' command before running the rest of the code. Any idea how to fix this?
try this ollama run mistral:latest
@@PromptEngineer48 It let's me run mistral:latest but how will this fix my error above? Do I have to add that to the python code somewhere?
Success!! Thank you!!
Welcome
I am getting answers from out of pdf also.How to restrict that to only pdf specific?
Ingest only the pdf.
@@PromptEngineer48 even after ingesting the PDFs,I am getting answers for some questions that are not in pdf.Is it because of the already trained model?
Hello, i keep getting error ERROR: Could not find a version that satisfies the requirement onnxruntime>=1.14.1 (from chromadb) (from versions: none)
ERROR: No matching distribution found for onnxruntime>=1.14.1
I had the error for ChromaDB too. If you Google for them you will probably find an answer. For ChromaDB this solved it for me:
- sudo apt install python3-dev
- sudo apt-get install build-essential -y
It would be great to feed local AI with project code or framework, and be able to ask it questions about the code.
nice idea. i will try to implement the same.
Hi, thanks for the tutorial. I am able to run the model using Ollama on windows but after everything the project runs and Asks for the Enter a query and if I add the questions it stucks there. In problems I am getting bekow issues such as
Import "langchain.chains" could not be resolvedPylance
Import "langchain.embeddings" could not be resolvedPylance
Import "langchain.callbacks.streaming_stdout" could not be resolvedPylance
etc
Can you please help me out in understanding this issue and resolving it.
Thanks.
There are bugs in windows.. you can try with Linux on windows.
Thank you very much, great video.
You are welcome!
Hi, I did what you said with some hesitation but it worked as you said . This is one good program to use to have a good look at a documents content and meaning. Thanks. BTW is there way to let the program give a full response with out telling it to continue with its explanation?
I think there should be a verbose flag, which you can set to False.
Wonderful, thank you! My dream would be to feed my local model with all my books in epub or pdf format just once and the model will keep a memory of those. Then from there I have so many ideas but not sure if feasible, maybe someone can help? 1) for each book create a mind map of concepts 2) a diagram of how each book is related to the others (citations, same authors, same topic, related concepts) 3) given a question or a topic the system can point me to which book is better to read. Probably impossible at the moment, isn't it?
Wonderful idea. I will dedicate time for a POC
@@PromptEngineer48 Awesome! Another one could be to automatically find the book category. Since the system has already ingested all the books it already knows the discussed topics in each book and from there it can assign each to an ontology of category and subcategories. Like non-fiction/self-help, fiction/novel, non-fiction/self-help/personal growth, and so on. I have other 100 of these ideas, ping me if you need more :D
I would also be interested in something like this if possible as it would be amazing to do this in a repository of books that you know. If you could do this with a non-trivial number of books, say, 100-150 (from Project Gutenberg), it would be a great application of local LLMs.
Thank you very much, all the videos you have made are great! However, when I was building the local llm, I found that there are multiple frameworks, such as privateGPT, localGPT, langchain, etc. Similarly, I found that there are very many choices for the llm as well as the vector database (e.g., hugging face vs. ollama), which gave me a big headache, and I was wondering if you could make a I was wondering if you could make a video that explains your recommendations for each part of the process of building a rag-based personal local document chat llm?
That would be a very good video but very less relevant as everyday we have so many updates. But this is something I can create. thanks for the idea. Definelty will work on that
@@PromptEngineer48 Thank you for your response! It's true that, as you say, the various programs are moving so fast. Since I'm hoping to be able to build something myself via langchain, I'm starting to work based on your github project, and some of the other out-of-the-box projects (e.g. open-webui, privateGPT, etc.) are just too heavy for me to get into and modify.
Cool.
Now we need a webui
Okay. On it
actually, i think it comes with a gradio webui now anyways, i was too quick to respond haha@@PromptEngineer48
I guess there is no requirement of video from my end now.. 🤣
Do it ! , might as well a video with the full package :) but don't have to make it a priority.@@PromptEngineer48
This has been very instructive, thanks!
Is there an LLM that's better than Mistral at working with very technical documents, i.e. lengthy API implementation documents? I set up my environment exactly as instructed, got my docs indexes and could get some answers on them. But things get murky when I go very specific, with incorrect or incomplete answers.
Try with mixtral 7x8b
man, I'm having separate conversations with different models using the UI, that's unbelievable..
Nice.
Outstanding work you did bro. I was able to get mine running following your instructions with no issues. Does this support llama2 as well? My other question, what are the steps to run privategpt ui? Thanks a lot, this is brilliant!
Yes. Llama 2 as well.. PrivateGPT UI I shall have to look.
How would you do this if you have ollama in docker? Or even cloud gpu like runpod? Or Linode? Seems like a good video for the future as you explain,and teach better than a majority of the creators.
Okay. Will create a video soon on the specific use case that you have mentioned.. I mean by hosting on docker and chatting with docs. Thanks for the comment
same here, I'm using dockerized ollama.. would be great to have privategpt with dockerized ollama..
Okay. Got the requirements
Now it's my turn to create that. 😄
Very interesting, easy to follow. I tested this with a music instrument manual, and it wasn't giving accurate answers at all. Any ideas on how to improve this? It's unusable for this type of document. Makes me wonder how accurate it would be with any content. Thank you!
That was pretty old stuff. Please watch the recent videos on my channel.
Is there a limitation on the max number of personal documents that I can upload to PrivateGPT?
No. but then the vectorstore gets confused and not able to get the relevant chunks.
Thank you for this great content! I was able to get this working on my M1 Mac. I was able to run `python ingest.py` and the `python privtagpt.py` commands. However, when I asked it to summarize the document I had uploaded, it referenced the "Think and Grow Rich" document that you showed in the video, rather than the test document I uploaded to the source_documents folder. How do I clear out the embeddings from the "Think and Grow Rich" document, or clear the chroma db database embeddings completely before running the ingest.py command again?
Delete the db and cache folder
@@PromptEngineer48 Thank you! Got it to work exactly as expected! Thank you for all the great content!
Welcome
I got an error when I typed ollama run mistral. Got a a message saying that ollama command not found. I get this from within the terminal of Visual studio code. But I can run Ollama from the mac's terminal window no problem. Did I forget to do something? Thank you!!!
try to change the type of terminal that you are using. i mean zsh or command prompt.
Where the knowledge base is stored? It keeps track of all pdfs already processed?
The are two folders created automatically names db and cache.
I am getting the sqlite3 error of using a unspported version, even the link specified does not solve my error can, I get the error when I run the ingest.py file
Were u able to solve?
Hi, can you create a video on how to clean our own data which is in my CSV file which is best to answer our query very effectively?
Yes I can. but to be more clear.. you want to use a data cleaner llm which will give clean you csv file?
@@PromptEngineer48 Okay pls go ahead
great video!
Thanks
can I use this to extract pdf information from 100 pdfs? I need the same information extracted from each pdf and store it in rows, so I need a table of 100x (items to extract), i.e. extract Total Assets from a Balance sheet (will be my 1st column) and I need to do this task for 100 companies. can i use this code to do that? I feel that its more like a many to one thing this one, rather than many to many. Thanks so much for your content!
If I understand that correctly, that could be hard coded. I think we don't need an LLM here
Thanks for your tutorial. I am trying this on Windows pc. I am in anaconda prompt. I am stuck at this error "Error: Could not build wheels for llama-cpp-python, which is required to install pyproject.toml-based project". I tried installing installing visual studio build tools but the error still persists. Any idea?
I believe we need .NET SDK, .NET Framework and other tools using the build installer
@@PromptEngineer48 all installed but still wont run.
got the same error, did you manage it to run?
Interesting idea of integration. But i think that since you have no bugs in this process, it must be automated. Also, ollama is nothing more than click-to-run tool to download and deploy different sorts of AI models, so in fact you don't really need it to run Mistral with PrivateGPT. Or am i wrong?
Now with more interesting integrations, we can scrape privateGPT itself and use Ollama to code up our projects natively. Yes you are right
Thank you for the amazing video! a quick question, are these local llms safe to use with private data? I want to use it for work but idk if the info will be shared with meta.
U can switch off internet. Safe 🔐 or not, we cannot guarantee, it may so happen that once u connect to internet the data may be transferred.
@@PromptEngineer48 okay, so the advantage of running an llm locally is just to save some $$ from API calls?
Hello @PromptEngineer48 , I new to python and want to learn this. I followed your instruction in your great video, but had many errors when installing modules in requirements.txt. Any update?
why dont i come up with an updated video.. please give me like a week or so.
@@PromptEngineer48 very nice... thank you sooo much....
Inspirational video, great work, subscribed. Could you look into integrating a web gui for requests and responses? I'm looking for a local LLM + Documents + URL for tailored and personalized chatbot knowledge store. I use perplexity right now but want to curate my own documents for tailored content. Keep up the amazing videos!
For this you can use the cat
th-cam.com/video/IsGVSZFaOpE/w-d-xo.html
Amazing!
Thanks!
do you know how to stop Ollama afterwards? it continues to run in the background even after trying to end the process multiple times.
An icon on the top.. appears. Press and quit
Can private GPT be run for a web type interface similar to your previous video ?
Is there anything which is not possible. 😅😀
Really great video! This has saved me a ton of time as I am building an AI model to ingest thousands of documents and query them. Very straightforward.
What made you choose mistral as the model to use? What is the largest model that Ollama has?
I liked and subscribed! God bless you!
Glad it helped! I choose mistral coz it's small and replies accurately to my use cases. Umm. the largest model I think is ollama.ai/library/megadolphin
Thanks for the great effort.
I just have a question. When a file is ingested, then i want to ingest different file, should i delete the db folder first? so not to mix older ingested files with the current one i want to ingest.
Also, after ingesting one of my csv files, i asked few questions. Then the answers i was getting were far off and was actually from another source, which i didn't provide, it was from a pdf book about getting rich or something. When i only ingested a csv file with numbers!
Yes u need to delete the db file first
Are we using poetry here or not? because that part was not there in the video.
Yes.
Is the full code available on the GitHub repository or are their some scripts missing using git ignore?
No. Everything in GitHub repo
Hi, Ollama is running, I input query but nothing receive answer, termial is blank. What should I do?
Try with different LLMs. a smaller version please
Will it recalculate embeddings everytime I add more pdf documents?
yes
@@PromptEngineer48 I mean it should only compute embeddings for the new documents only, not for entire set.
@promtEngineer, can you host it with basic UI interface, like people could upload their docs and ask questions, without doing all of this hardwork🙈, and also dont keep anyones personal documents with you
Ok
Is there a way for links, images and even videos to be parsed into PrivateGPT ???
Not with this setup. But yaa we could make that work
Great video, thanks for sharing it.
Can you create a Video content talking about making work Cheshire cat on ollama ? Must be interesting.
Thanks for the idea! On it.
wow 🙂
Thanks
is there the posibility to take responses only from your documents?
yes. a good prompt. in the begining of the prompt mention that you are a chatbot and answer based on these documents only.... some modifications. but this works for openai api. i have tested in my current project that I am doing on some RAG applications.
I can not update the db, when I ask the agent about the new added document, it still gives answers about the document in this video. It is kind of confusing.
Delete the db and cache folder.. then upload your own document.
Thank you, you are right, and really helpful!@@PromptEngineer48
How to remove old documents, i have a lot of data in the db
Delete the cache and the db folder
@@PromptEngineer48 thank you , and thanks for the guide. Really easy to follow along.
How does it knows it should use mistral if I have several models downloaded?
that is hardcoded
@@PromptEngineer48 actually not. This written in config. Take a look
is there a way to view the actual embeddings value from ChromaDB?
Sure it is
@@PromptEngineer48 How exactly can I view them ?
I want to try to give it some ability to troubleshoot for me, like conduct pings and traceroutes.
looks good
Hey! Are you able to show how to use auto-mem-local using Ollama? Thanks!
Noted !!!
great video! it is very cool. I noticed that you implemented the Ollama on your mac, but can you update this to install in a server or the cloud that can provide this service to your friends? that would be helpful for your friends
Yes you can! I will try to bring in a video
can you help me? after the python ingest.py i receive a error.. :-(
please mention the error !!
i try to copy here the error but it does't accept @@PromptEngineer48
i have a new Mac with chip M2
the problem is the hnswlib library
in Mac you should do python3 instead of python.. so please type python3 ingest.py
great video. now can u do it in streamlit so there's user interface to chat and how can we host it online? like render?
Okay. Streamlit and render integration got it. Will do that.