Talk to YOUR DATA without OpenAI APIs: LangChain
ฝัง
- เผยแพร่เมื่อ 1 ส.ค. 2024
- In this video, I will show you how to interact with your data using LangChain without the need for OpenAI apis, for absolutely free. We will be making use of huggingface hub embeddings for transforming our documents in to vector representation (embeddings). For Large Language Models, we will again using open-sourced models instead of Openai models (text-davinci-033, chatgpt etc.). All the step will be performed with FREE & open source tools with LangChain.
▬▬▬▬▬▬▬▬▬▬▬▬▬▬ CONNECT ▬▬▬▬▬▬▬▬▬▬▬
☕ Buy me a Coffee: ko-fi.com/promptengineering
|🔴 Support my work on Patreon: Patreon.com/PromptEngineering
🦾 Discord: / discord
▶️️ Subscribe: www.youtube.com/@engineerprom...
📧 Business Contact: engineerprompt@gmail.com
💼Consulting: calendly.com/engineerprompt/c...
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
Links:
Google Notebook: colab.research.google.com/dri...
LangChain: python.langchain.com/en/lates...
-------------------------------------------------
☕ Buy me a Coffee: ko-fi.com/promptengineering
Join the Patreon: patreon.com/PromptEngineering
-------------------------------------------------
All Interesting Videos:
Everything LangChain: • LangChain
Everything LLM: • Large Language Models
Everything Midjourney: • MidJourney Tutorials
AI Image Generation: • AI Image Generation Tu... - วิทยาศาสตร์และเทคโนโลยี
Want to connect?
💼Consulting: calendly.com/engineerprompt/consulting-call
🦾 Discord: discord.com/invite/t4eYQRUcXB
☕ Buy me a Coffee: ko-fi.com/promptengineering
|🔴 Join Patreon: Patreon.com/PromptEngineering
▶ Subscribe: www.youtube.com/@engineerprompt?sub_confirmation=1
Exactly what I was looking for...Langchain and HuggingFace... I do appreciate
Mi Mitch, I have a few questions that maybe you know the answers to? you look like you know a lot more than me about LLMs
Very cool, thank you so much Prompt Engineering, you've earned another subscriber!
Great video, was looking for this about using open source llm!
I tried to reproduce your script and I get this error : "Error raised by inference API: Model google/flan-t5-xl time out", can you please give us an example with model stored locally on PC hard disk?
Thank you very much small version of the model you said worked well with the system
This is wonderful, you should make a video on using no API’s (hosting the model locally) with vicuna or openassistant and langchain using web search, code execution, pdf searching and more! Thank you once again!
Will do!
Yes! I’d love to set something up that even approaches how Bing can do on-demand lookup and summary
That would be amazing! Definitely do!
So that means i can use the functionality of the A.I without using the API ?? What does it mean to host a model locally?? I'm still a beginner so all of this is new to me but it sounds really exciting for some reason
@@aeroartz4080 It means you're not using up the cash charged for utilizing their API, by utilizing the memory or disk on your own computer of what's available. If you have 8GB of RAM, this is not something you will be able to do really. You can convert down to 8bit for some smaller uses with less power, but may not be able to do what you want it to do.
Great Thanks! Another great idea would be to discuss using FOSS Vector stores options. Basically end-to-end FOSS implementation, I'm sure there would be great interest and more importantly is badly needed since OpenAi, Pinecone , are all closed and proprietary.
thanks for the suggestion, I was thinking about making an end to end implementation with pinecone but will also look into it.
I vote for the end to end FOSS Implementation. That would be brilliant!
I have been looking into Milvus and Weaviate for the vector database. OpenAssistant is also looking awesome.
@@heinzgericke I think this is where we're all at right now. Both Milvus and Weaviate do their jobs well, but it all feels half-baked (understandably) when you try to use langchain to implement self-hosted models, vectors, and embeddings.
@@engineerprompt FYI Pinecone, as of this weekend, have stopped new free accounts from using pinecone and started *forcing* people to sign up for paid accounts.
@@heinzgericke yeah, OpenAssistant is shockingly good, very impressive! PS If you're interested in partnering to develop FOSS AI stuff pls let me know,would love to work with passionate partners, i eat , breathe, sleep this stuff. Pls DM
will it query combining among multiple pdfs in this approach? For example, if I have couple of papers and ask about list all the authors from these paper, will this be able to generate from all paper or just find the most relevant one?
SORRY for the BAD sound quality. Not sure what happened here.
sounds perfectly clear to me. this is great infromation btw
Where is the exe file for this portable version that you open this AI?
can you explain how is this working like to a 5 year old? LOL
@@amandamate9117 😀
@@ShaneHolloman Thank you
Great video, going to have endless applications, thank you for sharing. Have you looked into OpenAssistant official release? A video on a full frotend and backend local deployment would be awesome.
Yes, I made a video on it before the official release. Need to look at it again. Just too many things to keep track of :)
If you liked this, you should check out this crash course on LangChain: th-cam.com/video/5-fc4Tlgmro/w-d-xo.html
You are always making amazing videos. I have not seen any videos on TH-cam on how to create an easy interactive mapmind or orgno gram by uploading CSV or excel and then share it with your manager. İf these is anything like that please let me know. I really need that.
💯 ⭐⭐⭐⭐⭐💯 Another Excellent video! Thanks for taking the time to share your knowledge and doing it in such a clear, concise, and timely manner. Really appreciate your effort, it's helping me learn.
chatgpt api is not free, so this is a great contribution
That's insane. Can you make a video on how to use gpt_index, langchain with openai api to train your bot with your own data. Make the bot multimodal (both text and voice option) and also make an interface for the bot using python(flask or django) or react.
This is a great video. Goos work
if I want this setup working in a closed env (without internet / API etc), how do I do it. I tried privateGPT, it just gave me parsed statements, no interactions. Like it looks for keywords from my prompts and looks for similar keywords in the truncated text and shares as if it is copy pasting the info
Hi, I tried running your code in Google colab. How long is the typical wait time to get an answer? Is it really long / slow?
very good tutorial. keep doing more of such FOSS RAG videos
Just sharing, I encountered some problems during testing. When doing through pdf part, it can through in an error that poppler and tesseract not installed. You can do !apt-get install poppler-utils tesseract-ocr do get through this problem.
Excellent Video. I have had challenges querying SQL Database with Langchain without OpenAI. Would love if you are able to do a video on it
Hello, are you able to connect the database to LLM model?
That's a very informative content. How can i interact with my CSV files without using OpenAI API?
Does this only work with english, or can you modify the language ?
Hello, Great thanks! Can I connect my database to the hugging face model?.... if it is possible .. plz make a video for this ... THANK YOU
Thank you for providing such a professional video. Due to the large number of flan-t5-xl and timeout reading, I would like to know how to download and read it locally?
Check this out: th-cam.com/video/Xxxuw4_iCzw/w-d-xo.html
Thank you so much 😍
Thank you!
MY GOD I NEEDED THIS. INSTANT SUB XD
Thanks for the video. I am getting the following error.
NotImplementedError: text/html output type is not implemented yet. You can pass `raw_response=True` to get the raw `Response` object and parse the output by yourself.
Could you please throw some light on this. Thanks
The program gives no response when I change the query... For instance when i change it to query = "What did the president say about troops" the programs hangs.
Where is the problem?
If I have the resources, can I fine-tune my own model from an opensource model and do this thing with my fine-tuned model? Does langchain support that?
Are embeddings apart from OpenAI safe and secure? We are looking to build a AI chatbot for a FinTech company. I was curious to know which is the best secure tech to use ?!
Dears, I created same but response is quite slow when the context is passed to LLM I am using metallama 7b chat llm. Any idea how to speed up? Even in small context It takes around 10s. In your video it's quite faster.
Did langchain remove instructions for how to use outsource embedings from their website? I looking for it but can't see
What if you want this to connect to database and query database using natural language example: "who are the top ten users this week"
when I do splitting my length of doc is 1. Is there full code somewhere? Did anyone faced with similar issue. I think in the meantime also url of text document is changed.
Hello, can you maybie do a video about a uncensored text to image generator that is downloadable for your own pc? Because things like midjourney get more and more restrictive.
Hi is there any ways to load up a locally installed LLM from my computer ? Using GPU ? Thanks !!
Hi there! Your content is great and I think you're amazing. While watching this video, I noticed that pinecone was not used in the process. I'm curious to know what the pinecone was intended for in the original GitHub Langchain repository, and whether it was not necessary for this particular case or if other methods were used instead. Thanks.
Thanks for nice comment. As for pinecone, its simply a vector store (think of it as database). Since I was showing a demo for small number of files, you don't really need pinecone for that but if you have large number of files and want to store the embeddings for them, pinecone is one of the options out of many others. Hope this helps.
@@engineerprompt Thanks for your kind reply! I have done some search myself for the last couple hours and kinda figure out what pinecone is. Then I was wondering if I can just store the vector data, in this case, Faiss index file or sort of things, in my local machine, and load it every time I run this program. It will be convenient for large-scale file processes but I am still trying dabbing on my code to make it work :)
@@engineerprompt One more thing...I wonder if you could help me with this. I tried your collab code hours ago and ran it successfully on my own machine. I tried to switch the model from hugging face to my own openai api and I think it works. It does process small files perfectly, but when I process a large file, say a 100 thousand character txt file, pycharm gave me this error: "openai.error.InvalidRequestError: This model's maximum context length is 4097 tokens. However, your messages resulted in 660677 tokens. Please reduce the length of the messages."
I think it might be some parameters that I didn't set right. I am still searching the langchain documents but so far got nothing.
Hey, thanks for making video on this. I implemented your code for multiple pdfs. But upon query, it gives me an error of 'google model timed out'. Any idea about this? I have tried increasing chunk size as pdfs were large size.. But no change in error..
Probably the api is timing out. Restart your notebook a few times, that should work.
Hello when i try to use the huggingface embeddings i get repository not found error. Any idea how i can resolve this?
Is Hugging Face Flan-T5 the best option in terms of quality amongst all the open source models? Is there another one I can try that is worthwhile looking into?
its actually a relatively old one. Look into GPT4all.
Hi !! I was trying to follow along with your video but i have the following error . I madee some changes though . i am trying to train it with a pdf of 70 pages.
Error raised by inference API: Input validation error: `inputs` tokens + `max_new_tokens` must be
How can you build a stand-alone LangChain LLM application for windows using python? I haven't programmed for years, so a bit out of touch with current computer software tools and trends, but have been following AI since 1977. Thanks!
Which offline embedding do you recommend?
this is what I searching for a week, can't wait
me too, i found solution file like GPT4All with langchain and chromadb and vectorsearch, but these are very very slow
@@taimoorneutron2940 slow due to CPU design, a workaround is to still embedd via hugging face or openai, store a file, then query. unfortunately querying is slow too - that fine-tuning staff is something I would like to see from this video!
Just to keep the expectation real, no fine tuning in this video, that's coming soon though :-)
@@engineerprompt Yeah I know, and I appreciate your work. This topic of tuning is something we all still cooking for
@@bakistas20 I found a tool that's very simple to use but need to test it to make sure it works 🙂
Great tutorial. What would be your approach to creating fully autonomous agents that do different things like searching the web, creating files, writing code etc?
Thank you. AutoGPT :-)
@@engineerprompt AutoGPT doesnt really do anything of value ATM. Too many wasted tokens on useless thought loops.
How do I add a chat history here as a reference for the bot's answers?
Very cool video and thanks a lot for sharing the Norebook sourcecode.
If I wanter to work with text/markdown files (instead od pdf) files, what would the code look like ? Ie replace UnstructuredPDFLoader with ?
UnstructuredMarkdownLoader
@@cashmoneybanks8442 Awesome !!! Thanks a lot 🙏
Now this I can use
thank you for the video, indeed I have 1 question : as I want to train a model for my company and could not find the right team of dev to do it yet, my question is : can this method be applied to retrieve information from a very long data set ? (the whole text of law of a country for example) . We intend to add this as an internal tool in my company to increase productivity. Plz any advice link or answer is welcome
Yes, you can use similar approach for that. You probably want to try more powerful embeddings/models. Email me if you want to discuss further.
@@engineerprompt i do have the similar use case for my company as explained by vulcan, but using the API key's will expose my data (Which is not be exposed as per the company policy).
is there any other way to create a chatgpt like bot to answer questions using natural language.
@@satishkumar-ir9wy Yes, now you can even use Llama embeddings with these models. You can use GPT4all with it to create more coherent responses.
Do you know the price and limitations to use this approach in a production environment?
You probably want to use newer models. In this example, I used T5 model which is relatively old (by today's standard). For better results, the best option is openai which will charge your per 1K tokens. Prices vary depending on model you use.
Thank you for the video. What GPU did you use?
I am running this on 1080Ti
What is the name of the software u r using for coding in this video?
I tired this but i had a probleme : when i ask questions the execution got stuck in endless loop and it gives me a time out at the end. Any solution please ?
I have the same problem. Did you find a solution ?
can this be done on Oogabooga WebUI? I'm trying to create D&D module making/running asistent bot and to do that I need Oogabooga WebUI to be able to read the pdf files
I haven't explored that much so can't say something yet.
how to increase the max length of response to our questions . Please help
Question: I have a bunch (tens of thousands) of sensitive legal/medical documents that I don’t want to expose to a third party, so I don’t think I can use OpenAI. Is there a way to use this “locally” so I can query my own date store of PDFs if I want to know “who has cancer” or “which claims were denied?”
Yes, you can do that with local models. Reach out to me if you nee help with it.
@@engineerprompt yes, I have the same issue. could you please provide some direction on how to go forward with this.
@@aldrinjenson Hey! I am also working on similar problem statement. Please help regarding this!
thanks a lot for this informative video!!!
I'm trying to follow the same method that you showed in this video, but sometimes my model answers out of the given text, do you have any idea on how can I solve it? I tried to play with the prompt but didn't help too much...
is there any way to guarantee never get any answer out of the given text?
My recommendation will be to use a prompt template for this case. the template will be something like this:
template = "
Answer the question based on the context below. If the question can not be answered with the information provided, say 'I do not know' "
and then you can add your question as part of the template. If you need further clarification, there is a video on crash course for langchain on my channel, check that out. Hope this helps.
@@engineerprompt thanks a million! I'm going to check it out
best!
Without OpenAi the similarity search works just like text in text out. It can not handle contextual questions. Are there any suggestions on how that can be implemented?
That's not totally correct. The embedding based approach is able to find the text within your document which are relevant to your query. Same operation happen whether you are using OpenAI's embedding or any of the open-source embeddings. For the next stage when you use LLM to create an answer based on the retrieved information as context, the process remains the same whether you are using OpenAI's LLM or any other. That being said, OpenAI's model are currently the best option for both operations. Hope that helps.
Your proposed solution is very interesting. Will it work in PythonAnywhere?
Yes, It should work. I might make a tutorial on it :)
I think the purpurs of the video is good, but many concepts and the goal isnt explained well enough. Things like how accurate the answers will be isn’t touched upon either
Will this work on enterprise data and not text document, say have a credticard data of million rows and customer data, without sending data to anywhere openai or pinecone or anywhere, can we use llm models and get good insights asked
Yes, that's possible. I have videos planned on the topic.
Can I only send data and ask questions in English? It does not work with data in Spanish?
I am not sure, but give it a shot :)
This is a amazing video i want to try it on a collection of specific topic pdfs. Id want to make it easier and try it with llama 2 langchain
Check out my localgpt project, already doing that :)
would you have a link to it I will see if I can fallow it chatpdf only allows like 5 before it shows a pay wall surprise.@@engineerprompt
How can I interact with this over an API?
Thanks for the video, I also tried to run on Google Colab with my small data but it raised error "ValueError: Error raised by inference API: Model google/flan-t5-xl time out"
I don't know if it supports English only or unicode? Please help. Thanks
Replace the flan-t5-xl with flan-t5-small
it is much better google/flan-t5-xxl
@@alejandromeadezorrilla442 Yes but probably free google colab will not work.
@@engineerprompt Hi I tested using the free version of google colab, the flan-t5-xxl only need 11 gb of RAM and Google Colab give you 12.3 gB of ram and 15 VRAM, always that exist available resources.
how can I apply it for csv file
What’s currently the best way to combine these technologies completely locally?
Use something like langchain and look at models like GPT NeoX, FLAN-T5 etc.
Your content makes learning both enjoyable and informative. Great work! #nurserytovarsity
Would prefer to have smth like that but for Excel.
one issue I noticed when I ask questions it just has an endless loop it cant answer different questions ...it does not say it does not know so it just gets stuck
The example including the colab does not work outside of those two questions you provided...
Yes, the huggingface hub integration appears to be broken. The two questions already in the colab have their answers cached, so they still return results rather quickly. But if you ask any other question, it gives a timeout error.
@@cashmoneybanks8442 lolol i spent like 6 hours on trouble shooting this lol i thought as much be was not confident enough to say since i have never used higgingface
@@FunwithBlender Yeah I was scratching my head for a while because one minute it was working for me, and the next I was getting timeouts no matter what huggingface model I used. Not sure if it's a problem with langchain or huggingface, but I hope it gets resolved soon
Nice video but receiving the error "ModuleNotFoundError: No module named 'sentence_transformers' " when loading HuggingFaceEmbeddings
Make sure you run the block containing
!pip install sentence_transformers
Seems like its not installed for you.
the url is not working...
that pdf does not seem to exist, 404 error
any other llms other than google t5? as google t5 shows time out error
Try this t5 small version. That will work on the free version of google colab.
Thanks
Thank you 😊
What are some use cases for this?
You can use it for reach or make chatbots for your own business.
what if you have multiple PDF files that host the data?
Check out my channel, there is a video on it :)
Actually you don't get longer responses if you increase the maximum length from 512. That's the best it gets, not more than that.
You are right, that's the max length for the selected model.
Hi, Has anybody implemented this video practically?
Hi, I am getting time out error
How it works in spanish?
hi its very good.
can i give him a python package github url or documentation page url and then ask him to write code according to package or i use that to learn the package
This is for information retrieval. You can simply ask it to get you something from the repo, it will not be able to produce new information. Hope this helps.
are there any limitations using this method compared to using openai api?
the responses are not as good as using openai api.
@@engineerprompt thanks for the answer!
Any chance to do it for JavaScript too ?
I am not familiar with JS but Langchain has support for it.
i hope this is not pinecone based but something like weaviate
Does hugging face embedding not cost anything, if so then why do you need HF API key?
It does not cost anything. You need the API key to access the model from HF hub. They are hosting the models. If you download them locally, you wouldn't need the API key.
@@engineerprompt using API key would run the models on hugging face hub or does it mean after running it once using API key the models are stored locally? Because the if the video mentioned that I could not see it.
@@jawadmansoor6064 If you are using the huggingfacehub the models will run in the cloud using your api key. To run them locally, you will need to download the models. I didn't cover that in this video but will make another video soon on this topic. Too many things are happening 🙂
@@engineerprompt please do make another video for embedding pdf, docx and txt documents for models hosted locally. Tq so much for your effort. Add the folder document bit too, if it's not too much to ask😊
@@engineerprompt Eagerly waiting for the next video. Also, please try with all the available options (locally running models) see what gives the best result, is vicuna really the best choice among them?
how much it costs using huggingface?
$0
sentence-transformers import error.Is anyone facing the same issue?
Use an older version of sentence transformer. I think 22 will work
Q: How was the GPT4all model trained?
A: LoRA
Understandable, have a nice day.
Haha, I'm only guessing it was very precise about its answer because the temp was set to 0?
Will this help with the creation of blimps (helium based) - used to clean up the air (giant vortex fans & filters)... we can't afford to have all these toxic fumes floating around, nor the volcanic ash blocking out the sun...
deleted
can you explain how is this working like to a 5 year old? LOL
>needs a third party account
lame, I'll just stay a luddite.
Without an OpenAI account but WITH a huggingface account 🤦
The difference is that huggingface is free
@@cashmoneybanks8442 ok. But until when? ;)
@@franklafortune1345 Considering it is THE hub for opensource collaborative machine learning, pretty sure it will always be free
@@cashmoneybanks8442 🤞
I get this error on some of my prompts:
ValueError: Error raised by inference API: The model google/flan-t5-xl is too large to be loaded automatically (11GB > 10GB)
SAME HOW CAN WE SOLVE THIS ERROR
https...all-mpnet-base-v2 says :{"error":"Repository not found"}