My sincere thanks to you for making such an insightful video, sir It would be great if you could make a video on how to create a custom LLM model. Currently, we are using another company's LLM, so I want to create a new one and publish it to the public or Huggingface. love from Gujarat
please check a playlist link in the video description (I have 3 end to end projects already on LLM in education sector, finance and this one is on retail)
For Big Database again inform queries by Prompt(like train LLM again) is huge job . May other way like Fine Tuning Model on Specific domain or Retrain it.
can we ask context based response? eg : Q1 . "how many XS size thsirt are there?" ans 50, Q2. "and S size?" LLM will understand the context of Q2 which is 'SiZE'
Is this free slot prompting scalable in production environments? There can be so many tables, do many different kinds of possible queries. We can not keep asking queries to prompts. Also these few shots examples would need to be part of the LLM context will be costly.
I want to make a report generating software using LLM on my ERP where nearly 200 tables are exists. In my report generating software I give a propmt and LLM will analysis this promps and make a sql query on the basis of my ERP databases. Can I do it using this procedure?
I'm able to run it on jupyter notebook but pycharm throws this error for the wrapper: validation error for SQLDatabaseChain database instance of SQLDatabase expected (type=type_error.arbitrary_type; expected_arbitrary_type=SQLDatabase)
31:53 Facing error at vectorstore = Chroma.from_texts(to_vectorize, embeddings, metadatas=few_shots) ValueError: Expected EmbeddingFunction.__call__ to have the following signature: odict_keys(['self', 'input']), got odict_keys(['self', 'args', 'kwargs'])
Another superb video. Can we build "End to End LLM Project Using Langchain, OpenAI in Finance Domain" which you shown 1 month ago with Google PaLM? I am asking because as you said google PaLM is free. So those who doesn't have OPENAI paid account then they can practice it. If possible then please make a video on it. Thanks in advance
Hey yes for that finance project you can use Google palm instead of open ai. you can get it done with few lines of code change. I am not sure if I will have that to make a separate video on that. consider this as an exercise for yourself 😃
@@codebasics Thank You very much for your kind support. You are a real gem of this industry. You are putting your great effort for India's bright future.
Hi , this video was super helpful for my work but can we retieve the SQL query without running it, actually I want only the SQL Query. I tried with logger but failed. Any good idea would help a lot.
I'm facing an issue with sentence-transformer. Even after installation when it comes to HuggingFaceEmbeddings, it is throwing error install no module named torch._c and also says to install sentence-transformers. Python, torch versions are matching though.
Facing error at vectorstore = Chroma.from_texts(to_vectorize, embeddings, metadatas=few_shots) ValueError: Expected EmbeddingFunction._call_ to have the following signature: odict_keys(['self', 'input']), got odict_keys(['self', 'args', 'kwargs'])
Re-check this things: 1. from langchain.vectorstores import Chroma this is correct 2. formate of your few_shots dict it correct. 3. to_vectorize = [" ".join(example.values()) for example in few_shots] This is correct. Might solves the issue
In production answers might change every day, it will be not possible to change answers every day right? And what about if I need data in table format, It will only answer in one single word, if we need data of 2 or more columns, so what would be the approach for it?
Hi, how I will decide that which GPT version will suitable for my project as below code? Please send any guideline/rule regarding this. Thank you. chatllm=ChatOpenAI(openai_api_key=os.environ["OPEN_API_KEY"],temperature=0.6,model='gpt-3.5-turbo')
sir how to give table and column name description to llm since most of the time the names are assigned based on domain knowledge so how to give custom description
I am trying to join the list but i am getting error like this "sequence item 3: expected str instance, dict found" how can i solve this error please help me to comeout from this
I have no science background in my school level and i completed my post graduation.is it eassy to shift data science filed.. If i can learn data science certificate programm.. If data science short term course are available in data science filed
Sir you are using a GPT api key internally how they work and what is providing in the code when you use the chain function then you are not going to encode query. And you pass LLM = LLM means what??
@@codebasics as the result in the browser I get this output "Question: what is the quantity of red tshirt? SQLQuery: SELECT sum(stock_quantity) as total_quantity FROM t_shirts WHERE color = 'Red'" not get output like you
@ 14:33 while executing this db_chain = SQLDatabaseChain.from_llm(llm, db, verbose=True) i am facing an error ValidationError: 1 validation error for SQLDatabaseChain database instance of SQLDatabase expected (type=type_error.arbitrary_type; expected_arbitrary_type=SQLDatabase) please someone help me
what if the quantity of t-shirt decreases which means someone buys after some time passed, do we need to change few_shots list's values and rerun the whole process? or llm figure out by its own?
Error of this line :--to_vectorize = [" ".join(example.values()) for example in few_shots] This is the error :---------------------------------------------------------------------------- TypeError Traceback (most recent call last) Cell In[31], line 1 ----> 1 to_vectorize = [" ".join(example.values()) for example in few_shots] Cell In[31], line 1 ----> 1 to_vectorize = [" ".join(example.values()) for example in few_shots] TypeError: sequence item 3: expected str instance, dict found If I change above line with this to_vectorize = [{" ".join(str(value) for value in example.values() if isinstance(value, str))} for example in few_shots] this is working fine but in next line vectorstore = Chroma.from_texts(to_vectorize, embeddings, metadatas=few_shots) it is giving me this error 86 texts = list(map(lambda x: x.replace(" ", " "), texts)) 87 if self.multi_process: 88 pool = self.client.start_multi_process_pool() AttributeError: 'set' object has no attribute 'replace'
I resolved it using the following changes:- in few_shots, make sure to take only the results from the qns1, qns2, qns3, qns4, qns5 . i;e., qns4['result'] in-place of qns4 and so on. and same for others.( if qns is alread a string value and not dictionary, then leave it.)
OSError: [WinError 126] The specified module could not be found. Error loading "C:\Users\Moin\anaconda3\Lib\site-packages\torch\lib\fbgemm.dll" or one of its dependencies. i m getting this error, not able to tackle
Check out our premium machine learning course with 2 Industry projects: codebasics.io/courses/machine-learning-for-data-science-beginners-to-advanced
Bro first line of code hi run nhi ho rha h pleade help kr do mera final year project h mera 😢😢😢
@@ritikkumarsah7 did you figure out how to do this bro
@@SatwikVarmaKunaparaju Yes. Some say API service is closed and some say you have to pay
This is one of the best tutorial I watched so far, it is comprehensive with perfect example. feeling lucky after going though this.
I am watching videos on Generative AI from last 7-8 days and this one is so easy to understand. Thanks
Bro first line of code hi run nhi ho rha h pleade help kr do mera final year project h mera 😢😢😢
Thank you uploading such detailed project videos. Learining newer tech like langchain becomes easier with such tutorials.
Bro first line of code hi run nhi ho rha h pleade help kr do mera final year project h mera 😢😢😢
Thank you sir, I took this project as base for my major project in college and it got selected , you helped me in my majors thanks sir thanks alot.
Your body language is excellent, folk! That makes the topic to understand easily!
You really are a fantastic teacher! Thank you for the time and effort that you have put in to create this outstanding video.
I binged on the playlist, how come this does not have millions of views?
Its such an amazing video. You explained such a complex topic in a simple language and the demo alongside nailed it. !! Hats off to you!
Thanks you Sir. you make really best videos with clear concepts. please make more projects it will really help
U always explain very easily😊
hats off , took me a while to understand fewshotprompttemplate
Dear sir, thank you very much for great tutorial! Keep it going on!
Amazing tutorial. really helpful. Thanks Dhaval Patel sir.
Thanks for the video. Concepts and flow explaination is really nice. Its taking some time to digest all of this.
Excellent video Dhaval Sir. 👌👏🏻👏🏻
This is goldmine of information!!
What if we change database every day like we insert or delete? This will work?
@@loading757Why will you change DB everyday?
@@2dapoint424 i mean if we alter, insert delete or make changes in db?
@@2dapoint424 😂😂😂😂
Bro first line of code hi run nhi ho rha h pleade help kr do mera final year project h mera 😢😢😢
Excellent !!! Can you make a video on (1) How to deploy custom LLM applications to Production (2) How to Evaluate LLM models
Amazing project
My sincere thanks to you for making such an insightful video, sir
It would be great if you could make a video on how to create a custom LLM model. Currently, we are using another company's LLM, so I want to create a new one and publish it to the public or Huggingface.
love from Gujarat
GREAT project!!!!!!!! Waiting for more LangChain Projects
please check a playlist link in the video description (I have 3 end to end projects already on LLM in education sector, finance and this one is on retail)
How humble dhaval sir who keeps us motivated
Thank you, Happy Learning.
@@codebasicsbut sir if we need answers of multiple coloumn what will we do? And database changes everyday, is it incorporate with our model?
I am not able to run, please help me, this is my final year project 😢😢😢 If anyone can help me, I am ready for paying.
@@ritikkumarsah7 ping me in hindi saat nau saat saat paanch saat ek do aat che
Please make a video from database to deployment...❤
i am getting error .. while importing file from .SQL (mySQL ) to MS SQL SERVER
Superb
Thanks for this tutorial !
For Big Database again inform queries by Prompt(like train LLM again) is huge job . May other way like Fine Tuning Model on Specific domain or Retrain it.
Thank you for this video! I was wondering if there is a way to do this with Gemini 1.5 Flash or Gemini 1.5 Pro
Nice teaching ❤
Hi Dhaval Please make a video on How to Evaluate LLM models? Which metrics need to use?
Well explained
This seems like a really good video, I'll be watching this later today. Wondering if you have a project where Databricks is used as data source?
Amazing video💪
Thank you 💯❤
Great 🎉
Thank you !
Great video Dhaval❤
Glad you liked it!
can we ask context based response? eg : Q1 . "how many XS size thsirt are there?" ans 50, Q2. "and S size?" LLM will understand the context of Q2 which is 'SiZE'
Hey, Great Video ... How can we do this for Question Answering LLM Task.
Very nice use case! Quick question though, we don't we create a dashboard out of this DBMS?
can I use custom databases which can be uploaded by the user?
❤❤❤
How can I do it if I have elastic search instead of mysql database?
Will this update the current data or we only deal with the historic data??
Please make a video on prompt engineering
which google-generativeai version have you got installed? i am getting tons of errors due to version mismatch...
Same here bro
I am not able to run, please help me, this is my final year project 😢😢😢 If anyone can help me, I am ready for paying.
Hii sir, 2nd year cse student sir, certification ka bhi workshop rakhiye if possible please
Can I perform the same with MongoDB (No SQL) too? Does langchain support that @codebasics?
Is this free slot prompting scalable in production environments? There can be so many tables, do many different kinds of possible queries. We can not keep asking queries to prompts. Also these few shots examples would need to be part of the LLM context will be costly.
I want to make a report generating software using LLM on my ERP where nearly 200 tables are exists. In my report generating software I give a propmt and LLM will analysis this promps and make a sql query on the basis of my ERP databases. Can I do it using this procedure?
Hi sir good afternoon
Can you please create a video using llama3 for getting answers from a pdf document
I'm able to run it on jupyter notebook but pycharm throws this error for the wrapper:
validation error for SQLDatabaseChain
database
instance of SQLDatabase expected (type=type_error.arbitrary_type; expected_arbitrary_type=SQLDatabase)
i got this error too, if you find a solution can you tell me how you managed? thank you in advance
Could you resolve the error?
@shreyasreedhar1225
@user-du4kl4by3y
@vms7597
Could you resolve the error?
I am not able to run, please help me, this is my final year project 😢😢😢 If anyone can help me, I am ready for paying.
module 'google.generativeai' has no attribute 'generate_text'
please how can i fix it?????????
31:53
Facing error at
vectorstore = Chroma.from_texts(to_vectorize, embeddings, metadatas=few_shots)
ValueError: Expected EmbeddingFunction.__call__ to have the following signature: odict_keys(['self', 'input']), got odict_keys(['self', 'args', 'kwargs'])
pip install chromadb==0.4.15
@@rohaanmanzoor3268even after installation am also facing the same error
still getting same error. did u fix it?
@@nandanab525 hey ! did you fix it? I'm stuck here.
heres what I did to resolve
!pip install -U langchain chromadb langchain-community
then restarted kernel and all set
can you make a video on finetuning llm for cyber security.
Is it possible to use hugging face llm model api key too
Another superb video. Can we build "End to End LLM Project Using Langchain, OpenAI in Finance Domain" which you shown 1 month ago with Google PaLM?
I am asking because as you said google PaLM is free. So those who doesn't have OPENAI paid account then they can practice it.
If possible then please make a video on it.
Thanks in advance
Hey yes for that finance project you can use Google palm instead of open ai. you can get it done with few lines of code change. I am not sure if I will have that to make a separate video on that. consider this as an exercise for yourself 😃
@@codebasics Thank You very much for your kind support. You are a real gem of this industry. You are putting your great effort for India's bright future.
Did you figured it how to use Google PaLM instead of OpenAI
I have been working on it for a long time but getting error at embedding
@@codebasicssir can this be connected sql server db?? .. I am using quite long
Hi , this video was super helpful for my work but can we retieve the SQL query without running it, actually I want only the SQL Query. I tried with logger but failed. Any good idea would help a lot.
I am getting issue with langchain libraries , pls advice python version and import lib versions -- Thanks
Dont trust this guy. You will stuck on midway 😢😢
I'm facing an issue with sentence-transformer. Even after installation when it comes to HuggingFaceEmbeddings, it is throwing error install no module named torch._c and also says to install sentence-transformers. Python, torch versions are matching though.
Hey bro run hua kya.. ye mera final year project hai. Run ho rha hai help kr do please i am ready for paying 😢😢😢
facing problems in embeddings = HuggingFaceEmbeddings(model_name='sentence-transformers/all-MiniLM-L6-v2')
did u find its solution
Can we use HuggingFace InferenceClient?
Obviously
Facing error at
vectorstore = Chroma.from_texts(to_vectorize, embeddings, metadatas=few_shots)
ValueError: Expected EmbeddingFunction._call_ to have the following signature: odict_keys(['self', 'input']), got odict_keys(['self', 'args', 'kwargs'])
Re-check this things:
1. from langchain.vectorstores import Chroma this is correct
2. formate of your few_shots dict it correct.
3. to_vectorize = [" ".join(example.values()) for example in few_shots] This is correct.
Might solves the issue
I am not able to run, please help me, this is my final year project 😢😢😢 If anyone can help me, I am ready for paying.
Any one have updated code
In production answers might change every day, it will be not possible to change answers every day right? And what about if I need data in table format, It will only answer in one single word, if we need data of 2 or more columns, so what would be the approach for it?
Same doubt
I have a doubt. Will this send my Database contents/schema along with API request to LLMs for queries I ask ?
Can we use langchain in corporates. I mean using API access key may be not secure to use in corporate industries because of data security
I am getting error at the starting stage itself while using the google palm
Activate your api key
Hi, how I will decide that which GPT version will suitable for my project as below code? Please send any guideline/rule regarding this. Thank you.
chatllm=ChatOpenAI(openai_api_key=os.environ["OPEN_API_KEY"],temperature=0.6,model='gpt-3.5-turbo')
how to print associated SQL query for the question in streamlit?
Can i write this code in vscode?
How to use Gemini?
Facing issue in establishing a connection b/w python and SQL. Can someone help?
sir how to give table and column name description to llm since most of the time the names are assigned based on domain knowledge so how to give custom description
hi sir, while doing langchain i found that model is passing unwanted quotes to the db. Can you suggest any solution to rectify this problem?
I am trying to join the list but i am getting error like this "sequence item 3: expected str instance, dict found" how can i solve this error
please help me to comeout from this
They have decommissioned google palm now what to do
I’m facing an error also with google palm what can be alternative do yoh have any idea?
@@kashifahmad9880 It's very difficult to select any other model because other model are not able to Make Sql connections
Can we access MongoDB as well?, becoz i could not find docs related to MongoDB.
I have no science background in my school level and i completed my post graduation.is it eassy to shift data science filed.. If i can learn data science certificate programm.. If data science short term course are available in data science filed
I've been facing a problem where I'm not been able to connect local host. Please help if anyone can.
Sir you are using a GPT api key internally how they work and what is providing in the code when you use the chain function then you are not going to encode query. And you pass LLM = LLM means what??
Bro you got a long way to go
@@johannanderson848 means
How do we add memory to this?
Can I do this project using Llama or any open source model?
Yes. Please check the latest video on this channel. It uses llama3.1, you can change this project as per that video.
@@codebasics as the result in the browser I get this output "Question: what is the quantity of red tshirt? SQLQuery: SELECT sum(stock_quantity) as total_quantity FROM t_shirts WHERE color = 'Red'" not get output like you
@@arshahin9788 @codebasics I am getting the same problem
@ 14:33 while executing this db_chain = SQLDatabaseChain.from_llm(llm, db, verbose=True) i am facing an error ValidationError: 1 validation error for SQLDatabaseChain
database
instance of SQLDatabase expected (type=type_error.arbitrary_type; expected_arbitrary_type=SQLDatabase) please someone help me
Not working this code now
@@shahrukhahmad4127
@lakshmikanth1988
Could you resolve the error?
Does this project include any paid Suprciptions for keys?
No
Which python version is used in this project & Why? There are new version available like 3.10, 3.11, 3.12
use 3.11 or 3.10 dont go for 3.12
Thanks @@5-10zoomergirls
what if the quantity of t-shirt decreases which means someone buys after some time passed, do we need to change few_shots list's values and rerun the whole process? or llm figure out by its own?
Exactly my doubt
You did this project?
Error of this line :--to_vectorize = [" ".join(example.values()) for example in few_shots]
This is the error :----------------------------------------------------------------------------
TypeError Traceback (most recent call last)
Cell In[31], line 1
----> 1 to_vectorize = [" ".join(example.values()) for example in few_shots]
Cell In[31], line 1
----> 1 to_vectorize = [" ".join(example.values()) for example in few_shots]
TypeError: sequence item 3: expected str instance, dict found
If I change above line with this to_vectorize = [{" ".join(str(value) for value in example.values() if isinstance(value, str))} for example in few_shots] this is working fine but in next line
vectorstore = Chroma.from_texts(to_vectorize, embeddings, metadatas=few_shots)
it is giving me this error 86 texts = list(map(lambda x: x.replace("
", " "), texts))
87 if self.multi_process:
88 pool = self.client.start_multi_process_pool()
AttributeError: 'set' object has no attribute 'replace'
I resolved it using the following changes:-
in few_shots, make sure to take only the results from the qns1, qns2, qns3, qns4, qns5 .
i;e., qns4['result'] in-place of qns4 and so on.
and same for others.( if qns is alread a string value and not dictionary, then leave it.)
@@prakashraushan2621 did u get a working project?
@@deepashenoy2301 yes it worked for me.
Feel bad for Lauki Lal
OSError: [WinError 126] The specified module could not be found. Error loading "C:\Users\Moin\anaconda3\Lib\site-packages\torch\lib\fbgemm.dll" or one of its dependencies.
i m getting this error, not able to tackle