LLM Project | End to End LLM Project Using Langchain, OpenAI in Finance Domain
ฝัง
- เผยแพร่เมื่อ 8 ก.ค. 2024
- End-to-end LLM project for beginners and intermediate users using langchain. In this LLM project, we will use langchain, openai API, and streamlit to build a news research tool that can be used by equity research analysts to conduct their research. This end-to-end NLP project will give you a good experience in building a real-life industry project that will add a lot of value to your data scientist or NLP engineer project portfolio.
Code: github.com/codebasics/langcha...
Langchain playlist: • LangChain Tutorials Pl...
Timestamps:
00:00 Introduction
01:20 Problem statement, tech architecture
13:45 Loaders (TextLoader, UnstructuredURLLoader)
21:31 Text Splitters (CharacterTextSplitter, RecursiveCharacterTextSplitter)
33:30 FAISS Index and Vector database
44:52 Retrieval (RetrievalQAWithSourcesChain)
56:49 Streamlit UI and Project Coding
Do you want to learn technology from me? Check codebasics.io/?... for my affordable video courses.
Need help building software or data analytics/AI solutions? My company www.atliq.com/ can help. Click on the Contact button on that website.
🎥 Codebasics Hindi channel: / @codebasicshindi
#️⃣ Social Media #️⃣
🧑🤝🧑 Discord for Community Support: / discord
📸 Codebasics' Instagram: / codebasicshub
📝 Codebasics' Linkedin : / codebasics
------
📝 Dhaval's Linkedin : / dhavalsays
📝 Hem's Linkedin: / hemvad
📽️ Hem's Instagram for daily tips: / hemvadivel
📸 Dhaval's Personal Instagram: / dhavalsays
🔗 Patreon: www.patreon.com/codebasics?fa...
![[안방1열 직캠4K] 베이비몬스터 치키타 'FOREVER' (BABYMONSTER CHIQUITA FanCam) @SBS Inkigayo 2404707](http://i.ytimg.com/vi/Tn3bGHiBWkg/mqdefault.jpg)
Check codebasics.io/ for intuitive, affordable tech courses taught by me.
I am always grateful to you. You are the my only teacher who teaches me basics of data science properly. Now with this LLM. langchain. You have superpower to explain everything in very easy way for a dumb student like me. I couldn't be a data scientist and fulfill my dream if you weren't there. Thanks for everything, please do continue this kinda projects for each domain🙏🙏🙏🙏💝💝💝
Did you tried this Rockybot
I am facing one error please help me to solve it
list index out of range
index = faiss.IndexFlatL2(len(embeddings[0]))
@@shaonsikder556
please resolve the index 0 issue
can I use Gemini API key instead of Open AI?
How model will work when 5 different users are providing different url to model simultaneously?
This is the best tutorial I have watched on LLM. Crisp, yet detailed. Thank you!
An end to end langchain project cannot be more clearer and perfect than this one !
It's just a perfect combination of depth and width!! Thank you so much!!
One of the best courses on TH-cam to learn GenAI. Thanks a lot for this
If anyone is facing list index out of range error then here is an open github issue: github.com/langchain-ai/chat-langchain/issues/68
Using a different version of langchain may help.
I tried 3 different versions but it did not work, on which version it might work?
Thanks a lot, I'm so happy that I came accross this video. Best tutorial.
i was working on a langchain project myself! perfect timing!
I like the way you explain the business scenario and toolset required to implement the solution in the PoC and Production phases.
Thank you! your trainings were always great and this one is even greater!
Thank you sir for this wonderful tutorial, which made me understand about the end to end LLM project in a simple way. Great work.
Absolute best explanations. To class. Thank you.
Thank you so much for this wonderful tutorial, really helps me in implementing a POC in my company
Best thing I love about this guy is, he let the audience feel the problem, so that when he is doing this he will be clear that I am solving this problem this should be the approach, love u man for all the content u make
Another excellent learning video! You explained the end-to-end LangChain application workflow very nicely. Thanks!
Coding... Coding... Coding... I don't like it! I avoid! But! Coding likes me! I can't avoid!
🤣🤣😜
😂😂
Waiting for this series!!
Thank you
The best one so far! Amazing
Found a lot of stuff on Langchain on TH-cam. But no one went this deep with individual langchain concepts as you did. A lot of confusing topics got cleared today. THANKS FOR THIS. ❤
Glad you enjoyed it!
sir please help me to solve this error
list index out of range
index = faiss.IndexFlatL2(len(embeddings[0]))@@codebasics
Thanks this explains the concepts really well and brings them to life.
what more simple explanation could be possible than this. Awesome explanation and project concept.
Amazing videos in the playlist! Thank you for your hard work!
Nice eee jumping in this weekend thank you
you are awesome teacher, I rarely post comments below yt movies.. you just have explained it in very easy and understable way.. ;-) well done
this is a movie or theory well explained and a powerful content thanks for this need more...
The Best best best video on langchain+llm in youtube Thank you sir for wonderful content.
Please upload the videos on how to finetune our own data and some more real world usecases like this video .............😇😇😇😇
Thank you so much Dhruw for this wonderful explanation :)
Keep rocking with your content ❤
Literally had this same project last month in my organisation
super useful thank you so much. you are great tutor expecting more videos like this
such a nice explanation sir. well understood. got an idea for my project work ,thank you sir .
Very good tutorial.Thanks for all the effort.
Great Explaination Thank you so much !
thank you so much sir. The project and explanation were very helpful
Thank you!. very much useful.
Thank you very much. its great.
Woow..my new hero Peter Pandey
This was an awesome overview! Thank you for all the work put into it!😍
Glad you enjoyed it Lisa!
Thank you for this!
Sir you are great learnt a lot!!!
Waiting for this series.
Thanks for this nice tutorial !
Thank you 👍
Amazing .... Would help us a lot ! Thank you :)
Rocky bhai made me watch the whole video😂 good job
fantastic work
You are the best!
hello sir the content is awesome and I will recommend it to my friends for sure.If possible it would be still more helpful if you can explain the conversion of table data into the contextual data in further vedio.
Great video Dhaval👍
Awesome project tutorial. Thank you. Learned a lot and will use in my projects. 🙏🙏🙏🙏
Glad it was helpful!
sir please help me to solve this error
list index out of range
index = faiss.IndexFlatL2(len(embeddings[0]))@@codebasics
sir please help me to solve this error
index = faiss.IndexFlatL2(len(embeddings[0]))
@@codebasicsI have used this technique with open source llm and completed a project of chat with local pdf files. For a lack of resources I can't able to build UI for that.i already posted this project documentation in my LinkedIn profile.
Amazing video Dhaval!!! Do you have MLOps playlist please share, if not could you please think about creating it using mlflow or zenml because i like the way you teach. Thanks!
Good evening respected sir
happy to see Rocky bhai reference :)
Great video !!!
Awesome
Sir too good...
Awesome bro..
You are awesome.
great video! could you please do a video on how to set up our jupyter environment?
Thank you. I have a question why not use a news API to retrieve article and company topics that individual urls?
Rocky boi hehe
thank you very much 😍. Can you post the slide, it's very helpful
Please do a video on LlanGraph...you will do it better than others for sure.
I am a beginner to this langchain, LLM world. Just wanted to understand in this example how is the PROMPT to the LLM being created - is it automatic? we are providing the query in the form of streamlit UI input & context in the form of retrieved chunks from vector DB this I understood.
Should have invested in Nvidia 5 months before to double the amount, Bro give the best financial advice 😜
I am working on same project
To learn llm what is prerequisites... please clearly mention
Is machine learning /deep learning required? @codebasics
It looks like a great project but I have some doubts on how the Vector is working, and fetching the correct results. I am tryingto understand and also have gone through the video again... can someone help me understand the reason as how converting the corpus to vector can help us find topics. I guess if I understand this, a project on Topic modeling will be easier
Getting an error:
module 'magic' has no attribute 'from_buffer'
tried multiple fixes, but unable to resolve.
Lets assume there is a cron job resding the urls. After a few days will the vector database has huge size? I have requirement where i want to read some documents(PDFs/docx) and get same data from each of it, answer a few questions. Do i need a vector db here? If so doesn't the size grow?
Great helped a lot. But if i want a source link of the document which i have in my knowledge base (vector database) how would i be able to do that?
Thanks a lot, Could you please create LLM builder on structured data.
Just a thought is or unniverse is quantified .
If we want to make a simulation what tokens numberwould be the max.
Sir please make a Data analysis project which dashboards are designed by AI.
Hello, Thank you for the video. I have a Question. Is there a technical way to measure the validity of responses?
Can we use scrapping and automation libraries like Selenium and beautifulsoup for long term project???
can we use other llms other than open ai - if so how
- also can you show how this is done on sagemaker
Sir can u make a session on making a chatbot like chatgpt that answer all the queries based on the info that we pass to it as pdfs
Sir this is really superb!! Can u plz generate high end machine learning real life projects with data cleaning preprocessing and modelling and end it with a web application.......it would be a very great help as being a student...
I think he has done it already. Potatoe disease project.
@@alphar85 that comes under Deep learning for my 5th sem project I have done that
@@alphar85 and moreover no data cleaning process is involved in potato disease classification. And plz respect it is sir not him.
Thank u
@@aishu2623 ok sorry i thought you just wanted an end to end project.
@@aishu2623 data cleansing is subject yo project. His data cleansing will most probably be different to your project esp if you are customising it. Otherwise if you are just copying and pasting then it is a different matter. Best of luck
Please make a project video using transformer.
why did you not do Vector Embedding in the project? You did it in the Interstellar Text.
Hi Dhaval - do you think streamlit has a future potential to implement a quick dashboard app at corporate level and get rid of tedious frontend coding ?
no
hello sir,
I was confused how can i create selenium test cases of a web application using langchain
can i directly import the selenium documentation and expect the ai to create the test case(for example: a button functionality).
This is the biggest challenge of AI - 99% of the AI TH-cam stars are inventing something which has already been solved 15 yrs back.
Could you please share the ppt through which you were explaining for future reference and study purpose
@codebasics sir when i am running this model in my localhost after spillting step it is showing index out of range
Why didnt we use SentenceTransformer and FAISS in the final project ?
As money control url have neft or stock data can this give answer of that?
can you suggest an open source embedding instead of openAI embedding, that works fine for this project?
I used bert and glove, and received nothing but endless errors
Sir a question now the chatbot can answer only the questions from those only 2 links ?
Can you implements such project like sentiment and question answering, NER end to end using openai and langchain
Do we need to buy openai tokens inorder to carryout this project???
❤
What are pre requisites, I have done python and library like numpy and pandas can I do this or I need to learn some thing first
Is it possible to "replace" the urls with github repositories of code or documentation? I'd also be important to keep the source metadata just like in this urls example to understand from which document inside which repo the answer came from. Thank you very much!
we can build it together
which python version should be used to run this project. Not all the dependencies are getting installed with the latest version of python which is 3.12
When loading the pkl file, I keep getting "ModuleNotFoundError: No module named 'langchain.schema.document'" error. I am using python 3.11.1 on Macbook Air. Thanks!
Is it possible to run the code first in jupyter notebook
Hi, Just one doubt. If we are saving the embedding using wb in a for loop, wouldn't the embeddings be getting overwritten in the same file?
Can this be done using llama index? Any reason to prefer langchain over llama?