Getting Started with RAG in DSPy!
ฝัง
- เผยแพร่เมื่อ 27 มิ.ย. 2024
- Hey everyone! Thank you so much for watching this tutorial on getting started with RAG programming in DSPy! This video will take you through 4 major aspects of building DSPy programs (1) Installation, settings, and Datasets with dspy.Example, (2) LLM Metrics, (3) The DSPy programming model, and (4) Optimization!!
The notebook used in the video can be found here: github.com/weaviate/recipes/b...
All future videos, as well as additional utils like data import scripts, will be in this folder: github.com/weaviate/recipes/t...
Please leave a star, it helps a lot!
DSPy on GitHub: github.com/stanfordnlp/dspy/
Join the DSPy Discord! / discord
DSPy GPT by Sean Chatman: chat.openai.com/g/g-cH94JC5NP...
New Signature class based on Pydantic by Thomas Ahle: github.com/stanfordnlp/dspy/p...
Knox, Ollama in DSPy: / 1756018720818794916
Stephen Byrne, Why I'm excited about DSPy: substack.stephen.so/p/why-im-...
Chapters
0:00 Intro!
1:35 Where to find the code
2:08 Community Notes
4:00 Getting Started with RAG
7:56 0. DSPy Settings and Installation
10:00 1. DSPy Datasets
11:56 2. LLM Metrics
19:22 3. The DSPy Programming Model
23:32 4. DSPy Optimization
30:45 Recap - วิทยาศาสตร์และเทคโนโลยี
DSPartY is bumpin’
Love your tutorials and the engagement you’ve been putting in within the DSPy community.
great tutorial Connor, looking forward to more advanced stuff like Agent application!
Connor - you can pan in and out as much as you want IMO, shows your excitement about the subject. The quality of the content is awesome. Also appreciate the shoutouts to the broader community. Thanks for sharing!
Thank you so much!! Haha, appreciated! Although I see far less negative comments this time around without the zooming haha! Thank you so much! Beyond grateful for all the help the DSPy community has given me in learning about this!
Great one Connor! good to see your progress that, naturally, help us all...
Thank you so much Jose, really happy to hear that! Learning a new tool is certainly quite the journey haha!
Very exciting stuff, thanks Connor!
Thanks Daniel!
Thank you Connor. This is exactly what the world needed.
Thank you so much Clint, appreciate it as always!
Thanks for the detailed video Connor, This is a great help. I am working on the lang graph and multi agent models. I had to optimise some of my prompts manually to reduce the number of agent hops to llm model. With BaysianSignature optimizer, I believe every prompt can be optimised and it'll reduce the hops made by agents
Ah, I need to learn a little more about LangGraph and I think Crew AI as the latest Multi-Agent framework before I can really comment on this!
Brilliant Connor, thanks so much for this video and looking forward to more about this subject!
Thank you so much Brad! Really happy to hear it! DSPy!
looking fresh my dude
Haha thank you!
Thanks for the great content. One of the things I am missing is how to save the optimized program so I can use it after that without constantly re-training.
Great walkthrough, thanks so much!
Great tutorial! I'm looking forward to building on this! Thank you
Nice video, Connor. Could you do a more in-depth video on the optimization process? In particular, looking at the series of prompts/examples selected throughout the optimization (analogous to doing a small lin. regression/backprop example by hand for intuition) and the overall token cost of these optimizations.
Ah indeed this is quite the test of my comprehension! Thank you so much, this is a fantastic idea! Give me a little bit of time to work through this though -- I will send this message through to the DSPy discord, I'm sure Omar, Michael, or Krista would be happy to walk us through this!
Thank You. This particular vid motivated me to SUBSCRIBE !
Awesome, super inspiring!
Thank you, Connor! Keep it up!
Thanks!
Conner - Thanks for the shoutout!! ❤
No thank *you*! Ollama in DSPy!! Amazing!
This was a fantastic walkthrough! Would love some insight into extracting structured data - I find this extremely useful, and being able to do this with a 7B/13B model (instead of GPT-4, for instance) would greatly decrease the cost of running my application. Thanks so much!
You should also make a video on each GPT call cost. I believe there are hundreds (if not thousands) of calls happening every execution. DSPy is best paired with local model like mistral 7b. Otherwise, it will be impossible to scale such a tool on hundreds of docs.
If you are not compiling the program there aren't that many calls happening... Just one call per module (ChainOfThough/Predict/ReAct..) actually. You can check every step in the pipeline with dspy-inspector for example
Yes this is exactly where my thinking is going after getting the bill for the video hahah, but more generally yeah my suspicion is that llamas connected in DSPy programs is where the value is -- need to test more to say for sure!
@@neoxelox Ah thank you! A new DSPy tool to try out haha `dspy-inspector`!
@@connorshorten6311 Can you make a video about it? and also another video on some of the pros & cons of using DSPy in production (if any - wrt cost, latency, scalability, & flexibility)?
lets gooo
So many haters wtf. Great video !! I been lazy in python bc of copy paste and langchain and llama-index. This video makes python more fun !
Hay man, great video! I have a few questions tho. Can you use other vector DB as retriever like Milvus? Also, is it possible to use LLM that are less known like Baichuan, Kimi etc? Thank you!
Thank you for fixing the zooming!
Haha you got it! Apologies for last time! The zooming has been fired!
could we cover the creation of the schema from an empty database such that the notebook flow actually runs through
Is it possible to run DSPY on local windows environment, say with Mistral 7b model? It fails to for me because of default value of url param, which I do know how to avoid.
You tried with weaviem is there any way you could do with pinecone ?
Any idea why the bootstrap with random search performed worse on the eval set? @ 29:00
Can I conclude that this framework is simply a prompt optimization framework? The rest is almost the same as llamaindex and langchain?
How can I load and use my own data to Weaviate and start implementing DSpy's implementation of RAG?
What we can do to have bigger answers? I want it to generate code, but after executing it gives me 4 lines of code
Someone have some idea?
Can you apply DSPy RAG on PDF files?
No offense but you took 4 min to get to the point. Time is the most valuable resource. And it is even more important with the plethora of information in the fast moving generative AI space. So please be ruthless in cutting down non-value add content. I'm not criticizing but I'm just voicing an opinion on great videos like this where you can be even more due diligent on our audience time
This comment took you longer than 4
@@edwardgao5388 true. I did that to see if the OP can save me some time in the future
Thanks. Saved my 4 minutes 😅
so for production, we'd just copy paste the optimized signature and few-shots into the system prompt? I'm a bit confused on how to wield this tool in production.