Thanks so much! You are the best channel when it comes to RAG. Please keep informing us about the latest advancements in this field. I have played with GraphRag. It can be expensive if you have tons of data, but considering how cheap GPT-4o and GTP-4o-mini have become, the price is not the biggest concern at least for my use case. I processes more than a thousand page document with GPT-4o and it cost me cents. The biggest problem with MS GraphRag is the inference latency. It is not very practical if you want to build a chatbot based on this. Also, it is less customizable in my recent experience. Hope LightRag is better in terms of accuracy, customizability and inference speed.
It looks like the pipeline is modular, and could work with different vector and graph databases. In the future, users could rate the responses, so highly rated responses could be stored and retrieved when similar queries are made by other users. Retrieving highly rated responses could improve overall system performance compared with generating responses for every query. Apply fast thinking instead of slow thinking when possible.
Great stuff, well done! I don't see any reason to use GraphRAG any more after that. I guess the technology will continue to surprise us, but the cost argument is powerful when you handle tons of data
Thanks for sharing , however this is another smart config changes resulting in low prompt tokens , Microsoft GraphRag uses higher overlap window ( 300:100 ) whereas here it is using (1200:100) which itself reduces number of prompt tokens used significantly. we should deep dive to understand if researchers have done something different to ensure much lower overlap window
Graphs solve the problem of chunk embeddings being de-contextualized. Late Chunking solves this problem. I think we need to re-consider the use cases for GraphRags.
It costs me $0.02 per 10 page pdf document using openai api. I think it's pretty decent. Future improvements should also include storing all these json files in a database. Also, have you figured out how to get the reference chunks used to generate the response to the query?
@@penguinmagehello No images. The example on github is using textract for pdf text extraction, but you can use any other approach and save the images. You could add them to the graph by sending them to a VLM for a descriptive summary. That would go to the entity/relationship resolution step. Your chunking would have to respect the figure description (See figure 1, which blah blah) and the figure title.
On the 'mixed' dataset, I don't think it had enough information on the various topics to create a comprehensive graph. I'm wondering if GraphRAG digs deeper or does some other technique with the graph meta-data like BM25.
How do you record your screen so that it follows your cursos? It’s super useful to follow along specially when reading papers! Kudos on this amazing expose of this methodology!
In a few years I suspect we will laugh at all these hacky RAG implementations as one will simply be able to dump everything into the context window and there will be some native mechanism to handle efficiency. What do you think?
I like everything about lightrag. I wish we could see the reference chunk/document for the returned query too. Also, if there's a better way to query from a sql/nosql db instead of querying from drive.
cool. Is there an implementation for Groq-API like gpt_4o_mini_complete-API in LightRAG? How can i use a GUI-Chatbot for LightRAG, e.g. chainlit/streamlit/...? thx
Its a hard question to answer. It will really depend on the task at hands. If you think its just search/looking up information, may be a standard RAG. If there are relationships between objects/entities then may be a knowledge graph.
nice video, I am just starting with RAGs, so sorry if my question is a little stupid, but if you spend 80k tokens running LightRAG, why the necessity to even implement a RAG when the book it self has 40k tokens? I mean wouldn't be easier to send the whole book to chatgpt?
> "I mean wouldn't be easier to send the whole book to chatgpt?" because this stuff is for knowledge graphs, not simple doc Q&A. The graphs create relationships *between* content. We used to do this in KBs by manually (and slowly) tagging and hyperinking stuff that we thought was related. Now models can create the relationships for us without having to hire 1,000s of librarians. The possibility to have a system trained to create meaningful relationships between content and dynamically pull that content on demand is mind blowing. Pre-trained models are already coming out.
![BOWKYLION - วิงวอน (ex-change) [Official MV]](http://i.ytimg.com/vi/cLdnZWDaO-w/mqdefault.jpg)
Thanks so much! You are the best channel when it comes to RAG. Please keep informing us about the latest advancements in this field.
I have played with GraphRag. It can be expensive if you have tons of data, but considering how cheap GPT-4o and GTP-4o-mini have become, the price is not the biggest concern at least for my use case. I processes more than a thousand page document with GPT-4o and it cost me cents. The biggest problem with MS GraphRag is the inference latency. It is not very practical if you want to build a chatbot based on this. Also, it is less customizable in my recent experience. Hope LightRag is better in terms of accuracy, customizability and inference speed.
Thanks for this. I was looking for a way to add to the KG without having to rebuild it. Crossing my fingers that this is it. And cheaper too 🙂
It looks like the pipeline is modular, and could work with different vector and graph databases.
In the future, users could rate the responses, so highly rated responses could be stored and retrieved when similar queries are made by other users. Retrieving highly rated responses could improve overall system performance compared with generating responses for every query.
Apply fast thinking instead of slow thinking when possible.
It is modular. For example, if you want to replace Textract with MinerU for doc/pdf parsing it would be very easy.
Definitely want to learn more about lite rag. Cost was a hindrance with regular rag. 😊
Pretty cool - if you do more with this, I'd love to see some experiements combining this with late chunking
good idea, when I get time, I want to implement a system that combines all these different approaches together.
@@engineerprompt Has anyone recreated this in LangGraph or LangChain JS ?
Thank you heaps for the diagrams and explanations!
such a great channel, thanks for this guide, i was just about to implement a knowledge base!
Great stuff, well done! I don't see any reason to use GraphRAG any more after that. I guess the technology will continue to surprise us, but the cost argument is powerful when you handle tons of data
Thanks for sharing , however this is another smart config changes resulting in low prompt tokens , Microsoft GraphRag uses higher overlap window ( 300:100 ) whereas here it is using (1200:100) which itself reduces number of prompt tokens used significantly. we should deep dive to understand if researchers have done something different to ensure much lower overlap window
Graphs solve the problem of chunk embeddings being de-contextualized.
Late Chunking solves this problem. I think we need to re-consider the use cases for GraphRags.
It costs me $0.02 per 10 page pdf document using openai api. I think it's pretty decent.
Future improvements should also include storing all these json files in a database.
Also, have you figured out how to get the reference chunks used to generate the response to the query?
Mind sharing how/ which packages you use to split pdfs especially those with images?
@@penguinmagehello google marker pdf to markdown
@@penguinmagehello No images. The example on github is using textract for pdf text extraction, but you can use any other approach and save the images. You could add them to the graph by sending them to a VLM for a descriptive summary. That would go to the entity/relationship resolution step. Your chunking would have to respect the figure description (See figure 1, which blah blah) and the figure title.
On the 'mixed' dataset, I don't think it had enough information on the various topics to create a comprehensive graph. I'm wondering if GraphRAG digs deeper or does some other technique with the graph meta-data like BM25.
Using LightRAG with gguf?
Please please please make a video in which we can use it with our local models.
on it :)
@@engineerprompt Thanks man🙌
@@engineerpromptThe Zamba2 family of models looks interesting for running locally with lower latency and more tokens/sec output.
For the same book, graphRAG uses 1.1 million input tokens and 200k output tokens.
How do you record your screen so that it follows your cursos? It’s super useful to follow along specially when reading papers! Kudos on this amazing expose of this methodology!
It's a video editing app called screen studio on mac
Thanks for the really great video, by the way.
wow so many RAG system this year already
Yup, its hard to track but nice to see the different ideas that are coming up.
In a few years I suspect we will laugh at all these hacky RAG implementations as one will simply be able to dump everything into the context window and there will be some native mechanism to handle efficiency. What do you think?
Or you just grant access to your document repositories and you're good to go
Needle in haystack problem of big context Windows....
To dump 10K documents each composed of at least 100 pages seems it will always be unrealistic. RAG of some sort will still be needed.
@ Will be interesting to see how this comment ages over the coming years. (Or mine!)
the native mechanism might probably be a graphrag implementation
Can I use this with Hugging face inference api or Vertexai api ? Actually I have access to this api only.
Use LiteLLM to standardize them to OpenAI API then configure BASE_URL and you are free!
Is Lightrag supposed to be used with a vector database, a graph database or both?
Both (nano vector database) along with the graph (which I believe is Neo4J by default)
Good Info
Have you tested it on structured data?
Thanks!
thank you!!!
how can i view the algo flowchart? a bit complex to understand
how does integrate the vector dataabse?
great video thank you
Hey 👋
I'm one of the most exciting fans of you and I have the opportunity to come to India I wish I can see you and have a conversation..
Please 🥺🙏
Thanks
I like everything about lightrag. I wish we could see the reference chunk/document for the returned query too.
Also, if there's a better way to query from a sql/nosql db instead of querying from drive.
thank you :)
cool.
Is there an implementation for Groq-API like gpt_4o_mini_complete-API in LightRAG?
How can i use a GUI-Chatbot for LightRAG, e.g. chainlit/streamlit/...?
thx
I think there are a couple of PRs for other models. Not sure about the GUI
How to create LightRAG object inside dicken folder
Which of all RAG is cost effective and quality better?
Its a hard question to answer. It will really depend on the task at hands. If you think its just search/looking up information, may be a standard RAG. If there are relationships between objects/entities then may be a knowledge graph.
so essentially they implemented triples / triplets (sciphi/triplex) 🤔
pretty much :)
nice video, I am just starting with RAGs, so sorry if my question is a little stupid, but if you spend 80k tokens running LightRAG, why the necessity to even implement a RAG when the book it self has 40k tokens? I mean wouldn't be easier to send the whole book to chatgpt?
Imagine large systems with terabytes of data.
> "I mean wouldn't be easier to send the whole book to chatgpt?" because this stuff is for knowledge graphs, not simple doc Q&A. The graphs create relationships *between* content. We used to do this in KBs by manually (and slowly) tagging and hyperinking stuff that we thought was related. Now models can create the relationships for us without having to hire 1,000s of librarians. The possibility to have a system trained to create meaningful relationships between content and dynamically pull that content on demand is mind blowing. Pre-trained models are already coming out.