LangChain Advanced RAG - Two-Stage Retrieval with Cross Encoder (BERT)

In this Video we discuss advanced retrieval techniques from Vector Databases with Query Expansion and Two-Stage Retrieval with a Cross Encoder. We use another Reranker on top to take the famous "lost in the middle" problem into consideration.
Code: github.com/Coding-Crashkurse/...
Timestamps:
0:00 Two-Stage Retrieval in theory
2:44 RAG and it´s issues
9:31 Query Expansion
11:50 Cross Encoder Reranking
13:50 LongContext Reorder
#openai #langchain

[@Canna_Science_and_Technology]

Thanks for the video. Perfect timing…. Need this for tomorrow.

[@sivi3883]

Best 15 mins of my day! You explained every single component in the code clear and crisp! Excited to check the other videos of yours. Thanks a bunch

[@henkhbit5748]

Very informative.👍 Love the umap visualization 2 see the query and the embeddings.

[@roguesecurity]

This channel is a gem 💎

[@user-ze9he1hw3d]

The most productive 14 minutes of my day watching and learning from this video :)

[@codingcrashcourses8533]

great! Thanks for your comment

[@fuba44]

UniqueList = list(set(ListWithDuplicates)) to replace those nested for loops. Love your content!

[@codingcrashcourses8533]

Does not work for complex objects in that way probably;)

[@micbab-vg2mu]

Thank you for the great video:)

[@codingcrashcourses8533]

Thanks for your comment. Glas you enjoyed it :)

[@kenchang3456]

This video is terrific, I'll give it a try!

[@codingcrashcourses8533]

Thank you!

[@felipecordeiro8531]

I don't know the umap library, its very interesting. Good explanation about RAG advanced techniques, sucess for you!

[@codingcrashcourses8533]

thank you :)

[@samyio4256]

Youre vids are insanely good. I doubt there is a better ai-prog-tuber

[@codingcrashcourses8533]

Thank you so much :)

[@andreypetrunin5702]

Спасибо!!

[@codingcrashcourses8533]

Your welcome andreij:)

[@andreypetrunin5702]

@@codingcrashcourses8533 I cannot run the code in VSCode.
When running the import:
From langchain_community.document_loaders import TextLoader, DirectoryLoader
Error:
File c:\Python311\Lib\enum.py:784, in EnumType.__getattr__(cls, name)
782 return cls._member_map_[name]
783 except KeyError:
--> 784 raise AttributeError(name) from None
AttributeError: COBOL
I have installed the langchain-community library.

[@austinpatrick1871]

Awesome video. So I glad I found this channel. Long shot question:
After testing several chunk/overlaps, my experimentation indicates an optimal chunk_size=1000 and overlap=200. My RAG contains about 10 medical textbooks (~50,000 pages). However, every video I see on RAG nobody uses chunks anywhere near that large. Does it seem improbable that my ideal chunk size is 1,000, or is there likely another variable at play?

[@sivi3883]

Did you find anything?
At least from my experience so far, with fixed chunk methodology (whatever be the chunk size or overlap) its easier to do POC but not for production grade quality. Did you try semantic chunking or chunking based on sections/headings and then capture relationship between the chunks via graph database?

[@maxlgemeinderat9202]

Thank, always nice videos!
Do you have a favorite german cross-encoder?

[@codingcrashcourses8533]

No, I don´t! I did not work that much with cross encoders to be honest

[@user-pr6nm2di6d]

Can u please make a video on retrieving data from SQL using SQL agents & Runnable using LCEL. If not possible here, if you can update the same in the udemy course. It helps alot

[@codingcrashcourses8533]

I would rather do it here than on my udemy course, since it´s quite specific. Give me some time to do something like that please ;-)

[@Sonu007OP]

Looking for a similar video with LangChain templates. Production level SQL-ollama app. Greatly appreciated 🙏❤

[@codingcrashcourses8533]

@@Sonu007OP have not worked with ollama yet, i am afraid my 7 year old computer wont get it running ^^

[@codingcrashcourses8533]

Video about this topic will be released on 03/25 and 03/28 :)

[@vinaychitturi5183]

Thanks for the video..
But while genering queries using llm_chain.invoke(query), facing exception related to output parser.
OutputParserException: Invalid json output:

[@vinaychitturi5183]

I resolved it temporarily by removing parser al together and formatted the output in the next step. Thank you again for the video. It is helpful.

[@codingcrashcourses8533]

Weird. Normally i never have Problems with that parser

[@verybigwoods]

How much computation resource (specifically GPU) required in running this cross encoder model?

[@codingcrashcourses8533]

It also works on a cpu

[@sumangautam4016]

LLMChain() is deprecated and the output_parser in the examples also cause json output error.
Would be nice, if you could update the github code. Thank you
If anyone having issue with json output, here is a fix:
from langchain_core.output_parsers import BaseOutputParser
class LineList(BaseModel):
lines: list[str] = Field(description="Lines of text")
class LineListOutputParser(BaseOutputParser[LineList]):
def __init__(self) -> None:
super().__init__(pydantic_object=LineList)
def parse(self, text: str) -> list[str]:
lines = text.strip().split("
")
return lines

[@erenbagc9164]

What's the best way to evaluate this RAG?

[@codingcrashcourses8533]

Difficult topic. Performance or output Quality?

[@erenbagc9164]

@@codingcrashcourses8533 well it should be advanced as much as possible since I got an advanced rag . I saw many cases that people used ragas,trulens etc. I'm indecisive

[@user-nu5ur6ud5f]

Is this open source/free?

[@codingcrashcourses8533]

You mean the cross encoder? Yes