Learn RAG From Scratch - Python AI Tutorial from a LangChain Engineer

แชร์
ฝัง
  • เผยแพร่เมื่อ 21 มิ.ย. 2024
  • Learn how to implement RAG (Retrieval Augmented Generation) from scratch, straight from a LangChain software engineer. This Python course teaches you how to use RAG to combine your own custom data with the power of Large Language Models (LLMs).
    💻 Code: github.com/langchain-ai/rag-f...
    If you're completely new to LangChain and want to learn about some fundamentals, check out our guide for beginners: www.freecodecamp.org/news/beg...
    ✏️ Course created by Lance Martin, PhD.
    Lance on X: / rlancemartin
    ⭐️ Course Contents ⭐️
    ⌨️ (0:00:00) Overview
    ⌨️ (0:05:53) Indexing
    ⌨️ (0:10:40) Retrieval
    ⌨️ (0:15:52) Generation
    ⌨️ (0:22:14) Query Translation (Multi-Query)
    ⌨️ (0:28:20) Query Translation (RAG Fusion)
    ⌨️ (0:33:57) Query Translation (Decomposition)
    ⌨️ (0:40:31) Query Translation (Step Back)
    ⌨️ (0:47:24) Query Translation (HyDE)
    ⌨️ (0:52:07) Routing
    ⌨️ (0:59:08) Query Construction
    ⌨️ (1:05:05) Indexing (Multi Representation)
    ⌨️ (1:11:39) Indexing (RAPTOR)
    ⌨️ (1:19:19) Indexing (ColBERT)
    ⌨️ (1:26:32) CRAG
    ⌨️ (1:44:09) Adaptive RAG
    ⌨️ (2:12:02) The future of RAG
    🎉 Thanks to our Champion and Sponsor supporters:
    👾 davthecoder
    👾 jedi-or-sith
    👾 南宮千影
    👾 Agustín Kussrow
    👾 Nattira Maneerat
    👾 Heather Wcislo
    👾 Serhiy Kalinets
    👾 Justin Hual
    👾 Otis Morgan
    👾 Oscar Rahnama
    --
    Learn to code for free and get a developer job: www.freecodecamp.org
    Read hundreds of articles on programming: freecodecamp.org/news

ความคิดเห็น • 88

  • @mr.daniish
    @mr.daniish 2 หลายเดือนก่อน +22

    Lance is the man! Love his content

  • @faisalmushtaq2287
    @faisalmushtaq2287 2 หลายเดือนก่อน +14

    I was waiting for this particular course. Thanks

    • @iqtech6065
      @iqtech6065 2 หลายเดือนก่อน

      Assala mu alaikum brother

  • @sagarkeshave5357
    @sagarkeshave5357 2 หลายเดือนก่อน +57

    Include more of langchain, llms, industry level based tutorials

  • @geekyprogrammer4831
    @geekyprogrammer4831 2 หลายเดือนก่อน +6

    This man is amazing!

  • @ser1ification
    @ser1ification 2 หลายเดือนก่อน +3

    This is great! Thank you so much!

  • @jplkid14
    @jplkid14 หลายเดือนก่อน +6

    The complete happenstance of the phrase "do rag" sounding like "durag" coming from this video was awesome. Sorry, totally unrelated...but it made me chuckle.

  • @bhavyajain7403
    @bhavyajain7403 22 วันที่ผ่านมา +1

    This felt like a semester condensed in a few hours. This dude reads a lot. I learned about so many of interesting things.

    • @lakshay1168
      @lakshay1168 14 วันที่ผ่านมา

      is there line by line explaination of full code ?

  • @jasonmuscat534
    @jasonmuscat534 หลายเดือนก่อน +2

    Lance thank you for sharing your deep insights on the subject of RAG and taking the time to share this with the community.
    Just a question, at 1:04:00 into the overall video concerning the subject of Query Construction. For the question:
    "videos that are focused on the topic of chat langchain that are published before 2024"
    Should the result have been?:
    latest_publish_date: 2024-01-01 as opposed to earliest_publish_date: 2024-01-01
    This would be more inline with question:
    "videos on chat langchain published in 2023"
    where the results where:
    earliest_publish_date: 2023-01-01
    latest_publish_date: 2024-01-01
    Thank you

  • @utk1000
    @utk1000 หลายเดือนก่อน +1

    VERY WELL EXPLAINED. THANK YOU

  • @mrchongnoi
    @mrchongnoi หลายเดือนก่อน +1

    I watched this twice. Very good.

  • @ComputingAndCoding
    @ComputingAndCoding หลายเดือนก่อน

    Awesome video! it helped a great deal to explain the concept.

  • @AD-np2sh
    @AD-np2sh หลายเดือนก่อน +4

    I rarely say that a tutorial is good - but this is an amazing tutorial, extremely underrated!!!

    • @wanderingtravellerAB99
      @wanderingtravellerAB99 23 วันที่ผ่านมา

      Underrated by who?

    • @lakshay1168
      @lakshay1168 14 วันที่ผ่านมา

      is there is line by line explaination of full code ?

  • @devloper_hs
    @devloper_hs หลายเดือนก่อน

    Awesome as always

  • @yashtiwari3565
    @yashtiwari3565 2 หลายเดือนก่อน

    Please let us know when the blog related to adaptive RAG will be uploaded, Lance mentioned that he will be uploading it in a day or so. Also I wanted to ask this question to general public, which one is better, State machines or Guardrails?? (In the context of creating complex flows using llms)

  • @janeslt
    @janeslt 2 หลายเดือนก่อน

    Thank you!!!

  • @cristian_palau
    @cristian_palau หลายเดือนก่อน +4

    Great video! What software is used to create these nice diagrams ?

  • @mukilloganathan1442
    @mukilloganathan1442 2 หลายเดือนก่อน

    Always a fan of a lance video

  • @souhaildahmeni9961
    @souhaildahmeni9961 หลายเดือนก่อน

    Thanks for the content !

  • @cyborg69420
    @cyborg69420 หลายเดือนก่อน

    I recommend this vid to everyone.

  • @izzatullobaltabayev8619
    @izzatullobaltabayev8619 หลายเดือนก่อน

    Thank you !

  • @kevinkawchak
    @kevinkawchak หลายเดือนก่อน

    Thank you for the talk.

  • @claudiodisalvo9925
    @claudiodisalvo9925 2 หลายเดือนก่อน +1

    This is great content. Speaking of that 95% of private data I guess a lot of practitioner are finding it hard to convince business people to share their data with an LLM provider. And of course concerns are very much understandable. I guess people would feel more comfortable if a RAG application would be able to clearly define a partition of data that it can work on for the benefit of the tool, and a partition that can be either used as obfuscated or simply never shared, not even by chance.

    • @Kalmaos
      @Kalmaos หลายเดือนก่อน +2

      Maybe the solution would be running the model locally?

    • @juanpablopenaloza5093
      @juanpablopenaloza5093 หลายเดือนก่อน

      NVDIA CHATRTX might just do the job

  • @ChristianBernhardt-tp1tn
    @ChristianBernhardt-tp1tn หลายเดือนก่อน

    i have an question. In the rag-fusion part in the fusion_rank function: why u using the index (rank) to upgrade your scores ? Isnt it better to use the variable "previous_score" ?? the variable rank is just an index, wich descripes in witch order you read in the chunks. btw ty for the video you are an livesaver

  • @FranciscoJPolo
    @FranciscoJPolo 2 หลายเดือนก่อน +1

    Great!!

  • @CodeKitchen
    @CodeKitchen 2 หลายเดือนก่อน +9

    Love the teaching style! at 9:00 you mention that you've walked through the code previously. Is there another video to go with this one or did I miss something?

    • @KOTAGIRISIVAKUMAR
      @KOTAGIRISIVAKUMAR หลายเดือนก่อน

      those are shorts videos and they combined them to form an long single video.
      when lance referring previous video means not another video.

    • @shraeychikker694
      @shraeychikker694 หลายเดือนก่อน +2

      I think this is the playlist from the videos are taken:
      th-cam.com/play/PLfaIDFEXuae2LXbO1_PKyVJiQ23ZztA0x.html

    • @CodeKitchen
      @CodeKitchen หลายเดือนก่อน +1

      @@shraeychikker694 Nice one - many thanks :)

  • @afrazhussain3778
    @afrazhussain3778 2 หลายเดือนก่อน

    great content

  • @karimelfa1394
    @karimelfa1394 2 หลายเดือนก่อน

    thank you

  • @tubege
    @tubege หลายเดือนก่อน

    Question:. Is it possible to do RAG across different vector stores that use different embedding strategies?

  • @vaioslaschos
    @vaioslaschos 26 วันที่ผ่านมา

    Are there any evaluation sets for RAG? If one gets a new RAG method, how can they compare it to other methods out there?

  • @simrangupta7689
    @simrangupta7689 12 วันที่ผ่านมา

    🎯 Key points for quick navigation:
    02:21:13 *🔄 RAG Evolution*
    02:22:20 *❓ Reconsider Chunking*
    02:23:42 *📑 Document-Centric RAG*
    02:25:20 *🔄 Multi-rep Indexing*
    02:26:30 *📊 Utilize Raptor*
    02:28:34 *🔄 Beyond Single-Shot*
    02:30:23 *🧠 Enhance with Reasoning*
    02:30:38 *🎯 Out-of-Scope Queries*
    Made with HARPA AI

  • @ashrafkaasamani209
    @ashrafkaasamani209 14 วันที่ผ่านมา +1

    i have a question about RAG, is it important to clean the data before we use it in RAG, or the LLM would not get affected by it, and it would not make a difference?

  • @therealtuyen
    @therealtuyen 9 วันที่ผ่านมา

    Thanks for sharing

  • @ivant_true
    @ivant_true 2 หลายเดือนก่อน

    Thanks

  • @willcheng8257
    @willcheng8257 2 หลายเดือนก่อน

    Like first and then watch

  • @eduardoconcepcion4899
    @eduardoconcepcion4899 หลายเดือนก่อน

    What is the best way to manage the chunk size?

  • @andrebarsotti
    @andrebarsotti 7 วันที่ผ่านมา

    Hello there!
    This material is amazing. I would love to read the articles cited in the video. Could someone please list them for me? Thank you!

  • @Yomi4D
    @Yomi4D หลายเดือนก่อน

    Amazing.

  • @utkarshkapil
    @utkarshkapil 2 หลายเดือนก่อน

    GOLD

  • @teddysalas3590
    @teddysalas3590 2 หลายเดือนก่อน

    is it possible i can do rag and combine data with huggingface models?

  • @GeandersonLenz
    @GeandersonLenz หลายเดือนก่อน

    What the name of this screen recorder used by Lance?

  • @pr00f
    @pr00f 16 วันที่ผ่านมา

    Where can we find this video about Chunking?

  • @tomasohara
    @tomasohara 12 วันที่ผ่านมา

    Very, very good! Can you post the links in the comments? Not all are in the notebooks (e.g., 2 of 4 at the 1:21:38 mark).

  • @iCeTainment
    @iCeTainment 2 หลายเดือนก่อน

    ❤❤❤

  • @VipinAp-iy9tt
    @VipinAp-iy9tt หลายเดือนก่อน

    How to add coverstional memory to it?

  • @nawaz_haider
    @nawaz_haider 2 หลายเดือนก่อน +97

    Udemy created 50 accounts to dislike this video

    • @vishwanathnb128
      @vishwanathnb128 2 หลายเดือนก่อน +3

      😂😂😂

    • @zaidnadeem4918
      @zaidnadeem4918 2 หลายเดือนก่อน +3

      I will create 50 accounts to like your comment 😂

  • @iuseh
    @iuseh 2 หลายเดือนก่อน

    llama 3 in 15T tokens, chart would be different if you released video 3 days later :)

  • @cboyley
    @cboyley 6 วันที่ผ่านมา

    Isn't this process cost prohibitive? It seems like its taking an input that would cost 10 tokens, and generating an input that costs hundreds if not thousands of tokens.
    Perhaps Im misunderstanding the costs and underworkings of these LLMs.

  • @Andrew-Tsegaye
    @Andrew-Tsegaye 2 หลายเดือนก่อน +1

    LLM Agents plzzz... ❤

  • @sanjaybhatikar
    @sanjaybhatikar หลายเดือนก่อน +4

    Thanks for the excellent video! If your goal is to democratize gen AI to as diverse an audience as possible, I suggest you stop using OpenAI in these tutorials. In many parts of the world, having a credit card is not an option and OpenAI quickly backs you into that corner. Use, promote and support open-sources alternatives instead. Thank you.

  • @riccardomanta372
    @riccardomanta372 12 วันที่ผ่านมา

    Is it just me or are there straight up always errors based on changed libs?
    Even tho when you simply try to execute their code.
    It's really frustrating working with langchain at this point.

    • @xxxxx8200
      @xxxxx8200 10 วันที่ผ่านมา

      Amen.

  • @Josholsan
    @Josholsan 2 หลายเดือนก่อน

    Hello, at 27:13 why is he using itemgetter to pass the question? What's the difference between doing that and setting a RunnablePassthrough() in there?

    • @flaviobrienza6081
      @flaviobrienza6081 2 หลายเดือนก่อน

      No difference, just that with RunnablePassThrough() you don't need a dictionary in the invoke

  • @Thomas_Leo
    @Thomas_Leo 2 หลายเดือนก่อน +1

    Amazing videos but how does this translate into careers or jobs? What positions are employers looking for? Would they even hire anyone without experience? How do you even get started? I'm aware this channel mostly focuses on the coding and hands-on experience but I wish there was an actual channel focused on employment. I'm pretty sure there are channels out there and if anyone has recommendations, I'll be grateful.

    • @samtx
      @samtx 2 หลายเดือนก่อน

      look on linkedin jobs title descriptions keywords if any with llm ai

    • @vcool
      @vcool หลายเดือนก่อน

      Start working on some AI projects first, on your own, in your spare time. Show some results. Once you have two to show, getting a job should be easier. You don't actually need this langchain thing. As for how you get started, if you've already used GPT-4, etc., thinking about larger workflows that chain inputs and outputs in creative ways to solve problems. Also think about when you need to use embeddings for distance computation. You can use the LLM+embedding APIs directly or via an SDK, optionally sometimes with a local vector database. You don't need to go fancy.

    • @samtx
      @samtx หลายเดือนก่อน

      @@vcool what results? What you done

  • @andyhall7032
    @andyhall7032 2 หลายเดือนก่อน +4

    And there was me thinking "how can it take over 2 hours to talk about applying RAG status to your project plans"

  • @roberth8737
    @roberth8737 2 หลายเดือนก่อน +2

    This is the way...

  • @drlordbasil
    @drlordbasil 13 วันที่ผ่านมา +1

    Lance uploaded this himself, this seems copy and paste no?

  • @imaspacecreature
    @imaspacecreature หลายเดือนก่อน

    Good job, but ' dict ' never again 😂

  • @vcool
    @vcool หลายเดือนก่อน

    This seems like it's going to pigeonhole me and tie my hands into a small dogmatic set of patterns, when what I need is broader freedom that I can accomplish without it.

  • @vcool
    @vcool หลายเดือนก่อน +1

    Does this video have everyone brainwashed? If you know basic programming, you don't need langchain at all. I don't like unnecessary abstractions.

    • @Haz2288
      @Haz2288 หลายเดือนก่อน +6

      Yeah but the building blocks are useful. Do you write your own sorting functions?

    • @bradyanderson2250
      @bradyanderson2250 หลายเดือนก่อน

      @vcool would you mind expanding on what the alternative is to langchain? Genuinely curious on learning, not attacking

  • @USER-A566
    @USER-A566 2 หลายเดือนก่อน +5

    This video getting massive viewership🇮🇱

    • @blacklight8318
      @blacklight8318 2 หลายเดือนก่อน +15

      You mean 🇵🇸 ?

    • @USER-A566
      @USER-A566 2 หลายเดือนก่อน

      @@blacklight8318 No

    • @farhanlabib2833
      @farhanlabib2833 2 หลายเดือนก่อน

      Get and eat dudu

    • @junaidiqbal4104
      @junaidiqbal4104 2 หลายเดือนก่อน +1

      Palestine

    • @USER-A566
      @USER-A566 2 หลายเดือนก่อน

      Thanks, but that was on the menu yesterday... today is rice and beans

  • @memezone3256
    @memezone3256 28 วันที่ผ่านมา

    bad

  • @anjanikumar8298
    @anjanikumar8298 9 วันที่ผ่านมา

    may be not the best, explanation and not understandable easily