Zeta Alpha
Zeta Alpha
  • 143
  • 83 376
ColPali: Document Retrieval with Vision-Language Models only (with Manuel Faysse)
In this episode of Neural Search Talks, we're chatting with Manuel Faysse, a 2nd year PhD student from CentraleSupélec & Illuin Technology, who is the first author of the paper "ColPali: Efficient Document Retrieval with Vision Language Models". ColPali is making waves in the IR community as a simple but effective new take on embedding documents using their image patches and the late-interaction paradigm popularized by ColBERT. Tune in to learn how Manu conceptualized ColPali, his methodology for tackling new research ideas, and why this new approach outperforms all classic multimodal embedding models. A must-watch episode!
Check out ColPali / ColQwen2 & the ViDoRe benchmark:
- arxiv.org/abs/2407.01449
- github.com/illuin-tech/colpali
- github.com/illuin-tech/vidore-benchmark
- huggingface.co/vidore/colpali-v1.2
- huggingface.co/vidore/colqwen2-v0.1
- huggingface.co/spaces/vidore/vidore-leaderboard
Timestamps:
0:00 Introduction with Jakub & Manu
4:09 The "Aha!" moment that led to ColPali
7:06 Challenges that had to be solved
9:16 The main idea behind ColPali
13:20 How ColPali simplifies the IR pipeline
15:54 The ViDoRe benchmark
18:23 Why ColPali is superior to CLIP-based retrievers
20:41 The training setup used for ColPali
24:00 Optimizations to make ColPali more efficient
29:00 How ColPali could work with text-only datasets
31:21 Outro: The next steps for this line of research
มุมมอง: 362

วีดีโอ

10 AI papers you should read for September 2024
มุมมอง 329วันที่ผ่านมา
〉Join us for Transformers at Work, live from the Bay Area on Friday, September 20th! zeta-alpha.com/events/transformers-at-work-2024 This is a 3-minute summary of the most trending AI research papers from our latest Trends in AI webinar. For a more detailed discussion, check out the blog post which includes the full recording of the live show: zeta-alpha.com/post/trends-in-ai-september-2024 Dis...
Next-gen reasoning with OpenAI's o1 (& much more) | Trends in AI - September 2024
มุมมอง 822วันที่ผ่านมา
〉Join us for Transformers at Work, live from the Bay Area on Friday, September 20th! zeta-alpha.com/events/transformers-at-work-2024 The AI landscape is buzzing with developments, from massive funding rounds to strategic acquisitions and groundbreaking model releases. In this installment of the Trends in AI webinar, we will unpack OpenAI's newest model, o1, which sets a new standard for reasoni...
Using LLMs in Information Retrieval | Neural Search Talks: Ronak Pradeep
มุมมอง 188หลายเดือนก่อน
In this episode of Neural Search Talks, we're chatting with Ronak Pradeep, a PhD student from the University of Waterloo, about his experience using LLMs in Information Retrieval, both as a backbone of ranking systems and for their end-to-end evaluation. Ronak analyzes the impact of the advancements in language models on the way we think about IR systems and shares his insights on efficiently i...
Designing Reliable AI Systems with DSPy | Neural Search Talks: Omar Khattab
มุมมอง 259หลายเดือนก่อน
In this episode of Neural Search Talks, we're chatting with Omar Khattab, the author behind popular IR & LLM frameworks like ColBERT and DSPy. Omar describes the current state of using AI models in production systems, highlighting how thinking at the right level of abstraction with the right tools for optimization can deliver reliable solutions that extract the most out of the current generatio...
The Power of Noise | Neural Search Talks: Florin Cuconasu
มุมมอง 273หลายเดือนก่อน
In this episode of Neural Search Talks, we're chatting with Florin Cuconasu, the first author of the paper "The Power of Noise", presented at SIGIR 2024. We discuss the current state of the field of Retrieval-Augmented Generation (RAG), and how LLMs interact with retrievers to power modern Generative AI applications, with Florin delivering practical advice for those developing RAG systems, and ...
Benchmarking IR Models | Neural Search Talks: Nandan Thakur
มุมมอง 2582 หลายเดือนก่อน
In this episode of Neural Search Talks, we're chatting with Nandan Thakur about the state of model evaluations in Information Retrieval. Nandan is the first author of the paper that introduced the BEIR benchmark, and since its publication in 2021, we've seen models try to hill-climb on the leaderboard, but also fail to outperform the BM25 baseline in subsets like Touché 2020. Plus some insights...
Neural Search Talks: Using LLMs for Evaluation and Agentic Search with Ragelo
มุมมอง 1682 หลายเดือนก่อน
Welcome to Zeta Alpha's first RAG Meetup in San Francisco! In this session, Fernando Rejon Barrera, CTO at Zeta Alpha, discusses the use of Large Language Models (LLMs) for evaluating and enhancing agentic search capabilities with the help of the Ragelo toolkit. He discusses the automation of the evaluation process by introducing an Elo-ranking system to compare retrieval strategies and RAG sys...
Neural Search Talks: Finetuning Embeddings for RAG
มุมมอง 2612 หลายเดือนก่อน
Welcome to Zeta Alpha's first RAG Meetup in San Francisco! In this session, Jakub Zavrel, Founder & CEO of Zeta Alpha, discusses the details of fine-tuning embedding models for Retrieval-Augmented Generation (RAG). You will find answers to why semantic search using neural embeddings outperforms traditional keyword search, and how combining BM25 with fine-tuned embeddings and cross-encoders can ...
The 10 most trending AI papers of the month | July 2024
มุมมอง 8312 หลายเดือนก่อน
This is a 3-minute summary of the most trending AI research papers from our latest Trends in AI webinar. For a more detailed discussion, check out the blogpost which includes the full recording of the live show: zeta-alpha.com/post/trends-in-ai-july-2024 Dissecting the current Trends in AI: News, R&D breakthroughs, trending papers and code, and the latest gossip. Live talk show from LAB42 with ...
Kyutai's Moshi and Claude 3.5 challenge GPT-4o (& much more) | Trends in AI - July 2024
มุมมอง 6182 หลายเดือนก่อน
Anthropic launched Claude 3.5, undercutting GPT-4o’s price with competitive performance, and Google’s Gemma 2 - 27B emerges as the new strongest ‘somewhat open’ model. ARC-AGI has put a $1M bounty for solving a deceptively simple task where current AI falls short. AI Startup Etched scores $120M to develop a Transformer-specific ASIC, and CuspAI, a new European startup secures millions for carbo...
Apple strikes massive deal with OpenAI (& much more) | Trends in AI - June 2024
มุมมอง 4993 หลายเดือนก่อน
In this month’s edition, we discuss the impact of GPT-4o, while we patiently wait for GPT-5. OpenAI is on a roll with the new Apple deal but remains embroiled in drama. A report on the latest AI essentials from Google I/O and Microsoft Build. A brand new version of the Sentence Transformers library is out, and a new embedding model from NVIDIA is topping the leaderboards. Mega funding rounds fo...
Fast & Verifiable | Retracing Generated Answers while Chatting with Several Docs Simultaneously
มุมมอง 1024 หลายเดือนก่อน
We build RAG solutions for companies too! We can integrate Generative AI with companies' internal knowledge bases to securely bring AI use cases into production. Reach out to us: www.zeta-alpha.com/contact Get accurate answers from trusted sources, and retrace the original chunks of text used to generate your answer. Upload numerous documents effortlessly while meta-data gets extracted automati...
LLaMA3 400B to beat GPT4? (& more) | Trends in AI - May 2024
มุมมอง 6484 หลายเดือนก่อน
In this month’s edition, we discuss the burning question: how good is LLaMA 3 really? Plus all the latest news and releases, like Phi-3, Reka Core & Snowflake Arctic, the new Atlas by Boston Dynamics, and developments in self-driving cars from Wayve and Mobileye. Are the famous Chinchilla scaling laws burnt toast? And as usual, we dive deeper into the most trending AI R&D papers of the month, i...
Baking the Future of Retrieval Models | Neural Search Talks: Aamir Shakir (mixedbread.ai)
มุมมอง 4935 หลายเดือนก่อน
Baking the Future of Retrieval Models | Neural Search Talks: Aamir Shakir (mixedbread.ai)
Devin: The end of software engineers? (& much more) | Trends in AI - April 2024
มุมมอง 4735 หลายเดือนก่อน
Devin: The end of software engineers? (& much more) | Trends in AI - April 2024
JIT Assembly to Build Exascale AI Infrastructure | Neural Search Talks: Ash Vardanian (Unum)
มุมมอง 3026 หลายเดือนก่อน
JIT Assembly to Build Exascale AI Infrastructure | Neural Search Talks: Ash Vardanian (Unum)
Claude 3 steals the throne from GPT4 (& much more) | Trends in AI - March 2024
มุมมอง 1.1K6 หลายเดือนก่อน
Claude 3 steals the throne from GPT4 (& much more) | Trends in AI - March 2024
Zeta Alpha Trends in AI - February 2024: Entering the year of the Dragon
มุมมอง 1.5K7 หลายเดือนก่อน
Zeta Alpha Trends in AI - February 2024: Entering the year of the Dragon
Zeta Alpha Trends in AI - January 2024: 10 predictions for AI in 2024
มุมมอง 5438 หลายเดือนก่อน
Zeta Alpha Trends in AI - January 2024: 10 predictions for AI in 2024
Zeta Alpha Trends in AI - December 2023 - Gemini, NeurIPS & Trending AI Papers
มุมมอง 3K9 หลายเดือนก่อน
Zeta Alpha Trends in AI - December 2023 - Gemini, NeurIPS & Trending AI Papers
A Guide to NeurIPS 2023 - 7 Research Areas & 10 Spotlight Papers to See
มุมมอง 6K9 หลายเดือนก่อน
A Guide to NeurIPS 2023 - 7 Research Areas & 10 Spotlight Papers to See
The Rise and Risks of Generative AI - interview with Marydee Ojala at KMWorld 2023
มุมมอง 5410 หลายเดือนก่อน
The Rise and Risks of Generative AI - interview with Marydee Ojala at KMWorld 2023
Neural Search, Fine tuning, and Vector Databases - interview with Amr Awadallah at KM World 2023
มุมมอง 11810 หลายเดือนก่อน
Neural Search, Fine tuning, and Vector Databases - interview with Amr Awadallah at KM World 2023
The Gen AI Wave in Search and Knowledge Management - Zeta Alpha at KMWorld 2023
มุมมอง 5510 หลายเดือนก่อน
The Gen AI Wave in Search and Knowledge Management - Zeta Alpha at KMWorld 2023
Ethics and risk management for AI in the Enterprise - interview with Tony Rhem at KMWorld 2023
มุมมอง 10710 หลายเดือนก่อน
Ethics and risk management for AI in the Enterprise - interview with Tony Rhem at KMWorld 2023
Meta-search, LLMs, and upgrading Enterprise Search with AI - interview Sid Probstein at KMWorld 2023
มุมมอง 6610 หลายเดือนก่อน
Meta-search, LLMs, and upgrading Enterprise Search with AI - interview Sid Probstein at KMWorld 2023
RAG Defects, Hallucinations and LLM Agents - KMWorld 2023 interview Colin Harman
มุมมอง 10110 หลายเดือนก่อน
RAG Defects, Hallucinations and LLM Agents - KMWorld 2023 interview Colin Harman
Zeta Alpha Trends in AI - November 2023 - US Executive Order, LLM Evaluation & Trending AI Papers
มุมมอง 26010 หลายเดือนก่อน
Zeta Alpha Trends in AI - November 2023 - US Executive Order, LLM Evaluation & Trending AI Papers
Zeta Alpha at KMWorld and Enterprise Search & Discovery 2023
มุมมอง 5411 หลายเดือนก่อน
Zeta Alpha at KMWorld and Enterprise Search & Discovery 2023

ความคิดเห็น

  • @abcthegreat1
    @abcthegreat1 2 วันที่ผ่านมา

    A+, have been curious about ColPali and this was both insightful and easily understood by a non-technical

    • @zetavector
      @zetavector 2 วันที่ผ่านมา

      Thanks, that was our goal! Glad you enjoyed our content.

  • @WemissYew
    @WemissYew 9 วันที่ผ่านมา

    Thank you for making the video!

  • @jmanhype1
    @jmanhype1 หลายเดือนก่อน

    ReBase: Training Task Experts through Retrieval Based Distillation

  • @420_gunna
    @420_gunna หลายเดือนก่อน

    Thanks for uploading!

  • @420_gunna
    @420_gunna หลายเดือนก่อน

    Great interview but encourage the host to give a little more rope to the speaker to explain themselves fully

  • @rufusrodah645
    @rufusrodah645 หลายเดือนก่อน

  • @GeoffY2020
    @GeoffY2020 2 หลายเดือนก่อน

    Hi please improve the audio quality by speaking closer to the mic , I listen to you guys every month, keep up the good work

    • @jakubzavrel8244
      @jakubzavrel8244 2 หลายเดือนก่อน

      Thanks for your feedback. We'll make sure to improve on this for the next session.

  • @GammaOmega-o3t
    @GammaOmega-o3t 2 หลายเดือนก่อน

    Great interview thanks ! 🎉

  • @nabilaabraham9503
    @nabilaabraham9503 2 หลายเดือนก่อน

    the discussion at 17:20 is mostly referencing results in table 1, not table 2 as mentioned but otherwise great ep!

  • @JustinVazquez1430
    @JustinVazquez1430 2 หลายเดือนก่อน

    That wasn’t horrible

  • @marianataglio5865
    @marianataglio5865 2 หลายเดือนก่อน

    Today I missed the webinar, but I'm happy to find out it's available on TH-cam.

  • @alexiskiri9693
    @alexiskiri9693 3 หลายเดือนก่อน

    Thanks. Any news on OpenAI's mysterious "Q" project? When asked about it, Sam said they weren't ready to talk about it yet but it is rumored it was behind part of the big split that happened at OpenAI.

  • @420_gunna
    @420_gunna 3 หลายเดือนก่อน

    salute 🫡

  • @micbab-vg2mu
    @micbab-vg2mu 3 หลายเดือนก่อน

    thank you for the update:)

  • @420_gunna
    @420_gunna 4 หลายเดือนก่อน

    I miss this podcast format! Can't get these discussions anywhere

  • @billykotsos4642
    @billykotsos4642 4 หลายเดือนก่อน

    These are always GOLD !

  • @420_gunna
    @420_gunna 4 หลายเดือนก่อน

    Thanks! :)

  • @yayo9796
    @yayo9796 4 หลายเดือนก่อน

    Great work! BTW how to generate this paper topic distribution visualization?

    • @zetavector
      @zetavector 4 หลายเดือนก่อน

      Thanks! The visualization is generated with our neural discovery platform, search.zeta-alpha.com. To achieve similar results: 1) Open the discovery tab 2) Search for your topic of interest and use filters of meta-data to achieve the desired overview 3) At visualization, press "explore more" 4) At visualization, press "explain clusters" To do this for topics unrelated to AI, we recommend uploading other documents yourself. It will be easy to filter out private documents with the "owner" filter.

  • @arimasters1980
    @arimasters1980 5 หลายเดือนก่อน

    Wooow !! Thank you so much its like falling in a gold mine while looking for copper

  • @Sapose11
    @Sapose11 6 หลายเดือนก่อน

    Lovely talk

  • @micbab-vg2mu
    @micbab-vg2mu 6 หลายเดือนก่อน

    nice:)

  • @drpchankh
    @drpchankh 6 หลายเดือนก่อน

    Good discussions and good depth. Thanks for sharing!

  • @EkShunya
    @EkShunya 6 หลายเดือนก่อน

    i really love your roundups i helps me soo much

    • @zetavector
      @zetavector 6 หลายเดือนก่อน

      We're glad to hear that!

  • @TLabsLLC-AI-Development
    @TLabsLLC-AI-Development 6 หลายเดือนก่อน

    Thanks for the roundup as always! Monthly round ups are long cycles for the field! :D

    • @zetavector
      @zetavector 6 หลายเดือนก่อน

      True! Sometimes the items almost feel irrelevant already :D

  • @420_gunna
    @420_gunna 6 หลายเดือนก่อน

    Always love the monthly roundup! Thanks guys, your TH-cam channel is awesome

    • @zetavector
      @zetavector 6 หลายเดือนก่อน

      Our pleasure, we love your consistent feedback!

  • @420_gunna
    @420_gunna 7 หลายเดือนก่อน

    Love this series and channel, please keep putting out awesome content!

  • @tilaboy
    @tilaboy 7 หลายเดือนก่อน

    Very interesting session! Love it, keep going!

  • @420_gunna
    @420_gunna 8 หลายเดือนก่อน

    Awesome interview! Love listening to Sarah and thanks for putting out awesome content ZA -- excited to see the series on IR methods with Colbertv2 coverage. Great job.

  • @rextran3464
    @rextran3464 9 หลายเดือนก่อน

    are emergent abilities of LLMs still there tho? i just saw jason weis rebuttal to those arguments of it being a mirage

    • @dainionwest831
      @dainionwest831 9 หลายเดือนก่อน

      Yes emergent properties still exist and likely will continue to so so as these systems scale to larger complexity Emergent properties aren't exclusive to LLMs but are a fundamental part of large complex adaptive systems like the stock market, bird flocks, or even us as humans!

  • @austinmw89
    @austinmw89 9 หลายเดือนก่อน

    Can the paper chat handle tables? Also using gpt multimodal to handle the figures would be awesome!

  • @codacoder
    @codacoder 9 หลายเดือนก่อน

    Great video and visualization!

  • @Chadpritai
    @Chadpritai 9 หลายเดือนก่อน

    Can you please share the slides??

  • @nitinkushwaha9540
    @nitinkushwaha9540 9 หลายเดือนก่อน

    Great conversation

  • @TheShadyStudios
    @TheShadyStudios 11 หลายเดือนก่อน

    Adding timestamps would make these videos much more accessible. Maybe y'all can use AI for this?

    • @zetavector
      @zetavector 11 หลายเดือนก่อน

      Good suggestion. The TH-cam transcript has timestamps, but the quality is not that great yet, will look into it.

  • @DjilBadr
    @DjilBadr 11 หลายเดือนก่อน

    very interesting tool.

  • @deeplearningpartnership
    @deeplearningpartnership ปีที่แล้ว

    Exciting times.

  • @gachaswhatitis590
    @gachaswhatitis590 ปีที่แล้ว

    Express yourself through a comment on my channel. I can't wait to read what you think! 📝

  • @gachaswhatitis590
    @gachaswhatitis590 ปีที่แล้ว

    Express yourself through a comment on my channel. I can't wait to read what you think! 📝

  • @TheAIEpiphany
    @TheAIEpiphany ปีที่แล้ว

    Great interview! Angela was a bit misleading in the "open source" section though, by omitting what is not open-sourced and instead focusing on what is open-sourced. Namely none of their models are open sourced (nor datasets to the best of my knowledge), meaning you can't use any of them for commercial applications and you have to retrain the models from scratch (due to the CC-BY-NC 4.0 license). Which means you have to have 100+ GPUs if you want to train 54B MoE in any reasonable timeframe. With 100 A100 GPUs you need roughly a month. :) (assuming of course you have MT experts and a crystal ball at hand otherwise a lot of additional inefficiencies creep in)

  • @brainwaves2389
    @brainwaves2389 ปีที่แล้ว

    how to retrain this on custom data?

  • @gsusin1
    @gsusin1 ปีที่แล้ว

    Great interview! Looking forward to the developments with agents and chaining (ChatGPT Plugins like Zapier, AutoGPT, etc.), wonder if/how lots of what has been developed for traditional ML methods can be reused here. Pipelining for complex Data Science use cases, including lots of data engineering pre-processing and multiple models in parallel or series has become a commodity, we can do this in code (e.g. Kedro) or in tools such as Alteryx, Dataiku, Sagemaker, etc.

  • @kiaragrouwstra4250
    @kiaragrouwstra4250 ปีที่แล้ว

    it seems noteable the mentioned AI optimists have career interests in AI. the mentioned regulations seem not to address some other concerns, like use in surveillance and the military.

  • @EkShunya
    @EkShunya ปีที่แล้ว

    Thank you for your effort I find these updates extremely helpful

  • @younesprog2629
    @younesprog2629 ปีที่แล้ว

    please add the timestamps

  • @user-wr4yl7tx3w
    @user-wr4yl7tx3w ปีที่แล้ว

    this is such a good interview. so informative and cutting edge.

  • @user-wr4yl7tx3w
    @user-wr4yl7tx3w ปีที่แล้ว

    how can we watch the full speech given by Professor Wang, during the conference event?

  • @user-wr4yl7tx3w
    @user-wr4yl7tx3w ปีที่แล้ว

    I wish the audio was better. TH-cam should provide subtitles when audio is not that great.

  • @wayne8863
    @wayne8863 ปีที่แล้ว

    For quality vs quantity. I feel the conversation is a little superficial, I hope the host will spend more time on more technical stuff.

    • @zetavector
      @zetavector ปีที่แล้ว

      Thanks for the feedback! I feel this time it was a bit harder to manage the depth as we wanted to cover a lot and didn't have a specific research paper to focus on. We'll keep this in mind going forward!

  • @mitragynin5442
    @mitragynin5442 ปีที่แล้ว

    Thank you so much for the helpful and interesting informations about this amazing topic! You definately gained another subscriber :) btw. I'm a student and tried to sign up for the free account on zeta-alpha. But when I click on the link in the FAQ section, I just get the message "unauthorized". Can you help me get access?

  • @mint-o5497
    @mint-o5497 ปีที่แล้ว

    great breakdown and explanations, thank you!