LlamaIndex Webinar: PrivateGPT - Production RAG with Local Models

Want to learn how to build a production-ready RAG stack using purely local models?
​In this video we're excited to host Ivan and Daniel, creators of PrivateGPT - the leading project for privacy-aware RAG.
​it’s powered by LlamaIndex, and you can also learn how to use LlamaIndex modules at the right level of abstraction to build your own production RAG system 🔥
​Besides this we will also have an open QA session about proper UX, productionization and privacy challenges.

[@ripeer-dq7bj]

Great webinar! Very informative.
I have a few questions:
1) Does anyone know nvidia's equivalent (4090, T400...) to Apple's M3 mentioned at the beginning of the webinar in terms of LLM inference use cases?
2) Still in the hardware section, can a single instance of LLM running in a single GPU answer multiple requests at the same time? How does it work for the GPU when 300 users are querying the pipeline at the same time?
3) On the question of attacks and people trying to poison the application, what do you think of the Nemo-Guardrails framework?
4) Concerning Private GPT's option of manually choosing between querying the documents and querying the llm directly, are there any solutions that would automatically detect whether the query requires the use of the domain-specific pipeline rag or whether the query can be sent directly to the llm?
Thank you very much for your time!

[@vgtgoat]

I'm interested in using PrivateGPT. Can I use it if I host the local model on a different server and access it through the oobabooga api?

[@scitechtalktv9742]

Can I run this on Google COLAB FREE VERSION? That is a Linux environment ?

[@imadahmad104]

This question never answered on many platforms, how to ensure that it support multiple user simultaneously. It's crashes

[@MrAlket1999]

What is the github repo of this project that you are presenting here? How can we make the same system as you ?

[@YoungSecurity]

Skip to 38:15 in the video for the links.