Building RAG-based LLM Applications for Production // Philipp Moritz & Yifei Feng // LLMs III Talk

// Abstract
In this talk, we will cover how to develop and deploy RAG-based LLM applications for production. We will cover how the major workloads (data loading and preprocessing, embedding, serving) can be scaled on a cluster, how different configurations can be evaluated and how the application can be deployed. We will also give an introduction to Anyscale Endpoints which offers a cost-effective solution for serving popular open-source models.
// Bio
Philipp Moritz
Philipp Moritz is one of the creators of Ray, an open-source system for scaling AI. He is also co-founder and CTO of ‪@anyscale‬, the company behind Ray. He is passionate about machine learning, artificial intelligence, and computing in general and strives to create the best open-source tools for developers to build and scale their AI applications.
Yifei Feng
Yifei leads the Infrastructure and SRE teams at ‪@anyscale‬. Her teams focus on building a seamless, cost-effective, and scalable infrastructure for large-scale machine learning workloads. Before Anyscale, she spent a few years at Google working on the open-source machine learning library TensorFlow.
// Sign up for our Newsletter to never miss an event:
mlops.communit...
// Watch all the conference videos here:
home.mlops.com...
// Check out the MLOps Community podcast: open.spotify.c...
// Read our blog:
mlops.community/blog
// Join an in-person local meetup near you:
mlops.communit...
// MLOps Swag/Merch:
mlops-communit...
// Follow us on Twitter:
/ mlopscommunity
//Follow us on Linkedin:
/ mlopscommunity

[@Insipidityy]

If the LLM evaluator is not trained on ray documentation, how can it be used to evaluate if the responses to ray-related questions are correct?
One more assumption is you're using GPT4 as the LLM evaluator. If GPT4 is asked to evaluate GPT4, isn't there an inherent bias at play?