DSPy Assertions: Computational Constraints for Self-Refining LM Pipelines // Arnav Singhvi // Talk

// Abstract
Chaining language model (LM) calls as composable modules is fueling a powerful way of programming. However, ensuring that LMs adhere to important constraints remains a key challenge, one often addressed with heuristic “prompt engineering”. We introduce LM Assertions, a new programming construct for expressing computational constraints that LMs should satisfy. We integrate our constructs into the recent DSPy programming model for LMs, and present new strategies that allow DSPy to compile programs with arbitrary LM Assertions into systems that are more reliable and more accurate. In DSPy, LM Assertions can be integrated at compile time, via automatic prompt optimization, and and/or at inference-time, via automatic self-refinement and backtracking.
// Bio
Arnav recently-graduated undergrad in CS at UC Berkeley. Researcher with Stanford NLP Group, contributing to DSPy framework
A big thank you to our Premium Sponsors, ‪@Databricks‬ and ‪@baseten‬ for their generous support!
// Sign up for our Newsletter to never miss an event:
mlops.communit...
// Watch all the conference videos here:
home.mlops.com...
// Check out the MLOps Community podcast: open.spotify.c...
// Read our blog:
mlops.community/blog
// Join an in-person local meetup near you:
mlops.communit...
// MLOps Swag/Merch:
mlops-communit...
// Follow us on Twitter:
/ mlopscommunity
//Follow us on Linkedin:
/ mlopscommunity

[@irrelevantdata]

Are "question", "answer", "long_document" and "summary" variables, and if so, where are they declared? 5:17 What operation does this symbol "->" perform?