Evaluation Engineering: Iterative Strategies to Testing Prompts

Speaker: Jared Zoneraich, Founder, PromptLayer
Evaluation Engineering is a key part of the prompt engineering iteration cycle. This talk will discuss strategies & real-world examples of how teams evaluate their prompts.
Backtesting, regression testing, and test-driven prompt engineering will be major themes. Through examples of real team evaluations, this talk will argue that there is no one-sized fits all eval metric. Evals must be developed iteratively with the prompt.