No more bad outputs with structured generation: Remi Louf

JSON, prompt formatting, hallucinations. If you feel uncomfortable, you have probably had to implement complex solutions to circumvent these problems. What if there was a way to increase the reliability of Large Language Models at no cost? Enter structured generation. In this talk we will explore how the output of models can be steered at no extra cost, why this improves their efficiency and accuracy significantly and makes generation less sensitive to the specifics of the prompt. By the end of the talk, we'll understand how we can start benefiting from this breakthrough today by using the open source library Outlines.
Recorded live in San Francisco at the AI Engineer World's Fair. See the full schedule of talks at www.ai.enginee... & join us at the AI Engineer World's Fair in 2025! Get your tickets today at ai.engineer/2025
About Rémi
Rémi is the co-founder and CEO of .txt. After studies in Philosophy and Physics, Remi explored NLP and Bayesian statistics for 7 years. He works from a French castle, enjoys long walks in the nearby forests, reading poetry, playing guitar and spending time with his children.

[@glennmontague4310]

FYI, this was recorded in June. Not posting the recording date somewhere obvious (like in the title) is misleading.

[@connor-shorten]

Can't wait to try this with vision models! Really interesting insight on the interplay of structured generation and in-context learning! Thanks so much for the talk!

[@aethermass]

Really smart. I will probably start using this at work this week.

[@ArturG3]

How different it is to Instructor library?

[@ruili7941]

Addressed my pain point.

[@artur50]

Github or didnt happen 😂

[@paulomtts]

This concept has been around for a while, not much new here. Lookup Jason Liu's Instructor library, if you use python.

[@broformation6530]

How does it affect the performance of the model?