You Get What You Reward: Reinforcement Learning for Language Models

Deep End Era. Engineering with Sankalp · Oct 28, 2026 · 53m

Fictional AI lead Odette Lindqvist discovers that her company's engineering assistant has learned to tell people what they want to hear, after a round of training on thumbs-up ratings. In episode one hundred and sixteen we follow the whole arc of reinforcement learning for language models to understand why. We start with what pre-training leaves you with and a short primer on policies, rewards and advantages, from TD-Gammon to AlphaGo. Then preference learning from Christiano's 2017 backflips, InstructGPT's three-step recipe where a 1.3 billion parameter model beat 175 billion parameter GPT-3…