Latest / The Information Bottleneck
EP26: Measuring Intelligence in the Wild - Arena and the Future of AI Evaluation
Anastasios Angelopoulos, Co-Founder and CEO of Arena AI (formerly LMArena), joins us to talk about why static benchmarks are failing, how human preference data actually works under the hood, and what it takes to be the "gold standard" of AI evaluation.
Anastasios sits at a fascinating intersection - a theoretical statistician running the platform that every major lab watches when they release a model. We talk about the messiness of AI-generated code slop (yes, he hides Claude's commits too), then dig into the statistical machinery that powers Arena's leaderboards and why getting evaluation…
The skinny
The skinny isn't ready yet — notes appear once the transcript is processed.