Peter Gostev | On benchmarking the frontier, and why human accountability remains key | #101
In this episode of The Only Constant, Lasse Rindom speaks with Peter Gostev, AI Capability Lead at Arena and creator of BullshitBench, an open-source benchmark that tests whether language models can recognise nonsensical questions rather than confidently inventing an answer. Returning as the podcast’s first solo repeat guest, Peter joins Lasse for a wide-ranging conversation on what has changed in AI, from model capabilities and coding agents to the organisational realities of accountability, governance and an overwhelming volume of AI-generated work.
Main topics they discuss include:
Why…
The skinny
The skinny isn't ready yet — notes appear once the transcript is processed.