EP 129 : AI Benchmarks Questioned: Study Claims Tech Bias + Microsoft Phi Models, Decentralized AI
Today, we dive deep into a new study questioning the leading AI benchmark, LMArena, with claims that it might give unfair advantages to major tech companies like Meta, Google, and OpenAI. Are AI model rankings being distorted? Plus, we explore Microsoft's impressive new small reasoning models in the Phi family, designed for efficiency on phones and laptops. Get updates on Claude's new research upgrades and integrations, FutureHouse's 'AI scientist' agents, and an ambitious experiment building the world's first user-owned, decentralized LLM. We also cover other key AI news and highlight some…
The skinny
The skinny isn't ready yet — notes appear once the transcript is processed.