Shows / The Information Bottleneck
The Information Bottleneck
Two AI Researchers - Ravid Shwartz Ziv, and Allen Roush, discuss the latest trends, news, and research within Generative AI, LLMs, GPUs, and Cloud Systems.
Episodes
The Information BottleneckDan Card: AI Hasn’t Changed Cybersecurity As Much As You Think
Dan Card has spent more than 26 years in cybersecurity as a penetration tester, defender and architect. He helped organizations respond to some of the biggest…
The Information BottleneckWhy Video Games Still Don’t Use Modern AI | Julian Togelius (NYU)
Julian Togelius, an NYU professor and co-founder of modl.ai, is living the dream of every kid who grew up as a computer geek and gamer. He’s been doing AI…
The Information BottleneckWhat’s Left for Mathematicians? | Daniel Litt
In this episode, we talked with Daniel Litt, a mathematician at the University of Toronto. In March 2025, Daniel bet that AI would not be able to autonomously…
The Information BottleneckThe Political Economy of AI: Governance, Bias, and the Future | Andy Hall (Stanford/Anthropic)
In this episode of the the information bottleneck podcast, we talked with Andy Hall, a Stanford professor and MTS at Anthropic. The conversation explores the…
The Information BottleneckJohn Platt (Google): AI for Climate and Science
John Platt is a Google Fellow and leads the Applied Science team at Google Research, the group working on climate and science. He's also the person behind SMO…
The Information BottleneckYuandong Tian on Recursive Self-Improvement
Yuandong Tian spent many years at Meta FAIR and recently left to co-found Recursive Superintelligence, a company building AI that improves itself. In this…
The Information BottleneckChris Manning: Language Is the Real Unlock for Intelligence
Chris Manning is a legend in NLP. He's a professor of linguistics and computer science at Stanford and ran the Stanford AI Lab. His course CS224N, Natural…
The Information BottleneckWhy AI Still Can't See wtih Andrew Dai
Andrew Dai spent over a decade at Google Brain and DeepMind, where he co-wrote the 2015 paper that introduced language model pre-training followed by…
The Information BottleneckCan AI Replace Doctors? Zachary Lipton on the Future of Healthcare and AI
Zachary Lipton is an associate professor at Carnegie Mellon University and a co-founder of Abridge, a healthcare AI company (and a jazz saxophonist!). His…
The Information BottleneckSara Hooker on the End of Static AI
What comes after scaling? We talk with Sara Hooker, co-founder and CEO of Adaptation Lab, about why the next generation of AI may look very different from…
The Information BottleneckTiny Recursive Models Beat the Giants - Alexia Jolicoeur-Martineau (Microsoft)
Alexia Jolicoeur-Martineau is a Principal Researcher at Microsoft and the author of "Less is More: Recursive Reasoning with Tiny Networks," the paper behind…
The Information BottleneckContinual Learning Is the Next Bottleneck | Rohan Anil (Core Automation )
Rohan Anil spent eleven and a half years at Google, where he went from writing memory allocators to large-scale linear solvers, then optimization at Google…
The Information BottleneckWorld Models | John Langford (Microsoft AI Labs)
John Langford, one of the heads of Microsoft's AI Labs, the creator of Vowpal Wabbit, and a co-inventor of CAPTCHA, joins us to talk about world models…
The Information BottleneckWhich Tabular Model Should You Actually Use? | David Holzmüller (INRIA)
Description Tabular data is still where most of machine learning actually happens in industry, and the field has changed a lot in the last few years. In this…
The Information BottleneckWhy You Can't Just Rent 1,000 GPUs | Charles Frye (Modal)
Charles Frye (Modal, ex-Weights & Biases, Berkeley PhD) joins Ravid and Allen to explain why modern AI research is bottlenecked by compute, and why simply…
The Information BottleneckStella Biderman (EleutherAI) - Open Source, AI Safety, and Who We Can Trust
Stella Biderman, Executive Director of EleutherAI, joins us the week an OpenAI model autonomously broke out of its sandbox and hacked Hugging Face. Stella…
The Information BottleneckWhy Deep Learning Finally Works on Tables | Frank Hutter (Prior Labs)
In this episode, Frank Hutter joins us to talk about TabPFN and why tabular data is suddenly the hottest problem in deep learning. Frank is a professor at the…
The Information BottleneckSurya Ganguli: The Physics of Intelligence
Surya Ganguli is a professor at Stanford and VP at General Catalyst, working at the intersection of physics, neuroscience, and AI. He started in string theory…
The Information BottleneckText Diffusion Models with Brendan O'Donoghue (Google DeepMind)
Brendan O'Donoghue, research director at Google DeepMind, makes the case for text diffusion as a real alternative to autoregressive generation. He walks…
The Information BottleneckNathan Lambert: Inside Post-Training and the Open Model Fight
Nathan Lambert spent three years as post-training lead at Ai2, where he built the OLMo models, and he writes Interconnects, one of the most-read technical…
The Information BottleneckDaphne Koller - The Future of AI in Biology and Drug Discovery
Daphne Koller wrote the book that many of us learned probabilistic graphical models from, founded Coursera, and now runs insitro, which is trying to make drug…
The Information BottleneckRL Was Broken at Every Level - With Joseph Suarez (PufferAI)
In this episode, Joseph Suarez from PufferAI explains why he thinks RL never had an algorithm problem, but it had a code problem. Every part of the standard RL…
The Information BottleneckThe Model Found a Way Out - with Florian Brand (Prime Intellect)
Florian Brand builds evals at Prime Intellect. The premise of the conversation is that writing a benchmark is the easy part now. Keeping the model from…
The Information BottleneckPierre-Carl Langlais on Building Models from Data You Can Account For
Most labs build language models by scraping the web and filtering afterward. Pierre-Carl Langlais runs it the other way around. At Pleias, the French-German…
The Information BottleneckDhruv Batra: The Browser Is a Robotics Problem - From Embodied AI at Meta to Web Agents at Yutori
Dhruv Batra spent years leading Embodied AI at Meta, training virtual robots to navigate photorealistic 3D scans of real buildings with pure reinforcement…
The Information BottleneckHow to Turn Research Into Billion-Dollar Companies, with Ion Stoica
Ion Stoica has done what almost no academic ever does — repeatedly turned university research into billion-dollar companies. He co-founded Databricks (now…
The Information BottleneckKaggle Grandmasters, Agent Skills, and Why Everyone Is Overfitting with Jean-Francois Puget (NVIDIA)
Jean-Francois Puget is a Director and Distinguished Engineer at NVIDIA, where he leads the Kaggle Grandmasters team, and he's ranked third on Kaggle's all-time…
The Information BottleneckAI Agents and The Golden Age of Asking Questions with Dimitris Papailiopoulos (MSR/UW-Madison)
In this episode, we talked with Dimitris Papailiopoulos, researcher at Microsoft Research's AI Frontiers lab and professor at the University of Wisconsin…
The Information BottleneckWhy All Models Learn the Same Thing with Phillip Isola (MIT)
Phillip Isola, professor at MIT, joins us to talk about representation learning: what makes a representation good, why different models seem to converge on…
The Information BottleneckAI for Science with Qichao Hu (Molecular Universe / SES AI)
Most AI-for-science companies are selling shovels. Qichao Hu wants the gold. In this episode, we talk with Qichao, the founder and CEO of Molecular Universe…
The Information BottleneckInfrastructure for AI at Scale - With Benny Chen (Fireworks AI)
We talk a lot on this show about RL, agents, and the move between pre-training and post-training, but not enough about the layer everything actually runs on…
The Information BottleneckBroken Peer Review, AI, and Worms — with Oded Rechavi
Oded Rechavi is a biologist at Tel Aviv University and the co-founder of QED, a company building AI to review scientific work. He's also spent years studying…
The Information BottleneckWill AI Take Our Jobs? With Alex Imas (Google/University of Chicago)
Will AI take our jobs? We put the question to Alex Imas, the new Director of AGI Economics at Google DeepMind and a professor at Chicago Booth, whose entire…
The Information BottleneckWhy AI Benchmarks Are Lying to You - with Wenhu Chen (Meta/University of Waterloo)
In this episode, we sit down with Wenhu Chen, research scientist at Meta MSL, assistant professor at the University of Waterloo, and the person behind MMLU-Pro…
The Information BottleneckJürgen Schmidhuber - Part 2: JEPA, the Road to AGI, and Who Really Invented Modern AI
In the second half of our conversation with Jürgen Schmidhuber, we focus on the key ideas he's pursued since the early 1990s and discuss why he believes these…
The Information BottleneckJürgen Schmidhuber - World Models, RL, and the Year that changed AI (Part 1)
In this episode, we host Jürgen Schmidhuber - the man, the legend, one of the godfathers of modern AI. His lab worked out many ideas behind today’s systems…
The Information BottleneckAI for Science and the Thermodynamics of Generative AI - with Max Welling (UvA, CuspAI)
In this episode, we sit with Max Welling, Professor of Machine Learning at the University of Amsterdam, co-founder and CTO of CuspAI, and a foundational figure…
The Information BottleneckAfter Math Falls, What's Next? with Julia Kempe (NYU/Meta)
Julia Kempe on Why Math Will Fall Next, Superhuman Provers, and the Return of the Renaissance Researcher In this episode, we sit down with Julia Kempe, a…
The Information BottleneckIntelligence in an Open World - with Mengye Ren (NYU)
We talk with Mengye Ren, Assistant Professor at NYU's Center for Data Science, about what intelligence actually means once you step outside a benchmark, and…
The Information BottleneckLanguage, Cognition, and the Limits of LLMs - with Tal Linzen (NYU/Google)
We host Tal Linzen, Associate Professor at NYU and Research Scientist at Google, for a conversation on the intersection of cognitive science and large language…
The Information BottleneckThe Principles of Diffusion Models - with Jesse Lai (Sony AI)
We host Chieh-Hsin (Jesse) Lai, Staff Research Scientist at Sony AI and visiting professor at National Yang Ming Chiao Tung University, Taiwan, for a…
The Information BottleneckInside xAI, and the Bet on AI Math - with Christian Szegedy (Math Inc)
We talked with Christian Szegedy, co-inventor of Inception and Batch Normalization, founding scientist at xAI, now at Math Inc, about what it takes to build a…
The Information BottleneckReasoning Models and Planning - with Rao Kambhampati (Arizona State)
We sat down with Rao Kambhampati, a Professor of CS at Arizona State University and former President of AAAI, to talk about reasoning models: what they are…
The Information BottleneckWhat Actually Matters in AI? - with Zhuang Liu (Princeton)
In this episode, we hosted Zhuang Liu, Assistant Professor at Princeton and former researcher at Meta, for a conversation about what actually matters in modern…
The Information BottleneckThe Future of Coding Agents with Sasha Rush (Cursor/Cornell)
We talked with Sasha Rush, researcher at Cursor and professor at Cornell, about what it actually feels like to we in the heart of the AI revolution and build…
The Information BottleneckThe Hidden Engine of Vision with Peyman Milanfar (Google)
How Denoising Secretly Powers Everything in AI Peyman Milanfar is a Distinguished Scientist at Google, leading its Computational Imaging team. He's a member of…
The Information BottleneckReinventing AI From Scratch with Yaroslav Bulatov
Yaroslav Bulatov helped build the AI era from the inside, as one of the earliest researchers at both OpenAI and Google Brain. Now he wants to tear it all down…
The Information BottleneckWhy Healthcare Is AI's Hardest and Most Important Problem with Kyunghyun Cho (NYU)
We talk with Kyunghyun Cho, who is a Professor of Health Statistics and a Professor of Computer Science and Data Science at New York University, and a former…
The Information BottleneckDiffusion LLM & Why the Future of AI Won't Be Autoregressive - Stefano Ermon (Stanford /Inception)
In this episode, we talk with Stefano Ermon, Stanford professor, co-founder & CEO of Inception AI, and co-inventor of DDIM, FlashAttention, DPO, and…
The Information BottleneckTraining Is Nothing Like Learning with Naomi Saphra (Harvard)
Naomi Saphra, Kempner Research Fellow at Harvard and incoming Assistant Professor at Boston University, joins us to explain why you can't do interpretability…
The Information BottleneckEP28: How to Control a Stochastic Agent with Stefano Soatto (VP AWS/ Pro. UCLA)
Stefano Soatto, VP for AI at AWS and Professor at UCLA, the person responsible for agentic AI at AWS, joins us to explain why building reliable AI agents is…
The Information BottleneckEP27: Medical Foundation Models - with Tanishq Abraham (Sophont.AI)
Tanishq Abraham, CEO and co-founder of Sophont.ai, joins us to talk about building foundation models specifically for medicine. Sophont is trying to be…
The Information BottleneckEP26: Measuring Intelligence in the Wild - Arena and the Future of AI Evaluation
Anastasios Angelopoulos, Co-Founder and CEO of Arena AI (formerly LMArena), joins us to talk about why static benchmarks are failing, how human preference data…
The Information BottleneckEP25: Personalization, Data, and the Chaos of Fine-Tuning with Fred Sala (UW-Madison / Snorkel AI)
Fred Sala, Assistant Professor at UW-Madison and Chief Scientist at Snorkel AI, joins us to talk about why personalization might be the next frontier for LLMs…
The Information BottleneckEP24: Can AI Learn to Think About Money? - with Bayan Bruss (Capital One)
Bayan Bruss, VP of Applied AI at Capital One, joins us to talk about building AI systems that can make autonomous financial decisions, and why money might be…
The Information BottleneckEP23: Building Open Source AI Frameworks: David Mezzetti on TxtAI and Local-First AI
David Mezzetti, creator of TxtAI, joins us to talk about building open source AI frameworks as a solo developer - and why local-first AI still matters in the…
The Information BottleneckEP22: Data Curation for LLMs with Cody Blakeney (Datology AI)
Cody Blakeney from Datology AI joins us to talk about data curation - the unglamorous but critical work of figuring out what to actually train models on…
The Information BottleneckEP21: Privacy in the Age of Agents with Niloofar Mireshghallah
Guest: Niloofar Mireshghallah (Incoming Assistant Professor at CMU, Member of Technical Staff at Humans and AI) In this episode, we dive into AI privacy…
The Information BottleneckEP20: Yann LeCun
Yann LeCun – Why LLMs Will Never Get Us to AGI "The path to superintelligence - just train up the LLMs, train on more synthetic data, hire thousands of people…
The Information BottleneckEP19: AI in Finance and Symbolic AI with Atlas Wang
Atlas Wang (UT Austin faculty, XTX Research Director) joins us to explore two fascinating frontiers: the foundations of symbolic AI and the practical…