Shows / The Experimentation Edge
The Experimentation Edge
How do product teams decide what to build and what not to? The Experimentation Edge is the podcast where product, growth, and engineering leaders share how A/B testing, feature flags, and experimentation drive real business outcomes — backed by named companies and real numbers. From DoorDash's 12,000 A/B tests a year to Atlassian's experimentation-led product win to UPS's $500M experimentation team, each episode goes deep with operators running experimentation programs at scale. Hosted by…
Episodes
The Experimentation EdgeHow RealPage finds its biggest A/B test wins, one funnel at a time
Summary Andrew Miller, Director of Product Management, Growth and E-commerce at RealPage, joins host Ashley Stirrup to share how experimentation works inside…
The Experimentation EdgeWhy you're already wrong 75% of the time
Summary In this special episode of The Experimentation Edge, host Ashley Stirrup looks back at lessons from more than 50 guests with two people from…
The Experimentation EdgeHow Preply runs close to 1,000 experiments a year
Summary Sebastien Polis, Senior Experimentation Manager at Preply, joins host Ashley Stirrup to explain how a learning marketplace with 25 to 30 product teams…
The Experimentation EdgeBuilding a Culture of Experimentation in Sports Analytics with New York Red Bulls
Summary Vamsi Krishna Kunapuli, Director of Analytics at New York Red Bulls, leads a small team that supports marketing, ticket sales and partnerships for one…
The Experimentation EdgeMotive's Shiva Manjunath on CRO as a mindset, not a role
Summary Shiva Manjunath, Lead Web Strategist at Motive and host of the From A to B podcast, joins Ashley Stirrup to explain why CRO works best as a mindset…
The Experimentation EdgeEngagement with safety: Headspace's guardrails for AI tests
Summary Mary Alfheim, VP Data and Analytics at Headspace, joins host Ashley Stirrup to explain why more engagement is not always better when your members come…
The Experimentation EdgeHow Walmart Connect learned to align cross-sell with shopper intent
Summary Arv Ramchandani, Senior Product Manager for Experimentation & Analytics at Walmart Connect, runs ad experiments on pages owned by other teams, from…
The Experimentation EdgeWhy more info lifted VidaXL's account creation nearly 40%
Summary Sapna Kashyap, CRO Lead at VidaXL, runs the experimentation program for an e-commerce company operating in more than 30 countries, where product…
The Experimentation EdgeThe auto researcher: How FanDuel finds a winning model in 300 attempts
Summary Michael Mathews, Director of Data and AI Product at FanDuel, runs the teams behind the auto researcher, an agent that takes a data set, an objective…
The Experimentation EdgeWhat Grubhub tests for big product bets before they ship
Summary Grubhub does not run much classic A/B testing. Michal Lenik, Associate Director of Product Design at Grubhub, explains what the company does instead to…
The Experimentation EdgeHow MilliporeSigma lifted add to cart with a smarter search
Summary Dorothy Crepin runs the A/B testing program at MilliporeSigma, the life sciences business of Merck KGaA, where one global catalog has to serve three…
The Experimentation EdgePayPal's $180 million experimentation win
Summary Gaurav Sethi joins Ashley Stirrup on The Experimentation Edge to explain how PayPal's experimentation program went from 700 to 800 tests a year with…
The Experimentation EdgeServiceNow's Customer Zero approach to AI experimentation
Summary ServiceNow runs its own platform on itself. Ashraf Karim, Senior Vice President, Connected Customer Experience, owns everything that happens after a…
The Experimentation EdgeWhy Twilio ships on signals instead of significance
Summary Wanli Lau spent eleven years at Expedia Group, most recently running consumer product analytics on a program that shipped more than 1,500 A/B tests a…
The Experimentation EdgeWhy GoPro stopped judging A/B tests by win rate
Will Guyeskey is Director of Digital Product at GoPro, where his team owns the e-commerce side of gopro.com. Before GoPro he ran personalization at Gap and cut…
The Experimentation EdgeWhat Samsung learned bringing B2C rigor to B2B
Summary What changes when you take a mature B2C experimentation practice and apply it to a B2B storefront? Anuradha Tempe, Lead Product Manager at Samsung…
The Experimentation EdgeEven a loss is a win: Charlie Health's approach to experiments
Summary On this episode of The Experimentation Edge, Ashley Stirrup talks with Joe Yevoli, Director of Growth at Charlie Health, a virtual intensive outpatient…
The Experimentation EdgeRealtor.com on using your AI as a junior data scientist
Summary What do you do when your biggest experiment win turns out to be a loss? Whitney Perez, Director of Product Management at Realtor.com, joins host Ashley…
The Experimentation EdgeHow Zalando connects every experiment to its North Star
Summary How do you keep 1,000 experiments a year pointed at one North Star? In this episode of The Experimentation Edge, host Ashley Stirrup talks with Mi…
The Experimentation EdgeLearneo on testing the opposite of every hypothesis
Summary Rich Liebling, senior director of engineering at Learneo, joins host Ashley Stirrup to explain the practice that came out of growing Shop It To Me from…
The Experimentation EdgeThe four questions Early Warning asks before any A/B test
Summary What separates a valid A/B test from an expensive guess? Priya Singhee, VP of Enterprise Analytics & Data Science at Early Warning — the bank-owned…
The Experimentation EdgeHow Supercell A/B tests 300 million players without breaking trust
Summary On this episode of The Experimentation Edge, Ashley Stirrup talks with Shan Huang, data scientist on the central experimentation team at Supercell, the…
The Experimentation EdgeHow Clover experiments when billions of dollars flow through daily
Summary How do you run an experimentation program when classic A/B testing is off the table? Ben Schein, Director of Product Management at Clover, joins host…
The Experimentation EdgeWhy JobLeads says one test won't move you, but 100 will
Summary What happens when the most logical feature you've ever built has zero impact? In this episode of The Experimentation Edge, host Ashley Stirrup, CMO of…
The Experimentation EdgeWhy The Aspen Group targets a 25% win rate
Summary What does a winning A/B test mean when your website serves 1,100 dentist owned offices? In this episode of The Experimentation Edge, host Ashley…
The Experimentation EdgeHow Principal A/B tests on customers who don't exist
Summary What if you could A/B test on customers who don't exist before spending a single live impression? Erika Dunn, assistant director of data science at…
The Experimentation EdgeWhy US Bank considers missing even 1% of customers unacceptable
Summary How does a major bank scale experimentation when even one percent of customers missing an experience is unacceptable? Vijay Lal, Lead Product Manager…
The Experimentation EdgeWhy Farfetch manages by learning rate, not win rate
Summary Luis Trindade, Principal Product Manager of Experimentation at Farfetch, joins host Ashley Stirrup to explain how one of the world's largest luxury…
The Experimentation EdgeHow Cogniteer Built an Experimentation Engine From Scratch
Summary In this episode of The Experimentation Edge, host Ashley Stirrup talks with Fabian Hans, founder and behavioral psychologist at Cogniteer, a…
The Experimentation EdgeHow Fin A/B Tests Millions of Samples in Days
Summary On this episode of The Experimentation Edge, host Ashley Stirrup talks with Pedro Tabacof, Principal Machine Learning Scientist at Fin (formerly…
The Experimentation EdgeHow Kargo turns losing experiments into competitive edges
Summary In this episode of The Experimentation Edge, host Ashley Stirrup, CMO of GrowthBook, sits down with James Falzone, Director of Product Management at…
The Experimentation EdgeThe 'wine effect' and other surprises that reshaped how Box runs e-commerce experiments
Summary In this episode of The Experimentation Edge, host Ashley Stirrup talks with Danielle Olean, Director of E-commerce at Box, about what it really takes…
The Experimentation EdgeDiligent explains why moving on from an experiment might cost you
Summary Dan Layfield, Director of Product Management at Diligent, joins host Ashley Stirrup on The Experimentation Edge to trace what fifteen years of A/B…
The Experimentation EdgeThe metric Stitch Fix says every experimenter should chase
Summary In this episode of The Experimentation Edge, GrowthBook CMO Ashley Stirrup sits down with Nick Beyler, data science manager at Stitch Fix, where he…
The Experimentation EdgeWhat the Expedia Group cannot measure, it cannot ship
Summary Amir Moghaddam, Director of Software Engineering at Expedia Group, joins host Ashley Stirrup on The Experimentation Edge to make the case that…
The Experimentation EdgeHow Fin went from weeks to hours of analysis using AI
Summary In this episode of The Experimentation Edge, host Ashley Stirrup sits down with Raunak Kumar, senior manager of GTM analytics at Fin (formerly…
The Experimentation EdgeInside The Home Depot's experimentation at a $25B scale
Summary What does experimentation look like inside a $150 billion retailer? In this episode of The Experimentation Edge, host Ashley Stirrup talks with Kim…
The Experimentation EdgeHow Disney picks which experiments to run
Summary What does it look like to kill a multimillion dollar feature before anyone builds it? In this episode of The Experimentation Edge, host Ashley Stirrup…
The Experimentation EdgeShip faster, measure better: experimentation tips from JPMorgan Chase
Summary How do you know if the thing you just shipped actually worked? On this episode of The Experimentation Edge, host Ashley Stirrup, CMO of GrowthBook…
The Experimentation EdgeTwitch on why false negatives kill product ideas
Summary How do you make a high-stakes product decision when the safe choice is to never test it at all? In this episode of The Experimentation Edge, host…
The Experimentation EdgeSquarespace killed its blank template and built something better
Summary What do you do when your big launch increases engagement and tanks conversion? On this episode of The Experimentation Edge, host Ashley Stirrup talks…
The Experimentation EdgeSignet Jeweler's "View All" page made more money by showing less
Summary Craig Kistler, VP of Experience Design, Personalization, and Experimentation at Signet Jewelers (the parent company of Kay, Jared, Zales, Peoples, and…
The Experimentation EdgeRingCentral's DART framework: The four metrics that actually measure AI agents
Summary RingCentral's Director of Product Management for AI Products, Mayank Agarwal, joins host Ashley Stirrup to dismantle the metrics most teams use to…
The Experimentation EdgeThe 2% close rate increase that turned Ford Credit's product teams into believers
Summary On this edition of The Experimentation Edge, Ashley Stirrup talks with Geoffrey Bell, Experimentation Product Specialist at Ford Credit, about building…
The Experimentation EdgeAtlassian on the talent product turnaround from A/B testing
This episode of The Experimentation Edge explores how A/B testing, feature flags, and user research transformed Atlassian's talent product after it failed with…
The Experimentation EdgeHow DoorDash saved millions with one A/B test
This episode of The Experimentation Edge unpacks how DoorDash's experimentation platform runs 12,000+ A/B tests per year across 42 million monthly active users…
The Experimentation EdgeHow UPS generated half a billion from 80+ apps with A/B testing
This episode of The Experimentation Edge shows how UPS's A/B testing program drove $500M+ in incremental revenue across 80+ customer-facing applications. Dave…
The Experimentation EdgeHow Experimentation Led to Annual Growth at Fanatics
Summary Most e-commerce companies test a handful of features each month. Fanatics runs nearly 100 experiments monthly and delivers a big portion of the…
The Experimentation EdgeInside Chess.com's Plan to Run 1,000 Experiments in a Single Year
Summary Chess.com ran its first A/B test in 2023. Two years later, the team is on track to run 1,000 experiments in a single year—and they've already shipped…
The Experimentation EdgeAncestry on AI Storytelling, Paywalls, and Decision Quality
Summary What happens when A/B testing stops being a tool and becomes your operating system? Suresh Teckchandani, VP of Product & Technology at Ancestry…
The Experimentation EdgeFyxer's engineering playbook to go from $1M to $35M ARR
Summary How do you drive hypergrowth without guessing? Kameron Tanseli, Head of Growth Engineering at Fyxer—an AI assistant for your email—breaks down the…
The Experimentation EdgeThe 5 pixels that cost LinkedIn a million dollars a month
Summary How do you build a culture where nothing ships without evidence—and leaders actually act on the data? Makram Mansour, Head of Marketplace at ID.me and…
The Experimentation EdgeTruist on shipping faster and more safely with AI and human-in-the-loop banking
How do you boost developer velocity in a highly regulated industry—without sacrificing safety or customer trust? Charles Williams, Senior Vice President and…
The Experimentation EdgeFrom chatbots to open-world agents at Microsoft: evals, go-live metrics, and copilot velocity
AI is moving so fast that what “good” looked like a few months ago is already outdated. So how do you measure value, ship safely, and scale what works? Marco…
The Experimentation EdgeUpwork on AI-Driven Ops at scale
Summary What do you test rigorously—and what do you ship fast and fix forward—when every change could impact millions? Vinoj Kumar, Vice President of…
The Experimentation EdgeHow Moxie Pest Control boosted conversions 5% with data and coaching
Think pest control isn’t a digital business? Think again. Raj Mehta, Vice President of Product and Technology at Moxie Pest Control, outlines how he turned a…
The Experimentation EdgeTypeform on how to stop running experiments and start earning them
If 80% of A/B tests fail, how do you de-risk decisions that touch pricing, product, and brand? Aleksandra (Aleks) Bass, Chief Product & Technology Officer at…