Latest / MBA Training Data / Churn models in telecom: why uplift beats accuracy
Transcript
- 0:05Host: Welcome back to Leaders Insights. How did Telefónica build a churn model that actually moved retention numbers?, and why it matters this week.
- 0:12Expert: The dashboard flagged a customer as high-risk churn — high risk of leaving — the morning after she'd already ported her number to a competitor. We caught her twelve hours too late, and that one screen told me everything wrong with how telcos do retention.
- 0:27Host: And that's what we're digging into today — how Telefónica built a churn model that actually moved the retention numbers, instead of just producing pretty charts. Start with the mistake everyone makes.
- 0:38Expert: They predict churn the week it happens. By then the customer's already emotionally gone — she's price-shopped, she's texted her friend on another network, she's halfway out. A model that's accurate at that point is a coroner, not a doctor.
- 0:51Host: That's a nice line, but Telefónica's models weren't magic either. They spent years before anything worked. What were those years actually for?
- 1:00Expert: Accumulating boring signals. Not the sexy ones. Everyone wants to model complaints and call-center sentiment. Telefónica got leverage from things like degraded data speed in a specific cell tower, payment timing drift, the gap between a plan and actual usage. Small betrayals that stack up months before someone cancels.
- 1:20Host: Degraded speed on one tower predicts churn? That sounds like the kind of correlation that falls apart in production.
- 1:27Expert: It falls apart if you treat it as a standalone feature — a single input to the model. It holds when you combine it with contract stage and competitor pricing in that postcode. The insight wasn't one variable. It was that network quality is a retention problem, not just an engineering problem, and those two teams never talked to each other.
- 1:47Host: Let me push on the timeline. "Years of accumulating signals" sounds like an excuse a CDO gives the board when the model's late.
- 1:54Expert: Sometimes it is. But here's the honest version — most of that time wasn't modeling, it was plumbing. dbt Labs, who make data transformation tooling, reckon analysts spend well over half their time just cleaning and reshaping data before they model anything. Worth noting they sell the thing that fixes that, so take the exact figure with salt — but every CDO I know nods when they hear it. The model was the easy part. Making the data trustworthy every single day was the slog.
- 2:27Host: So where do other telcos consistently fall short? Be specific.
- 2:31Expert: They build a model that's accurate and never ask if it's actionable. There's a difference. MIT Sloan Management Review has written about this gap — plenty of organizations have analytics they can't act on because the prediction arrives at someone with no authority or budget to do anything about it. Telefónica wired the score into the retention offer at the moment of risk. The agent sees it, and has a pre-approved thing to offer. No approval chain.
- 3:00Host: You said predicting churn the week it happens is useless. But an early prediction means you're spending retention money on people who might've stayed anyway. Isn't that just burning margin?
- 3:10Expert: Yes, and that's the second thing everyone gets wrong. They optimize for accuracy — catching every leaver. The number that matters is uplift — did the intervention actually change the outcome versus doing nothing. Some customers were going to stay regardless; discounting them is pure waste. Some were going to leave no matter what; discounting them is also waste. The money's in the movable middle, and most telcos can't even tell you how big that middle is.
- 3:39Host: How do you find the movable middle without running experiments that annoy real customers?
- 3:43Expert: You hold back a control group — a slice you deliberately don't contact — and measure the difference. It feels insane to let customers churn on purpose, and every revenue VP fights it. But without it you're flying blind on whether your retention spend does anything. Telefónica ran those holdouts. That's why their numbers moved and were believed.
- 4:04Host: Give me the one thing a CDO does Monday morning.
- 4:07Expert: Pull your last retention campaign and ask one question — did you have a control group? If the answer's no, you don't have a churn program, you have a discount program with a data science logo on it. Fix that before you touch the model.
- 4:21Host: A coroner with a logo. We'll end there.
- 4:24Host: Sources for today's episode: KDnuggets, MIT Sloan Management Review, The New Stack, Towards Data Science, dbt Labs (vendor — data tooling).
- 4:34Host: Done. Want to know where you actually stand? Take the CDO self-assessment at mba-training.com.