Latest / The CMO Podcast with Fexingo: Marketing Leadership, Budgets, and Executive Strategy / How CMOs Are Using AI to Forecast Churn Before It Happens
Transcript
- Lucas: So Luna, I want to start with a number: twenty-two percent. Luna: Twenty-two percent — that's a churn reduction number, I'm guessing? Lucas: Exactly. A mid-market SaaS company I've been following — about two hundred employees, five thousand accounts — deployed a churn prediction model eighteen months ago. Within two quarters, they cut voluntary churn by twenty-two percent. Not by changing their product or slashing prices. By using machine learning to flag accounts that were about to leave, sometimes weeks before the customer even knew they were unhappy. Luna: That's the promise, right? The crystal ball. But I've seen plenty of churn models that just... sit there. Lucas: That's the key distinction. Most churn models fail not because the algorithm is wrong, but because the data feeding it is stale or too narrow. This company didn't just throw login frequency and support tickets into a spreadsheet. They built what's called a customer health score — a composite of product usage intensity, support sentiment from ticket transcripts, payment timing patterns, and even changes in how many users at the account are active. Luna: So we're talking natural language processing on support tickets, plus behavioral data. That's a bigger lift than a lot of marketing teams realize. Lucas: It is. But the model they landed on wasn't some cutting-edge neural net. It was gradient boosting — XGBoost, specifically. Interpretable, fast, and it handled the imbalanced data well. Only about five percent of their accounts churn in a given quarter, so you need a model that doesn't just predict 'no churn' for everyone and get ninety-five percent accuracy. Luna: Right, the classic trap. And gradient boosting is still a black box for most marketers. How did they sell the C-suite on it? Lucas: They didn't lead with the algorithm. They led with a pilot: they ran the model on historical data from the prior twelve months and showed that if they had acted on the top twenty percent of at-risk accounts, they would have saved about three hundred and fifty accounts. That translated to roughly one point two million dollars in retained annual recurring revenue. The CFO didn't ask what algorithm they used. Luna: Which is the right way to pitch it. But once the model is live, you have to act on the signals. What did the marketing team actually do differently? Lucas: That's the part most case studies gloss over. They set up three tiers. Tier one — accounts flagged as red, highest risk — got a direct call from a customer success manager within twenty-four hours, plus a personalized offer: a free onboarding refresher or a one-on-one strategy session. Tier two — orange, moderate risk — got an automated email sequence from a senior account manager, not a generic newsletter. Tier three — yellow, low risk — just got a check-in from the support team. The key was that the intervention was tailored to the risk level and the specific signals the model flagged. Luna: So the model told them not just who was at risk, but why. That's the difference between a score and an action plan. Lucas: Exactly. For example, if the model flagged an account because usage had dropped off but support tickets were still low, the intervention was a product training session. If the flag was driven by negative sentiment in tickets, the intervention was a service recovery call. The model provided the diagnosis; the team prescribed the treatment. Luna: I want to push on the cost side. Building and maintaining a model like that — even a gradient boosting model — requires a data engineer, maybe a data scientist. That's a two hundred thousand dollar annual investment for a mid-market company. Does the math still work? Lucas: In their case, yes. They calculated that retaining a hundred accounts pays for the entire analytics team. And they saved three hundred and fifty. But you're right — for a smaller company, say fifty employees, the economics are tighter. That's where off-the-shelf tools come in. Platforms like Totango or Gainsight offer pre-built health scores that don't require a custom model. You lose some precision, but you gain speed. Luna: And you don't need a PhD to interpret the output. That's important for a CMO who's trying to build a data-driven culture without scaring the team. Lucas: It also solves a common problem: most churn models predict churn based on data that's already a week old. Real-time scoring — where the model updates the health score every time a user takes an action — requires streaming infrastructure. This company didn't have that. They scored their accounts once a day, overnight. And that was good enough. Luna: Once a day — that's a realistic starting point. Weekly is probably fine for a lot of B2B companies with longer sales cycles. The risk is over-indexing on recency and burning out the team with false alarms. Lucas: Speaking of false alarms — there's an ethical angle here too. What happens when the model flags a customer who's not actually at risk? You intervene, maybe offer a discount they didn't ask for, and suddenly you've trained them to expect a discount whenever they reduce usage. That can actually increase churn in the long run. Luna: Right, the classic 'crying wolf' problem. And there's the privacy side: if you're using sentiment analysis on support calls, are customers aware their conversations are being analyzed? Most terms of service allow it, but it's a trust question. Lucas: This company handled it by being transparent — they added a line in their privacy policy and in the support call preamble that said 'we analyze call sentiment to improve service.' They also built in a human review loop: any account flagged as red had to be manually reviewed by a senior CSM before the intervention went out. The model was a recommendation, not a decision. Luna: That's the right governance model. I think the broader lesson for CMOs is: churn prediction is not a one-time project. It's a muscle that has to be exercised. The model degrades as customer behavior changes, as the product changes, as the market shifts. You have to retrain it quarterly at minimum. Lucas: And that's where most organizations fall off. They build the model, get the initial win, and then six months later the model is predicting on last year's patterns. The company I mentioned — they have a monthly model refresh cycle. Every first Monday, the data science team retrains and validates. It's baked into their sprint. Luna: That's disciplined. And it's probably why they got the twenty-two percent reduction instead of a one-time bump. Lucas: Exactly. And you know, it's funny — we're talking about a hundredth episode today, and this topic really captures something about the show. We've spent ninety-nine episodes digging into how CMOs are using new tools and tactics, and at the core, it always comes back to a few fundamentals: understanding your customer, acting on data, and building systems that improve over time. Luna: And a willingness to invest in the infrastructure, not just the campaign. Which brings me to something quick — Lucas: Go ahead. Luna: We talk a lot about investing in marketing technology on this show. And the reason we can keep having these conversations without a paywall or ads is that a small group of listeners chips in monthly through buy me a coffee dot com slash fexingo. It's not a big ask — it's just the honest economics of keeping this ad-free and independent. Lucas: Yeah, and we don't ever want to sound like we're selling something. But if these episodes have given you an idea you've actually implemented, or saved you time figuring out a tool, that's the kind of value that a few dollars a month feels proportional to. Totally optional, obviously. Luna: And we're grateful either way. Back to the model — one thing I've been wondering: how do you measure the ROI of a churn prediction model versus just doing better customer success? Lucas: Great question. The standard metric is retained revenue attributed to the model. You compare the churn rate of accounts the model flagged as high risk against a control group that wasn't scored. But that's tricky because you can't ethically withhold an intervention from a high-risk account. So most companies use a time-series analysis: before versus after model deployment, controlling for seasonality. Luna: And what about the counterfactual — maybe the customer success team just got better at their jobs over the same period? Lucas: That's the hardest confound to eliminate. The company I mentioned actually ran a shadow test for three months: the model scored accounts, but the scores weren't shared with the CS team. They just logged what the model would have predicted. After the quarter, they compared the predicted churn to actual churn. The model's precision was about seventy-eight percent at identifying churners. That gave them confidence. Luna: Seventy-eight percent precision is solid for a first-generation model. And it's explainable enough to get buy-in. Lucas: Exactly. And that's the real takeaway for CMOs: you don't need a perfect model. You need a model that's directionally correct and that your team trusts enough to act on. Perfection is the enemy of progress here. Luna: So for a CMO listening right now, what's the first step? Should they go build a model, or start with a health score spreadsheet? Lucas: Start with the spreadsheet. Seriously. Pick your top hundred accounts by revenue. List the data points you think matter — login frequency, support tickets, payment timeliness, feature adoption. Score each one manually on a one-to-ten scale. You'll quickly see patterns. That manual exercise teaches you what data matters before you spend on a model. Then, if you see clear signals, graduate to a tool or a custom model. Luna: I like that. Low risk, high learning. And it gets the team thinking in terms of prediction instead of reaction. Lucas: That shift from reactive to predictive is really what we're talking about. And it's one of the highest-leverage investments a marketing org can make.