Latest / The Newsletter Podcast with Fexingo: Substack, Beehiiv, and Email Newsletter Business / How Newsletters Use Predictive Churn Scoring to Retain Subscribers
Transcript
- Lucas: So you're a newsletter writer. Your open rates have been drifting down for three weeks. Not crashing — just a few percentage points each issue. You tell yourself it's seasonal. And then one morning you wake up and forty people unsubscribed overnight. Luna: That sinking feeling. And by the time you see the churn, it's already happened. You can't undo it. Lucas: Right. The whole game is catching those people before they hit the unsubscribe button. And increasingly, newsletter writers are borrowing a technique from SaaS companies called predictive churn scoring — building a simple model that flags subscribers who look like they're about to leave. Luna: I've seen this pop up in the Beehiiv community especially. People are running basic lead-scoring logic right inside their email platform. How does it actually work at the newsletter level? Lucas: It's honestly not as complex as it sounds. The core idea is that you assign a point value to subscriber behaviors that correlate with eventual churn. Things like — did they open the last three emails? If no, add five points to their risk score. Did they click anything in the past thirty days? If no, add another ten. You sum those up, and anyone above a certain threshold gets flagged. Luna: So you're basically building a health score. Every subscriber starts at zero and accumulates points for risky signals. Lucas: Exactly. One writer I talked to runs a daily newsletter on venture capital — about fifty thousand subscribers. He built a five-signal scoring model inside Beehiiv's automation engine. The signals were: didn't open any of the last five emails, didn't click any link in the last fourteen days, subscribed more than sixty days ago but never engaged, submitted a spam complaint — even a single one — and finally, opened an email but closed it in under three seconds. Luna: The three-second open is interesting. That's a signal for 'I glanced at the subject line and decided not to read.' Lucas: Exactly. It's a negative engagement. So he weighted that at fifteen points. The spam complaint was an automatic red flag — thirty points. Anyone who hit forty points total got moved into a 'high churn risk' segment. And then instead of just letting them drift away, he triggered a re-engagement sequence. Luna: What did the sequence look like? Lucas: Three emails over ten days. The first one was a 'we miss you' style message with a survey — just two questions: 'What topics do you want more of?' and 'Is there anything I'm doing that's annoying?' The second email offered a content upgrade — a free PDF summary of his top five posts from the quarter. The third was a last-chance email that essentially said 'I'll assume you want off the list unless you click this link to stay.' Luna: And the results? Lucas: Over three months, he recovered about eighteen percent of the flagged subscribers — they started opening again. Another fifteen percent actively confirmed they wanted to stay via that last-chance click. But crucially, the ones who didn't respond — he removed them. So his list got smaller, but his open rates went up by six percentage points and his churn rate dropped by twenty-two percent. Luna: So you're trading list size for list health. Most writers I know are terrified of losing subscribers, even the dead weight. Lucas: And that's the biggest mental shift. Predictive churn scoring forces you to accept that some subscribers are better off gone. They're dragging down your deliverability by not opening, and they're distorting your engagement metrics. If a hundred thousand people never open a single email, you don't have a hundred thousand subscribers. You have a hundred thousand liabilities. Luna: Yeah, because email providers like Gmail and Outlook look at your overall engagement rate. If too many people ignore you, your emails start landing in spam for the people who actually want them. Lucas: That's the deliverability angle — and it's a strong one. But there's also a more subtle benefit. Once you have a scoring model, you can start testing interventions. Maybe you try a different send time for high-risk segments. Or you test a subject line that's more benefit-driven versus curiosity-driven. You can measure whether those interventions actually reduce the risk score over time. Luna: If today was actually useful to you — especially if you're a newsletter writer thinking about implementing something like this — the way these conversations stay ad-free is listener support. It's buy me a coffee dot com slash fexingo. Just a simple way to keep the show independent. Lucas: Yeah, and we really mean it — we don't run sponsors, we don't do affiliate links. The coffee link is it. So if you got something out of this episode, that's where you can help us keep going. Luna: Okay, back to the model. One thing I wonder about is the threshold. How do you decide that forty points is the cutoff? Is it just a guess? Lucas: It doesn't have to be. You can back-test it. If you have historical data — who unsubscribed in the past six months — you can look at what their behavior was in the thirty days before they left. Calculate the average score at the point of churn. For that VC newsletter writer, the average was around thirty-eight points. So he set the threshold at forty to catch people slightly before they hit that average. Luna: So it's a moving target. You'd want to re-calibrate every few months as your audience changes. Lucas: Ideally quarterly. Because the signals can shift. Maybe you change your content format and suddenly people who never opened are opening again. Their risk score drops organically. You don't want to flag them based on old data. Luna: What about writers on Substack? Their analytics are more limited than Beehiiv's. Can they still do this? Lucas: It's harder but possible. Substack gives you open and click data per subscriber, but you can't build automated scoring workflows natively. What some writers do is export their subscriber list weekly, run a scoring formula in a spreadsheet, and then manually import a segment label back. It's clunky, but it works. Luna: I've also seen writers use a third-party tool like Mailmeteor or Sheetsu to connect Substack data to Google Sheets and then use Zapier to tag subscribers. It's a workaround, but it's doable if you're technical. Lucas: And there's a simpler version that doesn't need a full model. Just track one signal: time since last open. If someone hasn't opened in thirty days, send them a re-engagement email. That single-signal approach won't catch everyone, but it can recover maybe ten to fifteen percent of at-risk subscribers. Luna: Which is still meaningful. For a ten thousand subscriber list, that could be a thousand people you save. Lucas: Exactly. And the cost is basically zero. You're just setting up one automated email. So there's no excuse not to do at least that. Luna: But I want to push back on something. Isn't there an ethical risk here? If you're scoring people and trying to keep them engaged, where's the line between helpful nudging and manipulation? Lucas: That's a really fair question. I think the line is in the intent and the transparency. If you're sending a re-engagement email that says 'We've noticed you haven't opened lately, is everything okay?' — that's respectful. But if you're using psychological tricks like fake urgency or guilt-tripping, you've crossed into manipulation. Luna: And also, if someone has clearly disengaged — like they haven't opened in three months — at some point you have to respect that silence. Not everyone wants to be 'saved.' Lucas: Right. That's why the last-chance email is important. It gives them a clear opt-out. You're not trying to trick them into staying. You're offering a final chance to say 'yes, I'm still interested.' If they don't, you remove them. That's fair. Luna: I think the newsletter writers who do this best are the ones who frame it as a service to their subscribers. 'I want to make sure I'm only sending you stuff you actually want.' That's a win-win. Lucas: It also builds trust. If I receive a 'we miss you' email from a newsletter I used to like, I'm more likely to re-engage than if I just get ignored. It shows they care about the relationship, not just the number. Luna: So what's the one takeaway for a writer who wants to start today? What's the simplest thing they can do by the end of the week? Lucas: Export your subscriber list. Sort by 'last opened' date. Anyone who hasn't opened in thirty days — create a segment. Then send them a single re-engagement email with a survey or a valuable piece of content. Track how many click and how many unsubscribe. That's your baseline. From there, you can start adding more signals. Luna: And if you want to get fancy, add a second signal — like 'last clicked' — and give it double weight. That alone will catch most of the at-risk subscribers. Lucas: Exactly. Predictive churn scoring doesn't have to be a machine learning model. It can be a simple spreadsheet. What matters is that you're proactively identifying the people who are slipping away — and giving them a reason to stay. Luna: I love that. It's a mindset shift from reactive to proactive. And the data is already there, sitting in your analytics. You just have to look at it differently. Lucas: That's the whole episode, really. You don't need a data science team. You need a willingness to ask: who is about to leave, and what can I offer them right now?