Latest / The CTO Podcast with Fexingo: Technical Leadership, Architecture, and Engineering Org / How Notion Cut Notification Noise by 70 Percent
Transcript
- Lucas: So, Notion just shared a deep dive on how they cut notification noise by seventy percent, and the numbers behind it are pretty striking. Engagement on their notifications went up, and they actually saw a dip in churn. Luna: Seventy percent is huge. How did they even begin to tackle that without just turning everything off? Lucas: Well, the core insight was that notifications weren't one-size-fits-all. They had users drowning in updates they didn't care about, while others missed critical mentions. So they built a preference-based system, letting people choose what they wanted to be notified about, and then layered a machine learning model on top to predict those preferences automatically. Luna: So they're not just putting the burden on the user to configure everything. The system learns what matters to them. Lucas: Exactly. The model looks at past behavior, like how often you open a page or react to a comment, and it adjusts the notification frequency and type. It's a really elegant approach because it respects the user's attention while still keeping them informed. Luna: I imagine that also means fewer people muting everything or uninstalling the app out of frustration. Lucas: Right, and that's where the business case comes in. They saw a measurable drop in churn, and people were engaging more with the notifications they did get. It's a classic win-win: users are happier, and the product becomes stickier. Luna: It makes me wonder, did they have to change their engineering culture to support this? I mean, building a recommendation system for notifications is a big shift. Lucas: It definitely required a mindset shift. Instead of just adding more notification triggers, product and engineering had to think about notification quality over quantity. They even had to set up experiments to measure sentiment, not just click-through rates. Luna: So it's not just a technical fix, it's a philosophical one. You're redefining what a notification is for. Lucas: Exactly. And that's something many teams struggle with because it's tempting to use notifications to drive immediate engagement metrics. But Notion's data shows that less can be more. Luna: I'd love to hear more about the machine learning part. How did they train the model without a ton of labeled data? Lucas: They used a combination of explicit user feedback, like snoozing or muting specific notifications, and implicit signals, like how quickly someone opens a notification. It's a classic collaborative filtering approach, similar to what Netflix does for recommendations, but applied to notification preferences. Luna: That's clever. So the more you interact, the better the model gets at predicting what you want to see. Lucas: Precisely. And they were careful about cold-start problems for new users, falling back to default preferences that are conservative and then quickly adapting as the user interacts. Luna: And this whole thing has a direct impact on the bottom line. Lower churn means more recurring revenue. Lucas: That's the beautiful part. When you solve a user pain point, the metrics often follow. They saw a reduction in churn, which for a subscription product like Notion is huge. Luna: It's a great example of how investing in user experience can be a growth strategy. Lucas: Yeah, and it's something more companies should consider. Notifications are often an afterthought, but they're a direct line to your users' attention. Getting that right can make or break the product experience. Luna: Before we go deeper into the technical details, I want to mention something. Conversations like this on Fexingo Business are ad-free, and that's because of listeners who support the show. A couple of dollars a month is genuinely what keeps these going — buy me a coffee dot com slash fexingo, if you've gotten something out of them. Lucas: Yeah, it really does make a difference. Every little bit helps us keep bringing you these deep dives. And speaking of deep dives, let's talk about how Notion actually implemented the machine learning pipeline. Lucas: So, they built a feature store that tracks user interactions and notification outcomes. This feeds into a model that's retrained regularly to adapt to changing user behavior. Luna: Retraining on a schedule or based on drift detection? Lucas: I believe they use a combination — scheduled retraining plus monitoring for performance degradation. It's a robust approach because user preferences can shift over time, especially as they use the product differently. Luna: And how did they handle the latency? I imagine you can't have a model call on every notification trigger. Lucas: Good point. They pre-compute a lot of the scores in batch jobs, so the actual decision of whether to send a notification is fast and doesn't add noticeable delay. They also use feature caching to keep it snappy. Luna: That makes sense. It's a classic trade-off between real-time personalization and system performance. Lucas: Absolutely. And they also had to think about the user's control. Even with the model, they give users the ability to override and customize, which builds trust. Luna: So it's not a black box. You can still take the wheel if you want. Lucas: Right, and that's important for adoption. If users feel like the system is making decisions for them without their input, they'll rebel. Notion's approach balances automation with transparency. Luna: I'm curious about the impact on team productivity. If notifications are more relevant, does that mean fewer distractions and better focus? Lucas: That was actually one of the stated goals. By cutting the noise, they're helping users get into flow state more often. And they saw a correlation with increased daily active usage, which suggests people are spending more quality time in the app. Luna: That's a strong signal. It's not just about opening the app; it's about meaningful use. Lucas: Exactly. And this is a lesson that applies beyond Notion. Any product that relies on notifications to drive engagement can learn from this playbook. Luna: Do you think this approach could backfire if a product's user base is very diverse? Like, different segments might have very different preferences. Lucas: That's a real concern, but the model actually handles that well because it's learning individual preferences. It might even uncover segments you hadn't thought of. The key is to have enough data and a flexible model. Luna: So it's almost like a personalization engine that scales. Lucas: Yeah, and Notion's case shows it can be done without a massive team. They had a small squad dedicated to this, and they were able to achieve a seventy percent reduction in notification volume. Luna: That's impressive. It makes me wonder if they shared any specific metrics on user satisfaction or net promoter score. Lucas: They mentioned that notification sentiment scores improved significantly, and they saw a reduction in users disabling notifications altogether. That's a strong vote of confidence. Luna: It's a smart move, especially in a market where attention is the scarcest resource. Lucas: For sure. And it's a reminder that sometimes the best feature is the one that removes friction, not adds it. Luna: Well said. Any final thoughts on what other product teams should take away from this? Lucas: I'd say start by auditing your notification types and ask yourself: is this helping the user or just our metrics? Then be willing to experiment and let data guide you. Notion's success is proof that less can be more.