Latest / The CTO Podcast with Fexingo: Technical Leadership, Architecture, and Engineering Org / How Pinterest Rebuilt Its Recommendation Engine for 500 Million Users
Transcript
- Lucas: So Pinterest rebuilt its entire recommendation engine. Not a tweak, not an A/B test on the ranking layer — a ground-up rewrite of how they figure out what you might want to pin next. Luna: That's a big deal. Pinterest has what, north of 500 million monthly active users now? And their whole value prop is discovery — showing you stuff you didn't know you wanted. Lucas: Exactly. And for years they ran on a fairly standard collaborative filtering pipeline. User-item matrix, some matrix factorization, batch-precomputed recommendations that refreshed every few hours. It worked, but it had two fundamental problems. Luna: Let me guess: stale recommendations and the cold-start problem. Lucas: Bingo. If a user pinned something new, the system wouldn't reflect that for hours. And a brand-new pin with zero interactions? Essentially invisible. So around 2023 they started working on a graph-based approach, and by mid-2025 they rolled it out to production. The project is called PinSage. Luna: PinSage — I've read the papers. It's a graph neural network that learns embeddings from the bipartite graph of pins and boards. But going from paper to production at Pinterest scale is a different beast. Lucas: Completely different. The graph has billions of nodes — each pin is a node, each board is a node, each user is a node — and tens of billions of edges. Training a GNN on that graph is a distributed systems problem in itself. They used a technique called random-walk sampling to generate mini-batches, because you can't load the whole graph into memory. Luna: Right, they sample neighborhoods around each node. So instead of looking at the full graph, they take a few random walks from the target node and aggregate features from the nodes they encounter. That's how they keep the computation tractable. Lucas: And they do it in a distributed way using TensorFlow with a custom data pipeline. But the really interesting part is how they moved from batch to near real time inference. In the old system, embeddings were recomputed nightly. In PinSage, embeddings for new pins are computed on the fly — within seconds of the pin being created. Luna: That must have required a massive shift in infrastructure. How did they handle the latency requirement? Lucas: They deployed the trained model on TensorFlow Serving, running on Kubernetes, with GPUs for inference. They optimized the model to run in under 50 milliseconds per query. The key insight was that they didn't need to recompute the entire graph embedding every time — they could cache the embeddings for stable nodes like popular boards and only compute new ones for fresh pins. Luna: So the hot path is fast, and cold starts are handled with a fast approximate embedding. What about the user side? When a user pins something, does that immediately update their recommendations? Lucas: It does. The user's action triggers an update to their user embedding in real time. They use a streaming pipeline based on Apache Kafka and Flink to process pin events and feed them into the inference service. So if you pin a recipe for sourdough bread, within seconds your home feed starts showing more baking content. Luna: That's a huge improvement in user experience. But it also means the infrastructure cost must have gone up significantly. Batch precomputation is cheap; real-time inference at 500 million users is not. Lucas: It did increase cost, but Pinterest claims the engagement lift more than justified it. They saw a 15 percent increase in re-pin rate and a 10 percent increase in session time. When you're an ad-supported platform, more engagement means more revenue. And they also saved some cost by no longer running those massive nightly batch jobs. Luna: Right, the batch jobs required a huge Spark cluster that sat idle during the day. Now they have a more balanced resource utilization with the Kubernetes cluster handling inference around the clock. Plus, they can scale down during low-traffic periods. Lucas: And there's another clever piece: the graph structure itself captures relationships that collaborative filtering misses. For example, if two pins are often saved to the same board, they're likely related even if they have no common users. That's especially powerful for long-tail content. Luna: That makes sense. In a collaborative filtering model, a niche pin with few interactions is hard to recommend. But in a graph model, it can be connected to a board that has many pins, and that board's embedding can propagate to the new pin. It's a form of inductive learning. Lucas: Exactly. And that's been a game-changer for content creators on Pinterest. Smaller creators with niche content suddenly get discovered because their pins land in the right boards. Pinterest actually published a case study showing that pins from creators with fewer than 100 followers saw a 25 percent increase in impressions after PinSage rolled out. Luna: That's a strong signal. So the system isn't just better for users, it's better for the supply side too. But I'm curious about the operational side — how do they monitor and debug a graph-based recommendation system? Traditional A/B testing doesn't always translate well. Lucas: They built a suite of offline metrics — recall@k, mean reciprocal rank, normalized discounted cumulative gain — but they also track online metrics like engagement and diversity. One thing they found was that the graph model could sometimes create echo chambers, recommending very similar content repeatedly. So they added a diversity penalty in the ranking layer. Luna: So the graph generates candidates, but there's a second stage that reranks them with business rules. That's a common pattern in recommendation systems. Lucas: Right. PinSage produces hundreds of candidate pins, then a lightweight gradient-boosted tree model reranks them based on freshness, diversity, and user preferences. That reranking model is updated daily, so it can adapt quickly to trends without retraining the entire graph. Luna: It's fascinating how they split the problem. The GNN captures complex structural relationships, but it's expensive to retrain. The reranking model is cheap and fast, so it can react to short-term signals. That's a smart architectural trade-off. Lucas: And it's a pattern you see in many production ML systems. The first stage is high-recall, the second stage is high-precision. Pinterest's contribution is making the first stage work at their scale with a graph neural network. Luna: Speaking of scale, I imagine the infrastructure team had to deal with some interesting challenges. Like, how do you serve a TensorFlow model on Kubernetes with GPUs and keep latency under 50 milliseconds? Lucas: They used TensorFlow Serving with model versioning and A/B testing baked in. They also implemented client-side caching — the top recommendations for a user are cached for a few minutes, so not every page load hits the model. That cut the inference load by about 60 percent. Luna: Smart. And for the cold-start case for entirely new users with no pins yet — how do they handle that? Lucas: They use a heuristic based on sign-up data: which boards they followed, which categories they chose. That initial embedding is rough, but as soon as they pin one thing, the embedding updates. Within three pins, the model is as effective as for users with a long history. Luna: That's impressive. It means the onboarding experience is good from day one, which is critical for retention. If a new user sees irrelevant pins, they're gone. Lucas: Exactly. And Pinterest has been public about their retention metrics improving after PinSage. They cited a 12 percent increase in day-7 retention for new users. Luna: Alright, so we've covered the graph, the real-time pipeline, the cold start, the monitoring. But there's one thing I'm still wondering about — how did they manage the migration from the old system to the new one without disrupting the user experience? Lucas: That's a great question. They ran the two systems in parallel for six months. Initially, they served recommendations from the old system and logged what PinSage would have recommended. They compared offline metrics, then started A/B testing with small fractions of traffic. They gradually ramped up to 100 percent over several months. Luna: So they had to maintain two infrastructure stacks simultaneously. That's expensive and operationally complex. Lucas: It is, but it's the safest way. They also built a shadow evaluation pipeline that could replay old traffic through PinSage and compare results. That helped them catch regressions before they affected users. Luna: It sounds like they did it right. And now they have a recommendation engine that's not only more effective but also more adaptable. I wonder if other platforms with similar scale — like Etsy or even Amazon — are looking at this approach. Lucas: They probably are. Graph neural networks are becoming more common in production recommendation systems. Pinterest's open-source release of parts of PinSage has definitely accelerated that. The paper and some of the code are available, so other teams can build on their work. Luna: That's the kind of engineering culture that moves the whole industry forward. And it's why we cover stories like this — because the architecture decisions made at Pinterest today might influence how your favorite apps recommend content tomorrow. Lucas: Absolutely. For anyone building or running a recommendation system, the PinSage case study is a masterclass in balancing model complexity with operational reality. And if you're finding these deep-dive conversations useful for what you're building, it's worth knowing that listener support is what keeps this show ad-free and focused on the engineering details. Luna: Yeah, it makes a real difference. If you'd like to chip in, there's a page at buy me a coffee dot com slash fexingo. No pressure, but every bit helps us keep bringing you stories like this. Lucas: Right. So, back to the graph architecture — one more detail I want to highlight. The PinSage model uses multiple layers of graph convolutions, but they found that two layers was the sweet spot. More layers captured too much global structure and diluted local signals. Luna: That makes sense. For recommendation, the immediate neighborhood — boards and pins directly connected to a user's pins — is more informative than distant connections. Two hops captures the board structure and the co-occurrence patterns. Lucas: Exactly. And they also use attention mechanisms in the aggregation function, so when combining features from neighboring nodes, the model learns which neighbors are more important. That's a nice touch that improved accuracy by a few percent. Luna: Attention in GNNs is pretty standard now, but applying it at Pinterest scale with billions of edges is still non-trivial. They must have done some serious engineering to make the training efficient. Lucas: They used a technique called neighbor sampling with importance weighting to reduce the fan-out. Instead of using all neighbors, they sample a fixed number — say, 50 — weighted by their PageRank score. That keeps the computation bounded. Luna: So they're essentially doing a biased random walk that prioritizes more influential nodes. That's clever. Lucas: It is. And it means the model can be trained on a single machine with multiple GPUs using data-parallel training. They didn't need a massive distributed training cluster, which kept the infrastructure cost manageable. Luna: That's a good reminder that even at Pinterest scale, you don't always need a supercomputer. Smart algorithmic choices can make training feasible on commodity hardware. Lucas: Exactly. And that's the theme of this whole story — Pinterest made deliberate, intelligent trade-offs at every layer: the graph sampling, the two-stage inference, the caching, the migration strategy. It's a textbook example of how to modernize a core ML system without breaking the bank or the user experience. Luna: Well said. And it leaves me wondering — what's next for Pinterest's recommendation system? Are they looking at multi-modal models that incorporate image features directly? Lucas: They've already started. In the latest version of PinSage, they incorporate visual embeddings from a convolutional neural network trained on pin images. So the model can understand that two pins are similar because they look alike, even if they're on different boards. That's especially useful for fashion and home decor. Luna: That's the natural evolution — from collaborative to content-based to hybrid. And with the graph structure, they can combine both signals seamlessly. Lucas: Right. And I think that's where the industry is heading. The line between recommendation and search is blurring, and systems like PinSage are at the forefront of that convergence. Luna: Great episode. I learned a lot about graph neural networks in production. Thanks, Lucas. Lucas: Thanks, Luna. And thanks to our listeners for sticking with us through the architecture details. Until next time.