Latest / The CTO Podcast with Fexingo: Technical Leadership, Architecture, and Engineering Org / How Netflix Rebuilt Its Encoding Pipeline for Bandwidth Savings
Transcript
- Lucas: One of the quietest engineering wins in the streaming world is how Netflix rebuilt its encoding pipeline to deliver the same video quality at roughly half the bandwidth. Not a new codec — an algorithm change. Luna: Half the bandwidth? That's not incremental. That's a rewrite of a core assumption. Lucas: Exactly. And the core assumption they challenged was this old idea that you encode a movie at a single bitrate — say 5 megabits per second — and that's your HD stream. Netflix realized that different scenes need different bitrates. Luna: So a static dialogue scene versus a fast-action explosion — they shouldn't be encoded the same way. Lucas: Right. A blank wall barely needs any bits. An explosion with smoke, sparks, rapid camera movement — that's a bitrate hog. So Netflix built what they call per-title encoding optimization. They analyze each movie, each episode, and find the minimum bitrate needed to maintain a given quality level. Luna: That's basically a custom encoding profile for every piece of content. Doesn't that explode the encoding compute cost? Lucas: It does. But they made a trade-off: more compute during encoding to save bandwidth during delivery. And bandwidth is the expensive recurring cost — especially for mobile users and markets with infrastructure constraints. Lucas: Let me give you a concrete number. Before this change, Netflix's 1080p stream averaged around 5.8 megabits per second. After per-title encoding, they dropped that to about 3.5 megabits for the same VMAF score — that's a video quality metric they developed internally. Luna: VMAF — Video Multimethod Assessment Fusion. They open-sourced that, right? Lucas: They did. And it's key to the whole system. VMAF lets them measure quality in a way that correlates with human perception, not just peak signal to noise ratio. So they can set a target VMAF score and then the encoder hunts for the lowest bitrate that hits it. Luna: That's a feedback loop — quality target, then encode, measure, iterate. Lucas: Exactly. And they took it further. They now use per-shot encoding — not even per-title. They break each video into shots, which are contiguous sequences from a single camera angle, and encode each shot independently. A two-hour movie might have 2,000 shots. Luna: So a static shot of a sunset gets a very different bitrate than a quick-cut action sequence. Lucas: Yes. And they do this with machine learning models that classify shots by complexity. The model predicts the optimal encoding parameters for that shot given the content type. It's not brute force — it's a learned model trained on millions of encoded clips. Luna: This is where the engineering gets really satisfying. They're not just applying a better codec. They're applying intelligence to an old problem. Lucas: And the codec story is interesting too. Netflix started with H.264, moved to VP9 and HEVC, and is now rolling out AV1. Each codec gives better compression, but per-title encoding layers on top of that. The two are multiplicative — AV1 with per-shot encoding gets them down to maybe 1.5 megabits for 1080p on some content. Luna: That's a 75% reduction from the original 5.8. Lucas: Right. And for 4K, the savings are even more dramatic because the dynamic range is wider. A dark scene in 4K can be encoded at a fraction of the bitrate of a bright scene, because the human eye is less sensitive to detail in shadows. Luna: So the algorithm can exploit perceptual blind spots. That's clever — give the bits where the eye actually sees them. Lucas: That's the whole philosophy. And it's not just about saving Netflix money on CDN costs. It's about enabling streaming in places where bandwidth is scarce. In India, for example, Netflix has a mobile-only plan that streams at 480p. With per-title encoding, they can deliver a 480p stream that looks like 720p on a phone screen. Luna: Because the phone screen is small and the viewing distance is close — the compression artifacts are less visible. Lucas: Exactly. They optimize for the viewer's context. And this is where the engineering meets business strategy. Better compression means lower server load, lower CDN costs, and happier users on constrained connections. Luna: It's also a moat. If you're a smaller streaming service, you probably don't have the engineering resources to build this kind of custom pipeline. Lucas: That's a good point. Netflix open-sourced VMAF and shared some of their encoding techniques in tech blog posts, but the full pipeline — the shot detection, the ML model, the integration with their encoding farm — that's proprietary. And it's hard to replicate. Luna: So they get a quality advantage without having to own the last mile. Lucas: Which brings up a broader topic — when should a company build custom infrastructure versus using off-the-shelf solutions? Netflix chose to build because compression is core to their product. Their entire business is streaming video. If they can deliver a better picture at lower cost, that's a direct competitive advantage. Luna: For a company like a bank or a retailer, it might not make sense to build your own encoding pipeline. But for Netflix, it's table stakes. Lucas: Right. And the thing is, they didn't do it all at once. They started with per-title, then per-shot, then ml driven shot classification. Each step was an incremental improvement that compounded over years. Luna: That's a lesson in itself — don't wait for the perfect solution. Start with one improvement, measure it, then iterate. Lucas: Absolutely. And if these kinds of deep dives into how real engineering teams solve hard problems are useful for what you're building or running, we hope you find them valuable. We deliberately don't run ads on these conversations because we want them to be as direct and uninterrupted as the thinking behind them. If you want to support that approach, the link is buy me a coffee dot com slash fexingo. Luna: Yeah, it's a small gesture that helps us keep the show exactly the way we want it — no sponsors, no scripts, just the real story. Lucas: So back to Netflix — one of the interesting knock-on effects of per-shot encoding is that it changes how they think about content creation too. If your encoding pipeline can handle high-complexity shots more efficiently, you might encourage filmmakers to use more dynamic camera work without worrying about the bandwidth cost. Luna: That's a feedback loop — the technology influences the art. Lucas: And that's where engineering becomes a creative partner. Netflix has a dedicated team that works with content producers to optimize the encoding for specific titles. For example, for the show 'The Crown', they used higher bitrates for the ornate costumes and sets because the detail mattered. Luna: So they're not just applying a blanket algorithm. They're treating each title with some nuance. Lucas: Exactly. And that nuance is what separates a good encoding pipeline from a great one. It's not just about the technical metrics — it's about the viewing experience. And that's a lesson that applies beyond streaming. Luna: Where else do you see that kind of optimization thinking? Lucas: I think about how Google optimized its search index — not just for speed, but for relevance. Or how Amazon optimizes its warehouse robot paths. It's the same principle: understand the real objective, then measure and optimize against that, not the proxy metric. Luna: Netflix optimized against VMAF, not bitrate. Bitrate is a cost, VMAF is the experience. Lucas: That's the key. And as engineers, we often optimize what's easy to measure — latency, throughput, cost — instead of what matters. Netflix's encoding story is a reminder to step back and ask: what are we really trying to deliver? Luna: And then build the measurement system to capture that. Lucas: Exactly. So if you're a CTO or a senior engineer listening, take a look at your own pipeline. What proxy metric are you optimizing that might be masking the real user experience? There might be a Netflix-style win hiding in plain sight. Luna: And you don't have to be Netflix to do it. You can start small — pick one metric, one flow, and see if you can improve it by a factor of two. Lucas: That's the takeaway. Thanks for listening, and we'll see you next time.