Latest / The CTO Podcast with Fexingo: Technical Leadership, Architecture, and Engineering Org / How Netflix Rebuilt Its CDN for 4K Streaming
Transcript
- Lucas: Netflix today delivers more than 15 percent of all downstream internet traffic globally. That's not a typo — fifteen percent. And most of that traffic is 4K HDR streaming, which can run at 25 to 40 megabits per second per stream. Luna: So for a popular show like a new season of Stranger Things, you're talking about terabits per second leaving their data centres simultaneously. Lucas: Exactly. And that's why Netflix doesn't just rely on AWS or a traditional CDN like Akamai or CloudFront. They built their own content delivery network called Open Connect. And I think the engineering story there is one of the most interesting edge infrastructure projects of the past decade. Luna: What was the trigger? Was it cost, or control? Lucas: Both, but control was the bigger factor. Around 2012, Netflix was using a mix of third-party CDNs. But they realised that for a premium streaming experience — especially as they moved into 4K — they needed direct control over the last mile. They couldn't let a third party decide which peering path to use or how aggressively to cache content. Luna: So they started putting their own hardware inside ISP networks. Lucas: Right. The Open Connect Appliance is a custom server that Netflix designed in-house and then gives to ISPs for free. The ISP plugs it into their network, and Netflix pre-populates it with the most popular content. The appliance runs a heavily tuned FreeBSD operating system and is essentially a giant SSD array with a very fast NIC. Luna: How much storage and throughput per box? Lucas: The current generation — they call it the Open Connect Appliance, or OCA, Gen 7 — has about 100 terabytes of SSD storage and can push 90 gigabits per second of sustained throughput. That's enough to serve roughly 3,000 simultaneous 4K streams from a single 2U box. Luna: And the ISP doesn't pay anything for that hardware? Lucas: Not a cent. Netflix foots the bill for manufacturing and shipping. In exchange, the ISP agrees to peer with Netflix directly — usually via a zero-cost peering agreement. That saves Netflix money on transit costs, and it saves the ISP money on backhaul because traffic stays local. It's a win-win. Luna: So Netflix is essentially subsidising the ISP's edge infrastructure to guarantee quality of service for their own subscribers. That's a fascinating business model wrapped in an engineering solution. Lucas: And listeners of this show often tell us they appreciate hearing about these kinds of real-world architecture trade-offs. If these conversations are useful for what you're building or running, there's a simple way to help keep them coming. A small group of listeners already chip in monthly over at buy me a coffee dot com slash fexingo, and that support literally funds producing as many of these as we do — no ads, no sponsors, just the content. So if you find value in it, that's the place. Luna: Yeah, it really helps us keep the show independent and focused on the technical details that matter. Anyway — back to the appliances. How does Netflix decide what to cache on each box? Lucas: That's the clever part. They use predictive algorithms. Netflix knows, based on your viewing history, watch time, and even the time of day, what you're likely to watch next. So before you even click play, the content is already sitting on the OCA in your ISP's data centre. They call this proactive caching. Luna: So it's not just the most popular titles globally — it's personalised per region, per ISP, and even per neighbourhood. Lucas: Exactly. They model demand at a very granular level. For example, a new anime series might be huge in Japan but barely watched in Brazil. The OCAs in Japan will cache that series, while the ones in Brazil won't waste space. And they update these predictions continuously — every few minutes, the machine learning models re-evaluate and reshuffle what's stored. Luna: What about the long tail? Niche documentaries or old catalog titles that only get a handful of streams per day? Lucas: Those come from the origin — Netflix's own data centres in places like Virginia and Oregon, plus some AWS regions. But they try to keep as much traffic local as possible. Currently, over 95 percent of Netflix's streaming traffic is served from these edge appliances. Only about 5 percent comes from the origin. Luna: That's an incredible hit rate. How do they handle software updates across thousands of appliances in hundreds of ISPs? Lucas: They built a custom management plane that sits on top of FreeBSD. Each appliance phones home to a control server, which pushes configuration changes and software updates. Because the appliances are stateless from a storage perspective — the content is purely cached and can be re-populated — they can safely reboot or upgrade without affecting availability. Luna: And the hardware itself — is it off-the-shelf or fully custom? Lucas: It's a mix. The Gen 7 appliance uses standard Intel Xeon processors and commodity SSDs, but the motherboard and chassis are custom-designed by Netflix to optimise airflow, power efficiency, and density. They also use a custom NIC that offloads a lot of the network processing to hardware, which is critical for hitting that 90 Gbps number. Luna: I'm curious about the software stack beyond FreeBSD. Did they build a custom HTTP server? Lucas: They did. They use a custom user-space TCP stack and an in-house HTTP server called GALA — it stands for Global Application Layer Accelerator. It's optimised for video streaming, which means it handles things like range requests, adaptive bitrate switching, and connection pooling very efficiently. They've open-sourced some of the components, like the FreeBSD kernel patches. Luna: And the peering logistics — how do they get the appliances into ISPs? I imagine some ISPs are reluctant to take free hardware. Lucas: It's a negotiation. Netflix has a team that works directly with ISPs to demonstrate the value: reduced transit costs, better customer experience, lower churn. The appliance comes with a service-level agreement from Netflix — if it fails, Netflix replaces it within 24 hours. Most ISPs agree because the alternative is that their subscribers have a poor streaming experience and blame the ISP, not Netflix. Luna: So Netflix essentially outsourced the last mile to the ISPs but owns the hardware and the software that runs on it. It's like a franchise model. Lucas: Exactly. And they've taken it a step further with what they call Open Connect for Content Providers. Other companies — like Amazon Prime Video — can now use Netflix's appliance design and peering relationships to deliver their own content. It's a separate programme, but it leverages the same infrastructure. Luna: That's interesting. So the CDN becomes a platform. Are there any downsides to this approach? Lucas: The biggest downside is that you need to have enough scale to justify building your own CDN. Netflix's volume is enormous — they ship millions of appliances and manage hundreds of gigabits per second of traffic. For a smaller streaming service, it wouldn't make economic sense. They'd be better off using a commercial CDN. Luna: Also, if an ISP goes down, the appliances inside that ISP become unreachable. There's no automatic failover to a different CDN provider. Lucas: Right. Netflix does have backup paths — they can serve from the origin or from OCAs in adjacent ISPs — but it's not as seamless as a multi cdn strategy. They've accepted that risk because the performance benefits outweigh it. Luna: How has the architecture evolved with the shift from 1080p to 4K and now to HDR? Higher bitrates must have stressed the system. Lucas: Definitely. When they started Open Connect, the average stream was about 5 megabits per second. Now with 4K HDR, it's 25 to 40. That means each appliance can serve fewer simultaneous streams. So they've had to increase throughput per box — from 40 Gbps on Gen 5 to 90 Gbps on Gen 7 — and also increase cache density to store the larger files. Luna: And the encoding side — they've built their own codec optimisations too, right? Lucas: Yes. Netflix is heavily invested in the AV1 codec, which gives about 30 percent better compression than H.265. They've also developed per-title encoding, where each movie or show is analysed and given a custom encoding ladder — not a one-size-fits-all bitrate profile. That helps reduce bandwidth without sacrificing quality. Luna: So the CDN and the encoding pipeline are co-designed. The encoding team reduces the bitrate, and the CDN team caches more content per gigabyte. Lucas: Exactly. It's a system-level optimisation. And they release public data about the performance. For example, in 2025, they published a paper showing that Open Connect appliances achieved a median cache hit rate of 99.5 percent during peak hours. That's exceptional. Luna: What about the environmental impact? Running thousands of appliances 24/7 has to draw a lot of power. Lucas: Netflix has made commitments to reduce their carbon footprint. The appliances are designed for power efficiency — they use low-power CPUs and SSDs instead of spinning disks. They also participate in ISP efficiency programmes where the appliances can enter low-power modes during off-peak times. And because they reduce the need for long-haul transit, they actually help the internet overall be more energy-efficient. Luna: That's a good point. Local caching avoids sending data across the backbone, which saves energy at the core network level. Lucas: Right. And they've open-sourced the hardware design and software stack, which has allowed other companies to build similar systems. That's a contribution to the broader internet infrastructure community. Luna: So if I'm a CTO at a company considering building a private CDN, what's the key takeaway from Netflix's approach? Lucas: First, don't do it unless you have massive scale. Second, invest in predictive caching — knowing what to cache is more important than the hardware. Third, own the full stack from codec to appliance. And fourth, build partnerships with network operators. The technology is important, but the business relationships are what make it possible. Luna: One last thing — what's next for Open Connect? Any clues about Gen 8? Lucas: They're rumoured to be working on appliances that support 400 Gbps NICs and using NVMe over fabrics to pool storage across multiple boxes. There's also talk of integrating compute — like transcoding at the edge — so that the appliance can adapt content on the fly to different device capabilities. But Netflix hasn't confirmed anything publicly. Luna: So the edge keeps getting smarter. It's a fascinating evolution from a simple cache to an intelligent delivery node. Lucas: Absolutely. And that's the direction a lot of edge infrastructure is heading — not just storing data, but processing it too. Netflix's Open Connect is a blueprint for that future.