Latest / The CTO Podcast with Fexingo: Technical Leadership, Architecture, and Engineering Org / How Netflix Rebuilt Its CDN for 300 Million Subscribers
Transcript
- Lucas: There's a number that's been rattling around in my head all week: Netflix accounts for roughly fifteen percent of global downstream internet traffic at peak. That's not a typo — fifteen percent. And the backbone that carries all that video is their own content delivery network, Open Connect. Luna: And that's not some outsourced AWS CloudFront thing either. They built it themselves, from the hardware up. Lucas: Right. Open Connect started around 2012, when Netflix had about 30 million subscribers and was still using commercial CDNs. But as they grew — and especially as they moved into HD and 4K — the cost and latency of third-party CDNs became unsustainable. So they made a bet: build your own CDN, embed it inside ISP networks, and take control of the last mile. Luna: Embed inside ISP networks — meaning they physically put servers in data centers owned by Comcast, Verizon, Deutsche Telekom, that kind of thing? Lucas: Exactly. Open Connect appliances are essentially storage servers that Netflix gives to ISPs for free. The ISP racks them in their own facilities, connects them to their own backbone, and Netflix fills them with the most popular content. The ISP saves money on transit, Netflix saves on CDN costs, and the user gets lower latency. It's a rare win win win. Luna: I remember reading that by 2015, Netflix was already serving over ninety percent of its traffic through Open Connect. That's a massive shift in just a few years. Lucas: It was aggressive, and it worked. But the real challenge wasn't building the first version — it was scaling from 50 million subscribers to 300 million. And that's where the architecture gets interesting. Luna: So what changed? What broke as they scaled? Lucas: The biggest bottleneck was storage. In the early days, Open Connect appliances used spinning hard drives — big, cheap, 4TB or 8TB drives. But as Netflix added more content — especially 4K HDR streams, which can be 15 to 20 gigabits per hour per title — the storage per appliance needed to grow. And spinning disks have a physical limit: you can only stack so many in a 2U chassis before heat and vibration become problems. Luna: So they moved to SSDs. But that must have been expensive. Lucas: It was. In 2016, Netflix started testing all-flash appliances. The performance difference was dramatic — SSDs could serve the same content with way lower latency and way higher throughput. But the cost per gigabyte was roughly ten times that of spinning disks. So they had to be smart about what they stored on flash versus disk. Luna: Did they end up with a hybrid architecture, or did they go all-flash eventually? Lucas: They went hybrid-first, then eventually all-flash for new deployments. The key innovation was a caching algorithm called 'least recently used with eviction based on content popularity.' Basically, each appliance caches the most popular content locally. But 'popular' isn't static — it shifts by time of day, by region, by device type. So the cache has to adapt. Luna: And that's where the long-tail problem kicks in. Netflix has thousands of titles — but maybe only a few hundred get the bulk of views in a given region on a given night. Lucas: Right. Netflix's own data shows that the top ten percent of content accounts for about eighty percent of views. So the cache hit ratio — the percentage of requests served from cache rather than fetched from origin — is critical. Open Connect appliances aim for a cache hit ratio above ninety-nine percent. Below that, you're backhauling traffic to Netflix's origin servers, which increases latency and cost. Luna: Ninety-nine percent is incredibly high. Most CDNs brag about ninety-five. Lucas: It's partly because Netflix controls both the content and the infrastructure. They can pre-populate caches based on predicted demand. They know when a new season of Stranger Things drops, they know which regions are likely to stream it most. So they can push those bits to the edge before anyone hits play. Luna: I've heard they also do something called 'predictive push' — where they anticipate what you'll watch based on your watch history and pre-cache it. Lucas: That's real. Netflix filed a patent for predictive prefetching back in 2015. The idea is: if you usually watch a show every Tuesday night, the system may pre-cache the next episode on your local Open Connect appliance during off-peak hours. It's not magic — it's pattern recognition applied to infrastructure. Luna: So the appliance itself is basically a smart cache. But what about the hardware? Did they design their own servers? Lucas: They did. Netflix published the hardware specifications for Open Connect appliances — they call them 'reference designs.' The latest generation uses Intel Xeon processors, NVMe SSDs, and custom firmware optimized for sequential reads. They also have a specialized network interface card that handles the TCP offload, so the CPU isn't bogged down managing connections. Luna: And they give these to ISPs for free. That's a big upfront investment. Lucas: It is, but the ROI is clear. Every gigabit of traffic served from an Open Connect appliance instead of a commercial CDN saves Netflix money. And as they've grown, the savings have compounded. By 2019, Netflix estimated that Open Connect saved them over a billion dollars annually in bandwidth costs. Luna: That's a staggering number. And it's not just cost — it's also quality. When you control the edge, you control the user experience. Lucas: Exactly. Netflix's own data shows that Open Connect reduces the time to first frame by roughly fifty percent compared to third-party CDNs. That translates directly into fewer people abandoning a stream because of buffering. Luna: Speaking of the listener experience — if these deep-dive conversations are useful for what you're building or running, the show stays ad-free because a small group of listeners chips in monthly. It's buy me a coffee dot com slash fexingo. No pressure, just a quiet way to keep this going. Lucas: Yeah, and we genuinely appreciate those who do. It lets us keep going deep on topics like this without sponsors dictating the angle. So thank you. Luna: Alright, back to the edge. One thing I find fascinating is that Open Connect appliances aren't just storage — they also run a lightweight web server that handles the HTTP requests. It's like a miniature CDN node in a box. Lucas: Exactly. The software stack is built on top of FreeBSD, and Netflix has contributed heavily to the open-source community around it. The web server is called 'OSS' — it's their own event-driven server designed for high-throughput video delivery. And they've released it as open source. Luna: So any company could, in theory, build their own CDN using Netflix's software? Not that many have the scale to justify it. Lucas: Right, the bar is high. But the software is out there. And the broader lesson is: when you own the stack end to end, you can optimize in ways that general-purpose CDNs can't. For example, Netflix can tailor the TCP congestion control algorithm for video streaming — they use a variant called 'BBR' from Google, but tuned specifically for video. Luna: BBR is designed to avoid bufferbloat. That's a huge win for video, where you want steady throughput, not bursts. Lucas: Precisely. And that's the kind of optimization that only works if you control the server and the client. Netflix also built their own client-side adaptive bitrate algorithm, and they've published research showing how it interacts with Open Connect to reduce rebuffering. Luna: So the real moat isn't just the content — it's the feedback loop between the edge and the client. Lucas: That's the key insight. Every time a user streams a video, the system learns. Which titles are popular in a given region? What time of day does demand spike? Which compression settings work best on which devices? Open Connect feeds that data back into the content encoding pipeline, creating a virtuous cycle of improvement. Luna: And all of that has to work at internet scale. What happens when a new season of a hit show drops? Is there a thundering herd problem? Lucas: That's the worst-case scenario for any CDN. Netflix handles it through a process they call 'flash crowds.' When a highly anticipated title releases, they pre-populate caches as much as possible. But if demand exceeds capacity, the system falls back to origin servers. They've designed the back-end to handle those spikes gracefully — using a tiered architecture where regional caches act as a second layer. Luna: Tiered — meaning there's a hierarchy of caches? Luna: Larger regional caches that serve a broader area, and local ones that serve a smaller region. The local cache is the first line, then the regional cache, then the origin. It's a standard pattern, but Netflix has tuned the eviction policies to minimize origin traffic during flash crowds. Lucas: They also use something called 'cache admission control' — not every piece of content is cached immediately upon request. If a title is requested once, it may not justify cache space. The system waits until a threshold of requests is hit before promoting content to the local cache. That prevents one-hit-wonders from polluting the cache. Luna: That's smart because cache space is effectively a scarce resource, even with SSDs. Every terabyte you dedicate to a rarely-watched show is a terabyte you can't use for 'Squid Game'. Lucas: Exactly. And the cost of cache misses isn't just latency — it's also transit costs. Netflix pays for the bandwidth from their origin servers out to the internet. So every cache miss has a direct dollar cost. That's why they obsess over hit ratios. Luna: What about the future? As Netflix moves into live sports and gaming, does Open Connect need to evolve? Lucas: Live streaming is a fundamentally different challenge. With on-demand, you can pre-cache. With live, you can't. So Netflix has been deploying a separate infrastructure for live events, using a multicast-like approach over HTTP. They've been testing it with events like the Netflix is a Joke comedy festival. And they've published a paper on how they handle live streaming at scale. Luna: So Open Connect is the on-demand backbone, and they're building a parallel system for live? Lucas: That seems to be the direction. And gaming adds another layer — low-latency, stateful interactions. That's a whole other architecture. But the core principle remains the same: own the infrastructure to own the experience. Luna: It's a reminder that even in the cloud era, the biggest players find value in building their own hardware. That's a lesson for any CTO thinking about build versus buy. Lucas: Absolutely. And the threshold for building is lower than most people think. Netflix built Open Connect when they had 30 million subscribers. That's big, but not Google-big. If your infrastructure cost is growing faster than your revenue, it's worth asking: could I take more control? Luna: And if you do, you might end up with a system that's not just cheaper, but better. It's a high-risk, high-reward strategy — and Netflix made it work. Lucas: They did. And the numbers speak for themselves: fifteen percent of global traffic, delivered with sub-second latency, on hardware they designed, running software they wrote. That's the kind of vertical integration that defines the biggest tech companies. Luna: I'd love to see a breakdown of what that means for their carbon footprint. Spinning disks versus SSDs, power draw, cooling... Lucas: That's a great angle for a future episode. For now, I think the takeaway is: infrastructure isn't just a cost center. When done right, it's a competitive advantage.