Latest / The CTO Podcast with Fexingo: Technical Leadership, Architecture, and Engineering Org / Why Netflix Rewrote Its Own Content Delivery Network
Transcript
- Lucas: If you ask most engineers what Netflix runs on, the answer is almost always AWS. And that's partly true — for control plane stuff, for recommendations, for the backend logic. But the actual video bits that reach your screen? Those have almost nothing to do with the cloud. Luna: Right, because streaming ten million simultaneous 4K feeds through AWS data centers would bankrupt you on egress fees alone. Lucas: Exactly. And that's why, starting around 2011, Netflix began building what is now called Open Connect — its own content delivery network. Today it's a network of roughly 17,000 purpose-built appliances sitting inside internet exchange points and ISPs around the world. Luna: Seventeen thousand — that number is bigger than most CDN providers' entire footprint. Did they really need to own it themselves? Lucas: That's the central question, and it's one every CTO faces at some point when the cloud bill starts growing faster than revenue. For Netflix, the answer came down to three things: cost, control, and quality of experience. Lucas: Let's start with the cost. By 2015, Netflix was delivering something like 40 percent of all downstream internet traffic in North America during peak hours. If you're paying standard CDN rates for that volume, you're looking at hundreds of millions a year in bandwidth charges. Luna: And that's before the data transfer out of AWS — which is famously expensive. Lucas: Right. So Netflix's engineering team ran the numbers and realized that if they built their own caching appliances and placed them directly inside ISP networks, they could eliminate almost all of that transit cost. The economics were: spend maybe a couple thousand dollars per appliance, put it in a colocation cage rent-free because the ISP benefits from reduced backbone traffic, and suddenly the marginal cost of delivering a stream drops to near zero. Luna: It's like building a warehouse inside the customer's garage. Lucas: Exactly. And the quality payoff was huge too. When you serve video from a box that's literally inside the same ISP that your viewer uses, you eliminate jitter, you reduce buffering, you can push much higher bitrates. Netflix's own data showed that Open Connect reduced startup latency by something like 20 percent compared to third-party CDNs. Luna: But wasn't there a governance challenge? I mean, asking ISPs to host your hardware — that requires a different kind of relationship. Lucas: Massively. And that's actually one of the less talked about engineering leadership challenges here. Netflix had to build a whole partner engineering team whose job was to design these appliances to be dumb, reliable, and remotely manageable — because you can't send a technician to every ISP closet every time you have a firmware glitch. The appliances had to be self-healing, they had to provision themselves via DNS, they had to report telemetry without a human touching them. Luna: So it's not just a hardware project — it's a distributed systems design problem at a scale most companies never touch. Lucas: Exactly. And the software side is fascinating. Open Connect uses a custom version of FreeBSD, heavily tuned for network performance. The caching logic had to handle things like: what happens when a new season of Stranger Things drops and millions of people request the same file simultaneously? The appliance had to coordinate with Netflix's control plane to pre-position content based on predicted demand. Luna: That predictive pre-positioning is a huge operational challenge — because you don't want to waste disk space on shows nobody watches in that region, but you also can't afford a cache miss during peak hours. Lucas: Right. And that's where the machine learning piece comes in. Netflix built a forecasting system that looks at regional viewing patterns, upcoming release schedules, even time-of-day traffic curves, and decides which titles to push to which appliances. The algorithm optimizes for a metric they call 'fill rate' — basically, the percentage of requests that can be served from the local cache without going back to the origin. Luna: And what's a good fill rate for them? 95 percent? 99? Lucas: They've disclosed that during peak hours, Open Connect serves over 95 percent of traffic from local caches. Only about 5 percent ever needs to hit a central origin server. That's a staggering efficiency gain. Luna: So if you're a CTO at a company that's spending eight figures on cloud bandwidth, the Netflix model starts looking really attractive. But the question is — at what scale does this make sense? Lucas: That's the million-dollar question. Netflix's build decision was driven by a very specific set of circumstances: they had enormous traffic volume, they had predictable demand patterns, they had the engineering talent to build and maintain a custom stack, and they had a culture that strongly favors ownership over outsourcing. Luna: And not every company has that. We should mention that if today's episode was useful to you and you want to keep it ad-free, buy me a coffee dot com slash fexingo helps. Lucas: Yeah, and it genuinely makes a difference — so thank you to those who chip in. Now, back to the scale question. I think the threshold is lower than most people assume. Luna: Really? I'd have thought you need to be a Netflix or a YouTube before you even think about it. Lucas: You don't need to build your own CDN from scratch. But you can take the same principle — co-locate your content closer to users — and apply it at a smaller scale. There are companies that use a hybrid model: they run a few dozen caching servers at strategic peering points, while still using cloud CDNs for long-tail content. Lucas: The key is to start with a clear economic model. Map your current per-gigabyte cost from your CDN provider, then estimate the total cost of ownership for a self-hosted appliance — including hardware, colocation, power, and maintenance. If the self-hosted cost per gig is 30 percent lower and you have enough volume to amortize the fixed costs, it's worth prototyping. Luna: But there are hidden costs too — like the operational burden of managing thousands of appliances. Netflix has a dedicated team for that. A 50-person engineering org can't do the same. Lucas: True. But you can outsource the hardware management to a colocation provider or use a platform like Equinix that offers 'smart hands' services. And you can buy appliances from vendors like Qwilt or Broadcom that are pre-built for caching. You don't need to build the entire stack from scratch like Netflix did. Luna: So the build-versus-buy decision here is really about core competency. Netflix views its delivery infrastructure as a competitive advantage — they control the entire pipeline from encoding to the last mile. For most media companies, that might not be the case. Lucas: Exactly. And there's a lesson in that for engineering leaders in any industry. The Netflix story is often cited as a pure cost-cutting move, but it's really a strategy play. By owning the delivery, they can experiment with new codecs, they can push adaptive bitrate updates without waiting for a third-party, they can even use their appliances as edge compute nodes for things like ad insertion or real-time transcoding. Luna: Which is exactly what they started doing with their recent experiments in server-side ad insertion and interactive content. Lucas: Exactly. So the lesson isn't 'build your own CDN.' The lesson is: when a cost center starts to dominate your P&L, don't just negotiate a better contract — ask whether you can change the architecture itself. Sometimes the engineering investment upfront unlocks years of strategic flexibility. Luna: I think a lot of CTOs hear 'Netflix built its own CDN' and feel intimidated. But the principle applies to other layers too — like, maybe you don't need to host your own database if you can use a managed service, but maybe you should if database latency is your product's core differentiator. Lucas: Right. It's about aligning infrastructure ownership with business moat. And that's the conversation every tech leader should be having with their CFO: is this cloud service a commodity we can outsource, or is it a strategic asset we should own? Luna: So for someone listening who's currently evaluating a big infrastructure decision — what's the first step you'd recommend? Lucas: Measure your unit economics first. What does it cost to deliver one unit of value to your customer — one video stream, one API call, one transaction. Then isolate the variable components. If a single line item — like egress bandwidth or database throughput — accounts for more than 20 percent of your infrastructure cost, that's a candidate for deeper investigation. Lucas: Then run a six-month prototype with a self-hosted alternative on a small percentage of your traffic. Netflix didn't flip a switch — they started with one ISP in one region, proved the model, then expanded. Luna: That's the key — start small, measure obsessively, and let the data tell you whether ownership is worth it. Lucas: And if the data says no? Then you've still learned something about your business that a bill negotiation never would have taught you.