Latest / The CTO Podcast with Fexingo: Technical Leadership, Architecture, and Engineering Org / How Cloudflare Built Its Global Network for 30 Million Requests per Second
Transcript
- Lucas: So Cloudflare handles something like 30 million HTTP requests per second across its global network. That's not a peak number — that's their average traffic as of mid-2026. Luna: Thirty million per second. That's roughly three times the number of Google searches per second. How do you even architect for that? Lucas: Well, you start with the fact that Cloudflare isn't a traditional content delivery network. It began as a reverse proxy and a security layer — originally a project called Project Honeypot — but quickly evolved into a distributed system spanning over 330 cities worldwide. Luna: Right, so every request hits a Cloudflare data center first. But the routing decision — which data center — that's where Anycast comes in. Lucas: Exactly. Anycast allows multiple data centers to advertise the same IP address. The internet's routing protocol, BGP, naturally sends a user to the nearest data center. That's step one for low latency. But the real challenge is that no single data center can hold all the data — or handle all the traffic. Luna: So how do they decide what to cache where? Is it a least recently used eviction policy across all nodes? Lucas: It's more nuanced. Cloudflare uses a tiered cache hierarchy. Each data center has a local cache, and if there's a miss, it goes to a regional cache — maybe one in the same continent — before hitting the origin server. The key insight is that they don't cache everything. They cache based on file type, popularity, and customer settings. Luna: And for dynamic content — things that can't be cached — they still route through Cloudflare for DDoS filtering and security. Lucas: Right. Every request passes through their layer 7 reverse proxy, which is built on a heavily customized version of Nginx. They've open-sourced some of those modifications, but the core is a multi-threaded event loop that can handle tens of thousands of concurrent connections per server. Luna: What about the DDoS scrubbing itself? I remember reading that Cloudflare mitigates some of the largest attacks ever recorded. Lucas: In 2020, they mitigated a 2.3 terabit per second DDoS attack. More recently, they've absorbed attacks exceeding 10 Tbps. The way it works is that their edge servers inspect each packet against a set of rules — rate limits, IP reputation, protocol anomalies — and drop malicious traffic before it ever reaches the application. Luna: But inspecting every packet at 30 million requests per second — that's computationally expensive. How do they keep latency under control? Lucas: They use eBPF — extended Berkeley Packet Filter — to run sandboxed programs in the Linux kernel. That lets them filter traffic at wire speed without context switching to user space. They also use DPDK for packet processing on some high-performance edge servers. Luna: So the edge is essentially a giant state machine, processing packets and applying rules. But what about consistency? If a customer updates a firewall rule, how quickly does it propagate to all 330 data centers? Lucas: That's a great question. Cloudflare uses a global control plane called Quicksilver — a low-latency key-value store that pushes configuration changes to every edge node in under 30 seconds typically. It's a separate system from the data plane, which is critical for availability. Luna: So the data plane can keep running even if the control plane goes down. Lucas: Exactly. Each edge node caches the latest configuration locally. If Quicksilver is unreachable, the edge continues operating with the last known config. That's a key architectural principle — never let a central failure take down the edge. Luna: That reminds me of how Google's global load balancer works. But Cloudflare is a much smaller company — how do they manage engineering at that scale? Lucas: They have around 3,000 employees, which is surprisingly lean for the traffic they handle. Their engineering team is organized around product verticals — like security, performance, and developer platform — but the core infrastructure team maintains the global network software. Luna: And they've open-sourced a lot of that software — like their L4 proxy, their DNS server, and even their internal tooling for deploying configs. Lucas: That's been a deliberate strategy. Open source helps with recruiting, and it also gives the community a chance to audit the code. Trust is a huge part of being a reverse proxy — customers need to know you're not tampering with their traffic. Luna: Speaking of trust, Cloudflare has had a few high-profile incidents — like the 2019 firewall bypass and the 2022 outage that took down a chunk of the internet. Lucas: They've been transparent about both. The 2022 outage was caused by a bad software deployment that triggered a cascade of reboots across the network. They've since implemented better canary testing and deployment rollback automation. Luna: So they treat their own network as a petri dish for chaos engineering. Lucas: Pretty much. They run regular game days where they simulate failures — like taking a data center offline or throttling a transatlantic link — to see how the system behaves. Luna: Let's talk about one specific capability: how they handle DDoS attacks on layer 7 — application-layer attacks that look like legitimate traffic. Lucas: Layer 7 is trickier because you can't just drop packets based on source IP. Attackers use botnets that mimic real browsers. Cloudflare's approach is to challenge suspicious requests with a JavaScript proof of work — like a CAPTCHA but computational — that a real browser can solve but a botnet can't at scale. Luna: They also use machine learning models that analyze request patterns — like the order of HTTP headers or the timing of mouse movements. Lucas: Yes, they have a product called Bot Management that scores each request on a scale from one to 100 — one being definitely a human, 100 being definitely a bot. Customers can set thresholds to allow or block traffic. Luna: And all of this happens in microseconds per request, at 30 million per second. Lucas: Right. The underlying architecture has to be incredibly efficient. They run a custom HTTP stack called Pingora, which they wrote in Rust to replace Nginx on some of their edge nodes. Pingora gives them more control over memory and concurrency. Luna: Rust seems to be the language of choice for high-performance networking these days. Lucas: It is. The memory safety without garbage collection is a big deal when you're handling millions of connections. Cloudflare's engineers have said Pingora reduced memory usage by about 70% compared to Nginx for the same traffic load. Luna: That's a massive efficiency gain. And it means they can run fewer servers per data center, which reduces power and cooling costs. Lucas: Exactly. And that's not just a cost save — it's also a green initiative. Cloudflare has committed to powering its network with renewable energy and offsetting historical emissions. Luna: It's interesting how architectural decisions cascade into business and sustainability outcomes. Lucas: Yeah, the whole system is designed around efficiency from the chip up. They even work with hardware vendors to customize servers — for example, using Intel's QuickAssist Technology for encryption offloading. Luna: Let's zoom out. For a CTO building a product that relies on real-time data at scale, what's the biggest lesson from Cloudflare's architecture? Lucas: I'd say: design for failure at every layer. Cloudflare assumes any single data center can go dark at any moment. Their routing, caching, and control plane all degrade gracefully. That's hard to do retroactively. Luna: So start with a distributed mindset from day one. Lucas: Even if you're only handling 1,000 requests per second today. The patterns — Anycast, tiered caching, control plane separation, eBPF — they scale down as well as up. Luna: And if these design patterns are something you're grappling with in your own work, shows like this one exist because of listener support. A couple of dollars a month at buy me a coffee dot com slash fexingo genuinely helps keep them ad-free and focused on the engineering details. Lucas: Yeah, it's a small thing that makes a big difference. We appreciate everyone who chips in. Luna: Alright, back to the edge. So we've talked about caching, DDoS, and routing. What's next for Cloudflare on the architectural front? Lucas: They're pushing compute to the edge with Cloudflare Workers — a serverless platform that runs JavaScript and WebAssembly on their edge nodes. That effectively turns the CDN into a distributed computing platform. Luna: So instead of just caching content, the edge can now run custom logic — like authentication, A/B testing, or even full applications. Lucas: Exactly. Workers execute in isolates — a lightweight sandbox — so they can spin up in milliseconds. The challenge is coordinating state across the edge, but they've built a key-value store and a distributed durable objects API for that. Luna: That's a major shift. For years, the edge was just for static assets. Now it's a platform. Lucas: And it changes the economics of web applications. You can serve a dynamic site entirely from the edge, with no origin server, if you design for it. That's what a lot of the Jamstack ecosystem is moving toward. Luna: Alright, so final thought: if you're building for the edge today, what's the one thing you should get right? Lucas: Idempotency. Because the edge is distributed, a user's request might hit multiple data centers if the network changes mid-flight. Your application logic needs to handle duplicate requests safely — something Cloudflare's Workers runtime helps with, but it's still a design consideration. Luna: Good advice. Thanks, Lucas. Lucas: Thanks, Luna. That's it for this episode of The CTO Podcast.