Latest / The Edge Computing Podcast with Fexingo: Local Compute, CDNs, and Distributed Infrastructure / How Edge Computing Is Making CDNs Smarter
Transcript
- Lucas: So there's this idea people have about content delivery networks — that they're just a bunch of servers caching static images and video files. And for a long time, that was basically true. Luna: Cache and forget, right? Get the file closer to the user, serve it fast, done. Lucas: Exactly. But over the last couple of years, the big CDN providers — Cloudflare, Fastly, Akamai — they've been quietly turning those edge nodes into general-purpose compute platforms. They're adding runtime environments that let you run code at the edge, not just serve cached content. Luna: And that's a pretty fundamental shift, architecturally. Lucas: It really is. And if today gave you something useful to think about, the way this show stays ad-free is through listener support. If that matters to you, you can help at buy me a coffee dot com slash fexingo. Luna: It's how we keep the conversation focused on what's actually happening, without the sponsored interruptions. So back to the edge — what's a concrete example of this shift? Lucas: Let's talk about dynamic content — things like personalized landing pages, real-time ad insertion, or even A-B test logic. Traditionally, that stuff runs on a central server, and every request goes back to origin, which adds latency. Luna: Right, the round trip can be hundreds of milliseconds, which for a person waiting for a page to load, that's noticeable. Lucas: So a CDN like Fastly, they have this compute platform called Compute@Edge. Instead of just caching, you can write a program in Rust or JavaScript that runs at the edge node — the server closest to the user — and it can make decisions, pull data from a database, assemble the page, and serve it, all from that node. Luna: So the edge node becomes a mini application server. What kind of latency numbers are we actually seeing? Lucas: There's a deployment from early 2025 — a large streaming platform using edge compute for ad insertion. They went from about a 200-millisecond delay for ad requests to under 10 milliseconds. Luna: That's a 20x improvement. And I'm guessing it also offloads the origin servers significantly. Lucas: Yes, in that case they cut origin server load by about 60 percent. Because the ad logic and the personalization logic are all running at the edge. The origin only gets called for rare cache misses or for data that truly needs centralized coordination. Luna: That's a massive cost saving in infrastructure. But it also raises a question: what about consistency? If each edge node is making decisions independently, how do you avoid users seeing two different personalized versions of the same page? Lucas: It's a real challenge. The typical approach is to use a distributed key-value store that's replicated across edge nodes. Something like Cloudflare's Durable Objects or Fastly's geolocated KV store. You write the personalization decision to that store, and any subsequent request from the same user hits the same node or retrieves the decision. Luna: So it's not fully independent — there's still a coordination layer. But that coordination is also distributed, not centralized. Lucas: Right. And that's the sweet spot. You avoid a single point of failure, but you still have eventual consistency. For most use cases — ad insertion, personalization — that's fine. You don't need strong consistency for a recommendation. Luna: What about security? If you're running arbitrary code at the edge, aren't you expanding the attack surface? Lucas: Absolutely. And the providers have been investing heavily in sandboxing. Fastly uses WebAssembly sandboxing — your code runs in a Wasm runtime that's isolated from the host system. Cloudflare uses a similar model with Workers, using isolates based on V8. Luna: So the same isolation pattern that made browsers safe is now powering edge compute. That's interesting. Lucas: Exactly. And it's not just security — it's also about performance. Wasm modules are small and start up in microseconds. So you can run a function on every request without the cold-start penalty you'd get from a serverless function in the cloud. Luna: That makes edge compute viable for high-frequency use cases. We're not just talking about a few thousand requests per second — we're talking millions. Lucas: And that's exactly what the big CDNs are seeing. Cloudflare Workers now handle over ten million requests per second at peak, and a significant fraction of those are running custom logic, not just serving cached files. Luna: Let's zoom out a bit. Does this mean the traditional three-tier architecture — load balancer, app server, database — is becoming obsolete? Lucas: Not obsolete, but the boundaries are blurring. The edge node is now acting as the load balancer and the app server in one. And with distributed data stores, the database layer is also moving to the edge. Luna: So the whole stack flattens. Instead of a central cluster, you have thousands of mini clusters. Lucas: Yes. And that has implications for how you think about deployment, monitoring, and debugging. You can't SSH into a single edge node; it's a fleet. So observability becomes critical. Luna: Are the CDN providers building tools for that? I know Fastly has an observability platform, but is it enough? Lucas: They're getting there. Most providers now offer real-time logging from edge nodes, distributed tracing, and metrics dashboards. But it's still early — debugging a request that hit a node in Tokyo versus one in Frankfurt is not as seamless as debugging a single server. Luna: So the trade-off is performance gains for operational complexity. That sounds worth it for the latency-sensitive use cases, but maybe not for every app. Lucas: Right. You wouldn't move your core transactional database to the edge. But for the presentation layer — the last mile of web delivery — edge compute is becoming the default. And I think we're going to see more and more of the business logic move out there too. Luna: It's interesting — the CDN started as a way to speed up delivery of static assets, and now it's becoming the platform for the entire user-facing application. Lucas: Exactly. And that shift is happening faster than most people realize. By the end of 2025, I'd guess that over half of the dynamic content on major websites is already being assembled at the edge. Luna: That's a bold prediction. But given the numbers we've seen — the latency improvements, the origin offload — it's hard to argue against the momentum. Lucas: So that's the story. The CDN isn't just a cache anymore. It's a distributed computer that happens to be really good at serving content. Luna: And the next time you load a webpage that feels instant, there's a good chance it wasn't a static file — it was a program running on a node near you. Lucas: That's exactly the future we're heading into.