Latest / The CTO Podcast with Fexingo: Technical Leadership, Architecture, and Engineering Org / How Etsy Cut Server Costs Without Breaking Deploys
Transcript
- Lucas: In the second half of 2025, Etsy did something that sounds boring on paper but is actually one of the hardest things a large engineering org can pull off. They moved about 200 microservices — roughly half their production footprint — into a smaller, cheaper Kubernetes node pool, and they did it without a single deploy slowdown or customer-facing incident. Luna: Okay, that's a lot of services. What was the motivation — just cutting costs? Lucas: Primarily, yes. Etsy's infrastructure bill had been creeping up. Their cloud spend was growing faster than revenue in some quarters. The engineering leadership, including their CTO, set a pretty clear target: reduce annual infrastructure costs by something like 6 to 8 percent without cutting capacity or performance. A classic FinOps goal. Lucas: But the interesting part is how they approached it. They didn't do a big re-architecture. They didn't consolidate services or force teams to rewrite code. Instead, they focused purely on the deployment layer — specifically, the Kubernetes node pools that those services run on. Luna: So they just packed the services more densely onto fewer machines? That sounds straightforward until you hit noisy-neighbor problems or resource contention. Lucas: Exactly. And that's where the story gets technical. Etsy's existing node pool was using relatively large instances — let's say 16 vCPU, 64 GB RAM — spread across multiple availability zones. The services were deployed with fairly generous resource requests and limits, partly because teams had padded them over the years to avoid OOM kills. Lucas: The new pool used smaller instances, about half the size, but with a much tighter bin-packing strategy. They wrote a custom scheduling plugin that considered not just CPU and memory, but also network IO and disk throughput for each service. They essentially ran a giant constraint-satisfaction problem to figure out the optimal placement. Luna: That's clever. Did they have to do any traffic shaping to avoid hotspots? Lucas: Yes, and that's the second big piece. They introduced per-service traffic caps at the load balancer layer — basically a token bucket that limited how many requests per second any single instance could receive. That prevented one busy service from hogging resources on a shared node, even if its CPU request was modest. Luna: Huh. That's almost like applying rate limiting internally, not just at the edge. Lucas: Exactly. They called it 'intra-cluster traffic shaping.' It was a middleware layer in their service mesh that injected a small delay when a pod approached its traffic quota. The delay was measured in milliseconds — imperceptible to users — but it smoothed out usage spikes across nodes. Lucas: The really impressive part, though, was how they validated the migration. Etsy's deploy culture is famously fast — they push code to production dozens of times a day. Any new node pool had to support that same cadence without teams feeling constrained. Luna: Right. So they couldn't just flip a switch. They must have used canary analysis. Lucas: They did, and at a fine granularity. They moved services over in groups of 10 to 15, each group running in the new pool for a full week before the next batch. During that week, they monitored not just CPU and memory, but also deploy times, rollback rates, error budgets, and p99 latency per service. Lucas: If any service saw a latency increase of more than 5 percent or a deploy failure rate above its baseline, that group was rolled back immediately. In practice, that happened twice — both times because a service had a hidden dependency on local SSD caching that the new node type didn't provide. They fixed it by adjusting the CPU manager policy. Luna: So the new nodes didn't have local SSDs? That's a significant hardware difference. Lucas: It was. The new instances used network-attached ephemeral storage instead of local NVMe. For most services that was fine, but a few had been relying on local disk for write-ahead logs or similar. Etsy's team actually built a small tool that analyzed each service's disk IO patterns and flagged mismatches before the migration — but even with that, two slipped through. Luna: That's a good validation of their methodology. They caught it in canary, not in production at scale. Lucas: Exactly. And because the rollback was automated — when a canary group failed its health check, the scheduler just moved those pods back to the old pool — the teams barely noticed. Luna: What was the final cost saving? Lucas: Etsy publicly disclosed that the project reduced their annual Kubernetes infrastructure spend by $4.2 million. That's about 7 percent of their total cloud bill. The migration took roughly five months from start to finish, including the tooling development. Lucas: And here's the thing — they didn't touch a single line of application code. This was purely an infrastructure and operations win. It's a great example of what a focused platform team can do when they're given the autonomy to solve a specific problem. Luna: That's a really strong case study for any CTO who's being asked to cut costs without breaking engineering velocity. It's not about squeezing developers — it's about optimizing the deployment layer. Lucas: Yeah. And if today's episode was useful to you and you want to keep it ad-free, buy me a coffee dot com slash fexingo helps. That's literally what keeps these conversations going. Luna: Absolutely. A handful of listeners chipping in really does make a difference. Lucas: So back to Etsy — one thing I really appreciated was their emphasis on making the cost savings measurable and attributable. They tagged every resource with cost-center labels, so after the migration they could generate a report showing exactly how much each product team saved. That built trust with the engineering org. Luna: That's smart. It turns a platform initiative into something every team can see the benefit of. Did they share any specific numbers per team? Lucas: Not publicly, but internally they published a dashboard. Some teams saw their infrastructure costs drop by over 20 percent. Others, especially services that were already efficient, saw more modest gains. But nobody's costs went up, which was a key requirement. Luna: What about the teams that were flagged as potential mismatches for the new pool? Did they have to refactor? Lucas: About 30 services were identified as high-risk based on the disk IO analysis. Etsy gave those teams two options: either refactor to use network storage, or stay on the old pool with a commitment to reduce their resource requests. Most chose to refactor, and the platform team provided migration guides and a test environment. By the end of the project, about half of those 30 had successfully moved to the new pool. Luna: So they didn't force everyone. The platform team gave options and let teams decide. That's good change management. Lucas: It is. And I think that's the real lesson for engineering leaders. Cost optimization is often framed as top-down and painful — like, 'we need to cut spend, everyone reduce your instance count by 10 percent.' Etsy showed that a bottom-up, tooling-driven approach can be more effective and less disruptive. Luna: It helps that they had a mature platform team and a strong Kubernetes practice already. Not every org has that foundation. Lucas: True. But the principles apply broadly. Invest in good observability. Automate canary analysis. Give teams choice. And think about the deployment layer before you ask developers to rewrite code. Luna: Is there a follow-up planned? Further optimizations? Lucas: Etsy's public roadmap mentions two related initiatives. One is applying the same bin-packing approach to their stateful workloads — databases, caches, that sort of thing. The other is exploring spot instances for even larger savings. But those are harder problems, and they're moving carefully. Luna: Interesting. So the infrastructure cost optimization playbook now includes a chapter on traffic shaping and canary-based migration. That's a useful template. Lucas: It really is. And it's a reminder that sometimes the biggest wins come from the most mundane-sounding projects.