Latest / The Cloud Business Podcast with Fexingo: AWS, Azure, GCP, and Enterprise Infrastructure / Cloud Contracts Now Meter Your Service Mesh Sidecar CPU
Transcript
- Lucas: So we've spent several episodes on the cloud providers' creeping granularity — metering Terraform state file reads, container registry scans, inter-region data replication. But there's a new line item appearing on enterprise cloud bills that caught my eye: service mesh sidecar proxy CPU usage. Luna: Wait — the sidecar that handles service to service communication? They're metering that separately now? Lucas: Exactly. And I don't mean the cost of running the sidecar itself on your own compute — that's always been there. I mean a specific line item on the cloud provider's invoice, usually under something like 'Additional Proxy vCPU Hours' or 'Sidecar Compute Surcharge.' AWS App Mesh started itemizing it late last year. And now Azure Service Mesh and Google's Traffic Director are following suit. Luna: So if you're running Istio on EKS, your cloud bill now has an explicit charge for all those Envoy sidecars? Lucas: Right. Let's use a concrete example. Consider a 500-node EKS cluster running Istio in production. Each pod gets an Envoy sidecar container. The cloud provider doesn't charge you for the sidecar's base image — they charge for the CPU time the sidecar consumes on the host. And here's the kicker: they meter it at a higher rate than your application containers. Luna: Higher rate? Why would sidecar CPU be more expensive per vCPU hour than, say, your web server? Lucas: Because the sidecar is doing work that the cloud provider considers 'infrastructure overhead' — mutual TLS handshakes, telemetry aggregation, access logging. And they argue that work consumes not just compute but also control plane resources. So they price it at roughly 1.5x to 2x the standard compute rate. I've seen an enterprise with 500 nodes, each running 20 pods, half of them with istio-proxy sidecars — that's about 5,000 sidecars. Each sidecar uses maybe 0.1 vCPU on average. That's 500 vCPU hours. At 1.5x the rate of standard compute, that's around $12,000 per month in sidecar-specific charges alone. Luna: Twelve thousand dollars a month, just for the sidecars. And that's on top of the compute you're already paying for the application containers. Lucas: Exactly. And it's easy to miss because it doesn't show up as a separate item on the first bill. It's buried in the EC2 or ECS cost line, then suddenly — after a provider audit — it gets broken out. I've talked to a FinOps lead at a mid-sized SaaS company who said their sidecar costs jumped from invisible to $18,000 per month after the provider enforced the new metering. Luna: So what changed technically? Why are they able to meter this now? Lucas: Two things. First, the sidecar proxies themselves have become more feature-rich and more cpu intensive. Mutual TLS handshakes, especially at scale, are expensive — each new pod connection requires a full TLS negotiation, which can eat 10 to 20 milliseconds of CPU per handshake. Second, the cloud providers have added telemetry hooks in the container runtime — they can now attribute CPU cycles to specific cgroups inside a pod. So they know exactly how much CPU the sidecar consumed vs. the application container. Luna: That feels invasive, but also — it's their infrastructure. They can measure it. Lucas: Right. And the providers say this pricing is more transparent — you can see what your service mesh actually costs. But the practical effect is that a design decision — 'let's use a service mesh' — now has a direct, recurring per-unit cost on your cloud bill. Luna: So what can teams do about it? Let me guess — first thing is set resource limits on the sidecar? Lucas: That's the first lever. Most teams don't set CPU limits on the istio-proxy container. By default, it can burst to the entire node's CPU. If you set a hard limit — say, 0.5 vCPU per sidecar — you cap the cost. But you also risk throttling under load, which can cause latency spikes. So you have to test thoroughly. Luna: Second lever: selective injection. Not every service needs a sidecar. Lucas: Exactly. You can use Istio's workload selector to only inject sidecars into namespaces that actually need mutual TLS or telemetry. Batch jobs, internal cron pods, certain monitoring agents — they can run without a sidecar. I've seen teams cut their sidecar count by 40 percent just by auditing which deployments actually need the mesh. Luna: And the third lever is more radical: avoid the sidecar entirely. Lucas: Yeah — the eBPF-based approach. Cilium, for example, implements service mesh functionality at the kernel level, using eBPF programs, so there's no separate sidecar container. The proxy logic runs in the kernel's networking stack. That means no additional CPU charge from the cloud provider — you're just using the host's kernel CPU, which is already covered by the instance cost. Luna: But that's a big architectural shift. You're replacing your entire service mesh. Lucas: It is. And Cilium isn't a full replacement for Istio's feature set yet — especially around fine-grained traffic splitting and fault injection. But for teams where the primary need is mutual TLS and basic telemetry, Cilium can reduce sidecar-related cloud costs to near zero. Luna: So the cloud providers are essentially taxing a pattern they themselves encouraged. They pushed service meshes as a best practice, and now they're charging for the privilege. Lucas: That's a fair framing. It's reminiscent of what happened with NAT gateways — they used to be included, then they became a separate line item. And honestly, if this episode helped you think about your own cloud bill, or if you're wrestling with a sidecar cost surprise, that's the kind of conversation we want to enable. If it was worth a coffee to you, you know the link: buy me a coffee dot com slash fexingo. Luna: Yeah, listener support is genuinely what keeps this show ad-free and focused on these specific, practical angles. Lucas: Alright, back to the sidecar. There's a fourth mitigation I want to mention: sidecar compression and tuning. You can reduce the CPU usage of Envoy by disabling access logs, reducing the number of telemetry metrics, and using a lighter cipher suite for TLS. Some teams have cut sidecar CPU by 30 percent just by disabling the default Envoy access log. Luna: That makes sense. Access logging is notoriously expensive — each request gets written to stdout, and the sidecar then has to format and emit that log. Lucas: Right. And the cloud provider meters that CPU usage. So if you don't need per-request access logs, turn them off. Use sampled logging instead — log every hundredth request. That alone can bring sidecar CPU down significantly. Luna: What about the control plane cost? The sidecar bill is one thing, but does Istio's control plane — Pilot, Mixer, Citadel — also get metered? Lucas: Interesting question. The control plane components run as their own pods, so they're metered as standard compute. But some cloud providers are experimenting with a separate 'Control Plane Surcharge' for managed service meshes. AWS App Mesh, for instance, charges a flat hourly fee per cluster for the control plane, plus the sidecar surcharge. It's not per-pod, it's per-cluster — around $0.10 per hour per cluster. That's another $72 per month per cluster. Luna: So for a company with multiple clusters — dev, staging, production — that control plane fee adds up. Lucas: Exactly. And some teams have separate clusters per team or per environment. I've seen an org with 12 clusters — that's $864 per month just for the control plane surcharge. Plus the sidecar compute surcharge. The total service mesh tax can easily exceed $2,000 per month for a mid-size deployment. Luna: So the question becomes: is the service mesh worth the cost? And that's a conversation teams need to have with their finance department, not just their DevOps team. Lucas: Right. And that brings me to a broader point: cloud providers are increasingly metering what were previously 'free' operational capabilities. Service mesh, DNS, container registry scans — all now have granular, separate charges. The era of the all-inclusive cloud is over. Every architectural decision now has a line item. Luna: And the people who design the architecture — the engineers — often don't see the bill. So the cost optimization has to be a cross-functional effort. Lucas: That's exactly right. The next time you're designing a service mesh, you need to factor in not just the operational complexity but the per vcpu hour surcharge. And that might push you toward eBPF-based approaches or even a simpler sidecar-less design. Luna: So what's the one thing a listener should do this week to check if they're being hit by this? Lucas: Log into your cloud cost management tool — AWS Cost Explorer, Azure Cost Management, GCP's Billing Reports — and search for line items containing keywords like 'proxy', 'sidecar', 'mesh', 'Envoy', or 'App Mesh'. If you see a charge that looks like a per-vCPU surcharge for a service mesh, you're being metered. Then, get the per-pod CPU usage of your sidecar containers — you can use cAdvisor or your container monitoring tool. Multiply by the surcharge rate, and you'll have your monthly sidecar tax. Luna: And then you can decide whether to tune, limit, or eliminate. Lucas: Exactly. And if you find that your sidecar costs are significant, the levers we discussed — resource limits, selective injection, tuning, and eBPF — are all viable paths. The key is to make it a deliberate choice, not a surprise on next month's bill. Luna: Alright, next episode we'll look at another new metering frontier — cloud providers now charging for VPN tunnel idle time. That's a fun one. Lucas: Yeah, that'll be a good one. See you then.