Latest / The Cloud Business Podcast with Fexingo: AWS, Azure, GCP, and Enterprise Infrastructure / Why Cloud Contracts Now Include AI Inference Guarantees
Transcript
- Lucas: So it's late May 2026, and a quiet change is rippling through enterprise cloud contracts — one that most procurement teams haven't fully clocked yet. Luna: I have a feeling you're not talking about another egress fee discount. Lucas: Not at all. This is about AI inference performance guarantees. For years, cloud SLAs have been about uptime — 'your virtual machine will be available 99.99 percent of the time.' But now, the hyperscalers are starting to guarantee how fast your model actually responds. Luna: So instead of 'the server is on,' it's 'the server will return your inference in under 100 milliseconds.' That's a much harder promise. Lucas: Exactly. And it changes the entire risk profile for enterprises running production AI. Let's start with Google Cloud, because they were first out of the gate. In March, they introduced what they call 'AI Optimized' compute SLAs for their TPU v5e and v5p instances. Luna: These are the chips they build specifically for training and inference, right? Lucas: Right. The TPU v5e is really the inference workhorse — it's designed to be cost-efficient for serving models. And the new SLA doesn't just say 'your TPU will be running.' It commits to a specific inference throughput, measured in queries per second, and a latency percentile — typically the p99 response time. Luna: But how do they even measure that? Is it per instance, per model, per request? The engineering complexity is enormous. Lucas: That's the crux of it. Google's approach is to define a 'reference model' — they use a standard bert large or a Llama 3 variant — and they guarantee performance when running that model with a specific batch size and input length. If you deviate from that, the SLA doesn't apply. Luna: So it's a benchmark, not a blanket promise. That creates a negotiation gap for enterprises running custom models. Lucas: Exactly. And that's where the real-world friction lives. I talked to a procurement director at a mid-size SaaS company last week — they're renegotiating their Azure contract right now. They wanted inference guarantees for their proprietary recommendation model. Microsoft said sure, but the premium over standard compute was about 25 percent. Luna: Twenty-five percent just for a promise on latency? That's significant. Lucas: It is. But here's the thing — without that guarantee, if your model slows down during peak hours, you lose users. For a real-time recommendation engine, every extra 50 milliseconds translates to measurable revenue drop. So the premium starts to look like insurance. Luna: And AWS? How are they responding? Lucas: AWS is taking a different route. They're not offering an explicit inference SLA on their GPU instances yet. Instead, they're pushing 'capacity reservations' for their latest NVIDIA H200 and upcoming B200 GPUs. The pitch is: reserve a specific number of GPUs for a one- or three-year term, and you get guaranteed access — but the performance itself isn't in the SLA. Luna: So they're guaranteeing the hardware will be there, but not how fast it runs. That feels like a step behind Google. Lucas: It depends on your use case. If you have a steady-state inference workload and you know your model well, you might prefer the lower cost of a reservation over a performance SLA. But if your traffic is spiky or your model is evolving, the reservation could leave you overpaying for idle GPUs. Luna: And that's the classic cloud dilemma — flexibility versus commitment. Lucas: Right. But I think the bigger story here is that these SLAs represent a fundamental shift in how cloud providers think about value. They're moving from 'infrastructure availability' to 'application-level outcomes.' And that has implications for enterprise procurement. Luna: Like, procurement teams now need to understand model architecture and latency distributions? That's a whole new skill set. Lucas: Exactly. I've seen some larger enterprises start building internal 'AI infrastructure engineering' teams that sit between procurement and the ML teams. Their job is to benchmark models on different cloud instances and then negotiate SLAs based on real data, not just marketing. Luna: It reminds me of how, a decade ago, companies had to hire 'cloud architects' just to manage migration. Now it's 'AI infrastructure negotiators.' Lucas: That's a great comparison. And it's worth noting that these inference SLAs aren't universal yet. Google has them for TPUs, but not for their A3 GPU instances yet. Azure has a preview program for 'AI guaranteed compute' but it's limited to certain regions. So we're in early innings. Luna: Which means early adopters — like that SaaS company you mentioned — are effectively beta testers. They're paying a premium but also shaping what the standard will look like. Lucas: Yeah, and if you're in procurement right now, that's both an opportunity and a risk. You can potentially negotiate better terms if you're willing to commit early, but you might also lock into a standard that gets cheaper or simpler in six months. Luna: So what's your advice for a listener who's about to sign a cloud contract with inference workloads? Lucas: First, benchmark your own model on the provider's hardware before you sign anything. Don't rely on their reference model numbers. Second, ask for a 'performance credit' clause — if your p99 latency exceeds a threshold for more than, say, 5 percent of requests in a month, you get a service credit. Some providers will push back, but it's worth asking. Luna: And third, I'd add: don't forget the regional dimension. If your inference SLA only applies in us-east-1, but your users are in Southeast Asia, the guarantee is almost meaningless. Lucas: Great point. And that's exactly the kind of nuance that procurement teams need to catch — because the hyperscalers are selling these SLAs as a differentiator, but the fine print matters. Luna: It's a lot like the early days of reserved instances. Remember when everyone thought they were saving money, but then realized they were locked into the wrong instance family? Lucas: Exactly. History repeating, just with GPUs and TPUs instead of general-purpose VMs. Luna: And speaking of repeating — we're able to have these conversations without any ad breaks because of listener support. If you find this kind of practical analysis useful for what you're building or running, you can help keep it ad-free at buy me a coffee dot com slash fexingo. Lucas: It's a small way to support the kind of detailed procurement talk we just covered — no sponsors, no interruptions. Just the signal. Luna: Right. And back to that — I'm curious, Lucas, do you think inference SLAs will eventually become table stakes, like uptime guarantees are today? Lucas: I think so, but it might take three to five years. Right now, only a fraction of enterprise cloud spend goes to inference. But as generative AI moves from pilot to production — which we're seeing accelerate in 2026 — the demand for predictable inference will grow. Luna: And when that happens, the SLA itself becomes a competitive weapon. The provider that can offer the tightest guarantees — with the least fine print — wins the next wave of cloud revenue. Lucas: Exactly. And if you're a cloud buyer, the best time to understand these terms is now, before they become standard boilerplate. Because once they're standard, the negotiation leverage shifts to the provider. Luna: Well, that's a solid warning for the procurement teams out there. Thanks, Lucas. Lucas: Thanks, Luna. And if you want to dig deeper, we'll put some links to the actual SLA documents from Google and Microsoft in the show notes.