Latest / The Cloud Business Podcast with Fexingo: AWS, Azure, GCP, and Enterprise Infrastructure / Why Enterprise Cloud Contracts Now Lock in GPU Access
Transcript
- Lucas: So you're an enterprise CIO. You've got a $50 million annual cloud commitment with one of the big three. You're training models, you're running inference, and you are hitting GPU capacity walls every other month. You ask your account rep for more H100s and they say, 'Sure, but we can't guarantee availability unless you sign a three-year GPU access rider.' That is the new reality as of mid-2026. Luna: A GPU access rider — that's a contractual clause that locks in a certain number of GPU instances for you, even if demand spikes elsewhere? Lucas: Exactly. And it's spreading fast. AWS calls it 'Capacity Reservations for AI/ML.' Azure has 'Reserved GPU Instances with Priority Access.' Google Cloud has 'Compute Engine GPU Commitment with Availability SLA.' They all do the same thing: you pay a premium — typically 15 to 30 percent above on-demand pricing — in exchange for guaranteed access to specific GPU SKUs during your contract term. Luna: Fifteen to thirty percent premium — that's significant. But if you're running a mission-critical inference pipeline, maybe it's worth it to avoid a two-week wait for spot capacity? Lucas: That's exactly the calculus. And the hyperscalers are banking on that fear. Let me give you a concrete example. I spoke with an infrastructure director at a mid-size fintech company — they process about 200,000 transactions a day using a fraud-detection model on AWS. Last November, they got bumped from their usual P4d instances during a Black Friday surge because someone else had reserved that capacity. Their inference latency tripled. They lost about $800,000 in fraudulent transactions slipping through before they could scale up on different instances. Luna: Ouch. So they learned the hard way that on-demand GPU is not reliable when demand spikes. That's the exact use case for these riders. Lucas: Right. So now they're signing a three-year deal with AWS that includes a GPU access clause. But here's the twist — the clause is not just about availability. It also includes an 'idle GPU penalty.' If they reserve, say, 50 H100s but only use 30 on average, they still pay for the full 50. That's a huge risk if your training workloads are cyclical. Luna: So you're paying a premium for guaranteed access, but you're also on the hook for underutilisation. That sounds like a hedge — you're betting that the cost of occasional idle GPUs is less than the cost of being caught without capacity during a critical workload. Lucas: Exactly. And the hyperscalers love it because it locks in revenue and shifts capacity-planning risk to the customer. But it's not just about the big enterprises. The real story here is what this does to everyone else. Luna: What do you mean? Lucas: When the hyperscalers sell guaranteed GPU access to large enterprises, they carve out a portion of their total GPU fleet. That reduces the pool available for on-demand and spot instances. So startups and smaller companies — the ones that can't commit to $10 million annual spends — get squeezed. They see higher on-demand prices, longer wait times, and fewer instances available in the spot market. Luna: I've heard from a couple of AI startup founders that they're now having to reserve GPU capacity three to six months in advance, which is brutal for early-stage companies that iterate fast. Lucas: It's a structural shift. Two years ago, you could spin up a cluster of A100s in a few hours. Now, even A100s — which are last-gen — are harder to get on demand because the hyperscalers are prioritizing H100 and B200 reservations for big customers. We're seeing a tiered GPU market emerge. Luna: Tiered meaning what — different GPU classes with different availability guarantees? Lucas: Exactly. Top tier: the latest Blackwell or Hopper GPUs, reserved for enterprises with multi-year contracts and GPU access riders. Middle tier: previous-gen GPUs like A100 and A10, available on-demand but with no availability guarantee. Bottom tier: spot instances of older GPUs like V100 or T4, which are cheap but can be reclaimed with two minutes' notice. So your ability to run AI workloads scales with your willingness to sign a long-term contract. Luna: And that's a problem if you're a startup that doesn't know what your compute needs will look like in six months. But for an established enterprise, it might make sense to lock in capacity for known workloads. Lucas: Which brings us to the financial trade-off. Let's run some numbers. Say you're a large bank with a steady-state inference workload that needs 100 H100s 24/7. On-demand pricing for H100 on AWS is roughly $40 per hour per instance. That's $96,000 per day for 100 instances, or about $35 million per year. If you sign a three-year reserved instance with a GPU access rider, you might get a 20 percent discount on the compute cost, but you pay a 20 percent premium on the GPU access guarantee. Net, you're probably paying around $33 million per year — a slight saving, but you get guaranteed capacity. Luna: So the GPU access rider doesn't necessarily save you money — it's more about risk management. You're paying a bit more for certainty. Lucas: Right. But here's where it gets interesting. Some enterprises are starting to negotiate 'burst pools' into their contracts. That means they reserve a baseline number of GPUs — say 50 — but have the option to burst up to 200 for a limited time, typically 48 hours, at a premium rate. This is useful for things like monthly model retraining or handling unexpected traffic spikes. Luna: That sounds like a sensible middle ground. But I imagine the hyperscalers are pricing those burst options aggressively to discourage overuse. Lucas: They are. The burst premium can be 50 to 100 percent above on-demand. But if you're only bursting a few times a year, it's still cheaper than reserving 200 GPUs full-time. And it gives you flexibility. Luna: So what's the broader market implication of all this? Is this a temporary GPU shortage, or is it a permanent shift in how cloud compute is sold? Lucas: Great question. I think it's a bit of both. The immediate driver is the GPU shortage — NVIDIA's H100 and B200 are still supply-constrained as of mid-2026, despite massive fab expansions. But the hyperscalers are also using this moment to lock in enterprise commitments for the long term. They see AI as the killer workload that will drive cloud growth for the next decade, and they want to capture that revenue with sticky contracts. Luna: So it's a strategic play. By creating scarcity, they incentivize enterprises to sign longer deals with higher commitment levels. Lucas: Exactly. And it's working. I've seen data from a recent survey by a major IT analyst firm — they found that 62 percent of enterprises with AI workloads have signed GPU access riders in their cloud contracts this year, up from just 18 percent in 2024. That's a huge jump. Luna: Sixty-two percent — that's a majority. So if you're an enterprise that hasn't been approached about this yet, you probably will be soon. Lucas: And when you are, you need to go in prepared. Know your actual GPU utilisation patterns. Don't over-reserve. Push for burst clauses. And be aware that the hyperscalers are also pushing 'AI infrastructure bundles' that include not just GPU compute but also storage, networking, and even software licenses. The total contract value can get very large very quickly. Luna: That reminds me — I read a recent SEC filing from a Fortune 500 company that disclosed a $100 million cloud commitment specifically for AI infrastructure. That's a big number, and it shows how seriously enterprises are taking this. Lucas: A hundred million is becoming the new baseline for large enterprise AI cloud deals. And in those deals, the GPU access rider is often the most negotiated clause. The account reps have a lot of flexibility on the premium — we've seen discounts of 10 to 25 percent on the rider itself if the customer commits to a longer term or higher baseline. Luna: So negotiation matters. Which is great for enterprises that have strong procurement teams. But for smaller companies, they might not have that leverage. Lucas: That's the concern. And it's why we're seeing the rise of 'GPU broker' firms that help enterprises negotiate these contracts. They aggregate demand across multiple clients to get better terms from the hyperscalers. It's a whole new intermediary layer in the cloud ecosystem. Luna: Speaking of new layers in the ecosystem — one thing I find interesting is how this ties into the broader conversation about cloud economics and the value exchange between providers and customers. If you're finding these deep dives useful for your own planning, I'll mention that listener support is what keeps this show independent and ad-free. It's a small gesture that makes a big difference — like buying a coffee for the team. If that feels right to you, you can head to buy me a coffee dot com slash fexingo. No pressure at all, but every bit helps us keep bringing you these conversations. Lucas: Yeah, and we really appreciate that. It allows us to stay focused on the substance rather than chasing sponsors. Alright, so back to GPU access — another trend I'm watching is the 'inference guarantee' that's starting to appear in contracts. Some hyperscalers are now offering SLAs specifically for inference latency, not just availability. They'll guarantee that 99.9 percent of inference requests complete within 50 milliseconds, for example. That's a new level of commitment. Luna: That's a big deal for real-time applications like fraud detection or autonomous driving. But it must be tied to the GPU access rider, right? You can't guarantee latency if you can't guarantee you'll have the compute. Lucas: Exactly. So the inference SLA is an add-on to the GPU access rider. You pay extra — maybe another 10 percent — for the latency guarantee. And the hyperscalers will only offer it if you reserve a certain number of GPUs. So it's another way to lock in commitment. Luna: Are there any regulatory considerations here? I'm thinking about the SEC's focus on ai related risks in corporate disclosures. If a company signs a GPU access rider and then can't use the capacity, that could be a material risk. Lucas: That's a really good point. In fact, we've seen some enterprises include clauses that allow them to reduce their reserved GPU count if their AI workload projections change, subject to a penalty. That penalty is typically 30 to 50 percent of the remaining contract value. So it's expensive to back out, but it's possible. And yes, the SEC has been asking companies about their AI infrastructure commitments in comment letters. So CFOs need to be aware of the financial reporting implications. Luna: It's fascinating how this is reshaping the cloud market. I wonder if we'll see a secondary market for GPU reservations, similar to how airlines sell ticket futures. Lucas: There's already a startup doing that — a company called ResVault that lets enterprises trade unused GPU reservations. If you've reserved 100 H100s but only need 80 for a quarter, you can sell the excess on their platform. The hyperscalers are okay with it because it doesn't affect their revenue — they get paid either way. And it gives customers more flexibility. Luna: So the market is evolving fast. What's your advice to an enterprise CIO who's about to negotiate a new cloud contract with a GPU component? Lucas: First, audit your GPU utilisation for the past 12 months — know your average, peak, and how often you hit capacity limits. Second, model two scenarios: one where you buy reserved capacity with a rider, and one where you rely on on-demand with a contingency plan. Third, negotiate burst provisions and an exit clause with a reasonable penalty. And fourth, consider using a broker if your spend is over $10 million per year. They can often get you better terms than you'd get on your own. Luna: Solid advice. I'd add: don't let the account rep rush you. These contracts are complex, and the GPU access rider is a relatively new feature. Take the time to understand the fine print. Lucas: Absolutely. And one last thing — keep an eye on the hyperscalers' capital expenditure disclosures. If they're building out GPU capacity faster than demand, the scarcity premium might ease. But if capex lags, these riders will only become more common and more expensive. Luna: So the GPU access rider is here to stay, at least for now. It's a tool for both sides — enterprises get certainty, providers get commitment. But the key is to use it wisely. Lucas: Exactly. And that's the takeaway: don't just sign it because you're afraid of being left out. Know your numbers, know your options, and negotiate the terms that fit your actual workload. The cloud is no longer an infinite resource — it's a market, and you have to trade smart.