Latest / The Cloud Business Podcast with Fexingo: AWS, Azure, GCP, and Enterprise Infrastructure / GPU Contract Clauses That Lock in Power Draw Limits
Transcript
- Lucas: If these conversations are useful for what you're building or running, there's a simple way to keep them coming. And honestly, if today was worth a coffee to you, that's the link — buy me a coffee dot com slash fexingo. Luna: No pressure, just if the show's been helpful. We keep it ad-free, and that's because of listeners who toss in a coffee now and then. Lucas: Exactly. So: GPU power draw limits. Not temperature caps, not lifecycle guarantees — actual wattage ceilings written into cloud contracts. This is new, and it's spreading fast. Luna: I saw that Azure started offering a 350-watt tier on their ND H100 v5 instances last quarter. AWS followed with something similar on p5.48xlarge. Lucas: Right. And GCP just announced a 400-watt option on their A3 Mega instances. The pattern is clear: providers are moving from thermal-only provisions — where they cap the temperature at, say, 85 degrees Celsius — to power limits that cap the actual draw. It changes the math for anyone running AI workloads. Luna: Why now? I mean, power has always been a constraint in datacenters, but why are they codifying it into SLAs? Lucas: A few reasons. One: the density of GPU clusters is pushing datacenter power infrastructure to the limit. A single rack of H100s can draw over 40 kilowatts. That's more than a small office building. Providers need to guarantee they don't overload the grid, and contracts are the way to do it. Luna: So it's partly a risk management play for the cloud providers themselves. Lucas: Exactly. But the second reason is customer demand. AI startups have been complaining about variable power costs — your GPU instance might draw 700 watts during a training run, then 300 during inferencing. That unpredictability makes it hard to budget. A fixed power cap means you know your ceiling. Luna: But a cap also means you can't exceed it, even if you want to. So there's a trade-off. Lucas: Huge trade-off. Let's talk about the specific tiers. The three most common right now: 300 watts, 400 watts, and 500 watts per GPU. The 300-watt tier is typically for inferencing workloads — lower compute, lower power draw. The 400-watt tier is the sweet spot for most training runs. The 500-watt tier is for high-performance training where you want maximum throughput, but it comes at a premium. Luna: I've seen pricing data that shows the 500-watt tier costs about 30% more per gpu hour than the 400-watt tier on AWS. But the performance gain isn't linear. Lucas: Right. An H100 at 500 watts might give you maybe 10-15% more FLOPS than at 400 watts, but you're paying 30% more. So you have to model the cost per training epoch, not just cost per instance. And that's where a lot of teams get tripped up. Luna: I want to dig into a real contract negotiation. You mentioned a mid-size AI startup that negotiated a 400-watt cap. What happened there? Lucas: So this is a company called Synthex — they do LLM fine-tuning for legal documents. They were on Azure's ND H100 v5 at the default 500-watt tier. But they realized that for their workloads, 90% of the time they were under 400 watts anyway. The only spikes came during the initial warmup phase of training, which lasted about 2 minutes. Luna: So they negotiated a contract that capped them at 400 watts, and Azure agreed to a lower per gpu rate. Lucas: Exactly. They moved to a 400-watt SLA and saved 18% on monthly compute costs. Their training time increased by about 4% because of those warmup spikes being throttled, but the cost savings more than made up for it. The key was they had good telemetry data to show Azure their actual power profile. Luna: That's something more teams should do before signing. Measure your actual power draw over a week of representative workloads. Lucas: Absolutely. And providers are actually encouraging it now. AWS has a tool called Instance Power Advisor that lets you simulate your workload on different power tiers. It's in preview. GCP has a similar thing in their Cost Optimization dashboard. Luna: What about the risk of hitting the cap during a critical training run? If you're training a model for 30 days and a power spike triggers throttling, you could lose progress. Lucas: That's the big downside. Contracts typically have a grace period — like 1% of the time you can exceed the cap by up to 10% without penalty. But if you consistently spike, the provider can throttle you, and that could mean slower training. For some workloads, like real-time inferencing, that's a dealbreaker. Luna: So the choice of power tier becomes a business decision, not just a technical one. If your model's time to market is critical, you might pay for the 500-watt tier. Lucas: Exactly. And that's why the contract language matters. Some providers offer a 'burst' provision — you get 500-watt performance for up to 5 minutes per hour, then you're throttled back. That's good for workloads that have periodic spikes. Luna: I've seen some contracts that tie power limits to carbon-aware scheduling. Like, you get a lower rate if you agree to a 300-watt cap during peak grid hours. Lucas: Yes, that's a new trend. GCP has a 'carbon-aware GPU' tier that dynamically adjusts your power cap based on grid carbon intensity. You save money, but you have less predictability. For batch workloads that aren't time-sensitive, it's a good deal. Luna: Let's talk about the legal side. What happens if the provider fails to deliver the agreed power cap? Do you get SLA credits? Lucas: Most contracts now include a power SLA with credits. If your instance is supposed to be 400 watts and it drops to 300 for more than 5% of the month, you get a 10% credit. But you have to monitor it. There's no automatic reporting yet. Luna: So teams need to instrument their own power monitoring. That's an added cost. Lucas: It is, but it's worth it. I know a startup that saved $200,000 over a year just by claiming SLA credits for power shortfalls. They built a simple script that checked GPU power draw every minute and logged violations. Luna: What about multi-cloud contracts? Can you negotiate power caps across AWS, Azure, and GCP in one deal? Lucas: Not yet. Each provider negotiates separately. But some enterprise agreements now include a 'power parity' clause — if you sign a three-year commit, they'll match the lowest power pricing across their regions. That's new as of this quarter. Luna: Interesting. So if you're a large enterprise, you could leverage that clause to get better terms. Lucas: Exactly. Let's zoom out. Power draw limits are one piece of a bigger shift: cloud contracts are becoming more like utility contracts. You're not just renting compute; you're buying a specific power envelope. That changes how you design your workloads. Luna: It also changes how you hire. I'm seeing job postings for 'GPU power engineers' — people who understand both hardware power profiles and cloud contract terms. Lucas: Yes. The role combines data center engineering with procurement. It's a niche but growing field. For anyone listening who negotiates cloud contracts, the takeaway is: get your own power telemetry, understand your workload's profile, and don't default to the highest power tier. Luna: And if you're an AI startup, the 400-watt tier is probably your starting point. Model the cost-performance trade-off before you sign. Lucas: Right. And the providers are making tools to help you do that. Use them. The days of 'one size fits all GPU instances' are ending. Power is now a first-class term in cloud contracts. Luna: It's a smart move for the industry. It aligns cost with actual usage, and it gives customers more control. Lucas: Agreed. Next episode, we'll look at how these power caps interact with carbon-aware scheduling — because that's where the real optimization happens. Thanks for listening. Luna: And if you found this useful, you know where to find us: buy me a coffee dot com slash fexingo.