Latest / The Cloud Business Podcast with Fexingo: AWS, Azure, GCP, and Enterprise Infrastructure / Why Cloud Contracts Now Include GPU Thermal Runway Provisions
Transcript
- Lucas: So you're negotiating a multi-year cloud commitment for a gpu heavy AI workload, and you get to the SLAs section, and there's a new clause that wasn't there in 2024: a GPU junction temperature limit, typically 85 degrees Celsius, with penalties if the hyperscaler exceeds it. Luna: Wait, they're putting thermal limits in the contract now? For the actual silicon temperature? Lucas: Exactly. It's being called a 'thermal runway provision.' And it's showing up in AWS, Azure, and GCP enterprise agreements for customers deploying clusters of NVIDIA H100s or AMD MI300X accelerators. The idea is that dense GPU racks generate so much heat that if the cooling infrastructure can't keep up, junction temps rise, the GPUs throttle, and your training job takes 15 or 20 percent longer. Luna: That's real money. If you're paying for reserved compute by the hour, throttling means you're burning budget without getting the throughput. Lucas: Right. And this isn't hypothetical. I talked to a fintech firm that renegotiated their cloud contract in September of last year — 2025 — and they specifically negotiated a temperature delta penalty clause. Their agreement with their hyperscaler capped GPU junction temperature at 85 Celsius, with a credit if the average exceeds 87 over any 24-hour period. They estimated it saved them about $2.3 million in potential throttling downtime over three years. Luna: That's a huge number for a single contract clause. How did they even know to ask for this? Lucas: They had a thermal engineer on staff who ran simulations on their cluster. They found that during summer peak loads — July and August — the data center cooling couldn't keep junction temps below 90 degrees for about 40 hours total across both months. Their training jobs were consistently running 8 percent slower during those windows. So when the renewal came up, they turned that data into a contractual ask. Luna: So the hyperscaler agreed? That seems like a pretty aggressive SLA for something they can't fully control — ambient temperature, power grid fluctuations. Lucas: They agreed because the customer was committing to a three-year, eight-figure GPU reservation. And the hyperscaler had just rolled out liquid-cooled racks in that region — they were confident they could meet it. The provision is becoming more common as liquid cooling spreads. If you've got direct to chip cooling or immersion, you can guarantee tighter thermal envelopes. Luna: If today's deep dive on GPU thermal runway provisions gave you a useful angle for your next cloud negotiation, the reason we can keep doing these episodes without ads or sponsors is listener support. It's a small thing, but it adds up — you can head to buy me a coffee dot com slash fexingo. That's buy me a coffee dot com slash fexingo. And it genuinely helps us stay independent and dig into the contract details that matter. Lucas: Yeah, it's a low-key way to keep this ad-free. And we appreciate anyone who throws a coffee in the jar. Now, back to the thermal runway — let's talk about the physics for a second. A GPU like the H100 has a maximum junction temperature of about 95 degrees Celsius. But NVIDIA recommends staying below 85 for sustained operation. Once you cross 85, the card starts to dynamically reduce clock speed to protect itself. That's the thermal throttle threshold. Luna: And in a dense cluster — say 4,000 GPUs in a single data center hall — if even 10 percent of them hit that threshold, your job completion time extends noticeably. Lucas: Exactly. And the interesting thing is that these thermal runway provisions are migrating from the hardware lease world into cloud contracts. In the colocation space, it's been common for years to have power and cooling SLAs with temperature ranges. But cloud hyperscalers historically treated cooling as their problem, not a contractual promise. That's changing as GPU density increases. Luna: What's the typical structure? Is it a hard cap with penalties, or more of a target with reporting? Lucas: It varies. The most aggressive versions — like the fintech firm's — have a hard cap with graduated credits. If the average junction temp exceeds 87 degrees for a rolling 24-hour window, the customer gets a 5 percent credit on that day's GPU compute charges. If it exceeds 90 degrees, the credit jumps to 15 percent. Others are softer — just a commitment to use liquid cooling for certain GPU SKUs, with a reporting dashboard so the customer can monitor temps themselves. Luna: And the hyperscalers are willing to do this because they're investing heavily in cooling. I know Azure has been rolling out two-phase liquid cooling in some regions, and AWS has its own direct to chip solutions. Lucas: Right. And GCP is partnering with cooling vendors for their TPU pods. The key driver is that without liquid cooling, you literally cannot pack H100s or B200s at high enough density to make the economics work in a standard air-cooled data center. So the hyperscalers are making cooling a competitive differentiator. And the contract follows. Luna: So for a company that's about to sign a big GPU reservation, how do you approach this? Do you need a thermal engineer on staff? Lucas: Ideally, yes. But at minimum, you need to run a thermal audit of your expected workload. You can use tools like NVIDIA's DCGM — data center GPU manager — to profile your jobs and see what junction temperatures they generate. Then you overlay that with the hyperscaler's published data center PUE and cooling specs. If you see a gap, you build the clause. Luna: What's the pushback from the hyperscaler typically? They might say 'we can't guarantee silicon temperature because it depends on your workload.' Lucas: That's exactly the pushback. And it's fair — a poorly optimized training loop can run the GPU hotter than a well-optimized one. So the clause usually includes a carve-out: if the customer's workload exceeds a certain power draw or thermal design point, the SLA is void. The fintech firm agreed to cap their GPU utilization at 80 percent sustained and to use a specific kernel optimization library. They proved their workload was well-behaved, and the hyperscaler took on the risk. Luna: So it becomes a shared responsibility model for heat. I like that framing. Lucas: Exactly. And the data center industry is moving toward this anyway. The Open Compute Project has a standard for thermal telemetry that lets customers and providers share real-time temperature data. Some hyperscalers are already offering dashboards that show per gpu junction temps for reserved instances. If you're not asking for that data in your contract, you're leaving money on the table. Luna: What about the newer GPUs like the B200 or the upcoming Rubin from NVIDIA? Are they more or less sensitive to thermal runaway? Lucas: More sensitive, actually. The B200 has a higher thermal design power — around 1000 watts per GPU compared to 700 for the H100. And the packaging is more complex, with tighter thermal margins. So we're likely to see these provisions become standard for any next-gen GPU reservation. I've heard from a lawyer who negotiates cloud contracts that thermal runway clauses are now in about 40 percent of enterprise agreements for gpu heavy workloads, up from virtually zero in 2023. Luna: That's a rapid shift. And it's not just about cost — it's also about predictability. If you're training a model that takes two weeks, you need to know it won't slow down on day three because of a heat wave. Lucas: Right. And that's where the business case intersects with the engineering reality. CFOs care about the dollar value of delayed time to market for AI models. If a thermal throttle adds three days to a training cycle that's worth $500,000 in revenue per day, that's a $1.5 million hit. A thermal runway clause is cheap insurance. Luna: So what's the practical checklist for someone listening? They're about to sign a GPU commitment — what should they look for? Lucas: Three things. One: ask for a thermal SLA that specifies junction temperature, not just ambient room temperature. Two: get a reporting mechanism — either a dashboard or a monthly report — that shows actual temps. Three: negotiate graduated credits that kick in before you hit the hard throttle point. If you wait until 95 degrees, you've already lost performance. The sweet spot is around 85 to 87. Luna: And if the hyperscaler says they can't do it, what's the fallback? Luna: It feels like we're watching the cloud contract evolve in real time. A year ago, nobody was talking about GPU junction temperatures. Now it's a deal point. Lucas: That's the pace of AI infrastructure. The hardware is advancing so fast that the contracts have to catch up. I expect within two years, thermal runway provisions will be as standard as uptime guarantees for CPU workloads. It's just physics meeting procurement. Luna: And for the listener who wants to get ahead of that curve, the takeaway is: start collecting your thermal data now. Even if your renewal isn't for six months, you'll have the ammunition when you sit down at the table. Lucas: Exactly. And if you want to see a sample clause, I'll include a link in the show notes to a template that one of the cloud contract lawyers shared with me. It's anonymized but it shows the structure. That's it for this episode — we'll be back next week with another deep dive into the fine print of cloud infrastructure.