Latest / The Cloud Business Podcast with Fexingo: AWS, Azure, GCP, and Enterprise Infrastructure / Why Cloud Contracts Now Include GPU Temperature Caps
Transcript
- Lucas: So I was reading a cloud contract last week—one from a large European bank that's negotiating a multi-year deal with one of the big three hyperscalers—and I noticed a clause I hadn't seen before. Luna: Let me guess: something about GPU temperature. Lucas: Exactly. A GPU temperature cap. The contract specifies that the junction temperature on any GPU provisioned under the agreement must stay below 85 degrees Celsius, averaged over any rolling 24-hour period. If it goes above, the bank gets service credits. Luna: Wow. That's pretty specific. I mean, GPUs run hot—everyone knows that. But putting it in the contract? That's a new level of granularity. Lucas: It is. And it makes sense when you think about what's happening inside these data centers. The latest NVIDIA H100s and the newer B200s—they're incredibly power-dense. A single H100 can draw 700 watts under load. Pack a rack full of them and you're looking at 40 kilowatts or more per rack. That's orders of magnitude more than a traditional CPU rack. Luna: Right, and heat is the enemy of performance and longevity. I've read that sustained temperatures above 85 degrees can degrade the chip's lifespan significantly. Plus, you start hitting thermal throttling, which kills your training throughput. Lucas: That's the key. For a bank running AI models for fraud detection or risk analytics, consistent training speed matters. If the cloud provider's cooling can't keep up during a heatwave, the GPUs throttle, the job takes longer, and the bank's model deployment deadline slips. That's real money. Luna: So the bank essentially said, 'We don't want to absorb that risk. If you can't keep the chips cool, you compensate us.' Lucas: Exactly. And the hyperscaler agreed. Now, this isn't widespread yet. I've probably looked at a hundred cloud contracts this year, and this is the first one I've seen with an explicit temperature cap. But I think it's going to become more common. Luna: Why now? I mean, GPUs have been hot for years. What changed? Lucas: Two things. First, the density per rack has gone up dramatically. Second, AI training jobs are now so long—sometimes weeks or months—that even short periods of throttling have an outsized impact. The bank I mentioned runs training jobs that last three to four weeks. If they lose even 5 percent throughput for a few days, that's a meaningful delay. Luna: And the cloud providers are offering these terms? That seems like a big operational risk for them. Lucas: It is, but they're betting on their cooling infrastructure. The hyperscalers have been investing heavily in advanced cooling. Google uses liquid cooling in some of its TPU pods. Microsoft is deploying two-phase immersion for some workloads. AWS has its own cooling innovations. They're confident they can meet the 85-degree threshold most of the time. Luna: Most of the time. But what about a heatwave in Northern Virginia, where a huge chunk of cloud capacity sits? Lucas: That's the risk. The bank's data center is in Frankfurt, where temperatures are generally moderate. But if there's a record summer, the chillers might struggle. The hyperscaler knows that, so they're likely pricing this clause into the contract—charging a slight premium for the thermal guarantee. Luna: So it's not just a technical clause, it's a financial one. The bank pays a bit more for the warm fuzzy feeling that their GPUs won't bake. Lucas: Exactly. And the hyperscaler uses that premium to invest in better cooling, or to self-insure against the occasional credit payout. Luna: What's the actual temperature spec? I know NVIDIA recommends a maximum junction temperature of 85 degrees for the H100. So the bank is basically asking the cloud provider to stay within the manufacturer's spec. Lucas: That's right. It's not an arbitrary number. It's the spec. The bank is saying, 'You're selling me H100s. Keep them within their operating range.' Seems reasonable. But the thing is, many cloud providers don't guarantee that in their standard terms. They'll say 'best effort' or 'typical performance.' This clause makes it a contractual obligation. Luna: And if the provider fails, they pay credits. Do you know what the credit structure looked like? Lucas: It was tiered. For every hour the average junction temperature exceeds 85 degrees, the bank gets a 10 percent credit on the GPU compute charges for that hour. If it exceeds 90 degrees, the credit jumps to 25 percent. And if it exceeds 95 degrees, the bank can terminate the contract without penalty. Luna: That's a strong incentive for the cloud provider to get cooling right. I'm curious—how do they measure that? It's not like they put a thermometer on each GPU. Lucas: They do, actually. Every modern GPU has on-die temperature sensors. The hyperscaler can report average junction temperature from the fleet. The bank's contract requires that data to be shared monthly, and they can audit it. So it's verifiable. Luna: Okay, so this is a clause that makes a lot of sense for ai heavy enterprises. But what about the broader market? Is this going to trickle down to smaller customers? Lucas: I think we'll see temperature-related SLAs become more common for any gpu intensive workload within the next two years. But for smaller customers, it might be bundled into a broader 'AI infrastructure guarantee' that covers performance, temperature, and maybe even interconnect bandwidth. Luna: It's almost like we're seeing the cloud contract evolve from a simple resource rental to a more nuanced performance contract. Similar to how we've seen carbon-aware scheduling clauses emerge. Lucas: Exactly. The contract is becoming a tool for managing operational risk, not just cost. And temperature is a perfect example—it's a physical constraint that directly affects compute output. Luna: Speaking of tools that make a difference—a couple of dollars a month is genuinely what keeps these conversations going. If you've gotten something out of this episode, buy me a coffee dot com slash fexingo. It's a small way to support the show and keep it ad-free. Lucas: Yeah, we really appreciate that. And it makes a real difference for us. Luna: So back to temperature caps—one thing I wonder about is the interaction between this clause and the cloud provider's own cooling strategies. For example, if they use evaporative cooling, water consumption might spike during a heatwave. Do those tradeoffs show up in contracts yet? Lucas: Not yet, but I think they might. Some hyperscalers are already publishing water usage effectiveness, or WUE. I could imagine a future contract that ties GPU temperature guarantees to a maximum WUE—so the provider can't just dump water on the problem. Luna: That would be fascinating. So the temperature cap isn't just about hardware preservation—it's also about environmental impact. Lucas: Right. And it's another example of how cloud contracts are becoming more sophisticated. We've seen carbon credits, now temperature caps. I think the next frontier might be latency guarantees tied to physical distance between GPUs—for distributed training. Luna: Makes sense. If you're training a model across multiple regions, the network latency between GPU clusters can kill performance. A contract that guarantees inter-region latency below a certain threshold would be huge. Lucas: Exactly. And that's a much harder problem than temperature, because it involves network routing, peering, and physical fiber paths. But I wouldn't be surprised if we see it in a year or two. Luna: So for now, temperature caps are the leading edge. For any enterprise negotiating a gpu heavy cloud deal, it seems like a no-brainer to ask for it. Lucas: Absolutely. And if you're a provider, you better have your cooling game ready. Because once one customer gets it, others will demand it too. Luna: And that's the kind of market pressure that drives innovation in data center design. Lucas: Exactly. So keep an eye on your GPU temps—they might become a line item in your next cloud bill.