Latest / The Cloud Business Podcast with Fexingo: AWS, Azure, GCP, and Enterprise Infrastructure / The Cloud Capacity Auction Enterprises Are Winning
Transcript
- Lucas: There's a quiet shift happening in how big enterprises buy cloud compute. It used to be a straightforward negotiation — you sign a one- or three-year commitment, you get a discount. But now some of the largest buyers are treating cloud capacity like a commodity auction. Luna: You mean like bidding for spot instances? That's been around for years. Lucas: Right, but spot has always been volatile — great for fault-tolerant batch jobs, but no one would bet a production pipeline on it. What's new is that enterprises are building internal brokers that blend spot, reserved, and on-demand capacity dynamically. They're treating the hyperscalers like interchangeable utilities. Luna: So they're effectively running their own mini cloud exchange? Lucas: Exactly. And the financial incentive is enormous. One large financial institution I spoke with — a global bank — uses a third-party capacity broker that monitors pricing across AWS, Azure, and GCP in real time. They shift workloads every fifteen minutes based on where compute is cheapest. Last year they cut their cloud bill by 37 percent. Luna: Thirty-seven percent — that's not marginal. But doesn't that add a lot of complexity? You need to containerize everything, make sure workloads are portable. Lucas: It does, but that's exactly the trade-off. The bank invested heavily in Kubernetes and a custom scheduler that understands the cost surfaces of each cloud. And the hyperscalers are actually enabling this — they're releasing more granular pricing APIs and shorter commitment terms. Lucas: If these conversations about building smarter infrastructure are useful for what you're running or building, listener support is what keeps this show ad-free and independent. People who find value here can help at buy me a coffee dot com slash fexingo. That's it — and it genuinely makes a difference. Luna: Yeah, and we appreciate it. So Lucas, you mentioned three-year commitments are changing — what does that look like in practice? Lucas: AWS just introduced one-year convertible reserved instances that can be swapped between instance families mid-term. Azure has something similar with their savings plans. And Google Cloud is offering what they call 'committed use discounts' that can now be applied across regions. The old model of locking into a specific instance type for three years is fading. Luna: That sounds good for buyers, but isn't it squeezing margins for the cloud providers? They built their business on sticky, predictable revenue. Lucas: It definitely compresses margins on compute. But the hyperscalers are making it up on the AI side — GPU instances for training are priced at a huge premium, and those contracts are still long-term. So you have this two-tier system: low-margin, flexible compute for general workloads, and high-margin reserved capacity for AI. Luna: So the auction dynamic is really only for cpu based workloads and some inference? Training is still locked in? Lucas: Mostly, yes. NVIDIA's H100 and B200 GPUs are still scarce enough that cloud providers can demand multi-year commitments. But even there, I'm seeing early signs of a secondary market — companies that reserved too much GPU capacity are reselling it on third-party exchanges. Luna: That's fascinating. So we might end up with a true spot market for GPUs. Lucas: Exactly. And that would change the economics of AI training significantly. If you can bid for idle GPU time at a fraction of the on-demand price, more startups can afford to train foundation models. Luna: Let's talk about the broker role. You mentioned a third-party capacity broker for that bank. Who are these players? Are they consultants, software platforms? Lucas: Both. There are pure software platforms like Cast AI and Spot by NetApp that automate workload placement. And there are consulting firms like Accenture and a few boutique shops that help enterprises build their own brokers. The key is the real-time pricing feed — you need to ingest cost data from each cloud every few minutes. Lucas: One platform I looked at — a startup called Zesty — claims it can reduce compute costs by an average of 30 percent just by continuously optimizing reserved-instance purchases. They have a feature that automatically buys and sells reservations as your usage changes. It's like a quantitative trading desk for cloud capacity. Luna: That sounds incredibly powerful, but also risky. What if the broker makes a mistake and your production workload doesn't have enough capacity? Lucas: That's the fear. The best brokers use machine learning to predict your baseline demand and only expose the variable portion to the auction. They keep a safety buffer of on-demand capacity. The bank I mentioned has a rule: never let more than 40 percent of total compute be on spot or short-term reservations. That way even if prices spike, they can fall back. Luna: So it's a risk management problem, not just a cost optimization one. Lucas: Exactly. And that's why CFOs are starting to hire cloud procurement specialists — people who understand both finance and infrastructure. It's a new role. I've seen job postings for 'Cloud Capacity Manager' with salaries in the two-hundred-thousand-dollar range. Luna: That's a big investment for a company. How many enterprises are actually doing this today? Lucas: Still a minority — maybe 15 percent of large enterprises with multi-cloud strategies. But the number is growing fast. Gartner estimates that by 2028, 40 percent of enterprise cloud spend will go through some form of dynamic capacity optimization. The ones who aren't doing it are leaving a lot of money on the table. Luna: Are there any downsides for the cloud providers? They must hate the margin compression. Lucas: They do, but they also see it as inevitable. AWS's CFO said in an earnings call last year that they're moving to a 'utility model' where price is determined by supply and demand. They'd rather cannibalize their own high-margin reserved instances than lose the customer to a competitor offering more flexibility. Luna: So it's a race to the bottom on compute, but the providers are competing on AI and services? Lucas: Exactly. Compute is becoming a loss leader. The money is in the higher layers — SageMaker, Vertex AI, Azure OpenAI Service. The hyperscalers are fine with thin margins on raw VMs if they can lock you into their AI platform. Luna: That explains why Google Cloud is so aggressive with their network pricing. They know if they can win the compute auction, they'll upsell you on their AI tools. Lucas: Right. And the enterprises doing this well are the ones that separate their compute procurement from their AI platform choice. They buy compute at the lowest cost, then use a multi-cloud ML platform like Kubeflow or MLflow that abstracts away the underlying cloud. Luna: So the real skill is architectural neutrality. Build so you can move, then optimize ruthlessly. Lucas: That's the playbook. And the enterprises that master it are going to have a structural cost advantage over competitors that just sign a three-year deal with one provider and forget about it. Luna: What's the single most important thing a cloud buyer should do in the next six months to start moving toward this model? Lucas: Audit your current reserved-instance portfolio. You'd be surprised how many companies are paying for reservations they aren't using. Then start small — pick one non-critical workload, containerize it, and run it on spot for a month. Measure the savings. Once you see the number, you'll be motivated to go deeper. Luna: That's a concrete starting point. And it doesn't require a massive upfront investment. Lucas: Exactly. One team at a time. The auction is happening whether you participate or not. The question is whether you're on the buy side or the sell side.