Latest / The Cloud Business Podcast with Fexingo: AWS, Azure, GCP, and Enterprise Infrastructure / How Cloud Regions Are Auctioning Compute to Fill Gaps
Transcript
- Lucas: So cloud providers are now running what are essentially commodity auctions for compute capacity — and enterprises that understand the game can cut their infrastructure costs by 80 percent or more. Luna: We're talking about spot instances, right? AWS started that years ago, but it feels like the whole model has matured. Lucas: Exactly. AWS's spot market alone now handles over 2.5 million instance launches every single day. That's not a niche offering — it's a massive secondary market for spare compute. Azure has its low-priority VMs, Google has preemptible VMs, and even Oracle is getting into the game with bare metal instances that can be interrupted. Luna: And the pricing is dynamic, right? It fluctuates based on supply and demand in real-time. Lucas: Yes. For a standard m5.large instance on AWS, the on-demand price is about 0.096 dollars per hour. The spot price can drop to 0.024 — that's a 75 percent discount. For GPU instances, it's even more extreme. I've seen p3.2xlarge spot prices at 0.45 dollars per hour versus on-demand at over 3 dollars. That's an 85 percent discount. Luna: But the catch is termination. If the spot price rises above your bid, or if AWS needs the capacity back, your instance gets terminated with a two-minute warning. That's scary for production workloads. Lucas: It's a risk, but enterprises are building around it. Take a fintech company I spoke with last month. They run Monte Carlo simulations for risk modeling — those are stateless, fault-tolerant workloads. They use spot instances for the heavy lifting and save about 80 percent compared to on-demand. They've built a checkpointing system that saves state every five minutes, so if an instance is terminated, they just spin up a new one and resume. Luna: So the architecture has to be designed for interruption from the start. Lucas: Exactly. That's the key insight. The companies that succeed with spot instances have built their applications to treat compute as ephemeral. They use auto-scaling groups that mix on-demand and spot, so if spot capacity dries up, the on-demand instances handle the overflow. Or they use AWS's Spot Fleet, which lets you define a target capacity and automatically bids across instance types and availability zones to maximize savings. Luna: I've seen that with a media company that does video transcoding. They use Azure's low-priority VMs for batch jobs. They save about 70 percent, but they also have a fallback pool of standard VMs for critical jobs. Lucas: Right. And the bidding strategies have become sophisticated. Initially, you placed a bid price, and if the spot price went above your bid, you lost the instance. But AWS changed that — now spot instances are priced the same for everyone, and you just pay that current spot price unless you set a max bid. The real innovation is that you don't have to bid at all — you can just use the 'request spot instances' feature and AWS figures it out. Luna: Wait, so there's no more bidding war? How does that work? Lucas: AWS still calls it a 'spot market' but the pricing is now set by AWS based on long-term supply and demand trends. It's more stable. The price changes infrequently — maybe once a day. That makes it easier for enterprises to budget. But the trade-off is that you still have the interruption risk. The price might stay low, but if capacity is needed, your instances get reclaimed. Luna: So it's not really an auction in the traditional sense anymore. Lucas: It's a pseudo-auction. But Google's preemptible VMs are simpler — they're always the same price, up to 80 percent off, but they last a maximum of 24 hours. After that, they're terminated. So you get a fixed discount but with a hard time limit. Azure's low-priority VMs are somewhere in between: you get a discount but the termination is less predictable. Luna: And what about the new players? Oracle has been pushing bare metal instances with interruptible pricing. Lucas: Yes. Oracle offers 'preemptible' bare metal instances for 50 percent off on-demand. That's interesting because bare metal gives you full control over the hardware, no hypervisor overhead. Some enterprises with latency-sensitive workloads or specific compliance requirements are using those for batch processing. Luna: But the real story might be the rise of third-party brokers. There are companies like Spot by NetApp and CAST AI that automate the bidding and instance selection across multiple clouds. Lucas: Exactly. These tools analyze your workload patterns and decide in real-time which instance type and which cloud to use. For example, if AWS spot prices spike in a particular region, the tool might shift GPU workloads to Azure or Google Cloud. That's true multi-cloud optimization. Luna: And that's where the cost savings compound. One enterprise I read about saved 60 percent on their cloud bill just by using CAST AI's automation for Kubernetes clusters. They had a mix of spot and reserved instances, but the tool continuously optimized the blend. Lucas: It's a whole new layer of cloud financial operations. We've talked about FinOps before, but this is a specific subset: capacity arbitrage. Enterprises are essentially trading compute like a commodity. Luna: And that brings up the question of risk management. How do you balance savings against the potential for disruption? Lucas: It depends on the workload. For stateless, fault-tolerant jobs — like batch data processing, CI/CD pipelines, rendering, and simulation — you can go all-in on spot. For stateful applications like databases, you probably want on-demand or reserved instances. But even there, you can use spot for read replicas or for scaling out during traffic spikes. Luna: Right. Netflix uses spot instances heavily for its encoding and recommendation workloads. They've built a system called 'Chaos Monkey' that intentionally terminates instances to test resilience — so their architecture is already designed for failure. Lucas: That's the perfect example. They treat instance failure as a feature, not a bug. And the savings fund more innovation. Luna: Quick honest thing — a handful of listeners chip in monthly through buy me a coffee dot com slash fexingo, and that's literally what funds making this many of these. If these conversations are useful for what you're building or running, that support keeps it ad-free and going. Lucas: Yeah, it really does. And we keep doing this because we love digging into the details. So back to the auction model — what's next? I think we'll see more cloud providers offering 'guaranteed spot' with a maximum interruption rate, like GCP's 'committed use discounts' but for spot. Luna: Google already has something like that. They have 'preemptible VMs with a duration' where you can guarantee up to 24 hours, but after that, it's gone. Lucas: True. But I'm talking about a service where you pay a premium for a lower interruption probability. Imagine an 'interruptible' tier that costs 40 percent less instead of 80 percent less, but AWS guarantees it won't be terminated more than once a month. That would open spot instances to more workloads. Luna: That's a smart idea. The economics would still work for both sides. Lucas: And we're already seeing it in the form of 'capacity pools' for AI training. Some cloud providers are offering discounted GPU access if you agree to allow interruptions. For training runs that can be paused and resumed, it's a huge win. Luna: So the trend is clear: the cloud compute market is becoming more like a financial exchange, with spot, futures, and options — reserved instances are like futures contracts, spot is like the spot market. Lucas: That's exactly the analogy. And just like in finance, the smartest players aren't just buying at the market price — they're using algorithms, hedging, and arbitrage to get the best deal. Luna: For our listeners, the takeaway is: if you haven't looked at spot instances for your non-critical workloads, you're leaving money on the table. Start with one job queue, build in checkpointing, and measure the savings. Lucas: And don't just look at AWS. Compare across clouds. Oracle's bare metal spot, Google's preemptible, Azure's low-priority — each has different characteristics. The best strategy might be a multi-cloud spot approach. Luna: And use a broker if you can. The automation pays for itself. Lucas: Right. The cloud auction model is here to stay. It's not just about saving money — it's about building resilient, efficient systems that treat compute as a variable resource. That's the future.