Latest / The Cloud Business Podcast with Fexingo: AWS, Azure, GCP, and Enterprise Infrastructure / How Cloud Regions Are Competing for Enterprise AI Workloads
Transcript
- Lucas: So here's a number that stuck with me this week. A recent survey of enterprise cloud architects found that 72 percent of them now rank 'AI inference latency' as their top consideration when choosing a cloud region for new workloads. That's up from 28 percent just two years ago. Luna: Wow. That is a massive swing. So the old criteria—cost per compute hour, data egress fees, maybe compliance—have been overtaken by how fast a model can return an answer? Lucas: Exactly. And it's a direct consequence of AI workloads moving from training to inference. Training can tolerate batch processing in a low-cost region with cheap power. But inference—especially for customer-facing applications like chatbots, real-time fraud detection, or code generation—needs to be close to the user. Luna: And that's forcing the cloud providers to rethink their geographic expansion. They can't just build in Northern Virginia and Frankfurt anymore. Lucas: Right. So let's look at what each of the big three is doing. AWS recently opened a region in Malaysia—their first in Southeast Asia outside of Singapore. And they specifically marketed it as an 'AI inference hub' for the region, with low-latency connections to Indonesia, Thailand, and Vietnam. Luna: Interesting. So they're not just saying 'here's compute capacity.' They're saying 'here's compute capacity for your AI models that need to be fast for users in this part of the world.' Lucas: Exactly. And Azure is doing something similar with their new region in Spain—but they're tying it to renewable energy. Microsoft has committed to 100 percent renewable energy for that region by 2027, which is a major selling point for enterprises with net-zero commitments. Luna: So the pitch becomes: low inference latency for Western European users plus green credentials. That's a double win. Lucas: Precisely. And GCP is taking a slightly different approach. They're doubling down on what they call 'edge inference locations'—smaller footprint data centers in tier-two cities like Milan, Warsaw, and Doha. These aren't full regions; they're essentially GPU pods that can run inference with single-digit millisecond latency. Luna: So it's almost like they're creating a distributed inference network rather than a handful of giant regions. That's clever if you have workloads that need to be everywhere. Lucas: And it's a direct response to a problem enterprises are hitting: the major cloud regions are getting crowded. In Northern Virginia, for example, AWS has been struggling with capacity for certain GPU instances. I've heard stories of enterprises waiting weeks for H100 availability. Luna: Weeks? That's a lifetime for a company trying to launch a new AI feature. Lucas: Exactly. So what we're seeing is a new form of cloud region arbitrage. Enterprises are deliberately distributing their AI workloads across multiple regions—not just for redundancy, but to ensure they can get the GPU capacity they need. Luna: Can you give a concrete example? How does that work in practice? Lucas: Sure. Take a financial services firm running a real-time fraud detection model. They might deploy inference endpoints in three regions: one in us-east-1 for primary traffic, one in eu-west-2 for their London office, and one in ap-southeast-1 for Asia-Pacific. But they also have a backup region in us-west-2 that they pre-provision with GPU capacity—just in case. Luna: So they're essentially paying for reserved capacity in multiple regions to guarantee uptime and latency. That must get expensive. Lucas: It does. But it's becoming table stakes. And here's where it ties back to contract negotiation: enterprises are now asking cloud providers for region-level GPU capacity commitments as part of their enterprise agreements. They want a guarantee that they can spin up a certain number of GPUs in a specific region within a specific timeframe. Luna: So it's like a service-level agreement for GPU availability, not just uptime. Lucas: Exactly. And the providers are starting to offer it—but with caveats. AWS has something called 'Capacity Blocks for ML,' which lets you reserve GPU instances in specific regions for a set period. But it's not available in all regions, and the pricing is dynamic based on demand. Luna: So it's still a seller's market for GPUs. But it sounds like the providers are at least acknowledging the problem. Lucas: They have to, because enterprises are getting more sophisticated. I talked to an architect at a large retail company who told me they now run a monthly 'region health check' for their AI workloads. They measure latency, GPU availability, and cost across five regions and dynamically shift traffic based on that data. Luna: So they've essentially built their own multi-region inference router. That's pretty advanced. Lucas: And it's becoming a competitive differentiator. If you can serve your AI features faster and cheaper than your competitors, that's a real advantage. Which brings us to another factor: data residency. Luna: Right. Because not all regions are created equal when it comes to regulatory compliance. You can't just put your European user data in a U.S. region and call it a day. Lucas: Exactly. So we're seeing cloud providers opening regions specifically to satisfy data residency requirements for AI. Azure's new region in Switzerland was explicitly marketed for financial services AI workloads. AWS's region in Israel is pitched to defense and cybersecurity AI. Luna: So the region itself becomes a product feature. 'Run your AI here, and you're compliant by default.' Lucas: Precisely. And that's a powerful selling point. But it also creates complexity for enterprises that operate globally. You end up with a patchwork of regions, each with different GPU availability, latency profiles, and pricing. Luna: That sounds like a nightmare to manage. Is there any tooling emerging to help? Lucas: There is. There are now third-party platforms that act as a 'cloud region broker' for AI workloads. They abstract away the region selection and automatically route inference requests to the optimal region based on latency, cost, and compliance rules. It's like a content delivery network, but for AI inference. Luna: So that's a whole new category of cloud management software—AI inference orchestration. Lucas: Exactly. And some of these startups are already seeing significant traction. One I spoke to said they're managing inference traffic across 12 regions for a single enterprise customer, and they've reduced average latency by 40 percent while cutting costs by 15 percent. Luna: Okay so that's the kind of number that makes a CFO sit up and pay attention. Lucas: For sure. And speaking of things that make the show possible—if these conversations are useful for what you're building or running, you can support us at buy me a coffee dot com slash fexingo. It's a small gesture that keeps the podcast independent and ad-free. Luna: Yeah, we really appreciate that. And honestly, if today's episode was worth a coffee to you, that's the link. Lucas: Alright. back to regions. One thing I want to highlight is the role of government incentives. A few countries are offering tax breaks and subsidies specifically for AI cloud infrastructure. Saudi Arabia, for example, has been aggressively courting cloud providers to build regions there, offering discounted land and power. Luna: And that's not just about data residency—it's about becoming an AI hub. They want to attract enterprises that will build AI applications in the region. Lucas: Exactly. And it's working. Oracle recently announced plans for a second region in Saudi Arabia, and Google has a region in Dammam. These are becoming real alternatives for enterprises serving the Middle East and Africa. Luna: So the cloud region map is being redrawn by AI demand. The old model of 'build a region in every major metro' is being replaced by 'build regions where AI inference needs to be fast and where incentives are favorable.' Lucas: I think that's exactly right. And the implication for enterprises is that region strategy is now a first-order AI decision. You can't just pick one region and forget it. You need to think dynamically about where your users are, where your data can live, and where you can get the GPU capacity you need. Luna: So what's the one thing you'd recommend an enterprise cloud architect do this quarter to prepare? Lucas: Start auditing your AI workload's latency requirements. Measure actual inference times from your current regions. You might find that moving some workloads to a new region—like AWS's Malaysia region for Southeast Asian users—could cut latency by 50 percent or more. And that directly impacts user experience. Luna: Good advice. And with that, I think we've covered quite a bit. Thanks Lucas. Lucas: Thanks Luna. For our listeners, we'll be back soon with more on cloud business.