Latest / The Cloud Business Podcast with Fexingo: AWS, Azure, GCP, and Enterprise Infrastructure / Why AWS and Azure Are Racing to Offer Custom Silicon to Enterprise Clients
Transcript
- Lucas: So a few weeks ago I was looking at a monthly cloud bill for a mid-size fintech company — about six hundred thousand dollars a month, mostly on AWS — and I noticed something I hadn't seen before. Nearly half their compute spend was on Graviton instances. Not Intel, not AMD. AWS's own Arm-based chips. Luna: That's a big shift. A year or two ago, Graviton was still seen as the experimental option, right? You'd use it for batch jobs or stateless workloads, not core production. Lucas: Exactly right. And that's the story I want to unpack today — because it's not just AWS. Microsoft is pushing their Cobalt chips hard on Azure. Google has had Tensor Processing Units for years, but they're also now talking about general-purpose Axion processors. The hyperscalers are basically becoming chip companies. Luna: And that has huge implications for Intel and AMD, but also for enterprise customers who are trying to figure out whether these custom chips actually save money or just lock them in deeper. Lucas: Let's start with the numbers, because they're striking. AWS says Graviton3 instances deliver up to 40 percent better price-performance than comparable x86 instances. That's not a marginal improvement. For a company spending ten million a year on compute, moving to Graviton could mean four million in savings — or the ability to run the same workloads for less. Luna: Forty percent is huge. But is that apples to apples? I've heard people argue that Graviton really shines on certain workloads — like web servers, containerized apps — but not on everything. Lucas: That's a fair point. The performance gain varies. For compute-intensive HPC workloads or software that relies on specific Intel instruction sets, the advantage narrows. But here's what's changing: AWS has been optimizing its own services — Lambda, ECS, EKS — to run natively on Graviton. So if you're using managed services, the migration path is getting smoother. And they've been at this for years — Graviton launched in 2018, and we're now on the third generation. Luna: Right, so the ecosystem has matured. But what about Microsoft? How far along are they with custom silicon? Lucas: Microsoft announced the Cobalt 100 chip in late 2023, and it's now available on Azure. It's an Arm-based processor built for general-purpose cloud workloads. Microsoft claims up to 30 percent better performance per watt compared to x86. And they're not stopping at general compute — they also have the Maia AI accelerator, which competes with NVIDIA's GPUs and Google's TPUs for AI training and inference. Luna: So both AWS and Microsoft are betting that Arm-based designs — which used to be associated with mobile phones and low-power devices — are the future of the data center. What about Intel? They must be watching this closely. Lucas: Intel is in a tough spot. For decades, they dominated the server market, with over 90 percent share at one point. Now analysts estimate their share in cloud data centers has dropped to around 60 percent, and it's declining. AMD has gained ground with their EPYC chips, but the bigger threat is the hyperscalers designing their own silicon. If AWS and Azure can squeeze better performance per dollar out of their own chips, why would they keep buying Intel? Luna: But isn't there a risk for customers? If you build your infrastructure around Graviton or Cobalt, you become more dependent on a single cloud provider. Portability goes down. Lucas: That's the classic lock-in argument, and it's real. If you optimize your software for Arm-based chips and the specific architecture of AWS's Nitro hypervisor, moving to Azure or Google Cloud becomes harder. But the counterargument is that the savings are so significant that even if you're locked in, you're still ahead on cost. And enterprises are increasingly comfortable with multi-cloud strategies where they might use Graviton on AWS and Cobalt on Azure for different workloads. Luna: Let's talk about the economics from the cloud provider's perspective. Why are they all racing to build their own chips? It can't just be about giving customers a discount. Lucas: No, it's about margin. Cloud providers have razor-thin margins on raw compute. By designing their own chips, they can eliminate Intel's markup, customize features for their own workloads, and optimize for power consumption — which is a huge cost in data centers. Think about it: AWS runs millions of servers. If each Graviton chip saves them fifty dollars in power per year, that's millions in extra profit. Plus, they can offer lower prices to customers and still make money, which helps them win market share. Luna: So it's a competitive moat. And it's also a way to differentiate their AI offerings. Microsoft's Maia chip is designed specifically for AI training, and Google's TPUs have been powering their AI services for years. AWS has Trainium and Inferentia for AI workloads. Lucas: Exactly. AI is the killer use case. Training large language models on NVIDIA GPUs is incredibly expensive. If AWS or Azure can offer a chip that does the job at half the cost, they become the go-to platform for AI startups and enterprises. And they're already seeing adoption. AWS says Trainium-powered instances are used by companies like Anthropic and Airbnb for AI training. Microsoft is positioning Maia to power its own Copilot services and for customers running OpenAI models. Luna: Let's bring this back to the enterprise listener. If I'm a CIO or a cloud architect, what should I be doing right now to evaluate these custom chips? Lucas: First, don't assume you need to migrate everything. Start with workloads that are stateless, containerized, and not reliant on legacy x86 instructions. Web servers, microservices, and batch processing are great candidates. Run benchmarks. Both AWS and Azure offer free trials for their custom chip instances. Compare price-performance with your current x86 setup. And look at your managed services — services like AWS Lambda now run on Graviton by default for new functions. Luna: And what about the long-term? Are we going to see all cloud compute move to custom silicon in five years? Lucas: I think it's heading that way, but not all at once. Intel and AMD will still have a place, especially for legacy apps and workloads that require specific instruction sets. But the hyperscalers control the hardware stack, and they have every incentive to push their own chips. If you're an enterprise starting a new project today, I'd seriously consider building on Arm-based cloud instances from the start. It'll likely be cheaper and future-proof you for the next wave of cloud infrastructure. Luna: That makes sense. And I think we should also mention the supply chain angle. By designing their own chips, these companies reduce dependence on Intel and AMD, which have faced supply constraints in the past. Lucas: Great point. TSMC manufactures all these custom chips — AWS, Microsoft, Google all use TSMC's fabrication. So they're still dependent on one foundry, but they have more control over the design and allocation. It's a strategic move to ensure they can scale their data centers without being bottlenecked by an external chip supplier. Luna: So to wrap up: the cloud custom silicon race is a structural shift that's already saving enterprises money, but it also introduces new lock-in risks. The smart approach is to test, benchmark, and migrate selectively. Lucas: Right. And keep an eye on the next generation. Graviton4 is on the horizon, and Microsoft's Cobalt 200 is likely in development. The performance improvements are coming fast. For cloud buyers, this is one of the most important trends to track over the next couple of years. Luna: Great discussion. Let's check back in six months and see how much more market share these custom chips have captured. Lucas: Absolutely. That's our show for today. Thanks for listening.