Latest / The CEO Diary with Fexingo: Leadership Lessons, Executive Decisions, and Corner Office Stories / How Nvidia Kept Its Edge After the AI Chip Boom
Transcript
- Lucas: Nvidia is worth over three trillion dollars. Its stock has quadrupled in two years. And the conventional wisdom is that it got lucky — that AI just happened to need exactly what Nvidia had been building for gaming. Luna: Right, the timing narrative. But I think that undersells the decisions Jensen Huang made long before ChatGPT existed. Lucas: Exactly. Today I want to look at one specific strategic move that most people overlook when they talk about Nvidia's moat. It's not the H100 chip itself. It's the ecosystem around it — specifically CUDA, the parallel computing platform Nvidia launched back in 2007. Luna: So this is the software layer that lets developers write code once and run it on any Nvidia GPU. I've heard it described as Nvidia's 'tax' on AI. Lucas: That's the cynical take, but it gets at something real. Nvidia invested billions into CUDA for over a decade before AI was a mainstream business. By the time researchers needed to train neural networks, CUDA was the only mature platform for GPU computing. TensorFlow and PyTorch — the two dominant AI frameworks — were built on CUDA from day one. Luna: So switching costs are enormous. If you're a startup or even a large tech company, retraining your models on a competitor's architecture means rewriting your entire software stack. Lucas: Right. And it's not just about rewriting. It's about the entire toolchain — debugging, profiling, optimization libraries. AMD has its own platform called ROCm, but it's years behind in maturity. The developer mindshare is overwhelmingly Nvidia. Luna: Let's put some numbers on this. Nvidia claims over four million developers use CUDA. That's a lot of inertia. Lucas: And that number has been growing for nearly two decades. Every computer science student who takes a parallel computing class learns on CUDA. Every research paper that benchmarks a new AI model runs on Nvidia hardware. It's a self-reinforcing cycle. Luna: But there is a counter-argument. The hyperscalers — Amazon, Google, Microsoft — are all designing custom AI chips now. Google has its TPU, Amazon has Trainium and Inferentia. If they can match performance, doesn't that break Nvidia's grip? Lucas: It's the biggest threat, and Nvidia knows it. That's why they moved aggressively into networking. In 2019, Nvidia acquired Mellanox for about seven billion dollars. Mellanox makes high-speed interconnects — InfiniBand and Ethernet switches — that connect thousands of GPUs in a data center. Luna: So Nvidia owns not just the compute but the fabric that ties it together. That makes it harder for a cloud provider to swap in a custom chip without also replacing the networking layer. Lucas: Exactly. And Nvidia has been integrating Mellanox technology into its own GPU systems, creating what they call the 'Nvidia network platform.' They now sell entire racks of AI infrastructure — DGX systems — that are pre-configured with their compute, their networking, and their software. A cloud provider could theoretically build a competitive system using custom chips and third-party networking, but the integration risk and engineering cost are high. Luna: It reminds me of how Apple controls both the hardware and software on the iPhone. The integration creates a premium experience that's hard to replicate piece by piece. Lucas: That's a fair analogy. Nvidia is essentially building a vertically integrated AI platform. And they're not stopping at hardware and software. They're also offering a cloud service called DGX Cloud, where companies can rent Nvidia's infrastructure directly without buying hardware. That shifts the business model from one-time sales to recurring revenue. Luna: So even if a customer eventually builds their own in-house AI cluster, they're already locked into Nvidia's ecosystem through software and training. The switching costs compound over time. Lucas: And that's the key insight. The AI chip market is often portrayed as a race between Nvidia, AMD, and custom silicon. But Nvidia is playing a different game. They're selling a complete platform, not just a component. Luna: Speaking of platforms, it's interesting how these kinds of deep dives into business strategy are exactly what we try to do on this show. I know listeners tell us they use these conversations to think more clearly about their own industries. If that sounds like you, and you want to support the show staying ad-free — no sponsors, no annoying interruptions — you can buy us a coffee at buy me a coffee dot com slash fexingo. It's a small gesture that makes a big difference in keeping this going. Lucas: Yeah, it really does help us keep the focus on the substance rather than the commercial breaks. So, back to Nvidia — let's talk about the competitive landscape from AMD. Luna: AMD has been trying to crack into the data center GPU market with its Instinct line. They have a new chip, the MI300X, that they claim matches or beats Nvidia's H100 on raw performance. But market share tells a different story. Lucas: Analysts estimate Nvidia holds over 80 percent of the data center GPU market. AMD is in the single digits. And it's not just about performance — it's about the ecosystem gap. AMD's ROCm software has improved, but it still doesn't support as many AI frameworks out of the box. Developers have to do more work to get the same results. Luna: And there's the matter of supply. Nvidia managed to secure massive allocation from Taiwan Semiconductor Manufacturing Co. early in the AI boom. They locked up capacity for high-bandwidth memory from SK Hynix and Samsung. That supply chain muscle is hard to replicate quickly. Lucas: Right. Nvidia's relationship with TSMC is strategic. Jensen Huang was personally involved in negotiating wafer allocation. They pre-paid billions to secure capacity. That means even if AMD designs a competitive chip, they may not be able to produce enough volume to challenge Nvidia's lead. Luna: So the moat is multi-layered. Software, networking, supply chain, developer ecosystem. Each layer on its own might be surmountable, but together they create a formidable barrier. Lucas: And don't forget the data advantage. Every time a developer trains a model on Nvidia hardware, they generate telemetry that Nvidia can use to optimize future chips. The more people use their platform, the better their next-generation architecture becomes. Luna: That's a flywheel. More users → more data → better hardware → more users. It's the same dynamic that made Intel dominant in the PC era, and Windows dominant in operating systems. Lucas: Exactly. But there's a risk. The hyperscalers are a concentrated customer base. Amazon, Google, Microsoft, and Meta together account for a huge portion of AI GPU spending. If they decide to collectively push their own silicon, Nvidia could lose a chunk of its business. Luna: Google already designs its own TPU for internal use and offers it to cloud customers. Amazon's Trainium is being adopted by AWS clients. Microsoft recently announced a custom chip called Maia. The question is whether these chips will ever be good enough to replace Nvidia across the board, or just supplement it in specific workloads. Lucas: Most analysts think it's the latter. Custom chips are optimized for inference — running already-trained models — but training still heavily favors Nvidia's general-purpose GPUs. And even for inference, the software stack matters. If your model was trained on CUDA, porting it to a custom chip requires recompilation and potentially retuning. Luna: Let's talk about the valuation. Nvidia trades at over 40 times forward earnings. That's expensive by any historical measure. The market is pricing in continued dominance for years. What could break the narrative? Lucas: The biggest near-term risk is a slowdown in AI infrastructure spending. If companies decide they've bought enough GPUs for now, Nvidia's revenue growth could decelerate sharply. But so far, the spending shows no sign of slowing. Every tech company is terrified of being left behind. Luna: There's also geopolitical risk. The US government has restricted exports of advanced AI chips to China. Nvidia has had to design lower-performance variants for that market. If tensions escalate further, they could lose access to a significant portion of global demand. Lucas: That's real. China used to be about 20 percent of Nvidia's data center revenue. Those restrictions have forced them to innovate around the rules, but it's a headwind. On the flip side, the restrictions also hurt Chinese competitors because they can't access leading-edge chipmaking equipment. Luna: So the moat is strong, but not unbreakable. What's the one thing you'd tell a listener to watch as a signal of whether Nvidia's edge is eroding? Lucas: Watch the developer conferences. If you start seeing major AI frameworks like PyTorch or TensorFlow officially supporting AMD's ROCm with first-class performance, that's a sign the ecosystem is cracking. Also watch for hyperscaler announcements of training runs on their own chips — not just inference, but training from scratch. Luna: And on the networking side, if anyone manages to build a credible alternative to Nvidia's InfiniBand-based cluster, that would reduce the lock-in. Lucas: Right. For now, Nvidia has built one of the most impressive competitive moats in tech history. But moats require constant maintenance. Jensen Huang knows that complacency is the enemy. Luna: It's a fascinating case study in platform strategy. And it's still unfolding. Lucas: Absolutely. Worth keeping an eye on.