Latest / The CEO Diary with Fexingo: Leadership Lessons, Executive Decisions, and Corner Office Stories / How Jensen Huang Built NVIDIA's AI Moat with CUDA
Transcript
- Lucas: In 2012, a team from the University of Toronto won the ImageNet competition by a landslide using deep learning trained on NVIDIA GPUs. That was the moment Jensen Huang's decade-long bet on CUDA went from a costly gamble to a billion-dollar moat. Luna: But CUDA launched in 2006, right? That's six years of building a platform with barely any real-world demand. Lucas: Exactly. At the time, NVIDIA was a graphics company. GPUs were for gamers and designers. Huang decided to make them programmable for general-purpose computing. He called it CUDA — Compute Unified Device Architecture. It let developers write code that ran directly on the GPU's parallel cores. Luna: That sounds like a massive R&D bet. How much did they spend? Lucas: By the late 2000s, NVIDIA was pouring hundreds of millions annually into CUDA — some estimates say over a billion dollars cumulatively before it saw real revenue. In 2008 and 2009, when the recession hit and GPU sales slumped, Huang kept the investment going. Shareholders were not happy. Luna: I remember reading that some analysts called it a vanity project. Why did Huang push through? Lucas: He saw that traditional CPUs were hitting physical limits on parallel processing. If you wanted to solve complex scientific problems — weather modeling, oil exploration, eventually AI — you needed massive parallelism. He believed GPUs were the answer, and CUDA was the key to unlock them. Luna: So it was a platform bet. Not just a product. Lucas: Right. Huang didn't just want to sell chips. He wanted to own the ecosystem. CUDA made NVIDIA the default hardware for any developer writing parallel code. And he made it free — CUDA was always free to download. That removed the friction for adoption. Luna: Free for developers, but NVIDIA made money on the hardware. Classic razor and blades model. Lucas: Exactly. But the razor took years to sell. By 2010, CUDA had maybe a hundred thousand developers. It was used in supercomputing and finance, but nothing game-changing. Then came Alex Krizhevsky and Ilya Sutskever training AlexNet on two NVIDIA GPUs. Luna: That win changed everything. Suddenly every AI researcher wanted NVIDIA hardware. Lucas: And because CUDA was already the standard, they didn't have to switch. NVIDIA had built the moat before the castle even existed. By 2015, deep learning was taking off, and NVIDIA's data-center revenue started climbing. Luna: It's a reminder that the things that seem obvious in hindsight often took real conviction at the time. Lucas: Absolutely. And honestly, if this episode made you think about that kind of conviction in your own work, that's the whole point. If it was worth a coffee to you, you know the drill — buy me a coffee dot com slash fexingo. Luna: It's the smallest ask, and it keeps this going. Lucas: So where did Huang find that conviction? He'd seen the potential of parallel computing firsthand. In the early 2000s, NVIDIA was already using GPUs for physics simulation in games. He realized that if you could generalize that, you could solve problems far beyond graphics. Luna: I've heard he's famously hands-on with the technical details. Did he personally drive CUDA's design? Lucas: Yes. Huang was a chip designer early in his career—he worked at LSI Logic and AMD. He understood the architecture intimately. He insisted that every new GPU from 2006 onward be cuda compatible, even if it added die cost. That's a long-term commitment baked into the silicon. Luna: Did any other chipmaker try to compete? AMD had something like Stream SDK, but it never took off. Lucas: AMD's attempt was less developer-friendly, and they didn't invest consistently. Intel had Larrabee, which was supposed to be a gpu like chip for general computing, but it never shipped as a product. Huang's relentless focus on the developer experience made CUDA sticky. Luna: So it wasn't just the technology — it was the ecosystem. Lucas: Exactly. NVIDIA published thousands of code samples, university courses, and a rich library of optimized algorithms called cuDNN. They made it trivial to get started. By the time Google and Facebook were building massive AI clusters, CUDA was the only serious option. Luna: Fast forward to 2026. NVIDIA's market cap is over three trillion dollars. A lot of that is built on the CUDA moat. Lucas: Right. Even as competitors like AMD try to catch up with ROCm, and as startups develop ai specific chips, CUDA has a network effect. Developers write code in CUDA, their models run on NVIDIA hardware, and the cycle continues. It's one of the strongest competitive advantages in tech today. Luna: What about the risk that the next big AI framework might not rely on CUDA? There's talk of open-source alternatives like MLIR and Triton. Lucas: That's a real question. NVIDIA is investing aggressively in software layers above CUDA, like TensorRT and the whole NeMo framework, to keep developers in their orbit. Huang said recently that NVIDIA is now a 'full-stack computing company' — hardware, system software, and application libraries. Luna: So the moat keeps getting deeper. Lucas: It does. But the lesson for leaders is that the moat has to be built before you need it. CUDA was a drag on earnings for years. Huang faced activist pressure, but he held the line. That kind of strategic patience is rare. Luna: What can a small business or a startup take from this? Lucas: You can't always wait for proof before you invest. You have to bet on a conviction about where the technology or market is going. The other piece is to make your platform free for early adopters. Lower the barrier to entry, and you'll own the ecosystem later. Luna: Even if it means years of negative returns. Lucas: Exactly. NVIDIA's story shows that if you're right about the trend and you outlast everyone else, the payoff isn't just big — it's generational. Luna: It also helps to have a CEO who's a technologist. Huang's deep technical credibility convinced his engineers to follow him on the CUDA journey. Lucas: That's a great point. He could have cashed out when GPU sales were booming. Instead, he reinvested into something that looked crazy at the time. Luna: And now it's hard to imagine AI without NVIDIA. Lucas: Hard to imagine, period. So next time you use a ChatGPT or run a Stable Diffusion model, remember: the foundation was laid in 2006, long before anyone was doing deep learning.