Latest / Minds, Bodies, and Terawatts / OpenAI and Broadcom Unveil Jalapeño, First Custom AI Inference Chip Designed in Nine Months

Transcript

OpenAI and Broadcom Unveil Jalapeño, First Custom AI Inference Chip Designed in Nine Months · Minds, Bodies, and Terawatts

• 0:00
  [Host] This is Minds, Bodies, and Terawatts, where silicon gets spicy. CNBC reported on June twenty-fourth that OpenAI and Broadcom unveiled Jalapeño, OpenAI's first custom-built AI chip, designed from scratch for inference in just nine months.
• 0:14
  [Guest] Nine months from blank page to working silicon, about a third of the usual timeline. Jalapeño is built for inference, the workload that fires every time someone uses ChatGPT, and OpenAI says it cuts those costs roughly fifty percent versus current GPUs.
• 0:31
  [Host] Nine hundred million people use ChatGPT every week, more than the entire population of Europe. Fifty percent cheaper at that scale is real money.
• 0:41
  [Guest] Billions in annual savings. OpenAI runs mostly on rented Nvidia GPUs through Microsoft Azure, and inference now dominates its compute bill. The company spends more running its models than training them, a shift the whole industry is living through.
• 0:57
  [Host] Broadcom's CEO Hock Tan told CNBC the demand from his six customers is, quote, simply insatiable. But how does Jalapeño actually stack up?
• 1:07
  [Guest] Early testing shows performance per watt above current accelerators, and engineering samples are running OpenAI's production models at target frequency in the lab. Broadcom already builds custom silicon for Google and other hyperscalers, so this isn't their first chip. The speed is the real story.
• 1:25
  [Host] Because OpenAI used its own AI to help design the chip.
• 1:30
  [Guest] Greg Brockman told CNBC the full cycle took nine months, with OpenAI's models accelerating the design. That's a feedback loop no chip company has ever had. Google has been building TPUs for over a decade and is on generation seven. OpenAI compressed that learning curve into three quarters.
• 1:49
  [Host] But Brockman also told CNBC that Jalapeño is not a replacement for Nvidia. If you still need the incumbent for training, how much independence does one inference chip buy?
• 2:01
  [Guest] Less than the headline implies. Jalapeño is an ASIC, purpose-built for one workload. Training still runs on Nvidia. But inference is where the volume lives, and owning your highest-cost workload changes the negotiating dynamic even if you never leave the supplier.
• 2:18
  [Host] When do they ship?
• 2:20
  [Guest] TSMC is fabricating on its three-nanometer process. Hock Tan told CNBC prototypes arrive late this year, full-scale ramp in the first half of 2028.
• 2:30
  [Host] Where does this leave Nvidia?
• 2:32
  [Guest] Still dominant. Jensen Huang announced at GTC a trillion dollars in orders for Blackwell and Vera Rubin through 2027. But every major AI company is now designing custom silicon, Google, Amazon, Meta, and now OpenAI. Nvidia keeps growing because total demand keeps growing, but its share of each customer's spend is shrinking.
• 2:53
  [Host] Does cheaper inference reach regular people, or does it just pad OpenAI's margins?
• 2:58
  [Guest] Margins first, then competition pushes prices down. Sam Altman has argued compute itself could become a public utility. If inference drops fifty percent per chip generation, the cost curve starts to resemble solar panels circa 2010.
• 3:13
  [Host] That only works if the companies designing those chips decide to share the savings.
• 3:19
  [Guest] And that is the tension. Patrick Deglon calls it the Star Wars path in Unscarcity, a small group controlling the means of intelligence production while everyone else rents access. Jalapeño makes OpenAI's inference cheaper, but the blueprint stays locked inside the building. For more on the infrastructure layer reshaping who controls AI, head to unscarcity dot ai. The book is on Amazon, the audiobook is on Spotify. This has been Minds, Bodies, and Terawatts.