Latest / Neural Newscast / How OpenAI Built the Jalapeño Custom Inference Chip [Model Behavior]
Transcript
- [00:00] Announcer: From Neural Newscast, this is Model Behavior, AI-focused news and analysis on the models [00:05] Announcer: shaping our world. [00:12] Nina Park: I'm Nina Park. [00:13] Nina Park: Welcome to Model Behavior. [00:15] Nina Park: Today is June 24, 2026. [00:19] Nina Park: We are examining a significant shift in the AI infrastructure landscape as OpenAI transitions [00:25] Nina Park: from being primarily a software and research organization to a custom hardware designer. [00:32] Nina Park: On this program, Model Behavior looks at how complex AI systems are built, deployed, and operated within professional environments. [00:40] Nina Park: We focus on the engineering realities behind the headlines to understand the true impact on the industry. [00:47] Thatcher Collins: I'm Thatcher Collins. [00:48] Thatcher Collins: Nina, the major news this morning is the official unveiling of Jalapeno. [00:53] Thatcher Collins: This is the first homegrown silicon chip from OpenAI, and it is designed specifically for inference tasks. [01:00] Thatcher Collins: They are positioning this as an intelligence processor, a piece of hardware optimized to handle the heavy volume of day-to-day user queries, rather than the massive computational load required to train new models. [01:12] Thatcher Collins: According to reporting from the Next Web, OpenAI has already moved past the prototype stage and is currently running laboratory samples for a specialized model identified as GPT 5.3 Codex SPARC. [01:25] Nina Park: The broader hardware strategy appears to rely heavily on strategic collaboration. [01:30] Nina Park: To bring Jalapeno to market, OpenAI partnered with Broadcom for the essential connectivity and networking components, [01:36] Nina Park: and with Celestica for the production of the boards and server racks— [01:41] Nina Park: Thatcher, what stands out is that OpenAI is not attempting to build every component from scratch. [01:46] Nina Park: They focused on the core logic and architecture, but they are leaning on the Broadcom Tomahawk switching silicon to ensure the entire system can function at a massive data center scale. [01:56] Thatcher Collins: The timeline is what really catches my attention, Nina. [02:00] Thatcher Collins: They claim to have moved from the initial design phase to a final manufacturing tape out in only nine months, but... [02:08] Thatcher Collins: In the semiconductor world, that is an incredibly aggressive development cycle. [02:13] Thatcher Collins: They are attributing this accelerated pace to a self-reinforcing loop, [02:17] Thatcher Collins: where their own existing AI models assisted engineers in the design and verification process. [02:23] Thatcher Collins: If AI can genuinely help human engineers design better chips faster, [02:28] Thatcher Collins: it significantly lowers the capital barrier to entry for custom silicon across the entire technology sector. [02:34] Thatcher Collins: We should also consider the financial and strategic motives behind this move. [02:38] Nina Park: The hardware lead at OpenAI, Richard Ho, suggests this initiative provides them with full-stack control. [02:45] Nina Park: Essentially, they want to reduce their position as a captive customer of NVIDIA. [02:49] Nina Park: By owning the model architecture, the software layers, and now the physical chip underneath, [02:54] Nina Park: they can tune the entire stack for maximum energy efficiency. [02:58] Nina Park: Early internal testing suggests that the performance per watt is substantially higher than current state-of-the-art GPUs, [03:05] Nina Park: although a full technical report is still several months away. [03:08] Nina Park: It is important to maintain some skepticism, Nina. [03:11] Nina Park: Vendor-provided benchmarks at a product launch always deserve a measured look. [03:16] Nina Park: While Palapeno might be highly efficient for running models, the training of those models remains the domain of NVIDIA hardware. [03:24] Nina Park: OpenAI has been transparent about the fact that NVIDIA remains a critical partner for training [03:28] Nina Park: their largest, most complex systems. This is not a total divorce from the existing supply chain, [03:34] Nina Park: it is more like diversification at the edges where they can gain some much-needed leverage [03:38] Nina Park: at the negotiation table. That leverage could become massive if they successfully hit their [03:43] Nina Park: internal targets. OpenAI aims to have these custom chips powering 10GW of total compute capacity by [03:50] Nina Park: For some perspective, that is roughly equivalent to the total electrical output of 10 nuclear [03:57] Nina Park: Broadcom CEO Hawk Tan has been quite blunt about the necessity of this shift, stating that at such a massive scale, an organization simply should not rely on a third-party GPU provider if they can avoid it. [04:08] Nina Park: It's the same logical path that Google and Amazon took with their own silicon years ago. [04:13] Nina Park: Exactly. Broadcom is effectively serving as the kingmaker in this market. [04:19] Nina Park: They were the primary partner behind the TPUs at Google, and now they are performing a similar role for OpenAI. [04:25] Nina Park: We are also seeing OpenAI spread their risks by utilizing Cerebris chips for specific inference workloads. [04:32] Thatcher Collins: This suggests that while the training market is difficult to disrupt, [04:36] Thatcher Collins: the inference market is where the most intense competition is happening right now. [04:40] Thatcher Collins: We will have a better sense of how Jalapeno performs once it is deployed in Microsoft data centers later this year. [04:47] Nina Park: This represents only the first step in what will be a very long and capital-hungry process for the organization. [04:54] Nina Park: Thank you for listening to Model Behavior, a Neural Newscast editorial segment. [04:59] Nina Park: You can find more of our analysis at mb.neuralnewscast.com. [05:04] Nina Park: Neural Newscast is AI-assisted, human-reviewed. [05:07] Nina Park: View our AI transparency policy at neuralnewscast.com. [05:11] Announcer: This has been Model Behavior on Neural Newscast. [05:15] Announcer: Examining the systems behind the story.