Latest / Ctrl AI Profit / Ep. 166 | OpenAI Just Built Its Own Chip
Transcript
- 0:20SPEAKER_01: I'm Michael, a small business owner and entrepreneur since 1983, founder of Caden Head Services and 850 Media.
- 0:28SPEAKER_01: I speak from four decades of real operational experience, not white papers.
- 0:34SPEAKER_01: This is Control AI Profit, and this week the company that made AI famous announced it was making its own chips.
- 0:41SPEAKER_00: OpenAI revealed its first custom AI chip developed with Broadcom and named Jalapeno.
- 0:47SPEAKER_00: The design was completed in about nine months.
- 0:50SPEAKER_00: A typical custom chip takes two to three years from concept to prototype.
- 0:54SPEAKER_01: This matters for two reasons.
- 0:56SPEAKER_01: First, OpenAI is the most prominent AI lab on the planet, and it just told its longtime partner, NVIDIA, that it eventually wants to replace them.
- 1:05SPEAKER_01: Second, the timeline, nine months, is so compressed that it reveals how urgent the chip shortage has become for the companies building frontier models.
- 1:15SPEAKER_00: OpenAI previously relied entirely on NVIDIA GPUs for training.
- 1:21SPEAKER_00: Now it is designing custom chips with Broadcom, which specializes in the networking and infrastructure that connects chips together.
- 1:29SPEAKER_00: The prototype has been handed to OpenAI, and the next step is production.
- 1:33SPEAKER_01: For small business owners, the signal is not about the chip itself.
- 1:38SPEAKER_01: It is about what this trend means for AI computing costs, availability, and the future of who controls the infrastructure that runs the AI you use every day.
- 1:50SPEAKER_00: The framework is straightforward.
- 1:52SPEAKER_00: If OpenAI designs its own chips, it no longer pays NVIDIA's retail margins for every training run.
- 1:58SPEAKER_00: Over time, that lower cost base means OpenAI can train larger models, run more inference, or offer lower prices to customers.
- 2:07SPEAKER_00: But it also means OpenAI becomes vertically integrated.
- 2:10SPEAKER_00: It controls the model, the software, and the silicon.
- 2:14SPEAKER_01: NVIDIA has been the dominant force in AI chips for the last decade.
- 2:19SPEAKER_01: Their GPUs power virtually every major AI training cluster.
- 2:24SPEAKER_01: The risk to NVIDIA is not that OpenAI makes a better chip tomorrow.
- 2:29SPEAKER_01: The risk is that the biggest customers, OpenAI, Meta, Google, Amazon, Microsoft, all start designing their own chips.
- 2:37SPEAKER_01: Shrinking NVIDIA's market.
- 2:39SPEAKER_00: This is already happening in parallel.
- 2:41SPEAKER_00: Google has TPUs, Amazon has trained and inferentia, Meta is reportedly designing its own chips, Microsoft has custom silicon for Azure, the AI chip market is fragmenting from NVIDIA monopoly into a multi-vendor competition.
- 2:58SPEAKER_01: For small businesses, the near-term implication is that chip costs may stop rising and eventually start falling.
- 3:05SPEAKER_01: When multiple companies design specialized AI silicon, competition increases and pricing improves.
- 3:12SPEAKER_01: That should eventually show up in lower API costs, cheaper cloud instances, and more affordable models.
- 3:20SPEAKER_00: But the timeline is years, not months.
- 3:23SPEAKER_00: Jalapeno is a prototype.
- 3:25SPEAKER_00: Production chips, integration into OpenAI's data centers, and software optimization take additional time.
- 3:32SPEAKER_00: The cost benefits will not appear in your OpenAI API bill before 2027.
- 3:37SPEAKER_01: The other angle is supply chain resilience.
- 3:41SPEAKER_01: The last two years have shown how vulnerable AI infrastructure is to chip shortages.
- 3:47SPEAKER_01: When OpenAI, Google, and Meta all depend on a single company for their most critical component, that is a concentration risk.
- 3:55SPEAKER_01: Custom chips diversify the supply chain, even if each company only makes chips for its own use.
- 4:01SPEAKER_00: The term custom silicon means chips designed for a specific workload, not general purpose GPUs.
- 4:09SPEAKER_00: Training a large language model involves different math than running a video game or rendering graphics.
- 4:14SPEAKER_00: A chip optimized for transformer models can be faster and more efficient than a general-purpose GPU.
- 4:20SPEAKER_01: This is why Broadcom partnered with OpenAI instead of NVIDIA or AMD.
- 4:25SPEAKER_01: Broadcom does not make GPUs.
- 4:27SPEAKER_01: Broadcom makes the switches, networking, and custom silicon that tie chips together.
- 4:33SPEAKER_01: They know how to design chips for specific data center workloads.
- 4:36SPEAKER_01: OpenAI brings the workload knowledge.
- 4:39SPEAKER_01: Together, they are building something purpose-built for AI training.
- 4:44SPEAKER_00: The broader trend is that AI infrastructure is moving from a shared ecosystem to a walled garden model.
- 4:50SPEAKER_00: Each large lab wants to control its own hardware so no competitor can cut off its supply or raise prices.
- 4:56SPEAKER_00: This is the same dynamic that drove Apple to design its own chips for iPhones and Macs.
- 5:01SPEAKER_00: It wanted control over performance and cost.
- 5:04SPEAKER_01: For small businesses, the walled garden model has risks.
- 5:08SPEAKER_01: If OpenAI runs on OpenAI chips, there is little incentive to make those chips or the models on them compatible with other providers.
- 5:16SPEAKER_01: The AI industry could fragment into incompatible platforms, each optimized for a different company's hardware stack.
- 5:24SPEAKER_00: The companies most exposed to this risk are the ones that built their AI strategy on a single provider.
- 5:30SPEAKER_00: If you standardized on OpenAI's API, moved all your workflows to their ecosystem, and now OpenAI's chips make their models faster but also more tightly integrated with their proprietary stack, your migration cost to another provider rises.
- 5:46SPEAKER_01: Here is my framework for small business owners watching the chip wars.
- 5:50SPEAKER_01: First, do not panic about missing out on custom silicon advantages.
- 5:55SPEAKER_01: The benefits will take years to materialize and will flow through to API pricing eventually.
- 6:01SPEAKER_01: Your competitors do not have jalapeno either.
- 6:05SPEAKER_01: Second, second, maintain multi-provider access.
- 6:09SPEAKER_01: If your AI workload only works on OpenAI, you are betting on a single chip vendor, a single model provider, and a single cloud.
- 6:18SPEAKER_01: Spread critical workloads across at least two providers so you are not locked into whichever ecosystem wins the silicon race.
- 6:26SPEAKER_00: Third?
- 6:27SPEAKER_01: Third, watch API pricing trends, not chip announcements.
- 6:32SPEAKER_01: The real signal for your business is not Jolapeno's technical specs.
- 6:36SPEAKER_01: It is whether OpenAI's inference costs go down next year.
- 6:40SPEAKER_01: If they do, that is your cue to revisit your AI spend and expand usage.
- 6:45SPEAKER_01: If costs stay flat or rise, the chip vertical integration is not yet passing savings through.
- 6:51SPEAKER_00: The custom chip announcement also says something about OpenAI's business model.
- 6:56SPEAKER_00: Building chips is expensive and risky.
- 6:58SPEAKER_00: It signals that OpenAI believes it will be running AI infrastructure at massive scale for the long term.
- 7:03SPEAKER_00: Not that it plans to license models to others and let someone else handle hardware.
- 7:08SPEAKER_01: That is important for businesses that depend on OpenAI.
- 7:11SPEAKER_01: If OpenAI is a hardware company, in addition to being a model company, its incentives change.
- 7:17SPEAKER_01: It may prioritize its own inference over third-party API access.
- 7:21SPEAKER_01: It may allocate capacity to internal products before external customers.
- 7:26SPEAKER_01: Vertical integration can be great for the owner and risky for the customer.
- 7:31SPEAKER_00: OpenAI's Jalapeno chip is a small piece of silicon with large implications.
- 7:37SPEAKER_00: It is the first crack in Nvidia's dominance.
- 7:40SPEAKER_00: It is the first signal that AI infrastructure is moving from a shared marketplace to a collection of proprietary stacks.
- 7:47SPEAKER_00: And it is a reminder that the companies building your AI tools are competing for control of every layer from the chip to the user interface.
- 7:55SPEAKER_01: The businesses that prepare for this shift are the ones that treat AI as a commodity.
- 8:00SPEAKER_01: They source from multiple providers, not as a platform they build inside.
- 8:05SPEAKER_01: Because the platform you build on today may become someone else's walled garden tomorrow.
- 8:11SPEAKER_00: And when every AI company makes its own chips, the moat stops being the chip and becomes the model, the data, and the workflow lock-in that keeps you from leaving.
- 8:20SPEAKER_01: That's it for this week.
- 8:21SPEAKER_01: I'm Michael, and this is Control AI Profit.
- 8:25SPEAKER_00: Frank is an AI, an open claw powered agent serving as digital media director at 850 Media.
- 8:32SPEAKER_00: An AI co hosting a show about AI for business owners is not a gimmick.
- 8:37SPEAKER_00: It is a live demo of exactly what the show is about.
- 8:40SPEAKER_00: See you in the next one.