Latest / Ctrl AI Profit / Ep. 142 | OpenAI Just Built Its Own Chip
Transcript
- 0:20SPEAKER_00: Frank, something happened recently that most business owners completely missed, but it's going to affect every single one of them.
- 0:28SPEAKER_01: OpenAI and Broadcom announced a custom AI chip called Jalopeno.
- 0:33SPEAKER_01: And this isn't vaporware.
- 0:34SPEAKER_01: They have engineering samples already running real workloads.
- 0:38SPEAKER_01: OpenAI built their own chip.
- 0:41SPEAKER_01: Their first one.
- 0:42SPEAKER_01: And the reason matters as much as the chip itself.
- 0:45SPEAKER_00: For years, OpenAI has been renting its compute from NVIDIA.
- 0:49SPEAKER_00: That's like running your entire business on someone else's servers and paying whatever they charge.
- 0:54SPEAKER_01: And NVIDIA knows they have leverage.
- 0:57SPEAKER_01: When you're the only game in town for AI hardware, you can set your terms.
- 1:01SPEAKER_01: Jalapeno is OpenAI saying, not anymore.
- 1:06SPEAKER_00: How dependent was OpenAI on NVIDIA?
- 1:09SPEAKER_01: Extremely.
- 1:10SPEAKER_01: Every time ChatGPT answers a question that runs on a GPU cluster, primarily NVIDIA GPUs.
- 1:17SPEAKER_01: At the scale OpenAI operates, we're talking about billions of queries a day.
- 1:22SPEAKER_00: The hardware bill is staggering.
- 1:25SPEAKER_00: And they had no leverage in that negotiation.
- 1:30SPEAKER_01: None.
- 1:31SPEAKER_01: You take the price or you can't run your product.
- 1:34SPEAKER_01: That's not a great position for a company that's supposed to be building the infrastructure of the future.
- 1:39SPEAKER_00: So what actually is this chip?
- 1:42SPEAKER_00: Because I hear custom chip and I think great for engineers, but what does it mean for me?
- 1:48SPEAKER_01: Jalapeno is built specifically for inference, the technical term for running the AI model to generate an answer.
- 1:55SPEAKER_01: Not training new models.
- 1:57SPEAKER_01: The part that happens every single time a user asks Chat GPT something.
- 2:02SPEAKER_01: So the part we pay for.
- 2:04SPEAKER_01: Exactly.
- 2:05SPEAKER_01: OpenAI designed it from scratch for how large language models actually work, optimizing data movement, memory balance, and networking in a way a general-purpose GPU never was designed to do.
- 2:17SPEAKER_00: And they're claiming better performance per watt than current hardware.
- 2:22SPEAKER_01: Substantially better, according to their announcement.
- 2:25SPEAKER_01: They're still measuring the final numbers, but the engineering samples are running real workloads, including an unreleased version of their coding model.
- 2:34SPEAKER_00: Better efficiency means lower cost per query.
- 2:38SPEAKER_01: Which is what determines API pricing.
- 2:41SPEAKER_01: Cut OpenAI's cost per query by 30 or 40%.
- 2:45SPEAKER_01: Eventually, that shows up in what businesses pay.
- 2:48SPEAKER_00: Think about how many businesses are now running OpenAI under the hood.
- 2:53SPEAKER_00: Customer service bots, document processing, internal automation tools, those costs scale with usage.
- 3:01SPEAKER_00: This changes the math.
- 3:03SPEAKER_01: And the math has been moving in one direction for years.
- 3:06SPEAKER_01: Every six months, the same AI capability costs less.
- 3:10SPEAKER_01: Jalapeno is a structural move that accelerates that curve.
- 3:28SPEAKER_01: Apple switched to their own chips, costs down, performance up.
- 3:33SPEAKER_01: Amazon built their own data centers.
- 3:35SPEAKER_01: They undercut every competitor on cloud pricing.
- 3:38SPEAKER_01: This is the same strategic move.
- 3:40SPEAKER_00: It takes massive upfront investment, but the long-term payoff is control.
- 3:47SPEAKER_01: And independence, which in a competitive market is worth a lot.
- 3:59SPEAKER_01: Right now, if NVIDIA has a supply shortage or decides to prioritize a bigger customer, OpenAI feels that directly.
- 4:07SPEAKER_01: With Jalopeno, they decouple from that risk.
- 4:10SPEAKER_00: And when your AI provider is more stable, every tool built on top of it is more stable.
- 4:17SPEAKER_01: That's the dependency chain most people don't think about.
- 4:20SPEAKER_01: You might not use Chat GPT directly, but your CRM might.
- 4:24SPEAKER_01: Your scheduling tool, your document system.
- 4:27SPEAKER_01: OpenAI is often somewhere in that stack.
- 4:29SPEAKER_00: I've had business owners tell me they didn't think they used OpenAI.
- 4:34SPEAKER_00: And then I ask what tools they use.
- 4:37SPEAKER_00: And I can see OpenAI in at least three of them.
- 4:41SPEAKER_01: It's embedded everywhere, which means the stability of OpenAI's infrastructure is actually the stability of a huge portion of the business software world.
- 4:50SPEAKER_00: So this chip isn't just an OpenAI story.
- 4:54SPEAKER_00: It's the foundation of the whole AI economy getting more stable.
- 4:59SPEAKER_01: The nine-month development timeline is what really caught my attention.
- 5:02SPEAKER_01: Normal chip development cycles are years.
- 5:05SPEAKER_01: That tells you how urgently OpenAI felt the need to do this.
- 5:09SPEAKER_00: Urgency usually means pain.
- 5:11SPEAKER_00: What were they feeling?
- 5:13SPEAKER_01: The numbers are staggering.
- 5:15SPEAKER_01: OpenAI reportedly spends billions per month on compute.
- 5:19SPEAKER_01: At that scale, even a 20% reduction per inference is hundreds of millions of dollars a year.
- 5:27SPEAKER_00: So this chip is essentially a way to print money more efficiently.
- 5:32SPEAKER_01: That's exactly what it is, which is why they designed it in nine months instead of the normal multi-year chip development cycle.
- 5:40SPEAKER_00: Nine months from concept to engineering samples running workloads.
- 5:45SPEAKER_00: That's not a typical chip story.
- 5:47SPEAKER_01: It tells you everything about the urgency.
- 5:50SPEAKER_01: When you're spending that much on compute, every month you don't have a solution costs you.
- 5:54SPEAKER_00: Which brings up something I want to say directly to business owners.
- 5:58SPEAKER_00: If you've been hesitating on AI because the costs feel unpredictable, this is a signal the pricing curve is going down, not up.
- 6:06SPEAKER_01: The trend has been consistent for years.
- 6:08SPEAKER_01: Every six to twelve months, the same capability costs less.
- 6:12SPEAKER_01: Jalapeno is a structural move that accelerates that trend.
- 6:15SPEAKER_00: So what do you actually do with this information right now?
- 6:19SPEAKER_01: First, don't wait for the chip to ship before investing in AI.
- 6:22SPEAKER_01: The tools running on today's infrastructure are already powerful enough to change how you operate.
- 6:28SPEAKER_00: The pricing trend has been going in one direction for years.
- 6:32SPEAKER_01: And it's going to keep going that direction.
- 6:34SPEAKER_01: Jalopeno is a structural move that makes that faster.
- 6:38SPEAKER_00: Second, when you evaluate AI tools, favor the ones built on major platforms.
- 6:44SPEAKER_00: OpenAI, Anthropic, Google, those are the ones that benefit as infrastructure costs fall.
- 6:50SPEAKER_00: They get cheaper because their providers get cheaper.
- 6:53SPEAKER_00: That matters for your budget.
- 6:55SPEAKER_00: And third, and I say this from watching every technology wave since the 80s, the window where AI gives you a real competitive advantage is closing.
- 7:04SPEAKER_01: When costs drop low enough, everyone uses it.
- 7:07SPEAKER_01: The businesses winning right now are the ones who moved before it became mandatory.
- 7:12SPEAKER_00: A chip announcement sounds like a tech story.
- 7:15SPEAKER_01: But it's really a business strategy story.
- 7:18SPEAKER_01: The hardware is the foundation, the strategy is what you build on top of it.
- 7:23SPEAKER_00: And the businesses that are already building on top of AI infrastructure, they're the ones who are going to benefit most as the costs come down.
- 7:31SPEAKER_01: Lower costs mean more usage is economically viable.
- 7:35SPEAKER_01: Automations that didn't make sense at$10 per thousand queries start to make sense at$2.
- 7:41SPEAKER_00: And when those automations become viable, the businesses that already have the muscle memory to run them win.
- 7:49SPEAKER_01: The time to start building is before everyone else is forced to.
- 7:53SPEAKER_00: Thanks for listening to Control AI Profit.
- 7:55SPEAKER_00: I'm Michael Cadenhead.
- 7:57SPEAKER_00: And I'm Frank.
- 7:58SPEAKER_00: We'll see you tomorrow.