Latest / Hardware Startups with Fexingo: Devices, Robotics, and Manufacturing Tech Companies / Hardware Startups Are Betting on Edge Computing Chips
Transcript
- Lucas: There's this assumption that if you build a smart hardware device, you'll just offload the brain work to the cloud. But a growing number of hardware startups are betting that the real edge — pun intended — is doing the compute right on the device itself. Luna: I've seen those numbers — cloud costs can eat a huge chunk of margin, especially if you're doing real-time inference. But isn't the hardware for on-device AI still expensive? Lucas: It was. But a wave of startups has been designing specialized chips called neural decision processors that are both cheap and ultra-low-power. Take Syntiant — they've raised over a hundred million dollars to build chips that consume less than a milliwatt and can do keyword spotting, sensor fusion, even basic computer vision. Luna: Less than a milliwatt? That's basically negligible battery drain. So where are we seeing these chips land first? Lucas: Battery-powered devices that would otherwise need to stream audio or video to the cloud. Think smart earbuds, wearable health monitors, industrial sensors. The value proposition is straightforward: if your device can process a voice command locally, you save the latency, the bandwidth cost, and the privacy headache. For a hardware startup shipping tens of thousands of units, those cloud savings can be the difference between a positive margin and a losing one. Luna: Right, and that per-unit cloud cost can be a recurring expense that scales linearly. It's not like software where marginal cost is near zero. Lucas: Exactly. And that's why we're seeing startups choose a different silicon strategy than the default. Typically, a hardware founder picks a general-purpose microcontroller from the usual suspects — you know, a Cortex-M4 or something off the shelf. But those aren't optimized for neural networks. A neural decision processor, on the other hand, has dedicated hardware for matrix multiplications and convolutions. It can do an inference in a fraction of the energy. Luna: But there's a trade-off, right? A custom chip means higher upfront engineering cost and longer timelines. You're committing to an architecture before you've validated product-market fit. Lucas: That's the tension. If these conversations are useful for what you're building or running, we want to keep them free of ads. We deliberately don't run ads on these shows. If you want to support that choice, the link is buy me a coffee dot com slash fexingo. Luna: Yeah, it's that simple — listeners like you help us stay independent and focused on the actual business decisions. No sponsors to please. Lucas: So back to that tension. The smartest founders we've seen use a two-phase approach. Early prototypes use a beefier off-the-shelf microcontroller. Then, once they've nailed the product and have some traction, they spin up a custom ASIC. Syntiant actually sells a development kit that lets you test their chip in your prototype without designing a custom board first. Luna: So the startup gets the benefit of edge compute early, but can transition to a custom chip later when volumes justify it. That sounds like a reasonable middle ground. Lucas: And the numbers do justify it at relatively low volumes. Syntiant's chips cost around a dollar in moderate quantities. Compare that to the cloud cost per device per year for a voice assistant that's always listening — that could be several dollars in data transfer and inference. So within a year, the chip pays for itself. Luna: That's compelling. But I imagine the software toolchain is still a pain point. If you're a hardware startup with a small team, you don't want to spend months porting your model to a niche architecture. Lucas: That's the biggest barrier. The incumbents like Qualcomm have spent years building out their SDKs and ecosystem. Syntiant and other startups have been investing heavily in that, too. They provide TensorFlow Lite integration, model conversion tools, reference designs. But it's not plug and play yet. Founders need to budget for a few weeks of software engineering to get the model tuned. Luna: And there's also the question of performance. For more complex tasks — like real-time video analytics — you might need something more powerful than a milliwatt-class chip. So is edge AI only for simple tasks? Lucas: Not anymore. There's a spectrum. Syntiant's core is for always-on sensing — think 'Hey Siri' level complexity. But other startups like Hailo and Groq are building chips that can run full neural networks at tens of watts. They're targeting edge servers and industrial cameras. The whole market is splitting into two tiers: ultra-low-power for battery devices, and mid-power for edge nodes that have a power budget. Luna: And that's exactly where the incumbents are pushing back. Qualcomm, for instance, has its Cloud AI 100 series — but it's still more power-hungry. The startups have a genuine power advantage. Lucas: Because they're building from scratch for neural networks, while Qualcomm's chips often start from mobile SoC designs and add AI accelerators. The startup chips can be smaller, cheaper, and more efficient for specific workloads. That's the classic disruptor advantage. Luna: So what should a hardware startup founder take away from this? If you're building a connected device that does any kind of inference, is it time to start talking to these edge AI chip companies? Lucas: I'd say yes, but with a clear-eyed view of the trade-offs. Start by quantifying your cloud costs per device. If they're above a dollar per year, it's worth looking at a dedicated edge chip. Then evaluate the software effort. If your model is a standard CNN or RNN, the migration is probably straightforward. If you're using a bleeding-edge transformer architecture, you might need to wait for better tooling. Luna: And I'd add: consider your volumes. If you're shipping a few hundred units, the engineering NRE is hard to swallow. But if you're targeting tens of thousands, the unit cost savings start to matter. Lucas: Exactly. And that's the pattern we've seen in successful hardware startups: they don't jump to custom silicon too early, but they also don't ignore the long-term economics. The ones that scale are the ones that treat the chip decision as a strategic lever, not just a component choice. Luna: It's a fascinating shift. For years, the assumption was that the cloud would do all the heavy lifting. Now the pendulum is swinging back to the edge, at least for the always-on, low-power use cases. Lucas: And the startups that enabled that swing are the ones we're talking about today. Syntiant alone has shipped over 10 million chips. That tells you the demand is real. For a hardware founder, the smartest move might be to build a device that assumes zero cloud dependency from day one. Luna: Zero cloud dependency — that's a bold design constraint. But it forces you to think about power, latency, and privacy from the start. It might actually make for a better product. Lucas: It does. And it aligns with what consumers increasingly want: devices that work without an internet connection, that respect their privacy, and that don't require a monthly subscription for basic functionality. The hardware startups that deliver on that promise are the ones that will win. Luna: So if you're prototyping a new device right now, what's the first step to evaluate edge AI chips? Lucas: Look up the development kits from Syntiant, Hailo, or even Google's Coral. They cost a few hundred dollars. Run your model on them. Measure the inference time and power draw. Compare that to your cloud costs. If the numbers work, you have a path. Luna: And if they don't? You still learned something concrete about your product's requirements. Lucas: Exactly. That's the hardware startup way: test the critical assumption early, even if it means spending a weekend wiring up a dev board. Luna: Edge computing isn't just a buzzword anymore. It's a design philosophy that's reshaping how hardware startups think about product architecture. Lucas: And it's only accelerating. With each new generation of chips, the power efficiency improves and the tooling gets better. I expect in five years, virtually every connected device will do some form of on-device inference. The startups that adopt that philosophy early will have a real advantage. Luna: Alright, that's a great place to leave it for today. Thanks, Lucas. Lucas: Thanks, Luna. And for listeners, we'll be back next time with another deep dive into the decisions that make or break a hardware startup.