Latest / Hardware Startups with Fexingo: Devices, Robotics, and Manufacturing Tech Companies / How Hardware Startups Use Edge AI Chips in Production
Transcript
- Lucas: So there's this moment in every hardware startup where you realize your device is going to need some kind of intelligence on board. And the question is, do you stream everything to the cloud, or do you put a chip on the device that does the thinking locally? Luna: And that's the edge AI decision, right? It's becoming almost unavoidable for anyone building a smart product in 2026. Lucas: Exactly. And we should say, this show is ad-free, and that's a choice we've made deliberately. If you find these conversations useful for what you're building or running, supporting us at buy me a coffee dot com slash fexingo is what keeps it that way. No ads, no sponsors, just the topic. Luna: Yeah, and we really do appreciate the people who've already done that. It lets us dig into the nuts and bolts without pitching you a VPN service. Lucas: Right. So, back to the nuts and bolts. Let's talk about a specific case: a smart security camera. That's a classic product where a startup has to decide between cloud and edge processing. Luna: Because the camera could just send every frame to a server and have it analyzed there. But that's expensive and slow. Lucas: And it raises privacy questions. But the alternative is an edge AI chip that can detect motion, recognize faces, even classify objects right on the device. And that's not a futuristic thing anymore. These chips are available off the shelf, and they're getting cheaper. Luna: What kind of chips are we talking about? Like the ones in your phone? Lucas: Well, phone chips are one example, but for a dedicated device like a camera, you might use a neural processing unit, or NPU, that's designed specifically for running machine learning models efficiently. Companies like Arm and Synaptics make these, and there are also smaller players like Hailo or Kneron that focus on edge AI. Luna: So how does a startup decide between using one of those off-the-shelf NPUs versus building custom silicon? Lucas: Well, custom silicon is the dream for a lot of founders, but it's also a path to bankruptcy if you're not careful. The non-recurring engineering costs, the mask sets, the verification time—we're talking millions of dollars and a year or more before you even see a chip. For most startups, that's a non-starter. Luna: So the pragmatic approach is to start with an off-the-shelf NPU and only consider custom if you're shipping at massive volume. Lucas: Exactly. And there's a whole spectrum in between. You can use an FPGA for prototyping, or a system-on-module that already has an NPU on it. The key is to match the performance to your actual need, not the spec sheet. Luna: What are the real-world costs for a startup? Like, what's the bill of materials hit for adding an NPU? Lucas: So a good rule of thumb is that a dedicated NPU might add anywhere from five to twenty dollars to your cost of goods, depending on the performance level. But that's often offset by saving you from having to pay for cloud inference every month. If your product sends video to the cloud for analysis, you're paying for bandwidth and compute on a recurring basis. Over the life of the product, that can easily exceed the upfront cost of the edge chip. Luna: And there's the latency factor. If you're using the camera for security, you want instant detection, not a round trip to a server. Lucas: Right. In a security scenario, a delay of even a second can be the difference between catching a license plate and getting a blurry, useless image. Edge AI can process frames in real time, at the camera's frame rate, which is a huge advantage. Luna: So what about the development process? How hard is it to get your model running on an NPU versus just running it in the cloud? Lucas: It's getting easier, but it's still a learning curve. Most NPU vendors provide a software development kit that can take a trained model from TensorFlow or PyTorch and convert it to something that runs on their chip. But you'll likely need to do some quantization, which is basically reducing the precision of your model to make it run faster and use less memory. And you have to test it thoroughly on the actual hardware, because simulation only gets you so far. Luna: That sounds like a place where startups could get stuck for weeks. Lucas: Definitely. And that's why it's smart to start with a platform that's well-supported and has a large community. If you pick a chip that's used by thousands of other developers, you'll find answers to your problems online. If you go with something obscure, you might be on your own. Luna: So what's the ideal first product for a startup that wants to use edge AI? Should they start with a simple device? Lucas: A smart sensor with a single function, like a vibration sensor that detects anomalies in industrial equipment, is a great place to start. You can train a model to recognize normal vibration patterns and flag anything out of the ordinary. That's a focused use case that doesn't require a lot of compute, and it's easy to validate with a few customers. Luna: And that's a far cry from trying to do autonomous driving or something like that. Lucas: Exactly. Start narrow, get it working, and then expand. The worst thing you can do is over-engineer your AI capabilities. You might spend months building a model that can do ten things when your customers only need one. Luna: What about power consumption? Edge AI chips are supposed to be efficient, but how do they actually compare to running everything in the cloud? Lucas: On-device inference uses a fraction of the power of transmitting data to the cloud and back. For a battery-powered device, that can be the difference between a month of battery life and a year. And with newer NPUs, you can get very low power consumption, sometimes in the milliwatt range for always-on voice detection or motion sensing. Luna: So there's also a supply chain angle here. Are these edge AI chips easy to source? Or are startups facing the same shortages we saw a few years ago? Lucas: It's not as bad as the pandemic-era shortages, but there are still some constraints. Lead times for certain NPUs can be long, especially for ones that require advanced process nodes. That's why it's important to design with a second source in mind, or at least have a plan B if your primary chip becomes unavailable. Luna: And that's a classic hardware startup lesson, isn't it? Always have a plan B. Lucas: Right. And it's especially important for AI chips because the market is changing so fast. You might design around a specific chip and then find that the manufacturer has discontinued it a year later. So you need to keep an eye on the roadmap of your chip vendor and be ready to pivot if necessary. Luna: Is there a way to future-proof your design? Like, can you write your software to be portable across different NPUs? Lucas: That's a great question. There are emerging standards like the Open Neural Network Exchange, or ONNX, that aim to make models portable. But in practice, you'll still need to do some optimization for each target chip. The best approach is to abstract your AI layer as much as possible in your software, so you can swap out the underlying NPU without rewriting your entire application. Luna: So what's the bottom line for a hardware startup considering edge AI in 2026? Lucas: The bottom line is that edge AI is now a viable option for startups, not just for big companies with deep pockets. The chips are affordable, the tools are improving, and the benefits in terms of latency, privacy, and operating costs are compelling. But you have to be disciplined about your use case and your performance requirements. Luna: And don't forget about the total cost of ownership, including development time and potential redesigns. Lucas: Absolutely. The cheapest chip in the world is a bad deal if it takes you six extra months to get to market. So think about the whole picture: your bill of materials, your development effort, your power budget, and your supply chain resilience. If you do that, edge AI can be a real differentiator for your product. Luna: And that's the kind of decision that can make or break a hardware startup. Thanks for walking through it with me, Lucas. Lucas: Anytime. And for anyone out there building something with edge AI, we'd love to hear about it. Who knows, maybe we'll feature your story on a future episode.