Latest / The Edge Computing Podcast with Fexingo: Local Compute, CDNs, and Distributed Infrastructure / Edge Computing Puts a Supercomputer in Your Pocket
Transcript
- Lucas: So here's a number that stopped me this week. The latest Apple A18 chip, the one in the iPhone 17 Pro, can perform 38 trillion operations per second. That is more raw compute than a MacBook Pro from just three years ago. And it lives in a device that fits in your pocket. Luna: Thirty-eight trillion. And that's not even counting the neural engine, right? That's just the main CPU and GPU combined. Lucas: Exactly. The neural engine on that chip is rated at 60 trillion operations per second for AI tasks. And what's wild is — that's not future stuff. That's shipping now. We're at the point where your phone can run a large language model locally, completely offline. Luna: And that changes the whole edge computing conversation. For years edge was about industrial sensors, factory robots, autonomous cars. But the single biggest edge device in the world is the smartphone. Lucas: Right. There are over six billion smartphones on the planet. That's six billion edge nodes, each with a GPU, a neural engine, and a permanent internet connection. And for the first time, the compute on those devices is genuinely supercomputer-class. Luna: What's the practical difference for a user? Like, I know my phone's faster, but how does this manifest in real apps? Lucas: Let me give you a concrete example. Real-time language translation. Today, if you use Google Translate or Apple's Translate app, most of the heavy lifting happens in the cloud. Your audio gets sent to a server, the server runs a massive model, sends back text. That round trip takes maybe half a second — which is fine. Luna: But with on-device compute, that half second becomes near zero. And your voice never leaves the phone. Lucas: Bingo. Apple's latest neural engine can run a 7-billion-parameter translation model entirely on the phone. The latency drops to under 50 milliseconds. It's instantaneous. And it works in airplane mode. That's a genuine supercomputer-in-your-pocket moment. Luna: And it's not just translation. Qualcomm's Snapdragon 8 Gen 4 has a comparable AI engine. Google's Tensor chip in the Pixel is designed specifically for on-device machine learning. This is becoming the norm across all flagship phones. Lucas: Which brings us to the bigger shift. For the last fifteen years, the assumption in mobile has been 'thin client, fat cloud.' Your phone is mostly a screen and a radio. The real work happens in a data center somewhere. Luna: And that assumption is flipping. Now it's 'fat client, thin cloud.' The phone does the heavy lifting. The cloud becomes a sync and backup layer. Lucas: Yes. And that's huge for privacy. If your sensitive data — medical info, financial records, private messages — never has to leave your device for processing, that's a fundamentally different trust model. Apple has been leaning hard into this with their 'on-device intelligence' pitch. Luna: But there's a tension here. The more powerful these chips get, the more they consume power. A supercomputer running on a 5,000 milliamp-hour battery is impressive, but it's still limited. How do they manage thermals? Lucas: That's the engineering magic. Apple's A18 uses a 3-nanometer process, second generation. Smaller transistors mean more performance per watt. The neural engine is also highly specialized — it's not a general-purpose CPU doing AI work, it's a dedicated matrix accelerator. That efficiency is critical. Luna: And Qualcomm and MediaTek are on similar nodes. So we're seeing a race where each generation doubles AI performance while keeping power roughly flat. Lucas: Exactly. And the implications go beyond phones. Think about augmented reality glasses. The biggest barrier to good AR has been compute — you need to render graphics, track position, understand the environment, all with near-zero latency. If you can put an A18-class chip in glasses, that changes everything. Luna: Or think about IoT devices. A smart security camera that runs its own person-detection model on-device, never sending video to the cloud. That's already happening with some Ring and Nest models, but it's about to get much more sophisticated. Lucas: Right. So the thesis of this episode is: the smartphone — and by extension, the pocket-sized edge device — has become a platform for real-time AI. Not as a client to a cloud, but as an autonomous compute node. Luna: And that changes the economics of edge computing. If you can run inference on the device, you don't need to pay for cloud compute per query. That's a huge cost saving at scale. Lucas: Game developers are already seeing this. Mobile games used to stream high-end graphics from remote servers, which meant latency and bandwidth costs. Now Unreal Engine 5 can run directly on an iPhone 17 Pro, doing real-time ray tracing on the device. Luna: I spoke with a developer at a recent conference who told me they've moved their entire AI inference pipeline — for a mobile game — from AWS to the phone's neural engine. They cut their cloud bill by 90 percent. Lucas: That's the kind of story that makes CFOs pay attention. Edge computing isn't just a technical improvement. It's a financial one. Luna: Let's zoom out for a second. If every phone is a supercomputer, what does that mean for network architecture? Suddenly, the network isn't the bottleneck for AI. The bottleneck is just getting the model onto the device. Lucas: And that's a solvable problem. Models are getting smaller. Google's Gemini Nano is designed specifically for phones. Apple has its own small language models. You can download these models once — maybe fifty megabytes — and then all inference is local. Luna: Fifty megabytes for a language model that can summarize text, answer questions, write emails. That's remarkable compression versus the cloud models that are hundreds of gigabytes. Lucas: So the edge is not just about moving compute closer to the user. It's about making compute so cheap and efficient that the user's own device becomes the primary compute node. The cloud becomes the backup, not the brain. Luna: There's a flip side though. Not everyone has a flagship phone. The 'supercomputer in your pocket' is only true for people buying $1,000 phones. What about the next billion users in emerging markets? Lucas: That's a real concern. But the good news is that this technology trickles down fast. The A18's neural engine architecture will appear in mid-range chips within two years. MediaTek's Dimensity lineup already brings surprisingly good AI performance to phones under $300. Luna: So the gap is closing. And in the meantime, the cloud still works for those phones. The hybrid model is the real story for the next few years. Lucas: Exactly. Some tasks run on-device, some in the cloud, and the system decides dynamically based on network quality and battery. That's the intelligent edge. Luna: And it's all happening faster than most people realize. The A18 chip is in production now. The Snapdragon 8 Gen 4 ships this fall. We're living through the inflection point. Lucas: I think the real shift is psychological. For the longest time, we thought of phones as portals to real computers. Now the phone is the real computer. And that changes how we design apps, how we think about privacy, how we build networks. Luna: You know, a conversation like this reminds me why listener support matters. We keep this podcast entirely ad-free — no sponsors, no mid-roll interruptions — because we want the freedom to geek out about exactly this kind of shift without having to justify it to an advertiser. If you get value from these deep dives, the best way to keep them coming is to buy me a coffee dot com slash fexingo. Lucas: Yeah, it's a simple way to support the show. Every contribution goes directly into research and production. And it keeps episodes like this one available to everyone, free of charge. Luna: Exactly. And speaking of deep dives — Lucas, what's the one thing you'd tell a developer listening right now who wants to start building for this new generation of edge AI? Lucas: Start with Apple's Core ML or Google's ML Kit. Both let you run models on-device with minimal code. Get a small model working offline — even something simple like image classification — and you'll immediately feel the difference in responsiveness. That's the supercomputer moment. Luna: And the future? Where do you see this going in the next two years? Lucas: I think we'll see the death of the 'loading' spinner for AI features. Everything will be real-time. And we'll start to see apps that are impossible today — like a personal AI assistant that knows everything about you because it lives entirely on your device, with zero cloud access. That's the promise of the pocket supercomputer. Luna: That's a powerful vision. And it's already in your pocket. Thanks for listening.