Latest / EDGE AI POD / Beyond the Edge: Cloud and AI Convergence

Transcript

Beyond the Edge: Cloud and AI Convergence · EDGE AI POD

• 0:00
  Speaker 1: You know, keeping up with tech, it does feel like a full-time job sometimes, doesn't it?
• 0:04
  Speaker 1: Especially areas like edge computing, AI.
• 0:07
  Speaker 2: Oh, absolutely.
• 0:08
  Speaker 2: And when you put them together it gets complex fast.
• 0:11
  Speaker 1: Exactly.
• 0:11
  Speaker 1: It's like trying to predict the weather years out.
• 0:13
  Speaker 1: So that's what we're doing today.
• 0:16
  Speaker 1: Welcome to the Deep Dive.
• 0:17
  Speaker 2: And we got some really interesting material to unpack Some excerpts from Pete Bernard over at the Edge AI Foundation, titled Beyond the Edge.
• 0:25
  Speaker 1: Right, and their logo too, which sums it up nicely Connecting AI to the real world.
• 0:31
  Speaker 1: So our mission today really is to cut through that noise.
• 0:35
  Speaker 1: We'll use these insights to get a real handle on where Edge AI is today and, maybe, more importantly, where it's headed.
• 0:42
  Speaker 2: Yeah, looking at the hurdles, you know what slowed things down, but also the opportunities that are starting to well, starting to look really promising.
• 0:49
  Speaker 1: And Pete Bernard jumps right into one of the big challenges just how varied current IoT deployments are.
• 0:56
  Speaker 1: It's a real mix.
• 0:56
  Speaker 2: It is an incredibly heterogeneous set is how he puts it and that variety it has its upsides.
• 1:04
  Speaker 2: You can tailor things very specifically, but it also creates big headaches.
• 1:08
  Speaker 2: We're talking totally different hardware, different capabilities, different ways they connect, different security.
• 1:12
  Speaker 1: Different management orchestration.
• 1:14
  Speaker 2: Exactly, it's flexible, sure, but it makes scaling up or simplifying things really tough.
• 1:19
  Speaker 2: It's a bit of a double-edged sword.
• 1:21
  Speaker 1: It reminds me of that initial buzz around IoT.
• 1:23
  Speaker 1: You know all those predictions.
• 1:26
  Speaker 2: Yeah.
• 1:26
  Speaker 1: The classic hockey stick growth chart.
• 1:28
  Speaker 2: everyone showed oh yeah, I remember those.
• 1:30
  Speaker 2: The expectation was this huge, rapid explosion of connected devices.
• 1:34
  Speaker 1: And the reality.
• 1:35
  Speaker 2: The reality has been well a bit more measured, slower, as Bernard points out, some of the dependencies, the things we thought would just be there, didn't quite arrive on schedule.
• 1:46
  Speaker 1: Right, and maybe some assumptions were just a bit premature.
• 1:49
  Speaker 1: That hockey stick turned out to be more of a gentle slope.
• 1:53
  Speaker 2: Definitely more gradual.
• 1:54
  Speaker 1: So you have this really complex, diverse setup and it leads to this question I heard from a tech veteran recently which really struck me why can't we just pick a winner and move on?
• 2:04
  Speaker 1: You know, just standardize.
• 2:06
  Speaker 2: It's such an understandable sentiment, isn't it?
• 2:08
  Speaker 2: That desire for simplicity, a clear path, totally.
• 2:12
  Speaker 2: But Bernard argues pretty convincingly that trying to lock things down now, pick that winner for edge, ai it would actually backfire.
• 2:22
  Speaker 1: How so.
• 2:23
  Speaker 2: Because we're still so early in the game.
• 2:24
  Speaker 2: The tech is moving incredibly fast.
• 2:26
  Speaker 2: I mean, look around, new AI accelerator cores are popping up constantly.
• 2:31
  Speaker 2: We're seeing huge strides in lower power silicon, new ways for devices to learn continuously, without the cloud.
• 2:38
  Speaker 1: Right, optimizing memory, so you can even run things like generative AI on smaller devices.
• 2:43
  Speaker 2: Exactly, and new platforms for managing it all.
• 2:46
  Speaker 2: It's just constant innovation.
• 2:48
  Speaker 1: So trying to impose rigid multi-year standards, like maybe happened in the telco world?
• 2:53
  Speaker 2: Could really stifle that innovation, freeze the tech before we even know it's truly possible.
• 2:57
  Speaker 1: Okay, so what's the alternative then?
• 2:59
  Speaker 2: What Bernard suggests is focusing on best practices, encouraging communication, collaboration, getting working groups together to figure out how to connect these different pieces.
• 3:08
  Speaker 1: So foster growth that way, rather than forcing it into a box too early.
• 3:14
  Speaker 2: Precisely.
• 3:15
  Speaker 1: That's what will accelerate the market, and it's not just about industry collaboration.
• 3:17
  Speaker 2: Yeah.
• 3:18
  Speaker 1: The material also stresses looking further ahead, like fostering the next wave of ideas.
• 3:23
  Speaker 2: Yeah, supporting R&D is key both, both in companies and, crucially, in academia universities.
• 3:29
  Speaker 1: Building that bridge between them.
• 3:31
  Speaker 2: Absolutely vital.
• 3:32
  Speaker 2: That's where you nurture new talent, explore really novel ideas maybe the radical ones develop the next generation of AI leaders.
• 3:40
  Speaker 2: You need that constant flow of fresh thinking.
• 3:42
  Speaker 1: Okay, so the current picture is diverse, dynamic, maybe a little messy.
• 3:47
  Speaker 1: The material then talks about moving beyond the edge.
• 3:50
  Speaker 1: What does that actually mean?
• 3:52
  Speaker 1: What's the shift?
• 3:53
  Speaker 2: Well, the core idea is to stop thinking of the edge as just a bunch of separate, isolated devices and start seeing it as part of a more coherent system that stretches all the way from the cloud down to the tiniest sensor.
• 4:04
  Speaker 1: An integrated whole.
• 4:05
  Speaker 2: Exactly A system where all these different edge AI approaches tiny ML on little sensors.
• 4:10
  Speaker 2: You know neuromorphic computing inspired by the brain.
• 4:13
  Speaker 2: Even local generative AI can all work together.
• 4:15
  Speaker 1: Tailored to what you need, like cost or deployment or function.
• 4:19
  Speaker 2: Precisely Cost deployment, functionality, support, making it all work coherently.
• 4:26
  Speaker 1: And Bernard highlights three key pillars driving this shift beyond the edge.
• 4:30
  Speaker 1: Let's maybe tackle the first one AI itself seems fundamental.
• 4:35
  Speaker 2: It really is, and AI has changed profoundly.
• 4:37
  Speaker 2: It's not just a nice to have feature anymore.
• 4:39
  Speaker 1: No, it feels foundational now.
• 4:41
  Speaker 2: It is foundational for so many things and there's a much deeper appreciation now for data sets, for good quality data.
• 4:48
  Speaker 2: The models are getting incredibly sophisticated.
• 4:51
  Speaker 2: Tool chains well, there are still lots of them, but we're seeing signs of them maybe coalescing a bit, becoming more interoperable and the big one semiconductors can now routinely accelerate neural networks.
• 5:03
  Speaker 1: Even the complex ones, like transformers for generative AI, Yep even those right out at the edge.
• 5:07
  Speaker 2: That's a huge leap.
• 5:08
  Speaker 1: Wow, okay, that is significant.
• 5:10
  Speaker 1: So second pillar, semiconductor innovation, Sounds like things are speeding up on the hardware front too.
• 5:16
  Speaker 2: Oh, the pace is just remarkable Designing, taping out, fabricating new silicon.
• 5:21
  Speaker 2: It's getting faster and the focus is on performance, power and price.
• 5:27
  Speaker 1: Not just data center chips.
• 5:28
  Speaker 2: No, not at all.
• 5:29
  Speaker 2: There's this huge trend towards silicon specifically tailored for the edge.
• 5:33
  Speaker 2: Rugged chips for harsh environments think water plants or farms Makes sense, and ultra low power designs really optimize to run AI models efficiently, maybe even on battery power.
• 5:45
  Speaker 2: Things like chips designed for sparse computations, which saves a ton of energy for certain AI tasks.
• 5:50
  Speaker 1: So AI literally getting baked into everything everywhere, which brings us to the third pillar connectivity.
• 5:56
  Speaker 1: That often feels like the bottleneck, doesn't it?
• 5:58
  Speaker 2: It definitely has been complex, lots of competing tech spectrum issues, but we're seeing a clear shift now towards more fit-for-purpose options, meaning Meaning, not just one size fits all.
• 6:08
  Speaker 2: You've got power over Ethernet, low power, wide area, stuff like LoRaWAN for long range low data Right, and things like 5G REDCap, offering a balance.
• 6:17
  Speaker 2: The idea is to pick the right connection for the job optimize for power or cost or coverage.
• 6:23
  Speaker 1: Tailoring the connection, not just the device.
• 6:26
  Speaker 2: Exactly and interestingly, Bernard notes a shift in Telcos too More focus on enabling solutions for customers, not just maximizing ROI on their infrastructure build-out.
• 6:36
  Speaker 1: Okay, so we've got better AI, faster and more specialized chips and smarter connectivity, lots of capability building up at the edge.
• 6:44
  Speaker 1: But how do you tie it all together?
• 6:46
  Speaker 1: How do you fix that fragmentation problem we started with?
• 6:49
  Speaker 2: Ah, and this is where the cloud comes in, which might sound a bit weird at first.
• 6:53
  Speaker 2: Right Edge is about decentralizing.
• 6:55
  Speaker 1: Yeah, moving compute away from the cloud, closer to the data.
• 6:58
  Speaker 2: Right but Bernard makes this really compelling case that the cloud is actually the essential binding agent.
• 7:04
  Speaker 2: It's the key to resolving that tension between edge diversity and the need for things like portability and supportability.
• 7:11
  Speaker 1: How does that work?
• 7:12
  Speaker 1: How do you avoid just creating lots of isolated, hard-to-manage edge silos?
• 7:17
  Speaker 2: Well, think back to old school embedded systems before widespread connectivity.
• 7:21
  Speaker 1: Okay, standalone boxes doing one job for years.
• 7:24
  Speaker 2: Exactly Fixed function, rarely updated, maybe never functional silos.
• 7:28
  Speaker 1: Right.
• 7:29
  Speaker 1: Then we connected them and suddenly security management updates became huge issues.
• 7:35
  Speaker 2: Huge issues, whereas the cloud, even early on, was built differently.
• 7:39
  Speaker 2: Virtualization, workload portability were baked in.
• 7:42
  Speaker 1: Developers didn't need to know the specific hardware underneath.
• 7:45
  Speaker 2: Right Technologies like containers, Kubernetes originally for cloud apps, now being adapted for the edge.
• 7:50
  Speaker 2: Let you package and move things.
• 7:51
  Speaker 2: Resources became elastic, Orchestration became essential.
• 7:55
  Speaker 1: So the idea is to take those cloud strengths.
• 7:58
  Speaker 2: Virtualization, containerization.
• 7:59
  Speaker 1: Orchestration and apply them to the edge, even though it's often resource constrained.
• 8:04
  Speaker 2: That's the core insight Leverage those proven cloud capabilities to tackle the edge's persistent problems, Security management updates, and think about this fundamental connection which is.
• 8:15
  Speaker 2: Most data in the cloud actually starts at the edge Water systems, farms, hospitals, cities, cars the sheer volume of Edge data is exploding.
• 8:25
  Speaker 1: Right.
• 8:25
  Speaker 1: The Edge is the source of truth for so much cloud data.
• 8:28
  Speaker 2: Precisely so.
• 8:29
  Speaker 2: The thinking now is let's extend cloud DevOps and the EBLUPS practices all the way down to the sensors.
• 8:36
  Speaker 1: You mean manage edge devices like cloud resources.
• 8:39
  Speaker 2: Exactly.
• 8:39
  Speaker 2: We can't scale this industry by building custom management for every single edge device type.
• 8:44
  Speaker 2: Every sensor, camera, gateway, edge server needs to look like a manageable resource in a unified plane Cloud to edge.
• 8:52
  Speaker 1: And edge AI isn't just another workload in this picture.
• 8:55
  Speaker 2: No, it's central.
• 8:56
  Speaker 2: Bernard positions it as the key catalyst, bridging cloud and edge.
• 9:01
  Speaker 1: Okay.
• 9:01
  Speaker 2: Running AI right where the data happens, whether it's tiny ML on a microcontroller, neuromorphic stuff or traditional models on edge platforms.
• 9:10
  Speaker 2: That local processing is critical.
• 9:12
  Speaker 1: It complements cloud AI.
• 9:14
  Speaker 2: It complements cloud AI, agentic AI, cloud analysis.
• 9:16
  Speaker 2: Bernard uses the analogy Peanut butter and chocolate.
• 9:19
  Speaker 2: They make each other better.
• 9:21
  Speaker 2: I like that, and we're even seeing learning happen at the edge now Reinforcement learning, continuous learning devices, getting smarter locally.
• 9:28
  Speaker 1: So this cloud to edge integration, it's not just about easier management, it makes the whole system more capable.
• 9:33
  Speaker 2: Way more flexible, responsive, powerful.
• 9:36
  Speaker 2: Think about application portability, deploying AI models easily everywhere, managing training data, getting telemetry back.
• 9:42
  Speaker 2: Imagine dynamically shifting workloads between edge and cloud based on cost or performance network conditions, maybe with a human in the loop.
• 9:53
  Speaker 1: Okay, that's powerful.
• 9:54
  Speaker 2: And because it's built with this sort of hierarchical snap in approach for edge components, building and changing these distributed systems gets much simpler.
• 10:02
  Speaker 1: All right, this paints a really compelling vision, this unified cloud to edge AI future.
• 10:10
  Speaker 1: But, as the material notes, we're not quite there yet.
• 10:12
  Speaker 1: Looking towards 2025, beyond what needs to happen to make this real.
• 10:18
  Speaker 2: Yeah, it's still work in progress.
• 10:19
  Speaker 2: Bernard lays out a few crucial things.
• 10:21
  Speaker 2: First, he says we need to make edge AI the default topology.
• 10:25
  Speaker 1: Default meaning.
• 10:26
  Speaker 2: Meaning if data starts at the edge, process it there first.
• 10:29
  Speaker 2: Only send it to the cloud if you really need cloud scale capabilities, like aggregating data from tons of sites.
• 10:34
  Speaker 1: And for that to work.
• 10:35
  Speaker 2: Power, efficiency, performance costs.
• 10:37
  Speaker 2: They become absolutely critical for making it scalable.
• 10:39
  Speaker 1: Makes intuitive sense.
• 10:40
  Speaker 1: Avoid the latency, the bandwidth costs.
• 10:43
  Speaker 1: If you can get insights locally, what else?
• 10:45
  Speaker 2: Second, we need more of a unified vision across the industry.
• 10:49
  Speaker 2: Keep innovating, yes, but work towards a common model for deployment and support.
• 10:54
  Speaker 2: Leverage those cloud best practices.
• 10:57
  Speaker 1: So avoid reinventing the wheel constantly.
• 10:59
  Speaker 2: Pretty much.
• 10:59
  Speaker 2: Third, and this is huge commercial demand, Big companies, cities, policy groups need to ask for solutions built this way.
• 11:07
  Speaker 1: Push for interoperability yes.
• 11:09
  Speaker 2: And resist the temptation for short-sighted, one-off proprietary deployments that seemed cheap initially but cost a fortune later in operational headaches and replacements.
• 11:19
  Speaker 1: Need that strategic, long-term view from buyers.
• 11:22
  Speaker 2: Absolutely.
• 11:23
  Speaker 2: Fourth, edge tech providers really need to embrace cloud DevOps, make their diverse edge stuff show up and act like cloud resources.
• 11:31
  Speaker 1: Making it easy to manage.
• 11:32
  Speaker 2: Right, that means engaging more with things like LF, edige, akri, kubernetes, k3s, investing in bridging that gap between the constrained edge and the powerful cloud.
• 11:42
  Speaker 1: Make the edge feel like a natural cloud extension.
• 11:44
  Speaker 2: You got it.
• 11:44
  Speaker 2: And finally, serious investment in orchestration software.
• 11:47
  Speaker 2: We need that ecosystem spanning cloud to edge, including good MLOs for edge AI models.
• 11:53
  Speaker 1: So models can work with the cloud, but also run fine on their own if the network drops.
• 11:57
  Speaker 2: Exactly and support multi-cloud, multi-edge environments.
• 12:00
  Speaker 2: That's reality.
• 12:01
  Speaker 1: It really feels like we've made amazing progress on the pieces you know the chips, the AI models, the connectivity options incredible advances.
• 12:11
  Speaker 2: Undeniable.
• 12:11
  Speaker 1: But there are still these fundamental hurdles in making it all work together smoothly, scalably.
• 12:14
  Speaker 1: Incredible advances, Undeniable.
• 12:14
  Speaker 1: But there are still these fundamental hurdles in making it all work together smoothly, scalably, manageably, in a way that delivers real value.
• 12:20
  Speaker 2: That's the paradox.
• 12:21
  Speaker 2: Bernard really highlights All this amazing tech, but the full transformative value is still a bit locked up by these integration and management complexities.
• 12:29
  Speaker 2: Well, the takeaway is the takeaway is we have to learn from the past learn from embedded systems, from the IoT hype cycle, from cloud AI's evolution, understand the pitfalls, the successes and then work together collaboratively to unlock the real scale of edge AI.
• 12:45
  Speaker 1: So, wrapping up, then, we're seeing this big shift right from that fragmented early IoT world towards something much more integrated Edge AI tied tightly to cloud power and manageability.
• 12:54
  Speaker 2: Yeah, driven by those advances in AI chips connectivity.
• 12:58
  Speaker 1: With the cloud as that crucial glue, that orchestration layer.
• 13:01
  Speaker 1: But getting there requires that shared vision, real commercial demand for interoperable stuff and really embracing cloud native ways of doing things right down to the edge.
• 13:12
  Speaker 2: Exactly, it's evolving beyond just connecting things to making connected things truly smart, autonomous and manageable wherever they are.
• 13:20
  Speaker 1: So here's something to chew on as you digest all this Thinking about AI operating right where data is born.
• 13:27
  Speaker 1: What completely new things, what solutions become possible that just aren't feasible today because of latency or connectivity limits or just too much data to move.
• 13:37
  Speaker 2: Yeah, what could you do if intelligence was just embedded everywhere?
• 13:41
  Speaker 1: It's pretty exciting to think about, isn't it?
• 13:42
  Speaker 1: Hopefully this deep dive gives you a clear map of that road beyond the edge.
• 13:46
  Speaker 1: Thanks for joining us.