Latest / EDGE AI POD / Why The Next Wave Of AI Lives At The Edge

Transcript

Why The Next Wave Of AI Lives At The Edge · EDGE AI POD

• 0:06
  SPEAKER_00: We got some great speakers this afternoon.
• 0:08
  SPEAKER_00: We're going to kick it off with Erie Trishansky from Siva.
• 0:14
  SPEAKER_00: And I think it's great to have Siva involved because if you remember from yesterday we were talking about sort of the layers and players in the ecosystem, right?
• 0:21
  SPEAKER_00: From silicon IP up to the cloud.
• 0:24
  SPEAKER_00: And so Steva kind of represents really the foundational stuff, the foundational goo that a lot of our stuff is being built on.
• 0:32
  SPEAKER_00: So really appreciate him coming and sharing his insights.
• 0:36
  SPEAKER_00: There you go.
• 0:40
  SPEAKER_01: Thank you very much, Pete.
• 0:41
  SPEAKER_01: Very nice news.
• 0:42
  SPEAKER_01: I really like them.
• 0:44
  SPEAKER_01: Thank you, everybody, and welcome.
• 0:47
  SPEAKER_01: As Pete mentioned, I'm from Siva.
• 0:49
  SPEAKER_01: What we do is provide foundational IP across Connect, Sense, and Infer, allowing devices to connect across multiple protocols, sense their surroundings, and later on inference.
• 0:59
  SPEAKER_01: And I'm going to talk a little bit more about my presentation today.
• 1:02
  SPEAKER_01: But what I want to talk today about today is not what's happening today, it's what's coming next.
• 1:06
  SPEAKER_01: And what do we need to prepare as an industry, especially as model makers, chip providers, and solution providers in order to really harness the promise of AI.
• 1:17
  SPEAKER_01: AI is going to be huge.
• 1:18
  SPEAKER_01: I don't know how many presentations you have where people spoke about how big AI market is going to be.
• 1:23
  SPEAKER_01: So there are numbers going from a billion to a trillion to several trillion, but it's going to be huge.
• 1:28
  SPEAKER_01: It's going to have a lot of impact.
• 1:29
  SPEAKER_01: I think all of us are already using it day to day.
• 1:31
  SPEAKER_01: There's a lot of impact all across.
• 1:39
  SPEAKER_01: But we're going to have another 40 million billion devices which are connected, can sense an intelligence around us in 2030.
• 1:47
  SPEAKER_01: Think about how much decisions of what those devices can do.
• 1:50
  SPEAKER_01: And we see the right of the rise of Agentic AI.
• 1:54
  SPEAKER_01: People ask me why is that connected?
• 1:56
  SPEAKER_01: Because Agentic AI allows you to make software autonomous.
• 1:59
  SPEAKER_01: It allows you to connect multiple models running in different locations in order to provide outcomes.
• 2:05
  SPEAKER_01: And when we look at where the industry is going to go, it's going to be hybrid.
• 2:08
  SPEAKER_01: It's going to be some on the edge, some on the cloud, but you have to be connected.
• 2:12
  SPEAKER_01: Leading to that, I think the intelligence is moving much closer to us today.
• 2:17
  SPEAKER_01: Still, most of the work, most of the inference work, and of course training is down in the cloud.
• 2:22
  SPEAKER_01: We have all of the mega models, the foundational models that require huge data sets, huge databases, and of course a lot of servers and GPUs, all of them are there and running in the cloud.
• 2:34
  SPEAKER_01: But we started seeing more and more inference percolating towards the edge.
• 2:39
  SPEAKER_01: And when I mean inference, it's going to run on my devices, can be a cell phone, can be PC.
• 2:44
  SPEAKER_01: We all see a copilot PC, but also on the edge on a 5G or on a satellite.
• 2:49
  SPEAKER_01: What we believe and what we see already, that all of those devices, as I mentioned before, 40 billion, that's not including the small sensors, are going to be able to do and run intelligence models.
• 2:59
  SPEAKER_01: They don't have to run all of the models, but they have to run the right models at the right place at the right time.
• 3:04
  SPEAKER_01: This is why we believe that the rest, the next wave of AI is going to be at the edge.
• 3:08
  SPEAKER_01: And there are a couple of reasons associated with that.
• 3:10
  SPEAKER_01: The first one, you want to have it faster.
• 3:13
  SPEAKER_01: Why do I need to send data back and forth to the cloud?
• 3:15
  SPEAKER_01: Why can I not run what I need on the device itself much faster and of course much cheaper?
• 3:22
  SPEAKER_01: Privacy, why do I need to move data and share it with others?
• 3:27
  SPEAKER_01: I just keep it on my device, secured and protected.
• 3:31
  SPEAKER_01: And cost, we are moving from an OPEX play, where if you are providing a solution, you need to send it, let's say, to AWS or to GCP or if you open it.
• 3:43
  SPEAKER_01: Run it on your own data server, it's gonna cost you the ingest, the processing number of tokens, and it's gonna be on your OPEX.
• 3:49
  SPEAKER_01: When you're moving more towards the devices, more devices than intelligent, you're actually gonna have the ability, you're moving it to CapEx, and that's something you don't pay, I think as a provider.
• 3:57
  SPEAKER_01: So this is a paid by somebody else, and you use the same capabilities to run the device.
• 4:02
  SPEAKER_01: And last but not least is power.
• 4:04
  SPEAKER_01: First of all, much lower power envelope, and also solving the power barrier we're gonna have, especially in the US.
• 4:10
  SPEAKER_01: I don't know how it's right now in Taiwan, but especially in the US, there is not enough power.
• 4:14
  SPEAKER_01: I just read an article today.
• 4:16
  SPEAKER_01: There is a huge uh NVIDIA data center sitting in Mountain View, but it's not ready to go, but there's no power.
• 4:24
  SPEAKER_01: So, how do you solve that?
• 4:25
  SPEAKER_01: Moving more towards the edge, small devices, you're already gonna turn them on and charge them, can solve some of the problems for some of the models.
• 4:33
  SPEAKER_01: And today we're just beginning our journey.
• 4:37
  SPEAKER_01: If we're today in, I call it this is the Gen AI era, starting in 2022 with OpenAI, gave us freedom and gave us the ability to create documents, videos, pictures, ask questions.
• 4:49
  SPEAKER_01: I think everybody is using it.
• 4:50
  SPEAKER_01: I'm using it on a day-to-day.
• 4:53
  SPEAKER_01: Tremendous impact on the way I work and the way that we actually do things in the company.
• 4:58
  SPEAKER_01: So that's the first one.
• 5:00
  SPEAKER_01: Moving more towards agentic.
• 5:02
  SPEAKER_01: So if the Gen AI today is running mainly on data servers, on data centers in the cloud, when you start looking at agentic AI, we start seeing movement around specific models and capabilities in specific devices.
• 5:15
  SPEAKER_01: So cell phones are much stronger.
• 5:18
  SPEAKER_01: PCs are much stronger as well.
• 5:20
  SPEAKER_01: My home edge gateway has capabilities.
• 5:22
  SPEAKER_01: And this is critical because when you're talking about Agentic AI, it's a combination of multiple models, multiple agents, call it executive agent, some of them running on my watch, some of them run on my cell phone, and some of them run on the cloud.
• 5:36
  SPEAKER_01: But the ability to connect between them, sending commands and understand what to do, that's the magic.
• 5:40
  SPEAKER_01: That's what's making software autonomous.
• 5:43
  SPEAKER_01: Lead us to where we I believe is coming next.
• 5:45
  SPEAKER_01: It's already there today.
• 5:46
  SPEAKER_01: That's the edge of physical AI.
• 5:48
  SPEAKER_01: Physical AI is everywhere around us.
• 5:51
  SPEAKER_01: From a tiny sensor to a satellite, all of those devices are gonna be embodied with AI.
• 5:56
  SPEAKER_01: They're gonna be able to connect and to sense their environment.
• 6:00
  SPEAKER_01: This is what we are building.
• 6:01
  SPEAKER_01: That's the extension of edge AI to some extent.
• 6:04
  SPEAKER_01: Don't ask me, ask Jensen.
• 6:06
  SPEAKER_01: He knows better from everybody else.
• 6:09
  SPEAKER_01: But even Jensen is saying, I think uh it's kind of funny to say that this is gonna be probably the largest industry of all.
• 6:15
  SPEAKER_01: And it's not just about robots and cars, it's about multiple other use cases.
• 6:20
  SPEAKER_01: It's AR or VR, driver monitoring, uh, advanced quality assurance.
• 6:26
  SPEAKER_01: All of them are gonna fall as the use cases under physical AI.
• 6:29
  SPEAKER_01: Those are things that are gonna happen actually locally.
• 6:32
  SPEAKER_01: In some cases, and I'll show you later, you're gonna run some areas or some models actually in the cloud because you you don't want to have your model, a 7 billion model running on a small on a small watch.
• 6:43
  SPEAKER_01: I think it's not gonna fit there.
• 6:45
  SPEAKER_01: But you can run a lot of, and I've seen a lot from EmbedUR and from other companies, a lot of models that are optimized and can start doing more and more.
• 6:52
  SPEAKER_01: Think about when you connect them to a chain and building a system, what can be done?
• 6:58
  SPEAKER_01: Eventually, user experience is gonna stay the same, but you're gonna be able more privacy is faster at a lower cost.
• 7:06
  SPEAKER_01: Leading us to billions of devices, trillions of decisions.
• 7:11
  SPEAKER_01: Each one of them, again, we'll make a couple of decisions a day, if not more than that.
• 7:15
  SPEAKER_01: But what those devices really need in order to deliver the promise of physical edge AI, they need to have the ability to connect, connect over multiple protocols, in order to transfer, become the backbone of moving data from one place to another or commands.
• 7:32
  SPEAKER_01: They need to sense their environment their environment, understand what's happening around you, but also provide the experiences.
• 7:39
  SPEAKER_01: And of course, only when they have understanding what's happening around them, they have the latest models, latest capabilities and instructions, then they can make inference.
• 7:46
  SPEAKER_01: They can actually make decisions and then can later all either execute them or move them to somewhere else, somewhere else in the different in the value chain.
• 7:54
  SPEAKER_01: Let's start with connectivity.
• 7:56
  SPEAKER_01: I think I think we all agree that we are working right now in a ubiquitous wireless wireless connectivity is ubiquitous.
• 8:04
  SPEAKER_01: It's everywhere.
• 8:05
  SPEAKER_01: If I cannot connect on my 5G or Wi-Fi, I'm just dead.
• 8:09
  SPEAKER_01: Then what are when actually it's so simple these days?
• 8:12
  SPEAKER_01: And more and more devices are gonna be connected.
• 8:14
  SPEAKER_01: I think by 2030, I think for McKinsey, all devices are gonna be out are gonna be connected somewhere or another because if not, they're gonna be useless.
• 8:22
  SPEAKER_01: But what's interesting, that's not one solution fits all.
• 8:26
  SPEAKER_01: Some of them will require Bluetooth, very, very low power just to stream music.
• 8:31
  SPEAKER_01: Some of them require Wi-Fi and high throughput in order to transfer data.
• 8:35
  SPEAKER_01: Some of them will have no network around them, so they will require 5G.
• 8:39
  SPEAKER_01: And this is what we at Siva, this is what we deliver.
• 8:41
  SPEAKER_01: We deliver those protocols, those standards, those capabilities, and IP, going from Zigbee, UWB, Fred, Matter, Bluetooth, all variations, Wi-Fi and 5G, enabling our customer to make those devices connect, which is critical.
• 8:59
  SPEAKER_01: It's not just about providing the modem, it's providing the entire stack, including the radio and the software.
• 9:04
  SPEAKER_01: More if I think about plug and play, small faster time to market.
• 9:08
  SPEAKER_01: After I connected and now I have the ability to connect between my various devices, I want to start understanding sensing.
• 9:16
  SPEAKER_01: What's happening around me?
• 9:18
  SPEAKER_01: Because if I don't know what's happening around me from audio perspective, from vision and temperature and other sensors, then it what what you know how what can I actually do?
• 9:29
  SPEAKER_01: Because the action is going to be triggered by sensing of a change of movement.
• 9:35
  SPEAKER_01: And what we believe is that at the end of the day, you need to have those capabilities.
• 9:39
  SPEAKER_01: So we are very focused on delivering DSPs, AI DSPs for audio and for vision, and of course a lot of software for sensor fusion and understanding movement, DWS headsets, etc.
• 9:50
  SPEAKER_01: But last but not least, only when you have those two is becoming inference.
• 9:54
  SPEAKER_01: This is where I'm actually gonna monetize.
• 9:56
  SPEAKER_01: This is where I'm actually gonna be able to create value for all my investment in training.
• 10:01
  SPEAKER_01: Think about it today.
• 10:02
  SPEAKER_01: Did we pay ChatGPT for the training module?
• 10:05
  SPEAKER_01: No.
• 10:05
  SPEAKER_01: We are paying them$20 a month and others for inference to give me response.
• 10:10
  SPEAKER_01: And this is where actually company will be able to monetize all of the investment.
• 10:15
  SPEAKER_01: Sorry.
• 10:17
  SPEAKER_01: Couple of interesting facts.
• 10:18
  SPEAKER_01: The first one is if you look at the data centers, the investment is gonna be much more towards inference capabilities compared to training, moving from 10, 90% to around you know 70% in inference.
• 10:31
  SPEAKER_01: And the number of tokens is gonna be going much higher.
• 10:33
  SPEAKER_01: I think I've seen 100k, I think it'll be actually more than like a thousand K tokens when we reach 2030.
• 10:39
  SPEAKER_01: And that will drive a lot of processing power or AI compute across the various locations across the edge to cloud.
• 10:49
  SPEAKER_01: But each device is gonna have a different requirement.
• 10:52
  SPEAKER_01: Some of them will need very, very low performance, low power to run on my watch, my earbuds, my sensor.
• 11:00
  SPEAKER_01: Some of them will need a little bit higher to run it on the helping with ADAS in my car.
• 11:05
  SPEAKER_01: And the last one, maybe I want to run it into cloud.
• 11:08
  SPEAKER_01: I need a you know 16,000, 32,000 Mac, something which is really, really fast, running at gigaops and not TerraOps.
• 11:15
  SPEAKER_01: And this is where we are focusing on building actually a solution that actually allows you to scale from a very small to a very large with the same software tools.
• 11:22
  SPEAKER_01: I think we have a lot of partners in the ecosystem here today enabling that.
• 11:27
  SPEAKER_01: And I think this is critical.
• 11:28
  SPEAKER_01: This is not just about the hardware and the module, it's about the software.
• 11:32
  SPEAKER_01: How do we build a mechanism that can help you and can help your customer to deploy those solutions altogether?
• 11:40
  SPEAKER_01: But let's look at Connect Sense and Infer in the real physical world and how it evolved and what's gonna start happening these days.
• 11:49
  SPEAKER_01: So, automotive, it started with sensing tire pressure, sensing your environment.
• 11:55
  SPEAKER_01: Later on, connectivity was added.
• 11:57
  SPEAKER_01: That was 2008, 2010, update and much more.
• 12:01
  SPEAKER_01: And now we're starting to see much more inference solutions.
• 12:04
  SPEAKER_01: Some of them coming with AI DSPs, some of them are actually coming with inference, say NPUs in the next couple of years.
• 12:11
  SPEAKER_01: Our earpods started with streaming music using Bluetooth.
• 12:17
  SPEAKER_01: That's where it's all started.
• 12:19
  SPEAKER_01: But what we have seen today now, they're sensing, they understand movement, they do headset tracking, they understand their environment, they can actually measure my heart rate as well.
• 12:28
  SPEAKER_01: And we're starting seeing ending more and more AI capabilities like headset, you know, uh my head movement, uh, ENC improvement, and voice improvement as well, all of them running by AI algorithm, all of them running on the AirPods itself.
• 12:44
  SPEAKER_01: Last but not least, this is becoming very big in the US right now.
• 12:47
  SPEAKER_01: I don't know how many how many have tried the glucose meters.
• 12:50
  SPEAKER_01: So you get it, you buy it for 40 bucks, you put it on your on your hand, and it tells you if what you're eating is good for you or not.
• 12:57
  SPEAKER_01: It's a very, it's a fun activity.
• 13:00
  SPEAKER_01: But what I found remarkable about it, that up to three years ago, I couldn't get it in the open market.
• 13:07
  SPEAKER_01: I had to pay hundreds of dollars, and the user experience was kind of eh.
• 13:13
  SPEAKER_01: Meaning I have to go take it off, connect it while you know on the wire to somewhere to take the data.
• 13:18
  SPEAKER_01: Now each one of them costs 40 bucks.
• 13:20
  SPEAKER_01: I have wire, I have Bluetooth inside, connect to my cell phone automatically, and they're starting adding in the future, they're all gonna have some AI capabilities.
• 13:28
  SPEAKER_01: So, what do we see?
• 13:28
  SPEAKER_01: We're just the beginning of the edge of AI.
• 13:30
  SPEAKER_01: Everybody are investing, and it's gonna come.
• 13:33
  SPEAKER_01: It's not that the question is coming, the question is when and how are we gonna be prepared towards that.
• 13:37
  SPEAKER_01: And each one of those is a different module, different capabilities.
• 13:41
  SPEAKER_01: It's not one module or one NPU or one engine built for all.
• 13:49
  SPEAKER_01: And this is what we call if when you're combining connect, sense, and infer, this is something we call AI fabric.
• 13:54
  SPEAKER_01: That's the foundation layer, allow you to have a transport layer that connects between various devices across multiple protocols, sensive, perseverance, presence, what's happening, and last but not least, inference, where you have to get the right model at the right time.
• 14:10
  SPEAKER_01: And you can start by having only NPUs in the data center that probably don't need wireless connectivity and they probably don't need sensing capabilities.
• 14:17
  SPEAKER_01: But as I extend it towards the edge, I see you need a lot of capabilities towards it.
• 14:22
  SPEAKER_01: Probably if I'm looking at the medical devices, I need Federal Matter, I need Bluetooth, maybe a small NPU, maybe some small DSP.
• 14:30
  SPEAKER_01: I'm looking at my headset.
• 14:32
  SPEAKER_01: Probably right now it's Bluetooth, moving towards Bluetooth HDT7, but later on adding NPU capabilities, some DSP for voice and others, and on and on.
• 14:43
  SPEAKER_01: Each one we're gonna have a different requirement, different set, but that's what we define as the AI fabric.
• 14:47
  SPEAKER_01: That's the foundational layer.
• 14:49
  SPEAKER_01: It's miss and match, mix and match them, but in order to create and bring the capabilities in order to harness the real value of AI across edge to cloud.
• 15:01
  SPEAKER_01: And let's look at that in a real life scenario.
• 15:04
  SPEAKER_01: I don't know how many of you have seen the new product from Apple, you know, the real-time translation.
• 15:10
  SPEAKER_01: I think it's actually pretty cool.
• 15:11
  SPEAKER_01: I've tried it.
• 15:12
  SPEAKER_01: Try to decipher how it's actually work.
• 15:15
  SPEAKER_01: So there is a you can use your AirPods, you can have some voice improvement, headset tracking, some other model using AI or some very low inference engine.
• 15:25
  SPEAKER_01: But when I start, and I I think in the future it don't have to touch, we can say, hey, start translation, like wake up word.
• 15:32
  SPEAKER_01: It's gonna start sending what I'm hearing as an input because it cannot run the full ASR model on AirPods.
• 15:38
  SPEAKER_01: Just too small, not enough capability, then we'll drain all the power.
• 15:42
  SPEAKER_01: And this is where you're starting to send a system.
• 15:44
  SPEAKER_01: This is where you connect it to your phone.
• 15:45
  SPEAKER_01: You're gonna run it on your phone, you have a lot more capabilities, can run much larger models, and it's gonna return you the information as a translation, probably over Bluetooth, high performance Bluetooth.
• 15:56
  SPEAKER_01: That's the first system that is created.
• 15:59
  SPEAKER_01: But what if I want to later on to real get real insight?
• 16:03
  SPEAKER_01: Let's assume it was a business meeting between me and my friend, both of us talking different languages, and I want to capture what happened.
• 16:09
  SPEAKER_01: I can definitely send it back to the cloud, the entire translation or transcription, and provide infights, insight, meeting summary, and I'm probably gonna do it with Wi-Fi or 5G, not Bluetooth.
• 16:22
  SPEAKER_01: All of that is what we define AI fabric.
• 16:24
  SPEAKER_01: It's the capability to allow you to connect the foundational capabilities in order to really harness the value of edge AI and physical AI.
• 16:33
  SPEAKER_01: And this is what we deliver as a company.
• 16:35
  SPEAKER_01: I'll give you like a one-minute spill about FIVA, whatever you need from IP perspective around connectivity.
• 16:41
  SPEAKER_01: If it's Bluetooth, Wi-Fi, 5G, we have 67% market share.
• 16:46
  SPEAKER_01: We are the de facto standard.
• 16:48
  SPEAKER_01: If you want to build something that connects, you want to add it, come to us, we're more than happy to help.
• 16:52
  SPEAKER_01: Sensing around software, sensor fusion, experiences, and of course AIDSPs, and inference, small to large, single portfolio, but providing with the same toolkit and software stack, which is critical in order to make it happen across multiple end markets.
• 17:10
  SPEAKER_01: And it's not just about hey, this is the future, you know, you need to start building.
• 17:15
  SPEAKER_01: Well, we're delivering that today.
• 17:17
  SPEAKER_01: So 30% of the overall Bluetooth devices in the world are running on Siva.
• 17:23
  SPEAKER_01: We ship around a billion new devices that are using Siva Bluetooth every year.
• 17:28
  SPEAKER_01: A lot of you are already using that, and we're gonna see a lot more.
• 17:32
  SPEAKER_01: 50% of the worldwide headsets are powered by Siva.
• 17:38
  SPEAKER_01: This is not just about our Bluetooth, it's including some inference model, DSPs, and software.
• 17:44
  SPEAKER_01: In automotive, two out of the largest free players in automotive are using our AIDSP for ADAS, studied ramping.
• 17:53
  SPEAKER_01: Five plus customers have signed a new automotive deal with us in the last 12 months.
• 17:59
  SPEAKER_01: We are running seven out of the top ten MCUs are using our technology all across, and we have a lot of customers that not only buy a single product from Siva, they buy one and then they expand.
• 18:10
  SPEAKER_01: And of course, of infrastructure 5G, this is our bread and butter, 30% of the 5G, I would say worldwide uh transport is running using Siva algorithms at the back end of it.
• 18:23
  SPEAKER_01: And we are able to do that and shipping 20 billion dollars because of our partnership with our customer.
• 18:29
  SPEAKER_01: We've been doing it for 20 years.
• 18:31
  SPEAKER_01: That's a testament to our scale, the trust, and the technology that we bring.
• 18:36
  SPEAKER_01: Think about 20 billion devices, two billion per year.
• 18:40
  SPEAKER_01: It's around 60 new device, 60 new devices powered by Siva every second.
• 18:47
  SPEAKER_01: For me, when I did the math first time, I say, hey, I don't believe that, but it's huge, and nobody believed that.
• 18:52
  SPEAKER_01: So we are the power that powers everything on the back end.
• 18:56
  SPEAKER_01: But last but not last but not least, I'm here with uh with HAI, and I think thank you very much for inviting us.
• 19:02
  SPEAKER_01: It's a very strong partnership.
• 19:04
  SPEAKER_01: I think we are very much focused on industry collaboration.
• 19:07
  SPEAKER_01: I think when new markets coming up, you need to have a common language, common place to everybody to share.
• 19:13
  SPEAKER_01: We don't always have to agree, but it's a very good place for us to discuss and help build this industry together.
• 19:20
  SPEAKER_01: We're very much involved in working groups.
• 19:22
  SPEAKER_01: I think Audio is a new working group we are running, and if everybody wants to join, we more than welcome you.
• 19:28
  SPEAKER_01: And of course, the promotion.
• 19:30
  SPEAKER_01: This is promotion using our branding and actually building a partnership because we believe you have to get not multiple organizations, one organization to help to promote it overall.
• 19:40
  SPEAKER_01: With that, I will finish.
• 19:42
  SPEAKER_01: I think I'm actually on time, 25 minutes.
• 19:46
  SPEAKER_01: Would like any one of you, we're actually having uh an event in Hinchu on Friday.
• 19:51
  SPEAKER_01: If you haven't registered, you want to join, please come over.
• 19:54
  SPEAKER_01: I think you have the link and the information around it.
• 19:57
  SPEAKER_01: And it's not about what we're doing today because today we're already power, all of us are powering the smart edge.
• 20:02
  SPEAKER_01: It's about how we work to enable what's next, which is physical AI.
• 20:06
  SPEAKER_01: With that, thank you very much.