Latest / EDGE AI POD / Bridging the Research-Reality Divide in Edge AI

Transcript

Bridging the Research-Reality Divide in Edge AI · EDGE AI POD

• 0:06
  Speaker 2: Hi, I'm Pete Bernard.
• 0:07
  Speaker 2: We're going to talk about a topic today.
• 0:09
  Speaker 2: I'll have everyone introduce themselves here.
• 0:12
  Speaker 2: You can see their little blurbs.
• 0:14
  Speaker 2: But this panel is really about one of my favorite topics, which is the gap between research and reality, and we do a lot of talks here about research and cool things that are happening and sparsity and blah, blah, blah.
• 0:29
  Speaker 2: It's awesome.
• 0:30
  Speaker 2: But then the issue is okay, how do I ship something, how do I support something, how do I scale something?
• 0:36
  Speaker 2: And that's where there's another type of magic out there, another type of science that we also need to focus on.
• 0:42
  Speaker 2: I think to Carrie's point at the end of the day, the KPI is cash, and so getting our incredible inventions out there and deployed at scale is really critical.
• 0:54
  Speaker 2: So we wanted to spend some time with this esteemed panel to talk about what that gap looks like and what are some of the things that could mitigate that gap.
• 1:02
  Speaker 2: So maybe we can start with some introductions and kind of go down the line there.
• 1:08
  Speaker 3: Hello, good afternoon.
• 1:09
  Speaker 3: My name is Rajesh, CEO and founder of EmbedUR.
• 1:13
  Speaker 3: We've been in the embedded space for the last 20 years and the past few years have been super exciting and looking forward to the discussion.
• 1:24
  Speaker 4: Thanks, rajesh.
• 1:25
  Speaker 4: Hi everyone, my name is Avijit Sinha.
• 1:27
  Speaker 4: I'm the Senior Vice President for Strategy and Global Business Development for Wind River.
• 1:31
  Speaker 4: Wind River is a leader in embedded system software development.
• 1:36
  Speaker 4: The company has been around 40 years.
• 1:38
  Speaker 4: We build software for embedded edge and cloud, primarily for mission-critical industries like aerospace and defense, automotive, telecommunications, robots, drones and such.
• 1:50
  Speaker 4: We pride ourselves in building software that is certifiable to industry standards for mission-critical industries.
• 1:57
  Speaker 4: We are taking the company now in the direction of becoming a leader in edge AI, because we have got the embedded and edge systems now and we want to enable our customers to build and deploy AI at the edge.
• 2:07
  Speaker 4: So this topic is very relevant and I'll share with you some of the challenges we see as it comes to commercializing edge AI in mission critical industries.
• 2:17
  Speaker 6: Oh, I've got my own.
• 2:18
  Speaker 6: So Jonathan Wood work for Intel Corporation, so my responsibility is related to software, primarily within the edge computing world and Intel.
• 2:30
  Speaker 6: We have over 200 million units out there deployed in edge.
• 2:34
  Speaker 6: We see lots of different use cases.
• 2:38
  Speaker 6: We come across different new trends and edge AI specifically is one that's fascinating when we get to some of these closing the gaps.
• 2:46
  Speaker 6: I'll talk a little bit about what Intel is doing.
• 2:49
  Speaker 6: It's a pleasure to be here.
• 2:50
  Speaker 6: Look forward to the discussion.
• 2:54
  Speaker 1: Thanks, so my name is Winkie Giesemann, ceo and co-founder of the Things Industries.
• 2:59
  Speaker 1: We exist now for 10 years and we are a global company that focuses on network software for low-power IoT networks, and we manage around 3 million edge devices on our software at the moment through a cloud service and, just like Kerry, the previous speaker speaker just says, we're moving from IoT Internet of Things to intelligence of things and there's definitely a trend that we see as well and that that's my contribution.
• 3:33
  Speaker 5: Thank you.
• 3:34
  Speaker 5: My name is Niklas Kornström, representing Advantech.
• 3:39
  Speaker 5: I'm an IoT wise consultant for that Been visiting hundreds of factories and machine creators in Europe, us and also Asia.
• 3:50
  Speaker 5: I think this topic is essential because I see a huge gap between everything that we do here, more or less like scientists, compared to looking at it as a system integration thing, and I can take a very simple example.
• 4:11
  Speaker 5: I attended machines, mines and machines in San Francisco 2015.
• 4:18
  Speaker 5: It was a PTC event and during that event it we started to discuss digital twins and I guess most of you knows what the digital twin is, even though I'm very manufactured centered.
• 4:33
  Speaker 5: But if you go to factories today smear factories, small and medium sized enterprises none are using that, even though it was a huge topic at 2015.
• 4:48
  Speaker 5: Example of Bosch, the big airplane from the US, boeing air motors, for example, and the assumption is that every company has its own development department.
• 5:05
  Speaker 5: If you go to a factory, they don't have that.
• 5:10
  Speaker 5: So I think it's a huge thing looking at it as a use case, for example, before a model.
• 5:17
  Speaker 5: It's very easy to say that we have a great model for this and that, but to connect and I think this is part of the topic of today to connect it to reality, because otherwise it's more or less that you are coming and explaining some Star Wars thing you know, the Death Star to someone who is not even familiar with the abbreviation and acronyms that you're using.
• 5:43
  Speaker 5: The gap, I think, is very important.
• 5:46
  Speaker 5: Thank you.
• 5:47
  Speaker 2: Great, actually interesting anecdote.
• 5:50
  Speaker 2: Speaking of mission critical, avijit was mentioning Wind River has 100% market share on Mars, which is pretty cool.
• 5:58
  Speaker 2: So the Mars rover actually runs Wind River software.
• 6:02
  Speaker 2: So talk about a remote deployment.
• 6:04
  Speaker 2: That's pretty intense.
• 6:06
  Speaker 2: I don't know if people can top that.
• 6:09
  Speaker 2: But yeah, it's really about connecting AI to the real world and how do we fill that gap?
• 6:15
  Speaker 2: I guess we can either go down the line or people can jump in.
• 6:18
  Speaker 2: As to, if you had sort of Nicholas you mentioned sort of an anecdote there what are some of the things experiences you've had and we have a lot of scar tissue here on the panel that have jumped out to you in your career about like lessons learned on kind of deployment and supportability of solutions.
• 6:40
  Speaker 2: Anyone can jump in here if you want, I can start.
• 6:43
  Speaker 2: Go for it.
• 6:44
  Speaker 6: I work with many of the colleagues here, so we have joint shared scars and experiences.
• 6:49
  Speaker 6: But one of the things that I've observed within and it comes to what to Nicholas was saying about trying to help with education and help make it easier for our customers to consume this technology is really people are really confused about what works best in terms of what software stack, what workload for a particular problem, and it's not generally shared across enterprise or the wider ecosystem.
• 7:16
  Speaker 6: So it is a struggle and events like this would be great and other forums to share those learnings.
• 7:23
  Speaker 6: So one of the things we did at Intel was because we obviously come at it from a very large ecosystem be great and other forums to share those learnings.
• 7:24
  Speaker 6: So one of the things we did at Intel was because we obviously come at it from a very large ecosystem is to try and create a kind of a horizontal approach to simplifying validation of specifications for Edge AI.
• 7:38
  Speaker 6: So, for example, in retail, if somebody wants to have a specific computer vision type application, what is a recommended software stack and what performance could I expect?
• 7:49
  Speaker 6: What kind of device type should I be looking for?
• 7:52
  Speaker 6: And so, working with the ecosystem our colleagues like Nicholas at Advantech is a great example when you're over here too, is to work with the ecosystem to provide those white papers, that information, both to academia, to enterprise, to our global system integrator partners that act in an advisory capacity.
• 8:09
  Speaker 6: So we launched a program called AGI Systems earlier this year.
• 8:12
  Speaker 6: It's getting great traction.
• 8:14
  Speaker 6: Everybody's sort of identifying that as a blocker because there is inconsistency of information and where to go to.
• 8:22
  Speaker 6: So we want to invest in that for our Intel Silicon platforms and anybody who's building any product, be it embedded or be it in an enterprise space, is to invest and do that work.
• 8:34
  Speaker 6: So there's one example of where there is a problem, where the ecosystem coming together can address it, but it requires an investment, it requires leadership and it requires collaboration Any other good gotcha, anecdotes or explanations.
• 8:54
  Speaker 4: I'll give you a couple of examples that we in Mid River have encountered with our existing customer base.
• 8:59
  Speaker 4: You can't hear, okay, thank you, is this better, wildly?
• 9:08
  Speaker 4: Okay, I'll try to speak up a little bit.
• 9:10
  Speaker 4: Thank you, though.
• 9:10
  Speaker 4: So at Windover.
• 9:12
  Speaker 4: I'll give you a couple of examples of companies that we've engaged with recently, evangelizing our AGI capabilities and the questions we got back from them.
• 9:21
  Speaker 4: So one is the example of a company in Korea that actually builds defense equipment aircrafts, evtols and drones and the other one is another US company and I'm sorry I can't name the customers because they are in the defense sector and this is about FAA certification of flight worthiness of AI as incorporated in software.
• 9:44
  Speaker 4: So when we met with this customer in Korea, they've been a long standing customer of our RTOS real-time operating system and we were showcasing to them some edge AI capabilities building in the cloud, deploying at the edge and the persistent question for literally 45 minutes that we got peppered was like what I see, but how are you going to safety certify this?
• 10:03
  Speaker 4: And then, second, how can I get explainability and trace?
• 10:04
  Speaker 4: Peppered was like what I see.
• 10:04
  Speaker 4: But how are you going to safety certify this?
• 10:05
  Speaker 4: And then, second, how can I get explainability and traceability to what data was used, how it was trained, how it was deployed and how it will be changed over the life cycle of the asset.
• 10:16
  Speaker 4: Because, think about this, this is a fighter aircraft or a drone and there are lives at stake and you know there can be dangerous consequences.
• 10:24
  Speaker 4: So the the most important thing for them is certification by industry standards and safety certification.
• 10:30
  Speaker 4: The other example I mentioned was FAA, where we are looking to employ AI to enable our developers to use AI to develop code, and you can see a lot of those examples in enterprise and other scenarios.
• 10:41
  Speaker 4: But the FAA has what is called a DFR, an individual who looks at the software development process, and they require three things.
• 10:49
  Speaker 4: Number one is a software bill of materials.
• 10:51
  Speaker 4: Second, as I mentioned earlier, explainability and traceability.
• 10:55
  Speaker 4: And number three is that they want to ensure that code written is actually written by a human being and not by AI.
• 11:03
  Speaker 4: So that makes it nearly impossible to employ AI to develop code that can be deployed within safety critical.
• 11:09
  Speaker 4: So we have to overcome these challenges and these are real things that our customers have shared with us now that we must find solutions to.
• 11:16
  Speaker 4: It's not a question, it's not a hurdle, it's a very, you know, right thing that they're doing, because, at the end of the day, you know there are consumers who fly in planes whose lives are at stake, so they're looking out for that.
• 11:26
  Speaker 4: Or there are defense scenarios where you know harm can come about if AI, you know, doesn't behave in the manner that it must and can be controlled and built in the right manner.
• 11:35
  Speaker 4: So you know, we've seen tremendous progress in AI, which is great for us to see.
• 11:38
  Speaker 4: We're employing these technologies, but we now are at the stage where, if you have to commercialize, we have to safety certify them and make them reliable.
• 11:50
  Speaker 4: The other thing I'll mention to you is that there is a requirement to change the development process and the culture as well, because much of certification up until this point has been done through how software has been written, typically through a V model or requirements to delivery.
• 12:01
  Speaker 4: Ai is different and I won't go into the details, but all of you know AI development is more iterative and more continuous and so that same certification model and the process wouldn't work for AI.
• 12:12
  Speaker 4: So, as an industry and as a community, we've got to develop techniques to help develop AI, deploy AI and sustain AI in a manner that is safety certified such that it can see real-world at-scale deployment.
• 12:27
  Speaker 3: Thank you.
• 12:27
  Speaker 3: Here is something that we've been experiencing in the last couple of years where we have had customers come to us and tell us they have designed in an NPU and they don't know what to do with it.
• 12:41
  Speaker 3: So the problem is don't do AI just because it's a buzzword that everybody's using and over-designing something just because you have the fear of missing out.
• 12:53
  Speaker 3: You end up making mistakes, right?
• 12:56
  Speaker 3: So that's something that we're seeing time and again, where customers over-design without understanding that they really don't need an AI accelerator.
• 13:04
  Speaker 3: You don't need NPU.
• 13:05
  Speaker 3: You can do things with a CPU as well.
• 13:07
  Speaker 3: So those are some of the learnings where everybody is jumping in on the AI bandwagon and trying to do something that doesn't fit quite well in the model I would go to you.
• 13:18
  Speaker 6: Thank you for saying that.
• 13:19
  Speaker 6: But actually just to build on that a little bit, that issue of what is know, which is a blocking factor in the industry, which is there's an expectation you need a high performance device.
• 13:30
  Speaker 6: High cost, therefore, you know, destroys the cash KPI straight away.
• 13:34
  Speaker 6: So power performance and price performance.
• 13:36
  Speaker 6: And that reverts back to some of the things I was saying earlier trying to demystify what is needed for to be optimal to fit for all different types of use cases, whatever the budget.
• 13:46
  Speaker 6: But the really important thing is to look where you're infusing AI into as many applications that kind of make sense.
• 13:53
  Speaker 6: So one of the things that we recognized and we were not doing successfully perhaps was was ensuring that we can look at both, not just the greenfield side, but the brownfield side of the devices and realizing, yeah, we have a footprint of CPUs.
• 14:09
  Speaker 6: Some of them have NPUs, some of them may have a discrete accelerator, some of them may come later editions with a new accelerator and everybody's like focused on selling the next thing.
• 14:17
  Speaker 6: We're definitely guilty of that, trying to drive people up in terms of the roadmap, but you can actually enable many of the devices and what I'm pleased to see with a software ecosystem is that they you know, like my colleagues at Wind River, but also Intel is that we're making that available to go back and just take a little bit of that data, provide a little bit of AI, maybe thinning some of the data before you're sending it back to the cloud, but just ensuring that we start with something that's realistic.
• 14:44
  Speaker 6: And there's so many use cases that we're seeing, and healthcare is a great example.
• 14:47
  Speaker 6: Somebody was talking about that.
• 14:48
  Speaker 6: I was surprised to see how they are adopting AI at such a large and rapid extent, because there are a lot of things where they can get immediate benefit in terms of time saving, in terms of prediction, in terms of helping with patient care, and so those kind of industries.
• 15:06
  Speaker 6: I think that gap has started to close.
• 15:09
  Speaker 6: But there are other industries, like manufacturing, which is highly regulated.
• 15:13
  Speaker 6: They will take a little longer, we know that, but yeah, that's the journey we're on.
• 15:20
  Speaker 2: I wanted to also riff a little bit off of what Abhijit was saying around.
• 15:23
  Speaker 2: I call it data provenance, which is something we don't really think about until you get a customer that says I need that and it's too late.
• 15:33
  Speaker 2: So what do folks think about that topic of how do we do a better job of providing transparency and data provenance through the whole kind of manufacturing chain so that you can go through some sort of certification?
• 15:51
  Speaker 2: Is anyone seeing like real progress being made in the industry in that, or is it still just like kind of a customer demand?
• 15:59
  Speaker 6: well, I just don't want to sort of occupy the whole of jenny, but yeah, I've seen that actually utilities, which is an interesting ecosystem which is very regulated, highly regulated, for obvious reasons.
• 16:10
  Speaker 6: We see lots of cyber threats and lots of issues there, but moving them to a software-defined journey was the first step which you know containerizing and moving that so I think we progressed there to get them over that hump.
• 16:27
  Speaker 6: Now that puts so maybe it is a baby step thing it's really make the change for digitization and then step into.
• 16:36
  Speaker 6: Okay, once you've done that, you're in a position where you can easily, you know, take advantage of the ecosystem Pete and start to think about adding AI to it.
• 16:45
  Speaker 6: This forum is an edge AI forum, but the edge piece is still a big advantage of the ecosystem Pete and start to think about adding AI to it.
• 16:47
  Speaker 6: This forum is an edge AI forum, but the edge piece is still a big part of the journey is really getting that in these industries like manufacturing, to become more software-dependent and it is happening and I was surprised to see that at utilities and to expand.
• 17:02
  Speaker 2: I don't know if, nicholas or Venky, you guys have any insight on that one, no.
• 17:08
  Speaker 1: so what I see is I'm from IoT being kind of heads down and that and I think one pattern that comes back every time and I think that's also something that you said and was said before is maturity alignment, and when I did my masters in business administration on the IT part, everything was about IT and business maturity alignment.
• 17:30
  Speaker 1: So you can bring in IT, but if the business processes don't kind of like are suitable, to do, anything in IT doesn't make sense, right?
• 17:38
  Speaker 1: So this was in 2003 and over the last 10 years of the IOT hype, that things comes constantly back.
• 17:45
  Speaker 1: You can bring in IoT, but if there's not a good kind of digital infrastructure kind of maturity, like you can have a sensor that sends data, but where do you put it right?
• 17:57
  Speaker 1: And then somebody says, oh, I have an IoT dashboard.
• 18:00
  Speaker 1: Okay, that's a definition of a failure if your IoT data ends up in an IoT dashboard because you still need a human to look at some red and green graphs, right, and so, moving forward, I think what kind of mature context we see that our customers are successful in AI is that AI can play, of course, at the edge, where it can filter and transform the data to be more valuable and then, of course, in the cloud.
• 18:35
  Speaker 1: But there you also see, there needs to be a level of digital maturity and we have utility customers that are rolling out with our software to do kind of one, two million water meters and for them, like these meters are so dumb, like even a simple if this, then that could make them 100% more or 50% more efficient.
• 19:01
  Speaker 1: So for them, of course they are talking at AI in their slides, but maybe just go back and just do if this, then that first.
• 19:11
  Speaker 1: So yeah, I very recognize what you just said and I feel the pattern that's coming back, that if you want to close the gap, it's about technology, maturity, alignment.
• 19:24
  Speaker 1: Yeah, so the gap it's about technology maturity alignment.
• 19:27
  Speaker 5: Yeah, so I agree with the previous speakers, of course, but I've been talking to a lot of you here earlier today as well, and I think it's obvious that you know you.
• 19:42
  Speaker 5: You come with your let's's call it Death Star from Star Wars to the factory, and many of you told me already here that's why I have some notes that the problem that you face is that, okay, here I have my AI black box, more or less, but what about the data?
• 20:03
  Speaker 5: And, as mentioned previously, it's like you're missing the part, and obviously, from what you've been telling me as well, is that the infrastructure is not there to get the data Meaning, connect, collect, store and maybe analyze that data.
• 20:25
  Speaker 5: And just to mention that, since I'm from Advantech, this is an initiative now that we have started.
• 20:32
  Speaker 5: We've done it with Intel, nvidia, qualcomm and others that we, since many of you might not know what Advantech is, it's the biggest provider of hardware gateways, routers, industrial stuff to factories in the world, so we already have that market.
• 20:51
  Speaker 5: But an interesting thing is that a channel partner of ours was very clear.
• 20:56
  Speaker 5: He said that now we need to make more out of that hardware because the factories the market that is is getting mature.
• 21:04
  Speaker 5: They are asking for more.
• 21:06
  Speaker 5: It's not enough with a gateway or a router, they want more on it initially.
• 21:19
  Speaker 5: So, with that said, we have created this now with NVIDIA, intel, qualcomm and others, where we do an API and an SDK for a certain solution to reach the GPU and NPU and other hardware stuff, but also to have what we do communication, connect to a PLC in a machine, for example, protocol conversions from OPC, ua, mod bus, mqtt and stuff like that.
• 21:40
  Speaker 5: So once those are done, they connect, connect, collect and store.
• 21:44
  Speaker 5: Those APIs and SDKs are available for all of you, since you do the AI, and there is an old saying if you say that you can do everything, you most likely can't do anything.
• 21:56
  Speaker 5: So you know we need to cooperate and we need to do this together.
• 22:02
  Speaker 5: And we need to do this together and, as mentioned earlier, coming with a true use case is essential, if you ask me, because otherwise it won't align with our business models and down the lane you will find that there is not enough investment or understanding from the C-level, and all of these things makes it so that it's viewed more like a project instead of a product that will be usable in the market.
• 22:34
  Speaker 2: Yeah, good point.
• 22:34
  Speaker 2: I mean, I think one of the big gaps is what we're hearing is.
• 22:38
  Speaker 2: One of the big gaps, frankly, is just having a good idea of the problem that you're trying to solve.
• 22:43
  Speaker 2: It kind of starts with that.
• 22:44
  Speaker 2: Having a good idea of the problem that you're trying to solve, it kind of starts with that.
• 22:45
  Speaker 2: And then this idea of abstraction to some kind of the fundamental infrastructure is important and I guess to that point and we actually had this discussion recently it was about I was going to use the S word standards Like in Edge AI.
• 23:03
  Speaker 2: We have not had our OpenGL moment in Edge AI, where there's just some API that everyone uses and it's all portable and maybe that will never happen.
• 23:12
  Speaker 2: But how important do you see standardization for portability and things like that in terms of closing this gap?
• 23:20
  Speaker 2: How important is that needed, especially at this point in the innovation cycle?
• 23:25
  Speaker 2: Got lots of microphones over there, let's go Jumping on standards.
• 23:32
  Speaker 6: Yeah, I spent two or three years on wireless working standards, so there's very good things about that.
• 23:38
  Speaker 6: And also from an innovation point of view, pete, there are things that really does slow it down in terms of control.
• 23:45
  Speaker 6: But also there are things that really does slow it down in terms of control.
• 23:48
  Speaker 6: I think working together to define open and practical solutions that the industry can adopt makes sense.
• 23:55
  Speaker 6: I think I'm very much for that.
• 24:01
  Speaker 6: There's a lot of our colleagues like Wind River has a commercial practice but also is very strong in the open source world.
• 24:07
  Speaker 6: I think being open and moving to more of an open source mindset across the industry, particularly some of our commercial software vendors that we collaborate with, I think will help stimulate working together in collaboration.
• 24:20
  Speaker 6: I think it's a practical step.
• 24:21
  Speaker 6: Yeah, I think, rather than standards, setting up practical policies that make sense for interoperability, for APIs et cetera would be good, and we're starting to see that already with Gen AI.
• 24:40
  Speaker 6: We're starting to see people adopting common themes.
• 24:43
  Speaker 6: But I'm also a little wary of standards if we go too heavy towards that.
• 24:47
  Speaker 6: So these industry-type-led initiatives make better sense for me than being more of the traditional standards point of view.
• 25:00
  Speaker 4: I think you made the point about NPUs and whether they need that or not.
• 25:04
  Speaker 4: What we're seeing is that, even though there has been some level of standardization in the AI model and runtimes and frameworks like whether it's Python, pytorch or, for that matter, onyx as a framework for expressing the model, but when you take that model and you have to make it run on a piece of silicon, every silicon provider has their own compiler and you need to fine tune it.
• 25:26
  Speaker 4: A lot of the startups that are, so to speak, demoing down there too, they specialize in fine tuning the model through quantization, pruning and even simple compilation techniques to make it run well on that particular NPU, tpu or even CPU, for that matter, because there's a lot of secret sauce in there, and so that is the lack of standardization across the industry, which in some ways opens up opportunity for startups to provide services, to find a new model, but at the same time it's a lot of cost to the end customer to actually have to do that.
• 25:56
  Speaker 4: It would be great to be able to develop a model and be able to take it across silicon families.
• 26:01
  Speaker 4: The concrete example we see is in autonomous driving.
• 26:04
  Speaker 4: We've got a full stack perception and planning model.
• 26:07
  Speaker 4: But every time we're asked by a particular OEM to go implement it on, let's say, silicon A or Silicon B or Silicon C, our team has to put in 10, 20 people for six months fine-tuning it down and using the compilers to really get it to have the power and performance characteristics on that piece of silicon.
• 26:25
  Speaker 4: So it's a lot of cost, not completely unnecessary, but quite a bit as well.
• 26:29
  Speaker 4: And so if there's standardization in that I do agree with you, because if you standardize everything you lose versatility and vitality and verticalization optimization.
• 26:38
  Speaker 4: So there's a balance to be had.
• 26:39
  Speaker 3: But today I think things are too fragmented, this lack of standardization which causes a lot of cost and pain this lack of standardization, which causes a lot of cost and pain, and I believe we can adopt some standardization, but we have to leave it loose because otherwise it's going to curb innovation, because we are accelerating the pace at which we are deploying products.
• 26:57
  Speaker 3: So, in the sense, that is where working with HCI labs, creating an ecosystem where everybody can share, becomes a very fundamental step in solving that problem.
• 27:09
  Speaker 3: That's why we launched something called Model Nova.
• 27:13
  Speaker 3: It's an open source model zoo for tiny models, small models that you can take and use it on a Raspberry Pi today, and we have close to 110 models today and by the end of this year, 150 models.
• 27:23
  Speaker 3: Open source, anybody can use it, anybody can try it.
• 27:27
  Speaker 3: It's dumbed down version of becoming an AI engineer.
• 27:30
  Speaker 3: So the big problem that we see is AI on one side and embedded on the other side.
• 27:35
  Speaker 3: There's practically no bridge in between, because AI engineers don't want to do embedded.
• 27:41
  Speaker 3: Embedded engineers don't want to do AI.
• 27:43
  Speaker 3: So we thought this would be the perfect bridge and embracing an ecosystem.
• 27:47
  Speaker 3: Bringing everybody together makes it a lot more easy.
• 27:53
  Speaker 1: Yeah, no.
• 27:54
  Speaker 1: So I totally agree, like standards are really for a phase.
• 27:57
  Speaker 1: I believe in the market where you are converging and we said, yeah, you always have to start on your solution when you know the problem, that is true.
• 28:04
  Speaker 1: But, uh, you also need to keep dreaming and we need to give people imagination of what's possible.
• 28:10
  Speaker 1: And we may not be 100 right, um, but that that's how we we drive forward.
• 28:16
  Speaker 1: I like this is, of course, a niche group of of innovators and it's funny that, like being 10 years in iot, I have a lot of familiar faces here for people that switch jobs because you want to be on a new, new thing, but that divergence that is still extremely important because that eventually the hype will be over and then you want to have the maximum amount of exposure to the maximum amount of crazy ideas.
• 28:42
  Speaker 1: So it's about dreaming, not hallucinating.
• 28:45
  Speaker 1: That's something else.
• 28:47
  Speaker 6: Before you go to the next point, I was just going to mention one thing, I think, because it's quite incumbent on companies like Intel and other silicon vendors to try and make life a little easier for solving some of these problems.
• 28:59
  Speaker 6: One of the things you have to do is not just do it from a point of view of different silicon vendors, but you also have to do it from different silicon platforms from the same vendor.
• 29:06
  Speaker 6: So very frustrating, and we have that multiple times with guys like you.
• 29:10
  Speaker 6: So one of the things we launched just in March was what we call a microvisor toolkit, a toolkit which really exposes our capabilities seamlessly through a very simple tool to use.
• 29:23
  Speaker 6: It's really making it easier for partners like some of our colleagues here, to take that and optimize their software more rapidly, and we believe we save what would be a 90, probably a six-month exercise 20 guys literally down to just a few weeks by doing that work to expose those capabilities more open and in an open source environment, which we were not doing previously.
• 29:47
  Speaker 6: So if we make those type of gestures as an ecosystem, we'll do a better job and compared to some of our other colleagues that are very closed because of their market advantage, particularly in AI.
• 29:59
  Speaker 6: We want to take a different approach and we want to make sure that we're speeding up that development process and taking costs out of where it's not really necessary.
• 30:08
  Speaker 5: Nicholas, yeah, yeah, since the topic was standards, I think standards are great, but they are also very dangerous if you make them too firm.
• 30:19
  Speaker 5: And an example of that is that, once again, the, the Boeing had to transport an engine through states in the US to assemble it, but they couldn't because the American trains couldn't carry it that distance through tunnels.
• 30:37
  Speaker 5: Why?
• 30:38
  Speaker 5: Because the railroads were too thin to carry it.
• 30:42
  Speaker 5: And why were they too thin?
• 30:44
  Speaker 5: Because back in the days, englishmen had to build the railroads in England, in the US.
• 30:51
  Speaker 5: Hence they used the English railroad system.
• 30:55
  Speaker 5: And what did the English railroad system build upon the Romans that was there and their wagons?
• 31:01
  Speaker 5: So standards can be dangerous.
• 31:05
  Speaker 5: So you need to build standards that are kind of flexible and if you look at a genetic AI right now, for example, and model context protocol that is now coming up since November, this is a way of making a standard for AI to actually have some kind of protocol to talk to each other.
• 31:30
  Speaker 1: Do you know, by the way, what kind of how they came to the width of the tracks, the wagons, oh sorry, and the horse that would occur.
• 31:39
  Speaker 1: It all goes back to the Romans.
• 31:44
  Speaker 2: Those Romans?
• 31:44
  Speaker 2: I'm sure we have some Romans here.
• 31:46
  Speaker 2: Sorry, it's your fault.
• 31:49
  Speaker 6: Be an English P.
• 31:50
  Speaker 2: I'll take responsibility for that there you go A little bit, A little bit of responsibility there.
• 31:54
  Speaker 2: We're going to open it up for questions in a little bit because we've got about 10 minutes left.
• 32:00
  Speaker 2: But I did want to give folks kind of you know, sort of a key takeaway, for how can the audience help us in this journey If you had your sort of Thanos snap?
• 32:12
  Speaker 2: What is the thing you want this crowd to do to kind of bridge that gap between research and reality?
• 32:20
  Speaker 3: Go to the Model Nova website and sign up as a user.
• 32:22
  Speaker 3: Modelnovaai, please, you can help us.
• 32:28
  Speaker 6: Well, I think that already we, as we move to an open source and use tools like Hugging, face and GitHub, there is an opportunity to provide feedback through kind of recognizable tools right now, and so, I think, continue to do that.
• 32:42
  Speaker 6: Edge AI we moved a whole platform which we call Open Edge platform, which is our instantiation of how to build Edge AI application, how to build vision applications on our silicon.
• 32:56
  Speaker 6: Provide that feedback to us.
• 32:57
  Speaker 6: Where do we need to improve what is missing?
• 33:00
  Speaker 6: From the academia point of view as well, I think this we do need to bridge that gap.
• 33:06
  Speaker 6: I don't think we talked about that too much on here is to provide the tools and SDKs in the forums.
• 33:11
  Speaker 6: More work would be needed, but what I would ask you to do and encourage you to do, is that as you're building your applications, you're starting to work with the software that's out there is.
• 33:21
  Speaker 6: Make sure you're providing feedback through those channels.
• 33:27
  Speaker 1: I very much agree with that.
• 33:29
  Speaker 1: Learning alone is so much more inefficient than learning together.
• 33:33
  Speaker 1: We've been always open source, also had a collaborative platform on the application and device creation layer, and that has been the core to the success of our company that we always were very open and also inspired our customers to share a lot as well.
• 33:49
  Speaker 1: And I think if you build that kind of collaborative learning and you build that into your ideas and not try to build bald gardens or to protect every little thing that you think is your IP, but really focus on one thing that do you well and give away the rest, yeah, that will help in the end yourself and everybody.
• 34:16
  Speaker 4: Yeah, I think I'll give you four words that I think are critical for this community to rally behind.
• 34:22
  Speaker 4: Number one is standardization, we talked about.
• 34:25
  Speaker 4: Number two is traceability.
• 34:27
  Speaker 4: Number three is explainability.
• 34:30
  Speaker 4: And number four is certification.
• 34:32
  Speaker 4: For AI to truly come alive at the edge, those four things are going to be important.
• 34:36
  Speaker 4: You know the things about data provenance right.
• 34:39
  Speaker 4: The last thing, I think the debate around standardization, or proprietary versus open source, if you will.
• 34:46
  Speaker 4: I think there's an African proverb that says if you want to go fast, go alone, if you want to go far, go together.
• 34:53
  Speaker 4: And I think that really speaks to community.
• 34:55
  Speaker 4: If this community can come together and work on those four things, we can really advance the state of the art of AIGI but, more importantly, take the community forward and far into the future.
• 35:06
  Speaker 4: So that's really the value of this community, that's a collaboration we must do, because that's how we can advance the state of the art.
• 35:16
  Speaker 5: I think you should always look at what new laws are coming.
• 35:22
  Speaker 5: We have the AI Act.
• 35:25
  Speaker 5: We have goals of, for example, energy in the EU.
• 35:30
  Speaker 5: Coming, if you look at manufacturers, for example, is a huge deal 2026 and all of that, and you know.
• 35:38
  Speaker 5: Looking once again at the clear use case, where this can be used to make them understand it, I think is essential.
• 35:47
  Speaker 5: So, for example, operational equipment effectiveness is a way to measure how good your machines are working in a factory.
• 35:56
  Speaker 5: But right now, for example, a great opportunity is to look at energy consumption in combination with that, meaning that you actually use AI to find out about your GHG, greenhouse gas emission, etc.
• 36:12
  Speaker 5: Because all of this is coming and all the factories knows it.
• 36:17
  Speaker 5: But even though they might not know AI per se, they might know the value of the combination if it's explained.
• 36:28
  Speaker 3: Oh yeah, well, I just want to throw out a fact.
• 36:31
  Speaker 3: I was reading this recently, especially in residences.
• 36:36
  Speaker 3: I'm sure all of you will associate with this.
• 36:38
  Speaker 3: Do you know?
• 36:40
  Speaker 3: Does anybody know how many devices connect to your residential router on an average in an average home?
• 36:46
  Speaker 3: I'm talking about the US and maybe Europe.
• 36:48
  Speaker 3: It's applicable in Europe as well.
• 36:50
  Speaker 3: Does anybody know what the number is?
• 36:52
  Speaker 3: Oh, okay, so the average is.
• 37:03
  Speaker 3: Did anybody mention any other number?
• 37:06
  Speaker 3: I'll go with 40.
• 37:07
  Speaker 3: How about 40?
• 37:08
  Speaker 3: Right on, it's actually between 35, yeah, 35 and 40, and this is the kicker.
• 37:19
  Speaker 3: They are expecting this to more than double in the next four years.
• 37:24
  Speaker 3: Imagine that is edge right there, just for the residential home.
• 37:30
  Speaker 2: I just want to leave that thought good, so let's see if there's any sort of pent-up questions from the audience.
• 37:38
  Speaker 2: Oh, we got a couple over here ready right off the bat.
• 37:41
  Speaker 7: Go for it thank you, very interesting discussion and I just wanted to bounce off something that was said.
• 37:48
  Speaker 7: So I love the comparison that was made to the protocols A2A, mcp and now there's NCP as well.
• 37:56
  Speaker 7: That's come out.
• 37:56
  Speaker 7: What I find interesting about it is A2A is an enterprise, google is the one behind it.
• 38:01
  Speaker 7: Mcp is Anthropic, so that's a startup.
• 38:05
  Speaker 7: And then you've got NCP, which is actually MIT Media Lab.
• 38:08
  Speaker 7: So there's three very different type of actors which are each pushing out individual protocols out there which are for some reason kind of becoming a de facto standard.
• 38:18
  Speaker 7: And when I look at AGI, I actually find it a bit surprising, because Margo margoorg was actually launched what two years, three years back, maybe even a bit more Intel's part of it right now Capgemini as well.
• 38:30
  Speaker 7: So your point that you were making Avijit was interesting, because I like the way that you divided these four things right.
• 38:38
  Speaker 7: You said there's four things to be done.
• 38:39
  Speaker 7: Does the Edge AI Foundation need to actually fight the standardization battle?
• 38:44
  Speaker 7: And how does it work with Margo?
• 38:47
  Speaker 7: Because you say Margo, you can take care of standardization, we'll take care of this.
• 38:51
  Speaker 7: And it just feels that that conversation hasn't happened, even though when we look at the agent-to-agent protocols over there, it seems to have happened organically, so what's the reason this is not happening in the edge AI world?
• 39:06
  Speaker 2: Well, I mean, I think we've had that conversation In fact, we had that conversation at our board meeting the other day around certification and is there something we should be doing to lean in harder in that space to help at least characterize what devices and what systems meet a best practices standard in some sort of certification, to drive that clarity?
• 39:25
  Speaker 2: So, yeah, it's kind of a hot topic.
• 39:26
  Speaker 2: It goes back to the conversation of how much do you lean in on that as a best practice versus a standard and things like that, but it's definitely a topic.
• 39:38
  Speaker 2: Yeah and I know.
• 39:39
  Speaker 6: Nicholas, obviously from your team was heavily involved in working with that ecosystem.
• 39:45
  Speaker 6: For those that don't know, that ecosystem is your traditional industrial PLC type companies that provide heavily into manufacturing.
• 39:51
  Speaker 6: What I would say is what I've seen, irrespective of the discussions that are going on between different forums, is that they are trying to adopt best practices.
• 40:01
  Speaker 6: They are looking to companies like yourself and companies like you to advise them.
• 40:05
  Speaker 6: So what I'm seeing in those type of forums really more the industry solving the problem of collaboration, because in particularly if you had a Schneider or a Rockwell or a 8, we'd be not really trying to work together because of obviously competitive practices.
• 40:20
  Speaker 6: So I think it's a really good step that they're doing that, and that's one of the industries that is highly regulated.
• 40:26
  Speaker 6: It's slower to adopt change, so I would encourage it.
• 40:30
  Speaker 6: But with regard to them defining standards, no Recommended deployment practices would be where I see that going and that makes a lot of sense and interfaces and simplicity too and just adopt the industry standards that are out there, the ones that are leading.
• 40:47
  Speaker 6: So that's kind of my observation with Margot.
• 40:50
  Speaker 6: I just wanted to share that perspective.
• 40:53
  Speaker 3: In my opinion, the whole industry.
• 40:55
  Speaker 3: We are still crawling right.
• 40:56
  Speaker 3: We're not running yet.
• 40:57
  Speaker 3: It's very nascent.
• 40:58
  Speaker 3: We are still experimenting, we are still identifying how do we bring AI to the edge, what are the things that we need to do, and once we have a full cycle use case that we have completely deployed and understood, then we can understand, understand the framework and go towards a little bit of standardization.
• 41:14
  Speaker 3: In my opinion, we're still.
• 41:16
  Speaker 7: We're still not there but agent technically is kind of like younger, so if it's happening over there, is there a lesson that we can learn yeah, yeah, definitely.
• 41:27
  Speaker 3: It's a lesson.
• 41:27
  Speaker 3: We can learn to copy it over a year, but the dynamics of small edge and IoT is a little different.
• 41:34
  Speaker 3: So once we have a grasp of what it takes, then we can adopt what we have learned from the agents.
• 41:49
  Speaker 8: This question is a bit to each panelist.
• 41:52
  Speaker 8: You all come from organizations that sit at a different layer in the stack, almost.
• 41:57
  Speaker 8: So the question is maybe two parts.
• 41:59
  Speaker 8: One, where are you actually seeing the most traction in a commercial engagement right now?
• 42:04
  Speaker 8: From a vertical and application-specific application, an application-specific application.
• 42:07
  Speaker 8: And the second part to that would be is there a blocker to you know, multiplying that commercial engagement right now that you would need solved, or you're seeing already that needs a solution?
• 42:20
  Speaker 3: Great question what is happening now with our customer base?
• 42:24
  Speaker 3: We find a lot of our customers.
• 42:28
  Speaker 3: They don't have a dedicated software department.
• 42:30
  Speaker 3: In fact, they've never used to even knowing what software is.
• 42:35
  Speaker 3: And today, a lot of these device manufacturers for example a door lock camera Take that as an example.
• 42:43
  Speaker 3: So there are many of these manufacturers who are now putting in an NPU in those door locks, with the camera, and they don't have the wherewithal to understand AI and embed it.
• 42:53
  Speaker 3: And that's the biggest thing that we are seeing.
• 42:56
  Speaker 3: We're getting so many requests that people are trying to understand how do I even use this?
• 43:00
  Speaker 3: Can you help us?
• 43:00
  Speaker 3: So that is where we see us growing tremendously in the next few years.
• 43:07
  Speaker 5: Yeah, I can also add to that Once again.
• 43:09
  Speaker 5: Then, from the manufacturer perspective, the first thing I do when I go to a factory is that I want to do a factory walkthrough on the floor.
• 43:16
  Speaker 5: And then I ask three simple questions Do you store any data today?
• 43:21
  Speaker 5: And they go no.
• 43:23
  Speaker 5: And then they think, well, we do.
• 43:25
  Speaker 5: Actually, we bought this new machine.
• 43:27
  Speaker 5: It actually stores some data.
• 43:29
  Speaker 5: Okay, good, what do you do with it?
• 43:31
  Speaker 5: Nothing, because we don't have developers or anything, but they still pay for it.
• 43:38
  Speaker 5: And the third question, to just make it even more silly, is do you know who owns that data in the machine?
• 43:44
  Speaker 5: It's not the factory that bought it.
• 43:46
  Speaker 5: At times it's in the small prints, it's the machine creator.
• 43:51
  Speaker 5: So they have to pay a lot of money to get it.
• 43:54
  Speaker 5: And since we discussed standards PLC's, programming Logical Controllers that are connected to machines most of those registers to get the data are vendor locked.
• 44:05
  Speaker 5: You can't even get that data.
• 44:07
  Speaker 5: You have to pay for it.
• 44:08
  Speaker 5: So, as many have mentioned here, being open, this is a great opportunity.
• 44:14
  Speaker 5: Since this is now starting and, as mentioned, we are crawling.
• 44:18
  Speaker 5: It's a great opportunity to actually tell everyone that in order for us to be interested, it has to be open.
• 44:25
  Speaker 5: That's number one, because you need to get the data without a fuss, because if it ends already there with the cost without even showing value, it's a hard thing to do yeah, you're gonna say something, but I'll just jump in.
• 44:43
  Speaker 6: I think you know if we depends how you define where you, you know what you capture within AI right.
• 44:48
  Speaker 6: So if you think about computer vision, I think we've solved a lot for that pretty much.
• 44:54
  Speaker 6: I think the industry's done a great job of getting that embedded into many different use cases and that's great.
• 45:03
  Speaker 6: But that is AI One of the things I still think.
• 45:07
  Speaker 6: Moving from POC projects, we see many of hundreds of POCs to get them into production, I think, providing better sandbox tools and environments to test and quickly validate a use case.
• 45:23
  Speaker 6: And that's a bit of a mindset thing for certain industries.
• 45:25
  Speaker 6: Some industries are more agile at doing that, but I think we can do more there to make it easy, because making that leap from a POC and a process to go through that can take many, many months.
• 45:36
  Speaker 6: So in often cases it's not necessary.
• 45:38
  Speaker 6: So some of the things that we've been doing with our software ecosystem is really say, do so, rather than spend a lot of time working with a partner to actually help them set up the POC is really just demonstrate, just get out there, provide the tool, we will show you what is working very quickly.
• 45:57
  Speaker 6: So to try and bridge that and some of the areas where we're investing is with the ISV ecosystem and some of our partners is to actually have those demonstrations available more rapidly so people can experience the capability of having AI, particularly Gen AI, which is very new and we're at the start of this journey on those Gen AI and Agenti, let's be very clear is just to have those use cases and examples more accessible to speed up this problem of taking that, you know, poc, pilot, commercial production, which is a year, maybe 14 months, depending on what industry we need, to collapse that whole process.
• 46:32
  Speaker 1: Yeah, so yeah, where we're sitting in the technology stack is basically the network and data routing of telemetry for constrained and battery-operated devices.
• 46:43
  Speaker 1: So, because it's small batteries, small devices, ai on the edges is harder, but it's very clear if you look into the future that there's a big role.
• 46:53
  Speaker 1: So here are like a brain chip in Inatera, so they have this kind of like analog and digital AI sensor components and that, for instance, will only drive the battery efficiency more of these devices and add metadata and kind of make more of these devices and add metadata and kind of make sure that these devices last on a battery longer and provide more relevant data.
• 47:15
  Speaker 1: And then, of course, like all our customers, they consume all this telemetry data and there is actually a lot and specifically kind of like a bit of a poor man's anomaly detection, like here's a data set of something, here's a JSON Do you see something strange?
• 47:34
  Speaker 1: And if you don't need a correct hit, 100% of the time, some Gen AI can actually already go through some data series of small chunks and just say, hey, this is strange, and maybe I'll send somebody that's responsible for operations an email or a text message.
• 47:50
  Speaker 1: So, yeah, these small things, but yeah, the future is very clear on this.
• 47:58
  Speaker 4: Given where we sit in the stack, mostly it's at the operating system layer, so north of us.
• 48:03
  Speaker 4: Basically, the challenge is enabling tooling to develop AI.
• 48:07
  Speaker 4: So we are actually partnering with a lot of startups that are able to do AI and LOPs and then optimization of the model to deploy it.
• 48:14
  Speaker 4: And then on the south side the operating system has to run on a variety of silicon.
• 48:19
  Speaker 4: So we have to partner with various silicon partners, intel and others in the room.
• 48:23
  Speaker 4: But the south end of the equation is very labor-intensive because, as I said earlier, compilers are different, the SDKs are different, so we have to throw people, bodies at it to make sure we have the drivers and the BSPs to make our operating system work on each family of silicon.
• 48:37
  Speaker 4: On the north end it's not as labor-intensive, so there's more chance for standardizing tooling because the model frameworks are fairly common, the tooling is fairly open source.
• 48:49
  Speaker 4: So there's that economics of it, you know labor at the south end of it and automation and technology more at the north end of it, which is where we see some of the challenges and the focus that we've got.
• 49:00
  Speaker 6: Yeah, Not a new point, but I think I missed something and I wanted just to round it out, which was when I talk about tools and et cetera and sort of speed up the process.
• 49:09
  Speaker 6: One of the things that we did do, which we were not doing previously, is we're donating a lot of the reference code that we built for retail, for manufacturing, for healthcare, for robotics real application code that's available and that we were open sourcing.
• 49:25
  Speaker 6: That Again, before we were kind of like a little shifty.
• 49:28
  Speaker 6: I guess in terms of this is IP that we may want to potentially monetize, or our ecosystem wants to monetize.
• 49:34
  Speaker 6: But really get out ahead of that and say you don't need to start from scratch, just like with the language models.
• 49:41
  Speaker 6: There's a lot of models out there.
• 49:42
  Speaker 6: There's a lot of application code that's been developed for specific verticals.
• 49:46
  Speaker 6: So get it out there, get the ecosystem engaged with it, and then, of course, they can improve it, they can modify and enhance it.
• 49:53
  Speaker 6: So that's the other thing we're doing that's available through what we call our Edge AI reference suites.
• 50:00
  Speaker 2: Awesome.
• 50:00
  Speaker 2: Well, I think we're at about time, and I just want to thank the panel here Plenty of experience, like I said, scar tissue and insights on this gap.
• 50:09
  Speaker 2: That is really critical for all of us to kind of monetize all the research that we're doing.
• 50:14
  Speaker 2: So thank you again.