Latest / EDGE AI POD / Beyond the Cloud: The Hidden Security Challenges of Edge AI

Transcript

Beyond the Cloud: The Hidden Security Challenges of Edge AI · EDGE AI POD

• 0:07
  Speaker 1: Thanks everyone.
• 0:08
  Speaker 1: So it's 2 o'clock Friday afternoon.
• 0:10
  Speaker 1: We are truly on the home straight and it's great to have you all still with us.
• 0:15
  Speaker 1: We have a really exciting panel this afternoon dealing with a really important topic of edge security.
• 0:22
  Speaker 1: My name is Patrick Ward.
• 0:24
  Speaker 1: I work with the Edge AI Foundation.
• 0:25
  Speaker 1: I look after partner development for Europe, so if we haven't had a conversation already and you're not a part of the foundation, let's have a conversation before you leave today.
• 0:35
  Speaker 1: Let me introduce why don't the panel?
• 0:38
  Speaker 1: Introduce yourselves and then we'll get going.
• 0:40
  Speaker 1: Munia.
• 0:41
  Speaker 2: Sure.
• 0:41
  Speaker 2: Thank you, Patrick.
• 0:43
  Speaker 2: So, hello, I'm Mounia Karouchari, so I'm mostly working for STMicroelectronics in France, mostly focusing on the different threats issues that we might face in the future for AI, so the really security issues that we will discuss today with this fantastic panel.
• 1:05
  Speaker 1: Looking forward to Munya.
• 1:06
  Speaker 3: Matthias.
• 1:06
  Speaker 3: Hi, I'm Matthias Kohlke and I work as a consultant for Thistle Security and for Korean AI accelerator DeepX and a bunch of other computer modules, and so I think it's going to be a great discussion.
• 1:22
  Speaker 1: Looking forward to Matthias and Ali Ali.
• 1:24
  Speaker 4: Ors.
• 1:24
  Speaker 4: I work for NXP Semiconductors.
• 1:26
  Speaker 4: I lead the overall AI strategy and innovation products that we have for AI enablement.
• 1:32
  Speaker 4: So that includes all our hardware and software offerings under the EIQ branding that we have at NXP across our microcontroller and applications processor products.
• 1:42
  Speaker 1: That's awesome, ali.
• 1:42
  Speaker 1: Great to have you with us today.
• 1:43
  Speaker 1: So, look, let's put a bit of definition around.
• 1:46
  Speaker 1: Oh, as long as I stay standing on this chair, let's put a bit of definition around security.
• 1:51
  Speaker 1: Look, when we create an architecture for a system, we make choices.
• 1:55
  Speaker 1: When we make those choices, it seriously affects the security landscape.
• 1:58
  Speaker 1: As we move AI workloads to edge devices, maybe I'd ask each of you how does that affect the security profile?
• 2:07
  Speaker 1: What are the kinds of risks that are emerging as we move AI to the edge Munia?
• 2:11
  Speaker 1: Let's start with you.
• 2:13
  Speaker 2: Yeah, that's a great question, Patrick, to begin this panel discussion, maybe I will begin with something that has been said on the first day by Lucas Garcia.
• 2:22
  Speaker 2: I found very interesting.
• 2:24
  Speaker 2: He was asking the audience do you trust your AI models?
• 2:29
  Speaker 2: And I would say to that question, honestly, personally, I don't trust them.
• 2:34
  Speaker 2: We should not trust them.
• 2:35
  Speaker 2: There are many stuff that could go wrong.
• 2:38
  Speaker 2: So, honestly, it could go wrong on, I think, on different levels actually.
• 2:44
  Speaker 2: So, model systems, for instance, if we consider AI models, the AI algorithms, I think for this audience it would be interesting.
• 2:52
  Speaker 2: We can maybe, yeah, divide them in two categories.
• 2:55
  Speaker 2: Maybe, yeah, let's go with algorithmic attacks, so it's a really attacks that will use the algorithms themselves, so take advantage of the algorithms in order to change its behavior, manipulate the behavior of the model.
• 3:09
  Speaker 2: You can give an example on that, patrick.
• 3:12
  Speaker 2: Let's say, yeah, let's take face recognition.
• 3:16
  Speaker 2: There is a model with the four of us where the model will just try to recognize who's actually, for instance, in front of the building, and then maybe I'm an attacker, I'm a bad person, I want to access a building to which I should not have access to.
• 3:31
  Speaker 2: Let's say, the model actually knows me like this.
• 3:38
  Speaker 2: So everyone, I will do like a Superman transformation there, please bear with me.
• 3:42
  Speaker 2: Transformation there, please bear with me.
• 3:50
  Speaker 2: So yeah, let's imagine the models the.
• 3:52
  Speaker 1: AI model knows us like this.
• 3:53
  Speaker 1: It's been trained on our faces.
• 3:54
  Speaker 1: It recognizes us for authentication, yeah.
• 3:56
  Speaker 2: And if I am like that, I would be recognized as Munoz normal.
• 4:00
  Speaker 2: But I printed these specific glasses, which actually appeared a couple of years ago in the literature I appeared.
• 4:06
  Speaker 2: With these glasses I now look like Patrick and I have access to billions of AGI foundation or things like that.
• 4:13
  Speaker 1: I'm using my imagination here, okay, so yeah for everyone from the ILLIS.
• 4:18
  Speaker 2: Hopefully you can see that it's me, but the AI model won't know that it's Munya.
• 4:24
  Speaker 2: But now I can access whatever building.
• 4:26
  Speaker 4: Patrick has access to.
• 4:28
  Speaker 2: So that's really the kind of things, threats, on which we should be informed of, be aware of and really try to protect against them.
• 4:37
  Speaker 2: This is the first category.
• 4:38
  Speaker 2: I will go really quickly on the second one because it's more.
• 4:41
  Speaker 2: It's less related to the model itself, but more on the hardware devices and maybe with matthias it will be closer also to his topic of focus within this technologies.
• 4:53
  Speaker 2: It's um, the physical attacks that we see in general way in cyber security.
• 4:59
  Speaker 2: So all these physical attacks actually have been shown in the late six, five, six years that it actually applies just as well on AI models than it was applying on cryptography.
• 5:13
  Speaker 2: So, for example, we have an AI model, the AI algorithm, and now we will try to modify some weights.
• 5:21
  Speaker 2: Again, if I take the same example than before, I have very big model, millions of parameters, but I want to change just a couple of I don't know 10 weights and again I will look like Patrick, same category.
• 5:39
  Speaker 2: I would say also we have the third threat, which is site analysis.
• 5:43
  Speaker 2: Where we have the third threat, which is site analysis, where we have very big model, trained, I don't know, with a very interesting data set.
• 5:50
  Speaker 2: It took the engineers maybe months, if not here maybe one year or two years to develop this kind of model, and personally, as an attacker, I want to recover that model with less effort possible and in as short time as possible maybe a couple of hours and then I get a model that has been trained for months, if not more.
• 6:17
  Speaker 2: So this is kind of threads that I think are really important to take into account today in when we consider AI and all the issues that this could lead when we develop models and how to implement them.
• 6:33
  Speaker 1: These are the things we need to be aware of and then put things in place to protect ourselves against them and, in case they happen, to be able to deal with them when they do happen.
• 6:40
  Speaker 1: Matthias, maybe I could come to you.
• 6:43
  Speaker 3: Yeah, right, it's always quite interesting when you talk about the AI security.
• 6:46
  Speaker 3: Someone like me, who comes from hardware, then moved to software, thinks about security like when we're protecting the model from excess data as rest, secure, ota.
• 6:57
  Speaker 3: But there is also that kind of attacks of changing the weight on the model.
• 7:02
  Speaker 3: But yeah, let's focus on that side of securing the communication.
• 7:07
  Speaker 3: So that's what we also do at Bizzle right there, I think I'm always surprised how many people are unaware that a good security there requires a combination out of software and hardware.
• 7:21
  Speaker 3: Nowadays you think everything is just an app, but no, where do we store the key?
• 7:31
  Speaker 3: What attacks do exist?
• 7:31
  Speaker 3: It's always a battle between attackers and engineers to do that, and so it's always a combination of this and if it's embedded in the chip from like secure boot on thrust zone already from the vendor, or if it's a third party element to do, and then at least then make sure that no one gets a hold in an easy way of the of the model, which would make it then for all the other products out there more easier to basically create tricks to change the the weight of this.
• 8:00
  Speaker 3: So yeah, it's on both sides.
• 8:04
  Speaker 1: So we're hearing kind of AI specific stuff, model data, algorithm, et cetera, but also this usual cybersecurity, data at rest, data in transit all of which still applies especially so.
• 8:16
  Speaker 1: In fact, ali, do you want to come in there?
• 8:19
  Speaker 4: Maybe I'll take it a bit back to the fundamentals.
• 8:21
  Speaker 4: We jumped right into the model protection.
• 8:23
  Speaker 4: When we look at edge deployments, you're also looking at the benefits and the pros and cons around that.
• 8:30
  Speaker 4: Edge versus cloud deployment or edge and cloud deployments there is the whole.
• 8:36
  Speaker 4: When you're running something in the cloud, you have a single point or an aggregated point of potential failure and potential attack.
• 8:44
  Speaker 4: It's a bigger target to attack.
• 8:46
  Speaker 4: So when you start moving into the edge, you start getting into resiliency out of that effect that you're now distributing your system.
• 8:54
  Speaker 4: So there's a level of resiliency that you get from that distribution, but you're also proliferating nodes that are open to attack.
• 9:02
  Speaker 4: So while you have the resiliency that comes from the distribution side, you end up getting into vulnerabilities, as Munian and Matthias mentioned, around opening yourself up for physical attacks because the cloud is less reachable from a physical point but the nodes of the edge, when you've deployed on the edge, you start opening yourself up to the the physical attacks, which are the traditional.
• 9:21
  Speaker 4: You're still open to traditional software attacks, but you're now open also to an accessible platform with edge attacks like fault injection, like side channel and denial of service type of aspects as well.
• 9:34
  Speaker 4: So you need to have a very robust protection of the traditional cybersecurity and traditional security elements on that edge deployment itself.
• 9:44
  Speaker 4: Other benefits of going into the edge, of course, if you're dealing with privacy-related data or proprietary data that you want to maintain, running AI on the edge gives you that benefit of maintaining data that is privacy-critical.
• 9:59
  Speaker 4: You're maintaining it on the edge.
• 10:01
  Speaker 4: You're potentially not pushing it back onto a transmission channel from where you could have vulnerabilities in the communication channel itself.
• 10:08
  Speaker 4: But that also means again that you need to have very robust protections on the edge device itself so that that private data that you're trying to maintain is protected.
• 10:18
  Speaker 4: And we're also getting into additional requirements around regulations for, like Cyber Resiliency Act, the UAI Act and these are also coming into play of pushing what you do on the hardware.
• 10:31
  Speaker 4: And from NXP's perspective, we're a semiconductor vendor, so we're on the hardware layer and the enablement layer.
• 10:37
  Speaker 4: But our customers and our customers' customers are sitting on the software stack more and more, so they need to be aware and enabling all of that from a protection perspective is critical.
• 10:48
  Speaker 1: It's a pretty complex picture.
• 10:49
  Speaker 1: There's a lot of layers to this.
• 10:50
  Speaker 1: There's a lot of things to be considered.
• 10:52
  Speaker 1: Munir, you talked about algorithmic attacks.
• 10:55
  Speaker 1: So we're talking about getting access to, and taking control of, and in some way modifying, the algorithm itself.
• 11:03
  Speaker 1: Can you just talk us through a little bit of what?
• 11:05
  Speaker 1: How does that happen and how can we protect against that?
• 11:09
  Speaker 1: Just dial into that a little bit more for me.
• 11:12
  Speaker 2: Yes, sure so algorithmic attacks, they actually could happen in different fashions.
• 11:17
  Speaker 2: So it depends on the means, the goals of the attacker.
• 11:23
  Speaker 2: So, yeah, very powerful attacker, we can imagine that he has access to the means, the goals of the attacker.
• 11:26
  Speaker 2: So, yeah, very powerful attacker, we can imagine that he has access to the model.
• 11:28
  Speaker 2: He has access to everything actually.
• 11:30
  Speaker 2: So if I know the model, I know maybe even the data set on which the model has been trained, it's easier so that to imagine images.
• 11:41
  Speaker 2: So, before I took the analogy for facial recognition, where my glasses misinterpret lead to the model to misinterpret the right class, the right person, yeah, the right person?
• 11:54
  Speaker 2: yes, and in that case it's just that because I choose some very minor perturbation that is realistic glasses many people here today have glasses, just like me, and it's realistic.
• 12:08
  Speaker 2: So we are trying to find the minimum perturbation within the input image that leads to a prediction error and when we know the model, so the weights, that topology of the algorithm that we are currently implementing, it's way easier and, for instance, yeah, we will base that most usually on gradients of the model, just to give you a bits of specifics.
• 12:32
  Speaker 2: But so yeah, this kind of thing are really yes, we see that quite a lot in literature.
• 12:39
  Speaker 2: But maybe I can go back to what Ali was mentioning for distributed, and in that case I think it's even more important to the algorithmic attacks because we can actually have in a decentralized architecture that we more and more see, also Queer and development.
• 12:57
  Speaker 2: At ST we have these different models at the edge and if we imagine one of them being malicious and performing this attack on the device for instance, I'm one of the one providing, you know, the data to the, to the server, and then just will, like that I can manipulate all the other clients.
• 13:22
  Speaker 2: So this is even more powerful in decentralized architecture and data poisoning features.
• 13:29
  Speaker 1: So by taking control of one point, you're getting then access to others and then can take more control of the distributors Exactly.
• 13:37
  Speaker 2: You can even manage to manipulate the whole circuit, the whole framework, even maybe creating backdoors leading to different functionality of the full model.
• 13:50
  Speaker 1: Right, okay, matthias, I know you wanted to come in there as well.
• 13:54
  Speaker 3: Yeah, that's the same thing I want to say, the difference between distributed and undistributed.
• 13:59
  Speaker 3: And then you might have actually the security on the data is all solid and touted but someone manip.
• 14:05
  Speaker 3: Security on the data is all solid and touted, but someone manipulates basically algorithmically the data and then still the security layers all intact but the data going back to the, to the server and and training and influencing, then synchronizing with the other nodes, basically gets the wrong input.
• 14:20
  Speaker 3: So you see, it's not only about the complete current security on the data exchange and cryptography thing, but there's still other attacks with the keeping the function intact but just manipulating some training data.
• 14:40
  Speaker 1: Okay, and Ali, I wanted to bring you in there also because I guess when we're thinking about the architecture where to run the workload cloud edge, some sort of hybrid configuration how how does security influence where we're going to make those choices?
• 14:57
  Speaker 1: So we've looked at it from we make those choices, the security implications.
• 15:00
  Speaker 1: Let's look at it from the other way how does it influence architecture?
• 15:04
  Speaker 4: typically.
• 15:04
  Speaker 4: I mean, the first thing that influences where you run the workloads, today at least, is coming from the cost structure.
• 15:12
  Speaker 4: Where do you have the best return on your investment of where you run?
• 15:16
  Speaker 4: It's a very economics-driven decision, I think.
• 15:20
  Speaker 4: More and more, especially with regulations and some of the threat levels coming up, it's become, you know, we want to shift that discussion to security-oriented discussion as well of where some of these workloads should run.
• 15:32
  Speaker 4: Now.
• 15:32
  Speaker 4: Cloud is very well suited for training, for higher-level compute and higher-level reasoning aspects, and there what you need to do with the cloud side is to have a very strong perimeter protection and a very good data governance structure around what you're running on the cloud when you start getting into the edge.
• 15:53
  Speaker 4: You're very good at latency and real-time operations.
• 15:56
  Speaker 4: You're very good at data privacy protection on the edge, but you need to have a very robust security of the device itself.
• 16:02
  Speaker 4: On the edge.
• 16:03
  Speaker 4: There's more and more capability that's actually able to be pushed on the edge.
• 16:07
  Speaker 4: Yesterday in another panel, we were talking about LLMs and generative AI.
• 16:12
  Speaker 4: Is it possible to run on the edge?
• 16:14
  Speaker 4: Yes, Like you can run these generative AI models now on the edge, which kind of leads to additional surfaces that open up and additional capabilities from a vulnerability perspective that open up when you start talking about generative AI running on very small edge nodes.
• 16:29
  Speaker 4: So the decision is typically from where does it make sense to run the workload?
• 16:34
  Speaker 4: A lot of the decision right now is around the economics of where you run it trying to run it on lower-cost devices, more distributed, trying to save on your connectivity costs and your, of course, cloud costs, which can get aggregated quite high, Moving down to the edge, but also moving down to the edge with a security-first mentality of where does it make sense From a security perspective?
• 17:00
  Speaker 4: Does it make sense to be on the edge?
• 17:02
  Speaker 1: So there's a tension there between the economics, the security, and you're making the optimal decision within those constraints.
• 17:08
  Speaker 1: I think you wanted to come in there.
• 17:09
  Speaker 3: Yeah, I think that's very interesting to say.
• 17:11
  Speaker 3: It's not anymore is it capable to run things.
• 17:14
  Speaker 3: You need to deserve infrastructure before.
• 17:16
  Speaker 3: On the device, it's really about the cost and a good example.
• 17:19
  Speaker 3: I've seen there a lot of these security cameras you see now in the market.
• 17:23
  Speaker 3: They send the picture to the cloud and there happens the detection maybe movement detection locally to save some battery.
• 17:30
  Speaker 3: And so I had these cameras and you have a monthly fee you have to pay, because they have to pay the data traffic as well.
• 17:41
  Speaker 3: We don't forget that a data center has to do that and so I switched to a solution where people say, oh, you have it on-premise, you know, over the network and all these cameras have edge AI capabilities I have.
• 17:52
  Speaker 3: But then I looked in the power consumption it takes me at home to have a PoE switch and the server with the hard drive and what it takes me for these cameras over the year, and I realized I'm ending up actually paying more money than the subscription for, okay, germany has a very high electricity cost, right, and this is something also where you can shift the cost to the customer.
• 18:20
  Speaker 3: And so therefore, I will think also that more and more application as to they can run in the edge people economically will run them there and be seen with some like Qualcomm and DeepX bringing out chips which can run even large language models and then you don't have to pay the data center.
• 18:37
  Speaker 3: You don't have to pay the traffic, you're dealing just with metadata, and so I've seen that right and in the end the person is happy not to pay a yearly subscription, but you pay the electricity.
• 18:51
  Speaker 1: But there's a total cost of ownership consideration that needs to be calculated there, muni, I want to bring you in there then, changing tack a little bit, and talk about standards and the role that standards play in security.
• 19:04
  Speaker 1: What standards are at play right now, what's emerging and what role are they going to play in helping the security agenda?
• 19:17
  Speaker 2: I think there is actually a point to make with the previous question related on that, on standards, for instance, for defenses.
• 19:21
  Speaker 2: So currently we are missing defenses that will be efficient for many use cases, realistic use cases Usually the defenses that are developed are whether there are really many use cases.
• 19:33
  Speaker 2: Realistic use cases Usually the defenses that are developed are whether there are really many of them.
• 19:36
  Speaker 2: So usually they have gotten after a couple of days or attacked just a week afterwards and then made inefficient or most of the cases, as I was saying, they are not useful.
• 19:51
  Speaker 2: So that's why it's a bit if we have standards, if we have regulations that make metrics.
• 19:58
  Speaker 2: We discussed actually quite a lot about metrics today mostly, for instance, even with Danilo's workshop.
• 20:06
  Speaker 2: So metrics is really important in that sense that these standards, more specifically, will be a make us able to design some command methodology between all the the main customers within worldwide, so that everyone respects the same inputs, the same constraints when they are designing their model, so that this is the really the main advantage I see on standards and why it's really important to have something that will be makes things easier for security.
• 20:41
  Speaker 2: And you are asking, patrick, for just a couple of examples.
• 20:45
  Speaker 2: Maybe there are quite some initiatives, mostly and Ali mentioned the CRA and the AI Act so this, for instance, are just regulations from Europe for the for the EU, I act, its regulations from Europe to define the criticality of a given application.
• 21:05
  Speaker 2: So, for instance, if we say, if we come back to fashion recognition and we say it's really critical, we need to have this kind of standard defined and for that, so the EU for instance, there is ETSI that is working on that- Okay.
• 21:20
  Speaker 2: With a working group and also the NIST in the US.
• 21:23
  Speaker 2: In the US, yeah, developing things related to that.
• 21:26
  Speaker 2: So, yeah, I think it's really important.
• 21:28
  Speaker 2: I wouldn't say big fan we need to be big fans of that but I think it's important to do.
• 21:35
  Speaker 2: But we still should keep some flexibility.
• 21:39
  Speaker 1: They have a role to play.
• 21:40
  Speaker 2: They have a role to play, but they should be flexible.
• 21:43
  Speaker 2: As we were saying from the beginning that our AI is fast evolving, security is really fast evolving.
• 21:49
  Speaker 2: So if the standards are too struck, too rigid for some specific functions security is really fast evolving.
• 21:54
  Speaker 2: So if the standards are too struck, too rigid for some specific functions, it will be impossible for us to make them actually apply them on the field.
• 22:00
  Speaker 2: So that's why there is this our initiatives that are ongoing.
• 22:05
  Speaker 2: Unfortunately, for now, it's still work ongoing, but yeah, hopefully we can reach something in the near future.
• 22:12
  Speaker 1: It's a great point, moon.
• 22:13
  Speaker 1: Yet Everything's evolving so fast right now that writing standards for this stuff must be a complete nightmare.
• 22:19
  Speaker 1: Matias, did you want to come in there as well?
• 22:21
  Speaker 3: Yeah, so I think again dividing it into the technical standard, securities, I'm pretty confident it's very globally.
• 22:29
  Speaker 3: Maybe it compared to when it's about privacy and what can you ethically do.
• 22:34
  Speaker 3: And on the technical side, I don't think it's so much a problem to have worldwide standard.
• 22:39
  Speaker 3: Maybe that's because these standards often involve engineers are involved in that and the other ones are politicians involved.
• 22:47
  Speaker 3: It's a dangerous mix.
• 22:48
  Speaker 1: And so I think it's a dangerous mix.
• 22:50
  Speaker 3: And therefore I think there's more challenges and you see that how European companies complain about regulations already that they cannot do certain usage of data and the United States is the opposite.
• 23:01
  Speaker 3: So I think that is probably the more complex topic to find a common path, also if you want to have built a better global product.
• 23:10
  Speaker 3: Absolutely compared to data data security.
• 23:13
  Speaker 3: I think there the engineers are basically speak a common language yeah, it's more, it's better established.
• 23:19
  Speaker 1: I guess I wanted to talk about a topic that's really talked about a huge amount it's very topical at the moment which is agentic AI and I guess specifically around running agents on endpoint devices and in a distributed way.
• 23:34
  Speaker 1: Ali, maybe I could bring you in here.
• 23:37
  Speaker 1: What are some of the considerations there around trusting these agents and managing them?
• 23:43
  Speaker 4: You shouldn't trust them, but anyway let me start by level setting, maybe around AI agents and agentic AI, so they're very distinct around AI agents and the Gentic AI.
• 23:50
  Speaker 4: So they're very distinct.
• 23:51
  Speaker 4: They're related but distinct terms.
• 23:54
  Speaker 4: Ai agents are pieces of software, pieces of applications that actually can take some task completion or have a task to complete and might have some level of autonomy in getting that task done with an AI model as part of it.
• 24:09
  Speaker 4: And this is like evolution with generative AI, like from LLMs, from getting into vision language models and vision language action models and agents that relate to it.
• 24:22
  Speaker 4: Then agentic AI is really an orchestration of these.
• 24:25
  Speaker 4: You know, sometimes multiple agents and more of a framework to create more autonomy around task completion and how they can run into each other.
• 24:36
  Speaker 4: You start getting into now one of the protocols or infrastructures that is somewhat of a de facto standard around.
• 24:44
  Speaker 4: This is something that Anthropic released called the Model Context Protocol, mcp, and it's interesting because it's I won't say it's a hype it's extremely powerful.
• 24:54
  Speaker 4: It's able to orchestrate and even have public MCP servers where you can run local models and local capabilities but also source from public sites like YouTube, like Salesforce from an organization's perspective, or even GitHub and Hugging Face, pull in data and, from a security perspective, what you see with MCP is, even though there is provisions for a token based authorization, as agents pass information from each other or access certain services, it is not mandatory to have authorization.
• 25:28
  Speaker 4: So suddenly you can get into having set up an agentic AI setup that has no security around it accessing public sources, accessing private sources, potentially accessing things that have been manipulated very easily in a public domain.
• 25:42
  Speaker 4: So there's a lot of layers of, again, vulnerability that this creates and, because it's such a hot domain, people are using it without thinking about the security.
• 25:54
  Speaker 4: It's kind of like it always happens where security sometimes is an afterthought and people are like, oh, wait a second, like we've really opened ourselves up.
• 26:00
  Speaker 4: Now we got to close it back down.
• 26:01
  Speaker 4: And you might not know that you've, by the time you figured out your vulnerabilities, that actually your IDs have proliferated, your authorization pieces have proliferated.
• 26:13
  Speaker 4: So you need to structure any of this agentic AI stuff with a lot of the fundamental security concepts that already exist and have existed even with HTTP, you know, ethernet-based protocols, et cetera, in terms of you've got to limit the accesses you're granting, the accesses you're granting, so the agent coming in only has certain authorization to be able to complete the task that it's given.
• 26:37
  Speaker 4: You have to work on a limited privilege principle or a restricted privilege principle so it only accesses resources that it needs to complete its task and nothing else.
• 26:50
  Speaker 4: You're sandboxing these agents so that they're contained.
• 26:54
  Speaker 4: So all these have to be put in place, and these are concepts that exist.
• 26:58
  Speaker 4: We just need to be careful as something new pops up in the AI world, like agentic AI, and next buzzword is physical AI, where it starts getting into moving agents.
• 27:08
  Speaker 4: We need to be careful in leveraging the technology that exists already in protecting these systems.
• 27:16
  Speaker 1: Okay, thanks, Ali, and I guess those agents are also accessing, as you say, resources, information, data, etc.
• 27:21
  Speaker 1: Matthias, we need to keep that in mind too.
• 27:24
  Speaker 3: Maybe I always like to talk a little bit in pictures for people who are not so technical.
• 27:29
  Speaker 3: So, to sum up, what Ali said is you have an agent and maybe you as a company have control over this, but there's external resources and it feels for me like I'm in a call with a service agent and you turn on your computer and then you see in that, whatever call it at Google zoom, naming them all, you see a lot of people without a picture and then you said, oh, who's somewhere else on the call?
• 27:50
  Speaker 3: Yeah, don't mind our intern, we train him how to do customer calls and that's a little bit this, especially if it then reaches, reaches out to other services providers and it creates them.
• 28:03
  Speaker 3: We talked about the security risk before this week.
• 28:08
  Speaker 3: You know what is.
• 28:08
  Speaker 3: If I have an idea and I want to check, oh, does someone already has a patent on this?
• 28:13
  Speaker 3: And I go on a search engine and I said, look for existing patent for this, then the one who earns the search engine says, oh, that's a great idea.
• 28:22
  Speaker 3: There's no pattern for this.
• 28:25
  Speaker 3: Okay, yeah, I think that's kind of the same thing that there the security risks is really in a managing and knowing the, the sources and the other agents you're basically connecting to.
• 28:39
  Speaker 1: So the very process of searching for a particular patent is alerting someone, some other party, to the fact that this is an area of innovation and maybe worth looking into.
• 28:48
  Speaker 1: Wow, okay, If you don't mind, could I jump in there?
• 28:52
  Speaker 1: Yeah?
• 28:53
  Speaker 2: yeah, yeah, I think.
• 28:55
  Speaker 2: Yes, I agree with Matthias.
• 28:56
  Speaker 2: I think he took us a bit, even a further way, you know, with a bigger view of we're getting a bit farther than IGN-GKI, maybe more applying to LLMs and GDI, and I wanted to make a point that in that case too, there are a lot of vulnerabilities on which we are today honestly not doing.
• 29:16
  Speaker 2: There are a lot of works, for sure, it's the beginning of that, but the numbers are there and we should take care of those, just like before algorithmic attacks with prompted injection resistance that could lead, you see, two very critical issues.
• 29:31
  Speaker 1: that's why, yeah, yeah, okay, okay, great point.
• 29:34
  Speaker 1: Look, we're gonna go to the audience now shortly, but before we do, I wanted to get a perspective from each of you on where this is all headed.
• 29:42
  Speaker 1: So I'm asking you to kind of look into the future here and just what's coming down the tracks in terms of security or in terms of other developments that could impact the security agenda.
• 29:52
  Speaker 1: Munia, maybe I could start with you Sure.
• 29:56
  Speaker 2: So for the future, that's, yeah, great question.
• 29:59
  Speaker 2: Well, I think that the main topic that's really interesting for us for the future, mostly as an IP provider, is really the hardware To be able to leverage the right hardware to be able to run all these workloads on the edge.
• 30:20
  Speaker 2: Hardware to be able to run all these workloads on the edge.
• 30:23
  Speaker 2: So for today, for instance, during this event, we discussed a couple of times about the neural art with multiple demos.
• 30:29
  Speaker 2: I think this kind of thing.
• 30:31
  Speaker 2: We have a very big gap that we are trying to bridge.
• 30:35
  Speaker 2: And there is the things that I think will be really important for the future, for instance, to respect privacy issues, to have them on the edge and why not have large language models inferred directly on the edge I think that's one of the major process are really important for us.
• 30:56
  Speaker 2: And maybe the second point I would make for the future was maybe to change the way we are working, actually the way we're working.
• 31:04
  Speaker 2: Yeah, the way we are working, and mostly for security purposes.
• 31:08
  Speaker 2: In security, it's always the very last point that's always coming, and maybe for that so Pete began introduced actually the event by asking how can we take AI to the next level, and for that we had very interesting topics all week, but I think security should be one of them.
• 31:29
  Speaker 2: There is always, yes, compression, quantization very important topics for performance-related efficiency, but if we don't take care of security as soon as the model is being trained, for instance, with adversarial training defenses that are efficient, it will not work or it will be too vulnerable for real implementation.
• 31:49
  Speaker 1: So it's about the fundamental principle think about security from the start, don't try to design it in at the end.
• 31:53
  Speaker 1: Okay, absolutely, matthias.
• 31:55
  Speaker 1: Looking to the future, what's coming?
• 31:57
  Speaker 1: I?
• 31:57
  Speaker 3: think when you look at you just told me that HAI foundation was more tiny ML at the beginning and I think it's the right thing to say just Edge AI, because we see the big AI also moving there large language models so that things can be unconnected, but also the other commercial power aspects or whatever, and latency and all of that which benefits the HAI.
• 32:21
  Speaker 3: So we will see that large language models coming there and then like also that it changes our life.
• 32:28
  Speaker 3: Just by the way, the tie in my picture here is AI-generated.
• 32:33
  Speaker 3: I didn't have the picture and someone helped me send the picture and I thought it's cool for a conference like this to sneak some AI in it, and original it was just without a tie, and so it changes the work now.
• 32:49
  Speaker 3: Already.
• 32:49
  Speaker 3: And then one thing I think particular will be important, like can I trust the models?
• 32:54
  Speaker 3: Who is who is?
• 32:55
  Speaker 3: It's kind of like statistics you know, don't believe statistics you have not falsified by yourself and so who is training what is?
• 33:04
  Speaker 3: Who defines what the truth is?
• 33:05
  Speaker 3: And we see that already.
• 33:07
  Speaker 3: I guess they're hiding a political.
• 33:08
  Speaker 3: You know where you ask this and that's going to be a challenge how you manage that, you know because what's the source of?
• 33:18
  Speaker 1: what's the source of this data?
• 33:19
  Speaker 1: What's the source of this and how can I trust it?
• 33:21
  Speaker 1: Ali same question to you in terms of looking to the future.
• 33:24
  Speaker 4: So a couple of points on touching back.
• 33:27
  Speaker 4: So I think, from security being a fundamental topic to consider as we're deploying systems, to actually make these systems really deployable and maintainable out in the field, security becomes extremely critical.
• 33:39
  Speaker 4: You need to still protect the overall system end-to-end With AR.
• 33:44
  Speaker 4: Without AI, I don't want to put myself out of a job as a marketer and a guy that talks about a lot of AI stuff, but it is still very fundamental in what we do, for any application still applies to AI-based applications where the model is part of the overall application.
• 33:57
  Speaker 4: And what changes is?
• 34:00
  Speaker 4: Ai, especially with generative AI, kind of creates additional angles of being able to attack the system, like, instead of fault injection now it becomes prompt injections of trying to decipher what the system and try to get to the keys, or the keys to the kingdom, in a sense, on these systems.
• 34:17
  Speaker 4: So protecting it end-to-end, finding ways of not just protecting the AI model but protecting the overall system On the AI model.
• 34:27
  Speaker 4: Now that's another asset that companies are spending time and resources, which means that it has an inherent value to it.
• 34:35
  Speaker 4: So you've got to protect the model itself as well from being copied or potentially reused or maliciously being modified for various purposes, things that come into play.
• 34:48
  Speaker 4: We talked about standards.
• 34:49
  Speaker 4: I'm not a big fan of standards because I sometimes feel that it limits innovation.
• 34:54
  Speaker 4: But one area where standards make a lot of sense is the security and the keys.
• 34:58
  Speaker 4: I mean we've been successful in creating protection with HEC, with RSA, public key encryption, and all of these are getting now to a stage where you know they're becoming vulnerable as we start talking about a post-quantum age with quantum computers, you know being seen as a way to break all these keys that we have that are fundamental protection that we have.
• 35:22
  Speaker 4: So PQC is an extremely critical aspect and sometimes people say well, you know quantum computers, they're not here yet.
• 35:29
  Speaker 4: So why are we worrying about that now?
• 35:31
  Speaker 4: Today, like the biggest quantum computer is about, ibm has it's like 1100 qubits, so about just over a thousand qubits capability.
• 35:39
  Speaker 4: These things I don't know if you've seen pictures don't look very stable.
• 35:42
  Speaker 4: They need to be super cool.
• 35:42
  Speaker 4: They look really fragile, and they are.
• 35:43
  Speaker 4: They break down very often.
• 35:46
  Speaker 4: They can't compute for very long periods of time, so why are we worrying about that?
• 35:50
  Speaker 4: Now, the overall quantum computer thing like to break an RSA key it's thought that you would need about a million qubits to break it down.
• 36:00
  Speaker 4: So we're at 1,000, getting to a million.
• 36:01
  Speaker 4: Estimates are about that's in a five-year span or maybe 10-year span that you'd get there.
• 36:08
  Speaker 4: But NXP is one of the companies that is working with NIST, that we have our algorithms that have been accepted and now we're already putting PQC into devices, because attackers harvest today and might crack it later.
• 36:22
  Speaker 4: So you need to be, or these devices that we're putting out into the system, into deployment, will be there for five years, 10 years, 15 years.
• 36:30
  Speaker 4: If you talk about microcontrollers and apps, processors in city infrastructures and electrical grids, for example, these things are sometimes 20, 30 years old.
• 36:39
  Speaker 4: So if you don't have the protection already in place, the attackers eventually get the technology and you're vulnerable.
• 36:46
  Speaker 4: So all of these need to be built early and you start protecting early.
• 36:51
  Speaker 4: It's a cat and mouse game, so there's always additional things that need to be put in place and you add those as things come up.
• 37:00
  Speaker 1: Wow, okay, great, ali, thank you.
• 37:02
  Speaker 1: So let's go to the audience and see what questions you have on a Friday afternoon.
• 37:07
  Speaker 1: Put the panel to the test.
• 37:09
  Speaker 1: Any questions, no questions?
• 37:20
  Speaker 1: Well, let me ask one more question then, as we wrap up.
• 37:24
  Speaker 1: I just want to get your perspective on if things do go wrong in this kind of new distributed world of AI on our endpoint devices.
• 37:35
  Speaker 1: How do we deal with that?
• 37:37
  Speaker 1: Are there differences in the way we deal with that and the way we contain that?
• 37:39
  Speaker 1: How do we deal with that?
• 37:40
  Speaker 1: Are there differences in the way we deal with that and the way we contain that?
• 37:43
  Speaker 1: Matthias, do you have a perspective on that?
• 37:48
  Speaker 1: Yeah, maybe you catch me on the wrong foot there, but I don't want to put you on the spot, ali or Munia.
• 37:57
  Speaker 3: Yeah, give me an example, a discrete example.
• 37:59
  Speaker 3: Well, I'm kind of considering this situation?
• 38:01
  Speaker 1: Yeah, give me an example, a discrete example.
• 38:02
  Speaker 1: Well, I'm kind of considering this situation where we've got some sort of penetration, we've had some sort of a security incident that's happened.
• 38:08
  Speaker 1: There are ways of dealing with that in the centralized world.
• 38:12
  Speaker 1: I'm just wondering how do we deal with that in a distributed world?
• 38:15
  Speaker 3: Yeah, okay, then.
• 38:17
  Speaker 3: this is quite why it's so important that you have the security in place, like the root of trust, that the device can really say it comes from the right authority because if someone basically preaches the main central point where you Get the updates from, then of course that can be fixed, right, it's gonna be for a short time and then you at least have to make sure you own, you can basically sign the correct, basically model and push it back to the devices and it can authenticate the source to do that.
• 38:52
  Speaker 3: Of course, if someone would preach the, the keys, that's another, another topic, right, but this is why you want to have the secure boot and have a verified image that you can, the device can verify where it comes for from, and so then you have them back, the ownership of your data center or whatever, that you can basically recover this and they don't have permanently the ownership of the edge.
• 39:17
  Speaker 2: Okay, okay, I might add something to what myia just said, I think there is also yes, this of course that is important between the server and the edge devices, but also the security on the edge device itself that is really essential and on which we should take care of.
• 39:34
  Speaker 2: Actually, for instance, what I was mentioning why not?
• 39:39
  Speaker 2: We can think about local devices that would be able to have intrinsic security for the models, so that we cannot perform data poisoning, attacks or things like that.
• 39:51
  Speaker 2: So this is the kind of thing against which so it's on two levels on the federated service workload, but also on the edge device, and I think that, depending on the criticality again, for instance, of the application, the response of the user would be quite different, depending on whether, yes, whether it's just not notification, for instance, of I don't know the level of beans in coffee maker, for instance.
• 40:21
  Speaker 1: Sure, so it's about the risk of the scenario, exactly.
• 40:25
  Speaker 2: If it's a very, yes, critical scenario, for instance, autonomous driving, in that case we can think about yes, securities, like whether shutting off the service or putting redundancies or things like that, so that we are sure that the actual response of the device is the, the one we were hoping for and the one we were mostly waiting for.
• 40:48
  Speaker 1: Which is a very nice tie-in back to where you've started saying do we put our trust of our life in these, in these models?
• 40:55
  Speaker 1: Munia, matthias Ali, I just want to say thank you so much for your participation today.
• 41:00
  Speaker 1: Great insights, thank you, great insights.
• 41:05
  Speaker 2: Thank you, thanks everyone, thank you.