Latest / EDGE AI POD / How to simplify and securely maintain up-to-date AI Models in the Edge

Transcript

How to simplify and securely maintain up-to-date AI Models in the Edge · EDGE AI POD

• 0:07
  SPEAKER_00: Okay, thank you for the extra time I have.
• 0:11
  SPEAKER_00: So, quick introduction about me.
• 0:13
  SPEAKER_00: I work as a consultant for embedded systems.
• 0:15
  SPEAKER_00: I worked for a lot of companies in this industry, particularly in security, IoT, and everything which has to do with embedded Linux computer modules.
• 0:25
  SPEAKER_00: And yeah, after all this time for different vendors, I started basically my own consultant business.
• 0:32
  SPEAKER_00: And I help companies like Thistle and Grin and DeepX accelerators to promote their products and support their customers on the European market, and I help European companies to find a place and partners in the United States.
• 0:49
  SPEAKER_00: And today we're talking about what we do with Thistle security.
• 0:54
  SPEAKER_00: And security is always seen as a boring topic and a must-have, and I want to look at this from the fun aspect.
• 1:02
  SPEAKER_00: And so here is something edge security is like flossing.
• 1:05
  SPEAKER_00: Everyone agrees that's important, but the most devices still skip it and hope no one noticed until it hurts.
• 1:13
  SPEAKER_00: And the best thing is I want to give a real life example from the conference.
• 1:18
  SPEAKER_00: Who has seen that in the conference here?
• 1:22
  SPEAKER_00: These this sign.
• 1:23
  SPEAKER_00: Can someone guess when that password has been changed the last time?
• 1:30
  SPEAKER_00: Maybe you, Christian?
• 1:32
  unknown: 22.
• 1:33
  SPEAKER_00: 22.
• 1:33
  SPEAKER_00: Okay.
• 1:34
  SPEAKER_00: So at least after that video here is on YouTube, they have to change that to let's say 2025.
• 1:41
  SPEAKER_00: So and but I don't want to look at this uh like from the standpoint it's a necessity.
• 1:49
  SPEAKER_00: There's also benefits with security.
• 1:51
  SPEAKER_00: And you have to look is we have billions of connected devices, they are growing.
• 1:57
  SPEAKER_00: And what we want to do is um what we see is we have even on edge devices now LMs.
• 2:03
  SPEAKER_00: We heard it at the Qualcomm presentation, deep access, an accelerator who can run large language models.
• 2:10
  SPEAKER_00: And you know, we want to uh we have uh attacks mainly looking here for your IP, right?
• 2:16
  SPEAKER_00: If you look especially at embedded Linux processors, it's not any more like compiled binary code on microcontrollers, which is maybe harder to make sense out of it and um and cross-compile that, but you have often things in plain text or things like Python, which is basically code, which is a documentation at the same time.
• 2:36
  SPEAKER_00: And then we have this topic of regulations, you know, in Europe now, um CRA or NIST2, and we have basically the commercial brand impact.
• 2:46
  SPEAKER_00: And so focusing on security basically can avoid regular pain.
• 2:50
  SPEAKER_00: The problem with security is whoever worked with this and with encryption, it's a little bit in the software, like for some people, plaque magic mathematically, uh similar like in hardware antenna design seems for some people like this.
• 3:05
  SPEAKER_00: And it's already hard to do if you just want to do it on a demo setup.
• 3:09
  SPEAKER_00: But it really gets a pain if you want to scale this, because usually when you scale, you also have different versions of your product, and with different processors, you have to basically manage.
• 3:22
  SPEAKER_00: So, what are the key threats to edge AI devices?
• 3:27
  SPEAKER_00: We talked about that the the the theft of your IP and your model, you have to imagine you invest money in training that for your particular market, and you don't want that anyone else has a benefit of that or tempers with it.
• 3:41
  SPEAKER_00: Then, of course, over-the-air updates, someone could hijack that and you want to avoid that someone deploys malicious code onto your device.
• 3:49
  SPEAKER_00: Um, you know, someone could extract credentials.
• 3:52
  SPEAKER_00: We heard about that people hacked into data centers because someone retrieved the password on a room thermostat from an air conditioning system which stored the password just in plain text and the flash memory.
• 4:05
  SPEAKER_00: And um and you know, people could downplay, uh downgrade your device, cause a lot of brand damage, or at least like a bad user um um experience.
• 4:16
  SPEAKER_00: So, what are basically, therefore, the core pillars of of security on edge devices?
• 4:22
  SPEAKER_00: It's like prevent an attack rather than notify that an attack has happened.
• 4:27
  SPEAKER_00: Then the secure boot, and I think it's a concept I will talk a little bit about uh because not everyone is aware what basically secure boot uh means.
• 4:37
  SPEAKER_00: And that makes it ensures that only trusted code runs on your device.
• 4:42
  SPEAKER_00: And there's different measures for that.
• 4:44
  SPEAKER_00: And then, of course, the OTA, everyone heard about that.
• 4:47
  SPEAKER_00: We want to make sure that what is transferred is encrypted and also lives in an encrypted way in in storage so that it's safe even if the storage is compromised.
• 4:57
  SPEAKER_00: And we want to have basically assigned firmware update so that the device can really verify that it comes from which source it comes, and that's that's an integer update.
• 5:07
  SPEAKER_00: And then we see a lot of this, like um we talked about dental flossing, so the root of trust has nothing to do with a root treatment.
• 5:14
  SPEAKER_00: It prevents that you one day basically need a root treatment, and that is often done to special key storage chips like secure element or trusted platform controllers.
• 5:26
  SPEAKER_00: And this is something I want to make aware of.
• 5:29
  SPEAKER_00: Security is not the pure software implementation thing for edge devices, it's a combination of hardware and basically software devices, and these um elements are specially hardened to um store uh store your thing.
• 5:44
  SPEAKER_00: And then, of course, secure data at storage, that means that even if someone would get access to the hard drive for the storage, the data is basically encrypted there.
• 5:54
  SPEAKER_00: So, what's the best practice to ensure this?
• 5:57
  SPEAKER_00: Of course, find and verify basically even the bootloaders.
• 6:02
  SPEAKER_00: Um, and that's the topic of secure boot.
• 6:04
  SPEAKER_00: You know, when the processor or microcontroller um uh boots the image, and you know, with higher-end MPU processors, it's ARM trust zone so that the bootloader um is verified, and then the bootloader verifies the image, and a root of trust.
• 6:22
  SPEAKER_00: Um and basically there's a few options for these so-called hardware secure elements.
• 6:29
  SPEAKER_00: And that is, for example, here in our in our sample downstairs we have a table that we use the Infini Obtiga, but back in the days when I worked at Atmel, we had secure elements, ST has secure elements, and that's special chips which hardens against all known attacks that someone could retrieve the the root and your private key from basically device.
• 6:52
  SPEAKER_00: And they offer hardware acceleration for security functions.
• 6:56
  SPEAKER_00: But some higher-end processors and all the microcontrollers have already secure boot built in.
• 7:02
  SPEAKER_00: But here we um basically that is part then of the chip.
• 7:06
  SPEAKER_00: It's different implemented across all the vendors, and you would have basically a place to securely store basically the key and then the processor or they had the fabric to verify um this.
• 7:20
  SPEAKER_00: And we support, or you if you want to do something with security, you have to support a lot of devices, and basically we support microcontrollers, um the NXP application processors, uh the PSOC, and also Qualcomm.
• 7:36
  SPEAKER_00: But whatever you bring, and if it's not already have been done, we uh implement basically the secure boot or verified boot for you onto your device.
• 7:48
  SPEAKER_00: Uh okay, and um here's the thing, you know, um best practice for secure update models.
• 7:55
  SPEAKER_00: Uh you want to basically update models.
• 7:58
  SPEAKER_00: This is kind of um when I started with microcontrollers, there they were not connected, and you programmed it, and over the lifetime they didn't change.
• 8:08
  SPEAKER_00: Now, with certain security requirements, regulatory requirements, you have to ensure that the product is secure over the lifetime.
• 8:16
  SPEAKER_00: And if it's a connected device, you basically have to maintain the firmware.
• 8:20
  SPEAKER_00: But on edge AI devices, we know we continuously have more data, retrain our models.
• 8:26
  SPEAKER_00: So also for the use case enhancement for the product, you want to update your models.
• 8:32
  SPEAKER_00: Um then you you want to make sure that it's end-to-end, basically encrypted, uh, the update is signed.
• 8:39
  SPEAKER_00: Um the you want to also have like a fail based uh fail-safe rollback feature, which is very important.
• 8:46
  SPEAKER_00: Could you imagine something would fail within a software update, and then you have a prick, a paper weight.
• 8:53
  SPEAKER_00: Um and and yeah, if you work with security, you will create, at least in the development process, a lot of paper weights until you uh know um how to do it right.
• 9:04
  SPEAKER_00: And um yes, and there is the in-house solution.
• 9:09
  SPEAKER_00: You know, engineers always like to do everything and uh and and learn new things, but it's always a question at the at the costs.
• 9:20
  SPEAKER_00: Uh the costs uh imagine if you are a small-sized company and you have to employ five people and spend 18 months to develop that.
• 9:27
  SPEAKER_00: Um, here using Thistle, you have like thousands of customers who you basically divide the costs on building the solution.
• 9:37
  SPEAKER_00: Okay, and that is basically um uh the the problem, a little bit of these in-house solution from companies, right?
• 9:45
  SPEAKER_00: Um it's expensive.
• 9:47
  SPEAKER_00: You have to um I always say the cost shares by you as a company rather than share across all the company of the thistle as basically this your secure OTA provider.
• 10:01
  SPEAKER_00: Um okay, and this is where the platform approach makes basically sense because um some someone in else maintains that for you, takes care of the security.
• 10:11
  SPEAKER_00: We work with many hardware vendors, so they are added.
• 10:14
  SPEAKER_00: So if you you basically change the uh the product over the lifetime, um you you you know that it will work on other devices.
• 10:24
  SPEAKER_00: So this is kind of a really good presentation here, um versus uh do it yourself.
• 10:31
  SPEAKER_00: And a lot there's a lot of OTA cloud solutions, but I'm I I think basically Thistle, we are like really focused from the ground up on security, not just of making an OTA update.
• 10:43
  SPEAKER_00: And if you look basically, we have the secure boot, we have the encrypted over-the-air update, we have hardware acceleration through the crypto elements, or what the processor basically provides, and basically the readiness for uh certification and industry requirements.
• 10:58
  SPEAKER_00: And we can optimize it for speed um um since since you know we can put more uh time into it um than like um a single company could could do resource-wise.
• 11:12
  SPEAKER_00: So and now if we want to update basically AI models, um so we have security risk, complex deployment, data integrity, compliance and trust.
• 11:22
  SPEAKER_00: And this is what we basically provide um all in our solution.
• 11:29
  SPEAKER_00: And um it's really hard to do well.
• 11:32
  SPEAKER_00: If you've been working on this, you will know this.
• 11:34
  SPEAKER_00: And I was very surprised how easy I could implement that into the product with um the infrastructure basically Thistle provides.
• 11:44
  SPEAKER_00: And here are like a few numbers, and I think you know that you have to already have a couple of people to get it done in that time frame.
• 11:52
  SPEAKER_00: And um it's usually not scalable because the customer develops for one product, we build it for multiple platforms.
• 12:00
  SPEAKER_00: Okay, that is basically why um the solution has been developed.
• 12:05
  SPEAKER_00: Here's a nice graphic from the marketing team, you know.
• 12:08
  SPEAKER_00: But really, I don't know if the line is here or here, but it's different than the graph, which gets more complex with the amount of devices.
• 12:18
  SPEAKER_00: So uh the physical secure end-to-end uh platform, so it's agnostic to the hardware, it's easy to deploy, and and we have the functionalities required basically by R by the RCA uh compliance.
• 12:36
  SPEAKER_00: And this is again, people think always only over the air update, but we talk about the secure boot.
• 12:42
  SPEAKER_00: Again, the functionality that the device itself, or with the help of the crypto element, can verify that the update it's receiving is really coming from the right source, and it's not uh someone else taking ownership of the device.
• 12:58
  SPEAKER_00: And then you have the protection of data at rest.
• 13:01
  SPEAKER_00: That means even if someone compromises the hard drive, the storage, the data there is encrypted.
• 13:07
  SPEAKER_00: Then uh secure software update, integrity verification, logging, which is required by that, and um and that is also embedded basically in the platform.
• 13:18
  SPEAKER_00: So if we have products, usually we have something like train a model, then you sign it, upload it to device, you collect new data, you read retrain, and um that is basically given by the thistle platform.
• 13:34
  SPEAKER_00: So and downstairs I have um uh an update, so I will not get too much in the technical details, but I want to talk about these three subjects: the secure boot factor, you know, the secure data, and the secure over-the-air uh update feature.
• 13:51
  SPEAKER_00: So I just have here a picture of um a device, and you see you have somehow the operating system, and there we have these thistle update client.
• 14:01
  SPEAKER_00: Very simple.
• 14:02
  SPEAKER_00: If it's embedded Linux system, it's a little bit easier than um than for a microcontroller.
• 14:07
  SPEAKER_00: You could install just took the thistle update client and it takes care of all the encryption, the security, and the contact to the platform.
• 14:17
  SPEAKER_00: And then you have somehow on the device usually a bootloader mechanism on an application processor, U-boot, for example, in Linux.
• 14:25
  SPEAKER_00: And either you have um you have um inside programmable fuses like ARM trust zone, but there is also if you don't have that, or if it's too complicated for you, because you have to imagine you have to somehow in the production program and flash the devices.
• 14:44
  SPEAKER_00: If something goes wrong, you have a paperweight.
• 14:46
  SPEAKER_00: So you have to manage also um um programming and fusing those devices in your production run.
• 14:54
  SPEAKER_00: But it could be also basically um a secure element.
• 14:58
  SPEAKER_00: So therefore we differentiate between secure boot, which uses the uh embedded functionality in the chip, or verified boot, which works with an external crypto element.
• 15:11
  SPEAKER_00: And um and if we compare these uh these both, the thistle verified boot, which basically would work with every processor, and the security is put into a secure element like um the Nfinian uh Optica, um there it's a basically relatively easy to integrate.
• 15:34
  SPEAKER_00: But if you have a processor with secure boot, we implement that for you, and this is basically uh fixed the by the manufacturer at stage.
• 15:45
  SPEAKER_00: Um this would be easier to integrate, but you can also have CRA compliance with the verified boot if your device doesn't have the secure boot feature.
• 15:58
  SPEAKER_00: And um, the secure boot, of course, is for processors who have that already um embedded.
• 16:07
  SPEAKER_00: Okay, so the security features we talked about, and I want to just repeat that here it's uh the cryptographic key generation, uh defending against attacks, support multiple security permequash, accelerated uh security generation and cryptographic functions, either in the host processor or via the coprocessor secure element.
• 16:36
  SPEAKER_00: And here you would see the um the um the process with the platform.
• 16:42
  SPEAKER_00: So we have the thistle platform and the control center.
• 16:47
  SPEAKER_00: You would have the client uh uh signing basically your image or your artifact, that's very important.
• 16:55
  SPEAKER_00: If you want to update an HAI model, you don't want to update um a five gigabyte um um image for embedded Linux, you only want to uh update a few bytes.
• 17:07
  SPEAKER_00: And on the device side, you run the thistle update client, which uses the root of trust for an external uh secure element or the internal function.
• 17:18
  SPEAKER_00: And people always say, yeah, what if someone compromises the platform?
• 17:22
  SPEAKER_00: And therefore, um I have uh uh another picture, so that would be the same for the update with the fallback to the previous image.
• 17:31
  SPEAKER_00: But again, someone asks, yeah, what if someone compromises the platform?
• 17:36
  SPEAKER_00: This picture says it all.
• 17:38
  SPEAKER_00: You device manufacturer, it's an end-of-end uh trust.
• 17:43
  SPEAKER_00: You basically sign your firmware image or artifact, and this will provides the infrastructure to do this.
• 17:51
  SPEAKER_00: If you decide to move away from Thistle as a security platform provider, you could do so, you know, because you're the owner of the root of trust to do so.
• 18:02
  SPEAKER_00: Of course, we have um a control center.
• 18:04
  SPEAKER_00: This can be also branded for your company, but a lot of company uses via an API in the background to monitor uh the devices.
• 18:15
  SPEAKER_00: Okay, and here you see it basically saves a lot of time of developing this.
• 18:22
  SPEAKER_00: It took us quite some time and resources to develop this.
• 18:25
  SPEAKER_00: And if you look in the embedded market, there's a lot of these companies who have uh just a few couple of hundred devices a year, but the devices are big CNC machines which cost half a million dollars, right?
• 18:38
  SPEAKER_00: And so for this amount of devices, you know, it's very expensive to build build that up.
• 18:44
  SPEAKER_00: Um, but you have many of those customers.
• 18:47
  SPEAKER_00: So it's many customers with a uh small m a smaller amount of devices compared to millions of devices, you know, for like smartphones or gadgets.
• 18:59
  SPEAKER_00: Okay, this is a partner network and it's uh from from us, and so we work with different silicon vendors which we already support, and also computer module manufacturers uh where this runs.
• 19:13
  SPEAKER_00: Um and if you have other vendors or it's needed, it's usually part of our work when you purchase these um uh the uh the services from us to implement it into your existing product.
• 19:27
  SPEAKER_00: So um so what are the final takeaways um to sum it up?
• 19:33
  SPEAKER_00: We talked multiple times about this.
• 19:35
  SPEAKER_00: You know, we have to have a security uh built in in order to prevent from attacks, print uh destroying our print, creating paper weights, um someone stealing our intellectual property.
• 19:49
  SPEAKER_00: And it's not easy to do with that platform.
• 19:53
  SPEAKER_00: It's already done, it's hardware agnostic, and you can implement it or we help you implement it in your device.
• 20:00
  SPEAKER_00: Not only on an application processor base like a Chetsen or Qualcomm processor running an operating system, also at a tiny connected microcontroller-based device.
• 20:11
  SPEAKER_00: If it has secure boot, we can leverage that, but we also can provide the root of trust, the secure ODA, and the secure boot via an external secure secure element.
• 20:24
  SPEAKER_00: And um and the platform uh future is future proof, you know.
• 20:29
  SPEAKER_00: We cr the the the vendors we crow we grow and um and support and support uh more and more different hardware vendors.