Latest / EDGE AI POD / Cloud Can’t Keep Up, So Your Toaster Gets A Brain

Transcript

Cloud Can’t Keep Up, So Your Toaster Gets A Brain · EDGE AI POD

• 0:08
  SPEAKER_00: This is on.
• 0:10
  SPEAKER_00: I am neuromorphic uh engineer in Nadera.
• 0:14
  SPEAKER_00: And uh I I am going to present uh end-to-end application generation and deployment for neuromorphic micro microcontrollers.
• 0:25
  SPEAKER_00: So there are sensors everywhere, right?
• 0:28
  SPEAKER_00: And there are also lots of types of sensors, uh ranging from acoustic, temperature, acceleration, vision, uh you name it.
• 0:39
  SPEAKER_00: Um not only they are everywhere, but they are going to be way more uh in the future.
• 0:45
  SPEAKER_00: So by 2030, uh billions of active sensors are going to be there and they are going to be uh outputting data that we cannot process in the cloud.
• 0:56
  SPEAKER_00: So uh the edge is not optim uh optional, uh it's inevitable.
• 1:03
  SPEAKER_00: Um we uh identify that uh concurrent uh normal microcontrollers uh mainly do uh tasks such as uh feature extraction or adjusting or calibrating a sensor, um which uh they are not quite compute intensive, right?
• 1:26
  SPEAKER_00: So like they lack some compute to be able to process all of this data, and that's where the chips that we're designing are going towards to um to enhance uh this uh computing power whilst uh staying in this in this power power range uh for the microcontrollers.
• 1:51
  SPEAKER_00: That allows us to uh develop applications for many domains such as consumer electronics, uh IoT and smart home, uh industrial applications and wearables.
• 2:04
  SPEAKER_00: Um we are going to focus a bit more on wearables in this talk because we have a gesture recognition um application that you can actually visit and and play with in our book.
• 2:19
  SPEAKER_00: Um we have said that there are many sensors and there are there are many types of sensors right that um that tells you that that uh uh to compute on the edge needs a certain flexibility.
• 2:37
  SPEAKER_00: That's why we have uh an heterogeneous architecture.
• 2:42
  SPEAKER_00: That also because we need to increase the the power as we established already, while uh being uh power efficient.
• 2:53
  SPEAKER_00: So um well this is the the heterogeneous architecture of of in Adidas chips.
• 3:00
  SPEAKER_00: Um as you can see um here, um well there there is uh CPU, a risk-file CPU, so that uh you can do uh general purpose tasks, uh also like some sort of common interfacing, and then we we mainly have uh we have a tiny CNN accelerator and then uh an SNN or spiky neural network accelerator, which you can also call a neural synapse array, which is uh our differentiation and and it's an accelerator for um being able to process uh the temporal data.
• 3:44
  SPEAKER_00: Yes, so going a bit deeper on on spiking neural networks that, as I have said, are so important to our chip.
• 3:53
  SPEAKER_00: Um normally the the typical flow or or pipeline that you would uh use to use them is that you would get certain input data that is decimal or or analog, um, and then you would encode this into binary events or spikes, uh which then would uh go through the network uh and then after that you would maybe want to make a bit of sense out of it, maybe like translate it into probabilities or something like that, such that uh you can determine if it's the output is pattern uh A or pattern B, or certain or there is a certain feature in the in the uh sensor uh data.
• 4:40
  SPEAKER_00: Um and we allow to we we uh can do this uh in our chip in many ways.
• 4:50
  SPEAKER_00: Uh as I said before, like we have quite an heterogeneous architecture and which is quite flexible.
• 4:57
  SPEAKER_00: So um we could imagine that uh somehow you get certain data from the CPU and then you could, for example, do this translation of the of the signal into spikes, you could do it in software or in some dedicated hardware that is quite efficient, that we have created uh for for you.
• 5:17
  SPEAKER_00: And uh then that would go through the SNN accelerator, and again, like if you want to create certain uh like uh fancy or like uh exotic um type of uh decoding, which again is like making sense of what the spikes meant, um you could do it in software and also you could do it in hardware, and then this would trigger an action uh probably by the by the CPU, right?
• 5:44
  SPEAKER_00: Um in order to be able to replicate this flow for any application that we uh target, uh we are using uh the Talamo Software Development Kit.
• 5:56
  SPEAKER_00: Um this uh software development kit uh allows you to train these really special uh networks um by uh training them as in normal PyTorch or PyTorch-like uh in a PyTorch-like manner.
• 6:15
  SPEAKER_00: Um it also integrates with uh standard software tools such as uh TensorBoard, just in case you want to know like okay what the network is doing, is it going to fail?
• 6:26
  SPEAKER_00: Is it good enough?
• 6:28
  SPEAKER_00: Um and uh then uh we're extending our our software development kit uh with with new tools, uh which would be the the system compiler that uh will allow you to get this high-level PyTorch uh pipeline that you just trained, and it will basically it will generate a binary file uh that that you can deploy in in our chip, for example.
• 6:59
  SPEAKER_00: Um we also extended uh the Talamos DK capabilities with an system of chip simulator um so that you can uh take into account all the interactions between every component in our heterogeneous uh chip.
• 7:20
  SPEAKER_00: I will exemplify this and the value of these tools uh with an example.
• 7:26
  SPEAKER_00: So actually I do not know how to play this, but uh okay, it doesn't matter.
• 7:35
  SPEAKER_00: We have uh this um application in our booth, but please uh come to to check it out.
• 7:42
  SPEAKER_00: Um in any case, the normal flow, as I was uh commenting before, is that uh you maybe have an IMU sensor.
• 7:49
  SPEAKER_00: This normally uh entails having certain accelerometer data and certain uh gyroscope data.
• 7:55
  SPEAKER_00: And you can uh you can uh translate translate this data into spikes using dedicated hardware.
• 8:05
  SPEAKER_00: Um it's maybe not necessary to know uh uh specifically what these encoders do.
• 8:11
  SPEAKER_00: Uh basically the IF encoder will kind of translate the magnitude of the signal into spikes, and the temporal contrast encoder will translate the the velocity of the of the hand gestures into spikes.
• 8:26
  SPEAKER_00: We already have our spikes, then we can pass them through the SNN and then make sense of them and create an action.
• 8:35
  SPEAKER_00: Um this is how it looks like by the way, if anybody is curious.
• 8:40
  SPEAKER_00: Um so what does the system compiler, this thing that I was talking about uh before, do?
• 8:45
  SPEAKER_00: Well, it it kind of uh from a high-level perspective, it it um it's targeting the fact that there are certain barriers of entry for for developers to uh use our chip um because you you have to know more stuff because there are more components, right?
• 9:06
  SPEAKER_00: So um for for the system compiler, we we maybe have certain high-level pipeline or high-level uh primitives that you have uh trained.
• 9:19
  SPEAKER_00: And uh in in this case, like in our chip, you would have maybe certain connection of these uh software entities of your simulation with uh certain dedicated hardware that implements them quite efficiently.
• 9:36
  SPEAKER_00: Otherwise, for example, there are since there are two temporal uh contrast encoders and this uh hardware may be dedicated, then you could also uh run them in software.
• 9:48
  SPEAKER_00: Um after the hardware assignment is done, then you basically can uh generate automatically certain uh low-level C code that then will be compiled and that you can deploy on our chip quite easily.
• 10:07
  SPEAKER_00: You could deploy it on our chip, but what if you don't have uh our chip or you are just prototyping something quickly, right?
• 10:15
  SPEAKER_00: Uh you need uh a system on chip simulator, not only an SNN simulator or or a CNN simulator, no.
• 10:24
  SPEAKER_00: You need the whole system to be um simulated and functionally good.
• 10:30
  SPEAKER_00: So we developed uh what we call the SOXIM, which is uh basically um so basically it it um satisfies the necessity that our subsystem needs to be tightly coupled because uh the of the time dependencies that that the SNN have, so like um the timing of of the spikes is really important in SNNs.
• 11:00
  SPEAKER_00: So this kind of addresses that problem and and solves it uh functionally.
• 11:07
  SPEAKER_00: Um it's a um cycle approximate uh simulation uh of the whole system that uh ensures that if you want to deploy something on our chip and you don't have it or you just want to verify something, then uh you can use this and it will have the same the same functionality.
• 11:28
  SPEAKER_00: Um well how good these tools are, right?
• 11:33
  SPEAKER_00: So we we evaluated them using basically like um a couple of KPS that we consider important.
• 11:41
  SPEAKER_00: Um one of them one one of them would be for the system compiler, which again is this um software that translates your high-level pipelines into um uh binary uh file basically.
• 11:57
  SPEAKER_00: Um we we saw that uh the static code size of this um of this uh generated code by the system compiler is in average 37% uh higher than uh an app created by an expert at the expense of it's automatic, like you do not need to really think about the implementation.
• 12:23
  SPEAKER_00: Um also we we see a a small drawback in in terms of power consumption because the system compiler is still not quite optimized, but um the the functionality is there and um it's we are working to optimize this even further.
• 12:43
  SPEAKER_00: Um regarding the the SOC simulator, there is 100% accuracy, or if you if you want to call it that way, so there is a like uh a one-to-one match between what our chip would do and what the SOC simulator has um has uh predicted there would be.
• 13:06
  SPEAKER_00: And um we also want to highlight that it's quite quite uh fast for this kind of simulation because um theoretically uh there can there it could be one to five times uh slower than than running the whole thing on the chip, but in in practically we saw that uh it's it's only 1.13 uh times uh slower.
• 13:40
  SPEAKER_00: Um well uh the conclusion is that um to be able to give access to developers to to be able to use our chip efficiently, we need to we need to and we are developing tools uh to make their life easier and to fully uh optimize for for the characteristics of our chip.
• 14:04
  SPEAKER_00: Um and we are going to continue developing these tools.
• 14:10
  SPEAKER_00: Also, um we just uh launched um our new commercial uh chip.
• 14:19
  SPEAKER_00: It's the first uh ultra-low power um neuromorphic chip, uh commercial neuromorphic chip uh in the market.
• 14:28
  SPEAKER_00: It has already earned an award for the most uh innovative chip.
• 14:35
  SPEAKER_00: Um and yes, it's it's uh really uh efficient and we are really looking forward uh for you to try it and engage it.
• 14:47
  SPEAKER_00: Um well also I wanted to to announce and recommend you to assist if you are more interested to join the workshop that there is going to be in 20 minutes, which um will be held by by my colleague uh Peter and and Basile, in which they will discuss uh commercial um the commercial future and the ecosystem of neurphic neuromorphics um in in the in the years to come.
• 15:28
  SPEAKER_00: Thank you very much.