Latest / EDGE AI POD / From Lab to Low-Power: Building EMASS, a Tiny AI Chip That Runs on Milliwatts

Transcript

From Lab to Low-Power: Building EMASS, a Tiny AI Chip That Runs on Milliwatts · EDGE AI POD

0:34
SPEAKER_05: Hello, hello, hello.
0:36
SPEAKER_05: Good morning.
0:39
SPEAKER_05: Okay, so here we are, HAI talks.
0:43
SPEAKER_05: We're back.
0:44
SPEAKER_05: Um folks are rolling in.
0:46
SPEAKER_05: They're elbowing their way into the virtual lobby here, so that's cool.
0:50
SPEAKER_05: Um I know we had over a hundred folks registered, so we'll we'll see um uh who's who's showing up today.
0:58
SPEAKER_05: But um good afternoon, Sarah, I should say.
1:00
SPEAKER_05: Good afternoon, because you're in Europe right now, right?
1:02
SPEAKER_02: Hi, Pete, yeah, so based in Sweden.
1:05
SPEAKER_02: So afternoon still, and still light.
1:08
SPEAKER_05: Oh, nice.
1:09
SPEAKER_05: Good.
1:09
SPEAKER_05: I'm in Bellevue, Washington, which it is still light because it is eight in the morning.
1:15
SPEAKER_05: So cool.
1:18
SPEAKER_05: All right, we're gonna have a good show today.
1:19
SPEAKER_05: Um I actually talked to Mohammed recently.
1:22
SPEAKER_05: So we have this so every other week we have these edge AI talks, we have these live streams, and then in between those weeks we have what we call edge I a partner, where we have discussions with different new partners, uh, new innovative partners that people maybe haven't heard of.
1:36
SPEAKER_05: So I did talk to Mohammed recently about um EMAS and what he was doing there.
1:40
SPEAKER_05: So I have a I have a little bit of a head start on all the cool stuff they're doing there.
1:44
SPEAKER_05: But um before we get into it, let me do a couple of PSAs.
1:49
SPEAKER_05: One is uh for those that are going to be in Amsterdam next week at the Things Conference, uh I'll be there.
1:55
SPEAKER_05: We'll have the whole Edge I Foundation community will be there.
1:59
SPEAKER_05: Um so I look forward to seeing you there.
2:02
SPEAKER_05: Uh please uh find me.
2:04
SPEAKER_05: Um we're actually gonna launch a uh a real-time scavenger hunt there where you can uh win cool prizes, and this is one of the prizes is this cool mug.
2:13
SPEAKER_05: Um and uh so keep an eye out for that on our website.
2:17
SPEAKER_05: And then we have if you haven't yet registered for edgei Taipei 2025.
2:24
SPEAKER_05: Um tapping November 11th and 12th in Taipei.
2:28
SPEAKER_05: Surprise, surprise, and um twice as big as last year.
2:31
SPEAKER_05: And it has some amazing keynotes and workshops and talks and uh and evening events.
2:37
SPEAKER_05: So um register for that.
2:39
SPEAKER_05: I think it's type2025.org.
2:43
SPEAKER_05: I should probably uh put that in a little banner.
2:46
SPEAKER_05: Um yeah, and so those are the two two uh calls to action for folks there.
2:51
SPEAKER_05: And of course, if you have not yet subscribed to our YouTube channel, please do so.
2:55
SPEAKER_05: I think we have maybe crossed the 50,000 subscriber boundary today.
3:00
SPEAKER_05: I'm not sure.
3:02
SPEAKER_05: That's pretty exciting.
3:03
SPEAKER_05: Sarah, what is new from Imagimob on your end?
3:06
SPEAKER_05: You've been traveling like crazy, you mentioned.
3:08
SPEAKER_02: So yeah, we just had a pretty exciting tour through um Singapore and Japan.
3:14
SPEAKER_02: So excited to hear our guest today, because he's from Singapore too.
3:18
SPEAKER_02: Hopefully he'll mention a bit about the industry.
3:20
SPEAKER_02: What I found there is quite innovative, uh, quite ahead.
3:24
SPEAKER_02: So um pretty cool.
3:25
SPEAKER_02: Um, and looking looking forward to learn more from him.
3:29
SPEAKER_05: Cool.
3:30
SPEAKER_05: Yeah, Singapore is a lot of fun.
3:31
SPEAKER_05: I'd mentioned I used to go there when I was at a previous company early in career.
3:36
SPEAKER_05: I used to go there all the time.
3:37
SPEAKER_05: And um it's such an incredible place, it's so unique.
3:42
SPEAKER_05: Uh the chili crab and the all that cool stuff going on there.
3:46
SPEAKER_02: So we've been said well, let's say that.
3:50
SPEAKER_05: That's the thing, like you know, traveling is hard, uh, but you know, I always appreciate the evenings because you can get an interesting dinner and you know, really understand the culture better through the food.
4:01
SPEAKER_05: So I think that's pretty cool.
4:03
SPEAKER_05: Um so yeah, hopefully get back to Singapore pretty soon.
4:08
SPEAKER_05: Good.
4:09
SPEAKER_05: Well, let's see.
4:10
SPEAKER_05: Without further ado, why don't we bring Mohammed on here?
4:12
SPEAKER_05: And um uh he can tell you his background, but he's actually a professor and a founder.
4:18
SPEAKER_05: So not unusual actually to find folks in academia doing cool research that then decide, hmm, this could be a company.
4:26
SPEAKER_05: So uh Muhammad's one of those folks, so let's bring him on.
4:30
SPEAKER_05: There he is.
4:31
SPEAKER_05: Welcome, sir.
4:33
SPEAKER_01: Hi, Pete.
4:34
SPEAKER_01: Hi, Sarah.
4:36
SPEAKER_02: Good to have you.
4:37
SPEAKER_01: Uh pleasure is all mine.
4:41
SPEAKER_05: Good.
4:41
SPEAKER_05: So where are you?
4:42
SPEAKER_05: You're calling in from Singapore?
4:44
SPEAKER_01: Yeah, today I'm calling from Singapore, yeah.
4:47
SPEAKER_01: So uh I'm a bit of an international citizen, but today I'm in Singapore currently.
4:51
SPEAKER_05: Yes, yes.
4:52
SPEAKER_05: I think the last time we spoke, you were in Egypt.
4:54
SPEAKER_01: So yes, indeed.
4:55
SPEAKER_01: Yeah, I was uh in the midst of like me and spending a little bit of summertime with the family and uh working with our RD office in Cairo, yeah.
5:04
SPEAKER_05: Yeah, so that's cool.
5:05
SPEAKER_05: Yeah, so uh different cities, same screen, as they say.
5:10
SPEAKER_05: Um, but uh cool.
5:12
SPEAKER_05: So so maybe give us before you launch into talking about all the cool stuff you're working on, why don't you give us a little bit of a background on yourself and how did we get here?
5:23
SPEAKER_01: Yeah, sure.
5:24
SPEAKER_01: All right, so uh hi everyone, right?
5:27
SPEAKER_01: So good morning, good afternoon, good evening, I think, mean depending on the time zone you're in.
5:32
SPEAKER_01: Um so my name is Mohammed Ali.
5:34
SPEAKER_01: Um I'm I'm wearing on multiple hats, but currently I'm mean the hat here, I'm the founder of uh an edge AI company called EMAS.
5:42
SPEAKER_01: In the meantime, as well, I'm also um a professor here at NTU Singapore.
5:48
SPEAKER_01: Um my journey started in Switzerland when I did my degrees in EPFL, then I went to Stanford.
5:56
SPEAKER_01: I did a little bit of research on like kind of um the whole thing that I was always working with.
6:01
SPEAKER_01: How can you bring new nanotechnology devices and integration technologies all the way to application level and benefit from them while at the same time overcoming their short uh like mean their challenges right in a scalable way?
6:18
SPEAKER_01: So I joined here, uh, I mean, I spearheaded a research project that's spanning multiple research institutes here in Singapore, and we were looking into the next wave of um AI hardware.
6:30
SPEAKER_01: That was in 2017, 2018.
6:33
SPEAKER_01: So we looked into, I mean, the main concept that's always been pitching in is that if you want to achieve significant gains, you have to look at the entire stack, see in between those layers that are these tiny hidden knobs that when you tweak them and you make all of these melded together, you will get significant gains, much larger than dealing with each layer of abstraction separately.
6:56
SPEAKER_01: So we were working on this and we're trying to get that.
6:59
SPEAKER_01: We built a couple of mean very large chips, and then we say that hey, this actually could be very efficient in like in very low power platforms, where um back in the day, right, edge AI was still emerging, like back in we'll talk about 2019, 2020.
7:14
SPEAKER_01: And the challenge there was like, I mean, can you really put AI there?
7:19
SPEAKER_01: And secondly, can you preserve the batteries right for a very long uh duration?
7:24
SPEAKER_01: And we started playing with all layers of abstraction, we came up with this nice concept, and then after all that, we said, hey, this is too good to just be in shelved out and waiting for the right uh I mean entrepreneur to come and take this uh to the light.
7:38
SPEAKER_01: So I said, hey, you know what?
7:39
SPEAKER_01: I'm gonna take the jump.
7:40
SPEAKER_01: And I founded the company, and uh the rest is history.
7:44
SPEAKER_01: So, I mean, four years down the road, mean we managed to get some chip development.
7:48
SPEAKER_01: We we worked so hard, we grinded like mean going crazy here and there with very few people.
7:53
SPEAKER_01: Then slowly, slowly, we started like getting more traction.
7:56
SPEAKER_01: We've been acquired by Nanovie, where we're helping us significantly through connections and funding.
8:03
SPEAKER_01: We grow now to the point that we're starting now going for larger mass production, much the newer generations of chipsets, customer tractions, even engaging in applications that we actually are excited about in various fields in terms of like mean wearables, drone technologies, predictive maintenance, and whatnot.
8:22
SPEAKER_01: So, yeah, so just a little bit to start it, mean as in research, and then we say, hey, this is too good, and now here we are.
8:30
SPEAKER_04: Yeah.
8:31
SPEAKER_04: Are you still doing uh teaching?
8:33
SPEAKER_04: Are you still doing lectures and things?
8:35
SPEAKER_01: Yes, indeed.
8:36
SPEAKER_01: Um yeah, I mean, I I think I mean I'm I'm getting more white hair than expected at my age because of the stress, but um uh currently really focused on the company, but teaching is something that I'm passionate about.
8:50
SPEAKER_01: So, I mean, I still teach undergraduate students as well.
8:53
SPEAKER_05: Yeah, yeah.
8:53
SPEAKER_05: No, that's cool.
8:55
SPEAKER_05: Yeah, I was saying it's not unusual to find uh in this space people doing, you know, professors and educators doing some really cool research, and then it's sort of like, hey, maybe we should productize this and uh you know get it to market.
9:09
SPEAKER_05: I I can probably just top of my head, three three to five at least folks that I know that yeah.
9:14
SPEAKER_01: I think it's the Silicon Valley uh mindset that when I was at Stanford just like me got in me, and then when I came here, I said, okay, let me apply that.
9:23
SPEAKER_05: Right.
9:23
SPEAKER_05: Just spin it out, raise some money, and yes, indeed.
9:27
SPEAKER_05: Uh yeah, that is awesome.
9:29
SPEAKER_05: And Nanoview is based in Australia too, I believe, since we're talking about geographies.
9:34
SPEAKER_01: Yeah, so Nanoview has the main office in Australia, they have here uh an RD office in Singapore.
9:39
SPEAKER_01: So we have now the headquarters here in Singapore, an RD office in Egypt, and we have uh managing a marketing office in North America, United States.
9:50
SPEAKER_01: So we are spreading across the globe as well.
9:53
SPEAKER_05: Yes.
9:54
SPEAKER_05: Yeah, I was talking to someone the other day about, you know, we've done a good job sort of flattening the earth um with uh these tools to meet and collaborate, but we have not solved the curvature of the earth problem with time.
10:08
SPEAKER_05: So time zones are still challenging uh for all of us to kind of get everybody on the same page.
10:14
SPEAKER_05: But uh no, it sounds cool.
10:17
SPEAKER_05: Um yeah.
10:18
SPEAKER_05: Sarah, do you have any uh questions for Mohammed before he jumps into his uh his spiel?
10:25
SPEAKER_02: No, I've been pretty curious, so let's get started.
10:27
SPEAKER_02: And I think the questions will pile up as you go.
10:30
SPEAKER_05: Yes, yeah.
10:31
SPEAKER_05: So a reminder to our visitors, this is a live stream, uh emphasis on the live.
10:35
SPEAKER_05: And so uh I see we've got a bunch of people here.
10:38
SPEAKER_05: So uh feel free to dig in, uh, shout out where you're from, uh, where you're calling in from, and then um also put your questions in here.
10:47
SPEAKER_05: And Sarah and I will monitor the stream and uh and politely cut in at various times to uh to pepper Muhammad with your questions.
10:57
SPEAKER_05: So don't be shy uh on that stuff.
10:59
SPEAKER_05: Okay, so why don't we bring up uh um uh your deck and uh we will give you the floor, we will hang back and uh let you kind of get into it.
11:10
SPEAKER_05: And like I said, we'll monitor the chat and uh oh, I see Tim here from San Jose.
11:16
SPEAKER_05: Okay.
11:17
SPEAKER_05: Um actually, why don't we just uh oh let's see.
11:20
SPEAKER_05: Well, let's just throw this up here just for just to show people how it works.
11:24
SPEAKER_05: So Tim from San Jose looking at a discussion about CubeSat launch from build by high school students.
11:30
SPEAKER_05: What is CubeSat?
11:32
SPEAKER_05: Is that uh I'm not sure what that is.
11:35
SPEAKER_01: Oh wow, okay.
11:36
SPEAKER_01: So high school students doing some sort of like um, I mean, a satellite cube and they're looking for a little bit of a power hardware into space.
11:44
SPEAKER_05: Yes, that is interesting.
11:46
SPEAKER_05: Um so sidebar on that for Tim is like there's a ton of action going on in the edge AI space um in space.
11:54
SPEAKER_05: Uh so a lot of like uh low Earth orbit satellite stuff happening to do um analysis of uh crops and uh environmental detection and things like that.
12:05
SPEAKER_05: So doing that monitoring on the platform in space is kind of the next thing.
12:09
SPEAKER_05: So lots of companies doing stuff like that.
12:11
SPEAKER_05: Um and probably uh you know the EMAS silicon could help there too at some point.
12:17
SPEAKER_05: But I I will defer to Mohammed to explain how that would work.
12:20
SPEAKER_00: Yeah, absolutely.
12:21
SPEAKER_00: Yeah, more than happy to explore uh I mean that beyond uh what's happening on the earth.
12:26
SPEAKER_05: Perfect.
12:26
SPEAKER_05: All right, well, we will we will hang back and let you let you do your thing.
12:31
SPEAKER_01: Sure.
12:31
SPEAKER_01: Okay, all right.
12:32
SPEAKER_01: Thanks, Pete and Sarah, for the introduction, opportunity, and the time.
12:36
SPEAKER_01: So, and thanks everyone for uh showing up.
12:39
SPEAKER_01: I'm really excited and honored to uh give you a bit of highlight, I mean what we're working on and what I mean, what's EMS about and where do we we stand at the moment and what are the secret sauce behind it, hoping that it can spark interest from uh potential collaborators or even others who can see what can innovate beyond that as well.
13:02
SPEAKER_01: So the main thing that I'm looking into or like being a focus on, the idea is going from atoms to applications, right?
13:09
SPEAKER_01: Which is really from the nanotechnology from these devices all the way to the application level.
13:14
SPEAKER_01: I mean, we're looking at the whole thing as one monolithic unit and see how we can optimize all of these layers together at the same time to achieve significant gains.
13:27
SPEAKER_01: So just a little bit of uh introduction, and I think we just like mean uh shed the lights on it before.
13:33
SPEAKER_01: Um, EMAS is short for embedded AI systems, and uh we actually founded 2020 in Singapore, and then as the journey went by, we finally had a RD center here in Singapore and in Cairo, Egypt, where I'm spearheading both of them, and sometimes I travel back and forth.
13:51
SPEAKER_01: Um, I mean the idea being in Egypt is that I mean I'm originally from Egypt and there's a lot of good talent there.
13:58
SPEAKER_01: Um, on a side note, actually, if you can see the logo of EMAS, is actually it is one of the what's called the Shen symbols, which is an ancient Egyptian, it's just a little bit rotated.
14:08
SPEAKER_01: Uh, that always when kings put their names inside that cartouche.
14:13
SPEAKER_01: Uh just um in a fun fact uh as we go.
14:17
SPEAKER_01: Uh, we kind of have also commercial business in the United States, and we look into um building a chip and the entire ecosystem around it, which is the compiler, the software, the board, right, that allows you to run AI with minimal uh retraining, right, to deploy any kind of solution with our uh chip in it.
14:41
SPEAKER_01: So the current leadership team, we have uh I mean Mark Garnson was the CEO of the semiconductor branch of None of You.
14:48
SPEAKER_01: Um he actually came out of retirement to uh lead this one and in the executive front.
14:54
SPEAKER_01: And I have uh Scott Smeiser, um, who I have the pleasure of working with.
14:57
SPEAKER_01: He's the vice president in sales and marketing, and myself, right?
15:01
SPEAKER_01: I mean, where I'm uh I mean now currently this acting as the CDO of EMAS and I'm the founder of it.
15:06
SPEAKER_01: At the same time, I said I'm I'm an associate professor at NTU Singapore here.
15:10
SPEAKER_01: Um, I mean, where I've been had the pleasure of conducting all the research before uh spinning off, right, in its premise.
15:19
SPEAKER_01: And just like before that, I was at Stanford.
15:21
SPEAKER_01: Uh before that, I was at EPL in Switzerland.
15:24
SPEAKER_01: So I consider myself somewhat of an international citizen where I've been lived in many continents.
15:30
SPEAKER_01: All right, so um I think that since we are talking about we are here now in the edge um AI community, right?
15:37
SPEAKER_01: We have the main challenge that we want to run a lot of applications in a battery constrained systems.
15:43
SPEAKER_01: And if you wanted to do this, right, the idea, if you wanted to get that done, you really have to look into the entire system as a whole, not into different pieces separately.
15:53
SPEAKER_01: Yes, you can solve some of the components there, but to get your maximum gains, right, you really to want to bring your AI applications very close to where the sensors are, redesign your semiconductor chip so that it knows uh mean what kind of uh where it directs the energy to where it's actually needed and it doesn't consume any other energy otherwise.
16:14
SPEAKER_01: And at the same time, make the application squeeze it and fit it into the current chip with its tight constraints, at the same time, do not uh reduce the accuracy, or otherwise you will not have a meaningful application coming back.
16:28
SPEAKER_01: So, but if you do this, right, if this is a very complex uh mean equation to solve, but if you manage to get that, it really unlocks new opportunities.
16:37
SPEAKER_01: And I mean, there's a lot of work happening globally, right, trying to tackle all of this, and just we are fortunate to be one of these uh I mean players on that field, right, to try to really crack this equation and try to get uh the whole thing uh mean serviceable and available right to the masses uh um in this context.
16:58
SPEAKER_01: So so how do we approach the uh this kind of problem, right?
17:03
SPEAKER_01: As I said, we look into it as one full uh mean monolithic uh layer.
17:09
SPEAKER_01: Is it from atoms all the way to the application?
17:12
SPEAKER_01: So when we design the silicon, right, we try to see what kind of emerging nanotechnology devices and integration techniques that we can benefit from that can give us, right?
17:22
SPEAKER_01: Mean an unfair advantage or can give us significant benefits.
17:26
SPEAKER_01: And if there's any kind of shortcomings after it, can we solve it at a higher level of abstraction?
17:31
SPEAKER_01: So we don't have to wait for the technology to really fully mature.
17:35
SPEAKER_01: If it's just ready, but there are some uh kinks around it, we can try to solve it at the application at the application level.
17:41
SPEAKER_01: From there, we can take to develop modules, right, that can be um more of a plug-and-play option and modular with different sensors that will be targeting different applications.
17:51
SPEAKER_01: But as you design the application and the AIs for that, you have to also bear in mind that it will run into these small chips.
17:58
SPEAKER_01: So you have to be very careful what kind of operations you all can only support, what kind of operations you can provide that maximizes accuracy and make uh give you the answer that you really need without requiring a significant amount of computational power, which will then cause a larger battery or will cause significantly shorter lifetime.
18:18
SPEAKER_01: So, what I will go, I mean, in uh in the next part of this talk, I will go bit by bit right into these abstraction layers and try to shed some of the technical highlights.
18:28
SPEAKER_01: I will try to give it in layman uh terms as much as I can, but if you want me to go deeper, I'm more than happy to do so.
18:36
SPEAKER_01: If you if you feel like it's got this is too much, I'm more than happy to try to even raise the abstraction level a little bit higher.
18:43
SPEAKER_01: So the way that I said, right, it is a full end-to-end approach, right?
18:48
SPEAKER_01: We look at the entire system.
18:49
SPEAKER_01: So we try to say that, okay, I mean, there's a link between the algorithms, right, and the data flow of your hardware, right?
18:57
SPEAKER_01: So with a chip design, you have to know exactly what kind of algorithms.
19:00
SPEAKER_01: Try to make your algorithms more aware of the underlying compute fabric and make the compute fabric also at the same time mean more have some sort of uh support for some of the algorithmic transformation that you would have at a higher level.
19:14
SPEAKER_01: Then you go at the lower level, even at the circuit design, when you actually lay out your chips, right?
19:19
SPEAKER_01: You try to make sure that there are not any kind of unnecessary delays, and at the same time, give power gaining support or like mean enabling the idea that you would only deliver the power to just the components that need to be run at the moment, at that exact specific point in time, and the other ones would be just like turned off.
19:40
SPEAKER_01: To be able to do this at such a fine granularity in time and in space, you would need some new nanodevices that can help you switch things on and off, right, in microsecond range.
19:53
SPEAKER_01: And that's where new non-volatile memories come in play.
19:56
SPEAKER_01: And then with this new non-volatile memories, they actually have some sort of shortcomings.
20:01
SPEAKER_01: So you have to redesign the algorithms to be aware of these shortcomings so not to stress them, right, and make expose this kind of inefficiencies in that.
20:10
SPEAKER_01: That's why the whole thing is end-to-end and everything is interconnected together.
20:14
SPEAKER_01: When you get everything done properly, right, then you achieve significant gains.
20:19
SPEAKER_01: But we will see now in each of these layers how this could be done.
20:24
SPEAKER_01: So the chip itself, if you look at the data flow that we have, right, it's a RISC-V-based processor.
20:30
SPEAKER_01: We have two AI accelerators in it.
20:33
SPEAKER_01: One, each one, as you can see here in the uh diagram or in the figure on your right, is actually responsible for certain kinds of kernel operations together, and we have also a support for AI compression, they can run the vast majority of the kernels in convolutional neural networks, multi-layer perceptrons, and recurrent neural networks, okay, that can run them sufficiently with very, very compressed weights that can give an average of 1.3 bits per weight, which means that while it is like non-power of two number, that there are some layers that will have one bits, some layer would have a little bit higher than one bit, and even when the layers with one bit can be compressed, right, to merge everything together.
21:15
SPEAKER_01: But at the application level, this does not harm the accuracy.
21:19
SPEAKER_01: And we will see like in the next slide how this is possible.
21:23
SPEAKER_01: Now, to merge to complement this, we need to increase the on-chip memory to have some sort of support for larger AI workloads.
21:32
SPEAKER_01: So we look into mean SRAM, that's our temporary memory to store our uh I mean variables, right?
21:38
SPEAKER_01: All and you have like this all partial sums and whatnot.
21:41
SPEAKER_01: But for your AI workloads, and the idea if you want to turn on the system on and off, we have this non-volatile memory, right?
21:49
SPEAKER_01: Magnetic RAM is an example, resistive memory is also another one.
21:52
SPEAKER_01: So we're using MRAM or R RAM, right?
21:55
SPEAKER_01: And we have experimented with both technologies as well that can help you store the AI weights.
22:01
SPEAKER_01: So when the chip is not in use, you can turn the entire chip off and back on, and it will retain its state.
22:07
SPEAKER_01: So there's no loss of information, you don't need to retrieve the data from any off-chip uh flash memory or even on-chip flash memory.
22:15
SPEAKER_01: Then when you are having this, right, you can also get your weights there, right?
22:21
SPEAKER_01: Keep it, keep it there, and then when you are done with your inference, or even layer by layer, you can selectively turn off your processor, you can turn off this accelerator or this accelerator or any kind of the other component that is not in use.
22:36
SPEAKER_01: Now, another thing that we also think about is that AI not just is compute intensive, it's also memory intensive, it's memory-centric.
22:45
SPEAKER_01: So you need to have a large accessibility to memory.
22:48
SPEAKER_01: While we don't have a lot of capacity, but still you need to have wide parallel access.
22:53
SPEAKER_01: That's why we have a lot of parallelism when accessing this kind of memory that can reach orders of magnitude over 12 gigabytes per second.
23:02
SPEAKER_01: Bear in mind that this runs at 50 megahertz, right?
23:05
SPEAKER_01: If it runs at a much higher frequency, we can reach multiple terabytes per second access bandwidth.
23:09
SPEAKER_01: But that's not the point here because we're running at the edge domain, very high speed is not needed in that context.
23:16
SPEAKER_01: But could having this non-volatile memory that acts also with this large uh or this high speed bandwidth or this wide bandwidth, right?
23:24
SPEAKER_01: Eliminated the need for any kind of high bandwidth memory or really aggressive DRAM that can give you this kind of bandwidth, but would be with either higher energy or the idea of having two-chip solutions, right, which would increase your uh footprint.
23:38
SPEAKER_01: And that could be limiting, right, for some applications such as smart rings or even specs or even any kind of application that require very small footprint applied there.
23:49
SPEAKER_01: So when you combine these things with the idea that you can compress your workload, you can get, we get actually around from one to five milliwatts running in continuous inference of the certain workloads.
24:01
SPEAKER_01: And with that, right, you can get that even in reality, when you have the idea that you can get data right from the uh environmental sensors, not continuously, because it the mother nature doesn't provide very high uh I mean uh bandwidth data, right?
24:17
SPEAKER_01: So you can turn off the system when you don't have data to run the AI, and then once you have sufficient data, you turn back the entire system and make it operate.
24:25
SPEAKER_01: So you would really save the juice.
24:27
SPEAKER_01: And not just we're talking about like going into ultra-low power, we're talking about complete shutoff.
24:33
SPEAKER_01: Now, to get this chip to be interfacing, we have the regular sensors, and also we say that it has a very small package, which is five plus five uh millimeter QFN package, and that would help us right to be run into these kind of systems that require very small footprint.
24:51
SPEAKER_01: So, how did we do the trick, right?
24:54
SPEAKER_01: So if you look at the algorithm right layer, I mean, I think many of uh experts here mean it's no surprise that looking into how we can reduce things, there are a number of options.
25:04
SPEAKER_01: You either you do some pruning, knowledge destination, which is this idea is removing unnecessary weights, right?
25:11
SPEAKER_01: Which remove unnecessary computations.
25:13
SPEAKER_01: So that's one way.
25:14
SPEAKER_01: Secondly, right, we do quantization, which basically means that each variable can use smaller and smaller bits at the same time, right, without any kind of affecting any kind of accuracy.
25:25
SPEAKER_01: Then, once you've done both of these together, now you have your weights.
25:29
SPEAKER_01: Can I really compress it right in such a way that when I bring it to the chip, it's really compressed and I can decompress it on the fly.
25:37
SPEAKER_01: So it goes to my AI accelerator.
25:40
SPEAKER_01: So we have applied the three of these techniques together, and that allows us to achieve significant production, right, and speed up compared to having a full precision or a full-blown workloads, right, uh, for this case.
25:53
SPEAKER_01: Now, this is something that's been available in the community.
25:56
SPEAKER_01: We look into this, we have done it in the convolution neural networks for recurrent networks, and currently we are also conducting a little bit of experiments to apply this in language models.
26:06
SPEAKER_01: So try to get small language models into tiny language models and large language models to medium-sized or small-sized language models as well.
26:16
SPEAKER_01: The other thing that I mentioned about is the on-chip memory, right?
26:19
SPEAKER_01: And we talk about we're using MRAM or RRAM, and that actually enables fine-grained power gating.
26:24
SPEAKER_01: What do I mean by fine-grained power gating is that you can select small instances of time, and we're talking about here milliseconds, right?
26:33
SPEAKER_01: You can turn off the entire system and then you can turn it back on.
26:36
SPEAKER_01: Now, when you're using MRAM and RRAM, it allows you to do this in microsecond rate.
26:41
SPEAKER_01: So you can turn off and turn on in microseconds, and that's almost 5,000 times quicker, right, than doing this with flash.
26:49
SPEAKER_01: So suddenly you have this opportunity that once you finish running your AI workload at the moment, while waiting for the data to come, instead of going for ultra-low power, will still leakage, it's gonna be a component, you just turn the entire thing off, right?
27:04
SPEAKER_01: And when you entire the entire thing off, more than 90% of it is being no consuming zero power at all.
27:10
SPEAKER_01: And all that's needed here is that once you have sufficient data, you will trigger the system, get it, turn it back on, and re with the previous retained state, and resume the application execution.
27:21
SPEAKER_01: And when you do something like this, right, then combined with the fact that resisted memory and magnetic memory are even more efficient than flash, right?
27:29
SPEAKER_01: You can read from them directly, right, instead of doing it through to SRAM and then using that, and you will be able to achieve significant gains, can get you up to 10x, right?
27:41
SPEAKER_01: Benefits in energy due to this uh this kind of trick here.
27:45
SPEAKER_01: Of course, now when you want to do this, now this sounds too good to be true.
27:49
SPEAKER_01: So, what's the catch here?
27:50
SPEAKER_01: That R RAM and RM MRAM memories are not perfect memories, right?
27:54
SPEAKER_01: They have a lot of challenges going on.
27:56
SPEAKER_01: I mean, some of them can actually, when you read them, you don't read it properly.
27:59
SPEAKER_01: Some of them, when you write them, if the writing to them is not done properly, you have to rewrite a number of times.
28:04
SPEAKER_01: Even they have very limited lifetime and endurance, so the lifetime of them is very low.
28:09
SPEAKER_01: It can be like 10 to the six times or million times, and then that cell goes bad.
28:14
SPEAKER_01: So, how would you be able to tackle this right as the technology matures by the foundries and still make use of that?
28:21
SPEAKER_01: And that's where we come in.
28:22
SPEAKER_01: That the applications, right, that when we this when we reprogram the applications, right, they are aware that you should not be doing a lot of rights here.
28:32
SPEAKER_01: That's one example.
28:33
SPEAKER_01: Another thing that if you do a lot of rights, you have to do it in a very smart way that you don't have to stress the same location of your memory again and again and again, right?
28:43
SPEAKER_01: Something that's called like like like where leveling.
28:45
SPEAKER_01: So, in a combination of the algorithms tuning and the architecture at the same side, right, you will be able to overcome this kind of limitations and you don't have to wait for years for the technology to be super mature, right?
28:58
SPEAKER_01: And you can actually go ahead and using, become an early adopter of this technology and gives you an edge compared to other players in the field.
29:08
SPEAKER_01: All right, so when we combine all of that together, right, if you look at where we stand compared to other uh I mean providers of edge e so we're doing using with RISC-5, we'd have two deep learning accelerators.
29:23
SPEAKER_01: Typical other ones will use ARM processors and they use like their own version of NPU.
29:28
SPEAKER_01: Sometimes they can use DSPs.
29:29
SPEAKER_01: Now, with the memory here, we don't require any external memories, although I mean there are some other players also doing the same thing, but the majority also some require some sort of a DRAM or external flash.
29:41
SPEAKER_01: Now we have one of the very high compressed models, right, on our side, and we can mean provide that to our chip.
29:48
SPEAKER_01: And others would require to have this kind of like full precision, whether it's at eight bits, four bits, or two bits, right?
29:55
SPEAKER_01: These models have to be supported in that way.
29:57
SPEAKER_01: We do not require the user to compress the model because.
30:00
SPEAKER_01: Because we will do it for them.
30:01
SPEAKER_01: And as a result, we can get to five milliwatt active power, and that's running on continuous inferencing.
30:06
SPEAKER_01: It's not just like one pod.
30:08
SPEAKER_01: This is doing inferencing again and again and again and again.
30:11
SPEAKER_01: In reality, that doesn't happen.
30:12
SPEAKER_01: So usually the power drops down to one milliwatt or even lesser than that.
30:17
SPEAKER_01: So all of this, right, require this enables us to run neural networks, classical applications, and we have multiple uh I/O uh ports that can get a lot of sensor data, fuse it, and give it to the AI engine to give even deeper insights, right?
30:33
SPEAKER_01: Despite the fact that this is a very, very small chip, but it can have a lot of power usage, a lot of applications in fields such as an audio, vision, right?
30:43
SPEAKER_01: An IoT and even wearables such as ring or mean like mean uh biometric devices and so on.
30:51
SPEAKER_01: Right.
30:52
SPEAKER_01: Do we have any questions, Pete, or should I still continue?
30:57
SPEAKER_05: A couple of questions, uh, some internally um in uh generated.
31:03
SPEAKER_05: Um there was a question here.
31:05
SPEAKER_05: I think you had mentioned also on your website around drone drone applicability.
31:10
SPEAKER_05: And you know, certainly in in kind of drone deployments, it's all about uh power and weight, right, and battery life.
31:17
SPEAKER_05: Uh, can you talk a little bit more about kind of the applicability for drones and and do you have any kind of drone trials that are in process?
31:25
SPEAKER_01: Yeah, so actually, I mean the drone is one of the examples that I will talk, I'll shed the lights on a little bit later, right?
31:31
SPEAKER_01: Um, I mean, the the up the use of AI in drones, right, have been captured the interest for quite some time.
31:39
SPEAKER_01: Like, and the idea is that the all the points that people talk about is enhancing the functionality of drones, right?
31:45
SPEAKER_01: That like will try to provide with certain vision, try to detect objects there, try to help in navigation and so on.
31:51
SPEAKER_01: But even if you have the magical chip that would do this, right, and you start speak to the uh that consume zero energy, right?
32:00
SPEAKER_01: And you speak to drone manufacturers, they will tell you, well, I mean, actually, our problem was never the electronics.
32:05
SPEAKER_01: Our problem is the energy consumed by the propellers.
32:08
SPEAKER_01: So that actually sparked, I mean, our uh quote unquote research.
32:13
SPEAKER_01: And look, can we use our trip, right, to re like distribute the workloads on these propellers and save the energy right while you're flying so you can achieve the same thing but consuming lower energy.
32:27
SPEAKER_01: So we did some analysis of that, and I will explain it a little bit later, right?
32:31
SPEAKER_01: But the bottom line, we achieved to get like enhancing the flight time in simulations, yeah.
32:36
SPEAKER_01: I mean, uh, I mean, granted, but by up to 50 or 60 percent, sometimes can even reach 70 percent improvement in your flight endurance, which is on the same charge, you cover wider distance or you cover longer flight time, and that's thanks to the idea that you will have some sort of automatic updates, right, of your AI application.
33:07
SPEAKER_01: At the same time, you would have another AI that can perform this kind of prediction using that or relating that to the inactivatable component, and that's what you achieve actually real savings in the in the drones.
33:27
SPEAKER_01: So, yeah, so that I mean 50%, 60% is not significant, but in drones actually the same.
33:36
SPEAKER_05: Yeah, so you're using the you're using kind of uh the AI workloads to optimize the propeller usage to getting more energy and uh flight time out of the drones, which is really interesting.
33:46
SPEAKER_05: Um we might be having a little bit of a bandwidth thing going on here.
33:49
SPEAKER_05: I'm not sure.
33:50
SPEAKER_05: Sarah, are you seeing a bandwidth thing going on here?
33:54
SPEAKER_02: Yeah, I was thinking we may need some AI to piece together your your answer, but I think most of it we could follow.
34:00
SPEAKER_05: There you go.
34:00
SPEAKER_05: Yeah.
34:02
SPEAKER_05: Mohammed, you might be getting a little choppy on us.
34:07
SPEAKER_05: Um but yeah, we can I I guess if you're gonna talk more about drones, that's pretty cool uh in in in a minute.
34:12
SPEAKER_05: So we we can dig into that.
34:14
SPEAKER_05: That is kind of uh a pretty canonical example of where edge AI is helping.
34:19
SPEAKER_05: I mean, obviously there's the AI vision part.
34:21
SPEAKER_01: Um close a lot of things.
34:25
SPEAKER_01: Okay.
34:28
SPEAKER_05: Um but uh but certainly I've not heard about uh AI being used to optimize uh some of the propellers and the uh propulsion and stuff, so that's pretty cool.
34:40
SPEAKER_05: Um another question uh we had uh here.
34:45
SPEAKER_05: Um well maybe we should wait for you to kind of get your bandwidth back together.
34:49
SPEAKER_05: Um we are live.
34:56
SPEAKER_05: Uh here, let me let me know.
34:58
SPEAKER_01: Okay, so I'll yeah, I'm back.
35:02
SPEAKER_01: I think it's just like when there was a lot of running workloads in my laptop, and I just I need to pull some of them to uh keep the CPU free.
35:10
SPEAKER_05: There you go.
35:10
SPEAKER_05: We got the CPU thing going on.
35:13
SPEAKER_05: Um there's a question here about uh uh is this architecture applicable to generative image applications?
35:22
SPEAKER_05: I think you might have mentioned this.
35:25
SPEAKER_05: I mean, you know, this is not a TOPS type of platform, it's more of a GOPS platform.
35:29
SPEAKER_05: So uh yeah, yeah.
35:32
SPEAKER_01: So yeah, so while while the principle is applicable, the current shape that we have, I think it will be a little bit like mean a bit of a stretch to try to do this, unless you do it very, very small images, yeah.
35:44
SPEAKER_01: Right, right, exactly.
35:46
SPEAKER_05: So not yet, I would say.
35:49
SPEAKER_05: Um another question here.
35:52
SPEAKER_05: Uh how do you scale from Alan?
35:56
SPEAKER_05: How do you scale to microwatts?
35:59
SPEAKER_01: All right, okay, very good question.
36:02
SPEAKER_01: So the the most obvious way to do this, right, when you do going now is to go down the technology roadmap.
36:09
SPEAKER_01: So this chip was fabricated at uh 22 nanometer.
36:15
SPEAKER_01: Um, I'm still audible, right?
36:17
SPEAKER_05: Yes, sorry, having a little bit of a layout here.
36:20
SPEAKER_05: Sorry, Sarah's getting hidden by this thing.
36:23
SPEAKER_01: Yes.
36:24
SPEAKER_01: So if I bring it down all the way to three nanometers, right?
36:29
SPEAKER_01: So, I mean, we're talking about like mean five or six generations of and reducing the power, and we're going from one milliwatt, this will be in like in the hundreds of microwatt already, if I just like with this kind of technology trick.
36:41
SPEAKER_01: But I think if you wanted to really sustain that, you have to rethink the data flow, you have to maybe incorporate a little bit of neuromorphic computing, but you have to do it in a scalable manner, right?
36:54
SPEAKER_01: So that when you do things in that front, you don't have to go back and forth inside and outside of the neuromorphic array.
37:02
SPEAKER_01: So it is possible you have to do a number of tricks.
37:05
SPEAKER_01: One of them is involving technology scaling, the other one will be like changing a little bit the data flow.
37:12
SPEAKER_05: Got it, got it.
37:14
SPEAKER_05: I had a quick question too, and then we have another question coming up here, but um, you spent a bunch of time talking about your memory controller.
37:20
SPEAKER_05: Is that so that is that a memory controller that's that was designed by EMAS as part of the system?
37:26
SPEAKER_01: Yes.
37:27
SPEAKER_01: So we have a memory controller inside that's like been designed by us.
37:30
SPEAKER_01: So the memory controller handles the uh the traffic that goes from the processor and the AI accelerators to the memory, and it handles also the transactions that goes to the MRAM or the R RAM to ensure that you don't you don't like mean stress it and cause any kind of endurance failures or kind of uh uh temporary uh read or write failures.
37:51
SPEAKER_05: Okay.
37:52
SPEAKER_05: Yeah, and for folks that don't know, I mean, I think the point you were making too is that a lot of um kind of your power optimization and your performance is in your moving from the AI acceleration logic to the memory logic and back and forth and uh making sure that that's all uh perfectly aligned and in sync.
38:11
SPEAKER_05: Um, because what you see in the out now in the kind of more general purpose computing world is you know completely separate systems for memory and processor and GPU and um that's even why companies like NVIDIA, when they get into data centers, like highly optimize the paths between these different systems to uh you know minimize uh any performance loss.
38:31
SPEAKER_05: So you're doing it sort of at the on-chip at the microphone.
38:35
SPEAKER_00: Very, very small.
38:36
SPEAKER_00: Yes, exactly.
38:37
SPEAKER_00: Yeah, that's awesome.
38:38
SPEAKER_00: That's awesome.
38:39
SPEAKER_05: Okay, before we get you back into your deck, Sarah, did you have any other questions you wanted to throw at him while we're oh yeah, actually, while we're at it, um I see that you mentioned a lot of applications.
38:50
SPEAKER_02: I would love to know kind of what's your sweet spot application where you've seen the technology work really well and really be able to extract the last bit of performance that you want.
39:00
SPEAKER_01: All right.
39:01
SPEAKER_01: Um, I think from from what I uh what I've seen from my experience, right, the app the time series data kind of applications is the one that benefit the most from this.
39:12
SPEAKER_01: Because in in reality, these kind of applications actually do not provide very high data, even the vibration data ones, as long as you're dealing with mechanical or even uh sound, right?
39:22
SPEAKER_01: These are very, very slow compared to the electrical or electronic, uh how electronics operate.
39:28
SPEAKER_01: And that gives you enough window between one sample and the other one that you will be able to run your AI, completely turn off the entire system and wait for it while it gets back on.
39:38
SPEAKER_01: If we're talking about like doing something in like particle accelerators and all that, yeah, that's gonna be very tough uh to do that.
39:45
SPEAKER_01: Videos, um it's it's always complicated, right?
39:48
SPEAKER_01: But if you have a very small, but still images would be uh also a workable solution.
39:54
SPEAKER_01: But when you work with still images, for example, fingerprint, or you're working with like facial identification, right?
40:02
SPEAKER_01: While we see some gains, right, mean a user would not be feeling the difference if you're running if he stays there for one second versus two seconds, he would not be like me seeing much difference in that.
40:13
SPEAKER_01: So time series data, wearables, right?
40:16
SPEAKER_01: Um, anything that uh interacts with this kind of sensors, we see the benefits on that front.
40:23
SPEAKER_05: Cool.
40:24
SPEAKER_05: All right, we have more questions, but why don't we let you finish your talk and then we'll circle back?
40:30
SPEAKER_05: So they're starting to pile up a little bit.
40:32
SPEAKER_05: So we're gonna exit and give you the stage for a minute.
40:36
SPEAKER_01: Okay, okay.
40:37
SPEAKER_01: I will um I will continue, right?
40:41
SPEAKER_01: So as a result, um, of we have here right the chip, just like more of a recap, right?
40:47
SPEAKER_01: With all of these components together, meanwhile we run at 30 uh gigaops, right, at two milliwatts, that gives us an efficiency of 12 trillion operations per watt, right?
40:57
SPEAKER_01: So this is a very, very efficient chip at this kind of uh at this kind of scale of this kind of like mean uh class.
41:05
SPEAKER_01: And we're having around four megabytes of on-chip memory.
41:08
SPEAKER_01: We can provide the compression of the models, and we have fabricated the chip at 22 nanometers, right?
41:13
SPEAKER_01: Which means that chip is actually quite cheap for selling, but also it gives us a lot of leverage as we scale down the technology, and as technology gets depreciated over time, we go down to the 12, the 10, the 6, the 4.
41:25
SPEAKER_01: We will see more and more benefits and even opportunities to have even bigger systems.
41:31
SPEAKER_01: So um, one thing that uh would say, okay, I mean, all of these numbers sounds good, right?
41:37
SPEAKER_01: I mean, did we run things or did we how did we benchmark?
41:40
SPEAKER_01: And we actually use the ML ML Commons uh tiny benchmark suite, right?
41:46
SPEAKER_01: It had like four or five of them, and we compared against the publicly released ones, and we see in a consistent way, we achieve right in terms of like execution time, it means better if it's not the same, right, as the fastest in its class.
42:01
SPEAKER_01: So where there's no kind of uh degradation of any kind of performance, but simultaneously, right, we can get from 20, 10, even 200 times lower energy compared to which platform you are uh I mean that's having this uh releasing their uh benchmarking results.
42:18
SPEAKER_01: So these are two applications for image and visual wave words.
42:21
SPEAKER_01: You can see that we achieve simultaneously 20 times lower energy, and the execution time is within the same range as the fastest.
42:29
SPEAKER_01: The um the actually the the operating point is where the circle of the EMAS is in.
42:35
SPEAKER_01: And we can show that also for other uh benchmarks as shown over here, right?
42:41
SPEAKER_01: Comparing with the best in class for each one of these benchmarks, we consistently the energy gets orders of magnitude, 10, 20, or even some cases 190x, right?
42:51
SPEAKER_01: While in performance, we are actually all doing quite very well, very close or even faster than the current uh state-of-the-art solutions for that.
43:02
SPEAKER_01: So this makes the system right really ideal for solutions, right, in wearables, in drones, right, in IoT and robotics, um, in healthcare and med tech, even some kind of automotive as well in agri tech and environmental sensors.
43:18
SPEAKER_01: So you can apply it in any system that requires that operation on a battery, you need to run it on a very long part uh length time and works with time series data.
43:27
SPEAKER_01: That's where our application or our chip really shines in.
43:33
SPEAKER_01: So now if we take a step back, okay, and we say that now we have a very good chip, we have very amazing energy efficiency.
43:41
SPEAKER_01: But if the user cannot use it very well and do it in a very short time or like requires a significant steep learning curve, right, for it to use this kind of chip, it becomes unappealing, right?
43:55
SPEAKER_01: And we've seen this again and again in the industry.
43:58
SPEAKER_01: So it doesn't become any kind of technological advantage anymore, but how can you become really uh well integrated with the entire ecosystem?
44:07
SPEAKER_01: So in this case, we have actually the development kit, right, that we can give you a virtual board environment.
44:14
SPEAKER_01: So you can be still okay, you say I have an idea, but I don't want to have the chip right now.
44:19
SPEAKER_01: We are providing this kind of remote access to the chips that are sitting in our uh server environment that you can run workloads on it and get the performance right on it.
44:31
SPEAKER_01: If you wanted to have the board and you want to work with the SDK, right, you can take the SDK that's easy compatible with all of our AI development suites.
44:41
SPEAKER_01: So it's compatible with TensorFlow, compatible with PyTorch.
44:44
SPEAKER_01: So if you have your models already there, you can very easily with as simple as a click of a button, right?
44:51
SPEAKER_01: You can create the library that would make things run into our chip, will give you the API that then you would need, you can write it.
44:58
SPEAKER_01: Now you would write the code, you can write it in bare metal or even on an operating system, which is uh one of them is the Zephyr operating system that's actually has a lot of other software layers.
45:10
SPEAKER_01: So if you have a full system that had other applications that not just AI, but you build it on top of Zephyr AS, you can easily port all of that and make it run into our chip.
45:21
SPEAKER_01: At the end of the day, we have a RISC-V core that can best pretty much run any kind of application on it, depending on the performance uh I mean, uh operating point that you would like to have your system running on.
45:34
SPEAKER_01: So, to be able to do this, we also give the opportunity for users to very quickly develop their application.
45:41
SPEAKER_01: So we have this is actually the third generation uh evaluation board, which is around 2.5 plus four centimeters.
45:48
SPEAKER_01: This is a very, very small chip, a very small board connected with USB type C.
45:53
SPEAKER_01: And we have also provided the support that you can integrate sensors, right, with this uh sensor expansion board.
46:01
SPEAKER_01: And that is done actually in collaboration with the tier one component providers that they are providing us the component for the board for the evaluation for the sensor boards, and we have developed this one.
46:12
SPEAKER_01: And I think that I'm happy to announce that in the Things Conference, we should be able to see a live demo running in this new evaluation boards.
46:24
SPEAKER_01: So the sensor board would look something very small like this, right?
46:28
SPEAKER_01: And then you would attach it here, you just like put up put it on it.
46:31
SPEAKER_01: We have developed the libraries, so once you press it on there, you can use the libraries, so it's more of a plug and play, and then you can quickly write have your own prototype for predictive maintenance, for wearables, for cold asset tracking, or for any kind of other kind of sensing environments.
46:47
SPEAKER_01: We will be able to help you using this as a uh using the blueprint of these kind of uh modular boards, and we attach the right sensors, and you should be able to have your entire PUC very quickly, even in the hardware front and on the software, since we provide you the APIs to access any kind of these sensors as well.
47:08
SPEAKER_01: So this brings us now the whole entire thing.
47:11
SPEAKER_01: So we have now the AI, really innovative, right?
47:14
SPEAKER_01: We have the chip that can run things, mean with very low power.
47:17
SPEAKER_01: We have the ability to integrate sensors really efficiently and even from a user perspective, right?
47:21
SPEAKER_01: It doesn't have to take a lot of time.
47:23
SPEAKER_01: But then what about applications?
47:25
SPEAKER_01: So we looked into a number of applications on that front.
47:28
SPEAKER_01: The one that I wanted to focus on is actually in drones.
47:31
SPEAKER_01: As I mentioned earlier, right, we we tackled the drone problem that in a in a new way that we wanted to help the drone to redirect its energies towards increasing flight time rather than enhancing the AI functionality, which is possible with our chip.
47:47
SPEAKER_01: I'm not saying it's not possible, but we wanted to create a bigger impact on that.
47:52
SPEAKER_01: So, we what we have done, we have created a simulation environment using state-of-the-art tools and with hardware in the loop, which means that the drone simulation environment would send data to our chip, our chip would control the virtual drone in this environment, and then we would see compared to a drone that's running with the default controller, how what is the difference in terms of energy consumption and how longer we can fly with the drone here.
48:16
SPEAKER_01: So we found that in actually we can see 60%, sometimes we can reach 80%, right?
48:21
SPEAKER_01: Increasing flight time, that's in just in quad in quadcopters, hexacopters, and in octocopters.
48:27
SPEAKER_01: So we looked in that in different weather conditions.
48:30
SPEAKER_01: So we are now looking into based on this kind of findings, to map into a real drone doing for live trials.
48:36
SPEAKER_01: So we're starting now the work with this, and at the same time, looking for OEM drone manufacturers who are willing to collaborate with us on that.
48:44
SPEAKER_01: So the bottom line is that thanks to the availability of our ship that can run at very low power and getting the time series data right from the drones, such as the altitude, the airspeed, the current orientation, and the target destination, right?
48:59
SPEAKER_01: The current state of charge of the battery, you'll be able to come up with a virtual model of the drone that uses to derive an AI model for what could be the best next action, of course.
49:11
SPEAKER_01: And you're doing all of this right in a power budget of less than two milliwatts.
49:17
SPEAKER_01: So when you're doing this very low power, so any energy is being saved is not consumed by the electronics, right?
49:22
SPEAKER_01: You don't need a very big uh GPU to get that done.
49:26
SPEAKER_01: You can actually do it on our trip, right?
49:28
SPEAKER_01: No problems at all.
49:29
SPEAKER_01: And then you'll be able to get that done, and you have also enhancing functionalities.
49:34
SPEAKER_01: In addition to this, you can even save that energy, do something else with it.
49:38
SPEAKER_01: But the main benefit here is that you can make your drone instead of flying for one hour, you can make it now go for one hour and 30 minutes or even slightly longer than that.
49:48
SPEAKER_01: So that's a huge thing that would help mean helping in missions for drone delivery, for rescue missions, and whatnot, and all of the other use cases that could be possible with a drone and now with this kind of enhanced functionality.
50:01
SPEAKER_01: Another thing that we were looking into also is in the wearables.
50:05
SPEAKER_01: And we have already examined our chip with fault detection, with prediction of mean arrhythmias or sleep rates.
50:13
SPEAKER_01: So we have successfully integrated our chip with an um with all of these kind of like mean ECG sensors and we and gyroscopic sensors and we managed to do some trials on it to me detect that if someone is falling and we can take a quick action, especially in nations where uh population is aging, right?
50:32
SPEAKER_01: And it becomes a problem that there's not enough care for them, and and these people could not be really a technology savvy.
50:39
SPEAKER_01: So if they if something happens to them, right, it raises an issue, right?
50:43
SPEAKER_01: And that stresses the government resources to be able to take care of them.
50:47
SPEAKER_01: So these kind of solutions actually could help not just in mean fault detection, but sometimes it can help in kind of a preventive uh measures.
50:54
SPEAKER_01: So we're looking to mean slowly in penetrate in this field, and we're having also trials, mean looking also uh potential partners on that field as well.
51:05
SPEAKER_01: So the technology at the moment, right?
51:08
SPEAKER_01: So before I conclude, so it's now we have uh developed it, examined it, we have uh all the solutions that can make the technology really sustain uh superior energy efficiency.
51:21
SPEAKER_01: We're even advancing it with newer generations, with a newer chip, with a smaller technology node with enhanced functionality on it.
51:28
SPEAKER_01: And at the moment, we are now into the commercialization phase.
51:32
SPEAKER_01: We're trying to uh work with potential partners to come to come up with POCs and even do some design in.
51:38
SPEAKER_01: And they are already working with some of these partners, uh, some of them actually even tier one electronic components providers that I mean I'm not allowed to uh disclose their name yet.
51:49
SPEAKER_01: But to conclude, right, I mean, just remind you that the idea of EMAS is to look into the entire stack, right?
51:57
SPEAKER_01: And hopefully that by this you would find this inner hidden knobs and you'll be able to achieve significant gains, and that allows you, right, to extend the battery lifetime of the electronic components that even have an impact into uh mean other components in the entire system as well.
52:16
SPEAKER_01: Thank you so much, I mean, for listening, and I'm more than happy to answer other questions.
52:20
SPEAKER_05: Yes, we have lots of questions.
52:22
SPEAKER_05: We do have a few minutes left, so hopefully we can power through some of these things.
52:26
SPEAKER_05: But uh thank you, Mohammed, for uh for that presentation.
52:29
SPEAKER_05: You were uh I could tell leveraging your professorial lecture skills there.
52:33
SPEAKER_05: So that was good.
52:36
SPEAKER_05: Um, a couple things to note, by the way, before we jump into the questions, too, is that uh if people are at the Things Conference in Amsterdam, uh there will be a demo there happening next week.
52:48
SPEAKER_05: And then key uh key item too is Muhammad himself will be at our Taipei event in person.
52:54
SPEAKER_05: So if you want to meet uh the professor and the founder, uh come to the Taipei event and uh and you'll be able to um see him there.
53:05
SPEAKER_05: And there's our little uh banner for getting your tickets to the Taipei event in November, November 11th and 12th.
53:11
SPEAKER_05: So you can meet Muhammad in person.
53:13
SPEAKER_05: Okay.
53:15
SPEAKER_05: Uh let's see.
53:16
SPEAKER_05: Questions, questions.
53:21
SPEAKER_05: Well, you mentioned this before.
53:22
SPEAKER_05: Uh so just a question from um uh hybrid robotics uh N7 PKT.
53:31
SPEAKER_05: So you have the virtual dev kit as well as MCU boards that people can purchase, right?
53:36
SPEAKER_05: So those are currently available.
53:38
SPEAKER_00: Yes.
53:40
SPEAKER_05: Cool.
53:41
SPEAKER_05: All right, good.
53:41
SPEAKER_05: So and you can go to your website to get it.
53:45
SPEAKER_01: Yes, indeed.
53:45
SPEAKER_01: Like if someone is very interested, I mean we will uh help him to set up the the connection and uh he will be able to develop his application and run uh perform the live runs remotely, and then he can see the performance of it.
53:58
SPEAKER_05: Yeah, here's a question that um I wasn't quite sure how to interpret this.
54:03
SPEAKER_05: The attention layers.
54:05
SPEAKER_05: What attention layers does your AI model mapping flow support?
54:09
SPEAKER_01: So oh okay.
54:11
SPEAKER_01: That's uh that's uh that's actually been an uh interesting question.
54:15
SPEAKER_01: So at the moment, right, we are uh doing the uh convolution layers and we're looking into uh all the attention layers are now being running on the uh RISC-V core.
54:26
SPEAKER_01: So the attention layers can run, right, as any kind of uh any kind of application, but it doesn't have any kind of hardware support yet, right?
54:36
SPEAKER_01: We are developing our newer uh I mean AI accelerator that would look into that, but given that our main focus was an edge AI, right?
54:45
SPEAKER_01: And uh transformers or generative AI uh models, right?
54:49
SPEAKER_01: Do not run on that platform yet.
54:51
SPEAKER_01: So that's why we do not have at the moment running on the attention layers, but convolutional layers, any kind of vector matrix layer, right?
54:59
SPEAKER_01: That would work very well on our accelerators.
55:02
SPEAKER_05: Cool.
55:04
SPEAKER_05: Good.
55:04
SPEAKER_05: Uh here's a question from Alex Miller.
55:07
SPEAKER_05: Uh got a thumbs up on this one.
55:10
SPEAKER_05: So for two DL accelerators, uh, network arc architecture is hard coded while weights are fully programmable.
55:17
SPEAKER_05: Uh, can you mention what type of network architecture is hard-coded?
55:22
SPEAKER_01: Um, actually, no, we we don't have any kind of hard-coded.
55:25
SPEAKER_01: So you can run uh mean kernels, like mean in like mean if we talk, let's take convolution for example.
55:30
SPEAKER_01: You can do one cross three, three cross one, uh three cross three, five cross five, seven cross seven.
55:35
SPEAKER_01: You can do residual networks, right?
55:37
SPEAKER_01: The idea of having two accelerators that one of them can actually do for these kind of shallow layers, right?
55:45
SPEAKER_01: If you're doing uh depth for depth uh first convolutions, right?
55:49
SPEAKER_01: So one of them is actually really, really superior in that compared to the other one, while the other one handles more dense connection much more efficiently.
55:57
SPEAKER_01: So the two of them have this kind of nice interplay between each other.
56:01
SPEAKER_01: Um, but then you can basically run any kind of net any network as long as they uh mean fit in the convolution and in any kind of fully connected layers.
56:12
SPEAKER_05: Cool.
56:12
SPEAKER_05: Uh Sarah, any of these questions here catch your eye for the next one?
56:19
SPEAKER_02: I'm muted, let me see.
56:21
SPEAKER_02: Oh I like the question about the enablement as well.
56:26
SPEAKER_02: So if you can comment on that.
56:28
SPEAKER_05: Which one's that?
56:29
SPEAKER_02: Enabling how do you enable building AI applications?
56:32
SPEAKER_02: Exactly.
56:35
SPEAKER_05: All right.
56:36
SPEAKER_01: Okay, so there are, I mean, there are two approaches for this.
56:39
SPEAKER_01: Let's say that you are someone who knows how to code your own AI application, right?
56:44
SPEAKER_01: And you develop your own application using PyTorch or TensorFlow.
56:47
SPEAKER_01: So all what you need to do is to have this, the the model, right, that you have, and then you would run it into our mapping uh mapping tool.
56:55
SPEAKER_01: So which basically it's a command line, you give it the latest checkpoint with the weights, right, and the biases, and you click convert, it does the conversion for you, and it will give you a library file and that could that would program the AI accelerators, another file that would load all the AI weights with the bias conversion.
57:14
SPEAKER_01: So then it will give you an API, right?
57:17
SPEAKER_01: Like use this API in your main code and your main application.
57:21
SPEAKER_01: At the end of the day, you got you we run in C, right?
57:24
SPEAKER_01: If you if you want to run it on PyTorch or you want to run directly on Python, you have to run it on top of the Zephyr operating system, which is still in the same approach, but then we tell you this is the API that you need to use, and you just need to import that library into Python.
57:38
SPEAKER_01: So that is one if you are someone who has developed his application.
57:42
SPEAKER_01: But let's say that you, someone who just have data and you don't even know how to code your application, right?
57:48
SPEAKER_01: This is something that we are uh we have a uh uh let's say a beta version of, but we are working to make it very mature through the what's a concept of called neural architecture search.
57:59
SPEAKER_01: So we take the data and we build the AI model for you, right?
58:03
SPEAKER_01: We let the AI search and come up with an AI model for you based on your accuracy metrics and satisfying the constraints of our chip.
58:11
SPEAKER_01: So we look at say our chip has this amount of memory, so you can only have this amount of parameters, right?
58:17
SPEAKER_01: It tried to select from different predefined architecture skeletons or scaffolds, then it tried to program or try to train some of them, it comes up with the one that gives you the best uh accuracy.
58:28
SPEAKER_01: Or you, if you want the entire parent of frontier, it will can spin it out for you and tell you, okay, which one would you like to choose?
58:34
SPEAKER_01: But this is for someone who's still exploring this a little bit further, and I think we're not the only ones, there are other companies doing this to expand the adoption of AI and edge AI by companies and use cases that do not have this already integrated into the system or enabled yet.
58:52
SPEAKER_05: So using AI to help develop the AI models, yeah, basically.
58:57
SPEAKER_05: What we have we're about to run out of time, but I had one more question that jumped up here.
59:01
SPEAKER_05: I'm a big fan of WASM, long story, but uh someone brought up uh do you have a WASM runtime for your hardware?
59:10
SPEAKER_01: Um, I think we haven't done that yet, right?
59:13
SPEAKER_01: But I mean we uh it's one of the things that we're looking into.
59:17
SPEAKER_01: Okay, cool.
59:18
SPEAKER_05: Yes, I'm a big fan of that.
59:19
SPEAKER_05: Uh cool.
59:20
SPEAKER_05: All right, great.
59:22
SPEAKER_05: We are at about time.
59:23
SPEAKER_05: I would say, you know, call to action.
59:25
SPEAKER_05: Uh, you know, go to uh Mohammed's website, uh EMAS, and um is it nanoview.com or what's the what's the yeah nanoview.com slash emass.
59:36
SPEAKER_05: Yeah, there it is.
59:36
SPEAKER_05: Nanoview.com slash emass.
59:39
SPEAKER_05: Uh come see uh in Amsterdam next week.
59:42
SPEAKER_05: Some folks from uh EMass that'll be there, meet Mohammed in person um and Taipei and um follow us on YouTube if you haven't.
59:52
SPEAKER_05: I think we're like about five users, uh subscribers short of 50,000 right now.
59:57
SPEAKER_05: So uh for everyone who's uh on the live stream.
1:00:00
SPEAKER_05: And subscribed, I think we'd we'd reach our number there.
1:00:03
SPEAKER_05: But uh really appreciate the time, Mohammed.
1:00:05
SPEAKER_05: Thank you, Sarah, uh, for co-hosting here.
1:00:08
SPEAKER_05: Right.
1:00:08
SPEAKER_05: And uh yeah, fantastic.
1:00:10
SPEAKER_05: Really appreciate it.
1:00:12
SPEAKER_01: Right.
1:00:12
SPEAKER_01: Thanks, B, for the opportunity.
1:00:14
SPEAKER_01: Yeah, pleasure with you, Sarah.
1:00:15
SPEAKER_01: And thanks everyone for uh attending.
1:00:18
SPEAKER_04: Cool.
1:00:19
SPEAKER_04: All right, bye-bye.
1:00:20
SPEAKER_01: Bye.