Latest / EDGE AI POD / What happens when AI learns from the fire hose—and tests itself on silicon

Transcript

What happens when AI learns from the fire hose—and tests itself on silicon · EDGE AI POD

• 0:33
  SPEAKER_01: Good morning.
• 0:35
  SPEAKER_01: Good afternoon.
• 0:39
  SPEAKER_00: Yeah, well, yeah.
• 0:40
  SPEAKER_01: You live there.
• 0:41
  SPEAKER_00: So I live here.
• 0:42
  SPEAKER_00: And I was we it was great seeing you last week at the Things Conference.
• 0:46
  SPEAKER_01: Yeah, no, that was great.
• 0:47
  SPEAKER_01: I you know I haven't been to Amsterdam in a few years.
• 0:50
  SPEAKER_01: And um yeah, it's such a nice city.
• 0:53
  SPEAKER_00: And uh did you get into the center though?
• 0:55
  SPEAKER_00: Because where we were at for the conference was not really.
• 0:58
  SPEAKER_01: Yeah, no, I did a little extra time.
• 1:00
  SPEAKER_01: I went to the week museum with all the things.
• 1:04
  SPEAKER_01: And uh yeah, and it was a great event.
• 1:06
  SPEAKER_01: So hats off to Nikki Geisman and team over there at the things.
• 1:12
  SPEAKER_01: And uh yeah, it was great.
• 1:14
  SPEAKER_01: It was the first time we had brought um a bunch of the edge AI community there.
• 1:17
  SPEAKER_01: And I know HMP has been a long time supporter as well.
• 1:20
  SPEAKER_00: But um And the Scavenger hunt was awesome.
• 1:23
  SPEAKER_00: I thought it was such a great idea because this was actually the year they had like double the partners, double the booths.
• 1:29
  SPEAKER_00: It was huge this year.
• 1:30
  SPEAKER_00: It was it was really fun.
• 1:31
  SPEAKER_00: So I can't even imagine what they're gonna do next year.
• 1:33
  SPEAKER_01: I know, no, it was good.
• 1:35
  SPEAKER_01: And we had like I was on four panels myself.
• 1:37
  SPEAKER_01: There's a bunch of AJ panels and uh you're a busy man, Pete.
• 1:42
  SPEAKER_01: It was a busy week.
• 1:43
  SPEAKER_01: Um, but um yeah, it's you know, the over I was telling people the intersection of low power wireless and low power AI is kind of a good overlap.
• 1:52
  SPEAKER_01: So I think cross-pollination there, so that's fun.
• 1:55
  SPEAKER_01: So I'm DJET lagged.
• 1:57
  SPEAKER_01: And um apologize to some of the folks that joined earlier.
• 2:00
  SPEAKER_01: We had a little scheduling.
• 2:02
  SPEAKER_00: Yeah, I saw the comments.
• 2:03
  SPEAKER_01: There was people here like I think we'd originally accidentally scheduled it for 7 a.m.
• 2:08
  SPEAKER_01: Um which we always have at Tuesdays at 8 a.m.
• 2:11
  SPEAKER_00: And so, but you know, blame always blame it on like daylight savings time, you know?
• 2:20
  SPEAKER_01: Uh okay, cool.
• 2:22
  SPEAKER_01: So we're here and we're going to bring on Evan and um we'll talk about um you know his topics, and they recently rebranded.
• 2:31
  SPEAKER_01: So some people may have known them as Eta Compute, but now they're called Mod Compute, which I think is yes, and I was just gonna say I think that's a much cooler name.
• 2:41
  SPEAKER_00: Their logo is also cooler, but I look it was it's so different.
• 2:44
  SPEAKER_00: I was like, wait, who is this?
• 2:48
  SPEAKER_00: But it's so exciting to have the CEO on today, Evan.
• 2:52
  SPEAKER_01: Yes.
• 2:53
  SPEAKER_01: Oh, let me do a quick DSA before we bring Evan on.
• 2:56
  SPEAKER_01: So if folks have not if you're in the area of Taiwan in November uh or or are interested in going, we're gonna have a two-day event in Taipei November 11th and 12th, and um AJI Taipei 2025.
• 3:11
  SPEAKER_00: I might be there.
• 3:12
  SPEAKER_01: Cool, cool.
• 3:13
  SPEAKER_01: Yeah, here I'll show the little banner.
• 3:15
  SPEAKER_01: Um if you go to that URL, there's still some seats available.
• 3:18
  SPEAKER_01: It's a bit of a limited seating thing, but um we're gonna have uh, well, Jenny might be there, Edge Impulse, but we're gonna have, you know, your parent company, Qualcomm, I think will be there too.
• 3:29
  SPEAKER_01: Um, NXP, ST Micro, Dell Technologies, Advantech, Intel, Climax, you know, and then we're gonna have professors from NTU and NTCU uh and a bunch of cool uh presentations on research.
• 3:43
  SPEAKER_01: And we're gonna have a cool mixer there.
• 3:46
  SPEAKER_01: Uh, we're gonna have a panel on startups, uh, a panel on deploying and and commercializing edge AI.
• 3:53
  SPEAKER_01: So really kind of a packed couple of days to meet who's who in the space and learn what's what.
• 3:58
  SPEAKER_01: Um so go to that URL and get some tickets and uh and I'll see you there.
• 4:02
  SPEAKER_01: So that'll be good.
• 4:03
  SPEAKER_01: So that's my that's my PSA.
• 4:05
  SPEAKER_01: Um now that we've done the PSA.
• 4:08
  SPEAKER_00: Well, another quick PSA.
• 4:10
  SPEAKER_00: If you have any questions today, please ask them in the chat and we will answer them live um at the end of Evan's presentation.
• 4:17
  SPEAKER_01: So yes, yes.
• 4:19
  SPEAKER_01: And we may even interrupt Evan depending on depending on how juicy the question is.
• 4:24
  SPEAKER_01: You kind of kind of kind of bundled in there.
• 4:27
  SPEAKER_01: But uh yeah, definitely.
• 4:28
  SPEAKER_01: This is interactive, and this is your opportunity to talk to the experts on what they're talking about.
• 4:33
  SPEAKER_01: So, yes, please do that.
• 4:35
  SPEAKER_01: Uh, why don't we bring Evan on board here?
• 4:37
  SPEAKER_01: There he is.
• 4:38
  SPEAKER_01: Good morning.
• 4:40
  SPEAKER_01: Good morning, Evan.
• 4:41
  SPEAKER_01: Yes.
• 4:42
  SPEAKER_01: Excellent.
• 4:44
  SPEAKER_01: Um, so yeah, thanks for joining us.
• 4:47
  SPEAKER_01: Uh let me get you into the guest configuration here.
• 4:51
  SPEAKER_01: So let's see.
• 4:54
  SPEAKER_01: I believe you lost you for a second.
• 4:57
  SPEAKER_01: There it is.
• 4:58
  SPEAKER_01: No, but I want to give you a guest spot.
• 5:00
  SPEAKER_01: There you go.
• 5:00
  SPEAKER_03: Here I am.
• 5:01
  SPEAKER_01: Okay, you got the prime time spot.
• 5:04
  SPEAKER_01: Thanks.
• 5:08
  SPEAKER_01: So you're the CEO of Model Cat.
• 5:12
  SPEAKER_01: Yep.
• 5:13
  SPEAKER_01: Welcome.
• 5:14
  SPEAKER_03: As you said, people may have known us as Etacompute, but uh we recently rebranded as Model Cat.
• 5:23
  SPEAKER_03: Wanted to really reflect our new mission, um, which we'll be hearing about a bit today.
• 5:31
  SPEAKER_01: Okay.
• 5:31
  SPEAKER_01: What was the is there a backstory to Model Cat?
• 5:34
  SPEAKER_01: Are you a cat person or what's what's this?
• 5:36
  SPEAKER_03: Uh yeah, you know, well, you know, like all of these things, we're back annotating the myth in real time.
• 5:43
  SPEAKER_03: So uh I am a cat person, and I do look, you know, uh I do have a cat, and in fact, we've got quite a few cats in our company.
• 5:52
  SPEAKER_03: Okay.
• 5:53
  SPEAKER_03: Um, but it was more a thematic thing, you know, the the the the way cats always land on their feet and have this kind of poise, do stuff that that seems difficult, but with you know, out getting their whiskers out of joint.
• 6:10
  SPEAKER_03: Um, so we we had this idea of of using the cat for that reason.
• 6:16
  SPEAKER_03: Got it, got it.
• 6:17
  SPEAKER_03: And you're based in Sunnyvale, I think you said, Yeah, in Sunnyvale in California, Silicon Valley.
• 6:22
  SPEAKER_01: The heart of Silicon Valley.
• 6:24
  SPEAKER_01: So that's exciting.
• 6:25
  SPEAKER_01: That's right.
• 6:25
  SPEAKER_01: Yeah.
• 6:27
  SPEAKER_01: Well, appreciate you uh joining us today and enlightening the audience, as Jenny said.
• 6:32
  SPEAKER_01: Uh throw some questions in the chat.
• 6:34
  SPEAKER_01: We may uh chime in uh on occasion if we get a good bundle of questions.
• 6:40
  SPEAKER_01: But um again, you had a I think you have a presentation you were going to go through, and you have a live demo too, right?
• 6:45
  SPEAKER_01: So yeah, I've got a pro and a demo.
• 6:48
  SPEAKER_01: Yeah.
• 6:48
  SPEAKER_01: We like live demos here because that's yeah, let's see it working.
• 6:52
  SPEAKER_01: Yeah.
• 6:53
  SPEAKER_01: Okay, let's bring that up.
• 6:54
  SPEAKER_01: Um, yeah, cool.
• 6:56
  SPEAKER_01: So I think Jenny and I will kind of hang back a little bit.
• 6:58
  SPEAKER_01: Okay.
• 7:11
  SPEAKER_01: Speaking of rebranding, in fact, I was sharing with Evan before that we got started, how it's been about a year where we changed the name from Edge AI Foundation to TinyM from TinyML to edge AI.
• 7:22
  SPEAKER_01: So yes.
• 7:24
  SPEAKER_00: Weren't you the mastermind of that decision too?
• 7:26
  SPEAKER_01: Uh yes, I guess I was.
• 7:27
  SPEAKER_01: Yeah.
• 7:28
  SPEAKER_00: I think it was a good choice because it really does expand what we can talk about and do.
• 7:32
  SPEAKER_01: Yeah, yeah.
• 7:32
  SPEAKER_01: And then we we have been, yeah, there's all kinds of generative edge AI things happening to computer vision and all kinds of goodies.
• 7:40
  SPEAKER_01: And uh yeah, so rebranding is uh, you know, is important.
• 7:44
  SPEAKER_01: So I appreciate that uh Evan and the team have made the leap.
• 7:47
  SPEAKER_01: But yes, we did our own rebranding last year.
• 7:49
  SPEAKER_01: So we're still we still still say from tiny ML to the edge of AI.
• 7:52
  SPEAKER_01: We love tiny ML, and it's it's a critical part of this whole thing.
• 7:55
  SPEAKER_01: It's a container.
• 7:58
  SPEAKER_01: So but yeah, okay, cool.
• 7:59
  SPEAKER_01: So I'll just I'll just do that.
• 8:01
  SPEAKER_01: Okay, so we'll step back a little bit.
• 8:02
  SPEAKER_01: Evan will give you the floor and uh take it away.
• 8:06
  SPEAKER_03: Great.
• 8:07
  SPEAKER_03: Well, thanks for having me this morning and um welcome to everyone watching.
• 8:12
  SPEAKER_03: Uh appreciate your time.
• 8:14
  SPEAKER_03: Um, I'm Evan Petritis, and I'm the CEO of Model Cat.
• 8:19
  SPEAKER_03: And as you just heard, we formally were called Eda Compute.
• 8:23
  SPEAKER_03: If you've been in this game for a while, you may have heard of Edacompute because um we were one of the originally one of the um inference, edge inference silicon companies.
• 8:36
  SPEAKER_03: But we pivoted away from that, and we we have a new mission, and the name Model Cat comes along with the new mission actually.
• 8:46
  SPEAKER_03: It and our really our goal now is to build tooling that gives you truly autonomous AI model production, and um, you know that that is uh actually a pure software um uh sort of proposition.
• 9:06
  SPEAKER_03: Um, but we actually have a bunch of hardware components that you'll see a little bit of as we go through today that that are part of our uh model cat solution.
• 9:18
  SPEAKER_03: Um, and they they you will see echoes of the um original origin story in that, probably.
• 9:28
  SPEAKER_03: So I wanted to spend a few slides and then we'll get to the demo.
• 9:33
  SPEAKER_03: Um, and I'll leave plenty of time for some questions at the end.
• 9:39
  SPEAKER_03: And I guess they're going to interrupt me if you if they get a really good one.
• 9:42
  SPEAKER_03: Um, but a few slides first to sort of explain our philosophy and what we're trying to do here.
• 9:50
  SPEAKER_03: Starting from what we see as one of the key observations, which is the velocity of change in uh in the AI domain.
• 10:00
  SPEAKER_03: So, you know, I've kind of been in the technology industry for a long time and seen, you know, I was very lucky to be part of the internet boom, see see the internet grow from um you know an academic network up to you know what what it is today.
• 10:16
  SPEAKER_03: And that was a lot of change, but it never had the velocity that AI technology, uh velocity of change that that AI technology has seen.
• 10:30
  SPEAKER_03: And you know, it's it's really like we're seeing a new innovation every day.
• 10:34
  SPEAKER_03: And the the problem that development teams basically have is they see you might read an abstract for a paper, or you might read a press release or a data sheet from a from a chip vendor, and then if you're trying to actually develop products, that very often the problem the question in your mind is is this innovation important enough for me to pay attention to?
• 11:02
  SPEAKER_03: Will this give me some kind of meaningful business outcome that I need?
• 11:09
  SPEAKER_03: And you know, examples of these things, you know, that every every day there's there's more neural network architectures.
• 11:16
  SPEAKER_03: The the chip guys have been extremely busy with different hardware architectures.
• 11:20
  SPEAKER_03: We're in the early, you know, we're in the in the phase where the the hardware architectures are still proliferating, actually, even today.
• 11:28
  SPEAKER_03: Um, you know, it could be something like you know, a new approach for a training optimizer or uh, you know, training scheduling.
• 11:39
  SPEAKER_03: So there's this there's basically a fire hose of research and new ideas in the field.
• 11:47
  SPEAKER_03: The issue is that that these are the are, or some of these at least are probably applicable immediately, but it's very hard to know whether you should invest your time in investigating, even reading the whole paper.
• 12:03
  SPEAKER_03: Should I read the whole paper?
• 12:05
  SPEAKER_03: It's a question that is actually a hard question for any of us to answer.
• 12:10
  SPEAKER_03: So this is this is like a characteristic of a of a technology that is got super high velocity and kind of unlike anything that we've seen in the past.
• 12:24
  SPEAKER_03: And you know, uh the other the background environment to this is if you're trying to build products, there's a pretty well established uh toolkit for how you do good product development, you know, you have good system architecture, you do decomposition.
• 12:45
  SPEAKER_03: This is all engineering, come software engineering kind of motherhood and apple pie.
• 12:53
  SPEAKER_03: You build in testability, you stand on the shoulders of the giants, you use platforms and mega components, you know, those platforms that we use all the time.
• 13:06
  SPEAKER_03: You maybe don't even think about the operating systems or the or the IP protocol stack, the whole internet stack.
• 13:15
  SPEAKER_03: Um, you you do a lot really of incremental improvement.
• 13:20
  SPEAKER_03: So trying to reuse because it's so hard to get stuff to work in the real world, that once you've got it to work, you don't want to lose that.
• 13:29
  SPEAKER_03: So there is a lot of building on modules, on components, and so on that you've already proven out or that have been shown to work.
• 13:39
  SPEAKER_03: So this is all great.
• 13:41
  SPEAKER_03: This is all great, but you spray a fire hose fire hose of AI innovation at this, and none of these kind of uh mechanisms and none of the processes in traditional product development are really capable of responding and absorbing those innovations fast enough.
• 14:01
  SPEAKER_03: It's uh it's it's really uh pretty tough situation if you're in the game of trying to build real real products.
• 14:11
  SPEAKER_03: So we wanted to address this, and so of course, you know, we aren't the first people to observe that this is a problem, this mismatch.
• 14:24
  SPEAKER_03: And and you know, people have tried many approaches, and some of the approaches, there's there's actually you know, a couple of other major ones as well, but some of the approaches are you know, better libraries, you know, maybe maybe if I make my compiler better, or maybe if I make my um you know my my training setup better, or my cluster management better.
• 14:47
  SPEAKER_03: These are this component by component improvement, it's part of the story, but it's not really been enough.
• 14:55
  SPEAKER_03: Another sort of uh angle is the the whole product angle, which is okay, the the issue here is that we just need an end-to-end tool chain which gives a development team everything they need, and that gets part of the way as well, but but has not been dramatically successful.
• 15:15
  SPEAKER_03: And of course, the one that that that many players in the in the game come back to because it's actually kind of known to work is here's it here's a can solution.
• 15:26
  SPEAKER_03: Okay, here's a can solution for X, for problem X.
• 15:30
  SPEAKER_03: Um this is known to work, and P it works, but it gives you these spot points in the in the product space where you've got a solution, then to get from that solution to actually something that you this useful for your application is not always easy.
• 15:46
  SPEAKER_03: So none of these have really been broadly successful.
• 15:51
  SPEAKER_03: And we set out to ask how we can actually solve that problem today.
• 15:58
  SPEAKER_03: Um, I really like this this this stock image, or don't like it, in the sense that you know they've put it at the word AI on a chip.
• 16:08
  SPEAKER_03: It must be easy, hey?
• 16:09
  SPEAKER_03: That's it.
• 16:10
  SPEAKER_03: There there's there's the picture.
• 16:12
  SPEAKER_03: So that's in some way that's what we want, but it's very it's not it's been proven to be not very easy.
• 16:20
  SPEAKER_03: But we have one hugely powerful weapon that that you know we have never had in the past to help to solve this problem, and that is AI itself.
• 16:35
  SPEAKER_03: Using AI as the core of the tooling for this to help solve this.
• 16:44
  SPEAKER_03: How do I build AI models for a particular application on a particular chip?
• 16:49
  SPEAKER_03: That is the key to um making this a broadly accessible and usable solution for across a variety of different domains.
• 17:02
  SPEAKER_03: And that's what I'm going to show you today.
• 17:04
  SPEAKER_03: That's what Model Cat is.
• 17:07
  SPEAKER_03: So, you know, as usual, AI is really critical and is the linchpin of this, but it's not a magic wand.
• 17:17
  SPEAKER_03: And anyone who you know uses any of the coding tools or even any of the chat chat AIs today knows that you need some scaffolding.
• 17:34
  SPEAKER_03: You can't just throw a general purpose AI at a problem where you actually want a very deterministic and well-characterized output, which is really what we want if we're trying to build a product.
• 17:49
  SPEAKER_03: And and so AI itself is not really a magic wand, but it's the central component.
• 17:56
  SPEAKER_03: And we've built this scaffolding and kind of characterized it as five pillars that you need for AI-created AI.
• 18:10
  SPEAKER_03: So the very first, and probably the I think the most important one, and and look, a lot of the uh the chat bots are now getting to this, is that you must have a closed loop around the AI.
• 18:22
  SPEAKER_03: That is, the AI is working towards your the user's goal in in a closed loop until it reaches their heads.
• 18:32
  SPEAKER_03: And so, of course, then that brings up how do you limit how much resource it's going to use and all of those sort of things.
• 18:38
  SPEAKER_03: Yeah, you have to think about that.
• 18:40
  SPEAKER_03: But this is really the this the central architectural um uh characteristic.
• 18:48
  SPEAKER_03: The second thing is it must be grounded in reality.
• 18:53
  SPEAKER_03: You know, this is we're not in the game of dealing with AI hallucinations or inaccuracies or estimates.
• 19:02
  SPEAKER_03: We need the AI to be continuously grounded in reality, and you know, I use that term very broadly.
• 19:10
  SPEAKER_03: The reality of what the chip can do, the reality of how much memory you've got, the reality of how much power it's using, it must be grounded in reality.
• 19:19
  SPEAKER_03: And I'll tell you a bit about how we do all these things in a minute.
• 19:24
  SPEAKER_03: Third thing is it's got to think in systems, you know.
• 19:27
  SPEAKER_03: Um, you know, my I'm I've got a sort of systems, I'm actually a hardware engineer originally, and I've got a systems background.
• 19:34
  SPEAKER_03: And the the the thing that we need to do here to make this broadly applicable is is provide a way that you can tell the sys the tool or the platform, Model Cat in this case, you can tell it in a natural way what you want out of it.
• 19:55
  SPEAKER_03: So this is not the same as giving it lots of little micro instructions about what neural network architecture you want to use, or exactly which transfer learning data set you want to use.
• 20:09
  SPEAKER_03: It's here's what I need from you.
• 20:13
  SPEAKER_03: Um, and you know, it's an input.
• 20:16
  SPEAKER_03: Now you give it to me.
• 20:18
  SPEAKER_03: The fourth thing is all about that fire hose.
• 20:21
  SPEAKER_03: We need a way uh that the system the system is modular enough that it can learn from this fire hose of new research, and that that can be highly systematized and highly automated, that learning.
• 20:36
  SPEAKER_03: And then the last one is that it needs to be a trustworthy single-step process, and um, you know, trustworthy in the sense that the user really trusts it.
• 20:50
  SPEAKER_03: And of course, that's something that builds up over time with using any any any of these kinds of um AI systems.
• 20:58
  SPEAKER_03: Okay, so now let me uh just skip over to very quick kind of demo walkthrough.
• 21:10
  SPEAKER_03: So ModelCAD itself is a SaaS or cloud-based system, and so I'm running this.
• 21:21
  SPEAKER_03: This is in my browser.
• 21:22
  SPEAKER_03: I'm running this in my browser, and you can see here the data sets page.
• 21:26
  SPEAKER_03: You know, this is showing a variety of different data sets.
• 21:30
  SPEAKER_03: And today I'm just gonna concentrate, show you one particular data set.
• 21:36
  SPEAKER_03: So, this is a data set that's a public and open source data set that's been bopping around.
• 21:42
  SPEAKER_03: Um, it's pill detection.
• 21:45
  SPEAKER_03: And so, you know, you can probably see that the um they've got a variety of you know, somewhat difficult um backgrounds and and distractions for the system to try to find pills within these images.
• 22:00
  SPEAKER_03: It's a good example, just simply because it's easy to understand.
• 22:04
  SPEAKER_03: So, within Model Cat, of course, there's a way that you can upload your data, and and this data set has been uploaded by one of our engineers actually.
• 22:15
  SPEAKER_03: Um the system provides a bunch of analytics on your data that helps you to quickly understand if you've got the typical sort of data SKUs, and and our AI will sometimes give you a warning that says that, like in this case, it's saying that it's found some distributional shift between the training and validation splits.
• 22:41
  SPEAKER_03: So there's there's the system automatically analyzes your data set, and then we'll give you warnings and sometimes even up to the error level if if there's um things that are wrong with your data set.
• 22:57
  SPEAKER_03: Um, this is all the a pretty simple uh press button process to get that data in.
• 23:07
  SPEAKER_03: Now, one other kind of core input into our model creation, or as we call it, the model build process, is our silicon library.
• 23:19
  SPEAKER_03: So we we have a variety of chips that have been onboarded onto Model Cat.
• 23:27
  SPEAKER_03: When I say onboarded, that means that Model Cat has characterized each of these chips for their ability to run these neural network models.
• 23:39
  SPEAKER_03: And it knows just about everything that you could need to know about the kinds of operations that are supported, how fast they are, how um how much memory they take, what the parameters that they accept are.
• 23:55
  SPEAKER_03: So you can think of this as a very detailed, highly automated profiling of the chip and its low-level runtime.
• 24:04
  SPEAKER_03: You know, most often there's a runtime that the chip vendor makes, and sometimes, you know, in some cases there's a there's specific kernels that chip provider provides, and in other cases there's actually uh compilation uh code or compilation tools that that the chip provider provides.
• 24:26
  SPEAKER_03: So all of that is characterized by model cat.
• 24:30
  SPEAKER_03: So you can think of this as the library of chips that you can target for your model inference.
• 24:43
  SPEAKER_03: And then when it comes to actually asking model cat to build models for you, it's a very simple process.
• 24:53
  SPEAKER_03: So you you basically can specify the um the data set here.
• 25:05
  SPEAKER_03: So, you know, so I might sp I might take this pills data set that we were looking at.
• 25:12
  SPEAKER_03: So I've just selected the data set, which is one of those data sets that we looked at earlier.
• 25:18
  SPEAKER_03: The system you might have noticed then that it auto-populated a bunch of stuff.
• 25:22
  SPEAKER_03: So this is this is the internal AI AI at work.
• 25:26
  SPEAKER_03: It knows something about that Pills data set because of all the analytics, and it makes a bunch of decisions about the best way to set up to build a model for that.
• 25:37
  SPEAKER_03: But the one thing that I still haven't told it, which is critical, is which chip do I want to target?
• 25:43
  SPEAKER_03: So here I have again that same library of chips.
• 25:48
  SPEAKER_03: So maybe I choose at this NXP IMX8M Plus chip.
• 25:56
  SPEAKER_03: And once I choose that chip, this left-hand pane is really everything I need to tell the system.
• 26:04
  SPEAKER_03: That's the the essentials of what I need to tell the system.
• 26:09
  SPEAKER_03: I want that model, I want models built to solve this PILS data set on this chip.
• 26:16
  SPEAKER_03: If I like, I can optionally specify.
• 26:21
  SPEAKER_03: Oh, I would like you know, I'd like the inference time to be less than five milliseconds, or I or I perhaps I've got a battery-powered device, and so I'm very worried about energy use or power.
• 26:33
  SPEAKER_03: So then I can specify the energy per inference.
• 26:36
  SPEAKER_03: So these are constraints that you can give the system, which are optional, and you can see here that there's a little AI advisor pane down at the bottom here.
• 26:50
  SPEAKER_03: So it it I don't know if you can see it on the on your if it's big enough, but it's saying that the job success prediction for this is high, which means that with this set of um constraints, that data set on this particular chip, and with the constraint of say a five millisecond inference time, it's it's expecting that it's a high chance of getting success on that data set.
• 27:17
  SPEAKER_03: Um before we leave this page, I just want to show you the advanced options.
• 27:26
  SPEAKER_03: So that when I click the advanced option button, there's a huge number of panes that pop up, and you can you can get inside and start giving the thing much more detailed directions.
• 27:41
  SPEAKER_03: So you know you can you can specify the input color space, you know.
• 27:47
  SPEAKER_03: If you if you think you know better than what the AI knows, then you can specify resolutions for the inputs, you can um specify target values for for accuracy, how much memory you want to give it, any of these parameters, and then a whole bunch of stuff about about how it does um the training and optimization.
• 28:13
  SPEAKER_03: Let's turn that off.
• 28:15
  SPEAKER_03: Now I wanted to so you you know when when you hit this create button, the system goes off and it does a lot of things.
• 28:24
  SPEAKER_03: And I just wanted to, I'm just gonna skip to back to a slide for a moment, just to tell you the things it's doing.
• 28:33
  SPEAKER_03: So behind that button, the sys the system is understanding, taking its understanding of your data set and its understanding of all everything it knows about the chip that you specified as a target, and it's starting to ask itself questions.
• 28:53
  SPEAKER_03: It's starting to ask itself questions about what would what would good model architecture be for this.
• 28:59
  SPEAKER_03: So this is neural architecture search.
• 29:02
  SPEAKER_03: It's what would a good transfer learning uh data set be for this if I have one?
• 29:11
  SPEAKER_03: Um, and then then it does some testing of those hypotheses, and this is really important for that grounding thing that I mentioned.
• 29:23
  SPEAKER_03: The system has access to what we call the chip farm, which is literally a hardware array of all of the chips that we've onboarded.
• 29:34
  SPEAKER_03: So, in that case, I was using the NXP IMX8M Plus.
• 29:39
  SPEAKER_03: So, within our chip farm, we have a bunch of those uh chips, the real hardware.
• 29:45
  SPEAKER_03: We have the NXP conversion tools, the latest version of the NXP kernels.
• 29:52
  SPEAKER_03: So the system is able, if it needs to, to go and test its hypothesis for oh well, this model.
• 30:00
  SPEAKER_03: Seems I think this model would be a good a good candidate, good model architecture for this for this problem.
• 30:06
  SPEAKER_03: It's able to go and test that on the real hardware if it hasn't tested it before.
• 30:13
  SPEAKER_03: And what it gets back from that are things like what's the accuracy, what's the inference time, and what's the energy per inference.
• 30:23
  SPEAKER_03: So all of that is rooted in real measured data.
• 30:29
  SPEAKER_03: And then the system can once it has some neural network architectures that it thinks are very you know likely to be successful and come in under the constraints, then it starts to expend more computation on training and hyperparameter optimization of those neural network architectures.
• 30:53
  SPEAKER_03: And it's continuously learning.
• 30:55
  SPEAKER_03: So every time every time it sort of tries out a new architecture on a particular trip chip, it learns all of that and essentially has a big knowledge store inside it.
• 31:10
  SPEAKER_03: Okay, so now if we go back to so I've I've created the job, and then I come back sometime later.
• 31:20
  SPEAKER_03: So during this time, the system has done all of those steps, spun up a cluster of computation for training and other computation-intensive stuff that it's doing, and then it starts to give you results.
• 31:37
  SPEAKER_03: And this is a pretty typical example of a results page.
• 31:42
  SPEAKER_03: So it says here that it's actually the pill detection data set, and we kind of collapsed all of the complexity of the data science results down into a single single page, which starts off with here's here's a graph of my prediction time versus my accuracy.
• 32:10
  SPEAKER_03: So x-axis is the prediction time or the the uh inference time, uh y-axis is the accuracy, and that that particular metric that it's chosen there is whichever the the um the original job chose as the primary metric.
• 32:29
  SPEAKER_03: So, you know, you might have seen um uh scatter plots like this before.
• 32:35
  SPEAKER_03: Every every dot on here is a model, and up and to the left is better.
• 32:41
  SPEAKER_03: The black line is the Pareto frontier.
• 32:45
  SPEAKER_03: So the most interesting ones are generally on the Pareto frontier.
• 32:49
  SPEAKER_03: Um, so if I click on one of these models, then I get you know a whole bunch of detailed parameters, all the different metrics there.
• 32:59
  SPEAKER_03: If I if I'm interested in different metrics, and you can see that there the system understands and reports measured uh accuracy metrics or data science type metrics, as well as these kinds of things, energy per inference, prediction time.
• 33:21
  SPEAKER_03: These are very, you know, very hardware specific things, or RAM usage, ROM usage.
• 33:28
  SPEAKER_03: Um, and so you get a complete picture here of the performance of that particular model.
• 33:38
  SPEAKER_03: Um, you may be satisfied, and at that point you can download that model.
• 33:42
  SPEAKER_03: And if you download and run that model on that same chip, remember this is all actually this was a this particular run was targeting an ST chip.
• 33:54
  SPEAKER_03: If you download and run that on that chip, you'll get the exact same performance.
• 33:59
  SPEAKER_03: So this is guaranteed because this is measured, these are measured performances.
• 34:04
  SPEAKER_03: Um, so this is avoiding the problem that you you often get where you know the the tools need to do an estimate, or you're doing you're doing your data science in a more abstracted environment, building the models and measuring them in an abstract environment, and then you've got this porting or deployment process to get it to work on the actual chip.
• 34:29
  SPEAKER_03: In this case, these the numbers are um measured on that chip.
• 34:34
  SPEAKER_03: Okay, and then you know, within each of these models, there's uh actually gruesomely detailed reporting, and this is all normalized, so it it kind of makes it nice for the end user because it's all normalized from the vendors' ways of describing this stuff.
• 34:58
  SPEAKER_03: So the low level that you know that the different SDKs describe this stuff or report this stuff in different ways.
• 35:05
  SPEAKER_03: We normalize all of that and basically provide very detailed reports which are very comparable, you know, one chip to another.
• 35:15
  SPEAKER_03: And you know, so this is this is actually called our our model benchmarking report, which is as you can see, is mainly about around layer timing and and sort of um uh resource usage on chip.
• 35:28
  SPEAKER_03: And then we also have this model scoring report, which is uh all the data science uh metrics in gruesome detail.
• 35:41
  SPEAKER_03: So just going back to here, I just wanted to show one more thing.
• 35:49
  SPEAKER_03: The uh within this graph, you you actually can access all of those different metrics.
• 35:56
  SPEAKER_03: So if you decide that, oh, you know, Coco MAP at 0.5 isn't what I'm interested in, I want it at 0.75, it's just a simple matter of you know selecting that off this axis.
• 36:09
  SPEAKER_03: So, you know, a lot of the sort of stuff that you might otherwise be doing in you know, writing a lot of Python code that the sort of day day-to-day stuff is done by Model Cat in a very easy way and accessible way.
• 36:26
  SPEAKER_03: Okay, so with that, I'm just gonna skip back.
• 36:31
  SPEAKER_03: I'll get I'm gonna finish up now so that we have time for questions.
• 36:40
  SPEAKER_03: One more thing I just wanted to mention.
• 36:43
  SPEAKER_03: Modelcat has a partnership with NXP, and we we co-developed a version of Model Cat with them called EIQ Model Creator, which is their their cloud-based ML enablement platform for all of their um chips, all the way from uh MCXA, which is uh a relatively inexpensive, low-powered microcontroller with no acceleration, up through the RT series and the IMX series, uh multiple series of IMX.
• 37:17
  SPEAKER_03: And so we've got Model Cat, which is uh our own brand and our own product, and then we've got the EIQ model creator, which is co-developed with with uh NXP.
• 37:30
  SPEAKER_03: All right, and I invite you all to do a test drive.
• 37:34
  SPEAKER_03: Give us a yell, and we'd be happy to give give uh test like um licenses, test drive licenses.
• 37:41
  SPEAKER_03: Okay, that's that's it from me.
• 37:45
  SPEAKER_03: I'll leave that up there for now.
• 37:47
  SPEAKER_03: Questions.
• 37:50
  SPEAKER_01: Well, thank you, Evan, for uh running through all that.
• 37:54
  SPEAKER_01: That was pretty cool.
• 37:55
  SPEAKER_01: And uh we have a few, uh quite a few in the uh queued up here in the chat.
• 38:01
  SPEAKER_01: Uh Jenny, do you wanna you wanna do the honors?
• 38:03
  SPEAKER_01: You had some good questions on your own too.
• 38:05
  SPEAKER_00: So we I got some questions in the chat.
• 38:08
  SPEAKER_00: However, we just got one from Dimitri in the comments, so maybe we'll do that one first.
• 38:13
  SPEAKER_00: Throw it up there.
• 38:16
  SPEAKER_00: So Dimitri asked, yeah, can you export your models to PyTorch or Keras?
• 38:21
  SPEAKER_00: Or is it only to the hardware that's supported by Model Cat?
• 38:25
  SPEAKER_03: It is.
• 38:26
  SPEAKER_03: We do have um, we're kind of more in the Keras universe, so we have a direct export that will get you into Keras, but not PyTorch today.
• 38:38
  SPEAKER_03: Okay.
• 38:43
  SPEAKER_03: You know, the the canonic the standard output the the the chip guys have made it relatively simple for us on the chip side, uh tier flight, you know, has is still uh the dominant um interchange format and we support that.
• 39:02
  SPEAKER_03: So for for almost all almost all the direct-to-chip uh interactions, we output tier flight.
• 39:11
  SPEAKER_01: Sounds good.
• 39:13
  SPEAKER_01: Uh what are some of the other questions we got here?
• 39:16
  SPEAKER_01: Jenny, you had some good ones you were covering.
• 39:18
  SPEAKER_00: Um yeah, I just asked some questions um sort of chronologically throughout your whole presentation.
• 39:24
  SPEAKER_00: Um, I guess the first one I had was um we I know you showed an image classification, maybe even bounding box object detection example, but what are the other types of models that you can generate with Model Cat?
• 39:37
  SPEAKER_00: Is it just computer vision or can you do time-based frequency?
• 39:43
  SPEAKER_03: We we've done um okay, so it's mainly computer vision, um, the usual computer vision use cases.
• 39:50
  SPEAKER_03: Uh the um we have done uh a bunch of what I what yeah single-dimensional, you know, um time series data type stuff.
• 40:06
  SPEAKER_03: And and um that so that yeah, people use it for that and have used it for that.
• 40:12
  SPEAKER_03: Um and we we are also working on uh other other modes, so you know obviously the language models and so on are coming.
• 40:25
  SPEAKER_03: Um probably not gonna reach these little microcontrollers, but you know, that we already have chips that are large enough to run language models, for example.
• 40:35
  SPEAKER_02: Cool.
• 40:36
  SPEAKER_01: Here's one that came in from Dale.
• 40:38
  SPEAKER_01: Uh will you be supporting more MCUs?
• 40:44
  SPEAKER_03: Yes, we we keep adding various types of, you know, some of them are MCUs, some of them are really microprocessors, some of them are these special purpose inference chips.
• 40:55
  SPEAKER_03: We keep adding more.
• 40:56
  SPEAKER_03: And the the nice thing about it is that we've now got a kind of factory for adding those.
• 41:02
  SPEAKER_03: This is not a highly, highly uh artisanal process to add to to onboard those chips.
• 41:09
  SPEAKER_03: So, yeah, we are, and we we work with you can see that we work with the major vendors to date, but we we've got some very interesting discussions going on with a bunch of the startups at the moment, which is which will will sort of supplement.
• 41:27
  SPEAKER_01: I could kind of create some libraries to plug into your stuff.
• 41:30
  SPEAKER_03: Yeah, yes, so we we've got we've got a we've got a uh roadmap bullet point for self self-onboarding, and we are working towards it's not there yet.
• 41:42
  SPEAKER_03: So, right now there's still a bit of engineering work that one of our engineers has to do.
• 41:49
  SPEAKER_00: So I I guess my my follow-up question for that would be are do the embedded engineers use AI to generate the support for a new platform for the AI generating AI, you know what I'm saying?
• 42:03
  SPEAKER_03: Uh yeah, no, I'm not saying um the the the uh yeah no.
• 42:10
  SPEAKER_03: Uh I think you know the deep embedded stuff, and and of course, the last bit that the embedded engineers on our side that is are doing is the deep embedded stuff, right?
• 42:22
  SPEAKER_03: You can imagine.
• 42:23
  SPEAKER_03: Yeah, that's we've got some great ideas about how to do that.
• 42:27
  SPEAKER_03: Uh today, today we we we haven't done it.
• 42:32
  SPEAKER_00: Awesome.
• 42:33
  SPEAKER_00: Well, we got a couple more questions in the chat.
• 42:35
  SPEAKER_01: Yeah, yeah, yeah.
• 42:36
  SPEAKER_01: Should we do the Qualcomm one next?
• 42:39
  SPEAKER_01: Why not?
• 42:41
  SPEAKER_01: Of course, the Q490.
• 42:44
  SPEAKER_03: Yeah, so we've got we've got uh not that Qualcomm chip, we've got another Qualcomm chip whose name I've immediately forgotten on there at the moment.
• 42:53
  SPEAKER_03: Um we we're pretty happy to work with whichever vendors and we we tend to be driven by you know the business opportunities in terms of sequencing the chips on.
• 43:06
  SPEAKER_03: We uh we I believe that that that 6490 has got uses the same um MPU, yeah, no, it doesn't really matter to us, but as long as the SDK is the same, uh and I think it does use the same SDK, and they've got a they've changed the name of their SDK a couple of times.
• 43:24
  SPEAKER_01: It's part of the dragon wing um world there.
• 43:28
  SPEAKER_01: Yeah, cool.
• 43:29
  SPEAKER_01: Uh actually I had a quick question.
• 43:31
  SPEAKER_01: So talking about architecture types and things like that.
• 43:34
  SPEAKER_01: I mean, we one of the panels we had in Amsterdam was our one called the farthest edge talking about spiky neural networks and neuromorphics and things like that.
• 43:42
  SPEAKER_01: Is have you done any experiments with that in terms of portability to or is it your uh supporting kind of more spiking neural network type?
• 43:50
  SPEAKER_03: Yeah, so we uh we're talking to a a few of the startups in that that and and and you know, a couple of those do these kind of spiking neural network type approaches.
• 44:04
  SPEAKER_03: I mean, it's it's it the the general architecture that we have is very very applicable to that, but the the lower level, you know, doing the the abstract jump jumping the abstraction gap between where they are with their runtime or whatever, you know, the chip vendors runtime and and where we are, which is you know, we speak pretty standard uh neural network architectures, uh and interchange formats.
• 44:37
  SPEAKER_03: That's that's a bigger, bigger, you know, bigger um step.
• 44:42
  SPEAKER_03: Now that said, I'm hoping that that in the next uh six months or so we have we have one or other of those that that gets mature enough that we we get it up into production.
• 44:55
  SPEAKER_01: Cool.
• 44:56
  SPEAKER_01: Here's a question from Dmitry Godofsky.
• 45:02
  SPEAKER_01: Do you have KWS or WWD models?
• 45:06
  SPEAKER_01: Maybe you can enlighten the audience as to what those acronyms mean.
• 45:10
  SPEAKER_01: Uh I don't know what those meaning.
• 45:14
  SPEAKER_00: Uh KWS is keyword spotting.
• 45:16
  SPEAKER_03: Yeah, yeah, yeah.
• 45:17
  SPEAKER_03: We've we've uh audio keywords we've done, yeah.
• 45:20
  SPEAKER_03: Yeah.
• 45:20
  SPEAKER_03: Uh WWE.
• 45:21
  SPEAKER_00: W D I need to look up.
• 45:23
  SPEAKER_03: Yeah, okay.
• 45:25
  SPEAKER_03: That's not one that I'm familiar with.
• 45:26
  SPEAKER_03: I'm sorry.
• 45:27
  SPEAKER_01: All right, Dimitri, if you can enlighten us in the chat, what is WWD?
• 45:32
  SPEAKER_01: Uh what else we have here?
• 45:35
  SPEAKER_01: Uh there's a question from Katya.
• 45:39
  SPEAKER_01: Uh latest research papers at the beginning as well as NAS.
• 45:43
  SPEAKER_00: Oh, wake word detection.
• 45:44
  SPEAKER_00: Sorry.
• 45:45
  SPEAKER_01: Wake word detection.
• 45:46
  SPEAKER_01: Oh, oh yeah, yeah, okay.
• 45:51
  SPEAKER_01: Yeah.
• 45:52
  SPEAKER_01: Okay, model building AI closer to NAS or indeed managing to learn from latest research, yes.
• 45:57
  SPEAKER_03: Yeah, yeah.
• 45:58
  SPEAKER_03: So so our general approach is a superset of the of both of those.
• 46:03
  SPEAKER_03: So so this is this is really important, actually.
• 46:06
  SPEAKER_03: You know, you think about how human learns, you know, you if you're someone who's working in research in that field, how do you learn about new ways of doing things?
• 46:22
  SPEAKER_03: I mean, I use that and it sounds like a very, very high-level description, but you have to jump up to that level to get to get this point.
• 46:30
  SPEAKER_03: Um, you you you might read a paper, or you might scrip scribble on the back of a bit of paper and go, oh, if I take a uh uh mobile net three and replace this block with uh something else, then I've got I've I've just done some you know neural architecture research, or at least ideation, and um and uh I've got I've got a new neural network architecture.
• 47:00
  SPEAKER_03: Um so it's both is really the answer to your to the question that's asked here.
• 47:12
  SPEAKER_00: Good.
• 47:13
  SPEAKER_00: Well, I had a quick question.
• 47:15
  SPEAKER_00: Um we we you showed the pill detection model, um, for example, in the in the demo.
• 47:21
  SPEAKER_00: Um, how are do you have to upload your own data?
• 47:25
  SPEAKER_00: Did someone take images of all those pills, or were they synthetically generated using some sort of Python script?
• 47:30
  SPEAKER_03: Or um that that's it's an open source data set.
• 47:36
  SPEAKER_03: Uh okay, all right.
• 47:37
  SPEAKER_03: So I don't I think I think it was basically uh look, I don't actually know the answer, but just looking at it, it looks like yeah, it's actual real images and they've done background replacement or something because it because it's got a lot of a lot of flack and hash kind of visual hash in it.
• 47:56
  SPEAKER_00: For sure.
• 47:58
  SPEAKER_03: It's pretty hard.
• 47:59
  SPEAKER_00: Yeah, some sort of image manipulation script for sure.
• 48:02
  SPEAKER_00: And just okay, but so you have to upload your own data sets.
• 48:05
  SPEAKER_03: Yeah, to come back to your question, yeah, that the we we provide a bunch of the open source data sets that we've already uploaded.
• 48:13
  SPEAKER_03: People are always uploading other open source data sets that we've never heard of, and then like for the real production quality models that when people are serious about building something for a product, they typically have to either create their own data in some way or or or gather their data in some way.
• 48:33
  unknown: Yeah.
• 48:33
  SPEAKER_00: Okay.
• 48:35
  SPEAKER_03: Cool.
• 48:37
  SPEAKER_00: Do you have to bring it in already labeled as well?
• 48:40
  SPEAKER_03: Yeah, we have a we have a labeling service that we partner with, and that's within the platform.
• 48:46
  SPEAKER_03: Um but but most of the time, yeah, it's it's already labeled.
• 48:50
  SPEAKER_00: Sure, sure, sure.
• 48:54
  SPEAKER_03: Good.
• 48:56
  SPEAKER_01: What else we got here?
• 48:58
  SPEAKER_03: Any more goodies?
• 49:00
  SPEAKER_02: Oh, also oh, sorry, go ahead.
• 49:02
  SPEAKER_03: I just saw one question there that I thought would be a good one.
• 49:04
  SPEAKER_03: Who is the audience for Model Cat?
• 49:06
  SPEAKER_03: Do you need to have strong knowledge of ML already to use it?
• 49:10
  SPEAKER_02: Yeah.
• 49:10
  SPEAKER_03: Um you might have been able to see in the user interface that that we we really worked hard to make it so that you can you can kind of think functionally and and not have to be an expert on the minutiae of data science and ml to use the thing.
• 49:34
  SPEAKER_03: And I I can say that we've we've we've you know we've come a hugely long way and we're continuing to to squeeze that, squeeze that.
• 49:44
  SPEAKER_03: It's it's actually a very hard thing to do because it pushes a lot of the functionality further down lower down into the stack.
• 49:50
  SPEAKER_03: The AI further down has to be more and more responsible, if you like.
• 49:55
  SPEAKER_03: And and so you know, the last thing any of us want is to be arguing with the thing about what you know what what it wants to do or what its ideas are or whatever.
• 50:04
  SPEAKER_03: That's not its job.
• 50:05
  SPEAKER_03: The job is to kind of just do what you ask.
• 50:08
  SPEAKER_03: So so the the our aspiration is to make it that it can a product manager can use it.
• 50:15
  SPEAKER_03: So that that you know that that is um, you know, analogous to what some of the the code um development tools in the in traditional software uh aiming at.
• 50:29
  SPEAKER_00: Cool.
• 50:30
  unknown: Cool.
• 50:30
  SPEAKER_00: Yeah, um, I I was looking at in the demo you showed there was a lot of options to show like advanced mode, and then the page got so big, and I was like, whoa, I I think that would be a little bit intimidating, but um I'm assuming you guys you have some sort of way to do, I suppose, is there some sort of like partner uh programming for a bit?
• 50:52
  SPEAKER_00: Is there a package like or do you just have to jump straight in on your own with your team?
• 50:58
  SPEAKER_03: It's you know, what we what we suggest is do a test drive if you're interested, or you you're you're asking yourself that question.
• 51:06
  SPEAKER_03: A user could well be asking themselves that question.
• 51:09
  SPEAKER_03: What we suggest is do a test drive, it costs nothing, it takes all of a few hours to actually get a flavor for what what you need to do.
• 51:21
  SPEAKER_03: Yeah, um, it really is that easy.
• 51:25
  SPEAKER_03: Uh and and um and so you know most users once they've done a trest drive, they they kind of get that you have to recalibrate your mental model because if you're if you're kind of used to Python note books and or you're an you know maybe a traditional software developer and you're looking at all the libraries that which you know which of these libraries do I really need for developing my AI model, um it, you know, this is you know a hundred or a thousand times less complex.
• 52:00
  SPEAKER_03: So, yes, we we we we have the advanced mode because we need to straddle.
• 52:05
  SPEAKER_03: Look, the reality is that many of the people listening here are data scientists or ML people in some in some capacity, and we need and they need control and they want to have control, and they probably do know, you know, and will be able to contribute beyond what our AI sort of can do.
• 52:24
  SPEAKER_03: And so we, you know, we want to, you know, we need to enable that as well.
• 52:28
  SPEAKER_01: Right, right.
• 52:29
  SPEAKER_01: So when you say qualified buyers, is this available to like folks in academia as well, or do you have some sort of program if they're yeah, yeah, yeah, yeah.
• 52:37
  SPEAKER_03: For the the test drives are, yes.
• 52:40
  SPEAKER_01: Okay, so qualified academic academics.
• 52:44
  SPEAKER_03: Um it's just qualified in the sense that you you know it's not it's not a completely automated process.
• 52:49
  SPEAKER_03: We that's an email address, and that's a that's a little form, and yep, okay.
• 52:55
  SPEAKER_01: We will we encourage people to click click on that.
• 52:58
  SPEAKER_00: Yeah, this and this sort of plays along with Katya's question earlier.
• 53:02
  SPEAKER_00: Um, how long does it take to get from start to finish?
• 53:05
  SPEAKER_00: Um, how long, you know, does it does it would it take for an average user to get from uploading their data, clicking start, and then deploying onto the device?
• 53:14
  SPEAKER_03: Yeah, yeah.
• 53:14
  SPEAKER_03: Um okay, so so it's an excellent question.
• 53:18
  SPEAKER_03: Everybody uh many people ask it.
• 53:23
  SPEAKER_03: Uh unfortunately, it's uh it's a how long is a piece of string question.
• 53:29
  SPEAKER_03: Um it can take between, so Katya, your range of estimates is is about right.
• 53:38
  SPEAKER_03: You know, hours up to a week is is about kind of what I would say.
• 53:46
  SPEAKER_03: So a week for these large data sets, like you know, we've got some, you know, people who've been doing automotive, you know, safety-related, high-resolution images, large data sets, um, very large transformer models that you're training.
• 54:05
  SPEAKER_03: Um, and then you know, uh that that that example that I showed, I mean, that was probably a few hours, I would guess.
• 54:14
  SPEAKER_01: Got it.
• 54:15
  SPEAKER_01: And do you run this on uh what's your back end?
• 54:18
  SPEAKER_01: Are you running this like an AWS or is it?
• 54:20
  SPEAKER_03: Yeah, it's AWS.
• 54:21
  SPEAKER_03: We use AWS, and so it's and it and Katya, you might be wondering how we do that in a few hours.
• 54:26
  SPEAKER_03: It spins up a parallel a cluster of machines for you for the particular job.
• 54:33
  SPEAKER_03: So for you for you, when you hit that build button or the create button, it spins up a cluster for for you.
• 54:40
  SPEAKER_03: It's not just one machine or anything.
• 54:43
  SPEAKER_00: Oh, you also mentioned, oh sorry, Pete.
• 54:46
  SPEAKER_00: Um, there you also mentioned that there was a I can't remember what you called it, but uh a farm of devices uh that you've already support.
• 54:55
  SPEAKER_00: Are these like like you have a room physically with the devices that it's deploying to to check the accuracy and the performance?
• 55:02
  SPEAKER_02: Yep, yep.
• 55:04
  SPEAKER_03: Yep.
• 55:04
  SPEAKER_00: I don't think a picture of that.
• 55:08
  SPEAKER_03: Uh yes, yes, there is a picture floating around.
• 55:11
  SPEAKER_03: I think I think our hardware guy is gonna do a beauty shop soon.
• 55:16
  SPEAKER_03: He said he was going to.
• 55:17
  SPEAKER_03: Yeah.
• 55:19
  SPEAKER_03: Yep.
• 55:20
  SPEAKER_00: Very cool.
• 55:21
  SPEAKER_01: Good, good.
• 55:23
  SPEAKER_00: Oh, Dale has a question, Pete.
• 55:25
  SPEAKER_01: Dale has a question.
• 55:27
  SPEAKER_01: How much does it cost?
• 55:30
  SPEAKER_03: Yeah, unfortunately, that's the piece of string again, Dale.
• 55:33
  SPEAKER_03: Um, but but you know, um we we have uh we have a little one pager which kind of gives some some kind of guidelines, small, medium, and that and that sort of production quality.
• 55:48
  SPEAKER_03: Um uh which we could share.
• 55:50
  SPEAKER_03: So if you if you if you just ping me afterwards, I'll dig that out and I'll just I'll just send that to you.
• 55:55
  SPEAKER_03: That just it again, it gives you a flavor, uh kind of like what um there was a tool where you could sort of do a cost estimator on your website.
• 56:04
  SPEAKER_03: Yeah, yeah.
• 56:05
  SPEAKER_03: You know, the the one of our engineers is promised me, promising me the cost estimation model.
• 56:12
  SPEAKER_03: Okay, and and and you know, it's coming, we've got enough data.
• 56:16
  SPEAKER_03: So the hard thing with that is is is the data.
• 56:18
  SPEAKER_03: It's actually not that hard a model, but it's just a yet another roadmap item.
• 56:23
  SPEAKER_03: So that little meter that said what your probability of success was will also say here's here's what your cost estimate is.
• 56:31
  SPEAKER_01: Right, okay.
• 56:32
  SPEAKER_01: So you do have human engineers, it's not all AI generated, you're not the only engineer.
• 56:36
  SPEAKER_01: No, no.
• 56:37
  SPEAKER_03: No, you know, that canonical slide of how development that's you know, for sure, a large platform development like this, we look like that.
• 56:46
  SPEAKER_01: No.
• 56:47
  SPEAKER_01: Okay, awesome.
• 56:48
  SPEAKER_01: We got we got we're coming in the last couple of minutes.
• 56:51
  SPEAKER_01: We got uh I want to remind folks, um, this show will be on our YouTube channel instantly after we finish today.
• 56:58
  SPEAKER_01: Uh so if you didn't catch the beginning, you can do that.
• 57:01
  SPEAKER_01: And then also um upcoming live streams are here at jfoundation.org live stream.
• 57:07
  SPEAKER_01: So we have one of these every two weeks.
• 57:09
  SPEAKER_01: Uh, you can subscribe to our YouTube channel or whatever your mechanism is, it's on LinkedIn and Twitch as well.
• 57:16
  SPEAKER_01: Um but yeah, I think Evan, this is fantastic.
• 57:19
  SPEAKER_01: Really appreciate your time.
• 57:21
  SPEAKER_01: And Jenny, thanks for co-hosting.
• 57:23
  SPEAKER_01: Um thanks everyone in the audience for accommodating our timing things.
• 57:29
  SPEAKER_01: Um, and uh any closing words of wisdom, Evan, for for the world here?
• 57:34
  SPEAKER_03: Uh all I'd say is ping ping us, and we're we're very open to um partnering.
• 57:43
  SPEAKER_03: Uh, you know, I know I noticed that there was a bunch of of uh questions about the chips.
• 57:47
  SPEAKER_03: We're very open to partnering with a variety of different chip vendors, and this proposition works for all of those, including the really innovative stuff that people are doing at the hardware level.
• 58:00
  SPEAKER_01: So yeah, and this is like this is like exactly the kind of innovation thing that you almost don't even think of that's happening in the space, like you know, actually using AI to sort of help with model portability and development and deployment of solutions.
• 58:13
  SPEAKER_01: So uh fantastic the that you're on the cutting edge of this.
• 58:17
  SPEAKER_01: And um again, kudos on the rebranding model cat.
• 58:20
  SPEAKER_01: Very cool.
• 58:21
  SPEAKER_01: Thanks very much.
• 58:23
  SPEAKER_01: I got my cat information merch.
• 58:25
  SPEAKER_02: We need some model catch or something.
• 58:29
  SPEAKER_01: I don't know.
• 58:30
  SPEAKER_01: Um cool.
• 58:31
  SPEAKER_01: All right, well, thanks everybody.
• 58:33
  SPEAKER_01: Jenny, thank you again, Evan.
• 58:36
  SPEAKER_03: Thank you very much both for hosting and thanks thanks to the audience for listening today.
• 58:42
  SPEAKER_02: Thanks again for take care.
• 58:44
  SPEAKER_03: Okay.