Latest / EDGE AI POD / Is it time to take control of your Edge AI Project? A conversation with SensiML

Transcript

Is it time to take control of your Edge AI Project? A conversation with SensiML · EDGE AI POD

• 0:00
  Speaker 1: data ingestion labeling model building and testing our Python SDK for programmatically interacting with our back-end engine, our embedded ML SDK that we use to generate ML pipelines for inferencing at the edge devices, and then that core engine that does all of the ML ops on the backend server.
• 0:28
  Speaker 1: So before I talk about that, I just wanna talk generally a little bit about building a real-time sensing algorithm for embedded devices and kind of what we, when we think of it.
• 0:37
  Speaker 1: What does it look like?
• 0:38
  Speaker 1: So for us, the first thing that we always think about is the raw sensor data coming in off of a sensor, and then we're going to do some signal preprocessing and that's really just to improve the quality of the signal by removing the noise, and if we know something about the signal, we might add some bias there to it.
• 0:57
  Speaker 1: And then there's typically an event triggering step, and so that can be, at the very simplest, like a sliding window where every hundred samples of data comes in.
• 1:05
  Speaker 1: We say, all right, this is a new event, let's pass this down the pipeline.
• 1:15
  Speaker 1: It can be more complicated where you have some sort of threshold settings that you're looking for.
• 1:17
  Speaker 1: You're looking for specific signals within a frequency range, and that becomes your event trigger, and so we typically refer to that as a segment or segmentation.
• 1:27
  Speaker 1: The next is feature extraction, and so in the feature extraction step we're taking this time series raw time series sensor data that's come in and been a little bit processed and we're using all of that to extract specific features.
• 1:41
  Speaker 1: So that could be something as simple as just taking the average across a channel.
• 1:45
  Speaker 1: So maybe you have 100 samples and you take the average of that and that becomes a feature.
• 1:50
  Speaker 1: Sometimes you'll see feature extraction as being extracting the frequency features using the MFCC coefficients or the MFE.
• 2:01
  Speaker 1: But so there's tons of different feature extractions and really the goal there is to simplify the signal into from this very raw time series into these individual features.
• 2:14
  Speaker 1: And then finally the classification step, where we take these features we've extracted and we use a trained model, a model that we've trained to run the inference of the device, and then we report it out or we act on that.
• 2:27
  Speaker 1: So some of the important parts about the event triggering and the feature extraction is it helps us to reduce false positives and class confusion.
• 2:38
  Speaker 1: The event triggering can be really important for preventing the classifier and the feature extractors from running unnecessarily.
• 2:47
  Speaker 1: So you can reduce the power and then, if you have really good features in your model, you can reduce the classifier complexity quite a bit and reducing the complexity is going to lead to lower latency and lower memory requirements for your model.
• 3:09
  Speaker 1: So, this is kind of a view of the event detection and feature extraction that we see it in our toolkit and just in general.
• 3:16
  Speaker 1: So on the left here we have kind of our time series sensor data coming in.
• 3:21
  Speaker 1: This is accelerometer data, so accelerometer x, y and z and what you see from the sensor is just every point in time you kind of see one data point from each of these and typically when we're talking about embedded devices you don't have a lot of memory so we can't necessarily store like three channels or six channels of data, multiple seconds of it.
• 3:43
  Speaker 1: So a lot of times we have to do real time processing of the data and detect those events in real time, because we can't store all of this memory in our buffer.
• 3:52
  Speaker 1: So these labels we put down right here, these are some sort of segmentation algorithm that we've applied, so they're detecting these different events.
• 4:02
  Speaker 1: It's probably triggering on the uh, seeing like a jump in the threshold and some of the signal, and says, okay, here's your start, here's where it kind of slows down.
• 4:10
  Speaker 1: So we do, we say this is a segment, some event happened here, and then we're going to extract features from it.
• 4:17
  Speaker 1: Um, and so those features that we extract you can kind of see over here on the right hand side.
• 4:23
  Speaker 1: So the features that we extracted from this purple one we have a number of instances of this purple one, so each of these dots here represents a feature from one of these segments that we've labeled, and you can see they kind of group nicely here, the blue ones here.
• 4:41
  Speaker 2: So even with these.
• 4:41
  Speaker 1: In this case, this is just two different feature extractors.
• 4:44
  Speaker 1: It's already doing a good job of separating different types of events that are happening in our data set.
• 4:53
  Speaker 1: So that's really what we're looking for when we initially start to do the event detection of the feature extraction is one.
• 4:57
  Speaker 1: Can we find an event detection algorithm that does a really good job of kind of separating out the events we're interested in from the rest of the noise, capturing the start and the end of that event and then the feature extraction?
• 5:08
  Speaker 1: What we're looking for is features that do a very good job of separating the different events.
• 5:12
  Speaker 1: So briefly, I'll just talk a little bit about training in ML algorithm.
• 5:21
  Speaker 1: So you can do linear regression it's just concerned with modeling the relationship between variables in a linear fashion.
• 5:30
  Speaker 1: You can do instance-based, such as a KNN classifier, where you're building up a database of examples and comparing new things that come in to that database of examples using some similarity measure.
• 5:44
  Speaker 1: There's decision tree methods, where you construct a model of decisions and each node splits on some feature in your data set.
• 5:51
  Speaker 1: We also have clustering, which are typically unsupervised techniques, where you try and identify clusters within your data set and then anything new that comes in you're trying to classify into one of those clustersusters.
• 6:05
  Speaker 1: And then deep learning, which everyone's very familiar with, is just a machine learning technique that uses layers and neurons to build a complex network.
• 6:14
  Speaker 1: And then there's ensemble methods which kind of come.
• 6:19
  Speaker 1: Typically the ensemble methods are a group of decision trees, shallow decision trees which can perform very well when trained together, as well as support vector machines.
• 6:31
  Speaker 1: And then when we're training these models this is typically what you would see, at least for training a neural network You'll start with some loss function that you're trying to minimize and you start iterating over time, using a method typically like gradient descent, to reduce the loss function and then, as you're training, you're monitoring the accuracy and typically you're splitting it into a train and validation set so you can watch your training accuracy and your validation accuracy and you want those two to kind of be in line.
• 7:16
  Speaker 1: Okay, so for once you've trained your model.
• 7:18
  Speaker 1: It's one of the most important things is how do you validate your model and that's just how can I estimate how well my model did that?
• 7:25
  Speaker 1: I trained on the real world data set and so typically you've gone out and you've collected a data set and you want to be able to tell whoever you're talking to whether it's your customer or your boss or just if you're making a paper how well this is going to.
• 7:39
  Speaker 1: How can it well can I expect it to do on a data set that I haven't collected yet.
• 7:44
  Speaker 1: And so some of the things we look for are first, is my model underfitting?
• 7:47
  Speaker 1: And so underfitting occurs basically when a model can't adequately capture the underlying structure.
• 7:53
  Speaker 1: So in this case, on the top, you see, it's underfitting because we're just drawing a straight line and the data doesn't really model the data.
• 8:01
  Speaker 1: On the bottom, when we're thinking about clustering, again we don't have enough clusters, so we can't really capture all of the information in this data.
• 8:11
  Speaker 1: And then there's also overfitting where your model corresponds too closely to that particular data set.
• 8:16
  Speaker 1: So you can see, in the top we're just drawing this very squiggly line here, and then in the bottom where our cluster are just all over the place, and so they're not necessarily capturing the structure of the data either.
• 8:29
  Speaker 1: And so typically what you want to do is you want to find a more balanced model, and that's just going to strike a balance between the bias of the model and the variance in the data set.
• 8:40
  Speaker 1: So a more balanced model for this one would kind of look like this graph, and a balanced model for this one would look like that, and then just briefly talk a little bit about model evaluation metrics.
• 8:53
  Speaker 1: So for binary classification.
• 8:56
  Speaker 1: A lot of times you'll see curves like this where you're looking at the ROC curve and essentially with binary classification, typically you'll have some threshold that's going to separate class A from class B and when they make a chart like this, they're just varying that threshold to get the range of how the classifier is going to perform when changing the threshold.
• 9:17
  Speaker 1: And then you select the best threshold for your model and then when you start talking about multi-class classification instead of just a true positive and true negative graph.
• 9:33
  Speaker 1: You'll see things like a confusion matrix, and so the confusion matrix is going to have all of your predicted classes here, all of your actual classes here, and then you can kind of see as data came in.
• 9:47
  Speaker 1: I predicted a cross.
• 9:49
  Speaker 1: It was actually a jab.
• 9:50
  Speaker 1: I predicted a hook.
• 9:51
  Speaker 1: It was unknown.
• 9:54
  Speaker 1: And so you can see where is your model getting confused between what the actual data is and what the model's predicting.
• 10:02
  Speaker 1: And then the last thing I want to talk about, in terms of setting up all of the things data is and what the model is predicting, and then the last thing I want to talk about in terms of setting up all of the things you kind of need to build these ML models, is how do you use AutoML?
• 10:12
  Speaker 1: So when you're doing AutoML typically we would do, it's doing a hyperparameter search, and so in order to avoid, you want to avoid underfitting, overfitting all of these different things.
• 10:26
  Speaker 1: So typically what you'll do is you'll take your training data.
• 10:28
  Speaker 1: So I have this much labeled data.
• 10:31
  Speaker 1: I'll take my training data and I'll put it in one chunk.
• 10:33
  Speaker 1: I'll take my test data.
• 10:35
  Speaker 1: I'll put it in this other chunk and then I'm going to do cross-fold validation on this training data chunk.
• 10:41
  Speaker 1: So I'm going to split it up.
• 10:42
  Speaker 1: So fold one validate train, train.
• 10:44
  Speaker 1: Fold two I'm going to switch what I'm validating and use train and train.
• 10:53
  Speaker 1: And then fold three, I'm going to have train and train and validate, and by doing this you can kind of predict how your model is going to perform on data it hasn't seen before, using all of the data that you have available to you.
• 11:03
  Speaker 1: And so when you run the AutoML process, you're going to look across a lot of different classifiers.
• 11:09
  Speaker 1: Pretty much we will usually look across all the ones that I mentioned earlier.
• 11:12
  Speaker 1: You're going to look across a number of different features, a number of different parameters for training those classifiers, and then you're going to rank them based on some sort of score.
• 11:21
  Speaker 1: So if it's an F1 score, the accuracy or the sensitivity, and you're going to take the, typically you'll take the average of the validation scores across all of these folds and that kind of helps you figure out which models are being overfit or biased and kind of narrow down on the ones that are going to work well for your data set.
• 11:42
  Speaker 1: And then, once you've finished all of that, you can go ahead and test the model on this test set and that'll give you a good idea of how it's going to perform in the real world.
• 11:55
  Speaker 1: Ok, so I kind of went through ML 101.
• 11:57
  Speaker 1: I just wanted to talk a little bit about all the things that we think about when we're building models.
• 12:03
  Speaker 1: So now I want to talk a little bit about Piccolo AI, which is our open source initiative and so, like I mentioned, it has Sensible's ML Engine, which is a server application for model building, sensor data management, model tracking and embedded firmware generation.
• 12:17
  Speaker 1: Our embedded ML SDK and that's our inferencing and DSP SDK that we use to develop DSP and ML pipelines for edge devices.
• 12:26
  Speaker 1: Analytics Studio UI, which is our web interface for working with ML Engine.
• 12:31
  Speaker 1: Our ML Engine to build tiny ML applications.
• 12:34
  Speaker 1: And, finally, our Python client that we use to programmatically access the server through REST APIs from a Jupyter notebook and you can go to our website here at sensibleorg, and this is the home of the open source initiative.
• 12:50
  Speaker 1: So we're really looking for the community to come and get involved in this, and I'll talk a little bit more about the end at the end of the presentation, about ways that you can get involved.
• 13:04
  Speaker 3: Okay, bruce.
• 13:05
  Speaker 3: Yeah, go ahead.
• 13:09
  Speaker 3: I want to.
• 13:09
  Speaker 3: I want to jump in because we had a couple of questions building up here, so before you jump into the next topic, maybe you can address a couple of them here.
• 13:13
  Speaker 3: Here's one uh from john and I don't know.
• 13:17
  Speaker 3: It's interesting, let me just show it.
• 13:19
  Speaker 3: It says I'm streaming results to an sd card for data.
• 13:22
  Speaker 3: Have you addressed any projects to write data to an SD card with the M5 Stixie Plus, versus having to extract results over local Wi-Fi to another model with an SD module?
• 13:33
  Speaker 3: Is that something that is applicable here?
• 13:38
  Speaker 1: So I would say that kind of falls a little bit out of the things that we focus on, but that's definitely something that an embedded developer can do to store results.
• 13:51
  Speaker 1: So it just comes down to the dev kit.
• 13:55
  Speaker 3: Yeah, okay, there was that one.
• 13:57
  Speaker 3: Then there was this one here from Johnny Model value and how to deal with data shifting.
• 14:03
  Speaker 3: I'm not sure if we got the full gist of that question, maybe.
• 14:07
  Speaker 3: Johnny can elaborate.
• 14:12
  Speaker 2: Maybe you mean data drift.
• 14:14
  Speaker 1: Data drift probably.
• 14:17
  Speaker 3: After the evaluation.
• 14:18
  Speaker 3: Then you get data drift.
• 14:19
  Speaker 3: How do you deal with that?
• 14:22
  Speaker 1: Yeah, typically with data drift you have to set up some way to monitor that the data is drifting and then you'll either have to retrain your model or, if you have some sort of preprocessing that you can figure out, that can remove that drift.
• 14:38
  Speaker 1: But I think a lot of times it comes down to the particular application, how your models are deployed and things like that, for how you'd want to attack that problem.
• 14:48
  Speaker 3: Okay, so Johnny just replied.
• 14:50
  Speaker 3: Yes, that is actually the question that he had.
• 14:51
  Speaker 3: So sounds good, all right.
• 14:54
  Speaker 3: Why don't we get back back to your?
• 14:58
  Speaker 3: You're shifting into open source mode here with Piccolo, ai right, yeah, yeah, all right, we will.
• 15:05
  Speaker 1: We will, uh, remove ourselves okay, great yeah, thanks for the questions um, so yeah, so now I want to talk about how you can get up and running with the software that we're releasing and kind of go through how you, how you can use it, um, and how you can contribute.
• 15:21
  Speaker 1: So, uh, we have the.
• 15:23
  Speaker 1: We put everything on our GitHub repo, githubcom, slash sensimall, slash piccolo.
• 15:30
  Speaker 1: So I'll just click on this right here, okay, so here's, our.
• 15:41
  Speaker 1: GitHub repo.
• 15:42
  Speaker 1: You can come here and let me just pop open the window.
• 15:49
  Speaker 1: So if you just type into your terminal git clones the repo, it'll pull it down the repo, it'll pull it down.
• 16:05
  Speaker 1: And then to get started with it, all you have to do is cd into the piccolo directory.
• 16:10
  Speaker 1: If you have Docker set up already, and if you're on Linux, all you need is just to have Docker.
• 16:17
  Speaker 1: If you're on Windows, there's a couple of prerequisites you have to go through to make it work properly, and those are kind of listed here.
• 16:22
  Speaker 1: So you need to install Docker engine.
• 16:24
  Speaker 1: You need to make sure you enable and those are kind of listed here.
• 16:27
  Speaker 1: So you need to install docker engine.
• 16:28
  Speaker 1: Uh, you need to make sure you enable the wsl2 backend.
• 16:32
  Speaker 1: There's instructions on how to do that here and then there's some post installation instructions.
• 16:35
  Speaker 1: So once you kind of go through that, you can go through our prerequisite list here and it'll be.
• 16:40
  Speaker 1: Everything will be set up and ready for you to go.
• 16:42
  Speaker 1: So I I've already done that.
• 16:45
  Speaker 1: And then so you just cd piccolo and you type docker-compose up and that's going to start the server up.
• 16:53
  Speaker 1: So I already have mine running.
• 16:55
  Speaker 1: So if I hit docker ps, that's gonna list all of the containers that are running, and so I'm not gonna go too much into the overall architecture of the server in this talk, because I just wanna show how to get everything up and running.
• 17:12
  Speaker 1: So Docker compose up, hit enter, so for me it's already running.
• 17:21
  Speaker 1: What's gonna happen when you type Docker compose up is a Redis database is going to start, our redis server is going to start, postgres database is going to start, and those are going to be inside the containers, and then we have a helper script that runs inside a container really quick, and then we have a database initialized script that runs.
• 17:42
  Speaker 1: So what you're seeing on the output right here is our database initialized script, and so that's a script that runs.
• 17:48
  Speaker 1: You only have to run it when you start up the server if you make changes to the models or the migration scripts and that's gonna run, and it's gonna set up the initial database.
• 17:58
  Speaker 1: It's gonna compile our initial libraries and get everything ready for you to start working on the server.
• 18:06
  Speaker 1: So that's what's running right now.
• 18:08
  Speaker 2: So you'll see a bunch of output.
• 18:10
  Speaker 1: You can just wait for that to finish, and once that's done, your server will be all set up.
• 18:15
  Speaker 1: And when?
• 18:19
  Speaker 1: it's all set up you can go to localhost 8000 in your browser and it's going to bring you to the web UI here and you can log in with piccolo at since I'm all calm is the default, so that is right here.
• 18:43
  Speaker 1: So you can go to your browser, log in, pick a load at Syncsmall and then the password's tinyemlforlife, and then when you log in, so the web server is going to serve the UI to you.
• 19:02
  Speaker 1: So everything will be right here.
• 19:05
  Speaker 1: I've already uploaded a couple of projects, so yours will be empty when you first start it up.
• 19:11
  Speaker 1: And just to go over, if you're not familiar with our tools at all, you can do this if you just go to sensiblecom.
• 19:19
  Speaker 1: Oops, you just go to sensiblecom.
• 19:26
  Speaker 1: You can just click on free trial and sign up and you can get access to all of these tools on our managed instance the piccolo iai has meant, if you want to run it on your own or start helping develop some of the software that we're building.
• 19:41
  Speaker 1: So let me just give you an overview of what's going on here.
• 19:45
  Speaker 1: So, if I open up this, this is the UCI human activity recognition data set that we've uploaded.
• 19:50
  Speaker 1: So inside the UI you have a data manager screen, data manager screen, and so if I open up this, so here is one of the data sets from the human activity recognition and this is just someone laying down, and so they've captured a number of different sensors and they've gone through and they've labeled those.
• 20:14
  Speaker 1: I think this data set is from a cell phone, so they're capturing the accelerometer, and the gy set is from a cell phone, so they're capturing the accelerometer and the gyroscope from a cell
• 20:22
  Speaker 1: phone and then they're removing gravity in the body, acceleration, and then the total acceleration gives you the gravity vector also.
• 20:29
  Speaker 1: So you can use the data manager to label data, add different labels, explore your data and things like that, and you can also add different metadata to your data set.
• 20:42
  Speaker 1: So a lot of times the metadata for us is like all right, this was set 10, subject you can put anything else about the context, and we use that inside of our query.
• 20:52
  Speaker 1: So here I went and built a query and so here it says label session.
• 20:56
  Speaker 1: So here I went and built a query and so here it says label session.
• 21:01
  Speaker 1: So this is pulling in all of those segments that I label, and this is telling it.
• 21:07
  Speaker 1: Let me specify what sensors do I want to pull into my model.
• 21:11
  Speaker 1: So in this case I want to pull in all of them.
• 21:13
  Speaker 1: But if you just want it to build a model off of some of the sensors, you could specify that here.
• 21:18
  Speaker 1: Then right here I'm doing a query filter.
• 21:20
  Speaker 1: So I only want to pull in the set that's for my training data.
• 21:24
  Speaker 1: I don't want to pull in also the test data when I'm building the model.
• 21:26
  Speaker 1: Then you say save and it's going to give you an overview of the distribution for the data.
• 21:33
  Speaker 1: In your query I can see the label distributions by segments, the label distributions by the number of samples.
• 21:38
  Speaker 1: So you can see, here I have 1,400 segments that I've added to the data set for laying down and that's 180,000 actual samples of data, and then each of my segments is all 130 samples long.
• 21:53
  Speaker 1: So once you've done that, you can come in and set up a pipeline.
• 21:59
  Speaker 1: So we have a number of pipeline templates that we've created.
• 22:01
  Speaker 1: So this is another area where you can get involved in helping us to add new pipeline templates.
• 22:06
  Speaker 1: So the pipeline templates are essentially, if you have a particular application in mind, you can build up a template and add it here and that way when someone comes, they can just and they have a similar application they can select that template and it'll walk them through.
• 22:21
  Speaker 1: So, like, here's different models that have been trained, they can specify their data and it's going to just create the pipeline that specifically works for that application.
• 22:30
  Speaker 1: And then, if I want to create a pipeline, so I can come in here this.
• 22:38
  Speaker 1: This is a pipeline I already created and basically it gives you your pre-processing steps, your feature extraction steps and your model training steps, just like we were talking about previously in the presentation.
• 22:50
  Speaker 1: These are kind of all of the steps that you need to build an inferencing engine.
• 22:55
  Speaker 1: You can also add different steps in between, so if I wanted to put a sensor transform or a sensor filter, so different pre-processing steps.
• 23:04
  Speaker 1: There's augmentation built in.
• 23:06
  Speaker 1: You can add that there.
• 23:07
  Speaker 1: Here's our segmentation step.
• 23:10
  Speaker 1: So this model is using a sliding window with the window size of 120 and the slide is 120.
• 23:17
  Speaker 1: And so we have different segmentation algorithms built in that you can use or you can add your own.
• 23:22
  Speaker 1: And then, if I come down to the feature generator, so this model is going to generate these different features right here you can come in and edit the parameters.
• 23:33
  Speaker 1: We have a number of feature generators that we've built already parameters.
• 23:42
  Speaker 1: We have a number of feature generators that we've built already, and you can add your own, and I'll show you how to add your own after I finish going through this as well.
• 23:47
  Speaker 1: So then we have a feature selector step, and so we have a bunch of different feature selectors built in.
• 23:53
  Speaker 1: The feature selection also runs as part of the Auto ml process, so it'll try a bunch of different feature selectors, uh, and try and pick the with different parameters as part of that, and then typically we will quantize the features.
• 24:07
  Speaker 1: So up to this point, um, so typically data comes in as int16 values, feature generators run and they emit floating point values, and then, prior to sending it to a classifier, you run it through a quantization step, and then finally, there's the model training step, and so right now it's set up for AutoML mode, and so the only thing you can set is what sort of validation do you want to do?
• 24:32
  Speaker 1: And so when I was talking about AutoML, picking the validation method for your dataset is really important to avoid overfitting and underfitting and to direct the AutoML in the right place.
• 24:42
  Speaker 1: So you want to spend a little bit of time thinking about the different validation methods that you want to use.
• 24:46
  Speaker 1: If you click on the icon.
• 24:49
  Speaker 1: Here you can see the information about what this validation method is going to do, and then all you have to do, and then so this is, and then all you have to do next is you just hit run pipeline and it's gonna kick off this AutoML job, and that job is gonna run locally on your server.
• 25:10
  Speaker 1: So let me go ahead and kick it off or kick off a job and we can look at what happens.
• 25:14
  Speaker 1: So when you kick off the job, it's gonna submit it to a pipeline queue that's gonna be picked up by your server.
• 25:20
  Speaker 1: So if I open up this window here, this is where I ran Docker Compose up and all the logs are just getting dumped right here right now and so you can see different things like sensible steps.
• 25:31
  Speaker 1: So this is the step where data processing is happening, and so in this case I spun up 10 step, 10 step workers.
• 25:38
  Speaker 1: So basically any job that I submit is being split across 10 processors or 10 different containers, and they're all running in parallel and so you can see the logs here as it dumps things out to figure out if there's any issues or how far along it is.
• 25:56
  Speaker 1: So so we use this basically to debug and so here in the log screen is telling us how long it's run for what step it's at.
• 26:04
  Speaker 1: So it's at the feature generator step right here and it has 19 batches.
• 26:08
  Speaker 1: So that means there's just 19 shards of data.
• 26:12
  Speaker 1: Essentially that it's processing.
• 26:13
  Speaker 1: So it's gonna process that in parallel and then the training summary will pop up.
• 26:20
  Speaker 1: As it goes through the model training, it'll tell you how for each iteration that it does in the AutoML with the model classifiers were built and how they performed.
• 26:36
  Speaker 1: Just going to go ahead and stop.
• 26:37
  Speaker 1: So I'm just going to stop the pipeline, so it terminated it.
• 26:39
  Speaker 1: So if we go look at our logs, everything will be stopped over here, and so the next thing we'll look at is once your models are generated they're all stored.
• 26:49
  Speaker 1: So you have you can store it in a database and so you can access them later, download them.
• 26:56
  Speaker 1: I'll show you how to do that.
• 26:58
  Speaker 1: So basically, here we can click on one of these so you'll see the confusion matrix for your training and validation sets.
• 27:07
  Speaker 1: You can see the outperformed in each of the folds right here.
• 27:12
  Speaker 1: There's a feature visualization tab that you can come in and look at which features were selected.
• 27:18
  Speaker 1: What did those features look like?
• 27:20
  Speaker 1: For your validation set.
• 27:21
  Speaker 1: There's a feature embedding tab which is going to compute the embeddings for the features that you did generate and give you kind of a 2D view across all the features so you can look here and see how well they're doing at separating it.
• 27:33
  Speaker 1: The feature summary is going to give you an overview of basically all of the features that you have, broken down by class, and then you can look at the model summary, which is going to tell you how it was trained, pipeline summary and knowledge pack for each of these models that you've generated, and then you can go in and test the model.
• 27:53
  Speaker 1: So we have the labeled session or the data that we've already labeled.
• 27:58
  Speaker 1: I can come here and click on my test data and I can run the model.
• 28:02
  Speaker 1: It's going to compile the firmware, run the model against this data set and give you the results, and then you can go into the download screen, and so we have a bunch of compilers already built in.
• 28:12
  Speaker 1: So if you want to compile for ARM, gcc or Espress expressive, if you want to compile it for Windows or just generic GCC compiler, you can use that.
• 28:21
  Speaker 1: And then these are the development platforms that we're supporting as part of the open source initiative.
• 28:27
  Speaker 1: If other people want to contribute, we can add more development platforms.
• 28:32
  Speaker 1: It's pretty straightforward and we have developer documentation that can walk you through how to do that.
• 28:41
  Speaker 1: Okay, so I wanted to get into the code a little bit and just show you how to add a new feature extractor.
• 28:48
  Speaker 1: So, say, whatever model you're building, you need a particular feature, and the ones we have right now Don't cover what you want.
• 28:57
  Speaker 1: So, hey, chris't cover what you want.
• 28:59
  Speaker 3: So hey Chris.
• 29:00
  Speaker 1: Yeah, go ahead.
• 29:02
  Speaker 3: Actually, I'm trying to unmute Jenny's mic here, sorry.
• 29:06
  Speaker 2: Oh, I mean I'm on.
• 29:07
  Speaker 3: Oh, I'm sorry, no.
• 29:08
  Speaker 3: I just wanted to check in for a second, so before we get into the next section, we had a couple of questions going on here.
• 29:15
  Speaker 3: So one of the questions here was yeah, from James what kind of?
• 29:21
  Speaker 3: Database are you using?
• 29:22
  Speaker 3: Pretty straightforward.
• 29:24
  Speaker 1: Yeah.
• 29:25
  Speaker 3: So, the.
• 29:26
  Speaker 1: We use Postgres for the database, and then all of the data is stored as CSV files.
• 29:34
  Speaker 1: So if you upload data, get stored at CSV files and then the database we track.
• 29:39
  Speaker 1: The database will track all of your metadata and all of your segments and everything like that.
• 29:46
  Speaker 3: Okay, sounds good.
• 29:47
  Speaker 3: Apologies for my screen manipulations there.
• 29:50
  Speaker 3: And then there was one other question.
• 29:55
  Speaker 3: Here it is oh yes, by develop a model using the tools, do I have access to the source code or just the binary?
• 29:57
  Speaker 3: There was one other question here it is oh yes, by develop a model using the tools, do I have access to the source code or just the binary?
• 30:03
  Speaker 3: And a little bit of a biz question here Additional royalty for this.
• 30:06
  Speaker 3: Can you address that?
• 30:10
  Speaker 1: So I can talk about basically developing the model part.
• 30:16
  Speaker 1: So all of this is running locally on your computer, and so you're going to see all of the source code and everything, so you do have access to all of that.
• 30:24
  Speaker 1: So, for example so I'll walk through it a little bit actually in this, so if there's no other questions, I'll do that.
• 30:33
  Speaker 3: Cool, all right.
• 30:34
  Speaker 3: Well, let's kind of get back into it then.
• 30:36
  Speaker 1: Sorry Didn't want to interrupt your flow, but so far so good All right, here we go, all right, yeah.
• 30:43
  Speaker 1: So basically say, you come to a point where you want to add a new feature generator or something to the toolkit because you need that for your pipeline and it's not there yet pipeline and it's not there yet.
• 30:59
  Speaker 1: So we put together some documentation for how to do that and I'm just going to walk through it right now.
• 31:05
  Speaker 1: So if I open up, uh, um, so this is, this is my GitHub, the piccolo GitHub repo, and I've opened it up in VS code and I've come to server library, core functions, feature generators, which is where all the feature generators live, and I've opened to server library, core functions, feature generators, which is where all the feature generators live, and I've opened up the FG stats, which are statistical feature generators.
• 31:24
  Speaker 1: So these are written in Python right here, and then they call C functions directly.
• 31:32
  Speaker 1: So say, we want to do, we want to make a new one which is some column, so it's going to sum up all of the channels.
• 31:39
  Speaker 1: So if I have accelerometer XYZ, it's going to take and sum across all of those channels and give me a new feature.
• 31:44
  Speaker 1: So the first thing I want to do is create a new database entry.
• 31:47
  Speaker 1: So if I come to our functionsprodyaml file I can add this new database entry.
• 31:54
  Speaker 1: It basically tells me the name, what type it is, what subtype it is, where I can find the Python file, what the function in that file is called, and then does it have a C version yet?
• 32:06
  Speaker 1: So we haven't created the C version yet.
• 32:07
  Speaker 1: So I just say that's false and so I'll save that, and then I'll go to FGStats and I'm going to create the python function so you can follow through.
• 32:19
  Speaker 1: But basically come and copy this here, so copy the python function in, and so now here's fgstat, sum columns, and we put a doc string that'll be showed in the ui, and then we also need an input contract and that's going to tell us what are the input parameters for this function.
• 32:40
  Speaker 1: So there's documentation about exactly how to set that up.
• 32:44
  Speaker 1: But basically it lets you specify the parameters that should be shown in the UI or in the Python client to call this function, as well as what should be generated when you generate the embedded code.
• 32:54
  Speaker 1: Okay, it when you generate the embedded code.
• 32:58
  Speaker 1: Okay, and then.
• 32:59
  Speaker 1: So I've added the function to my YAML file and I've added the Python code.
• 33:05
  Speaker 1: So the next thing I need to do is to actually load these functions into the database so the server knows where to find it.
• 33:12
  Speaker 1: So to do that, I'm going to the Docker PS.
• 33:20
  Speaker 1: I'm just gonna use one of these things right here.
• 33:24
  Speaker 1: So if I say Docker exec and then I pick one of the container IDs, I say slash bin, slash bash, it's actually gonna bring me into a bash shell of one of the server or docker containers that's running.
• 33:40
  Speaker 1: And then I just need to say Python managepy, load functions.
• 33:44
  Speaker 1: So the server uses Django underneath and load functions is just our function loader.
• 33:51
  Speaker 1: So it's going to go through this YAML file and it's going to upload all of the functions again, and so you'll see, it's just going to say all right loading all of this stuff.
• 34:00
  Speaker 1: So it's gone and it's loaded everything into the database and now all I need to do so let me just log out.
• 34:07
  Speaker 1: Oops.
• 34:08
  Speaker 1: So now if I come back and I log in again and let's just jump to our pipelines and we want to see if that feature extractor is in there.
• 34:23
  Speaker 1: So if I click feature generators and I say add feature I look at statistical and it's right here.
• 34:32
  Speaker 1: So we've added the sum columns and if I hit info button, um oops.
• 34:39
  Speaker 3: So if I hit info button.
• 34:41
  Speaker 1: here's the doc string that we put in there.
• 34:43
  Speaker 1: So all that got loaded up and now I can just use it as part of my pipeline and my auto ML process.
• 34:49
  Speaker 1: And then the other part of that is if you want to actually add the embedded code too.
• 34:58
  Speaker 1: So there's this other link right here in the presentation so you can click on that and it's going to walk through adding the embedded SDK code.
• 35:07
  Speaker 1: So I won't walk through that right now, but it's pretty straightforward too.
• 35:14
  Speaker 1: So basically, you update your database file to say it does have a C version and point to it, and then you add your source code to the embedded ML SDK and then run a code gen script which is going to generate everything and set it up.
• 35:32
  Speaker 1: And then now not only will you be able to use it in your training pipeline, which is what the Python code is doing you'll be able to generate the firmware and it'll include that function as well.
• 35:49
  Speaker 1: Okay, so that's just kind of.
• 35:51
  Speaker 1: If you're interested in getting started developing, those are places that we always can use help, and the project can always use help.
• 35:59
  Speaker 1: There is just adding new feature generators, adding new transforms, all of those, these different dsp algorithms that might be useful to you are probably useful to other people, um, and so let's see, I want to talk a little bit about using the data studio with piccolo ai.
• 36:17
  Speaker 1: So the studio is one of our applications that we've developed to kind of help manage tag-series sensor data.
• 36:24
  Speaker 3: Quick interrupt.
• 36:25
  Speaker 1: Yeah, go ahead.
• 36:26
  Speaker 3: Hey, chris.
• 36:27
  Speaker 3: Yeah, I just had a couple of questions come in from Alan.
• 36:30
  Speaker 3: I just wanted to address them before you got too deep into the next thingy-ding, is that all right?
• 36:33
  Speaker 1: Yeah, sure, go ahead.
• 36:40
  Speaker 3: Yeah, so here's the first one.
• 36:40
  Speaker 3: Uh, does the current tool support data augmentation?
• 36:41
  Speaker 3: What's the procedure for of adding data augmenters?
• 36:45
  Speaker 1: yeah, so it does um.
• 36:46
  Speaker 3: Let me see if I have I'm going to move jenny up here so that she's not permanently blocked by the comments.
• 36:57
  Speaker 3: I can squeeze in here like this so that she's not permanently blocked by the comments.
• 37:00
  Speaker 1: I can squeeze in here like this, yeah, so if I come to augmentation here, I can add a number of different.
• 37:09
  Speaker 1: So we have a number of different augmentations built in.
• 37:11
  Speaker 1: You can look at how the augmentation works in the code and add your own through the same process.
• 37:18
  Speaker 1: I just showed, uh, if you wanted to add your own, so cool, um okay, sounds good.
• 37:25
  Speaker 1: And then another question from alan uh purpose of having two different versions of functions c and python uh, yeah, so um, basically a lot of times when we're developing we might develop just the Python code first and we want to mess around with it and get that function right, but when we go to actually generate the embedded code for the device, it has to be C code.
• 37:51
  Speaker 1: So basically the strategy we use to handle that is we write the C, c code and then we call it from Python and that way we know whatever we're generating in our training pipeline is going to be the same code that runs on the embedded device, so we don't have any mismatch between the training data and the actual data on the device.
• 38:14
  Speaker 3: OK, cool, alright, Carry on.
• 38:19
  Speaker 1: All right Okay.
• 38:24
  Speaker 1: Okay so, yeah, now I want to just go ahead and talk about the Data Studio and how it interfaces with Piccolo AI server.
• 38:33
  Speaker 1: So our Data Studio is our basically our training data application for collecting data, labeling data.
• 38:41
  Speaker 1: We have some generative tools in there as well and testing models, so you can see right here.
• 38:48
  Speaker 1: Here's kind of like an overview of the interface.
• 38:49
  Speaker 1: Basically, we have our ground truth data that we labeled right here, and then we have one of our models that we've downloaded and run against the ground truth and this is the results it's showing here and you can compare those two in the graph.
• 39:04
  Speaker 1: And then it also shows you a confusion matrix between those two.
• 39:08
  Speaker 1: So to download it, you can go to our download the software, you can go to our download the software, you can go to our, our website and download it, and then you need to connect to the piccolo.
• 39:24
  Speaker 2: AI server.
• 39:25
  Speaker 1: So let me see, so I'll just go ahead and walk through it so here, if you go to settings, network.
• 39:33
  Speaker 1: You can connect to the piccolo AI server.
• 39:35
  Speaker 1: So I'm just setting this to my localhost 8000, which is where my local server is running, and then you just sign into it with the same username and password.
• 39:46
  Speaker 1: So here's all of the projects that I have.
• 39:49
  Speaker 1: So if I open up, sorry, my computer's not happy.
• 40:04
  Speaker 1: Okay, all right.
• 40:22
  Speaker 1: So basically here is the UCI HR data set that we were looking at earlier when we were building the model.
• 40:29
  Speaker 1: Here is some of the models that we've generated.
• 40:35
  Speaker 1: So I can, if I've connected to the server and I've generated a model, I can actually just download that model and run it locally here.
• 40:41
  Speaker 1: So if I just say download, it's going to download that file, oh, failed to compile.
• 40:51
  Speaker 1: I haven't set it up to to download those models yet.
• 40:53
  Speaker 1: So you can download the models locally and you can run them against this.
• 40:57
  Speaker 1: You can also import Python models.
• 41:00
  Speaker 1: So one of the things we'll do.
• 41:02
  Speaker 1: So if I say plus, I can import a model from PLL, I can import a Python model.
• 41:08
  Speaker 1: So, for example, if you wanted to build a model in sklearn and run it in the data studio, you can do that and you can run the data against it.
• 41:17
  Speaker 1: So here's a simple sliding window.
• 41:19
  Speaker 1: So if I wanted to figure out how do I want to develop a um, a segmentation algorithm, I can write it in python, I can run it here.
• 41:30
  Speaker 1: So see it ran.
• 41:32
  Speaker 1: And it put the labels in my test model and then it's comparing.
• 41:35
  Speaker 1: You got down here.
• 41:36
  Speaker 1: You have your confusion matrix, compare against things.
• 41:39
  Speaker 1: You can also import transforms.
• 41:42
  Speaker 1: So, for example, I can import dll transforms.
• 41:49
  Speaker 1: so if they're written in uh compiled to a dll, or if you have a python transform that you want to write, or if you want to import your own, you can do that, and so this is just the absolute value.
• 42:00
  Speaker 1: You can see it down here in the chart so you can set the absolute value.
• 42:05
  Speaker 1: And now not only can I see my sensor data here, but I can also see a transform of the data.
• 42:10
  Speaker 1: So basically, we use the Data Studio as a way to analyze our data, build, test.
• 42:17
  Speaker 1: The models that we've built collect that data as well.
• 42:22
  Speaker 1: So in this project you can see kind of the distribution for the two different classes that we have or the two different labels, so the test model label and then the ground truth label.
• 42:37
  Speaker 1: One of the other nice things let me switch to the robot arm In our project Is if you do have.
• 43:03
  Speaker 1: So this was one of the projects we worked on.
• 43:09
  Speaker 1: Let's get one that has labels on it, so if you do have a video that you've recorded, you can actually play the video and sync it up with your time series sensor data.
• 43:32
  Speaker 1: So a lot of times we'll use this to help us label the data after the fact.
• 43:37
  Speaker 1: So we go through and we collect video and the sensor data at the same time, we sync the video and the data up here and then we can go through and label it.
• 43:46
  Speaker 1: So this was one where we did, where we basically had a set of predefined motions that the robot was going to do and we learned to train on those motions and then detect if it did something it wasn't supposed to do as well.
• 44:04
  Speaker 1: And then the last thing I'll talk about for the Data Studio is we just added so.
• 44:11
  Speaker 1: A lot of times when we're building out keyword spotting models, we want to do text-to-speech.
• 44:15
  Speaker 1: So we just added support with 11 labs as well.
• 44:19
  Speaker 1: So if you're building an audio project, okay, I do have one, okay, so for an audio project.
• 44:45
  Speaker 1: So this was a voice ID model we were building.
• 44:47
  Speaker 1: Basically, the Data Studio has kind of nice built-in spectrogram to help you kind of visualize your data as you're labeling it.
• 44:57
  Speaker 1: But now we also have the ability to use, to directly use kind of text-to-speech and AI voice generators.
• 45:05
  Speaker 1: So we have support for 11 labs right now.
• 45:08
  Speaker 1: We also have kind of a Python package for doing azure and google's as well, to kind of add it to the data studio.
• 45:16
  Speaker 1: So if I come in here and I just select the different voices, I can tell the number of files I wanted to generate and just say generate.
• 45:25
  Speaker 1: It'll tell me how many it's going to do, it'll go and generate these files and then it just adds them to the project and they already have labels in them right now.
• 45:34
  Speaker 1: So that's one of the ways that we're doing data augmentation for our audio stuff.
• 45:45
  Speaker 1: And then you can just go ahead and play the file and listen to it as well.
• 45:52
  Speaker 3: Okay, all well.
• 45:52
  Speaker 2: OK, all right yeah so that covers the Data Studio.
• 45:56
  Speaker 1: So you can use the Data Studio with your local server.
• 45:58
  Speaker 1: If you have videos, the videos will be synced up with the server.
• 46:03
  Speaker 1: They can upload and download them as well, and so, finally, I just want to end with how you can get involved.
• 46:10
  Speaker 1: So for this project to be successful, we really need help from the community and interest from the community for people to start contributing.
• 46:17
  Speaker 1: We think it can be a great kind of confluence of makers, researchers and industry partners to kind of build this project together to help reduce some of the fragmentation.
• 46:31
  Speaker 1: Um, one of the challenges we've always had is, as we're developing something or someone's using our tools, it's like they can get so far, but if you're missing, like that particular feature generator or something, it's very difficult to kind of get there.
• 46:45
  Speaker 1: So we feel like opening up this way is going to make it much easier for people to go in and kind of tailor it to their needs, so you can use piccoloid to build your own projects, share your feedback with us.
• 47:01
  Speaker 1: It's one of the most important ways develop new pipeline templates.
• 47:03
  Speaker 1: So we have some documentation around how to do that, and so those templates for a particular application work well.
• 47:09
  Speaker 1: Adding edge tuned models to the Piccolo AI model zoo Didn't really go over that, but there is a model zoo of models that we've trained, particularly around things like keyword spotting on corpus of data.
• 47:22
  Speaker 1: So adding more of those models to the toolkit, along with the pipeline templates with how to use them.
• 47:30
  Speaker 1: New DSP algorithms and feature extractors One of the things we're working right now is to bring the CMSIS library in so it's much easier to develop against CMSIS for feature extractors.
• 47:42
  Speaker 1: So that's on our enterprise right now.
• 47:44
  Speaker 1: We're trying to bring that over to the open source as well.
• 47:47
  Speaker 1: Create new feature requests so you can go to our projects and look at what's going on there.
• 47:51
  Speaker 1: Create new feature requests so you can go to our projects and look at what's going on there.
• 47:53
  Speaker 1: Create new feature requests there.
• 47:55
  Speaker 1: Help us on some of the features that are already proposed.
• 47:59
  Speaker 1: So we typically go through an RFC process for adding any new major features to the toolkit so as we write them up there you can work with us on those.
• 48:12
  Speaker 1: You can check out our contributing contributor guidelines for kind of how to go about that.
• 48:16
  Speaker 1: You can join the community and contribute.
• 48:20
  Speaker 1: So we have a forum at sensibleorg that we just launched.
• 48:23
  Speaker 1: So go ahead and sign up there.
• 48:25
  Speaker 1: Join the community there.
• 48:26
  Speaker 1: If there's a platform that you want to add, you can add it.
• 48:30
  Speaker 1: We have some documentation for how to do that.
• 48:32
  Speaker 1: We also support AI.
• 48:34
  Speaker 1: You can add it.
• 48:35
  Speaker 1: We have some documentation for how to do that.
• 48:37
  Speaker 1: We also support AI accelerators on chips.
• 48:39
  Speaker 1: So there's an example for how to do it with the Scilabs XG24.
• 48:44
  Speaker 1: Adding some more ML algorithms there's a couple ones that we'd like to add that haven't gotten around to yet just to flesh out the AutoML Testing documentation, tutorials and then becoming a sponsor.
• 48:52
  Speaker 1: And of course, one of the most important things for us right now is just to start the project and follow us, to keep following what we're doing and be part of that community.
• 49:07
  Speaker 3: Awesome.
• 49:08
  Speaker 1: So thank you.
• 49:09
  Speaker 3: We actually have.
• 49:10
  Speaker 3: Jenny had a couple of questions.
• 49:11
  Speaker 3: I think she's still on mute, though I'm sorry.
• 49:12
  Speaker 3: Can I have?
• 49:12
  Speaker 3: Uh, jenny had a couple of questions.
• 49:13
  Speaker 3: I think she's still on mute, though I'm sorry.
• 49:14
  Speaker 3: Can I unmute you?
• 49:15
  Speaker 3: Yeah?
• 49:16
  Speaker 2: I got it.
• 49:16
  Speaker 2: I got it.
• 49:17
  Speaker 2: Uh, great presentation, Chris.
• 49:19
  Speaker 2: That was awesome.
• 49:21
  Speaker 3: Thanks, real good, real good.
• 49:23
  Speaker 3: Go ahead, jenny.
• 49:26
  Speaker 3: I had a couple of good cues there.
• 49:27
  Speaker 2: Yeah, so I just had a couple of questions, um, maybe to help spark some people in, some people in the audience as well, to ask some follow up questions.
• 49:34
  Speaker 2: But you briefly mentioned the auto ML tool that SenseML has.
• 49:39
  Speaker 2: You said that you vary the classifier parameters in order to view the outcomes for the auto ML tool for your model.
• 49:48
  Speaker 2: But do you also change, like DSP code parameters or other things of that nature?
• 49:55
  Speaker 1: Yeah.
• 49:55
  Speaker 1: So there's a couple of different ways you can do that.
• 49:58
  Speaker 1: So our AutoML pipeline right now typically we do like a feature explosion, so we'll generate lots and lots of features, and then we narrow those down to the best ones, and so in the AutoML pipeline it'll go and generate a whole lot of features, and then different feature selectors will run as part of the AutoML process and that'll select different ones that way, and so that's one of the parameters that gets changed, and then of course, the classifiers and the parameters of the classifiers.
• 50:28
  Speaker 1: And then we also have grid search built in and with grid search you can change earlier steps in the pipeline as part of that.
• 50:36
  Speaker 2: Sweet.
• 50:37
  Speaker 3: You mentioned also too about Jenny, about the 11 Labs integration for the Gen AI stuff.
• 50:43
  Speaker 3: I've used the 11 Labs stuff too.
• 50:45
  Speaker 3: It's pretty amazing, the voices are crazy, yeah.
• 50:49
  Speaker 1: So 11 Labs is great.
• 50:52
  Speaker 1: We've also pretty heavily used, like google and azure um, just because the quantity of data you can get out of there is really high, uh, so, versus 11 labs, which is a little bit more, I think when we were first doing a little um, more expensive than kind of those services.
• 51:11
  Speaker 1: So, yeah, uh, we'll just kind of use all of them.
• 51:15
  Speaker 1: They all have different, slightly different stuff, and so the diversity is what's important.
• 51:21
  Speaker 2: Yeah, diversity, and it really just depends on the quality of the incoming keyword that you need to spot as well.
• 51:27
  Speaker 2: You know, the Google services that they've had for years now are sometimes good for that, whereas 11Labs is when you really are.
• 51:38
  Speaker 2: Maybe you're putting your device in a hospital or something and you really need to know exactly what's heard.
• 51:40
  Speaker 2: So that's a good point, but, yeah, so my question about that I asked this question before you even mentioned the 11Labs integration that you had.
• 51:48
  Speaker 2: But, yeah, I was just wondering what other plans for synthetic data generation or augmentation do you have planned at SenseML?
• 51:57
  Speaker 1: Yeah, I guess I mean with the time series sensor data it can be kind of challenging to do.
• 52:02
  Speaker 1: Synthetic data Audio is the one that's kind of obvious and easy to do because we know like what, how it should change to kind of become a different speaker and how it improves the model.
• 52:15
  Speaker 1: With things around like IMU sensor data or other types of sensor data, it's never quite as obvious what should happen.
• 52:24
  Speaker 1: So like, if you change, try and augment the data a little bit, now it's probably a completely different gesture, so you can kind of like slow it down or speed it up a little bit, but if you're not capturing the physics, then you're not doing a good job.
• 52:38
  Speaker 1: So yeah, that's found as much success there, yeah.
• 52:42
  Speaker 2: That's something we're looking at at edge impulse as well, um physics simulators and for, specifically, for sensor data data generation are.
• 52:49
  Speaker 2: They're at an interesting spot where it requires a lot of extra knowledge in order to manipulate those type of 3D environments in order to get that data.
• 52:59
  Speaker 2: So I think there's a little bit more work to be done there for sure, but I'm curious to see what we'll all do in the space.
• 53:04
  Speaker 2: I'm really excited and I have one more question what does SenseML provide that enables model validation pre-deployment?
• 53:16
  Speaker 2: So, like for Alexa certification, you have to run a keyword spotting model on a continuous, 24-hour stream of incoming data in order to get certified, and it has to have very few false reject rates and false acceptance rates.
• 53:32
  Speaker 2: Do you have anything that can do that sort of capability in SenseML or something you're looking into?
• 53:41
  Speaker 1: So typically for things like that, we use the Data Studio.
• 53:45
  Speaker 1: So, it does have a live capture mode and a test model mode.
• 53:51
  Speaker 1: So basically we'll stream data to the device and then we can use this tool here to kind of validate that.
• 54:02
  Speaker 1: So it's typically where we handle that.
• 54:11
  Speaker 2: Cool, fantastic.
• 54:17
  Speaker 3: Sounds good.
• 54:17
  Speaker 3: Well, chrisris, I really appreciate the time, by the way, for our listeners out there.
• 54:19
  Speaker 3: You may have noticed that we actually started the stream.
• 54:20
  Speaker 3: We started talking before we actually hit the live stream button.
• 54:22
  Speaker 3: So that's that's on me.
• 54:23
  Speaker 3: So, um, so you miss, like, the cool opening video.
• 54:28
  Speaker 3: So I'm gonna have to play the closing video very similar to the opening video.
• 54:31
  Speaker 3: So I'm going to have to play the closing video very similar to the opening video, and that'll make up for my goof.
• 54:34
  Speaker 3: But, chris, any final words of wisdom, or you know, you got a lot of good call to action there.
• 54:40
  Speaker 3: I mean, it's fantastic You're doing all this stuff in open source and getting the community contribute, and this is the kind of area where you know people need more and more tools and more and more support.
• 54:49
  Speaker 1: So I think it's pretty awesome, but any, any closing thoughts that, uh, our audience should take away today uh, yeah, we're just, you know, really excited to be able to do this in open source and, um, we hope, to grow the community so awesome.
• 55:06
  Speaker 3: Well, thank you, jenny, for tuning in from amsterdam.
• 55:08
  Speaker 3: Enjoy amsterdam.
• 55:10
  Speaker 3: It's always fun to be in amsterdam, uh, great, and then, uh, we're gonna be back.
• 55:15
  Speaker 3: This is gonna be live streamed every two weeks, so stay tuned for the next one and subscribe to our youtube channel, follow us on linkedin, follow us on twitch, follow us on instagram.
• 55:27
  Speaker 3: All that good stuff, uh, but really appreciate everyone coming in, and I like the comments too.
• 55:33
  Speaker 3: That was really some really good questions as well.
• 55:35
  Speaker 3: So until next time, tiny ml talks.
• 55:39
  Speaker 3: All right, we're going to take it out with the cool video.
• 55:41
  Speaker 2: Thanks everybody thank you so much.
• 55:43
  Speaker 2: Outro Music