Latest / EDGE AI POD / Low Code No Code Platform for Developing AI algorithms

Transcript

Low Code No Code Platform for Developing AI algorithms · EDGE AI POD

• 0:07
  Speaker 1: Good morning everybody.
• 0:08
  Speaker 1: So I'll talk about a low-code no-code platform that we have built for deploying AGI solutions and algorithms.
• 0:17
  Speaker 1: My co-authors are Swapnil Sayansaha and David Ali Priyandi Sayan Saha and David Ali Priyandi.
• 0:24
  Speaker 1: So, as we heard in many talks this morning, there are billions of IoT devices now that are transmitting zettabytes of data to the cloud.
• 0:37
  Speaker 1: This was the very first talk.
• 0:41
  Speaker 1: We heard about 50 billion devices.
• 0:43
  Speaker 1: Who knows?
• 0:44
  Speaker 1: The number might be much bigger or smaller, depending on how you define the IoT devices.
• 0:49
  Speaker 1: However, high-quality data is essential, as we all know, for building good AI ML algorithms.
• 0:57
  Speaker 1: One of the important aspects here is how do you bring in the right kind of data?
• 1:03
  Speaker 1: So I'll talk a little bit about that.
• 1:05
  Speaker 1: And the second thing is that, if you consider building a sensor gateway to cloud model, it's something that is fairly intensive and complex.
• 1:16
  Speaker 1: So one of the approaches that we are taking in this is to put together a low-code no-code solution which will reduce this complexity and solve this problem.
• 1:27
  Speaker 1: So you would have seen you know, a demo of this in a workshop yesterday that was held by my colleague, davide Sergi, in the workshop in the morning.
• 1:37
  Speaker 1: So I'm talking more about you know, this particular solution that we have at ST.
• 1:41
  Speaker 1: So what exactly is ST-IoT Craft solution that we have at ST?
• 1:48
  Speaker 1: So what exactly is ST-IoT Craft?
• 1:49
  Speaker 1: So it's an online, scalable platform and you know, essentially in this particular platform you start out with.
• 1:55
  Speaker 1: You know this SensorTilepro that we have, which has a variety of sensors in it accelerometers, gyroscopes, temperature sensor, pressure sensor, magnetometer, microphone all of these different sensors.
• 2:04
  Speaker 1: It has Bluetooth connectivity and, essentially, some storage on it.
• 2:09
  Speaker 1: So this particular solution now can talk to the cloud solution through a gateway, and this gateway can be your mobile phone, your PC, whatever else it may be.
• 2:24
  Speaker 1: The STI to craft solution has a variety of examples that are available, so you can start with an example that might suit your use case scenario.
• 2:32
  Speaker 1: To become familiar, you have a bunch of example data sets that you can work with in order to try out the whole system.
• 2:39
  Speaker 1: There are project APIs, data set APIs, even job control APIs that are available so you can configure different jobs to put the solution together.
• 2:47
  Speaker 1: You also do have your own cloud app that you can eventually build using a cloud app that we provide.
• 2:55
  Speaker 1: So if you consider all the components that are used to build this thing, you would need a piece of hardware to start with.
• 3:03
  Speaker 1: So we have a bunch of example hardwares that are available, fully functional, that you can use to build this thing.
• 3:04
  Speaker 1: You would need a piece of hardware to start with.
• 3:05
  Speaker 1: So we have a bunch of example hardwares that are available, fully functional, that you can use to build your solution.
• 3:09
  Speaker 1: One of them is Sensortilepro.
• 3:10
  Speaker 1: I'll talk about a few others that you can utilize.
• 3:14
  Speaker 1: Then the next piece of this whole solution is in-sensor AI technology.
• 3:18
  Speaker 1: So we have smart sensors that can run machine learning algorithms in them.
• 3:21
  Speaker 1: Some of these sensors are, you know, available and are being utilized in commercial products such as your laptops, you know, for four or five years now, for detecting things like is the laptop on the lap or on the desk, or is the laptop in the bag, or is the laptop on your, you know, is the person walking with it?
• 3:40
  Speaker 1: All of those solutions are implemented inside the sensor.
• 3:43
  Speaker 1: So in-sensor AI technology is one of those pieces.
• 3:46
  Speaker 1: And then ST software solutions.
• 3:48
  Speaker 1: You know there are a variety of software solutions that we have, which can you know, let you analyze data, let you crop data, prepare this data for machine learning training process and finally, you know this ST AI-AT craft solution that I'm talking about.
• 4:03
  Speaker 1: You know it does have data set and project privacy.
• 4:06
  Speaker 1: It's a very serious aspect of it, which the previous talk that we heard about.
• 4:10
  Speaker 1: So this particular solution also does take care of that, using components such as CubeArmor, grafana, all those such components that you utilize on the cloud side.
• 4:23
  Speaker 1: So let me talk a little bit more about the in-sensor compute part.
• 4:26
  Speaker 1: So, as you can imagine, in-sensor compute part, the fundamental thing it needs is a sensor which interacts with the physical world.
• 4:33
  Speaker 1: So you would essentially use a sensor to measure acceleration, angular rate, magnetic field strength, pressure, temperature, humidity, whatever it may be.
• 4:43
  Speaker 1: Microphone would measure.
• 4:44
  Speaker 1: You know sound, so you know this.
• 4:44
  Speaker 1: You, whatever it may be, microphone would measure sound.
• 4:46
  Speaker 1: So this sensing data essentially will be the first piece.
• 4:51
  Speaker 1: Then there's a computational block.
• 4:53
  Speaker 1: So if you can apply filters on this sensor data, your machine learning algorithms will perform better.
• 4:58
  Speaker 1: So we have a method of automatically selecting the filters.
• 5:03
  Speaker 1: I'll talk a little bit more about in the next few slides.
• 5:05
  Speaker 1: And then you have, you know, predefined feature blocks available inside the sensor.
• 5:10
  Speaker 1: So put together an algorithm.
• 5:12
  Speaker 1: You know there are a variety of feature blocks, such, as you know, mean variance, energy in the band, peak-to-peak value, zero crossing.
• 5:20
  Speaker 1: All those features can be utilized and then, finally, you can run your decision tree inside the sensor and you can even apply a meta-classifier on that decision tree's output.
• 5:30
  Speaker 1: So now to implement this using STI IoT Craft.
• 5:38
  Speaker 1: I'll talk about the pieces for that.
• 5:40
  Speaker 1: So the very first is the supported hardware.
• 5:42
  Speaker 1: So there are a variety of hardware blocks that are available.
• 5:46
  Speaker 1: One of them is this STWinbox, so it contains industrial grade sensors accelerometers, gyroscopes, temperature sensor.
• 5:56
  Speaker 1: It has Bluetooth, low energy connection, so you essentially can extract the data, the stored data, or you can extract the data in real time.
• 6:04
  Speaker 1: Sensortilebox Pro it has consumer-grade sensors, so accelerometers, gyroscopes, pressure sensor, humidity sensor, microphone, and there are two flavors of these S-T-Winbox, which contains industrial-grade sensors.
• 6:26
  Speaker 1: On S-T-A-O-T-E-Craft, you have a variety of examples that are available.
• 6:29
  Speaker 1: So we you know one of the features of this is called as flash and run.
• 6:37
  Speaker 1: So there are six examples available on STI IoT Craft.
• 6:39
  Speaker 1: So, sti IoT Craft, you can access that through a browser, so you can go to stcom, you can search for STI IoT Craft Craft and essentially, you know you'll be at a landing page where you will see six of these examples.
• 6:49
  Speaker 1: So for industrial, the examples are smart asset tracking there's an example for fan coil monitoring.
• 6:57
  Speaker 1: There is an example for a smart power tool.
• 7:00
  Speaker 1: So using the sensor on a power tool, you can determine the state of the power tool like a drilling machine For consumer applications.
• 7:08
  Speaker 1: There are three more examples, like human activity recognition, gesture recognition, and there are two flavors of gesture recognition.
• 7:17
  Speaker 1: So these examples will allow you to work with an available set of data that's already on the cloud and you can actually flash this down to the sensor tile all the way from your browser.
• 7:31
  Speaker 1: So main idea here is that you have a complete working solution that you can try out without having to write any chunk of code.
• 7:38
  Speaker 1: So here is an example of what you will be able to see on the browser.
• 7:44
  Speaker 1: So the top trace that you see here is a real-time trace, which is, of course, a screenshot for accelerometer data.
• 7:52
  Speaker 1: So the three axes of accelerometer data are shown here.
• 7:55
  Speaker 1: So in case of a smart asset tracking example, you would have four different classes in this case that are being shown, would have four different classes in this case that are being shown.
• 8:03
  Speaker 1: So the output of algorithm running on the edge in the sensor dial is visible to you on the browser in real time.
• 8:11
  Speaker 1: So there are four classes here.
• 8:13
  Speaker 1: So the first class is that the asset is stationary but upright.
• 8:18
  Speaker 1: The second one is asset is not upright.
• 8:22
  Speaker 1: The third one is asset is in motion.
• 8:24
  Speaker 1: The fourth one is asset has been shaken.
• 8:26
  Speaker 1: So all of this can be seen in real time on the cloud using your browser on the cloud.
• 8:38
  Speaker 1: So how do you create a new project?
• 8:39
  Speaker 1: So the very first thing that you would do is you would define a data set.
• 8:42
  Speaker 1: This data set essentially would contain whatever.
• 8:46
  Speaker 1: If you are collecting data for this smart asset tracking example, you would be logging data for when the box is stationary or when the box is in motion.
• 8:56
  Speaker 1: You would be collecting data when somebody is shaking, this box is sitting upright or not upright.
• 9:02
  Speaker 1: All of that.
• 9:03
  Speaker 1: So you collect all of this data.
• 9:04
  Speaker 1: So you define your data sets.
• 9:05
  Speaker 1: Then you import your data into the project.
• 9:08
  Speaker 1: This could be done using something like sensor tile or you could bring in your own data.
• 9:14
  Speaker 1: If you have logged this data previously, you can import this data.
• 9:17
  Speaker 1: You define your new project and then you define your new model that you want to use.
• 9:21
  Speaker 1: You train the model and you can test this model in real time.
• 9:27
  Speaker 1: So for data management, you know there are two, three different aspects.
• 9:32
  Speaker 1: Number one is data acquisition.
• 9:33
  Speaker 1: So the data acquisition essentially can be done using this STI IoT Craft web portal.
• 9:37
  Speaker 1: So web portal will allow you to define your sensor.
• 9:41
  Speaker 1: You know, will essentially establish the path all the way from cloud through the gateway to the sensor node, and then you know, will essentially establish the path all the way from cloud through the gateway to the sensor node, and then you know, you basically define, you know which sensors you're collecting data from.
• 9:52
  Speaker 1: You can collect this data from multiple such sensors or you can use the mobile app.
• 9:56
  Speaker 1: There's a mobile app that's available which will allow you to do the same thing.
• 10:00
  Speaker 1: You can upload previously acquired data.
• 10:03
  Speaker 1: So we have a CSV wizard.
• 10:04
  Speaker 1: So this CSV wizard will allow you to transform this data into the format that is expected by STI IoT Craft.
• 10:12
  Speaker 1: So you could have different sampling rates for sensor data.
• 10:16
  Speaker 1: You could have different scales for the sampling data, so all of those could be transformed.
• 10:21
  Speaker 1: So essentially, it will be able to deal with files that are in heterogeneous data sets.
• 10:27
  Speaker 1: So some files might contain sensor data with, let's say, 50 hertz of sampling rate.
• 10:33
  Speaker 1: Others might contain 200 hertz of sampling rate.
• 10:36
  Speaker 1: So you can downsample the sensor data to the chosen sample rate and the CSV wizard will guide you through that process.
• 10:43
  Speaker 1: You can also download those data sets onto your laptop, your desktop device, and do your own analysis.
• 10:49
  Speaker 1: So STI IoT Craft provides you that capability.
• 10:57
  Speaker 1: So the other aspect is a visualizer that's available to you so you can visualize this data.
• 11:02
  Speaker 1: You can zoom into the data, crop the data, you can label the data.
• 11:07
  Speaker 1: The cleaning of all of this data can be done utilizing this particular tool.
• 11:12
  Speaker 1: And now let me come back to this one.
• 11:16
  Speaker 1: So now next step is the model training.
• 11:18
  Speaker 1: So in case of model training, it's a one-button solution that we have.
• 11:22
  Speaker 1: So the very first piece that you see here is that there is training data that's available.
• 11:26
  Speaker 1: So this tool lets you see, in case of asset tracking example, which is what I'm describing here, there are four classes the device is stationary, or asset is stationary, it is in motion, asset is upright, or it's not upright, or it's been shaken.
• 11:43
  Speaker 1: So you can visualize this using this data that all of the data that you have is sort of equally distributed, that you don't have too much of data, let's say, stationary data, that your 90% data is for stationary.
• 11:55
  Speaker 1: That is something that you want to avoid.
• 11:57
  Speaker 1: So you would be able to see that Next step is you can select the sensors.
• 12:03
  Speaker 1: Next step is that you can select the filters and features and train your solution.
• 12:07
  Speaker 1: So it can all be done using one button.
• 12:09
  Speaker 1: You know, a single button, without writing any line of code.
• 12:13
  Speaker 1: So what you see here is all the default features that have been selected.
• 12:17
  Speaker 1: So for filters, there are multiple methods that you can utilize a simple feature selection method or, you know, exhaustive feature selection method.
• 12:24
  Speaker 1: Two different methods that we can utilize a simple feature selection method or an exhaustive feature selection method, two different methods that we have developed.
• 12:28
  Speaker 1: For features, same thing the tool will automatically select the features for you based on the label data that you have provided.
• 12:35
  Speaker 1: So how does it work?
• 12:38
  Speaker 1: So the very first step is your hyperparameter tuning in this case, which is you would select basic or exhaustive, one of those two methods of selecting filters.
• 12:50
  Speaker 1: These filters will be applied to XYZ axis of the sensor data or the norm of the sensor data, or norm squared.
• 12:57
  Speaker 1: You can choose that.
• 13:00
  Speaker 1: Next step is automatic window length selection.
• 13:03
  Speaker 1: So, as you can imagine, when you are applying these features, you'd be applying that to a window of data.
• 13:09
  Speaker 1: So, looking at the spectral content, essentially this algorithm will be able to pick up what is the right window length.
• 13:15
  Speaker 1: So imagine this example of human activity recognition.
• 13:19
  Speaker 1: So for human activity recognition, if you want to detect if a person is walking, running, jogging, a one second update would be good enough.
• 13:27
  Speaker 1: So if your sensor data is acquired at, let's say, 25 hertz, your window length in this case would be 25 samples.
• 13:35
  Speaker 1: Automatic feature selection.
• 13:37
  Speaker 1: So there are five different methods that compose this automatic feature selection.
• 13:43
  Speaker 1: So you have random forest, you have ANOVA, you have AdaBoost, you have sequential feature addition or recursive feature elimination.
• 13:51
  Speaker 1: So using any of these five methods, or a combination of these five methods, you can arrive at a set of features that will be utilized to train your model.
• 14:01
  Speaker 1: On Now, sensor you know that we talked about smart sensor.
• 14:07
  Speaker 1: It offers you 31 such features.
• 14:08
  Speaker 1: You know such features include mean, variance, energy in the band, peak-to-peak value, zero, crossing, all of those.
• 14:15
  Speaker 1: So essentially, you will go through that process.
• 14:18
  Speaker 1: Next step would be, you know, generating a decision tree.
• 14:21
  Speaker 1: So this can all be done automatically.
• 14:24
  Speaker 1: Now, once the decision tree is generated, you would generate the.
• 14:30
  Speaker 1: You would transform that decision tree into a file that will be downloaded ultimately into the sensor.
• 14:35
  Speaker 1: So, from cloud through gateway, this new decision tree would be downloaded onto the sensor tile or stwinbox one of those two devices or your own platform.
• 14:45
  Speaker 1: So there's a bidirectional communication that you would need to establish, which is what I'll talk about next.
• 14:50
  Speaker 1: So bottom line here is that you have this whole process of extracting filters and features, applying filters and features to the sensor data that you have made available, generating a decision tree and then preparing that to be downloaded down into the sensor.
• 15:13
  Speaker 1: Finalizing all of that, you know, preparing that in the right format.
• 15:17
  Speaker 1: It is all done using one button.
• 15:19
  Speaker 1: So you know, just you know, try.
• 15:22
  Speaker 1: Mlc button will do all of this, so it will prepare the solution for the machine learning core in the sensor.
• 15:27
  Speaker 1: Of course, it will also give you the statistics when you provide label data.
• 15:32
  Speaker 1: Through this process, you will arrive at a decision tree and that decision tree, the statistics on that, will be presented in that.
• 15:42
  Speaker 1: So now let's briefly talk about the IoT system itself.
• 15:44
  Speaker 1: So the IoT system essentially has these three components.
• 15:48
  Speaker 1: So sensor nodes, so sensor nodes.
• 15:50
  Speaker 1: You can use these use cases that are shown on the portal.
• 15:54
  Speaker 1: There are six examples I talked about.
• 15:56
  Speaker 1: So you can use one of those to start with, or you can bring in your own data.
• 15:59
  Speaker 1: So, using the CSV wizard that I talked about, you can upload your own data for processing.
• 16:05
  Speaker 1: The second piece is the gateway.
• 16:07
  Speaker 1: So gateway runs a logic called as data sufficiency logic, which means the data that has been transferred from the sensor tile is processed on Gateway, where you're conserving the communication bandwidth.
• 16:22
  Speaker 1: You're not shipping stationary data because if you just consider the asset, most of the time it'll be stationary.
• 16:30
  Speaker 1: You don't want to keep on shipping the stationary data all the time to the cloud.
• 16:33
  Speaker 1: You want to ship relevant data for all the selected classes.
• 16:37
  Speaker 1: That will be done on the gateway.
• 16:39
  Speaker 1: So gateway.
• 16:40
  Speaker 1: The other thing it does is it does protocol translation.
• 16:43
  Speaker 1: So you can connect your sensor tile to the gateway using either BLE Bluetooth Low Energy Connection or using USB Bluetooth low energy connection or using USB.
• 16:51
  Speaker 1: So the protocol translation essentially does, you know, take this Bluetooth BLE data and then prepares that in the format which can be processed by the data sufficiency logic.
• 17:02
  Speaker 1: The next piece it does is identity translation.
• 17:06
  Speaker 1: You can imagine hundreds of thousands of those sensor tiles that are deployed, or your sensors that are deployed all over.
• 17:15
  Speaker 1: So you need to set up a bidirectional communication between the sensor smart sensor through this gateway to the cloud.
• 17:19
  Speaker 1: So this identity translation is done for each sensor with this digital twin in the cloud.
• 17:27
  Speaker 1: So there are multiple paths of building a solution.
• 17:29
  Speaker 1: So the very first one that can be is you take your sensor tile using a USB connection to your laptop.
• 17:36
  Speaker 1: You can essentially try this out.
• 17:37
  Speaker 1: You have your solution STIoT craft solution which is running in the cloud.
• 17:42
  Speaker 1: You have a web browser to access it and essentially it does this.
• 17:45
  Speaker 1: Next one is you can bring in a Raspberry Pi kind of a gateway.
• 17:49
  Speaker 1: This gateway essentially will run your data sufficiency logic, will do some storage for you and then communicate through a STIoT Craft client app to the cloud solution.
• 18:03
  Speaker 1: Next implementation you can have is you can build your own client app if you do not want to utilize the one that is provided with STIoT Craft.
• 18:09
  Speaker 1: And finally, if you are happy with this solution, then you can essentially to utilize the one that is provided with STI IoT Craft.
• 18:12
  Speaker 1: And finally, if you're happy with the solution, then you can essentially take all the logic that you have developed, which you have tested using Sensortilepro, onto your own device.
• 18:22
  Speaker 1: Your device could be your smartwatch, your IoT device, it could be your you know, whatever that you know industrial sensor node, whatever it may be.
• 18:30
  Speaker 1: And finally, just describing this, what are the different components that we have?
• 18:36
  Speaker 1: This is sort of to give you a full idea of what this IoT system is.
• 18:39
  Speaker 1: So, in case of IoT system, I talked about protocol translator, identity translator, the client app, all of those pieces.
• 18:51
  Speaker 1: There is Cosmos DB that is used to essentially store all of this data in, you know, globally.
• 18:57
  Speaker 1: You can have, you know, this solution deployed using Cosmos DB kind of storage mechanism.
• 19:02
  Speaker 1: So, in summary, we talked about a platform that uses distributed computing paradigm.
• 19:08
  Speaker 1: You run, you know, your AI logic inside the sensor, which essentially will give you a solution that is very, very effective in terms of reducing the battery power consumption for a battery-powered device.
• 19:24
  Speaker 1: You run logic, such as data sufficiency logic, on the gateway and the third chunk of logic runs in the cloud, which is you, you know, generate the best AI algorithm for the selected sensor.
• 19:36
  Speaker 1: Data and adaptation will occur here as well, in the cloud.
• 19:39
  Speaker 1: So you know, we have a solution that can be used to, you know, put together a solution that is very effective in terms of being power efficient, you know, fully deployable, worldwide and scalable.
• 19:51
  Speaker 1: That's all I have.
• 19:53
  Speaker 1: I can take any questions that you may have.