Latest / EDGE AI POD / TinyML Implementation for a Textile-Integrated Breath Rate Sensor

Transcript

TinyML Implementation for a Textile-Integrated Breath Rate Sensor · EDGE AI POD

• 0:09
  SPEAKER_00: I'm a senior scientist there.
• 0:10
  SPEAKER_00: My name is Jorgios Kokinis, and today on a bit of a different note from most of the inputs from the industry, I will be presenting our work on a tiny ML implementation of for a textile integrated breath rate sensor, a bit from an academic point of view.
• 0:33
  SPEAKER_00: Some words about Silicon Austria Labs is a research institute publicly funded, based in Austria, in three cities in Austria, and is focusing its research on all kinds of electronics-based systems, from power electronics to embedded systems, system integration technologies, RF systems, and sensor systems.
• 0:58
  SPEAKER_00: My team is electronic sensors, and we are conducting research on sensor elements but also readouts systems for sensor nodes.
• 1:12
  SPEAKER_00: Thus, naturally, we get into the realm of edge AI and recently started working on the field.
• 1:24
  SPEAKER_00: Since I'm gonna be discussing textile integrated electronics, some driving factors behind this niche of electronics.
• 1:36
  SPEAKER_00: One is the miniaturization driving factor.
• 1:41
  SPEAKER_00: Since the traditional transistors, we have come a long way to grain sites MCUs.
• 1:50
  SPEAKER_00: And on the other hand, we have the other main driving factor, which is additive manufacture, which also incorporated in our project with multimaterial 3D printing.
• 2:02
  SPEAKER_00: Of course, we cannot forget about market potential and societal benefits.
• 2:10
  SPEAKER_00: The last years there are all kinds of new usages of e-textiles in diverse applications from safety features, heating, health monitoring, rehab, fitness, and every now and then you have a lot of new applications popping up.
• 2:31
  SPEAKER_00: Of course, you can already purchase, and there's a lot of products in the market that are really highly advanced, but academia still spearheading the field.
• 3:00
  SPEAKER_00: In the context of flexible of textile integrated electronics, flexible interconnects is a holy grail.
• 3:10
  SPEAKER_00: Most of the solutions we find in academia and in industry is basically consisting of the electric islands where the circuits have been assembled, and then there is a search for interconnecting technologies that could deem these systems viable and robust.
• 3:40
  SPEAKER_00: But what is the need for textile integrated electronics and sensor in healthcare?
• 3:46
  SPEAKER_00: Of course, you are aware of the aging of population, especially in Europe, and this and the substantial burden it poses on the social healthcare systems.
• 3:59
  SPEAKER_00: So is there a way through adaptation of such systems to reduce the hospitalization days in hospitals?
• 4:11
  SPEAKER_00: To these burdening issues, our project Text Hypes, Textile Integrated Hybrid Printed Electronics, comes to bring some solutions to answer some questions on the one hand by remote patient monitoring systems where alerts can be provided to healthcare professionals about a deterioration in the health of a patient, and on the other hand to provide a seamless integration system for electronics and sensors into garments.
• 4:46
  SPEAKER_00: So our approach to the issue is similar to other systems that are that are in an academic mainly field.
• 4:59
  SPEAKER_00: So is to have a 3D printed multi-layer dielectric island where the assembly of the circuit comes later on and then use in our case switch back embroider interconnects with a swing machine, but also to develop embroider embroider uh breath rate sensing elements.
• 5:38
  SPEAKER_00: So we came to develop the Tex Hype Belt.
• 5:41
  SPEAKER_00: We have all these kinds of modules, the power module and 3D printed LED, a heart rate sensor based on a PPG sensor module, an MCU and accelerometer, and finally a breath rate sensor, and here you see an early prototype of the system.
• 6:03
  SPEAKER_00: I'm focusing here on the breath rate sensor.
• 6:06
  SPEAKER_00: We have the different layers of the dielectric island upon which, which was already 3D printed, upon which uh conductive ink was also dispensed.
• 6:20
  SPEAKER_00: And uh we used an embroidered carbonized silicon yarn to basically form a strain gauge that is measuring the expansion of the chest.
• 6:32
  SPEAKER_00: So we developed uh this sensor module with the MCU and then we had to decide what we're gonna do with it.
• 6:40
  SPEAKER_00: Uh the first obvious solution is to use some peak detection algorithm, but then we came with uh really big hurdles because such sensors have a burn-in effect, but also there is a lot of offsets appearing at random points during operation that also the peak detection algorithm can not cannot tackle.
• 7:04
  SPEAKER_00: That's the obvious uh solution it was to look for into an edge AI implementation.
• 7:12
  SPEAKER_00: First of all, we screened for public data sets.
• 7:16
  SPEAKER_00: Uh unfortunately, most of them used ECG and PPG data that were then labeled using impedance monography or capnography, and we only found one data set that it was using a sensor similar to our to our system.
• 7:36
  SPEAKER_00: Umfortunately, that one as well had several limitations, which I will touch on this on the next slides.
• 7:43
  SPEAKER_00: Uh so we opted to create our own custom data set.
• 7:47
  SPEAKER_00: Uh volunteers uh was were recording data in a seated or in standing position, and of course we took care of anonymity in the labeling.
• 8:02
  SPEAKER_00: Going back to the wog dataset, main hurdle was that it was actually missing labels.
• 8:10
  SPEAKER_00: So ironically, we used the peak detection algorithm to label the data on ourselves.
• 8:17
  SPEAKER_00: Um here you see a good a nice example of how this worked perfectly well, but then we had data that looked like this, and actually it was not very easily usable.
• 8:31
  SPEAKER_00: Here we also have an image of the raw signal and and the spectrum.
• 8:39
  SPEAKER_00: So we gathered i data, we uh went through a data pre-processing pipeline where we cleaned filter with a bandpass filter of 0.1 to 1 Hz.
• 8:51
  SPEAKER_00: This is the region that is of interest.
• 8:54
  SPEAKER_00: We segmented it into three-second splits, normalized them, and labeled them via an event push button.
• 9:03
  SPEAKER_00: That was also a uh a pretty good strategy as we could have a very well-defined uh birth rate on our data.
• 9:17
  SPEAKER_00: Moving on to the model architecture, here we heavily depended on uh edge impulse and the neural architecture search feature via the Eon tuner.
• 9:31
  SPEAKER_00: There was an automated process to find the optimal neural network architecture design.
• 9:37
  SPEAKER_00: We narrowed down our search to two uh two distinct models.
• 9:43
  SPEAKER_00: One used a convolutional neural network where the IDMU input was the raw signal with some pre-processing steps on the MCU, and the second was a deep neural network with a wavelet transformation.
• 9:59
  SPEAKER_00: Naturally, the CNN uh net model had uh many more parameters than the DNN uh and we proceed on training both on both datasets, and we had a really good convergence, and in the metrics it was uh obvious that the text hype dataset uh was performing uh slightly better, and the CNN model was performing always better than the DNN.
• 10:41
  SPEAKER_00: This followed the model deployment.
• 10:43
  SPEAKER_00: Uh you we use an STM32L4 microcontroller, some initial pre-buffering uh pre-processing steps to provide provide the data for the CNN network and the DNN network.
• 10:58
  SPEAKER_00: Uh we had we acquired the data through an analog to digital converter via DMA, and we use two timers, one to precise control the sampling intervals, and the second just as uh a solution to measure the intervals on the inference time.
• 11:16
  SPEAKER_00: Um the pre-processing on the MCU consists of an uh FIR filter from the CMC's DSM library, um some buffering and event flag, and finally some um minimum and maximum normalization.
• 11:31
  SPEAKER_00: The post-training optimization uh significantly reduced the size of the models by threefold, and uh here I included an image of the filter data from the sensor.
• 11:48
  SPEAKER_00: Finally, we run the inference at the edge.
• 11:53
  SPEAKER_00: For the convolutional network, um we had a relatively uh large inference time, close to six seconds.
• 12:04
  SPEAKER_00: Uh but also as we've seen before the results were much better, and we have uh a power consumption between 3.3 to 6.6 millibytes.
• 12:16
  SPEAKER_00: Well from the DNN that was the maximum in power consumption, on the other hand, was much faster in inference time.
• 12:28
  SPEAKER_00: Now we had two datasets um and we wanted to try uh the public dataset that was trained uh from sensor data from a different sensor to see how it well it j it generalizes on the sensor data from our model.
• 12:49
  SPEAKER_00: Um unfortunately we see that uh the model does not generalize very well.
• 12:57
  SPEAKER_00: We have a main average error of 8.3 for the CNN and the 10 for the DNN.
• 13:05
  SPEAKER_00: And so the next step would be to fine-tune the model with the data from the text-hype dataset and see if we can uh improve the performance.
• 13:14
  SPEAKER_00: On the other hand, uh of course the dataset that was used to train the model and was uh getting input from the same sensor uh perform much more better.
• 13:30
  SPEAKER_00: So the next step uh is to use uh to further collect uh data and train AI and models towards the remote patient and monitoring system that I uh look uh that I mentioned in the beginning.
• 13:46
  SPEAKER_00: Uh currently we are focusing on uh hydration uh prediction and finally of course use some uh embedded AI integration of uh such a system.
• 14:01
  SPEAKER_00: So that brings me to the end of my presentation.
• 14:05
  SPEAKER_00: Thank you very much for