Latest / EDGE AI POD / Pioneering Personalized Medicine with Advanced AI Algorithms

Transcript

Pioneering Personalized Medicine with Advanced AI Algorithms · EDGE AI POD

• 0:00
  Davis Sawyer: Good morning, good afternoon, good evening to everyone joining us from around the world.
• 0:04
  Davis Sawyer: My name is David Sawyer.
• 0:06
  Davis Sawyer: It's my great pleasure to welcome everyone on Zoom, everyone on YouTube and everyone on LinkedIn to today's TinyMail Talk.
• 0:12
  Davis Sawyer: Today, we're going to talk about unleashing the power of TinyMail Network models in medical devices from Jim Wang and Steve Yu at the University of Sydney.
• 0:20
  Davis Sawyer: It is now 8 am Pacific, so let's go.
• 0:22
  Davis Sawyer: It is now 8 am Pacific, so let's go.
• 0:29
  Davis Sawyer: First of all, as always, we'd like to thank our sponsors who make time possible AI, zip, arduino, arm, brainship, edge, impulse, embedur, bfissions, greenwave Technologies, gravity Inc, imax, imagimo, infineon, inatera, noda, ai, mxb, procter, gamble, orvo, qualcomm, renaissance, schneider Electric, senseml, sony, silicon Labs, stmicro, synaptics, sensein and TDK.
• 1:00
  Davis Sawyer: Thank you for making TinyMail possible.
• 1:07
  Davis Sawyer: If you are familiar with TinyML, if you're new to it, both of these are great places to get involved.
• 1:09
  Davis Sawyer: We have a growing meetup group with over 20,000 members in 42 countries, and the TinyMail community on LinkedIn is available here at the URL or QR code, with over 4,000 members and 16,000 very active followers.
• 1:21
  Davis Sawyer: In addition to the meetup groups and LinkedIn, we have a couple of ways you guys can get involved, also the YouTube channel.
• 1:32
  Davis Sawyer: So the YouTube channel is where all the material from TinyMail is available for free all the time.
• 1:36
  Davis Sawyer: All the talks like this one will be posted there so you can subscribe and stay notified from all the stuff happening all across the world in TinyMail and coming up.
• 1:47
  Davis Sawyer: This is a super exciting in-person event.
• 1:48
  Davis Sawyer: We do have a few in-person events that happen around the year around the globe.
• 1:52
  Davis Sawyer: TinyML EMEA is in Milan, italy, june 24th to 26th, so be sure to register now.
• 1:59
  Davis Sawyer: You know this is one of the most fantastic ways to get involved by actually going to one of these live events.
• 2:04
  Davis Sawyer: So a great pleasure to welcome our first of two speakers.
• 2:07
  Davis Sawyer: Jim Wang is a second-year PhD student in the School of Biomedical Engineering at the University of Sydney.
• 2:12
  Davis Sawyer: His research focuses on the application of tiny ML in the analysis of biosignal data, particularly in the realm of medical diagnostics, with a keen interest in leveraging cutting-edge technology for healthcare advancements.
• 2:22
  Davis Sawyer: He is committed to exploring the potential of machine learning and addressing critical challenges in biomedical engineering, and Jim will be joined by his colleague, Steve Yu.
• 2:33
  Davis Sawyer: Steve is a second-year master's student at the University of Sydney School of Biomedical Engineering.
• 2:38
  Davis Sawyer: He's dedicated to researching circuit design, signal processing and system development, particularly in the realm of biosignal hardware, showcasing a strong interest in exploring diverse devices for biosegnal measurements.
• 2:49
  Davis Sawyer: Steve Jim, thanks very much for being here, looking forward to your talk.
• 2:52
  Davis Sawyer: Please take it away.
• 2:59
  Jim Wang: All right, thank you, Davis, for the introduction.
• 3:03
  Jim Wang: Yeah, my name is Jim and I'm here with my colleague, steve, and we are going to talk about, you know, how we can apply, you know, tiny neural networks in wearable devices, and we currently are based in Sydney and it's late evening here.
• 3:25
  Jim Wang: So if we act a bit slow, and you know, please be, you know, patient.
• 3:31
  Jim Wang: Thank you, and now, first of all, I'll just pass on to my colleague, steve, and he'll be talking about some backgrounds and, you know, some challenges that we'll be facing in the research fields, and then I will be carry on from there to talk about, you know, the current progress that we'll be facing in the research fields, and then I will carry on from there to talk about the current progress that we have and the future directions of our research, and we will welcome questions along the way.
• 3:55
  Jim Wang: So if you have any questions, please drop in the Q&A chat box and then we'll be happy to take the questions along the way.
• 4:04
  Jim Wang: So now I'll hand it over to my colleague, colleague Steve, to start with the presentation of the day.
• 4:11
  Steve Yu: Thank you, Okay, thank you, jim.
• 4:13
  Steve Yu: So the presentation is divided into five different sections.
• 4:21
  Steve Yu: For the first two sections, we will be talking about the background and the current challenges of the field.
• 4:26
  Steve Yu: Well, for the later sections, we are going to talk about the context regarding to our research.
• 4:31
  Steve Yu: Then we start with the introduction part.
• 4:36
  Steve Yu: Our research is currently focusing on the biosignal devices.
• 4:42
  Steve Yu: Well, among all different device, variables these days are becoming a more and more popular type of device for recording bell signals.
• 4:50
  Steve Yu: Variables are the devices that you can wear every day.
• 4:54
  Steve Yu: Various variable devices with diverse applications are available to gather various physiological data.
• 5:01
  Steve Yu: These devices also process the captured signals using techniques shown on the screen.
• 5:10
  Steve Yu: In the health care domain, these devices are utilized for various purpose applications.
• 5:15
  Steve Yu: For example, the current device shown on the slide is a device for recording EEG signals.
• 5:23
  Steve Yu: It can have the different applications, like brain-computer interface, so that you can control your light in your home with the source of your brain, or it can also be used for like the recording, the seizure detection and medical field.
• 6:06
  Steve Yu: Machine learning can help health professionals to identify parameter patterns within data that have been related to some specific disease.
• 6:11
  Steve Yu: This not only helps them to save their time and the decision-making time, but also allows doctors to focus on their time to on the more important things, like how to cure the disease.
• 6:22
  Steve Yu: By automatically marks a certain time part of the data, machine learning is also simplified the patient care and treatment procedures, and this leads to a more efficient and effective healthcare delivery.
• 6:36
  Steve Yu: Moreover, machine learning helps in the early identification of the potential issue from the bell signals, enabling early treatment.
• 6:45
  Steve Yu: This approach contributes to better patient outcomes and overall health care management.
• 6:53
  Steve Yu: The integration of machine learning into health care not only enhance the diagnosis accuracy, but also helps to develop more personalized patient management strategies.
• 7:07
  Steve Yu: Therefore, achieving precise AI processing on resource-limited devices becomes more necessary now.
• 7:19
  Steve Yu: With that in mind, our goals here are to optimize bell signal models so that they can operate with low latency and minimal power consumption, while maintaining the performance of the AI model.
• 7:33
  Steve Yu: Then we are going to talk about several current challenges we are meeting in the research field.
• 7:37
  Steve Yu: Currently.
• 7:39
  Steve Yu: There are several different challenges in the field and we are going to focus on the five ones.
• 7:44
  Steve Yu: We think think the most critical ones Model generalizability, which is the ability for the model to operate on external datasets.
• 7:53
  Steve Yu: After training on the existing datasets, the model will need to operate on a totally different patient out of the dataset and it's still required to have a very good performance.
• 8:06
  Steve Yu: Personalization is the ability to customize the algorithms for specific user preference to offer like a better accuracy, like it is usually done by training the model with the user's own data.
• 8:23
  Steve Yu: Model size basically is the size of the model and the memory needed for the model to operating.
• 8:30
  Steve Yu: This is important because we are operating our model on the HDAC device and on HDAC device there is very limited computation resource, so we need to be careful with the model size.
• 8:43
  Steve Yu: Model performance basically this is the accuracy, latency and the resource efficiency of the model and we need to have a very good performance so that it can really do its job.
• 8:56
  Steve Yu: Talk consumption is basically the power needed for the model to operating on the device.
• 9:04
  Steve Yu: This is important because for the wearables, they are basically rely on the batteries to operating.
• 9:20
  Steve Yu: Generalizability is one of the major challenges of the model.
• 9:23
  Steve Yu: There are three different aspects of the things that will influence the generalizability of the model.
• 9:35
  Steve Yu: For example, different group of people will have different features regarding to the signal and this will lead to the difference in the bell signals.
• 9:41
  Steve Yu: Therefore, we need to be careful with that.
• 9:44
  Steve Yu: And regarding to the environmental factors, for example, noise For different regions in the world.
• 9:53
  Steve Yu: For the polar noise it can be 50 Hz or 60 Hz.
• 9:57
  Steve Yu: Also, when you are walking around with like a machine which is operating, it will also lead to a noise with a certain frequency in the signal.
• 10:08
  Steve Yu: Therefore, the model needs to be able to handle all these different conditions of the noise and also motion artifacts, while using our model on the wearables.
• 10:22
  Steve Yu: This means, like the patient will not sit there and do the recording.
• 10:26
  Steve Yu: The patient may be walking around, running around or even jumping up and down, and this means, like this will cause a huge amount of motion artifacts and the model needs to be able to handle all these motion artifacts and also so the long-term adaptability.
• 10:43
  Steve Yu: The patient is not.
• 10:49
  Steve Yu: They will change their daily behavior, like if they start drinking coffee in the morning, and this will also lead to the changing of the bell signals of the patient.
• 10:58
  Steve Yu: Therefore, the model needs to be able to handle all these different conditions.
• 11:05
  Steve Yu: Regarding to the personalization, the major solution for the personalization is fine tuning.
• 11:10
  Steve Yu: Fine tuning of the model is based on basically use the patient's own data to train the model to fine tune the model.
• 11:19
  Steve Yu: However, this method is kind of limited on the aged device.
• 11:24
  Steve Yu: It is because on the aged device, it is because on the aged device the memory is constrained.
• 11:31
  Steve Yu: Therefore, the memory, therefore the complexity and the size of the model is very limited.
• 11:39
  Steve Yu: For our current topic here, which was running a model on one Azure device, model size is also critical to consider.
• 11:55
  Steve Yu: For most of Azure devices they have very limited memory and if a large model will lead to the memory overflow and operational crashes and this will basically lead to the whole failure of the system.
• 12:07
  Steve Yu: Therefore, it is necessary to create a compact, efficient models and to find a balance between the model size and accuracy.
• 12:21
  Steve Yu: Model performance is the critical challenge of the machine learning.
• 12:24
  Steve Yu: So this is important because model performance is basically the major reason why we're using this model.
• 12:32
  Steve Yu: It needs to be very accurate.
• 12:35
  Steve Yu: A hierarchy needs to be very accurate, otherwise the result, the outcomes, will not be trustworthy and not reliable and then it will be meaningless.
• 12:46
  Steve Yu: Regarding to the latency, latency is something also very important for wearable device Cause, like it can produce, a low latency can produce a very low response time.
• 12:58
  Steve Yu: This means, like for the applications like seizure detection, patient may hurt them, may hurt themselves when a seizure occurred, so they need to have an instant response when the device detect a seizure, but if the latency is too long, then this has no meaning for them to like detect a seizure and to prevent them from being damaged by themselves.
• 13:21
  Steve Yu: And also, like the resources need to be considered well because still we are operating on aged devices and this means the memory, the power consumption of the device is limited and therefore we need to try to still find the balance between the performance and the resource.
• 13:44
  Steve Yu: Regarding to the whole consumption of the model, some devices here we used, they are not only operating for, like, several hours.
• 13:59
  Steve Yu: They may need to be operating for days, weeks or even years and, seeing all the true extent of battery life or or reduce the frequency of recharging, we need to design a model which is crucial to achieve the optimal battery efficiency.
• 14:14
  Steve Yu: This includes developing an energy efficiency model for data processing, interference and communication to minimize the computation load and the power usage, and communication to minimize the computation load and the power usage.
• 14:27
  Steve Yu: Additionally, choosing the low power consumption components in the hardware domain is essential to reduce power while ensuring optimal performance levels, like the different chips or different pre-processing components, like the different upper amps.
• 14:40
  Steve Yu: Then we pass the presentation to my co-worker, jim.
• 14:51
  Jim Wang: Oh yeah, okay, I'm just wondering up to this point.
• 14:54
  Jim Wang: If there's any questions, I'm happy to take.
• 15:01
  Davis Sawyer: Yeah, so there's a couple of questions here, jim or Steve, whoever wants to answer it?
• 15:04
  Davis Sawyer: First question is from Farid what type of biosensors besides ECG and EKG applications are being used, considered for tiny amount of medical monitoring Use cases, and who makes these use cases?
• 15:14
  Davis Sawyer: That's the first question.
• 15:17
  Jim Wang: Okay.
• 15:18
  Jim Wang: So I think for what the research we are currently doing, we are actually using ECG and EEG most of the time doing.
• 15:30
  Jim Wang: We are actually using ECG and EEG most of the time.
• 15:32
  Jim Wang: I'm aware that there has been other kind of biosignals that have been tested on, for instance, the PPGs and I think there's like muscle movements as well.
• 15:44
  Jim Wang: Even so, there has been some growing applications of biosignals in these kind of areas.
• 15:54
  Davis Sawyer: Cool, and there's a follow-up to that who makes these use cases?
• 15:58
  Davis Sawyer: Is it of the manufacturers like, I guess, medical device manufacturers and the models?
• 16:03
  Davis Sawyer: And there's a subsequent question to that, which is is there a database for these sensors?
• 16:07
  Davis Sawyer: So I think they're wondering who's providing the relevant, you know, materials for machine learning?
• 16:11
  Davis Sawyer: Uh, for these uses.
• 16:13
  Jim Wang: So so for some of the uh applications for, for instance, the, the um, eg and ecg applications that we deal with on daily basis and there are some public data sets for certain applications, for instance, for ECG data sets, there's abnormality data sets which are public so you can actually use them to train a model.
• 16:39
  Jim Wang: And I think for EEG there's a seizure data set that's publicly online.
• 16:45
  Jim Wang: And apart from that, we also work with you know, hospitals and on using some of the private data sets to actually to see, to check if the, the model that we have viewed, is actually useful in real cases.
• 17:08
  Davis Sawyer: Right Nice, so that's that's super helpful, and this is also it's kind of in the same vein useful in real cases.
• 17:12
  Davis Sawyer: Right Nice, so that's super helpful, and this is also it's kind of in the same vein.
• 17:14
  Davis Sawyer: This is a different question from Ladi Saleh.
• 17:17
  Davis Sawyer: The question is is there specific international standards for certifying the use of ML algorithms on medical devices?
• 17:22
  Jim Wang: In terms of standards.
• 17:23
  Jim Wang: I'm not quite sure what kind of standard it's referring to, but I mean, if you're talking about ethics, I think there's certain international standards when you think of the FDA or the guidance we have at a national level.
• 17:37
  Davis Sawyer: So I think the question says international, which maybe there's not as much there, but I mean in some cases maybe EMA.
• 17:41
  Davis Sawyer: I know the FDA has made some progress in how they want to certify software and ML devices and you'd find guidance there, there perhaps, but not from the ML community as far as I know.
• 17:49
  Davis Sawyer: If there's anything that's come from machine learning standards.
• 17:55
  Jim Wang: Yeah, me too, I think.
• 17:58
  Davis Sawyer: Yeah, okay.
• 17:59
  Davis Sawyer: Well, that's also something that a professor involved in the field, vijay Reddy, is very excited and adamant about.
• 18:04
  Davis Sawyer: That might be interesting, that question.
• 18:06
  Davis Sawyer: We'll keep it moving.
• 18:07
  Davis Sawyer: There is also more question that you might want to address.
• 18:13
  Davis Sawyer: It's uh, how do you convert deep learning models into c, c++ code or be implemented on the edge devices?
• 18:15
  Davis Sawyer: So do you use on an x or xxt tool to convert tf light to c?
• 18:19
  Davis Sawyer: And then, what type of models have you tried this for?
• 18:21
  Davis Sawyer: So the question is about converting.
• 18:23
  Davis Sawyer: How do you deploy these models on the edge devices?
• 18:25
  Jim Wang: so we have.
• 18:25
  Jim Wang: We have tried um converting, which I will the edge devices.
• 18:27
  Jim Wang: So we have tried converting, which I will be going through later.
• 18:30
  Jim Wang: We have been trying to convert some of the models to be implemented on microcontrollers, which requires to be in C++ formats.
• 18:43
  Jim Wang: I think for TensorFlow they have actually a package in C++ formats.
• 18:49
  Jim Wang: I think for TensorFlow they have actually like a package which you can use to actually transfer or transform some of the models that you have trained to the format that is compatible to microcontrollers.
• 19:01
  Jim Wang: And I think I will be going through some other details regarding because for microcontrollers it's actually just for inference, usually just for inference and for medical applications we have talked about.
• 19:18
  Jim Wang: There's one challenge that we have talked about is called personalization, which means you have to actually fine tune the model to actually better fit the patient's needs.
• 19:28
  Jim Wang: That we have, you know, for that purposes we have tried not converting that but using like a slightly more powerful edge devices that is actually powerful enough to actually fine-tune a small model with reasonable time on that.
• 19:47
  Jim Wang: So I will be going through that in the later part of the presentation.
• 19:53
  Davis Sawyer: Okay, let's save that for then.
• 19:53
  Davis Sawyer: There is one thing I wanted to clarify because I think I misunderstood the question earlier.
• 19:57
  Davis Sawyer: That was about is there a database for these sensors?
• 20:00
  Davis Sawyer: I mistook that as whether there is a dataset around these sensors.
• 20:02
  Davis Sawyer: The question is is there are database on these devices?
• 20:05
  Davis Sawyer: So some of the devices you're talking about earlier, um from the question on the ecg adpj, the ekg um, is there a data set, a database of devices?
• 20:16
  Jim Wang: database of devices.
• 20:17
  Jim Wang: Um, I don't think there's a specific data data set for devices, but I I know um there has been some, you know, manufacturers or industrial players that have been making or manufacturing certain devices for certain applications, but to my understanding, I haven't found any datasets for devices.
• 20:41
  Davis Sawyer: Okay, cool.
• 20:42
  Davis Sawyer: Well, thank you everyone for all these questions that will be dropped in the chat.
• 20:45
  Davis Sawyer: Jim, do you want to keep moving, because I think some of the stuff will be addressed in your discussion and we can jump back in as you like.
• 20:52
  Jim Wang: Yeah, I think, yeah, I'll be going, I'll be going forward with the presentation, but if you have any questions, just dropping in the chat box and I'll be happy to take it along the way.
• 21:01
  Jim Wang: Perfect, so I think I just taking on from my colleague has just presented.
• 21:05
  Jim Wang: So I think we have presented.
• 21:08
  Jim Wang: You know I talk about the challenges that we have faced in the research field.
• 21:17
  Jim Wang: So the strategies that we have, you know, used to actually going forward is to actually, you know, try to.
• 21:25
  Jim Wang: For my personal research, I've used ECG data as the primary data set that we use to actually test, um, you know, certain models and also the compatibility of the models for, you know, uh, potential on device purposes, applications and, um, you know, um, and we have also, uh used to, um, done certain research and studies to actually, uh, try to improve the generalizability of the models and in order to achieve some of the goals that we have or the challenges that we have faced, we have moved away from, you know, using really deep neural networks to actually shallower networks so that it has less parameters, so that's model size smaller and it might be, it can be more, it requires less computational resources and is more applicable on devices.
• 22:20
  Jim Wang: And also we have studied certain applications where the model can actually use less energy to run on devices.
• 22:31
  Jim Wang: And we have done all this to actually in our research.
• 22:37
  Jim Wang: This is kind of like the strategies and goals that we're trying to achieve from our research and I think I'll be going through some of the researches we have done so far to actually give the audience this idea that we have what we are trying to achieve.
• 22:59
  Jim Wang: So we have done a very small but interesting study on improving the generalizability of the models.
• 23:09
  Jim Wang: So I think my colleague Steve has mentioned about the idea of generalizability is sometimes when you train the model on a data set and then when you test the model on another data set which is not within the same data set, and you always see quite a big drop in performance, and this is some of the areas that has been caught our attention.
• 23:40
  Jim Wang: So what we have done is the idea is how can we actually improve the generalizability of the model with the same data set with minimum efforts?
• 23:55
  Jim Wang: And we would like to see you know if there's any improvements on the generalizability of the model.
• 24:02
  Jim Wang: So for that purpose we have taken this.
• 24:05
  Jim Wang: So for that purpose we have taken this model and this is the model that has been published in one of the papers in 2020.
• 24:15
  Jim Wang: I think it claims that it has been trained on the world's largest 12-foot ECG data set and also the model is fairly large compared to the model that we have used in the later part of the research.
• 24:30
  Jim Wang: But we have used this data to this model to actually test the generalizability.
• 24:36
  Jim Wang: You know if we can improve that.
• 24:47
  Jim Wang: So how we did it is very, quite simple.
• 24:49
  Jim Wang: Actually, the idea is we focused on the characteristics of the data sets, so we actually, instead of just using the whole data sets this whole data sets actually comprises of about 2 million individual data status and we actually made a subset from this large data set and with different criteria.
• 25:09
  Jim Wang: So you can actually see here that how we actually selected the data.
• 25:15
  Jim Wang: So, basically, there are a number of abnormalities in the data and one way we have tried to do is try to get a very balanced data set.
• 25:27
  Jim Wang: We have tried to do is try to get a very balanced data set.
• 25:31
  Jim Wang: So you can see that for every abnormalities here it's all.
• 25:34
  Jim Wang: For example, the first data sets, it's much more balanced compared to the rest, and we have also tried to actually select the data based on other criteria, say, I think that the second data set we tested was trying to pick on patients who has more than one abnormality in their ECG, and the third one, for example, we actually focused on older and younger patients instead of just basically randomly picking the patients.
• 26:13
  Jim Wang: So from the test I think this next slide I actually have so I think from the test we have done, we actually have observed something very interesting that you know we actually, with the data set we have, we have, you know, you know, selected, compared to the large data sets, this data set is only 1% of the large data set.
• 26:43
  Jim Wang: But in performance I'm talking about generalization performance, not just in sample performance the model actually performs slightly better than the model that has been trained on the, you know, on 2 million data sets.
• 27:00
  Jim Wang: So it's quite surprising that you know not, you know, when focusing on the balance of the data or characteristic of the data, we can actually improve the performance of the model in generalization.
• 27:16
  Jim Wang: So I think that's the first kind of small work that we have done to address some of the challenges that we have talked about earlier and before I move on to the next work, if there's any question I'm happy to take regarding this.
• 27:33
  Davis Sawyer: Yes.
• 27:33
  Davis Sawyer: So just pulling up here, there are two questions.
• 27:36
  Davis Sawyer: We have the first one.
• 27:37
  Davis Sawyer: I think it relates to something Steve presented as well.
• 27:41
  Davis Sawyer: It says you mentioned something along the lines of reducing latency for timely disease detection.
• 27:46
  Davis Sawyer: Could you elaborate on that further?
• 27:49
  Jim Wang: the lines of reducing latency for timely disease detection.
• 27:50
  Jim Wang: Could you elaborate on that further?
• 27:51
  Jim Wang: Uh, line time disease detection, okay.
• 27:53
  Jim Wang: So, for instance, I think what steve was referring to is, for instance, that um, um, for seizure detections, uh, we, we need to process the EEG data on device on the spot and you will give the detection to the patient that says, oh, your seizure is hitting you and you have to take the medicine or the precautions.
• 28:20
  Jim Wang: So the latency is how long does the model take to actually figure it out?
• 28:28
  Davis Sawyer: So it's quite important in addressing certain illness, for example, that the seizure I have talked about is one of them.
• 28:41
  Davis Sawyer: Nice, that's very cool.
• 28:41
  Davis Sawyer: That's definitely a clear benefit.
• 28:43
  Davis Sawyer: The other question is I am just curious to know the amount of RAM and ROM size used by the project that you are currently flashing on your medical devices.
• 28:54
  Jim Wang: Okay, I think I'm going to present a few of the works.
• 29:00
  Jim Wang: So I think on one of them, for inferences, we actually run it on a STM32 microcontroller and I think we, the RAM I think we use is about 94% if I'm not wrong.
• 29:18
  Jim Wang: I have it in our slides, in the later slides, but yeah, I think that's the number on my head, nice.
• 29:25
  Davis Sawyer: Okay, and one more question here is asking you to elaborate on the rules and tools that you use to reduce the size of the training data set, to make it more vital.
• 29:35
  Jim Wang: The rules and tools.
• 29:36
  Jim Wang: So for this work, actually, the rules and tools we use is basically on data selection, is basically on data selection.
• 29:48
  Jim Wang: So, for instance, I have elaborated that for one data set that we have created, it's just, to you know, trying to get equal amount of every abnormality in the data set.
• 30:00
  Jim Wang: And, for instance, for another data set we have used, we try to select the people, the patients with older age or younger age, instead of just focusing on, you know, the middle ages.
• 30:18
  Jim Wang: So these are the selection criterias.
• 30:20
  Jim Wang: So, and with these selection criterias, we have actually created a number of data sets and we have trained them and we have compared them and we have drawn a conclusion from there.
• 30:36
  Davis Sawyer: Awesome, that's really interesting.
• 30:37
  Davis Sawyer: One more quick question, jim.
• 30:38
  Davis Sawyer: For you it's actually from YouTube.
• 30:40
  Davis Sawyer: The question is which controller I think the microcontroller you are using.
• 30:45
  Jim Wang: For the work that we have done is STM32 microcontroller.
• 30:49
  Jim Wang: I think you mentioned that For that.
• 30:51
  Jim Wang: Yeah, stm32, got it.
• 30:53
  Jim Wang: Thank you, ok.
• 30:54
  Jim Wang: That's all the questions for right now.
• 30:55
  Jim Wang: Ok, so I'll move on to the next part.
• 31:04
  Jim Wang: Ok, so here is some tests that we have done for certain models that we have used For this particular work.
• 31:14
  Jim Wang: We did not test it on any hardware devices because we were just trying to figure out if this is the right model that we should be using for some of the applications that we are trying to address.
• 31:31
  Jim Wang: So for this model is the diagonal state-space sequence model and I think if you are following the machine learning community closely, you probably have heard about the MUM model which has been released earlier, early this year.
• 31:49
  Jim Wang: So this is from the same kind of work.
• 31:53
  Jim Wang: I think this is like more like a simpler version of Mamba and it shares the same foundation with that model.
• 31:59
  Jim Wang: So they claim that this model is very efficient in terms of, you know, parameters and computation and it also has some certain advantages in terms of model size, and so for this particular work, we actually have tested this on ECG data, and also this model works well with time sequence data, which is the form which ECG data is in the form of that.
• 32:32
  Jim Wang: So we have actually tested this model and we actually used, compared, using multiple layers of the S-ball 40 layers to actually see if this model is, you know, is working, if this model is working.
• 32:54
  Jim Wang: So, drawing from the work that we have done on this model, we actually found something quite interesting regarding this model.
• 32:57
  Jim Wang: So I think my colleague Steve has actually talked about, you know, noise.
• 33:05
  Jim Wang: The noise can be quite destructive in biosignals and this model has shown some resistance to the environmental noises.
• 33:15
  Jim Wang: So it's quite an interesting phenomenon that we have observed from this model and also the generalizability of the model is quite strong.
• 33:26
  Jim Wang: So we have actually in this work, because we have the conclusion that we have drawn from the previous work, that training on a more balanced dataset actually improves the generalization capability of the model.
• 33:41
  Jim Wang: So we actually used a very balanced dataset to actually train the model and test it on another dataset which is not the same one that we have trained it on and we have seen, you know, relatively quite strong holding out of the performance when you test on another data set.
• 34:00
  Jim Wang: And also another very interesting, you know, phenomenon that we have observed from this this model is that, um, you know, usually we use 12 lit ecgs to make you know predictions or detections of the abnormalities that, um, you know happening in your heart.
• 34:21
  Jim Wang: For this model we have.
• 34:23
  Jim Wang: We actually, you just used one channels instead of the 12 channels provided and we actually have tried to see if this model is still holding up and, very surprisingly, the performance didn't drop by too much.
• 34:36
  Jim Wang: So it's actually holding up quite strong with just one lead of data.
• 34:44
  Jim Wang: So we are saving 11 leads of data and this will save a lot of resources on computation, and that's why we think this could be a good candidate for, you know, wearable devices.
• 34:59
  Jim Wang: Okay, I think that's the summary of this work.
• 35:04
  Jim Wang: So if you have any questions regarding this before I move on to the next one, are you happy to drop in the chat box?
• 35:12
  Davis Sawyer: So yeah, first one, I think this is a great question actually.
• 35:14
  Davis Sawyer: Do you think this model code will be made available?
• 35:18
  Jim Wang: We have published this work and the code access has been put on the work, so I think you can access that through GitHub.
• 35:26
  Jim Wang: The link will be provided on the page, so I think you can access that through GitHub.
• 35:29
  Davis Sawyer: That link will be provided on the page.
• 35:30
  Davis Sawyer: Yeah, perfect, yeah, that's what I was gonna say.
• 35:31
  Davis Sawyer: If you have a link or something, you can actually even type it in the answer, and that way everyone can see it.
• 35:36
  Jim Wang: Yeah, yeah, you can share it.
• 35:37
  Jim Wang: I just realized actually, I can see that question.
• 35:41
  Davis Sawyer: Yeah, yeah, absolutely, so I think that those could definitely be shared.
• 35:44
  Davis Sawyer: This is another good question, um, under what scenarios do the models on the device need?
• 35:54
  Jim Wang: to be updated.
• 35:54
  Jim Wang: Uh, I'm, I don't quite get what the so what do you mean?
• 35:58
  Davis Sawyer: updates, um, maybe the weights, like do some fine tuning or or update, retrain the model, essentially, I think, or maybe change the model more deeply.
• 36:07
  Jim Wang: Okay, so for some instances, like we have talked about personalization, and this is actually linked to generalization as well.
• 36:16
  Jim Wang: So when you train a model on a public data set or on a large data set, there might be a chance that you know when this model is applied on this person.
• 36:27
  Jim Wang: It does not work as what it should be because of certain features you have or certain the environment might be different.
• 36:45
  Jim Wang: So in that case you might need to fine tune the model with your own data to actually make the predictions more accurate.
• 36:50
  Jim Wang: If that answers the question, yeah.
• 36:53
  Davis Sawyer: Right.
• 36:53
  Davis Sawyer: And even once you've done that like, say, you've taken a model and you've used your own data, your own annotations to now train it and you've deployed it to the devices maybe after a month or two months you notice that the accuracy starts to degrade or you have a new type of data, type of drift in the yeah, inputs yeah, that's that could happen.
• 37:12
  Jim Wang: Yeah, that could happen.
• 37:13
  Jim Wang: So, for instance, the user habit changes and that could.
• 37:18
  Jim Wang: That could result in your you know, the changing your signal and also, of course, the uh you might need to find in that model as well.
• 37:25
  Jim Wang: In that case, right, OK, next one.
• 37:33
  Davis Sawyer: Yeah, so there's one more question that's probably going to be addressed if you share the link or the code, but the question is do you have some precise figures to share when you're talking about the performance?
• 37:44
  Jim Wang: of your model.
• 37:44
  Jim Wang: I don't have it on the slides, but I'm pretty sure it's staying in the paper, in the published paper.
• 37:51
  Jim Wang: So maybe, steve, can you help me to actually put a link of the paper there.
• 37:56
  Davis Sawyer: So, yeah, you can actually check over there.
• 37:58
  Davis Sawyer: Awesome.
• 37:59
  Davis Sawyer: Well, thanks guys.
• 38:00
  Davis Sawyer: Yeah, that's super helpful.
• 38:01
  Davis Sawyer: I think that's a really interesting part of what's been new in what you presented here is the model contribution, so definitely a lot of interest in that.
• 38:08
  Jim Wang: OK, that's good.
• 38:09
  Jim Wang: So I think that's all the questions for now.
• 38:11
  Jim Wang: Yeah, maybe I'll just move on to the next part.
• 38:14
  Jim Wang: Ok, so this is another work that we have done so far.
• 38:17
  Jim Wang: So we have tested another interesting model.
• 38:21
  Jim Wang: It's called neural circuit policy and this is a very interesting network that they have presented, that you know.
• 38:31
  Jim Wang: It's bio-inspired and they claim that these neurons are much more efficient than the MLPs that we use in our daily machine algorithms.
• 38:47
  Jim Wang: So this model in the paper where they published this work, they claim that they are actually able to drive a car with just 19 neurons.
• 39:02
  Jim Wang: And we were interested to know if this network can actually be applied to biosignals, where we will be able to use less neurons, which means less parameters, and to make the accurate predictions.
• 39:24
  Jim Wang: So we actually tested this, and with a very, very simple model.
• 39:31
  Jim Wang: So, as you can see, we processed the raw ECG data with short-term Fourier transformation, which is a typical thing in biosignal area that you use to actually transform the data from time domain to frequency domain, and then we used just one layer of COM, lstm layer to actually try to extract the features from the the accepted ECG data, and then we actually put those features into a very simple neural network, I think it was.
• 40:21
  Jim Wang: I think there was just 20 neurons in there, 20 NCP neurons in there, and we were able to get quite an accurate predictions for the results.
• 40:33
  Jim Wang: So, as I mentioned also, and also we have actually deployed this model on our STM32 microcontroller to actually see if this model is actually compatible for on-device inferences.
• 40:57
  Jim Wang: So we actually and, as I mentioned, we always test for generalization of the model.
• 41:03
  Jim Wang: So we have actually trained this model on one dataset and tested on another dataset.
• 41:07
  Jim Wang: So, and there are two ways you can actually group the NCP neurons and that's the two models that we have tried.
• 41:20
  Jim Wang: It's called CLTC and CCFC and then we have also compared these two models and they behave very similar but also slightly different.
• 41:32
  Jim Wang: But we found that, you know, the CCFC model is actually much slightly quicker or slightly faster than the other model.
• 41:42
  Jim Wang: So it's actually more, it's actually better to, for you know, you've got to reduce the latency to actually get a quicker predictions from the model, and so I think that's kind of the summary that this about this work and this work is also published and I think the code is in the, in the, in the um, in the paper as well.
• 42:05
  Jim Wang: So if you're interested, you can have a look, and if you have any question regarding this book, um yeah, feel free to drop it now yeah, jim, thanks for both those links, I think, because we've also shared one with everyone in the chat as well.
• 42:17
  Davis Sawyer: For the previous paper.
• 42:18
  Davis Sawyer: There's one question um related to what you just presented.
• 42:21
  Davis Sawyer: Do you think that this model could generalize well to other types of biosensors, ie biomarkers?
• 42:29
  Jim Wang: This is a good question, because I've never thought about this.
• 42:35
  Jim Wang: But I think these neurons can be very useful in many areas, because we have tested on multiple types of biosignals and I have tested on EEG and we also have tested on EEG and the performance has been always holding up.
• 42:58
  Jim Wang: So my feeling about this is it could be useful for other biomarkers, but I haven't tested it, so I don't have the definite answer for that.
• 43:11
  Davis Sawyer: That's exciting.
• 43:12
  Davis Sawyer: Is this something you think people could replicate themselves if they had interest?
• 43:16
  Davis Sawyer: To try your model out essentially on their own sensors or biomarkers and try to see if they can answer that question?
• 43:23
  Davis Sawyer: Yes yes, love you.
• 43:26
  Davis Sawyer: Yeah, so I think that was yeah.
• 43:28
  Jim Wang: Your colleagues actually asked you the next question, so I think yeah, I think Steve will be putting that on in the chat.
• 43:38
  Jim Wang: So, yeah, awesome, okay, very cool, okay.
• 43:40
  Jim Wang: So I putting that on in the chat.
• 43:41
  Jim Wang: So, yeah, awesome, okay, very cool, okay.
• 43:43
  Jim Wang: So I should move on to the next one.
• 43:45
  Jim Wang: Okay, so this is another area that we have tried on.
• 43:52
  Jim Wang: So I think in the past, in the earlier part of the presentation, my colleague Steve has talked about the power consumptions of the models.
• 44:02
  Jim Wang: So in certain cases, as he mentioned, we require the devices to be lasting a lot longer than just a couple of hours.
• 44:15
  Jim Wang: So there's an interesting neural network called the spiking neural network, and I think it's also called neuromorphic in certain areas.
• 44:27
  Jim Wang: So what the claim about this kind of neural network is that it's very power efficient because it works differently from the traditional neural networks.
• 44:42
  Jim Wang: It works on, you know, event-based processing, so it detects the changes rather than the whole image In that example, if you understand.
• 44:55
  Jim Wang: So the power consumption of this kind of neural network is a lot lower because you only focus on, on, on the, the changes, and the, the network generate, generate spikes, instead of just transmitting between layers that, the, the weights, so when there's a, so the spikes are very energy efficient because that's what they believe that how human brain works, because your brain doesn't consume that much of energy when you try to learn or looking at some images, compared to the current machine learning that is consuming a tremendous amount of energy to train on GPUs.
• 45:45
  Jim Wang: So this is a very interesting network that we have looked at.
• 45:55
  Jim Wang: Also, in continuation from the previous work we have discussed, that we have actually merged the previous, the NCP neurons, the model which is a spiking neural network, and you can actually see from the charts.
• 46:16
  Jim Wang: So instead of you know, we have used the same processing of the ECG and you can actually see the GIF, you can see the spikes and those are actually you know when, when we made this ECG data neuromorphic.
• 46:35
  Jim Wang: So make a spike, make it event based, and then we have input this spike, make it event-based, and then we have input this network or this data to spiking ConvLSTM layer and before it fit into the neural network, the same NCP network that we have used in our previous study.
• 47:10
  Jim Wang: And the key takeaway from the study that we have found is that there is a claim that when you use spiking neural network, there is usually a big drop in performance.
• 47:19
  Jim Wang: But in this work, where we combine the spiking neural network and the NCP network, we actually did not observe a huge drop in performance.
• 47:32
  Jim Wang: Instead, actually, the performance holds up quite closely to the original model and because it's now neuromorphic or it is using spiking neural network, it's actually saving a lot of energy, as we have presented here.
• 47:52
  Jim Wang: So it's actually almost 100 times less energy usage compared to the traditional neural network that we have tested on and also to actually-.
• 48:07
  Davis Sawyer: Sorry, jim, I think there were some questions about the spiking, the neuromorphic approach, actually, because this is a super interesting topic itself.
• 48:14
  Davis Sawyer: So I think the question is can you elaborate on the biomimetic approach, which I think is referring to the neuromorphic or spiking event-based types of models you're using?
• 48:23
  Davis Sawyer: Are there other models based on this kind of approach and is it specifically interesting for tiny systems and resource-constrained devices, which I think it definitely is?
• 48:31
  Davis Sawyer: But if you want to just elaborate more on the kind of inspirations for these approaches, Sorry, I'm just actually reading the second question.
• 48:39
  Davis Sawyer: Yes, exactly, this is a question for Mark.
• 48:49
  Jim Wang: In general.
• 48:50
  Jim Wang: Are there other models based on this kind of approach?
• 48:55
  Jim Wang: There has been a growing attention in this field because of the energy problem that I have discussed earlier.
• 49:06
  Jim Wang: So what I have been focused on is on the network, which is the software of the neuromorphic, and there has been a growing interest in actually designing neuromorphic chips.
• 49:22
  Jim Wang: There has been some commercial product actually released, but at the moment I think neuromorphic chips and neuromorphic models are still at the earlier stage.
• 49:36
  Jim Wang: So there's still a lot of research and developments going on in this field and I have seen there are new type of neuromorphic chips that has been released in the recent years and there are some new ways actually to train the neuromorphic models as well has been released in the past one or two years.
• 49:57
  Jim Wang: So it's still early this model, but there has been a growing attention and application of this kind of a model, at least in medical domain.
• 50:08
  Davis Sawyer: Yes, Nice Well, thanks for that insight, definitely beyond even the biomedical engineering.
• 50:14
  Davis Sawyer: That topic of biomorphic has been interesting in tiny mill for some time and across a lot of domains, so definitely huge interesting inside email for some time and across a lot of domains.
• 50:25
  Jim Wang: So definitely definitely huge.
• 50:25
  Jim Wang: Okay, I think I addressed all the questions.
• 50:31
  Jim Wang: So, um, yeah, I haven't finished on that work.
• 50:33
  Jim Wang: So this is just the um, you know, finishing up slides for that.
• 50:37
  Jim Wang: Um.
• 50:37
  Jim Wang: So we actually um try to deploy this model on a microprocessor and to actually to see if on-device fine-tuning can happen, because at the moment for neuromorphic models or for spiking neural networks, it's harder to train on a normal GPU or CPU.
• 51:01
  Jim Wang: But just to demonstrate our proof of concept, we actually deploy this model on a microprocessor and use it to actually train or fine tune on the edge to actually see if this can be done.
• 51:19
  Jim Wang: And we have also tested this model on how it handles with missing channels.
• 51:32
  Jim Wang: This is because when you are, as I mentioned, for ECG data, there are 12 leads and sometimes there might be one or two leads.
• 51:42
  Jim Wang: That is not in good contact with your body and the signals gets rough or it's just missing.
• 51:49
  Jim Wang: So it's just like a habit that we have that we always test these models on missing channels to actually see how these models behave when there's a missing data and is there a huge impact on the predictions of the model and, compared to the non-spiking version of the model, the spiking ones essentially handles the missing channels better.
• 52:20
  Jim Wang: This may be because the neuromorphic models works on event-based data instead of the full data, so it naturally handles some of the missing data better than traditional models.
• 52:42
  Jim Wang: And also we have tested on one single lead ECG data to see if that can be done with just one lead ECG and to see if this model can be applicable for tiny devices, because usually for tiny devices you don't collect 12 lead ECGs, you just collect just one lead.
• 53:03
  Jim Wang: Yeah, I think that's all for this work, so I think there's more questions.
• 53:10
  Jim Wang: Should I take that?
• 53:18
  Davis Sawyer: Yeah, so the question that you could address here is how do you envision the on-device?
• 53:24
  Davis Sawyer: Supervised training model to work in a real-life application and where is the label data coming from in medical applications?
• 53:33
  Jim Wang: Yeah, I think the labeling will need to be done.
• 53:38
  Jim Wang: So we haven't figured out that part yet.
• 53:41
  Jim Wang: So how you actually do the labeling?
• 53:43
  Jim Wang: Because you actually need a doctor or a practitioner to actually label data.
• 53:53
  Jim Wang: But for some applications the patient might be able to label the data themselves and use that to actually train on device.
• 54:01
  Jim Wang: And this is a very tricky area where there has been some works in fine-tuning that you send the data outside the device for training or for fine-tuning and then you redeploy the models.
• 54:26
  Jim Wang: But there's a growing concern about privacy issues in that approach because if you send the data outside the data, it's very easy for anyone with a bad intention to actually hack into the data, to hack into the server to get your personal data.
• 54:48
  Jim Wang: So it has been a lot of work in this area as well recently to actually address this issue.
• 54:56
  Davis Sawyer: Got it and I think there are a few people who are definitely interested to collaborate with you and Steve.
• 55:01
  Davis Sawyer: So I don't know if you have it at the end of your presentation, but your email contact or however, to get in touch with you Definitely a few people interested here, so we'll save that for when you want to share with the other part of the.
• 55:18
  Jim Wang: So I think there's one more work that I would like to just briefly present.
• 55:24
  Jim Wang: So this is just another study that we have done.
• 55:29
  Jim Wang: So we have actually introduced the application of the S4D model and the NCP model that we had tested on.
• 55:38
  Jim Wang: So we just had a very interesting idea on both model have shown compactness or like high efficiencies in model size, in parameters, when dealing with this kind of at least in biosignal data.
• 55:56
  Jim Wang: So we had an interesting idea on what if we combine the two models.
• 56:02
  Jim Wang: So we actually tested, you know, this model.
• 56:06
  Jim Wang: We actually merged these two models.
• 56:09
  Jim Wang: We only used one layer of s4d layer and then we have merged it with an NCP model or NCP network with just 20 neurons, and we actually tested on broad ECG data means without any pre-processing, and we actually achieved a model that's so small it's just like 242 kilobytes in terms of architecture, in terms of the model size, and also we have also, as we are moving towards trying to see if this personalization can be happening on device, we have also tested this model to actually for on-device fine-tuning, to actually see if that can be happen, if that model is trainable on a relatively small edge devices, and I think it was possible.
• 57:07
  Jim Wang: And also, we have actually tried to find the limitation of this model.
• 57:14
  Jim Wang: And how we did it is that we actually tried to push the limit of the model by further reducing the neurons and the architecture within the S4D layers, and we actually have achieved a model that is just 25 kilobytes and it's still holding up the performance quite well.
• 57:44
  Jim Wang: And also, we have also tried to test, you know, if instead of using a long ECG data, we just use a very short ECG data, is the model able to make very accurate predictions?
• 58:03
  Jim Wang: And we actually have tried just using 2.5, I think it was 2.5 seconds of ECG data and the performance was still holding up nicely.
• 58:13
  Jim Wang: So we think this model is quite interesting because not only it tackles the problem of reducing the number of parameters and also it reduces the problem of latency, because if you are only using two seconds of data, the latency for on-device inference will be much shorter.
• 58:39
  Jim Wang: On-device inference will be much shorter.
• 58:41
  Jim Wang: So I think that's the summary of this work.
• 58:44
  Jim Wang: So if you have any question, happy to address that now.
• 58:52
  Davis Sawyer: Ben, we'll give just a minute to yourself for anyone who has some last minute questions here before we wrap up.
• 58:56
  Davis Sawyer: But thank you very much for the detailed and dynamic presentation.
• 59:00
  Davis Sawyer: I guess, quickly to wrap up with this, final thoughts what's next for your research direction?
• 59:04
  Davis Sawyer: What?
• 59:04
  Davis Sawyer: What are you as the working on next in alignment with this work?
• 59:08
  Jim Wang: oh okay.
• 59:08
  Jim Wang: So, um, actually, I have, uh, actually just one one slides on, you know just just future future works so um.
• 59:16
  Jim Wang: So I think, uh, after actually we have tested on several, you know, promising networks that is small and efficient, we are still trying to actually improve the performance of the model and we are also very interested in.
• 59:40
  Jim Wang: I think one of the questions earlier was regarding how on-device fine-tuning can actually happen.
• 59:52
  Jim Wang: That is kind of the area that we are also interested in at the moment.
• 59:54
  Jim Wang: So we actually and I mentioned to you there's a conflict, there's a balance that we need to address where, you know, uh, sending anything outside the device and keep everything on the edge because of the privacy issues.
• 1:00:07
  Jim Wang: So we are also trying to address that with some new uh methodologies that we have recently, uh have done some work.
• 1:00:15
  Jim Wang: So I think, um, there's some incoming.
• 1:00:18
  Jim Wang: I think in the incoming days we might be releasing that work.
• 1:00:22
  Jim Wang: It's kind of interesting because it's kind of a new methodology that we have proposed to actually maybe address this issue.
• 1:00:29
  Jim Wang: Yeah, Awesome.
• 1:00:32
  Davis Sawyer: Well, that's super cool.
• 1:00:33
  Davis Sawyer: I think we have a bunch of questions here, so we'll get through them in order here.
• 1:00:37
  Davis Sawyer: So first question is how can a person with the heart remember volunteer to obtain this device and wear it and send you lifetime data during rest and during sports activity?
• 1:00:49
  Jim Wang: to obtain the device and where the inside you know rest of.
• 1:00:52
  Jim Wang: Uh, I'm not quite sure about this because for what we have done so far is we use usually use data sets that is either public or we have an agreement with.
• 1:01:10
  Jim Wang: So we actually did not.
• 1:01:14
  Jim Wang: Even if it's personal data, we actually get it from through the agreements that we have with hospitals or any other corporation.
• 1:01:24
  Jim Wang: So we haven't really tested on applying that on a real person and try to do that.
• 1:01:32
  Davis Sawyer: Got it Okay.
• 1:01:33
  Davis Sawyer: Then the next question is for a single lead ECG.
• 1:01:36
  Davis Sawyer: Where did you physically attach the sensor to the scalp?
• 1:01:41
  Jim Wang: ECG?
• 1:01:41
  Jim Wang: Where did you physically attach the sensor to the scalp?
• 1:01:43
  Jim Wang: I think there is certain devices that actually has been released in the market that is just this big and I think you just attach there to actually collect just one lead ECG.
• 1:01:59
  Jim Wang: So I think you can actually design a device and to train the model with the type of data you're going to connect with just one lead ECG and to actually see if the model actually makes good predictions.
• 1:02:19
  Davis Sawyer: Okay, and then another question.
• 1:02:21
  Davis Sawyer: We talked about this a bit, but I think the question relates to are you trying to take learning or fine tuning on device?
• 1:02:30
  Jim Wang: For some of the work.
• 1:02:31
  Jim Wang: We have tried that and I think that is some of that's one of the directions that we actually have.
• 1:02:37
  Jim Wang: So we think that keeping the fine-tuning process on devices beneficial comparing to sending it out to the device in general.
• 1:02:48
  Davis Sawyer: Okay nice, so we do have a couple of minutes here, so we'll go through these three questions we have.
• 1:02:52
  Davis Sawyer: So have you explored federated learning using your models?
• 1:02:57
  Jim Wang: I have not used the federated learning for the models that have.
• 1:03:01
  Jim Wang: But I can see where the question is coming from Because instead of sending your data outside, you can share the gradients and actually fine-tune the models.
• 1:03:12
  Jim Wang: But there's one I'm not sure if you are aware there's a couple of papers that has been released in one or two years.
• 1:03:22
  Jim Wang: There's a growing concern with federated learning because there has been study done that you know, to show that you can actually replicate the training data with the gradient that you share outside the device.
• 1:03:39
  Jim Wang: So that's quite, and with just 20 lines of code, if I'm not wrong.
• 1:03:45
  Davis Sawyer: Yeah, that's a good incentive to use federated learning as one way to update models and achieve the learning and the on-device learning.
• 1:03:53
  Davis Sawyer: So that kind of leads to this other question which we talked about again.
• 1:03:57
  Davis Sawyer: I think a bit, but it's how do you address model drift?
• 1:04:01
  Jim Wang: How do I address model drift?
• 1:04:02
  Jim Wang: How do I address model drift?
• 1:04:03
  Jim Wang: So that part of the question we haven't really thought on but we believe on device training can solve some of the uh drifting problems okay, and I know, I know it's late for you here.
• 1:04:15
  Davis Sawyer: so the last question before we give you a break and we conclude here so also, are you trying to do non-invasive EEG collecting devices?
• 1:04:24
  Jim Wang: Yes, I think in the group that the research group we have, I think, a part of us is actually working on non-invasive and invasive EEG connection devices and yeah, so we have done a bit of working data as well.
• 1:04:43
  Davis Sawyer: Fantastic, fantastic.
• 1:04:45
  Davis Sawyer: Well, on that note, I think we also have our forums as well we can use to keep the conversation going, jim, steve, I'd love to thank you on behalf of TinyML.
• 1:04:55
  Davis Sawyer: If you can share your email address or get involved through the chat, whatever you like, that'd be fantastic.
• 1:04:59
  Davis Sawyer: I think there are a few people interested to collaborate with you guys.
• 1:05:02
  Davis Sawyer: So, on that note, I think it'd be great to thank both of you again for your time for making this happen today.
• 1:05:09
  Davis Sawyer: Thank you to our attendees for being here on our live stream and on Zoom.
• 1:05:13
  Davis Sawyer: Thank you again to our sponsors, to our executive strategic partners at TinyML, thank you to our platinum strategic partners at TinyML.
• 1:05:21
  Davis Sawyer: Thank you To our platinum strategic partners as well EmbedUR to Sony.
• 1:05:31
  Davis Sawyer: Ai Scales.
• 1:05:31
  Davis Sawyer: Deploy Vision, ai Scale, the gold strategic partners.
• 1:05:34
  Davis Sawyer: Arm, build the Future of TinyML.
• 1:05:36
  Davis Sawyer: Edge Impulse, the leading edge-building platform for Edge ML.
• 1:05:40
  Davis Sawyer: Infineon, driving decarbonization and digitalization together.
• 1:05:44
  Davis Sawyer: Neuromorphic intelligence for the sensor edge from Inotera.
• 1:05:49
  Davis Sawyer: Renesas SD provides extensive solutions to make tiny machine learning easy.
• 1:05:56
  Davis Sawyer: Synaptics engineering, exceptional experiences.
• 1:06:00
  Davis Sawyer: And also the Silver strategic partners, which is here.
• 1:06:03
  Davis Sawyer: So thank you everyone for joining us.
• 1:06:05
  Davis Sawyer: That concludes today's time of talk.
• 1:06:07
  Davis Sawyer: We'll see you next week.
• 1:06:08
  Davis Sawyer: Thank you very much.