Latest / EDGE AI POD / Revolutionizing Nano-UAV Technology with Cutting-Edge On-Device Learning Strategies

Transcript

Revolutionizing Nano-UAV Technology with Cutting-Edge On-Device Learning Strategies · EDGE AI POD

• 0:09
  Speaker 1: So it's my pleasure to chair the next session, which is called On-Device Learning, basically, and which deals with on-device learning basically, and we have two talks.
• 0:21
  Speaker 1: The first talk will be given by Elias Reda.
• 0:25
  Speaker 1: It's titled Training on the Fly On-Device Self-Supervised Learning Abort Nano-UAVs Within 20 mW of Power.
• 0:34
  Speaker 1: Elias is a second-year PhD student at the Dalai Lama Institute of Artificial Intelligence in Lugano in Switzerland.
• 0:42
  Speaker 1: He actually received a double MSc degree from Lugano and University of Milano Vigoca, and his research currently focuses on AI-based autonomous algorithms for pocket-sized robotic platforms such as nano-UAVs.
• 1:01
  Speaker 1: So, without any further ado, elia, please.
• 1:06
  Speaker 2: Hi everyone, I'm Elia Chereda.
• 1:08
  Speaker 2: I'm happy to be here to present our work on device self-supervised learning today, which is literally training on the fly, because we do it on nano UAVs.
• 1:17
  Speaker 2: In recent years, we've seen AI spreading to many different fields, and hardware improvements have actually allowed this to move across the whole spectrum of power and payload scales, down to the embedded level devices that we are all here today to talk about and we all care about.
• 1:43
  Speaker 2: Here we have stronger constraints in terms of power consumption, computational resources and memory.
• 1:51
  Speaker 2: What we've seen, however, is that deploying this system in the real world is a significant challenge and, in particular, what this presentation is all about why we care about this problem specifically on nano-UAVs is the challenge of domain shift, which applies across the entire spectrum but becomes more and more challenging as you go down in scale.
• 2:15
  Speaker 2: So what I'm talking about is the problem of training models on a certain domain and then deploying on another one, which, in robotics, is an extreme instance in the task of sim to real transfer, since data in robotics is very expensive to collect and requires sensors, infrastructure and also a lot of human time.
• 2:38
  Speaker 2: Training in simulation gives you the way to automate a lot of the collection process.
• 2:44
  Speaker 2: But this, however, poses a problem when you need to deploy to the real world, because a model that works in training may fail in the real world, and this, in robotics, can have very real consequences, such as crashes or other physical damage.
• 3:00
  Speaker 2: And with TinyML this problem becomes more and more significant because, due to the small size of models, you typically have struggled to make them generalize.
• 3:14
  Speaker 2: And this further challenges the traditional deployment pipeline where you have a single model that you use in all environments, which, on device learning, on the other hand, tries to replace with a setup in which you want to adapt your own models directly on device, so you deploy and then, directly in the environment where you want to use your model, you can collect data and adapt using only the resources.
• 3:44
  Speaker 2: On device addressing domain shift as close as possible to the source, which also has other benefits in terms of, for example, privacy and security.
• 3:53
  Speaker 2: In this fits with our work on nano drones, which we, which are very interesting platform for us because they, thanks to their small size, are especially interesting platform for us because they, thanks to their small size, are especially interesting to use in proximity of people and in cluttered environments.
• 4:12
  Speaker 2: However, their true potential would be enabled only through autonomous navigation, by being able to have them perform tasks autonomously, which hits with many of the problems that I was talking about, such as the computational power on board, which is limited to microcontroller units, since with just a few tens of grams you don't have the battery capacity.
• 4:41
  Speaker 2: You don't have the power to do something more powerful.
• 4:44
  Speaker 2: Also, they are a very interesting setup because they require challenging perception tasks.
• 4:51
  Speaker 2: You have CNNs processing images and also real-time constraints, because ideally you would use this model for closed-loop control of the drone.
• 5:01
  Speaker 2: And finally, the small models and the problem with collecting data turns into a very real domain shift problem.
• 5:09
  Speaker 2: So, from what we've seen in previous research, having models that actually generalize well is a challenge.
• 5:17
  Speaker 2: In this work we address the task in the context of human pose estimation, where a drone needs to estimate the relative pose of a person.
• 5:25
  Speaker 2: So these X, y, z coordinates in position and your orientation from an image of the drone camera, and this is processed with a state-of-the-art CNN which is trained entirely in simulation and fully deployed on device.
• 5:39
  Speaker 2: So this is a model that needs to predict spatial information.
• 5:47
  Speaker 2: So you can imagine that performing on-device learning with this kind of model it's challenging because the outputs that you want to present, that you want to predict, are not really measurable by the drone itself.
• 6:01
  Speaker 2: It has no way of knowing where the person is when it's actually deployed on the field.
• 6:06
  Speaker 2: So there has been a lot of work recently on the embedded level to enable on-device learning in terms of software, infrastructures, of methods and also in terms of robotics.
• 6:23
  Speaker 2: There have been already instances of reinforcement learning deployed on device.
• 6:28
  Speaker 2: However, what we've seen in by looking at the literature is that a lot of this work typically are limited to experimenting on a static data set.
• 6:38
  Speaker 2: They also often tackle just the classification or detection problems and they often skip the problem of dealing with the ground truth, this kind of spatial information that in our case is critical to train the model.
• 6:53
  Speaker 2: So in this work we tried to address these issues in the context of a full robotic application that we actually deploy in the real world and we try to measure with the entire pipeline.
• 7:06
  Speaker 2: So in actual closed-loop control with the drone in the field, we target the crazy fly 2.1 extended with processing and sensing capabilities.
• 7:17
  Speaker 2: In particular, we focus, we work with the gap family of system on chips from GreenWaves where you have access very quickly the data on the device in just a power budget of under 100 milliwatts and, in particular, we compare the two generations of system-on-chip GAP8 and GAP9.
• 8:00
  Speaker 2: So what we propose is to introduce an infield data collection phase that allows the drone to then fine-tune its model directly on board before using it to perform autonomous control, and we explore a number of practical implications of deploying this setup in the real world.
• 8:23
  Speaker 2: First is, how much data do we actually need to improve this model?
• 8:29
  Speaker 2: Because we already have a model training simulation.
• 8:32
  Speaker 2: So we want to do better than that, but we need to deal with the memory constraints of our platform.
• 8:38
  Speaker 2: Then we look at how to avoid the problem of the ground truth data and, finally, we explore how to best exploit the limited computational power.
• 8:51
  Speaker 2: So, for the first point, we test the effectiveness and robustness of our fine-tuning by evaluating the performance that we obtain on a real world test set by measuring regression performance on the three human subjects that are present in the data set, and we can see that in with these are to score, which is a normalized metric, where 100% corresponds to a perfect predictor and zero it's a predictor that always gives you a constant output.
• 9:25
  Speaker 2: Our baseline is really struggling.
• 9:28
  Speaker 2: It's negative, so worse than predicting always a constant output because it has never seen the real world and in particular, this challenging environment in the test set.
• 9:41
  Speaker 2: When we fine-tune the model in ideal condition, so we adapt the entire model and we assume that we have ground truth data, we can see that fine-tuning and testing on the same subject is strongly beneficial.
• 9:54
  Speaker 2: It gives more than 100 points of improvement.
• 9:58
  Speaker 2: But also if we fine-tune and test on different subjects, we see that the model is able to adapt to the real world, even the real world that it's never seen before in this setup.
• 10:11
  Speaker 2: It still gives it the ability to improve its predictions.
• 10:15
  Speaker 2: And we then extend this analysis by evaluating the size of the data set that we use for fine-tuning.
• 10:25
  Speaker 2: So we have here the regression error average over all subjects for the baseline in which we don't do any fine-tuning, and we would like to in this case reduce it because the size of the data set gives us a very real it's a very real constraint on the size of the memory.
• 10:48
  Speaker 2: So for the GAP8 system we only have 8 megabytes, which is already full with just these 502 samples of images and the prediction outputs.
• 11:02
  Speaker 2: So ideally we would like to reduce this and we have the previous experiment corresponding to these points, and we would like to act on two different parameters to reduce the size of the data set the acquisition frame rate and the flight time over which we recorded this data.
• 11:25
  Speaker 2: The flight time in particular, gives us a way to exploit the mission time more effectively because, as such a small platform, the battery time is usually limited to around five minutes.
• 11:40
  Speaker 2: So all the time that we spend for collecting data in the initial phase is time that we cannot use for performing the mission.
• 11:49
  Speaker 2: However, by acting just on the flight time, we can see that we have a significant reduction in the regression error, which is instead much more contained if we reduce the data set size by acting on the frame rate.
• 12:06
  Speaker 2: So we see that collecting data over a long time, it's quite beneficial for the model.
• 12:12
  Speaker 2: And then we test all the combinations in the middle of the two parameters frame rate and flight time, identifying two interesting points in which we have the minimum regression error or this sweet spot in which we have a much smaller data set without paying a lot in terms of regression performance.
• 12:33
  Speaker 2: On the other hand, this was done assuming that we have the ground truth data, which in our case corresponds to a system, an external system that gives us the poses of the drone and subject and allows us to perform ordinary supervised loss, which tries to map the prediction of the model towards these known targets.
• 12:58
  Speaker 2: So it tries to act on individual predictions of the model.
• 13:02
  Speaker 2: In this work, we apply instead a self-supervised state consistency loss, so it tries to act on individual predictions of the model.
• 13:06
  Speaker 2: In this work, we apply instead a self-supervised state consistency loss, which operates on a pair of model predictions every time and tries to make them consistent with the estimation that we have of the drone movement coming from the onboard odometry.
• 13:19
  Speaker 2: Coming from the onboard odometry, so we have sensors such as IMUs, that gives us this relative displacement of the drone itself between time steps, and we want the prediction to be consistent with this movement, which in this case, in this figure, would correspond to the two blue dots overlapping the same.
• 13:43
  Speaker 2: Like being coincident, we do this with a loss that allows us to handle uncertainty in the odometer estimation and also the unpredictability of the target movement, since, which allows the people to move while we collect data, allowing us to get more diversity, and this shows that when we do supervised training, we are able to reduce the regression error by around 56%, while with the self-supervised loss we are still able to achieve a significant improvement of 30%.
• 14:24
  Speaker 2: At this point we looked into how much does this process cost to execute on device and how to reduce this by looking at the architecture that we have in our model and how the different parts impact on the training workload.
• 14:44
  Speaker 2: So we have an ordinary architecture with convolutional batch normalization and ReLU layers and a fully connected layer at the beginning with a series of repeated convolution blocks in the middle.
• 15:01
  Speaker 2: So our baseline that we looked at until this point was to fine-tune the entire model, which updates the most parameter but has also the highest expected cost.
• 15:12
  Speaker 2: We then look from the literature for ways to reduce this cost by focusing just on some subsets of the parameter in the model, such as fine-tuning just the batch normalization parameters, the fully connected layer and the bias of the models.
• 15:33
  Speaker 2: This also shows that we are able to strongly reduce the fine-tuning cost while still achieving large improvements in regression performance with this sweet spot of the bias fine-tuning, which reduces the time quite a lot while still improving by 24%.
• 15:56
  Speaker 2: We implemented this on device with full pipelining of the memory transfers and the computation of the forward and backward phases, and explored the optimization of the software kernels that we use for these parts, showing up to 100 times improvement compared to a baseline implementation, and measure the performance of this on-device execution on our target system on chip, achieving the 120 milliwatts of power consumption on average that we were showing in the beginning, and this we also have a demonstrations of the.
• 16:42
  Speaker 2: We have also a demonstration of this system in which we show the drone predictions and ground truth against the model prediction for the baseline and the self-supervised model coming in a second.
• 16:56
  Speaker 2: So we can see that the baseline has a significant error compared to the ground truth, while the self-supervised baseline is able to track it much more closely.
• 17:07
  Speaker 2: To conclude, we have shown that it is possible to apply on-device self-supervised learning in the field on a real robotic application.
• 17:16
  Speaker 2: With this work, we have fine-tuning data that we can fit on the onboard device and collect it, just with the drone sensors.
• 17:25
  Speaker 2: We have this self-supervised approach that exploits these sensor data without relying on external infrastructure, and we have on-device implementation that is able to train in just 32 seconds.
• 17:43
  Speaker 2: Parts of this result we have already published in ICRA in a big robotics conference this year, and more will be coming later in a follow-up paper.
• 17:55
  Speaker 2: But in general we have shown that this fine-tuning is able to make the difference between mission success and failure With this in-field experiment, where we show that the, having defined two model, is able to track much more closely and avoid losing track of the subject and crashing, in practice completing the mission successfully.
• 18:20
  Speaker 2: So with this, I thank you for your attention and if you have any questions, thank you, elia.
• 18:24
  Speaker 1: Yeah, with this, I thank you for your attention and if you have any questions.
• 18:25
  Speaker 1: Thank you, Elia.
• 18:25
  Speaker 1: Yeah, we have time for maybe one or two questions.
• 18:39
  Speaker 2: Oh, Francesco, it's just a curiosity.
• 18:40
  Speaker 2: The one that you're showing here that actually works in the field is the one where you train the biases, or what?
• 18:46
  Speaker 2: Yes, in this case it's the bias.
• 18:53
  Speaker 2: Thank you.
• 18:55
  Speaker 1: Any more questions, so I have one.
• 19:02
  Speaker 1: One of the challenges of on-device or online learning is how to forget things.
• 19:08
  Speaker 1: How can this be implemented, or could it be implemented?
• 19:15
  Speaker 2: So in our work we didn't explore the access.
• 19:18
  Speaker 2: That is something that we want to focus on in follow-up work, because indeed here we do fine-tuning with just the new data collected in the field and we don't explore how much we are still able to perform in the old environment.
• 19:32
  Speaker 2: So what we are looking into also right now is how to apply continual learning methods to prevent forgetting from previous information.
• 19:41
  Speaker 2: That is something that we don't have yet.
• 19:46
  Speaker 1: Oh, there's one more question.
• 19:51
  Speaker 3: Hi, thanks for the presentation.
• 19:53
  Speaker 3: It was really interesting and on point.
• 19:55
  Speaker 3: Can I ask you to comment a little bit more on the self-supervised metric In particular?
• 20:02
  Speaker 3: I'm interested in understanding it does not require any label at all at the beginning right and later on.
• 20:15
  Speaker 3: If it's so, the part in which you are performing the fine-tuning requires the target to be still correct in a fixed position.
• 20:30
  Speaker 2: So okay, it's getting a little long, but okay.
• 20:33
  Speaker 2: So we do need the odometry data, which is like the green relative pose that you see here, which is actually computed just from the drone sensor.
• 20:44
  Speaker 2: So that one we need because it gives us information on how also the model predictions need to change.
• 20:52
  Speaker 2: And in the baseline version, yes, we do need the person to be still, but you can compute this same loss also if the person is moving.
• 21:03
  Speaker 2: You just ignore the movement of the person.
• 21:06
  Speaker 2: Basically, in the loss you can assume that the person stayed constant and the person movement becomes like measurement noise in the data.
• 21:16
  Speaker 2: And if you do this for if you collect enough data and you like, we have a few tricks to ensure that there is no bias in the movement of the person that we collect Then like tricks that we apply to the data in post-processing, like the person just moves and then we select data so that the movements are not biased Then we show that we can improve also without knowing the movement.
• 21:44
  Speaker 2: You can check the paper if you like the details.
• 21:48
  Speaker 2: There is the method there.
• 21:51
  Speaker 1: Thank you, Elia.