Latest / Satellite image deep learning / AutoML for Spaceborne AI
Transcript
- 0:04SPEAKER_00: Hi, Roberto.
- 0:05SPEAKER_00: Welcome back.
- 0:05SPEAKER_01: Hi, Robin.
- 0:07SPEAKER_01: It's always a pleasure.
- 0:08SPEAKER_00: Third time now.
- 0:09So let's have a little intro to yourself and what's been going
- 0:13on recently.
- 0:15SPEAKER_01: All right.
- 0:16Like last time,
- 0:16I still work in ESA,
- 0:18the Philop,
- 0:20and I serve as an internal research fellow.
- 0:22SPEAKER_01: So it's like a postdoc position.
- 0:25My domain of expertise is how to bring artificial intelligence
- 0:28on board satellites.
- 0:29SPEAKER_00: Mm-hmm.
- 0:30SPEAKER_01: Lately,
- 0:30I've been working on a very specific topic which concerns the
- 0:35automatic machine
- 0:36SPEAKER_01: learning or AutoML.
- 0:38SPEAKER_00: Fantastic.
- 0:40SPEAKER_01: And what I've been working on is like,
- 0:42how can speed up the development of models to deploy it on our
- 0:47satellites in a more
- 0:48SPEAKER_01: fast and efficient way.
- 0:50SPEAKER_01: Whereas the result is kind of overhead in training your model,
- 0:53SPEAKER_01: selecting your model,
- 0:54SPEAKER_01: fine tune it,
- 0:56SPEAKER_01: deploy it on your satellite,
- 0:57try to strike a good balance or a good trade off between the
- 1:02accuracy of your model
- 1:04SPEAKER_01: and the latency.
- 1:05SPEAKER_01: So how fast they can go.
- 1:07Right,
- 1:08so on the satellite,
- 1:09you don't really obviously get to choose among different
- 1:11hardwares.
- 1:12SPEAKER_00: They've selected something.
- 1:13SPEAKER_00: You need to optimize your solution for what's already there.
- 1:17SPEAKER_01: Exactly, exactly.
- 1:18SPEAKER_01: And you don't have all the power that you will have on the ground, right?
- 1:21SPEAKER_00: Right, interesting.
- 1:23SPEAKER_01: You have a lot of things.
- 1:26SPEAKER_00: Yeah, I mean, just ballpark, how would it compare on the ground?
- 1:30SPEAKER_00: I've got my GPUs available in space.
- 1:32Are we talking about like 10% of the available compute,
- 1:365%,
- 1:36something like that?
- 1:38SPEAKER_01: So let's say on ground, you could have something like 300 watts of power.
- 1:43SPEAKER_00: Mm-hmm.
- 1:44on board you can have something like three watts but it
- 1:47strongly depends on how big
- 1:48is your satellite for cubesat maybe 10 watts right so we're
- 1:53talking a couple of
- 1:54orders of magnitude in difference in available power generally
- 1:58speaking very
- 1:59interesting right so you have a paper that you've been working
- 2:03on let's cover some
- 2:04SPEAKER_00: of the details
- 2:06SPEAKER_01: Yes,
- 2:06SPEAKER_01: so I was questioning myself,
- 2:08SPEAKER_01: okay,
- 2:09we always try to bring this model from literature when we need
- 2:13to solve a task.
- 2:14SPEAKER_01: But most of these models that comes from literature,
- 2:17SPEAKER_01: they've been hardly optimized on specific data set and some specific data
- 2:21SPEAKER_01: distribution.
- 2:23SPEAKER_01: But when it comes to the satellite and to the new space segment,
- 2:27SPEAKER_01: we're always looking at verticalized application.
- 2:29So I think like a satellite with onboard AI that generally has
- 2:33to do only one thing
- 2:35SPEAKER_01: and it has to do it very,
- 2:36SPEAKER_01: very quickly.
- 2:37SPEAKER_01: Like, I don't know, detect a wildfire here.
- 2:40So they have a segment with the constellation of these
- 2:43AI-enabled satellites.
- 2:45SPEAKER_01: and they want to do only a single task.
- 2:48SPEAKER_01: So in that case,
- 2:49the model that we bring on orbit must be tailored and optimized
- 2:53towards that task
- 2:54SPEAKER_01: and data distribution.
- 2:57But we fail and do that because we usually take models already
- 3:02pre-made by the
- 3:03SPEAKER_01: literature to solve very different problems like,
- 3:08SPEAKER_01: I don't know,
- 3:08SPEAKER_01: classification on ImageNet or object detection on Cocoa dataset,
- 3:13which the data is completely different and therefore their
- 3:16architecture is made for
- 3:17SPEAKER_01: doing something completely different.
- 3:21SPEAKER_00: So you have a specific task, you know, fire detection.
- 3:24SPEAKER_00: You've curated a specific data set for that satellite.
- 3:28SPEAKER_00: And the next optimization is curating the model.
- 3:32exactly exactly the thing is that you want to squeeze as much
- 3:35as you can the
- 3:36performance of that model and on that hardware that is
- 3:40available on your satellite
- 3:42because you want to reduce the duty cycle so you want to use as
- 3:46much as less power
- 3:48as you can because if you use less power you use less energy of
- 3:52course and the
- 3:53battery gets benefits from it so you can extend the lifetime of
- 3:56your mission and so
- 3:58SPEAKER_01: it's a lot of money that you can save in the end
- 4:01SPEAKER_00: Yeah,
- 4:01SPEAKER_00: so the best performance for the least possible power,
- 4:04SPEAKER_00: that's the point you're trying to find.
- 4:07So the point is that best performance for satellite application
- 4:10and this kind of a
- 4:11SPEAKER_01: segment is not always having the best accuracy,
- 4:15SPEAKER_01: in my opinion.
- 4:16You don't want to have a model that performs three,
- 4:19four,
- 4:19five points of percentage at maximum.
- 4:22You don't want to have that delta increase if that costs you a
- 4:25lot of power on orbit.
- 4:26SPEAKER_01: Mm-hmm.
- 4:28SPEAKER_01: You want to strike a good balance between the accuracy,
- 4:31SPEAKER_01: so how performant is your model in that sense,
- 4:34but also how fast that model is in processing the whole
- 4:39acquisition on your
- 4:40SPEAKER_01: satellite.
- 4:41SPEAKER_00: Right.
- 4:41SPEAKER_00: So we're not just talking about a metric that we're targeting now,
- 4:44we're talking about different kind of characteristics of a
- 4:47model that we want to
- 4:48SPEAKER_00: achieve.
- 4:49SPEAKER_01: We can say that we are looking at a multi-objective optimization.
- 4:54which we have two concurrent objectives,
- 4:56and we try to optimize both of them.
- 4:59SPEAKER_00: Right, okay.
- 5:01That segues nicely into how you're doing that with the
- 5:04architecture search.
- 5:06SPEAKER_01: So I've been looking a little bit,
- 5:08and I discovered that this kind of technology was already
- 5:14developed for computer
- 5:15SPEAKER_01: vision tasks,
- 5:16SPEAKER_01: but it was not really applied a lot in the geospatial community.
- 5:19SPEAKER_01: So I decided to make my own framework.
- 5:25SPEAKER_01: And yeah,
- 5:26what I did is like applying a genetic algorithm optimizer and
- 5:32like build this
- 5:32SPEAKER_01: framework,
- 5:33SPEAKER_01: which is called Pinus,
- 5:34SPEAKER_01: that works in this way.
- 5:36SPEAKER_01: Essentially, it creates population of networks.
- 5:38And then you just see which of these networks are the most top
- 5:44performing.
- 5:45SPEAKER_01: And then you optimize the topology of those network in a cycle.
- 5:51But the key part in this framework is that we profile each
- 5:55model on the specific edge device.
- 5:58SPEAKER_01: So we have similar devices to the one that are flying on the satellites,
- 6:03SPEAKER_01: and we see how these models are performing on these devices.
- 6:06SPEAKER_00: Okay,
- 6:07SPEAKER_00: so you have on your desk literally these different computers plugged in,
- 6:11SPEAKER_00: and you run your software,
- 6:12SPEAKER_00: and you get scores for each of the different hardwares.
- 6:17SPEAKER_01: Exactly, exactly.
- 6:20SPEAKER_01: So...
- 6:21to expand a little bit on that each vendor mixes memories on
- 6:26the device in a very
- 6:27different way so you might find that some layers are better
- 6:31suited for some devices
- 6:33or in some other devices seems they use their own compilation
- 6:38some layers are not
- 6:39even supported so you cannot use all the layers of modern deep
- 6:43learning on some
- 6:44SPEAKER_01: devices
- 6:46really a matter of finding what is the best solution for that
- 6:50kind of hardware that
- 6:51you have software for your hardware interesting yeah what are
- 6:57some of the sort of
- 6:58SPEAKER_00: constraints there what kind of layers are we talking about typically
- 7:02SPEAKER_01: So,
- 7:02SPEAKER_01: for instance,
- 7:04SPEAKER_01: in a Miriam 2 that is flying on a Fisat,
- 7:07SPEAKER_01: on Fisat 2,
- 7:08SPEAKER_01: and was flying on Canini and Fisat 1,
- 7:11SPEAKER_01: well,
- 7:11SPEAKER_01: on that kind of device,
- 7:12SPEAKER_01: vision transformers are not very well supported.
- 7:16As you know,
- 7:17transformers are kind of the basic block of the modern deep
- 7:20learning architectures.
- 7:23So these are generally examples,
- 7:25but there are also other layers that are not supported.
- 7:30In other cases,
- 7:31during compilation times,
- 7:32the kernel gets fused together.
- 7:34SPEAKER_01: So sometimes,
- 7:37SPEAKER_01: let's say,
- 7:38three convolution with a kernel of 3x3 works better than a
- 7:42single convolution with
- 7:43SPEAKER_01: a kernel of 7x7,
- 7:45SPEAKER_01: or for some other devices.
- 7:49SPEAKER_01: It's really a challenge of discovering what is the best for your device.
- 7:55SPEAKER_00: Right.
- 7:56SPEAKER_00: So for each device,
- 7:57you have a table which says these are the kind of optimized
- 8:00layers that you should
- 8:01SPEAKER_00: utilize.
- 8:02Now,
- 8:02design and architecture that utilizes them and gives me the
- 8:05best result.
- 8:07Yeah,
- 8:07it works kind of like that,
- 8:09but I let the optimizer do all the job for me.
- 8:11SPEAKER_00: Mm-hmm.
- 8:13SPEAKER_01: by simply profiling each of these models on the devices,
- 8:17he's looking at which is the top architecture that is
- 8:21performing the best.
- 8:23SPEAKER_01: And then after tweaking that architecture a little bit,
- 8:26SPEAKER_01: discovering path,
- 8:27SPEAKER_01: changing the topology,
- 8:29SPEAKER_01: trying to see if that kind of architecture can be improved.
- 8:33SPEAKER_00: Right.
- 8:33That's the zone in a parallel way,
- 8:36employing different models at the same time.
- 8:39SPEAKER_00: Right.
- 8:39SPEAKER_00: I just want to understand the algorithm a little bit more there.
- 8:41SPEAKER_00: So it starts with a kind of grid search,
- 8:43SPEAKER_00: finds some promising architectures,
- 8:46SPEAKER_00: and then experiments further on those,
- 8:48SPEAKER_00: does it?
- 8:49SPEAKER_01: Yeah.
- 8:49SPEAKER_01: No, this class of algorithm are known in literature as meta-heuristic.
- 8:54SPEAKER_01: and are used to solve non-convex problems.
- 8:58SPEAKER_01: This one in particular is a population-based, is the genetic algorithm.
- 9:02SPEAKER_01: So you can consider it a grid, I see it more like a population.
- 9:07SPEAKER_01: You have a population made of several different individuals,
- 9:10SPEAKER_01: and each individual for you represents a model.
- 9:14SPEAKER_01: And after each generation,
- 9:16you kill the worst performing and then you recompile together
- 9:22the top performing,
- 9:23SPEAKER_01: creating a new architecture.
- 9:26The process continues in an iterative way until you reach a top
- 9:32performing architecture.
- 9:33SPEAKER_00: Right.
- 9:34So you basically are watching these different models kind of
- 9:37rising up a ramp
- 9:38towards some kind of best solution with the low performers
- 9:41being picked up.
- 9:43exactly actually we at the end of this optimization we obtain a
- 9:48set of top
- 9:48performing model and one can cherry pick uh the model that best
- 9:53satisfies his
- 9:53trade-off maybe someone is more interested in having a little
- 9:57bit of more of
- 9:58accuracy maybe someone else for another device is interested in
- 10:02having a little bit
- 10:03more of latency so at the end you obtain this solution which
- 10:06has like the pareto
- 10:08efficient solutions and you can cherry pick the model based on
- 10:11on your requirements
- 10:13So is there a lot of user skill required to get the best
- 10:16solution or is it pretty
- 10:17SPEAKER_00: much sort of push the button and it will arrive at the best solution?
- 10:22SPEAKER_01: Well,
- 10:22SPEAKER_01: you can tweak a little bit the parameters,
- 10:24SPEAKER_01: but more or less the optimizer should do the job for you.
- 10:28SPEAKER_01: You can't change how big are this population,
- 10:31SPEAKER_01: so how many models you want to train at the same time,
- 10:34SPEAKER_01: how much these models can be big,
- 10:36SPEAKER_01: so how many layers,
- 10:38SPEAKER_01: which layers they can use.
- 10:40SPEAKER_01: You can tweak this,
- 10:41SPEAKER_01: but on a general level,
- 10:43I would say that the optimizer really simplifies what you need
- 10:46to do because it
- 10:48SPEAKER_01: does all the job in the back end and you don't have to
- 10:50SPEAKER_01: to watch and to select which layers you need to use.
- 10:54SPEAKER_01: It does that for you.
- 10:56SPEAKER_00: Okay, so no transformers based on what's supported by the hardware.
- 10:59SPEAKER_00: What kind of architectures did you arrive at for your different tasks?
- 11:04SPEAKER_01: Well,
- 11:04usually they are kind of similar to mix of ConvNext and ResNets
- 11:10or mobile and
- 11:11SPEAKER_01: virtual bottlenecks.
- 11:14SPEAKER_01: Different kind of modules stack one on top of each other.
- 11:18SPEAKER_01: That's the kind of solution that he gets.
- 11:21But the remarkable thing is that these models are very,
- 11:23very tiny compared to the literature.
- 11:26We're talking over thousands of parameters compared to
- 11:29multi-million parameters
- 11:31that you will find in another model taken from another task for
- 11:35a computer vision
- 11:36SPEAKER_01: problem.
- 11:38SPEAKER_00: And what's the sort of trade-off in performance look like there?
- 11:42SPEAKER_01: So for defining the trade-off, I used a mathematical formula.
- 11:46SPEAKER_01: So it's basically combining two values in a single one.
- 11:51SPEAKER_01: And that is the value that I try to optimize.
- 11:53It's like having a sort of a weighted function that tries to
- 11:57balance both the
- 11:58SPEAKER_01: accuracy and the latency of my task.
- 12:04then at the end i can see the results on these graphs and then
- 12:06i can cherry pick
- 12:08which is like the the model that mostly satisfies my
- 12:12constraints that can be
- 12:13different depending on the on the type of mission that they
- 12:16have or on the type of
- 12:18task that i need to satisfy right okay so these results are
- 12:22presented in the paper
- 12:23um sounds very exciting what which model is going to be
- 12:27deployed first then
- 12:30SPEAKER_01: I think we have a lot of promising models here.
- 12:32SPEAKER_01: We have a model for wildfire detection.
- 12:34SPEAKER_01: This is further described in the paper.
- 12:38We also have a model for vessel detection that has been
- 12:42presented to a conference.
- 12:46SPEAKER_00: And I just want to have a look at the bigger picture here.
- 12:47SPEAKER_00: So you have this model that's on the satellites making some detections.
- 12:51SPEAKER_00: Maybe the accuracy has been traded a bit.
- 12:53Do they then integrate it to some wider pipeline where other
- 12:56models are maybe
- 12:58SPEAKER_00: looking at the images again on the ground?
- 13:02SPEAKER_00: Or is it just that single model that's going to do all the work?
- 13:08SPEAKER_01: So, okay, you're touching a bit of a pain of a problem.
- 13:12SPEAKER_01: That's like the domain gap.
- 13:14Because usually we train models on different data distribution
- 13:18on a different
- 13:18dataset compared to the data that is actually sensed by this
- 13:23new space satellite.
- 13:24Because they mount different sensors that are typically
- 13:27different from the data
- 13:29SPEAKER_01: that we already have collected.
- 13:31SPEAKER_01: And so it's always like this problem on filling that gap.
- 13:36SPEAKER_01: Because we try and with simulators,
- 13:38SPEAKER_01: we try to simulate the kind of data that we expect from that satellite.
- 13:42SPEAKER_01: But even in that case,
- 13:43there's still an existing gap between what we make synthetic
- 13:46expecting what the
- 13:47kind of sensor is acquiring and what is actually acquiring on
- 13:52that satellite.
- 13:53SPEAKER_01: So there is always this process of training one time,
- 13:56then you downlink again your images and then you fine tune
- 14:00again your images,
- 14:01SPEAKER_01: your model on these new images.
- 14:05SPEAKER_01: I will not say it's a continual learning process,
- 14:07SPEAKER_01: but we try to fine-tune to tweak the performance,
- 14:10SPEAKER_01: and we need to re-up the model in most of the cases.
- 14:14SPEAKER_01: This is something that happened in the past.
- 14:17SPEAKER_01: And I will say that having models that are very small is an advantage,
- 14:21SPEAKER_01: because it's kind of easy to fine-tune it,
- 14:23SPEAKER_01: and it's kind of easy to update again these models,
- 14:26SPEAKER_01: because their weight in terms of megabytes is very,
- 14:29SPEAKER_01: very low.
- 14:29SPEAKER_01: We're talking about...
- 14:30megabytes of models or sometimes even more even less it's kind
- 14:35of easy uplink so
- 14:38you anticipate updating the models in response to gathering
- 14:41more data of course um
- 14:45uh and also how practical is it to deploy these models if you
- 14:48want to deploy a new
- 14:49version is it kind of a straightforward deployment like cicd
- 14:53kind of operation or
- 14:55is there a lot of work to sort of get date and model ready for
- 14:58gag on a satellite
- 15:00SPEAKER_01: I would say most of the work here is mostly on the data side.
- 15:04SPEAKER_01: It's not really on the model side.
- 15:07SPEAKER_01: Because this pipeline here,
- 15:09SPEAKER_01: they're not really in CI,
- 15:11SPEAKER_01: CD,
- 15:12SPEAKER_01: but they are kind of optimized and automatized.
- 15:14SPEAKER_01: So there is not too much work on that.
- 15:17SPEAKER_01: But on data side, you really have to make a pipeline for your new data.
- 15:23SPEAKER_01: And then you need to update that model.
- 15:25SPEAKER_01: Because mostly of this data is not labeled.
- 15:28SPEAKER_01: So you really have to bear that in mind.
- 15:30SPEAKER_00: Right, okay.
- 15:32SPEAKER_00: So there'll be a period where you're basically onboarding this model and,
- 15:36SPEAKER_00: you know,
- 15:37viewing the predictions maybe and adding some more data to the
- 15:41training data set
- 15:42and potentially tweaking the architecture a bit and kind of
- 15:45going around this loop
- 15:46SPEAKER_00: a few times until you're happy.
- 15:48SPEAKER_01: That's something that usually happens.
- 15:51SPEAKER_01: Maybe not tweaking again the architecture,
- 15:53SPEAKER_01: but it really depends,
- 15:54SPEAKER_01: but at least giving a bit of fine tuning on this new data.
- 15:58SPEAKER_01: Sometimes it's not necessary to have a lot of this data.
- 16:01You can do a small fine tuning on only some very good quality
- 16:05information,
- 16:06SPEAKER_01: but you have to know that there is this gap and you need to close it.
- 16:10SPEAKER_00: Right, of course.
- 16:12SPEAKER_00: Moving on to the actual hardware that's available,
- 16:14what are the brands and what are the performance look like of
- 16:17those hardwares?
- 16:20SPEAKER_01: So, okay.
- 16:23SPEAKER_01: We have, of course, Nvidia with the Jetson chipset.
- 16:29SPEAKER_01: We have the Myriad from Intel.
- 16:31SPEAKER_01: And some of the boards developed by Botica.
- 16:35SPEAKER_01: And we have some boards developed from KP Labs as well.
- 16:39SPEAKER_01: I hope to not forget someone, but I think I'm going to forget someone.
- 16:43SPEAKER_01: But still,
- 16:44it's a lot of hardware and there's a lot of push on this kind
- 16:48of processing unit
- 16:49SPEAKER_01: that can do accelerators for small satellites.
- 16:54SPEAKER_00: Very interesting.
- 16:55SPEAKER_00: And do these different providers,
- 16:56do they take different approaches or target different kind of
- 16:59applications?
- 17:02SPEAKER_01: Let's say that they are very different,
- 17:04SPEAKER_01: one from each other,
- 17:05SPEAKER_01: also in the way they work from a model during the compilation time.
- 17:11SPEAKER_01: And that's,
- 17:12SPEAKER_01: again,
- 17:12SPEAKER_01: the problem that we were referring at the beginning,
- 17:15SPEAKER_01: that not each layer is supported by each one of those hardware.
- 17:19SPEAKER_00: Right.
- 17:20SPEAKER_00: Okay.
- 17:21I'm curious if there'd be a feedback loop that you would
- 17:23basically discover certain
- 17:24classes of models were optimal for certain kinds of tasks and
- 17:28that would drive you
- 17:29SPEAKER_00: towards particular hardware and give an edge to one provider.
- 17:34It's very difficult to extrapolate this generic rule because it
- 17:37also depends on the
- 17:38SPEAKER_01: way you make your optimization.
- 17:41SPEAKER_01: In the sense that if you look at the paper,
- 17:43you will notice that the model strain for the myriad were much
- 17:47strongly optimized
- 17:49SPEAKER_01: towards the latency.
- 17:51That is coming from a physical problem that the Myriad is a
- 17:55very low power device.
- 17:57So when you set that function to optimize this fitness between
- 18:02accuracy and
- 18:03SPEAKER_01: latency,
- 18:05SPEAKER_01: this is strongly pushing towards the latency compared to the accuracy.
- 18:09SPEAKER_00: Interesting.
- 18:10SPEAKER_00: And so a different provider might head in a different direction,
- 18:13SPEAKER_00: sort of utilize more power,
- 18:14SPEAKER_00: but be targeting a different kind of task.
- 18:18I would say that NVIDIA hardware is more general purpose and in
- 18:23particular is more powerful.
- 18:24Intel devices are much more focused towards certain kind of
- 18:29layers that are a bit
- 18:30SPEAKER_01: more efficient and they use much less power consumption that this
- 18:35nvidia chipset that we have right now but we are also in an
- 18:38historical period where
- 18:39things change very very quickly new hardware gets updated and
- 18:44new satellite gets
- 18:45launched uh every every six months i know for sure that next
- 18:50year we're gonna have
- 18:51new mission like ix there is also a constellation of uh ai
- 18:57enabled satellite by
- 18:58SPEAKER_01: planet that's uh the constellation so um
- 19:03SPEAKER_01: I think that the thing is going to change quickly in the future,
- 19:05but there is something that will be always constant in each one
- 19:09of these missions,
- 19:10SPEAKER_01: that is the efficient optimization of the models for a specific hardware.
- 19:18SPEAKER_01: That is something that you cannot overlook.
- 19:20SPEAKER_01: You need,
- 19:20SPEAKER_01: of course,
- 19:21SPEAKER_01: to use as much less power as you need,
- 19:24SPEAKER_01: because this is going to extend the lifetime of your mission.
- 19:27SPEAKER_00: Fantastic.
- 19:28SPEAKER_00: Well, really interesting topic.
- 19:29SPEAKER_00: Congratulations on the paper.
- 19:31SPEAKER_00: That's already published or that's to be released soon?
- 19:35SPEAKER_01: Published.
- 19:36SPEAKER_01: It's open source.
- 19:37SPEAKER_01: The code is available.
- 19:38SPEAKER_01: The data sets are all freely available.
- 19:40But in case you have any doubts,
- 19:43I feel free to,
- 19:44I ask you to connect with me.
- 19:46SPEAKER_01: Just send me an email or a message on LinkedIn.
- 19:49SPEAKER_01: I would be happy to provide with more information.
- 19:52SPEAKER_00: Brilliant.
- 19:52SPEAKER_00: I'll drop all the details in the show notes.
- 19:54SPEAKER_00: Well,
- 19:55SPEAKER_00: once again,
- 19:55thank you for joining me and all the best with the deployments
- 19:59and let's catch up
- 20:00SPEAKER_00: again in the future.
- 20:02SPEAKER_00: Ciao, bye-bye.
- 20:02SPEAKER_01: Thank you.
- 20:03SPEAKER_00: Ciao.