Latest / Satellite image deep learning / Earthquake detection with Sentinel-1

Transcript

Earthquake detection with Sentinel-1 · Satellite image deep learning

• 0:05
  SPEAKER_01: Hello and welcome to the Satellite Image Deep Learning Podcast.
• 0:08
  SPEAKER_01: I'm Robin Cole and it's my pleasure to present another technically focused episode in the series.
• 0:13
  SPEAKER_01: In this episode,
• 0:14
  SPEAKER_01: I caught up with Daniela Rege-Cambrian to learn about earthquake epicentre
• 0:18
  SPEAKER_01: detection with Sentinel-1 images.
• 0:20
  SPEAKER_01: Daniela has a key role in organising a new competition on this task.
• 0:24
  SPEAKER_01: We had a great discussion about the logistics of this competition and the lessons
• 0:28
  SPEAKER_01: he learned from organising a previous one.
• 0:31
  SPEAKER_01: I hope you enjoy this episode.
• 0:32
  SPEAKER_01: Hi Daniela, welcome to the podcast.
• 0:34
  SPEAKER_01: How are you?
• 0:36
  SPEAKER_00: Thanks for the invitation, and I'm glad to be here with you.
• 0:41
  SPEAKER_00: And yeah, really thanks.
• 0:43
  SPEAKER_01: Fantastic.
• 0:44
  SPEAKER_01: Now, I know you from your involvement in the ChadBud challenge.
• 0:47
  SPEAKER_01: Do you mind telling us a bit about that challenge?
• 0:50
  SPEAKER_00: Okay, so the last year, we ran a machine learning competition in ECML pick-up that we did.
• 0:57
  SPEAKER_00: And it was a great experience from a point of view.
• 1:01
  SPEAKER_00: It was quite interesting for us learning how to run competitions,
• 1:09
  SPEAKER_00: to deal with people,
• 1:10
  SPEAKER_00: deal with technical issues,
• 1:12
  SPEAKER_00: etc.
• 1:13
  SPEAKER_00: It was mainly...
• 1:14
  SPEAKER_00: about some topics on remote sensing.
• 1:18
  SPEAKER_00: In particular, we focused on a problem that was quite, I mean, explored.
• 1:24
  SPEAKER_00: It was burnout detection, mainly.
• 1:28
  SPEAKER_00: In this case, we want to address the delineation problem, so segmenting the areas.
• 1:34
  SPEAKER_00: And so we collected, I think, a reasonable number of people, 20, 30 people or less.
• 1:40
  SPEAKER_00: So we
• 1:42
  SPEAKER_00: I have to deal with a reasonable number of people,
• 1:44
  SPEAKER_00: not too much large people,
• 1:46
  SPEAKER_00: but it's quite challenging from a first experience point of view,
• 1:50
  SPEAKER_00: I will say.
• 1:51
  SPEAKER_00: And yeah, now help to go for another run about another competition.
• 2:00
  SPEAKER_01: Yeah, so that challenge was segmenting fire burned areas.
• 2:03
  SPEAKER_01: I'm just curious, how did it work?
• 2:07
  SPEAKER_01: Like how did people innovate their solutions?
• 2:11
  SPEAKER_01: Did they take the baseline and make iterations or did you see completely different solutions emerge?
• 2:17
  SPEAKER_00: So from what I've seen, mainly people have not changed too much the baseline.
• 2:23
  SPEAKER_00: They tried different solutions that are completely different from what we proposed.
• 2:30
  SPEAKER_00: Some people try to start with the classical indexes,
• 2:33
  SPEAKER_00: for example,
• 2:35
  SPEAKER_00: or other classical machine learning algorithms.
• 2:38
  SPEAKER_00: It seems to me they
• 2:41
  SPEAKER_00: These indexes and classical solutions do not provide any good results,
• 2:45
  SPEAKER_00: at least from what I've seen from the leaderboard,
• 2:48
  SPEAKER_00: I would say.
• 2:50
  SPEAKER_00: And the other people focused on,
• 2:53
  SPEAKER_00: for example,
• 2:54
  SPEAKER_00: more unit-like architecture,
• 2:56
  SPEAKER_00: I will say,
• 2:57
  SPEAKER_00: and their variation.
• 2:59
  SPEAKER_00: And also they tried to work with the different spectral bands of the images.
• 3:06
  SPEAKER_00: For example,
• 3:07
  SPEAKER_00: one of the winners focused only on the false composite color,
• 3:11
  SPEAKER_00: for example,
• 3:12
  SPEAKER_00: without using all the available channels in the images.
• 3:15
  SPEAKER_01: This is Sentinel-2 channels, was it?
• 3:18
  SPEAKER_00: Yeah, it was 12 channels because we used the images from Sentinel, too.
• 3:23
  SPEAKER_00: So we have both RGB and infrared and also an ultra-blue channel, if you remember well.
• 3:31
  SPEAKER_00: But yeah, people tried to improve and use other solutions from UNetLife.
• 3:37
  SPEAKER_00: Our baseline was a simple DeepLab v3, if you know the model.
• 3:42
  SPEAKER_00: And yeah, they...
• 3:44
  SPEAKER_00: do pretty good i will say since they beat the baseline on 20 points i will say so
• 3:54
  SPEAKER_01: So successful outcome for yourself as organizers of the competition.
• 4:00
  SPEAKER_01: Good quality solutions on that task.
• 4:04
  SPEAKER_00: Yeah, I will say they work at least.
• 4:07
  SPEAKER_00: So this is important for sure if we are doing some applicative,
• 4:11
  SPEAKER_00: I will say,
• 4:12
  SPEAKER_00: some application of current machine learning.
• 4:14
  SPEAKER_00: So they work at least.
• 4:16
  SPEAKER_00: They propose some good solution without,
• 4:17
  SPEAKER_00: for example,
• 4:18
  SPEAKER_00: as I said,
• 4:19
  SPEAKER_00: using every data disposal,
• 4:21
  SPEAKER_00: every channel disposal.
• 4:22
  SPEAKER_00: So they...
• 4:23
  SPEAKER_00: The innovator will say,
• 4:25
  SPEAKER_00: using only a part of the data,
• 4:27
  SPEAKER_00: this is good since this means that you don't really need everything that is
• 4:33
  SPEAKER_00: available from our point of view.
• 4:37
  SPEAKER_00: And yes, as I said, I was quite satisfied from the results since they beat our results, I would say.
• 4:47
  SPEAKER_00: So this is a good point.
• 4:52
  SPEAKER_01: Was there a particular metric that they were judged on?
• 4:56
  SPEAKER_00: They tried to maximize the classical intersection over union, I would say.
• 5:01
  SPEAKER_00: This is a binary problem, so it was quite good to evaluate the intersection over union.
• 5:08
  SPEAKER_00: And yeah, as I said, I've not seen any ensemble, big ensemble, more or less.
• 5:15
  SPEAKER_00: So this was a point that we tried to force people using a small model as possible to
• 5:26
  SPEAKER_00: improve existing models, use existing models without simply saying we assembled 10 models.
• 5:33
  SPEAKER_00: And yeah, for sure, building more models is good than using a single one.
• 5:37
  SPEAKER_00: But we forced, I think, a constraint to not use more than one GPU.
• 5:45
  SPEAKER_00: I would say it's arbitrary.
• 5:47
  SPEAKER_00: I would say since a GPU can have 80 gigabyte or 40 or 12 or well,
• 5:55
  SPEAKER_00: was simply a manner to avoid people using, as I said, larger samples.
• 6:01
  SPEAKER_01: Right.
• 6:01
  SPEAKER_01: So I think the challenge there is,
• 6:03
  SPEAKER_01: you know,
• 6:03
  SPEAKER_01: individuals might not have the same resources as a team,
• 6:06
  SPEAKER_01: a large organization.
• 6:08
  SPEAKER_01: So you're trying to level the playing field a bit there and,
• 6:11
  SPEAKER_01: I guess,
• 6:12
  SPEAKER_01: get a bit more engagement as a result.
• 6:14
  SPEAKER_00: Yes,
• 6:15
  SPEAKER_00: as you said,
• 6:16
  SPEAKER_00: people have to be,
• 6:17
  SPEAKER_00: I mean,
• 6:18
  SPEAKER_00: they must have the same instrumentation,
• 6:21
  SPEAKER_00: the same abilities to...
• 6:25
  SPEAKER_00: It's a game,
• 6:26
  SPEAKER_00: more or less.
• 6:27
  SPEAKER_00: So every people have to...
• 6:29
  SPEAKER_00: be in the same setting of other people to play right and fair with others.
• 6:35
  SPEAKER_00: Everyone has the possibility to win.
• 6:37
  SPEAKER_00: I'm quite passionate in game and I know this.
• 6:40
  SPEAKER_01: That's fantastic.
• 6:41
  SPEAKER_01: And so you've moved on to another competition this year.
• 6:43
  SPEAKER_01: Do you mind telling us what that one is about?
• 6:45
  SPEAKER_00: Yeah,
• 6:46
  SPEAKER_00: as you mentioned,
• 6:47
  SPEAKER_00: we want to run another competition on a different topic that I think is more under
• 6:53
  SPEAKER_00: explored is linked to earthquakes this year.
• 6:57
  SPEAKER_00: So we want to focus on their detection,
• 7:00
  SPEAKER_00: in particular to the detection of the epicenter,
• 7:04
  SPEAKER_00: mainly since from what I've learned from literature and other people working in geophysics,
• 7:10
  SPEAKER_00: et cetera,
• 7:10
  SPEAKER_00: et cetera,
• 7:12
  SPEAKER_00: uh currently is quite time consuming from what i've learned you need more stations
• 7:18
  SPEAKER_00: and each station have to detect the waves and then to detect the epicenter you have
• 7:23
  SPEAKER_00: to combine all the
• 7:26
  SPEAKER_00: detection of the different stations to understand where is the epicenter.
• 7:32
  SPEAKER_00: So it seems more time consuming,
• 7:35
  SPEAKER_00: while in this case simply having an image and then say the epicenter is here,
• 7:42
  SPEAKER_00: is not here.
• 7:44
  SPEAKER_01: So hopefully we can come up with a solution that just takes satellite images rather
• 7:50
  SPEAKER_01: than doing what is triangulation with ground sensors,
• 7:53
  SPEAKER_01: right?
• 7:54
  SPEAKER_00: Yeah,
• 7:55
  SPEAKER_00: we have also seen people working with these type of images from Sentinel-1 to
• 8:01
  SPEAKER_00: detect the area affected by earthquakes manually,
• 8:05
  SPEAKER_00: I will say.
• 8:06
  SPEAKER_00: So this seems feasible.
• 8:09
  SPEAKER_00: using the SAR instrument for this type of challenges, I will say.
• 8:14
  SPEAKER_00: So understanding earthquakes.
• 8:18
  SPEAKER_00: This is why now we have to propose a baseline.
• 8:22
  SPEAKER_00: I already tried something and it should work.
• 8:26
  SPEAKER_00: I hope people will try to propose a better solution for sure.
• 8:32
  SPEAKER_00: Since, as I said, they could be useful to
• 8:37
  SPEAKER_00: the humanity will say in this case, but also to geophysics, et cetera, et cetera.
• 8:43
  SPEAKER_01: So the gauntlet is thrown down,
• 8:44
  SPEAKER_01: a more challenging task potentially,
• 8:46
  SPEAKER_01: and a different sensor this time,
• 8:49
  SPEAKER_01: Sentinel-1 being the SAR sensor,
• 8:50
  SPEAKER_01: of course.
• 8:52
  SPEAKER_01: And what kind of data do you provide?
• 8:53
  SPEAKER_01: Is it a single image?
• 8:54
  SPEAKER_01: Is it a time series?
• 8:57
  SPEAKER_01: Francesco D' Okay,
• 8:58
  SPEAKER_00: as as done in the previous year we focused on to be temporal time series,
• 9:04
  SPEAKER_00: it will say so,
• 9:05
  SPEAKER_00: two images if you prefer calling them in this way,
• 9:12
  SPEAKER_00: and this is done because this is done because.
• 9:17
  SPEAKER_00: from our experimentation,
• 9:18
  SPEAKER_00: but also from other people's experimentation in manual,
• 9:22
  SPEAKER_00: I would say,
• 9:23
  SPEAKER_00: detection of these hazards,
• 9:28
  SPEAKER_00: you need two images because you need the interferogram if I'm
• 9:32
  SPEAKER_00: calling it well you are better than me in this and so yes also in this case we want
• 9:41
  SPEAKER_00: to be what people focusing on change detection i will say that oriented in this way
• 9:49
  SPEAKER_01: Yeah, just to summarize there, you obviously have a before and after SAR image.
• 9:53
  SPEAKER_01: The ground has moved.
• 9:54
  SPEAKER_01: You can do this, generate an interferogram, as you say, which will tell you how the ground has changed.
• 9:59
  SPEAKER_01: So presumably people will be using single image with interferogram.
• 10:04
  SPEAKER_01: And then the actual annotated data, you said it's the center of the epicenter of the
• 10:11
  SPEAKER_01: the earthquake, so is it a point, is it a segmentation mask?
• 10:14
  SPEAKER_00: It's a point, as you said, since the epicentre is one point.
• 10:18
  SPEAKER_00: Our,
• 10:20
  SPEAKER_00: I mean,
• 10:20
  SPEAKER_00: trustable source,
• 10:21
  SPEAKER_00: I would say,
• 10:21
  SPEAKER_00: that is the International Seismological Centre,
• 10:26
  SPEAKER_00: so we can trust them,
• 10:27
  SPEAKER_00: I hope.
• 10:28
  SPEAKER_00: And from their data, we tried to map all the epicenters on images, I will say.
• 10:35
  SPEAKER_00: So yes, as I said, you have to regress a single point.
• 10:38
  SPEAKER_00: But I think it's not really straightforward, as you said before.
• 10:42
  SPEAKER_00: Seems more complex than simply segmenting, for example.
• 10:48
  SPEAKER_00: Since you can rely on visual features, while SAR, as you mentioned, is not really
• 10:56
  SPEAKER_00: a picture, an image is more radiometric, if I'm calling this type of image as well.
• 11:06
  SPEAKER_01: Well, that sounds like a really interesting task.
• 11:08
  SPEAKER_01: I'm really looking forward to seeing the solutions people come up with.
• 11:11
  SPEAKER_01: And as you say, something a bit more, I think, involved than just segmentation.
• 11:15
  SPEAKER_01: Obviously, this is your second competition.
• 11:18
  SPEAKER_01: I'm curious, what's it like organizing these competitions, running them?
• 11:22
  SPEAKER_01: Did you learn anything significant in the first one that you're going to change the
• 11:26
  SPEAKER_01: way you run the second one?
• 11:28
  SPEAKER_00: So from our point of view, I think we will change the platform.
• 11:34
  SPEAKER_00: So this is not really linked to our previous competition,
• 11:39
  SPEAKER_00: I will say,
• 11:40
  SPEAKER_00: but it's a change from our point of view.
• 11:43
  SPEAKER_00: We have to learn to deal with another type of management.
• 11:47
  SPEAKER_00: Also, as I said, from our previous competition, from what I've seen, at least,
• 11:58
  SPEAKER_00: I would say you are not busy for all the time of the competition.
• 12:05
  SPEAKER_00: You have really low effort times and other times you have to deal with many things.
• 12:13
  SPEAKER_00: Since at the beginning you have to release something, then you have to...
• 12:19
  SPEAKER_00: give people time to deal with your data,
• 12:22
  SPEAKER_00: then you have to release the test,
• 12:24
  SPEAKER_00: you have to deal with the final part in collecting all the results,
• 12:28
  SPEAKER_00: understand their reports,
• 12:31
  SPEAKER_00: for example,
• 12:32
  SPEAKER_00: try to,
• 12:33
  SPEAKER_00: yeah,
• 12:33
  SPEAKER_00: you know,
• 12:34
  SPEAKER_00: there are many things that they will take in account.
• 12:37
  SPEAKER_00: At least in dealing with 30 people, as I said, 20 people.
• 12:41
  SPEAKER_00: Probably if 100 people is not the same.
• 12:44
  SPEAKER_00: I've also taken part in some other competitions.
• 12:46
  SPEAKER_00: So I'm pretty sure that larger competition are more complex.
• 12:51
  SPEAKER_00: But as I said,
• 12:52
  SPEAKER_00: since our goal is not simply dedicate all our time into this competition,
• 12:59
  SPEAKER_00: but letting people try solving interesting problems,
• 13:05
  SPEAKER_00: probably.
• 13:08
  SPEAKER_00: We have to manage our time in dealing with the pollution and other things.
• 13:14
  SPEAKER_00: So this is, I think, if you want to run a competition, you must have in mind that these type of things.
• 13:21
  SPEAKER_00: So there are moments in which you have
• 13:25
  SPEAKER_00: you are involved more in the competition,
• 13:29
  SPEAKER_00: in dealing with things other times in which you have to do nothing.
• 13:33
  SPEAKER_00: So this is a point that you have to take into account.
• 13:37
  SPEAKER_00: Other problem is releasing the things that is not so straightforward because also
• 13:44
  SPEAKER_00: speaking to another guy he told me the size of the things have to be comfortable
• 13:51
  SPEAKER_00: for people because as you said probably are
• 13:56
  SPEAKER_00: common people, not big societies, big companies.
• 13:58
  SPEAKER_00: So they have to deal easily with your data, with models.
• 14:03
  SPEAKER_00: They have to be fair to competition, as you said.
• 14:06
  SPEAKER_00: So it's not so straightforward.
• 14:08
  SPEAKER_00: We have to think,
• 14:08
  SPEAKER_00: well,
• 14:09
  SPEAKER_00: which are the constraints you want to pose,
• 14:12
  SPEAKER_00: if they are feasible,
• 14:13
  SPEAKER_00: if you could evaluate the results well.
• 14:17
  SPEAKER_00: So there are many points that I've heard.
• 14:19
  SPEAKER_00: Yeah.
• 14:20
  SPEAKER_01: Okay, so a few lessons learned around the organization and scheduling, I suppose.
• 14:26
  SPEAKER_01: Change of platform.
• 14:27
  SPEAKER_01: Which platform are you going to be hosting this competition on then?
• 14:29
  SPEAKER_00: So probably we will go for QuotaLab.
• 14:32
  SPEAKER_00: That is quite well renowned, I think, this year.
• 14:36
  SPEAKER_00: Since, as I said, the point is simply progressing a point.
• 14:41
  SPEAKER_00: So it will be more straightforward for people sending their questions
• 14:48
  SPEAKER_00: uh i would say their predictions and it is easier for us uh evaluating their
• 14:55
  SPEAKER_00: results so i would say we will go for quota this is free for these type of
• 15:01
  SPEAKER_00: challenges is it is working
• 15:04
  SPEAKER_01: And do people, do they submit code?
• 15:07
  SPEAKER_01: Do they submit model weights?
• 15:08
  SPEAKER_01: Do they somehow, you know, generate some predictions that you evaluate?
• 15:14
  SPEAKER_00: So for sure they have to submit the predictions.
• 15:16
  SPEAKER_00: Also this year we would like to,
• 15:19
  SPEAKER_00: also the previous year,
• 15:20
  SPEAKER_00: we suggested people opening their code for evaluation with
• 15:25
  SPEAKER_00: have not run their code.
• 15:30
  SPEAKER_00: This year instead, we would like to understand better the resource consumption of their models.
• 15:39
  SPEAKER_00: We want to prove that, I will say, bigger is not better, I will say.
• 15:45
  SPEAKER_00: So also from my experimentation,
• 15:48
  SPEAKER_00: from my understanding,
• 15:50
  SPEAKER_00: empirical understanding,
• 15:53
  SPEAKER_00: I will say these tasks are not so complex.
• 15:57
  SPEAKER_00: And so you don't really need to involve, for example, I will say, same like or something similar.
• 16:03
  SPEAKER_00: Using the mobile net, you got good results.
• 16:07
  SPEAKER_00: So forcing people in using low results models will be a constraint to force people
• 16:15
  SPEAKER_00: simply not saying,
• 16:16
  SPEAKER_00: I will go for bigger models,
• 16:18
  SPEAKER_00: as I said,
• 16:19
  SPEAKER_00: and
• 16:19
  SPEAKER_00: I think everyone could take part with us or a simple GPU with classical things.
• 16:29
  SPEAKER_01: Just one question.
• 16:30
  SPEAKER_01: If they're providing the predictions, that means they have access to the test dataset.
• 16:34
  SPEAKER_01: How do you prevent people overfitting the dataset?
• 16:37
  SPEAKER_00: In the first part, we will provide, as done in the previous year, only public validation, I will say.
• 16:44
  SPEAKER_00: Also, in this case, we will provide
• 16:48
  SPEAKER_00: the ground truth, I would say, also for the public.
• 16:53
  SPEAKER_00: But since it's more like a self-devaluation,
• 16:56
  SPEAKER_00: I would say,
• 16:57
  SPEAKER_00: and to let know if you want other people how you are performing,
• 17:03
  SPEAKER_00: it's like a way to...
• 17:07
  SPEAKER_00: force people in saying i'm doing better than you so keep the challenge running and
• 17:15
  SPEAKER_00: doing better this is simply the rationale i would say around the decision and from
• 17:21
  SPEAKER_00: my point of view i think it can work and also from the previous year i've seen that
• 17:27
  SPEAKER_00: people are submitting try to
• 17:29
  SPEAKER_00: doing better.
• 17:30
  SPEAKER_00: Some people have overfitted,
• 17:32
  SPEAKER_00: but at a certain point,
• 17:34
  SPEAKER_00: it could be clear,
• 17:35
  SPEAKER_00: since if you go up 90%,
• 17:38
  SPEAKER_00: I would say,
• 17:39
  SPEAKER_00: it can be suspicious.
• 17:43
  SPEAKER_00: And instead,
• 17:43
  SPEAKER_00: for the real private test set,
• 17:45
  SPEAKER_00: we are simply releasing the data without the prediction,
• 17:50
  SPEAKER_00: as done in the previous year,
• 17:51
  SPEAKER_00: so people cannot overfit in any case.
• 17:55
  SPEAKER_00: Fantastic.
• 17:56
  SPEAKER_01: Okay, well, thank you for the overview and the history of the series.
• 17:59
  SPEAKER_01: Why don't you tell us the basics when the competition begins?
• 18:02
  SPEAKER_01: When does it end?
• 18:04
  SPEAKER_01: So,
• 18:05
  SPEAKER_00: for now,
• 18:06
  SPEAKER_00: we have to deal with the proposal,
• 18:08
  SPEAKER_00: since we are waiting,
• 18:10
  SPEAKER_00: it should be in these weeks,
• 18:13
  SPEAKER_00: I will say the call for the proposals.
• 18:20
  SPEAKER_00: And then if
• 18:23
  SPEAKER_00: The timeline is the same of the previous year from March, April until the end of the summer, I will say.
• 18:33
  SPEAKER_01: Okay, so spring and 2024 ending in summer.
• 18:38
  SPEAKER_00: Yeah, according to the previous year, as I said.
• 18:41
  SPEAKER_00: I don't know this year, but I hope they will come to us with the deadlines for expectations.
• 18:49
  SPEAKER_01: Fantastic.
• 18:50
  SPEAKER_01: Okay, well, I'll include the details for the competition in the show notes.
• 18:54
  SPEAKER_01: People can check them out there.
• 18:55
  SPEAKER_01: If people want to follow you personally, Daniele, what's the best platform?
• 18:59
  SPEAKER_00: Following me, you said?
• 19:01
  SPEAKER_00: Yeah.
• 19:02
  SPEAKER_00: You can follow me probably...
• 19:04
  SPEAKER_00: Yeah, searching for me on the internet is my name and surname, Daniele Rege Cambrin.
• 19:10
  SPEAKER_00: It's unique.
• 19:11
  SPEAKER_00: I'm the only one with this name.
• 19:13
  SPEAKER_00: So you can easily, I will say, you can reach me by email.
• 19:18
  SPEAKER_00: You can find me on Discord or other social LinkedIn, for example.
• 19:23
  SPEAKER_00: You can follow me.
• 19:25
  SPEAKER_00: uh or what do you prefer as i said i'm also a game passionate so you can find me on
• 19:29
  SPEAKER_00: a game platform i would say oh okay people are quite passionate and want to play
• 19:35
  SPEAKER_01: yes that's another invite then two competitions are running yeah yeah yeah for sure
• 19:40
  SPEAKER_00: for sure as i said i'm really passionate this is why i'm running this competition
• 19:44
  SPEAKER_01: it's like a game i like games that's great wonderful well once again thanks again
• 19:50
  SPEAKER_01: and uh looking forward to the competition
• 19:53
  SPEAKER_00: Okay, thanks.
• 19:54
  SPEAKER_00: I hope to see many people this year as in the previous year.
• 19:58
  SPEAKER_00: And thanks to you,
• 19:59
  SPEAKER_00: Robin,
• 20:00
  SPEAKER_00: for the invitation and to discuss with you about these topics and new things,
• 20:07
  SPEAKER_00: I would say,
• 20:07
  SPEAKER_00: in machine learning field.
• 20:10
  SPEAKER_01: It's my pleasure.
• 20:10
  SPEAKER_01: Okay, ciao.
• 20:12
  SPEAKER_00: Ciao.