Latest / Satellite image deep learning / Location Embedding with SatCLIP, with Konstantin Klemmer

Transcript

Location Embedding with SatCLIP, with Konstantin Klemmer · Satellite image deep learning

• 0:05
  SPEAKER_00: Hello and welcome to the satellite image deep learning podcast.
• 0:08
  SPEAKER_00: I'm Robin Cole and it's my pleasure to present another technically focused episode in the series.
• 0:14
  SPEAKER_00: In this episode I catch up with Konstantin Klemmer to discuss SatCLIP, which is a new global and general purpose location encoder trained on Sentinel-2 imagery.
• 0:23
  SPEAKER_00: Our conversation covered the training of encoders such as CLIP and discussed the implications for downstream applications.
• 0:29
  SPEAKER_00: I hope you enjoy this episode.
• 0:31
  SPEAKER_00: Hi Konstantin, welcome to the podcast.
• 0:34
  SPEAKER_01: Thanks for having me.
• 0:35
  SPEAKER_00: Fantastic.
• 0:35
  SPEAKER_00: Why don't you get the ball rolling and tell us where you work and what you do.
• 0:41
  SPEAKER_01: Right.
• 0:42
  SPEAKER_01: So my name is Konstantin Klemmer.
• 0:44
  SPEAKER_01: I'm a postdoctoral researcher at Microsoft Research New England in Boston, Massachusetts.
• 0:51
  SPEAKER_01: And yeah, I have a PhD in computer science.
• 0:56
  SPEAKER_01: I have a master's in civil engineering.
• 0:59
  SPEAKER_01: I have an undergrad in economics and throughout my research career, I've been quite an interdisciplinary researcher.
• 1:05
  SPEAKER_01: But what all my academic journey has in common is that I've always been very excited about geospatial data.
• 1:14
  SPEAKER_01: Starting with more business analytics focused on problems like car sharing and bike sharing, which originally got me into this in my undergrad.
• 1:26
  SPEAKER_01: Satellite image deep learning featuring Robin Klemmer
• 1:44
  SPEAKER_01: I guess what is colloquially referred to as foundation models, so large neural network models that are trained on big data, big data without labels, right?
• 1:56
  SPEAKER_01: Which is something we have a lot in the geospace.
• 1:58
  SPEAKER_01: If you think about the kind of vast amounts of satellite imagery, for example, that exists, usually they're not labeled, right?
• 2:04
  SPEAKER_01: It's just a satellite image.
• 2:05
  SPEAKER_01: There's no kind of further information we have.
• 2:08
  SPEAKER_01: Like, I don't know.
• 2:10
  SPEAKER_01: is there a power plant on this image?
• 2:11
  SPEAKER_01: We don't necessarily have that information.
• 2:13
  SPEAKER_01: Often that requires kind of careful hand labeling, but there's still a lot we can learn from the unlabeled data.
• 2:20
  SPEAKER_01: And this is what I'm excited about currently.
• 2:21
  SPEAKER_00: I'm curious, was the fact that there's lots of unlabeled data part of the attraction for geospatial or did you...
• 2:27
  SPEAKER_01: No, not originally.
• 2:28
  SPEAKER_01: Originally, and I would say even now, my main motivation are impactful applications.
• 2:34
  SPEAKER_01: So I come from the more urban space, originally focused very much on urban mobility and transportation, which motivated me to get into this field.
• 2:44
  SPEAKER_01: I've kind of expanded my application focus a little bit.
• 2:49
  SPEAKER_01: So I'm also working on a lot of climate-related problems now.
• 2:55
  SPEAKER_01: and I strongly believe that there's so much opportunity for deploying machine learning specifically for geospatial data that can help us tackle really like grand human challenges from climate change to the kind of increasing urban densification that we see and yeah I believe it can really help people.
• 3:18
  SPEAKER_00: Yeah, I totally agree with that.
• 3:20
  SPEAKER_00: Talking of foundational models, that brings us on to the topic of this conversation, SatCLIP.
• 3:24
  SPEAKER_00: This is a paper that you've been involved in.
• 3:26
  SPEAKER_00: Do you mind giving us a bit of an introduction to SatCLIP?
• 3:30
  SPEAKER_01: Yeah, for sure.
• 3:31
  SPEAKER_01: First, let me shout out my amazing co-authors.
• 3:33
  SPEAKER_01: So this paper is written together with Mark Rosworm, who's an assistant professor at Wageningen University, Esther Rolf, who's a postdoc here in Boston at Harvard, and then two of my Microsoft colleagues, Lester Mackey, who's also with me in the New England lab, and Caleb Robinson, who is at the AI for Good lab in Redmond.
• 3:56
  SPEAKER_01: Right.
• 3:57
  SPEAKER_01: So yeah, let me start to explain the paper by talking about a different paper that kind of very much motivated this.
• 4:05
  SPEAKER_01: And that is the original CLIP paper.
• 4:07
  SPEAKER_01: So SatCLIP already has CLIP in the title.
• 4:11
  SPEAKER_01: And CLIP is a paper that was written by OpenAI, and it's not something that necessarily people in the general public have heard about, even though they've probably used it.
• 4:22
  SPEAKER_01: So let me explain that quickly.
• 4:25
  SPEAKER_01: What CLIP does is basically trying to translate in neural network space, in neural network terms, between image and text modalities.
• 4:39
  SPEAKER_01: Think about a large dataset of images with corresponding text, right?
• 4:43
  SPEAKER_01: So you might have an image of a, I don't know, a shepherd dog, and then the text description says this is a German shepherd in a green meadow, something like that.
• 4:55
  SPEAKER_01: So you have like a very large dataset, hundreds of thousands of images with corresponding texts.
• 5:01
  SPEAKER_01: And the idea of CLIP is to train a neural network to learn kind of a joint embedding that allows neural network models to translate from one modality to another.
• 5:12
  SPEAKER_01: And the way this is achieved in neural network terms again is to feed your image through an image encoder.
• 5:18
  SPEAKER_01: So go from the image space, so width, height, and then three RGB channels to some embedding space, let's say 512 characters, and do the same for the corresponding text.
• 5:28
  SPEAKER_01: So go from the text sequence or the token space to the same 512-length embedding space.
• 5:34
  SPEAKER_01: Now you have image embeddings and text embeddings, and you have a large batch, let's say 1,000 images and 1,000 texts.
• 5:41
  SPEAKER_01: What you then do is you randomly kind of shuffle those around.
• 5:44
  SPEAKER_01: So you have randomly shuffled images and randomly shuffled text embeddings.
• 5:49
  SPEAKER_01: And then the learning objective for the neural network is to match the correct image with the correct text embedding.
• 5:57
  SPEAKER_01: If you do this for a long time and with lots of data, you will learn to align the embedding space, right?
• 6:03
  SPEAKER_01: So you'll learn to align the image embedding with kind of the corresponding text.
• 6:11
  SPEAKER_01: In and of itself, this is not like super intuitive and not like super applicable to downstream tasks.
• 6:18
  SPEAKER_01: But, and now I come to where people have actually interacted with this.
• 6:22
  SPEAKER_01: OpenAI has used this CLIP model as a pre-training step and then have used kind of the learned embeddings from CLIP to train image diffusion models.
• 6:35
  SPEAKER_01: And we are all familiar with those because these are the DALL-E family of models, right?
• 6:39
  SPEAKER_01: So DALL-E 2, for example.
• 6:42
  SPEAKER_01: is powered in large parts by this CLIP pre-training and by these CLIP embeddings.
• 6:48
  SPEAKER_01: And that makes perfect sense, right?
• 6:50
  SPEAKER_01: Because what DALI does is exactly the same.
• 6:53
  SPEAKER_01: You enter a text prompt, and it translates that into a synthesized image.
• 6:59
  SPEAKER_01: Okay, so far for CLIP.
• 7:02
  SPEAKER_01: Now, what is SatCLIP?
• 7:03
  SPEAKER_01: At SatCLIP, we kind of change, we have the same idea, but we change two things.
• 7:08
  SPEAKER_01: First, we go from natural RGB images to satellite images.
• 7:13
  SPEAKER_01: So these are
• 7:14
  SPEAKER_01: Multispectral 13-channel Sentinel-1 satellite images.
• 7:18
  SPEAKER_01: Sorry, Sentinel-2, which is one of the ESA missions, European Space Agency.
• 7:23
  SPEAKER_01: And we also don't have corresponding texts to go with these images.
• 7:27
  SPEAKER_01: We have corresponding locations, right?
• 7:29
  SPEAKER_01: So these are just simply the latitude-longitude coordinates of the image centroid.
• 7:35
  SPEAKER_01: on the globe, on the sphere of our planet.
• 7:39
  SPEAKER_01: But then we try to do the same thing.
• 7:40
  SPEAKER_01: We embed the images in an image embedding, we embed the locations in a location embedding, and we try to match the correct image with the correct location embedding.
• 7:52
  SPEAKER_01: Now, the idea here again is to translate between this kind of location and image space.
• 7:58
  SPEAKER_01: And I think an intuition for how the models go about this is kind of twofold.
• 8:07
  SPEAKER_01: First, for the image embedding, the image embedding will try to capture features within the image that make it very unique to a specific location.
• 8:20
  SPEAKER_01: Satellite image deep learning featuring Robin Cole titled Location Embedding with SatCLIP, with Konstantin Klemmer
• 8:39
  SPEAKER_01: You will then have to kind of identify or guess where on the earth you are.
• 8:43
  SPEAKER_00: You could be in Paris and you'd say, oh, I see the Eiffel Tower.
• 8:47
  SPEAKER_00: That would be easy.
• 8:47
  SPEAKER_00: Exactly.
• 8:48
  SPEAKER_01: That would be very easy.
• 8:49
  SPEAKER_01: Or you see like a street sign in a specific language and you're like, oh yeah, this is, I don't know, Japanese letters.
• 8:55
  SPEAKER_01: I'm probably in Japan.
• 8:58
  SPEAKER_01: So the image encoder will kind of try to identify features that are unique to a given location.
• 9:05
  SPEAKER_01: And now think about this the other way around from a location's perspective.
• 9:09
  SPEAKER_01: If you're given like a specific location on the map of the world, let's say like just a pin that is dropped, let's say in Sahara in Africa, your immediate intuition for what you would expect at this location would probably be something like a desert.
• 9:24
  SPEAKER_01: Likewise, if you see a location in Greenland, you'll probably expect glaciers, mountains.
• 9:30
  SPEAKER_01: So we all have this kind of intuition in our mind.
• 9:33
  SPEAKER_01: And so we're trying to do the same with a neural network model.
• 9:36
  SPEAKER_01: We try to teach a neural network model for a given location, what kind of image or image features would you expect there?
• 9:44
  SPEAKER_01: And for a given image, which kind of location would you expect this image to come from?
• 9:48
  SPEAKER_00: Right.
• 9:48
  SPEAKER_00: So you need a very large dataset covering pretty much the entire landmass, do you?
• 9:53
  SPEAKER_01: Yes.
• 9:54
  SPEAKER_01: And so that gets us to the kind of dataset creation.
• 9:58
  SPEAKER_01: And this is exactly right.
• 10:00
  SPEAKER_01: The success of SatCLIP and doing this kind of pre-training very much depends on the design of the dataset.
• 10:07
  SPEAKER_01: And we sample 100,000
• 10:10
  SPEAKER_01: Image scenes from Sentinel-2 approximately uniformly randomly distributed over landmass of planet Earth.
• 10:19
  SPEAKER_01: So this includes all continents, all landmass.
• 10:25
  SPEAKER_01: Also, we have this kind of restriction that there should be no cloud coverage because we use Sentinel-2, so that can see through clouds, unlike radar, which can be a bit tricky in, for example, tropical areas where there's lots of cloud coverage.
• 10:37
  SPEAKER_01: So finding cloud-free imagery there is a bit harder.
• 10:42
  SPEAKER_01: But overall, I think we succeed with our training data set, and it looks quite nicely distributed over the map.
• 10:48
  SPEAKER_00: And are these single season images or do you have to sample across seasons so it understands, well, this is a tropical forest or this is a temperate forest?
• 10:56
  SPEAKER_01: That's a great question.
• 10:58
  SPEAKER_01: So right now, the kind of location encoder we use really just takes in latitude-longitude coordinates.
• 11:03
  SPEAKER_01: We don't use the time information that we would also have for our satellite image.
• 11:08
  SPEAKER_01: We don't use that at all.
• 11:10
  SPEAKER_01: What we do, though, is we sample these sentinel scenes over two years.
• 11:15
  SPEAKER_01: So all seasonalities across the globe are technically included in our
• 11:22
  SPEAKER_01: Satellite image deep learning featuring Robin Cole titled Location Embedding with SatCLIP, with Konstantin Klemmer
• 11:40
  SPEAKER_01: Now, the same observation has approximately the same likelihood to have been captured in July versus December, which will look vastly different on a satellite image because in July we might have much less snow coverage or no snow coverage, whereas in December we will most certainly have that.
• 11:58
  SPEAKER_01: And so the current form of SatCLIP can be seen as kind of averaging over seasons.
• 12:06
  SPEAKER_00: And that obviously requires a particularly large dataset for the averaging to work out across four different seasons.
• 12:13
  SPEAKER_01: Yes, again, it's 100,000 scenes across the globe.
• 12:17
  SPEAKER_01: I wouldn't call it a massive dataset yet.
• 12:21
  SPEAKER_01: And I would call SatCLIP a prototype more than anything.
• 12:25
  SPEAKER_01: I think it's a very useful prototype, as we show in the kind of experiments we do in our paper.
• 12:29
  SPEAKER_01: But still, I think
• 12:31
  SPEAKER_01: There's clearly benefits from just making that dataset much bigger and integrating the timestamp directly into them.
• 12:38
  SPEAKER_00: Yeah.
• 12:39
  SPEAKER_00: And just a question on the dataset you chose Sentinel to because of the wide coverage and the ease of access, I suppose.
• 12:46
  SPEAKER_01: Yes, so there's two reasons we chose Sentinel-2.
• 12:49
  SPEAKER_01: First, it is globally available for any location on the planet.
• 12:53
  SPEAKER_01: And again, with this kind of caveat of cloud coverage, it's also equal quality everywhere.
• 12:59
  SPEAKER_01: So that is reason one.
• 13:01
  SPEAKER_01: And then reason two is a bit more, I guess, up for discussion and like almost a philosophical question.
• 13:09
  SPEAKER_01: And that is the question, what describes a location best, right?
• 13:13
  SPEAKER_01: What kind of data
• 13:15
  SPEAKER_01: Would anyone use to best describe a given location?
• 13:19
  SPEAKER_01: And we believe that Sentinel-to-satellite imagery or satellite imagery in general is a very good descriptor of a location, right?
• 13:27
  SPEAKER_01: Because you'll be able to see the kind of ecological factors at a location.
• 13:31
  SPEAKER_01: Is there lots of vegetation?
• 13:33
  SPEAKER_01: Does it look like a hot area, right?
• 13:36
  SPEAKER_01: Or does it look like a very dry or a very wet area?
• 13:40
  SPEAKER_01: Satellite image deep learning featuring Robin Cole titled Location Embedding with SatCLIP, with Konstantin Klemmer Satellite image deep learning featuring Robin Cole titled Location Embedding with SatCLIP, with Konstantin Klemmer Satellite image deep learning featuring Robin Cole titled Location Embedding with SatCLIP, with Konstantin Klemmer
• 14:03
  SPEAKER_01: Satellite image deep learning featuring Robin Cole titled Location Embedding with SatCLIP, with Konstantin Klemmer
• 14:24
  SPEAKER_01: And it's also not the case that you have to choose one data set, right?
• 14:29
  SPEAKER_01: I think future iterations of these kind of models will most certainly be multi-model and combine different data sources.
• 14:36
  SPEAKER_00: Right.
• 14:37
  SPEAKER_00: Fascinating.
• 14:38
  SPEAKER_00: So once you've trained the model, you then can create these basically representations of any location on Earth.
• 14:46
  SPEAKER_00: You just input a lat and long, is that right?
• 14:48
  SPEAKER_01: That's right.
• 14:48
  SPEAKER_01: So the location encoder of the pre-trained SatCLIP model takes in raw lat-long coordinates on the whole planet, and it will return embedding that is 265 characters long in our case that will describe that location.
• 15:06
  SPEAKER_01: Now, I say we can generate that for the whole planet, which is true, but it's not necessarily meaningful for the whole planet.
• 15:14
  SPEAKER_01: And it is especially not meaningful for the oceans.
• 15:16
  SPEAKER_01: We don't have any satellite imagery over oceans.
• 15:19
  SPEAKER_01: We only focus on landmass.
• 15:22
  SPEAKER_01: Also, it would be very hard to tell apart, like to localize to satellite images of an ocean.
• 15:27
  SPEAKER_01: It's not very information rich.
• 15:31
  SPEAKER_01: And so while we can generate embeddings over oceans, these are meaningless.
• 15:36
  SPEAKER_01: They're basically just default back to the kind of random initialization of the neural network.
• 15:42
  SPEAKER_00: Out of curiosity, have you tried visualizing these embeddings to see if they correlate?
• 15:46
  SPEAKER_01: Yes, especially in the appendix of our paper.
• 15:50
  SPEAKER_01: Let me quickly look which figure in the appendix it is.
• 15:54
  SPEAKER_01: But we have a figure like that in the appendix of our paper.
• 15:57
  SPEAKER_01: And I think it also shows kind of both the meaningful embeddings over landmass and the embeddings that are over oceans.
• 16:07
  SPEAKER_00: Okay.
• 16:09
  SPEAKER_00: Fantastic.
• 16:12
  SPEAKER_00: So essentially, given any geospatial dataset with lat-long as one of its features, we can then add these embeddings as additional features and then train for downstream tasks.
• 16:24
  SPEAKER_00: Is that right?
• 16:25
  SPEAKER_01: Yeah, that is right.
• 16:27
  SPEAKER_01: So the idea here is the following.
• 16:30
  SPEAKER_01: The embeddings can basically replace image features.
• 16:35
  SPEAKER_01: That is our intuition.
• 16:37
  SPEAKER_01: Now, if you already have a downstream task, like let's say, I don't know,
• 16:41
  SPEAKER_01: population density estimation from satellite imagery.
• 16:44
  SPEAKER_01: If you already have the satellite images or already have a pre-trained model that can analyze them, SatCLIP embeddings will most likely not be better.
• 16:55
  SPEAKER_01: However, in cases where you're resource constrained, you don't have the capacity to download large chunks of satellite data or to train these very intense vision transformers or CNN models that will then extract features from these raw satellite images.
• 17:12
  SPEAKER_01: In this kind of resource-sparse environment, SatCLIP embeddings are particularly useful.
• 17:19
  SPEAKER_01: This kind of line of work already exists and I want to also shout out here work of my co-author Esther who has kind of worked on this in the past and they've developed the mosaics model which tries to do something similar.
• 17:35
  SPEAKER_01: They basically also extract features from satellite images without you actually having to download any satellite imagery.
• 17:44
  SPEAKER_00: Right.
• 17:45
  SPEAKER_00: And at high level, what's the main difference between SatCLIP and mosaics?
• 17:50
  SPEAKER_01: All right.
• 17:51
  SPEAKER_01: So mosaics, I think, at least in my intuition, can be best seen as a database.
• 17:58
  SPEAKER_01: So the kind of input is the same.
• 18:01
  SPEAKER_01: You only put in a latitude-longitude coordinate, and you will give in return, I think, feature vector of length 4,000 or something like that.
• 18:10
  SPEAKER_01: What Mosaics does is for any given input location, it looks up the nearest available satellite image and then it extracts random convolutional features.
• 18:21
  SPEAKER_01: So these are random features from an untrained CNN.
• 18:24
  SPEAKER_01: So there's no kind of pre-training happening, which is one of the key differences to SatCLIP.
• 18:30
  SPEAKER_01: And as I said, the Mosaics embedding is, I think, 4,000 characters long, at least the ones we used.
• 18:36
  SPEAKER_01: And so it's kind of much bigger than SatCLIP embeddings.
• 18:41
  SPEAKER_01: Okay, and do you think there's any sort of compromise there if you have a smaller embedding that you have less rich information in it or it's not really... So there is... I don't think there's a compromise or like we didn't investigate that much for the kind of different size of the embeddings but there is a key difference between Mosaics and SatCLIP and that is
• 19:10
  SPEAKER_01: SatCLIP is basically, since it's trained on 100,000 images, if the location that is plugged into a pre-trained SatCLIP location encoder is not one that is available in the training set, which obviously most locations aren't if we only choose 100,000, the SatCLIP basically returns location embeddings that are interpolations of the image features of the nearest locations available.
• 19:36
  SPEAKER_01: Now,
• 19:37
  SPEAKER_01: Satellite image deep learning featuring Robin Cole titled Location Embedding with SatCLIP, with Konstantin Klemmer
• 19:51
  SPEAKER_01: and it will not do any kind of smoothing.
• 19:54
  SPEAKER_01: SatCLIP, it does interpolate over space and so we can also see that in the visualizations in our paper, kind of creates a somewhat smooth surface.
• 20:02
  SPEAKER_01: Whereas mosaics are very, very fine-grained features that are very different for a given location.
• 20:08
  SPEAKER_01: And so,
• 20:09
  SPEAKER_01: What that means for applications is that if your applications are on a more larger global scale, what we find in our paper is that we outperform mosaics quite handsomely.
• 20:22
  SPEAKER_01: If the kind of task is much more restricted to like a specific area, it might look differently and mosaics might be the better choice.
• 20:30
  SPEAKER_00: And presumably you could also combine them both.
• 20:33
  SPEAKER_01: Yes, so we did try that a little bit.
• 20:36
  SPEAKER_01: We didn't go super much into detail because our first experiments just showed that this didn't work at all.
• 20:42
  SPEAKER_01: But yeah, so table five in the appendix of a paper does kind of a little investigation where we combine different embeddings that exist and see if they together improve performance, which we don't find.
• 20:55
  SPEAKER_01: But then again, we didn't investigate this very thoroughly.
• 20:58
  SPEAKER_00: Yeah, fantastic.
• 21:00
  SPEAKER_00: And what's the future then?
• 21:03
  SPEAKER_00: I mean, you've described two different approaches to creating embeddings, pre-training, not pre-training, obviously Sentinel-2 at the moment.
• 21:14
  SPEAKER_00: What's the future then?
• 21:15
  SPEAKER_00: Different modalities, different techniques, do you think?
• 21:19
  SPEAKER_01: Yeah, so the future is, I think, wide open, which makes this, I think, a very exciting research field.
• 21:28
  SPEAKER_01: So, first of all, I think the
• 21:34
  SPEAKER_01: The way we do these kind of location encoders can be adapted, right?
• 21:37
  SPEAKER_01: So I just said that we just use latitude, longitude as inputs.
• 21:41
  SPEAKER_01: I think very intuitively, the next step would be to also include the timestamp.
• 21:46
  SPEAKER_01: So you can kind of query location embedding at a given location and point in time.
• 21:52
  SPEAKER_01: This is, I think, the next clear step, but also kind of changing the architectures of the location encoders, making them bigger, allowing them to basically integrate more location knowledge.
• 22:06
  SPEAKER_01: These are clear next steps.
• 22:07
  SPEAKER_01: And then, like you mentioned, the training data is a
• 22:12
  SPEAKER_01: The other, I think, big open question.
• 22:14
  SPEAKER_01: We use Sentinel 2.
• 22:16
  SPEAKER_01: We think it's great, but I think multimodal is also the way to go here.
• 22:22
  SPEAKER_01: And if you look at kind of the emerging foundation models in the geospace, we see more and more multimodal models.
• 22:31
  SPEAKER_01: So I think
• 22:31
  SPEAKER_01: The Clay Foundation model that was launched only a few weeks ago is a great example.
• 22:35
  SPEAKER_01: And they stack Sentinel-2, Sentinel-1, and the digital elevation model all together in like one data cube.
• 22:44
  SPEAKER_01: Something like that, I think is very, very clear next step.
• 22:48
  SPEAKER_00: Well, very exciting time, lots of potential, even just for the applications that then don't need to access imagery but can just query a model.
• 22:56
  SPEAKER_00: That's a really exciting step change in functionality.
• 23:02
  SPEAKER_00: Final, I'm just curious how this model fits into the context of what the lab is doing in general and also Microsoft?
• 23:10
  SPEAKER_01: Right, so I'm a postdoc at Microsoft, which means I have like ultimate research flexibility and independence.
• 23:17
  SPEAKER_01: So this does not necessarily fit in super well within my lab, though I will say there is one of my co-authors, Lester Mackey at MSR New England, is very much looking in different
• 23:32
  SPEAKER_01: Earth science projects as well.
• 23:35
  SPEAKER_01: He's a project on sub-seasonal forecasting that is somewhat related to this.
• 23:41
  SPEAKER_01: I think also the kind of interesting statistical properties of geospatial data are something that is very interesting to the kind of more statistical machine learning side of Microsoft.
• 23:53
  SPEAKER_01: Yeah, I guess the keyword here is just a kind of autocorrelation that is very apparent in all sorts of geospatial data and how to best use that in neural network models.
• 24:04
  SPEAKER_01: And the kind of the CLIP objective that we use very much makes use of that, right?
• 24:08
  SPEAKER_01: So the intuition is, you know, nearby satellite images will look quite similar, whereas far away ones will look very different.
• 24:15
  SPEAKER_00: Yeah.
• 24:16
  SPEAKER_01: Yeah, but then I will say I think the SatCLIP work is very much of interest to other parts of the organization and I think especially also to the machine learning, sorry, AF for Good lab in Redmond where Caleb Robinson is from.
• 24:33
  SPEAKER_01: They're doing a lot of work on remote sensing and geospatial machine learning and this just falls right in there.
• 24:39
  SPEAKER_00: Fantastic.
• 24:40
  SPEAKER_00: And of course, the paper's on Archive, the code is on GitHub, so check them out.
• 24:44
  SPEAKER_00: If people want to follow along, the updates, LinkedIn, is that the right place for people to follow you?
• 24:49
  SPEAKER_01: Yeah, LinkedIn or Twitter or Excel, it's now called, I think, places where I will post updates.
• 24:56
  SPEAKER_01: And my website, I have a website too.
• 24:58
  SPEAKER_00: Okay, fantastic.
• 24:59
  SPEAKER_00: Well, once again, thanks for joining me today.
• 25:00
  SPEAKER_00: Fascinating conversation.
• 25:01
  SPEAKER_00: I look forward to hearing future updates.
• 25:04
  SPEAKER_01: Thank you so much.
• 25:05
  SPEAKER_01: Thanks for having me and hope people enjoy this.