Latest / Satellite image deep learning / Planetixx competition approach

Transcript

Planetixx competition approach · Satellite image deep learning

• 0:05
  Hello,
• 0:06
  I'm Robin Cole,
• 0:06
  and welcome to the Satellite Image Deep Learning Podcast.
• 0:10
  SPEAKER_01: In this episode,
• 0:10
  SPEAKER_01: I caught up with James Doherty and Donald Hill,
• 0:13
  SPEAKER_01: co-founders of Plastic Eye,
• 0:15
  a company using satellite imagery and AI to tackle ocean
• 0:18
  plastic pollution.
• 0:20
  Their platform not only detects plastic,
• 0:22
  but also predicts its origin and trajectory.
• 0:25
  SPEAKER_01: enabling more effective interventions.
• 0:27
  Beyond Plastic,
• 0:28
  they've expanded into monitoring algae blooms,
• 0:31
  a growing environmental concern.
• 0:33
  We discussed the technical and practical challenges of building
• 0:36
  AI models that work
• 0:37
  SPEAKER_01: at a global scale,
• 0:38
  SPEAKER_01: as well as their newly launched platform.
• 0:41
  We also recorded a live demo of the platform,
• 0:43
  which is available as a separate video.
• 0:46
  SPEAKER_01: If you're watching this on YouTube,
• 0:47
  SPEAKER_01: please consider liking the video and subscribing to the channel,
• 0:50
  SPEAKER_01: as this really helps to reach a wider audience.
• 0:52
  SPEAKER_01: Thank you.
• 0:53
  SPEAKER_01: Hi, James.
• 0:54
  SPEAKER_01: Hi, Donald.
• 0:54
  SPEAKER_01: Welcome to the podcast.
• 0:57
  SPEAKER_01: Hi, Robin.
• 0:58
  SPEAKER_02: Hi, Robin.
• 0:59
  SPEAKER_01: Wonderful.
• 1:00
  SPEAKER_02: Good to be here.
• 1:01
  SPEAKER_02: James and I are already talking over each other.
• 1:05
  SPEAKER_01: Three ways has its own dynamic, but we'll master it.
• 1:08
  SPEAKER_01: James, let's begin with you.
• 1:10
  SPEAKER_01: Where do you work and what do you do?
• 1:13
  Yeah,
• 1:13
  so we are the co-founding team of Plastic Eye along with our
• 1:18
  third co-founder,
• 1:19
  Jack.
• 1:21
  And we're a UK-based company and we monitor the health of
• 1:26
  oceans and freshwater
• 1:27
  SPEAKER_00: bodies using satellites and AI.
• 1:31
  So we have been around for about three years as a company and
• 1:36
  had a pretty kind of
• 1:37
  SPEAKER_00: fun experience.
• 1:38
  journey during that period and i'm sure we'll uh we'll dive
• 1:41
  into some of the those
• 1:42
  interesting points of discussion absolutely i believe that you
• 1:46
  you started or you
• 1:47
  SPEAKER_01: met at a competition is that right
• 1:50
  SPEAKER_00: Yeah, we did.
• 1:51
  Well,
• 1:51
  actually,
• 1:52
  we met originally in Charlie O'Neill's Irish bar in Geneva.
• 1:58
  SPEAKER_00: So I had just handed in my resignation at a law firm in London,
• 2:03
  SPEAKER_00: having been practicing law for four years,
• 2:06
  SPEAKER_00: and somehow managed to get myself an internship at CERN.
• 2:11
  So I flew to Geneva,
• 2:12
  dropped my bags and my accommodation,
• 2:14
  had a knock on the door,
• 2:16
  And my old friend from university called Cian was there and he
• 2:19
  said a single word,
• 2:20
  SPEAKER_00: which was pub.
• 2:22
  SPEAKER_00: So one hour later,
• 2:23
  SPEAKER_00: I was sitting in a pub,
• 2:25
  which with a guitar in my hands and a nice cool glass of Magnus
• 2:30
  or something which
• 2:32
  SPEAKER_00: we'd paid extortionate money for.
• 2:33
  SPEAKER_00: And about four hours later,
• 2:35
  SPEAKER_00: doing as usual,
• 2:36
  SPEAKER_00: turning up late in the day,
• 2:39
  SPEAKER_00: this tall violinist walked in who was excellent.
• 2:44
  SPEAKER_00: And,
• 2:45
  SPEAKER_00: yeah,
• 2:45
  SPEAKER_00: Donald and I kind of realised we had some mutual,
• 2:48
  SPEAKER_00: you know,
• 2:49
  SPEAKER_00: mutual kind of origins,
• 2:50
  SPEAKER_00: both coming from Northern Ireland.
• 2:52
  Obviously,
• 2:52
  there was the particle physics bit on the folk music bit,
• 2:55
  which we seemed to enjoy.
• 2:56
  SPEAKER_00: So we hit it off in a big way.
• 2:57
  SPEAKER_00: And then it was probably another five years later where we...
• 3:04
  SPEAKER_00: Yeah, started collaboration on Plastic Eye.
• 3:07
  SPEAKER_00: And this, as you say, was through a competition called Hack the Planet.
• 3:12
  It was focused on the use of remote sensing for promoting the
• 3:16
  health of the oceans.
• 3:18
  SPEAKER_00: We had this idea around...
• 3:20
  producing a tool and a kind of interactive tool showing plastic
• 3:24
  pollution in the
• 3:24
  SPEAKER_00: oceans on a global scale,
• 3:27
  SPEAKER_00: supporting cleanup efforts and supporting policy decisions,
• 3:29
  SPEAKER_00: especially as the global treaty on plastics is soon to come into force,
• 3:34
  SPEAKER_00: all being well.
• 3:36
  So yeah,
• 3:36
  that was the origins of the company and we've taken it from there.
• 3:40
  SPEAKER_01: Right.
• 3:41
  SPEAKER_01: And in that competition, was it a technical task?
• 3:43
  SPEAKER_01: Was it more business oriented?
• 3:45
  SPEAKER_01: How did you participate?
• 3:48
  SPEAKER_00: It was an ideas competition more than anything.
• 3:50
  SPEAKER_00: So it ran over six months.
• 3:53
  SPEAKER_00: There was what they called a knowledge exchange program,
• 3:56
  SPEAKER_00: which was super useful for me,
• 3:58
  SPEAKER_00: actually.
• 3:58
  And this was run by the Commonwealth Secretariat and the
• 4:02
  Satellite Applications Catapult.
• 4:04
  And it was really a big piece of education on the ocean and
• 4:09
  challenges around the ocean.
• 4:11
  but also really useful kind of interaction between the cohorts
• 4:14
  on that as well.
• 4:16
  SPEAKER_00: So the idea that we went in with originally, yes, changed quite a lot.
• 4:21
  SPEAKER_00: It was always kind of marine plastics focused,
• 4:23
  SPEAKER_00: but coming out the other side,
• 4:24
  we refined that a lot into what was kind of a more realistic
• 4:29
  use of kind of
• 4:30
  SPEAKER_00: satellite data and being able to
• 4:33
  know really try and make some immediate impact with that um and
• 4:37
  it was it was good
• 4:38
  timing really uh there was just you know in the year or two
• 4:41
  beforehand you know the
• 4:43
  first academic publications on on papers of of using remote
• 4:48
  sensing for for
• 4:49
  monitoring plastic pollution so really it was a play on
• 4:53
  productionizing some of
• 4:54
  SPEAKER_00: that evolving it turning it into a practical tool
• 4:57
  SPEAKER_01: So the technical solution was kind of understood,
• 5:00
  but it's about productionising that and building a sort of
• 5:03
  scalable business.
• 5:04
  SPEAKER_01: Is that right?
• 5:06
  SPEAKER_00: To an extent, yeah.
• 5:07
  SPEAKER_00: I mean,
• 5:07
  SPEAKER_00: there's certainly been a lot of evolution on the technical front as well,
• 5:10
  SPEAKER_00: which I'm sure Donal will attest to.
• 5:14
  SPEAKER_00: But yeah,
• 5:15
  SPEAKER_00: then putting a business case around that,
• 5:17
  it's one of these ones where the business case isn't
• 5:21
  necessarily obvious as to
• 5:24
  SPEAKER_00: whose responsibility is it to...
• 5:26
  SPEAKER_00: you know,
• 5:27
  clean up plastic from the ocean whose responsibility that it's
• 5:29
  there in the first place.
• 5:30
  And,
• 5:30
  you know,
• 5:31
  there's often challenges with with attribution chains,
• 5:34
  etc.
• 5:36
  SPEAKER_00: So so thinking around the business cases for that and certainly we had
• 5:41
  of existential ponderings over whether this you know was going
• 5:45
  to be a non-profit
• 5:46
  or a for-profit etc but but certainly we felt in the longer
• 5:49
  term to have a
• 5:49
  self-sustaining business that can really have a kind of ongoing
• 5:53
  impact um you know
• 5:54
  we've we've gone down the the for-profit route and and um we
• 5:58
  still exist three
• 5:59
  years later so it's uh you know it's it's going well enough i'd
• 6:02
  say absolutely and
• 6:04
  SPEAKER_01: you've had some recent successes and projects
• 6:09
  SPEAKER_00: Yeah, absolutely.
• 6:10
  SPEAKER_00: So,
• 6:11
  SPEAKER_00: I mean,
• 6:12
  we've remained the development around technology on the
• 6:18
  plastics front has remained
• 6:20
  SPEAKER_00: central to what we do.
• 6:20
  SPEAKER_00: So they are quite an exciting milestone.
• 6:25
  SPEAKER_00: Upcoming is the release of the Plastic Eye Observatory.
• 6:29
  SPEAKER_00: So that is this tool,
• 6:31
  SPEAKER_00: essentially,
• 6:32
  SPEAKER_00: you know,
• 6:33
  SPEAKER_00: the idea has crystallized and we've developed a product around it.
• 6:36
  SPEAKER_00: So this will be released over the course of the next month.
• 6:41
  And it's a fully integrated tool,
• 6:44
  drawing down satellite data,
• 6:45
  conducting machine testing.
• 6:48
  learning analysis on that and then pushing the insights that we
• 6:52
  gain to a front-end dashboard.
• 6:55
  SPEAKER_00: It's got lots of heat maps,
• 6:57
  SPEAKER_00: lots of hindcasting and forecasting and all sorts of bells and whistles.
• 7:02
  SPEAKER_00: So, you know, we're hoping that's going to be a really practical tool.
• 7:07
  But yeah,
• 7:07
  as you referred to,
• 7:08
  we've kind of diversified the technology somewhat as well.
• 7:12
  SPEAKER_00: So one of the...
• 7:15
  SPEAKER_00: Yeah,
• 7:15
  the kind of higher profile projects we're working on at the
• 7:17
  moment is using
• 7:19
  satellite and drone technology for monitoring and forecasting
• 7:24
  the formation of
• 7:25
  SPEAKER_00: algal blooms on Loch Ney.
• 7:28
  SPEAKER_00: So it's a bit of a different signal,
• 7:30
  SPEAKER_00: but yeah,
• 7:31
  it's a really exciting project and something we're really
• 7:34
  enjoying getting our
• 7:35
  SPEAKER_00: teeth stuck into at the moment.
• 7:36
  SPEAKER_01: Congratulations on the successes.
• 7:38
  SPEAKER_01: And so the platform will be going probably March 2025.
• 7:42
  SPEAKER_01: That's when we're recording this episode.
• 7:44
  SPEAKER_01: Excited to see that.
• 7:45
  SPEAKER_01: I think mentioning the algae blooms is maybe a good place to start.
• 7:49
  How do you even detect something as small as a piece of plastic
• 7:51
  or algae on the
• 7:53
  SPEAKER_01: ocean from space?
• 7:55
  SPEAKER_02: Indeed.
• 7:56
  SPEAKER_02: So maybe this is where I can chip in on the tack front a little bit.
• 7:59
  SPEAKER_02: So a lot of scientific literature...
• 8:02
  already existed on the plastics piece and algal blooms as well
• 8:06
  when James and I
• 8:07
  SPEAKER_02: were spinning the company up.
• 8:08
  SPEAKER_02: So it was really early on, it was an exercise in understanding
• 8:12
  SPEAKER_02: how this works from first principles.
• 8:14
  SPEAKER_02: And generally,
• 8:16
  it's relying on the spectral signature of floating debris or
• 8:21
  algae relative to the
• 8:23
  SPEAKER_02: surrounding water.
• 8:24
  And one of the benefits of satellite-based remote sensing is
• 8:28
  that it doesn't just
• 8:29
  SPEAKER_02: see in the RGB,
• 8:31
  SPEAKER_02: the red,
• 8:31
  SPEAKER_02: green,
• 8:31
  SPEAKER_02: blue that our eyes see in,
• 8:33
  SPEAKER_02: but we have access to a lot of other colors in the spectrum as well.
• 8:38
  And it turns out that one particular wavelength in the
• 8:40
  near-infrared band in
• 8:42
  SPEAKER_02: Sentinel-2 data,
• 8:43
  SPEAKER_02: so this is an ESA open-source data set,
• 8:46
  that offers you extremely high contrast between water and
• 8:51
  floating material.
• 8:52
  The main reason is that water absorbs very strongly in the
• 8:56
  near-infrared band,
• 8:57
  SPEAKER_02: so the water looks really,
• 8:58
  SPEAKER_02: really dark.
• 8:59
  SPEAKER_02: in the imagery, but the floating material reflects strongly.
• 9:04
  SPEAKER_02: So that kind of contrast,
• 9:05
  spectral contrast between the object of interest and the
• 9:09
  surrounding water is what
• 9:10
  SPEAKER_02: makes this kind of approach work.
• 9:12
  SPEAKER_02: And it's a very similar premise for algae,
• 9:15
  albeit the spectral bands where you find the contrast tend to
• 9:19
  differ a little bit.
• 9:21
  SPEAKER_02: So from a science perspective,
• 9:22
  you're dealing with differences in spectral response of the
• 9:25
  reflected light.
• 9:27
  SPEAKER_02: The Sentinel is a passive sensor.
• 9:29
  SPEAKER_02: You're actually looking at the light from the sun hitting off the water,
• 9:33
  SPEAKER_02: and you're looking at the reflected sunlight.
• 9:35
  There are other types of satellite where you can shine light
• 9:39
  onto the surface and
• 9:40
  SPEAKER_02: look for what's reflected back.
• 9:42
  SPEAKER_02: So for example, synthetic aperture radar.
• 9:45
  SPEAKER_02: And you can also make use of that type of data,
• 9:47
  SPEAKER_02: look at interference signals,
• 9:49
  SPEAKER_02: and that can help you understand the surface profile of the water.
• 9:54
  So the way we approached this was we weren't going to reinvent
• 9:56
  the science.
• 9:57
  SPEAKER_02: As James mentioned earlier,
• 9:58
  it was really about integrating the feature engineering that
• 10:01
  already existed,
• 10:03
  SPEAKER_02: looking at ratios of frequencies,
• 10:05
  and trying to build that into a machine learning pipeline on
• 10:09
  the cloud that could
• 10:10
  SPEAKER_02: do inference quickly and at scale.
• 10:14
  SPEAKER_02: because oceans are huge.
• 10:15
  SPEAKER_02: And,
• 10:16
  SPEAKER_02: you know,
• 10:16
  SPEAKER_02: if you want to service a large enough customer base,
• 10:19
  SPEAKER_02: you need to be able to do,
• 10:21
  SPEAKER_02: you know,
• 10:21
  SPEAKER_02: parallelization and all of the compute side of things efficiently.
• 10:24
  SPEAKER_02: And this is where Jack,
• 10:26
  SPEAKER_02: our other co-founder,
• 10:27
  SPEAKER_02: has really,
• 10:27
  SPEAKER_02: you know,
• 10:28
  SPEAKER_02: driven things forward tremendously.
• 10:29
  So just to give a shout out to Jack for all of his great work
• 10:33
  on the productionizing side.
• 10:34
  SPEAKER_02: So,
• 10:35
  SPEAKER_02: This is how it works in practice.
• 10:36
  And then when you're trying to do prediction and inference and
• 10:39
  detection of objects
• 10:41
  SPEAKER_02: at scale,
• 10:42
  SPEAKER_02: the first thing we actually tried was pixel-level analysis.
• 10:45
  You tend to start simple with these use cases and look at
• 10:49
  whether per pixel you can
• 10:52
  use labeled examples of floating plastic to help you assign a
• 10:56
  probability of a
• 10:57
  SPEAKER_02: given pixel of water containing plastic.
• 11:00
  SPEAKER_02: So we were looking at nothing more complex than a decision tree.
• 11:05
  um as our first as our first product but the the achilles heel
• 11:09
  there is that you're
• 11:10
  not using the correlation structure in the imagery properly as
• 11:14
  you know that each
• 11:15
  each pixel of water is not independent from its neighbors so a
• 11:19
  very natural next
• 11:20
  step for the attack evolution was to look at you know
• 11:23
  convolutional neural networks
• 11:25
  SPEAKER_02: and really exploiting both the spectral
• 11:28
  SPEAKER_02: properties of the satellite data, but also the rich spatial detail.
• 11:33
  SPEAKER_02: And there's already been a lot of work on terrestrial use of confnats.
• 11:39
  So there are even off the shelf models that you can use to
• 11:42
  segment fields and
• 11:43
  SPEAKER_02: buildings and rivers and stuff.
• 11:45
  But the ocean domain is a lot less thoroughly explored in terms
• 11:50
  of being able to
• 11:50
  SPEAKER_02: use deep learning because the amount of labeled data is quite low.
• 11:55
  so we've started off like our attack to this day remains
• 11:59
  reasonably simple in terms
• 12:00
  of architecture we've used units trained on on a few thousand
• 12:05
  labeled examples
• 12:06
  where we have to be you know really strict on i'm not
• 12:09
  overfitting it and um you
• 12:11
  know you get paranoid about uh false positives as well because
• 12:15
  it's it's important
• 12:16
  SPEAKER_02: that we don't just color in
• 12:18
  wave caps and trails made by boats and bits of bridges you know
• 12:22
  there's all sorts
• 12:22
  of stuff that can fake your your signal so we focus mostly on
• 12:26
  simple architectures
• 12:27
  that you know we can control the the false positive rate of and
• 12:32
  that's that's where
• 12:32
  a lot of the focus has been so simple model and the complexity
• 12:35
  is more on the sort
• 12:36
  of the data side of things covering all these edge cases having
• 12:40
  something that is
• 12:41
  SPEAKER_01: distributed across the whole world potentially
• 12:43
  SPEAKER_02: I think that's exactly it.
• 12:45
  SPEAKER_02: Yeah.
• 12:45
  SPEAKER_02: And a lot of work on iterative labeling as well.
• 12:48
  SPEAKER_02: So when we put a model into production,
• 12:51
  SPEAKER_02: it's coloring in things that we can then go back and check,
• 12:54
  SPEAKER_02: you know,
• 12:55
  SPEAKER_02: is this a verifiable signal of debris or is that actually,
• 12:59
  SPEAKER_02: you know,
• 12:59
  SPEAKER_02: a fake?
• 12:59
  And we can go and investigate cross-check Sentinel against
• 13:03
  higher resolution data
• 13:04
  SPEAKER_02: from planet and worldview,
• 13:07
  SPEAKER_02: and then add more labels of difficult examples into your training set.
• 13:11
  SPEAKER_02: And, you know, over time, this has become,
• 13:14
  As much an important piece of IP for the company as the models themselves,
• 13:18
  it's actually feeding it a lot of difficult examples to learn
• 13:22
  from and gradually
• 13:24
  SPEAKER_02: improve your error scores.
• 13:27
  SPEAKER_02: So it's been a patience game,
• 13:29
  SPEAKER_02: I think,
• 13:29
  SPEAKER_02: but the model that we will release in the next coming weeks,
• 13:34
  SPEAKER_02: I think,
• 13:34
  SPEAKER_02: is testament to all of that patient work over the last few years.
• 13:38
  SPEAKER_01: Yeah.
• 13:39
  It feels like almost a task that could continue forever,
• 13:42
  presumably with diminishing returns.
• 13:44
  SPEAKER_01: How do you decide, okay, this is done, we release?
• 13:48
  In my experience in data science and software,
• 13:50
  I'm not sure anything's ever really finished.
• 13:53
  I think at some point,
• 13:55
  a lot of engagement with early adopters will be important.
• 13:58
  So where we deploy the model in regions of interest where
• 14:02
  people are doing cleanup
• 14:03
  SPEAKER_02: operations and we can start to deliver them
• 14:07
  SPEAKER_02: you know, hindcasts of what's been seen in the past.
• 14:10
  SPEAKER_02: I also do a little bit of predictive modeling,
• 14:11
  SPEAKER_02: which I'll get into in a minute,
• 14:13
  but really that the acid test will be getting it into the hands
• 14:16
  of users and asking
• 14:18
  SPEAKER_02: for feedback and then integrating that feedback into,
• 14:21
  SPEAKER_02: you know,
• 14:21
  SPEAKER_02: an extra round of training.
• 14:23
  SPEAKER_02: And I think once we've,
• 14:24
  SPEAKER_02: once we've done that enough and we feel we're,
• 14:27
  SPEAKER_02: we're stable in control of the errors,
• 14:29
  SPEAKER_02: you know,
• 14:29
  SPEAKER_02: you could probably consider it a stable,
• 14:32
  SPEAKER_02: a stable product that we retrain,
• 14:34
  SPEAKER_02: you know,
• 14:34
  SPEAKER_02: if a scheduler maybe once a month.
• 14:36
  SPEAKER_03: Yeah.
• 14:38
  SPEAKER_02: extend it to multiple geographies.
• 14:39
  SPEAKER_02: So you're never really finished in the world of MLOps, I think.
• 14:45
  SPEAKER_02: Model drift is a thing, as I'm sure you're aware.
• 14:47
  SPEAKER_02: You need to continue to keep the model relevant and not just freeze it.
• 14:53
  Oceans are dynamic and there's seasonality and there's a lot of
• 14:57
  factors that you
• 14:59
  SPEAKER_02: need to keep under control.
• 15:00
  SPEAKER_01: Yeah.
• 15:01
  SPEAKER_01: One more question on that point.
• 15:03
  How do you keep track of all of these sort of features that
• 15:06
  you're bringing in?
• 15:07
  Do you have like a big table and you're like,
• 15:09
  we've done this now,
• 15:09
  we've done that now?
• 15:12
  SPEAKER_02: Yeah,
• 15:12
  SPEAKER_02: I mean,
• 15:13
  SPEAKER_02: at a boring level,
• 15:14
  SPEAKER_02: a lot of use of JIRA and GitLab and,
• 15:16
  SPEAKER_02: you know,
• 15:17
  SPEAKER_02: tickets and all of that kind of thing.
• 15:18
  SPEAKER_02: But also we've made very heavy use of tools like TensorBoard where,
• 15:22
  SPEAKER_02: you know,
• 15:22
  SPEAKER_02: you can keep track of specific model runs.
• 15:26
  SPEAKER_02: We built the models in PyTorch.
• 15:27
  SPEAKER_02: So, you know, when we run specific tools,
• 15:31
  SPEAKER_02: models at a given date,
• 15:33
  SPEAKER_02: we can keep track of the loss curve on all of the metrics,
• 15:37
  and also keep a log of what was changed in software relative to
• 15:41
  the previous run,
• 15:42
  and also keep track of whether any features were swapped out or
• 15:45
  did we redefine a
• 15:46
  SPEAKER_02: particular satellite index.
• 15:48
  SPEAKER_02: So TensorBoard,
• 15:49
  SPEAKER_02: there are other tools out there,
• 15:51
  but it helps you have almost a lab notebook of your science
• 15:57
  experiments that you've
• 15:58
  SPEAKER_02: done in the past.
• 16:00
  SPEAKER_02: We're flexible in terms of being able to revert.
• 16:03
  SPEAKER_02: If we do a model run that ends up making the performance worse,
• 16:07
  SPEAKER_02: we're not bound to use the latest run.
• 16:10
  SPEAKER_02: We can revert back to previous trained weights.
• 16:14
  SPEAKER_02: And TensorBoard's really helpful in that regard.
• 16:17
  So I think this is a super useful lens on what's going on with
• 16:22
  these deep learning models.
• 16:23
  SPEAKER_01: Absolutely.
• 16:24
  SPEAKER_01: So that's the detection side.
• 16:26
  SPEAKER_01: You mentioned prediction.
• 16:28
  Can you talk a bit about the challenges there and how you're
• 16:30
  tackling them?
• 16:31
  SPEAKER_02: Absolutely.
• 16:32
  SPEAKER_02: Absolutely.
• 16:32
  Is it worth providing a bit of background on that,
• 16:35
  though,
• 16:35
  just on the general project?
• 16:38
  SPEAKER_00: Yeah.
• 16:40
  SPEAKER_02: Yeah,
• 16:40
  I think I was going to explain the difference between detecting
• 16:45
  a bit of plastic on
• 16:46
  SPEAKER_02: a given date and actually making it a useful thing for a customer.
• 16:51
  So it's one thing to say,
• 16:52
  look,
• 16:53
  we've spotted a slick of debris last Tuesday.
• 16:57
  But by the time that that information gets into the hands of
• 17:00
  the customer,
• 17:01
  SPEAKER_02: it's maybe too late to intercept it.
• 17:03
  SPEAKER_02: The debris may have dispersed.
• 17:05
  SPEAKER_02: might have already washed up on the shore.
• 17:06
  So one thing we identified early was perhaps it would be useful
• 17:10
  to take ocean
• 17:12
  SPEAKER_02: current and wind data and given an initial detection location,
• 17:17
  try to do some sort of forward and back propagation of the
• 17:21
  likely future location.
• 17:22
  SPEAKER_02: So a bit of a weather forecast where you have
• 17:25
  SPEAKER_02: confidence intervals that you can propagate.
• 17:28
  SPEAKER_02: So as a hindcast tool,
• 17:30
  that's useful because you can backtrack where the debris likely
• 17:34
  originated from,
• 17:35
  SPEAKER_02: which is very useful if you're trying to identify hotspots where,
• 17:40
  SPEAKER_02: for example,
• 17:40
  SPEAKER_02: river mouths that are close to dump sites might be leaking material.
• 17:44
  SPEAKER_02: So if you can sort of backtrack to the likely origin spot,
• 17:48
  That's really useful for customers when they're thinking about
• 17:51
  where to target
• 17:51
  SPEAKER_02: their cleanup interventions.
• 17:53
  But the other use case is if you have active deployment of
• 17:58
  recovery vehicles,
• 17:59
  SPEAKER_02: so boats that would intercept the material and clean it up,
• 18:03
  then they need to know where is that debris likely to be in 24
• 18:07
  to 48 hours time so
• 18:08
  SPEAKER_02: that they can...
• 18:09
  you know,
• 18:09
  they can actually have an active response that goes to the
• 18:12
  right place.
• 18:13
  SPEAKER_02: And,
• 18:13
  SPEAKER_02: you know,
• 18:14
  SPEAKER_02: when we ran simulations on this,
• 18:15
  we were seeing large improvements in operational efficiency for
• 18:20
  these kinds of
• 18:20
  SPEAKER_02: fastball operations.
• 18:21
  SPEAKER_02: So you're saving on fuel,
• 18:23
  SPEAKER_02: personnel,
• 18:24
  SPEAKER_02: you know,
• 18:24
  SPEAKER_02: time,
• 18:25
  SPEAKER_02: and also probably an increase in the recovery rate of material.
• 18:29
  SPEAKER_02: Because rather than just,
• 18:31
  SPEAKER_02: you know,
• 18:31
  SPEAKER_02: driving around in a boat and hoping to,
• 18:34
  SPEAKER_02: you know,
• 18:34
  SPEAKER_02: encounter material,
• 18:35
  SPEAKER_02: you can be a lot more data-driven
• 18:37
  in and where you look so this this kind of predictive stuff is
• 18:42
  is becoming
• 18:42
  important yeah the other side and maybe this veers into another
• 18:46
  use case beyond the
• 18:47
  plastic it's the algal bloom um project that we're doing at the
• 18:51
  moment um in that
• 18:54
  project call one of the main um requirements was that the tool
• 18:58
  wasn't sort of
• 18:59
  retrospective that it had a a forward look so we've we've
• 19:03
  actually tackled it more
• 19:04
  SPEAKER_02: as a time series problem where given
• 19:07
  SPEAKER_02: historical seasonality patterns of the algae from space.
• 19:11
  SPEAKER_02: Can we say something about what's happening in the upcoming 12 months?
• 19:15
  And then you can backtest on historical data and check that
• 19:20
  your error rates are stable.
• 19:21
  So it's really not,
• 19:23
  we're not just detecting the current algae levels in Loch Ness.
• 19:27
  SPEAKER_02: We're trying to tell them, here's the outlook for the next 12 months.
• 19:32
  SPEAKER_02: And also looking at which
• 19:34
  external features like weather and agriculture patterns and
• 19:38
  temperature um do they
• 19:40
  impact the eventual bloom signal or not so you know it's really
• 19:44
  with a look forward
• 19:46
  rather than just being retrospectives and i think that's one of
• 19:49
  the huge powers of
• 19:50
  you know predictive modeling and ai it's it's not just
• 19:53
  inference today it's you
• 19:55
  know giving you a lens into the future so we're very excited
• 19:58
  about that and it's
• 19:59
  it's a it's a completely new direction for for the remote
• 20:02
  sensing tech but
• 20:03
  SPEAKER_00: It's cool stuff.
• 20:05
  Maybe as well just to mention why that's important in the
• 20:08
  context of Loch Ness.
• 20:09
  SPEAKER_00: So Loch Ness is the biggest freshwater body in the UK and in Ireland.
• 20:13
  SPEAKER_00: It's around 400 kilometres squared.
• 20:17
  SPEAKER_00: Its catchment area covers 45% of the land coverage in Northern Ireland.
• 20:22
  It's providing equivalently 40% of the population with its
• 20:27
  drinking water.
• 20:27
  SPEAKER_00: So it's a really important natural resource.
• 20:30
  SPEAKER_00: It also supports important industry.
• 20:33
  SPEAKER_00: So there's eel farms.
• 20:35
  SPEAKER_00: which are a million point industry,
• 20:38
  Hundreds of tons of eels being shipped across Europe every morning,
• 20:42
  especially to Amsterdam.
• 20:45
  SPEAKER_00: The Dutch are very fond of a jelly deal or something.
• 20:50
  And then obviously really important for kind of tourism and
• 20:54
  leisure as well.
• 20:54
  SPEAKER_00: So increasingly, you know, people are fond of swimming in the lake.
• 21:00
  Over the past kind of 10 years,
• 21:02
  we've seen increasing activity of these algal blooms.
• 21:06
  SPEAKER_00: So especially in 2023,
• 21:07
  SPEAKER_00: you know,
• 21:08
  SPEAKER_00: pretty much,
• 21:08
  SPEAKER_00: you know,
• 21:09
  SPEAKER_00: huge swathes of the lake turned green,
• 21:11
  SPEAKER_00: you know,
• 21:11
  SPEAKER_00: with this green kind of sludge washing up as well.
• 21:15
  SPEAKER_00: Within these blooms,
• 21:17
  SPEAKER_00: you're getting both kind of normal algae,
• 21:19
  SPEAKER_00: but there's also a type of bacteria,
• 21:21
  SPEAKER_00: a photosynthesizing bacteria called cyanobacteria.
• 21:25
  SPEAKER_00: And it's a bit nasty because it's producing toxins.
• 21:28
  So this is problematic because you're getting toxins then going
• 21:30
  into your water supply,
• 21:32
  SPEAKER_00: into water that you've got people swimming in.
• 21:36
  SPEAKER_00: So being able to somehow predict when these instances are going to occur.
• 21:43
  and then be able to have some foresight of that and be able to
• 21:45
  take mitigation
• 21:47
  SPEAKER_00: actions is really important.
• 21:50
  SPEAKER_00: So we are one project looking at that forecasting piece.
• 21:55
  SPEAKER_00: And then there's another set of three projects as well,
• 21:58
  SPEAKER_00: looking at the kind of mitigation piece as well.
• 22:00
  SPEAKER_00: So how do we actually interrupt the formation of these algal blooms?
• 22:06
  And very quickly,
• 22:07
  the reasons why they've got so bad in the past kind of decade
• 22:11
  are
• 22:12
  SPEAKER_00: kind of threefold.
• 22:14
  SPEAKER_00: So there's a lot of nutrient runoff.
• 22:16
  SPEAKER_00: It's obviously heavily farmed land around the loch.
• 22:21
  SPEAKER_00: So there's a lot of nutrients, phosphorous and nitrogens, et cetera.
• 22:25
  SPEAKER_00: So this is essentially food for these blooms.
• 22:29
  SPEAKER_00: The water temperature is going up.
• 22:32
  Modest rises,
• 22:32
  but it still pushes them towards a more kind of optimal stage
• 22:36
  of growth.
• 22:37
  And then there's also an invasive species of mussels called
• 22:40
  zebra mussels,
• 22:41
  SPEAKER_00: and they've basically increased the clarity of the water.
• 22:44
  SPEAKER_00: So you're getting more penetration by light into the water,
• 22:48
  SPEAKER_00: meaning it's better conditions for photosynthesizing.
• 22:53
  SPEAKER_00: And certainly we...
• 22:55
  SPEAKER_00: When we're looking at these prediction models,
• 22:57
  we want to consider the variables as widely as possible to try
• 23:02
  and refine and tune
• 23:04
  SPEAKER_00: these predictions to be as accurate as possible.
• 23:07
  SPEAKER_01: So lots of inputs to the system, essentially.
• 23:11
  Do you find that the models are just going to use remote
• 23:14
  sensing data or they're
• 23:15
  SPEAKER_01: going to need to use some other kind of data as well?
• 23:17
  SPEAKER_01: How do you see them evolving?
• 23:19
  SPEAKER_02: It's a great question.
• 23:20
  SPEAKER_02: I was going to touch on it just now.
• 23:21
  So one of the real strengths of the satellite data and the
• 23:24
  remote sensing is the
• 23:25
  SPEAKER_02: historical archive.
• 23:27
  SPEAKER_02: So Sentinel-2 coverage dates back to 2017.
• 23:31
  SPEAKER_02: So we have many years of effectively weekly observations,
• 23:36
  SPEAKER_02: modular cloud cover,
• 23:37
  SPEAKER_02: which is a problem in Northern Ireland.
• 23:38
  SPEAKER_02: That's why James and I are so pale.
• 23:41
  But you can look back over many years and look at the
• 23:44
  seasonality patterns and
• 23:46
  SPEAKER_02: levels and trend analysis.
• 23:49
  One of the limitations of other approaches is that you might
• 23:52
  not have such an
• 23:53
  SPEAKER_02: extensive archive.
• 23:54
  SPEAKER_02: But saying that,
• 23:55
  we have also commissioned drone data acquisition in the project
• 23:59
  to give us a view
• 24:00
  SPEAKER_02: below the clouds with richer spatial resolution.
• 24:05
  And one of the objectives in the next couple of weeks is to
• 24:08
  integrate that and fuse
• 24:09
  SPEAKER_02: it with the satellite-based data,
• 24:11
  SPEAKER_02: feed it into the predictive models and backtest and see
• 24:16
  Do the features derived from other types of sensors like drones
• 24:19
  lead to a reduction
• 24:21
  SPEAKER_02: in error rate or not?
• 24:23
  SPEAKER_02: So I'm generally quite pragmatic.
• 24:24
  It always comes down to whether a given predictive model has a
• 24:28
  better performance
• 24:30
  SPEAKER_02: historically or not.
• 24:31
  SPEAKER_02: So our job now is to test as many combinations of inputs as we can.
• 24:37
  Another potential candidate there is the synthetic aperture
• 24:40
  radar data as well,
• 24:42
  SPEAKER_02: which sees at night.
• 24:43
  SPEAKER_02: It sees through cloud.
• 24:44
  and can give you rich detail on the water surface and roughness
• 24:49
  due to the interferometry.
• 24:52
  So I think at the moment we're still scoping whether there is a
• 24:56
  relative value add
• 24:57
  SPEAKER_02: on top of our bread and butter optical satellite data.
• 25:01
  SPEAKER_02: But I think we're
• 25:02
  We're optimistic that we can squeeze a bit more juice out of
• 25:05
  the lemon than what
• 25:06
  SPEAKER_02: you just get from any single data source.
• 25:08
  The same is true for all of the use cases that we've
• 25:11
  encountered in the company at the moment.
• 25:14
  It really seems like a multimodal approach where we leverage
• 25:18
  data from different
• 25:19
  SPEAKER_02: sensors is generally the best thing you can do.
• 25:22
  SPEAKER_02: But it obviously changes the cost profile.
• 25:25
  And whether or not it's an affordable solution in the long run
• 25:29
  is also important
• 25:31
  SPEAKER_02: from a business perspective.
• 25:32
  Sentinel-2 data is so powerful because you can do a lot with
• 25:37
  the open source data
• 25:38
  SPEAKER_02: as your baseline.
• 25:42
  SPEAKER_01: Really exciting problem.
• 25:43
  SPEAKER_01: Lots of avenues to explore there.
• 25:46
  SPEAKER_01: Lots of different constraints feeding in as well.
• 25:49
  SPEAKER_00: Absolutely.
• 25:50
  SPEAKER_00: Well, I guess that's... Can I just add one point to that?
• 25:54
  SPEAKER_00: Just because it's kind of interesting.
• 25:57
  SPEAKER_00: So last week, Donald mentioned collection of data with drones.
• 26:00
  SPEAKER_00: Obviously, this is very helpful to be flying onto the clouds.
• 26:04
  SPEAKER_00: So last week,
• 26:05
  SPEAKER_00: our partner in this project,
• 26:07
  SPEAKER_00: Marble Aerospace,
• 26:08
  SPEAKER_00: were out flying drones over the lock over the course of last week.
• 26:13
  SPEAKER_00: Didn't start well due to the tail ends of one of the storms.
• 26:17
  SPEAKER_00: But it picked up and they had a brilliant day on Wednesday,
• 26:20
  SPEAKER_00: last Wednesday,
• 26:21
  SPEAKER_00: collecting data over the lock.
• 26:24
  SPEAKER_00: So they have fixed-wing drones.
• 26:28
  SPEAKER_00: They fly extremely quickly, exceeding 100 kilometers per hour.
• 26:34
  SPEAKER_00: And they can collect information,
• 26:38
  SPEAKER_00: obviously,
• 26:38
  SPEAKER_00: with a higher resolution,
• 26:40
  SPEAKER_00: which we would otherwise have accessible through satellite data.
• 26:43
  SPEAKER_00: Yeah.
• 26:45
  SPEAKER_00: Because they are so fast,
• 26:47
  they could potentially fly and collect data and imagery over
• 26:52
  the entire lake over
• 26:54
  SPEAKER_00: the course of about four hours.
• 26:56
  We didn't do that last week because that would involve flying
• 26:59
  beyond visual line of
• 27:00
  SPEAKER_00: sight and that requires additional permissions,
• 27:03
  SPEAKER_00: et cetera.
• 27:04
  SPEAKER_00: But I think the idea would be
• 27:06
  SPEAKER_00: to use that technology,
• 27:08
  perhaps when you've had red alerts from the satellite imagery
• 27:13
  to get a higher
• 27:15
  SPEAKER_00: fidelity data set available.
• 27:19
  SPEAKER_00: So that's pretty exciting.
• 27:21
  And I think that the first time kind of these kind of
• 27:27
  state-of-the-art type drones
• 27:28
  SPEAKER_00: have done monitoring work over the lock.
• 27:31
  SPEAKER_00: So it's a cool aspect of the project as well.
• 27:34
  SPEAKER_00: All right.
• 27:36
  SPEAKER_02: such as on the continuous monitoring piece as well,
• 27:38
  SPEAKER_02: but we don't consider this work ever really fixed and frozen.
• 27:43
  We will always continue to feed the updated satellite data in
• 27:48
  as a feature set,
• 27:48
  SPEAKER_02: and have really a rolling forecast,
• 27:51
  SPEAKER_02: and the drone data will be an important aspect of that as well.
• 27:54
  SPEAKER_02: Longer term,
• 27:55
  SPEAKER_02: if you want to be really ambitious with this,
• 27:57
  SPEAKER_02: you acquire drone data continuously through the year,
• 28:00
  SPEAKER_02: and you fuse it with the satellites,
• 28:03
  continue to orbit and we really produce a rolling forecast and
• 28:07
  see how far,
• 28:08
  SPEAKER_02: you know,
• 28:08
  SPEAKER_02: we can push the horizon of that model in the long run.
• 28:11
  So,
• 28:11
  yeah,
• 28:12
  it's really exciting from a,
• 28:14
  you know,
• 28:14
  from a productionization aspect.
• 28:17
  SPEAKER_02: I think we're really keen to see where we can take this.
• 28:20
  SPEAKER_01: Well,
• 28:20
  SPEAKER_01: really looking forward to seeing the platform soon,
• 28:23
  SPEAKER_01: seeing how the system evolves and,
• 28:24
  SPEAKER_01: you know,
• 28:25
  SPEAKER_01: hearing sort of future upgrades to it.
• 28:28
  SPEAKER_01: Where's the best place to follow the announcements from yourselves?
• 28:33
  SPEAKER_00: Yeah, probably LinkedIn at the moment is where we're quite vocal.
• 28:38
  SPEAKER_00: If you go to our website as well,
• 28:40
  SPEAKER_00: which is www.plastic-i.com,
• 28:44
  SPEAKER_00: we'll also make announcements on there.
• 28:47
  SPEAKER_00: And there's a link to the observatory.
• 28:50
  So you can sign up and be one of the first to access that when
• 28:56
  we release that in March.
• 28:59
  SPEAKER_00: So yeah, and we're always available and happy to take queries as well.
• 29:04
  SPEAKER_00: So there's a form on the website there where we can submit queries to us.
• 29:08
  SPEAKER_00: So yeah, we're delighted to receive those as well.
• 29:11
  SPEAKER_01: Fantastic.
• 29:12
  SPEAKER_01: I'll drop the details in the show notes,
• 29:14
  SPEAKER_01: but once again,
• 29:15
  SPEAKER_01: thanks for joining me and all the best for the launch.
• 29:18
  SPEAKER_00: Thank you very much, Robin.
• 29:18
  SPEAKER_00: Cheers.
• 29:19
  SPEAKER_00: Thanks, Robin.
• 29:20
  SPEAKER_00: Pleasure.