Latest / Satellite image deep learning / Deepness QGIS plugin

Transcript

Deepness QGIS plugin · Satellite image deep learning

• 0:05
  SPEAKER_01: Hello, I'm Robin Cole and welcome to the Satellite Image Deep Learning Podcast.
• 0:10
  SPEAKER_01: In this episode, I caught up with Marek Craft to learn about the Deepness QGIS plugin.
• 0:14
  SPEAKER_01: QGIS is a widely used open source tool for working with geospatial data.
• 0:19
  SPEAKER_01: It's written in Python and its functionality can be extended with plugins.
• 0:23
  SPEAKER_01: One plugin that recently caught my attention is Deepness, developed by Marek and his team.
• 0:27
  SPEAKER_01: Deepness makes it straightforward to use deep learning models in QGIS.
• 0:31
  SPEAKER_01: You don't need specialized hardware like GPUs,
• 0:33
  SPEAKER_01: and it offers a range of pre-trained models through a model zoo.
• 0:37
  SPEAKER_01: As a long-time QGIS user,
• 0:38
  SPEAKER_01: I was thrilled to discover Deepness,
• 0:40
  SPEAKER_01: and I believe it has the potential to make deep learning much more accessible to
• 0:43
  SPEAKER_01: geospatial practitioners without deep learning expertise.
• 0:47
  SPEAKER_01: Marek shared some fascinating examples of how the plugin is being used,
• 0:50
  SPEAKER_01: and discussed the growing community around it.
• 0:53
  SPEAKER_01: Note that there's also a demo video showcasing Deepness in Action.
• 0:57
  SPEAKER_01: If you're watching this on YouTube, please consider liking the video and subscribing to the channel.
• 1:01
  SPEAKER_01: It really helps to reach more people and grow the audience.
• 1:03
  SPEAKER_01: Thank you.
• 1:06
  SPEAKER_01: Hi, Marek.
• 1:06
  SPEAKER_01: Welcome to the podcast.
• 1:08
  SPEAKER_00: Hello, Robin.
• 1:10
  SPEAKER_00: Nice.
• 1:10
  SPEAKER_00: Thanks for having me.
• 1:12
  SPEAKER_00: It's a pleasure.
• 1:13
  SPEAKER_01: So let's begin with where you work and what you do.
• 1:17
  SPEAKER_00: Okay,
• 1:18
  SPEAKER_00: so I'm with the Poznań University of Technology in Poznań,
• 1:21
  SPEAKER_00: Poland,
• 1:22
  SPEAKER_00: with the Institute of Robotics and Machine Intelligence,
• 1:26
  SPEAKER_00: where I work mostly on image processing.
• 1:31
  SPEAKER_00: Back in the day,
• 1:32
  SPEAKER_00: it was mostly for robotics,
• 1:33
  SPEAKER_00: but recently we're also into space applications,
• 1:35
  SPEAKER_00: which obviously includes also earth observations.
• 1:38
  SPEAKER_00: Right.
• 1:39
  SPEAKER_00: That's why I'm here, I suppose.
• 1:41
  SPEAKER_01: Yes.
• 1:42
  SPEAKER_01: And what kind of topics of research do you do?
• 1:47
  SPEAKER_00: uh so aside from observation mostly from drones uh also some satellite uh earth
• 1:53
  SPEAKER_00: observation because we kind of started close to the ground and we're moving up but
• 1:59
  SPEAKER_00: also space robotics recently uh we've been involved in this topic too so that's a
• 2:05
  SPEAKER_01: very cool area so we connected over deepness do you want to give us a short
• 2:09
  SPEAKER_01: introduction to deepness
• 2:12
  SPEAKER_00: OK, so Deepness is essentially a plugin for QGIS software.
• 2:18
  SPEAKER_00: If maybe your viewers are not familiar with that package,
• 2:22
  SPEAKER_00: it's an open source software that is very rich in geographical information system functionalities.
• 2:28
  SPEAKER_00: And a lot of people use it for different purposes,
• 2:30
  SPEAKER_00: like land management,
• 2:32
  SPEAKER_00: agriculture,
• 2:33
  SPEAKER_00: planning,
• 2:34
  SPEAKER_00: et cetera,
• 2:34
  SPEAKER_00: et cetera.
• 2:36
  SPEAKER_00: And our plugin enables the users to use pretty much any
• 2:42
  SPEAKER_00: typical deep learning neural network so detection regression segmentation
• 2:50
  SPEAKER_00: classification and run it from within the software so that it is really nicely
• 2:56
  SPEAKER_00: available until this common interface that they already know and they don't really
• 3:01
  SPEAKER_00: have to have any deep learning expertise they just can't use it can use it within i
• 3:06
  SPEAKER_00: guess like a few clicks of a mouse button so
• 3:10
  SPEAKER_01: Very cool.
• 3:11
  SPEAKER_01: And what was the motivation behind doing this?
• 3:14
  SPEAKER_00: It actually started as a small project that we needed for our purpose because we
• 3:21
  SPEAKER_00: were working on assessing the damage that is done by wild animals to some crop.
• 3:27
  SPEAKER_00: And
• 3:28
  SPEAKER_00: instead of trying to reinvent the wheel,
• 3:31
  SPEAKER_00: which is obviously also our first intuition that,
• 3:35
  SPEAKER_00: hey,
• 3:35
  SPEAKER_00: we should do some kind of interface for that because it will look nicer and we'll
• 3:40
  SPEAKER_00: have more control and we'll be able to view the results,
• 3:43
  SPEAKER_00: et cetera,
• 3:43
  SPEAKER_00: et cetera.
• 3:44
  SPEAKER_00: I think that's like engineers first thought that, hey, let's build something.
• 3:50
  SPEAKER_00: Actually, the second thought was, hey, maybe there's something out there that's available.
• 3:55
  SPEAKER_00: Maybe we'll integrate with some web services.
• 3:58
  SPEAKER_00: But that's how we got into this piece of software.
• 4:02
  SPEAKER_00: And it turned out that it's very rich in functionalities.
• 4:05
  SPEAKER_00: It enables you to do a lot just right out of the box.
• 4:09
  SPEAKER_00: And it also can be extended fairly easily with Python.
• 4:15
  SPEAKER_00: And yeah,
• 4:17
  SPEAKER_00: as a test,
• 4:19
  SPEAKER_00: one of my colleagues wrote inference engine that was capable of running from within
• 4:25
  SPEAKER_00: this software.
• 4:26
  SPEAKER_00: And it turned out that it works fairly good.
• 4:29
  SPEAKER_00: And once we had this basic segmentation pipeline that was fully serviceable from within the software,
• 4:39
  SPEAKER_00: From that, there was really an easy way to extend it and add some functionalities.
• 4:43
  SPEAKER_00: And I know that the guys from our team work on adding things,
• 4:47
  SPEAKER_00: but we also have a fairly lively community,
• 4:52
  SPEAKER_00: and there's been some contributed functionality too.
• 4:54
  SPEAKER_00: And pretty much anyone who can train a deep learning model and convert this to this
• 4:59
  SPEAKER_00: common ONNX exchange format and stick to some conventions regarding the output data format,
• 5:08
  SPEAKER_00: can use this to run their own model and maybe see the results,
• 5:13
  SPEAKER_00: you know,
• 5:13
  SPEAKER_00: confront this with some migrant map,
• 5:15
  SPEAKER_00: maybe edit some results or produce some training data even with this plugin.
• 5:21
  SPEAKER_00: I think it's a very useful tool.
• 5:23
  SPEAKER_00: And I'm also fairly excited that it kind of enables people that don't have this really, really
• 5:29
  SPEAKER_00: deep learning expertise to use this tool to actually do something useful because we
• 5:34
  SPEAKER_00: have a lot of requests for functionalities and we have some feedback that people
• 5:39
  SPEAKER_00: were using it for some things that we didn't even think,
• 5:42
  SPEAKER_00: you know,
• 5:42
  SPEAKER_00: are possible or
• 5:45
  SPEAKER_00: the tool can be used for that so it's really nice and encouraging to do this that's
• 5:50
  SPEAKER_01: very cool yeah QGIS is an immensely popular tool written in python as you say lots
• 5:55
  SPEAKER_01: of users both you know government open source etc obviously when we think about
• 6:01
  SPEAKER_01: machine learning and deep learning in particular we think about needing powerful
• 6:04
  SPEAKER_01: computers to run these models that's true during training but i think what you've
• 6:08
  SPEAKER_01: shown is that you don't necessarily need a big powerful computer just to use the
• 6:11
  SPEAKER_01: model and that's what you've demonstrated with this plugin right
• 6:16
  SPEAKER_00: Yeah, yeah.
• 6:16
  SPEAKER_00: Inference can be run using a lot less resources.
• 6:22
  SPEAKER_00: It can even run on CPU.
• 6:23
  SPEAKER_00: It will be a bit slower.
• 6:25
  SPEAKER_00: But generally, it doesn't really require you to have a very powerful computer.
• 6:32
  SPEAKER_00: And if you just want to test drive it,
• 6:34
  SPEAKER_00: I think that pretty much any modern-day PC should be able to handle that.
• 6:38
  SPEAKER_01: Yeah.
• 6:40
  SPEAKER_01: And can you tell us a little bit more about ONNX?
• 6:44
  SPEAKER_00: So ONNX is basically a common neural network exchange format.
• 6:51
  SPEAKER_00: I think that a lot of tools use it like a portable neural network description if
• 6:58
  SPEAKER_00: you just want to run inference,
• 6:59
  SPEAKER_00: right?
• 7:00
  SPEAKER_00: So you're able to import your neural network in pretty much every major framework.
• 7:06
  SPEAKER_00: I think there's a lot of things that use it kind of like an intermediate format, for example.
• 7:13
  SPEAKER_00: If you have some inference engines,
• 7:15
  SPEAKER_00: for example,
• 7:16
  SPEAKER_00: for those embedded NVIDIA boards,
• 7:18
  SPEAKER_00: they often make use of ONNX.
• 7:21
  SPEAKER_00: There's a lot of tools in the embedded space that use ONNX as an exchange format.
• 7:27
  SPEAKER_00: Nice thing is that it's well documented and a lot of people actually agreed to use it, which is
• 7:35
  SPEAKER_00: not that common for standards.
• 7:39
  SPEAKER_00: So basically,
• 7:41
  SPEAKER_00: if you use Chex or Keras or TensorFlow or PyTorch,
• 7:46
  SPEAKER_00: they all have built-in functionalities for converting into this format.
• 7:49
  SPEAKER_00: So your prepared model can be run from within our tool too, thanks to it all.
• 7:56
  SPEAKER_00: Yeah, so that's pretty much it, I guess.
• 7:58
  SPEAKER_01: Oh, that's really cool.
• 7:59
  SPEAKER_01: So any model being trained anywhere pretty much can be converted to this format.
• 8:02
  SPEAKER_01: And as long as it complies to the expectations of the output input data formats,
• 8:07
  SPEAKER_01: it's going to be compatible with your plugin.
• 8:09
  SPEAKER_01: Really nice.
• 8:11
  SPEAKER_01: And the tool itself lives on GitHub, right?
• 8:15
  SPEAKER_00: Yeah, yeah.
• 8:15
  SPEAKER_00: It's fully open source.
• 8:17
  SPEAKER_00: We have a model zoo.
• 8:18
  SPEAKER_00: So we have quite a few models that you can just download and test drive yourself.
• 8:22
  SPEAKER_00: We are always looking for some more developers,
• 8:25
  SPEAKER_00: some maintainers,
• 8:26
  SPEAKER_00: because like any open source project,
• 8:29
  SPEAKER_00: it doesn't live without the people that are willing to contribute some time to work
• 8:34
  SPEAKER_00: on it.
• 8:36
  SPEAKER_00: It's also available from the QGIS plugin download websites.
• 8:42
  SPEAKER_00: Pretty much if you look for it from within the tool or from the website, you can just find it.
• 8:49
  SPEAKER_00: It's been downloaded over 25,000 times now from this plugin repository.
• 8:55
  SPEAKER_00: So it's a number we didn't really expect when we started.
• 9:00
  SPEAKER_00: Yeah,
• 9:01
  SPEAKER_00: it's really exciting to see where it gets us because the amount of things that
• 9:05
  SPEAKER_00: people would like to use it for range from finding mistletoe in forests or flooded
• 9:13
  SPEAKER_00: terrain after some natural disasters or finding wild animals.
• 9:19
  SPEAKER_00: I recently seen someone develop a plugin and model that counts sheep in fields.
• 9:28
  SPEAKER_00: Not sure if that's for counting insomnia or whatever, but it's funny.
• 9:35
  SPEAKER_00: But if someone has a need for that and has a UAV that flies over the fields maybe
• 9:40
  SPEAKER_00: to check on the inventory of their animals,
• 9:44
  SPEAKER_00: I can see how this can be useful.
• 9:47
  SPEAKER_00: And, yeah, basically, you have the right tool for that.
• 9:50
  SPEAKER_00: You have the interface.
• 9:52
  SPEAKER_00: You can perform some counting,
• 9:54
  SPEAKER_00: some spatial analysis just right off the bat because you have this whole power of
• 9:58
  SPEAKER_00: this tool behind you,
• 9:59
  SPEAKER_00: right?
• 10:00
  SPEAKER_00: It's not just the inference, but all the things that are associated with QGIS are basically built in.
• 10:05
  SPEAKER_00: So it's really nice.
• 10:06
  SPEAKER_01: So the predictions are just polygons, and you can just export those, the standard formats.
• 10:10
  SPEAKER_00: Yeah, it's polygons.
• 10:13
  SPEAKER_00: You can do some basic logical analysis or apply some rules to that or calculate the
• 10:24
  SPEAKER_00: areas or do whatever you want.
• 10:27
  SPEAKER_00: Actually, I'm not that much of an expert in this tool.
• 10:31
  SPEAKER_00: I always promise myself that I need to learn more.
• 10:33
  SPEAKER_00: But yeah, generally, we also got really nice reception from the community.
• 10:40
  SPEAKER_00: We participated in this QGIS Polish user meeting, and the response was very, very, very nice.
• 10:48
  SPEAKER_00: I didn't expect there would be that many people and that they will have a lot of
• 10:53
  SPEAKER_00: needs that they would ask us,
• 10:55
  SPEAKER_00: hey,
• 10:55
  SPEAKER_00: can you do this?
• 10:56
  SPEAKER_00: Can you do that?
• 10:57
  SPEAKER_00: Could that be possible?
• 10:58
  SPEAKER_00: It's really like completely different experience for what we're used to in image
• 11:04
  SPEAKER_00: processing conferences because,
• 11:06
  SPEAKER_00: like I said,
• 11:06
  SPEAKER_00: we
• 11:07
  SPEAKER_00: don't even envision what people might try to use the tool for.
• 11:11
  SPEAKER_00: And there's a lot of unmet needs.
• 11:13
  SPEAKER_00: And the people are not really aware what kind of power those neural network tools
• 11:17
  SPEAKER_00: can give them in order to automate or ease their pain with spatial processing.
• 11:22
  SPEAKER_00: So that's really, really nice.
• 11:24
  SPEAKER_00: There's a lot to do in this field.
• 11:25
  SPEAKER_01: Yeah.
• 11:25
  SPEAKER_01: Seems like you've really identified like a good opportunity there with this
• 11:30
  SPEAKER_01: software that already has broad adoption and just that,
• 11:33
  SPEAKER_01: you know,
• 11:33
  SPEAKER_01: making it accessible to use these most powerful models.
• 11:36
  SPEAKER_01: So really, really exciting to watch this progress.
• 11:40
  SPEAKER_01: Yeah.
• 11:40
  SPEAKER_01: Thank you.
• 11:41
  SPEAKER_01: Fantastic.
• 11:42
  SPEAKER_01: Okay.
• 11:42
  SPEAKER_01: We're going to record a video demo of the,
• 11:45
  SPEAKER_01: of the,
• 11:46
  SPEAKER_01: of the tool that'd be in a separate recording,
• 11:49
  SPEAKER_01: but for now we'll,
• 11:50
  SPEAKER_01: we'll end this,
• 11:51
  SPEAKER_01: this episode here,
• 11:52
  SPEAKER_01: but if people want to follow along deepness,
• 11:54
  SPEAKER_01: obviously head to GitHub.
• 11:56
  SPEAKER_01: Do you put out announcements anywhere else on sort of,
• 12:00
  SPEAKER_00: We usually announce on LinkedIn.
• 12:02
  SPEAKER_00: Some of it is also announced on my Twitter account.
• 12:04
  SPEAKER_00: But generally,
• 12:05
  SPEAKER_00: if you subscribe to notifications on GitHub or you're a QGIS user,
• 12:10
  SPEAKER_00: you'll get notifications of the new version of the plugin.
• 12:14
  SPEAKER_00: We're always welcome contacts and suggestions on how we can improve that,
• 12:19
  SPEAKER_00: on what maybe our next model should be.
• 12:22
  SPEAKER_00: Obviously,
• 12:23
  SPEAKER_00: we can't do it all because,
• 12:25
  SPEAKER_00: like I said,
• 12:25
  SPEAKER_00: there's a lot of people that are trying to use it and they have a lot of needs.
• 12:29
  SPEAKER_00: and we're only a few people,
• 12:30
  SPEAKER_00: this is why we also would welcome some help with developing the tool very much.
• 12:39
  SPEAKER_00: So, yeah, that's pretty much it, I guess.
• 12:42
  SPEAKER_01: Perfect.
• 12:43
  SPEAKER_01: I'll put all the links in the show notes,
• 12:44
  SPEAKER_01: and hopefully this encourages people to come and try it out and hopefully
• 12:48
  SPEAKER_01: contribute as well.
• 12:49
  SPEAKER_01: And I'll see you shortly for the demonstration.
• 12:51
  SPEAKER_01: Thank you.