Latest / Ctrl-Alt-Speech / Spotlight: Building Better CSAM Detection with Resolver’s George Vlasto

Transcript

Spotlight: Building Better CSAM Detection with Resolver’s George Vlasto · Ctrl-Alt-Speech

• 0:00
  Mike Masnick: Hello and welcome to Control Alt Speech.
• 0:02
  Mike Masnick: As we mentioned in last week's episode, we do not have a regular episode for you this week because it is Thanksgiving here in the us, but we do have something special for you instead.
• 0:13
  Mike Masnick: It is one of our sponsored Spotlight episodes in which we get to go deep with an expert in the trust and safety field and have a really.
• 0:21
  Mike Masnick: Interesting thought provoking conversation this week.
• 0:24
  Mike Masnick: The spotlight is brought to you by Resolver, talking about how they've just reached their 20th anniversary, supplying trust and safety services, which really is kind of incredible given just how drastically the internet world has changed over the last 20 years.
• 0:40
  Mike Masnick: In this episode, Ben got to sit down with George Blato, who's the head of Resolvers Trust and Safety Division to talk about their unique approach to scaling the detection of child sexual abuse material, better known as CSAM.
• 0:52
  Mike Masnick: They reminisce on how much the online world has changed in these past 20 years, as well as talk about how Resolver latest innovation called Athena is an incredibly powerful tool in the fight against online harms.
• 1:05
  Mike Masnick: Please enjoy this spotlight episode with.
• 1:08
  Mike Masnick: Ben and George Valeto from Resolver.
• 1:18
  Ben Whitelaw: George, welcome to Control Al Speech.
• 1:20
  Ben Whitelaw: Thanks so much for joining me today, and thanks again to Resolver for sponsoring the podcast and making Mike and I's Weekly JAB Rings possible.
• 1:29
  Ben Whitelaw: we're gonna dive straight in and talk a bit about Resolver Trust and Safety, which is a company that's been around a long time.
• 1:35
  Ben Whitelaw: It's been around 20 years.
• 1:37
  Ben Whitelaw: this week, in fact.
• 1:38
  Ben Whitelaw: For people who are new to the company who maybe weren't paying attention back in 2005, tell us a bit about Resolver, what it does in the space.
• 1:46
  George Vlasto (Resolver): Thanks Ben, and thank you for having me on the podcast.
• 1:47
  George Vlasto (Resolver): It's uh, it's a pleasure to be with you today.
• 1:50
  George Vlasto (Resolver): as you say, we are celebrating our 20th anniversary this week, which is a huge milestone for us.
• 1:55
  George Vlasto (Resolver): 20 years, is, almost an eternity in technology.
• 1:59
  George Vlasto (Resolver): and I was talking to our, our founder, Adam Hildrith the other day, and we were reflecting on the earliest days of Resolver, the sort of early days of the social internet and, really.
• 2:08
  George Vlasto (Resolver): The journey we've been on from being at the, cutting edge of discovering these kind of emerging harms, which at the time were really not well understood or, or well thought through.
• 2:17
  George Vlasto (Resolver): and the sort of earliest bad actors operating in places where vulnerable users congregated, particularly children.
• 2:23
  George Vlasto (Resolver): And how we, started, really in the early days, quite a sort of scrappy, Largely sort of keyword based effort to identify problematic behavior and how that's evolved over 20 years.
• 2:34
  George Vlasto (Resolver): And now we have data scientists and risk detection engineers working with our human specialists to be able to detect not just, you know, problematic content, but behavior.
• 2:43
  George Vlasto (Resolver): Trade craft as we called it, the sort of, you know, methodology that bad actors use.
• 2:47
  George Vlasto (Resolver): all the way up to our kind of, you know, latest, product launch, which is our unknown CSA detection service.
• 2:52
  George Vlasto (Resolver): And it's, it's an extraordinary number of people who've played a part in that whole range of disciplines, from some people with academic backgrounds through to deep data scientists, engineers, and everyone in between.
• 3:01
  George Vlasto (Resolver): And it's, it's pretty remarkable actually to reflect on that 20 year journey.
• 3:05
  Ben Whitelaw: Indeed, it's very few companies have been around for that length of time.
• 3:09
  Ben Whitelaw: You know, many have, have come and gone.
• 3:10
  Ben Whitelaw: I think it's over interesting for a number of reasons.
• 3:13
  Ben Whitelaw: You, you are not your kind of typical product driven company.
• 3:17
  Ben Whitelaw: address a range of harms, as you've alluded to there rather than just one.
• 3:21
  Ben Whitelaw: you have lots of different expertise kind of underneath the resolver, umbrella.
• 3:25
  Ben Whitelaw: Can you talk a bit about how you've, as a company developed.
• 3:29
  Ben Whitelaw: To cover all of these harms and with this broad range of expertise.
• 3:32
  Ben Whitelaw: 'cause I think it is fairly unique.
• 3:34
  George Vlasto (Resolver): It is we, we have the great advantage, I think, in this space that unlike a lot of our clients and partners, the only thing we think about is trust and safety.
• 3:41
  George Vlasto (Resolver): So for even the most sophisticated of our clients, trust and safety is a component, is an important component, but it's a component, part of what their overall organization thinks about.
• 3:48
  George Vlasto (Resolver): For us, it's everything which has allowed us to be, although we do have, products and product management function, we are actually problem focused and I think that's the kind of our ethos ever since our earliest days in 2005.
• 4:01
  George Vlasto (Resolver): Problems evolve.
• 4:02
  George Vlasto (Resolver): Problems emerge both through human behavior and technology innovation and being nimble enough to respond to those in a kind of dynamic way has been pretty key to our success thus far.
• 4:13
  George Vlasto (Resolver): And also, I think is a reflection or is reflected in the fact that we have this very diverse group of skills and people, globally now in, four or five different countries operating kind of 24 7.
• 4:24
  George Vlasto (Resolver): And it's about being problem focused rather than having a product that we're trying to market for its own sake.
• 4:29
  George Vlasto (Resolver): It's about constantly adapting our approach to, fix the problem or address the problem that our clients and partners are are struggling with at that given moment in time.
• 4:37
  Ben Whitelaw: Great.
• 4:37
  Ben Whitelaw: That's really helpful.
• 4:38
  Ben Whitelaw: you, you mentioned when you explained Resolver about its focus on behaviors and kind of identifying harmful behaviors as much as kinda harmful content, and there is an emphasis on intelligence and intelligence led approach.
• 4:52
  Ben Whitelaw: What does that mean in practice and, and how does, Resolver think about the balance of not only kind of human intelligence, but also increasingly AI intelligence too.
• 5:02
  George Vlasto (Resolver): that's a really good question.
• 5:03
  George Vlasto (Resolver): And think the advent of ai, at huge scale with remarkable capabilities obviously has changed our, the way that we address this problem, actually.
• 5:12
  George Vlasto (Resolver): So the way we think about it is there are, emerging threats, emerging behaviors, sometimes.
• 5:18
  George Vlasto (Resolver): In the context of new regulations or new regulatory requirements that our clients and, partners are required to think about.
• 5:25
  George Vlasto (Resolver): And being able to understand the risk on your platform or your service and how that compares to others, and also, interacts with the sort of regulatory framework in which you're operating is really important.
• 5:35
  George Vlasto (Resolver): And that is.
• 5:36
  George Vlasto (Resolver): Still, and I can't see a world in which it's not, in the future driven by human expertise, deep subject matter expertise that enables to act in a, you know, almost a consultative way with our partners to help them understand the risks that they might be facing and how those are evolving.
• 5:49
  George Vlasto (Resolver): AI is obviously a really important part, and almost all of our, partners use their own internal AI or AI that they, bring in for the job to detect harm at scale as sort of classifiers.
• 6:00
  George Vlasto (Resolver): And a lot of what we are doing with our human expertise is informing those classifiers and helping them operate at the kind of highest level of efficiency possible to drive real value to the companies, that we work with.
• 6:11
  George Vlasto (Resolver): In one case in particular, it's very hard for our partners to train their own models, and that's in CS a detection, which is why we brought to market what we call Athena, which is our sort of best in class, new generation csam detection service.
• 6:25
  George Vlasto (Resolver): And that's kind of our, unique.
• 6:27
  George Vlasto (Resolver): AI offering a machine learning model that can solve that problem for you in a way that your own classifiers can't, because you just can't train them on that kind of material.
• 6:33
  George Vlasto (Resolver): So as humans informing policy, training classifiers and then, in the hardest and most egregious harm in the case of CSA, are actually providing a solution directly to our partners, rather than leaving'em to sort of fumble in the dark around a very, very challenging, legally, ethically, and, you know, psychologically challenging kind of issue.
• 6:52
  Ben Whitelaw: Yeah.
• 6:53
  Ben Whitelaw: great to hear more about that.
• 6:54
  Ben Whitelaw: we're gonna talk a bit about Athena shortly, but I want to kind of talk a bit about you and, your part in Resolver journey, over the, the last 20 years.
• 7:04
  Ben Whitelaw: you had a 15 year career in the UK diplomatic service, and you've been in the role at Resolver as head of trust and safety, for three years now.
• 7:13
  Ben Whitelaw: Talk us about, that transition from UK government service into this role and give us a sense of what your kind of day-to-day looks like in the broader resolver machine.
• 7:24
  George Vlasto (Resolver): Yeah, so, so I, sort of went into public service and, diplomacy because I wanted to work on, you know, some of the most challenging consequential issues that there were.
• 7:32
  George Vlasto (Resolver): And I was lucky to do that in various parts of the world.
• 7:35
  George Vlasto (Resolver): But it became pretty clear to me a few years ago that actually the sort of generational challenge, that we.
• 7:40
  George Vlasto (Resolver): As a society, global society, not just in the UK or, just in sort of Western countries have to grapple with is, online harm, how we harness the power of the internet, which is obviously.
• 7:52
  George Vlasto (Resolver): Exponentially improved, you know, communication, academic collaboration in some cases, political freedom, and how do we balance that against the harm, especially to vulnerable users, particularly children.
• 8:03
  George Vlasto (Resolver): So for a number of years, I was looking for an opportunity to move across and then I was very fortunate to find this opportunity, it resolver.
• 8:09
  George Vlasto (Resolver): So it was actually oddly psychologically, quite a natural transition, although it's a very different job than what I was doing before.
• 8:15
  George Vlasto (Resolver): So my day at the moment, really it entails.
• 8:19
  George Vlasto (Resolver): Creating space for our team.
• 8:20
  George Vlasto (Resolver): So as I said at the beginning, our strength comes from our ability to.
• 8:24
  George Vlasto (Resolver): Understand quickly and respond rapidly to emerging problems.
• 8:29
  George Vlasto (Resolver): And the only way we can do that is by having teams who are enabled to be pretty entrepreneurial, actually in their own areas.
• 8:35
  George Vlasto (Resolver): obviously, you know, within a structure.
• 8:37
  George Vlasto (Resolver): So my, basic job is to make space for, our engineering teams or our risk detection teams or our human intelligence specialists to focus on what they need to, to have the tools at their disposal, to have the resources and the sort of framework to operate within, which enables'em to respond really quickly on behalf of our clients and partners to these problems, which can come out of nowhere.
• 8:56
  George Vlasto (Resolver): You know, we can see these things, particularly as harm verticals merge together.
• 9:00
  George Vlasto (Resolver): We see different manifestations here.
• 9:01
  George Vlasto (Resolver): We sit with com networks at the moment.
• 9:03
  George Vlasto (Resolver): You need to be very, very agile to be able to respond to those kind of effectively.
• 9:07
  George Vlasto (Resolver): So that's really my job, is to try and create the, framework that enables agility.
• 9:11
  George Vlasto (Resolver): That's.
• 9:12
  George Vlasto (Resolver): Harder said than done, I think
• 9:15
  Ben Whitelaw: for sure.
• 9:15
  Ben Whitelaw: just explain to us what you mean now by Comm Network.
• 9:18
  Ben Whitelaw: 'cause there might be listeners who don't fully understand what that means.
• 9:22
  George Vlasto (Resolver): So I think this kind of emerging trend, this, blending of, nihilistic sort of ideology, suicide and risk extortion, sextortion in some cases.
• 9:32
  George Vlasto (Resolver): this really, these really nasty groups like 7, 6 4 coming to the fore.
• 9:37
  George Vlasto (Resolver): who.
• 9:38
  George Vlasto (Resolver): Don't fall neatly.
• 9:40
  George Vlasto (Resolver): No one falls neatly in any case, but doesn't fall in turn of the traditional harm verticals that most of our partners and most of the industry look at.
• 9:46
  George Vlasto (Resolver): You know, whether it's child safety or.
• 9:48
  George Vlasto (Resolver): sextortion or suicide and self-harm, and they, cut across, they're using new techniques.
• 9:53
  George Vlasto (Resolver): They're sometimes interoperable with organized crime and money laundering.
• 9:57
  George Vlasto (Resolver): These are really, really complex challenges that have emerged very rapidly and even have kind of major law enforcement agencies kind of on the back foot trying to respond to them.
• 10:05
  George Vlasto (Resolver): They sit somewhere outside of the kind of regulatory framework in some cases.
• 10:10
  George Vlasto (Resolver): and a lot of our partners Are extremely keen, obviously, to understand how these groups operate, how they manifest on their platforms, the kind of behaviors they're using and the risks they post to their users.
• 10:18
  George Vlasto (Resolver): and that's one of those kind of examples where really that's a kind of almost a, you know, a kind of war room of people at Resolver who will come together, look at this from different angles, and try and develop a solution to the problem that enables our partners to get ahead of it before it starts to, really damage their wellbeing of the users on their platforms.
• 10:35
  Ben Whitelaw: Interesting.
• 10:36
  Ben Whitelaw: you mentioned needing to respond quickly to the problems and issues the partners bring to you.
• 10:42
  Ben Whitelaw: Can you give an example that kind of sticks in your mind from the time you've been a resolver so far, where that's been particularly stark or you've had to kind of move particularly quickly or, or to address, uh, how maybe the, you never saw coming.
• 10:56
  George Vlasto (Resolver): Yeah, so I'll give you a couple examples.
• 10:57
  George Vlasto (Resolver): It's slightly different.
• 10:58
  George Vlasto (Resolver): So, so one was, we were working with one of our partners, on, suicide and self harm prevention.
• 11:04
  George Vlasto (Resolver): and detected by our technology and then, and then reviewed by analysts in pretty much real time, we saw an indication that a school aged, individual.
• 11:13
  George Vlasto (Resolver): was planning to go to their school and, at least according to their social media post, take their own life.
• 11:20
  George Vlasto (Resolver): We saw that in real time.
• 11:22
  George Vlasto (Resolver): obviously we alerted our partner to that.
• 11:24
  George Vlasto (Resolver): but we also in parallel alerted law enforcement in the jurisdiction to which this applied, who were able to intervene in that individual's kind of journey to school.
• 11:32
  George Vlasto (Resolver): and we were able to offer that person of psychological support, and ensure that they weren't able to, go through with their plan, which had had been sort of fueled by some of that.
• 11:41
  George Vlasto (Resolver): Exposure to material online.
• 11:43
  George Vlasto (Resolver): that's, you know, we actually see that relatively frequently, that kind of, very, very rapid response to a threat online where we can work with our partners, work with law enforcement or other kind of social organizations to intervene, to stop things happening.
• 11:56
  George Vlasto (Resolver): and it's obviously incredibly powerful when it happens and it kind of resonates through the whole business or people who know about it within the business.
• 12:01
  George Vlasto (Resolver): 'cause it's, it's such a.
• 12:02
  George Vlasto (Resolver): Such an acute demonstration of what we do, you know, particularly when it's at that individual scale.
• 12:07
  George Vlasto (Resolver): So that'd be one example.
• 12:08
  George Vlasto (Resolver): And then, and then another, which we had recently where I think we benefit sometimes when we work with our partners.
• 12:13
  George Vlasto (Resolver): 'cause we don't just look at our partners platforms, we also look around them, the ecosystem around them.
• 12:18
  George Vlasto (Resolver): And in one particular case, recently.
• 12:20
  George Vlasto (Resolver): was an individual who was posing a risk to children on our partner's platform.
• 12:24
  George Vlasto (Resolver): but their activity on our partner's platform was, it was policy violative, but not illegal.
• 12:30
  George Vlasto (Resolver): and they, they were able to circumvent some of the countermeasures that were put in place to try and stop them using the platform.
• 12:35
  George Vlasto (Resolver): But using our sort of off platform research, we could see this person actually posed a real world threat to children.
• 12:39
  George Vlasto (Resolver): Uh, it was quite clear they were engaged in illegal activity and soliciting illegal activity elsewhere.
• 12:43
  George Vlasto (Resolver): So we were able to work in coordination with our partner to pass that information we had to law enforcement in the location that that person was, was in.
• 12:51
  George Vlasto (Resolver): and then they were able to take appropriate action.
• 12:53
  George Vlasto (Resolver): So, you know, in one case, sort of helping somebody who was, Potentially a victim of, suicide self-harm or suicide self-harm material that they've been exposed to.
• 13:02
  George Vlasto (Resolver): And on the other, looking to take action, against a potential perpetrator of, child exploitation.
• 13:08
  George Vlasto (Resolver): so those kind of examples, you know, actually occur on a remarkably regular basis actually.
• 13:13
  Ben Whitelaw: And so you're kind of acting as a, as the kind of eyes of a platform or a partner.
• 13:18
  Ben Whitelaw: in other spaces on the internet perhaps where, that partner or that platform might not have visibility.
• 13:23
  Ben Whitelaw: Is that fair to say?
• 13:24
  George Vlasto (Resolver): Exactly.
• 13:25
  George Vlasto (Resolver): Yeah.
• 13:25
  George Vlasto (Resolver): and sometimes they may not even have the latitude to act on information, you know, elsewhere because they haven't collected it, they haven't seen it, but also because it sits outside of their ecosystem.
• 13:33
  George Vlasto (Resolver): Whereas, we can sort of act as that intermediary sometimes with, know, law enforcement or, social support agencies or something if, intervention is required.
• 13:42
  Ben Whitelaw: And so many of the stories that Mike and I talk on the podcast feel like they start on one platform, emanate to another, move to another.
• 13:50
  Ben Whitelaw: And so that, is, definitely an, a kind of thread in many of the stories we discuss.
• 13:54
  Ben Whitelaw: And as I'm sure you're aware, we're gonna talk today, George, in a bit more detail about Athena and about child sexual abuse material, which you touched on.
• 14:02
  Ben Whitelaw: And again.
• 14:04
  Ben Whitelaw: It crops up all too frequently in the podcast each week.
• 14:07
  Ben Whitelaw: Its kind of proliferation as an online harm has seemingly only kind of increased over time.
• 14:12
  Ben Whitelaw: I'd love your thoughts on the landscape, child sexual abuse material now and how that has evolved over time.
• 14:18
  Ben Whitelaw: Are we seeing it get worse in the way that it feels like?
• 14:21
  Ben Whitelaw: What's your sense?
• 14:22
  George Vlasto (Resolver): So, so I think in terms of scale, certainly, in terms of severity, I think that's hard to say.
• 14:28
  George Vlasto (Resolver): that's a somewhat subjective, call to me.
• 14:31
  George Vlasto (Resolver): But certainly in terms of scale it is, and the, the way it manifests, the, the different environments that we're seeing it on, increasingly.
• 14:38
  George Vlasto (Resolver): are really concerning.
• 14:38
  George Vlasto (Resolver): And, and the advent of generative AI clearly has, of supercharged the problem.
• 14:43
  George Vlasto (Resolver): and I think there's a misperception in some quarters that, CSUN that's generated using artificial intelligence is a kind of victimless crime.
• 14:50
  George Vlasto (Resolver): There is a huge body of, academic and, social research that suggests that is, that's a fallacy.
• 14:55
  George Vlasto (Resolver): And actually it is, you know.
• 14:56
  George Vlasto (Resolver): Equally problematic, and can lead to, further offending from, child predator communities and also, I mean, just fundamentally is treated in most jurisdictions as being, you equally illegal and egregious as, real world CS a.
• 15:09
  George Vlasto (Resolver): So yeah, it is definitely getting, bigger in scale.
• 15:12
  George Vlasto (Resolver): it's becoming, More, prevalent in more places on the internet, and I think that child predators or consumers of CSA, becoming more.
• 15:22
  George Vlasto (Resolver): Experts actually how they generate, disseminate, and crucially monetize this kind of stuff.
• 15:28
  George Vlasto (Resolver): and that's, that's really kind of one of the biggest changes over the last three or four years, and we've discussed this extensively with the likes of, uh, the IWF and in Hope, and other partners.
• 15:36
  George Vlasto (Resolver): How do we tackle this monetization of, child sexual exploitation material and, and CSA in particular because it's a growing problem and financial drivers, Only exacerbate what was previously a kind of or less entirely sexual crime.
• 15:49
  Ben Whitelaw: Yeah.
• 15:50
  Ben Whitelaw: listeners who, who may be kind of, don't come across child sexual abuse material and, and maybe aren't familiar with it, can you just talk about how platforms, and, at Resolver you would identify child sexual abuse material, if it kind of came through, a moderation queue or, or a system.
• 16:07
  Ben Whitelaw: 'cause again, I think there are a few important distinctions there.
• 16:10
  Ben Whitelaw: just to outline before we talk a bit about, how you are, you are using your Athena tool to, to mitigate it.
• 16:16
  George Vlasto (Resolver): Sure.
• 16:16
  George Vlasto (Resolver): so are kind of a, a number of basic ways to detect CS a materials.
• 16:20
  George Vlasto (Resolver): So the simplest, is human review.
• 16:23
  George Vlasto (Resolver): So a human being looking at an image and saying, this is CS A, that can either be a moderator like working within queues, for a platform or technology service or, increasingly user reporting.
• 16:34
  George Vlasto (Resolver): as more and more platforms have, implemented sophisticated user reporting mechanisms.
• 16:38
  George Vlasto (Resolver): But it ultimately revolts around a person looking at it.
• 16:41
  George Vlasto (Resolver): The second way, which is kind of fairly embedded now, across the industry, is what's called known CS a detection.
• 16:47
  George Vlasto (Resolver): So ASH matching.
• 16:48
  George Vlasto (Resolver): So any image that's submitted to.
• 16:50
  George Vlasto (Resolver): For example, ncmec or, the Internet Watch Foundation, or others is hashed.
• 16:55
  George Vlasto (Resolver): So it's given a digital signature.
• 16:57
  George Vlasto (Resolver): It's then shared, with partners.
• 16:59
  George Vlasto (Resolver): so whenever that image reappears in the ecosystem, a platform should be able to call out against hash database and get a confirmation that this image that's come through your platform is known to be.
• 17:09
  George Vlasto (Resolver): Child sexual abuse material, therefore it must be removed and reported, and so on and so forth.
• 17:13
  George Vlasto (Resolver): And then the final way, which hopefully will lead us onto a conversation about Athena is what's called unknown cs a detection.
• 17:18
  George Vlasto (Resolver): So use using machine learning technology at scale to, identify and detect previously unseen or novel child sexual abuse material, whether it's real world, cs a from, from real world abuse, or, csam that's been generated using artificial intelligence, but crucially, that's never been seen before.
• 17:35
  Ben Whitelaw: And my, my understanding, George, is that a lot of the kind of child sexual abuse material that, is on platforms is unknown or unmatched.
• 17:44
  Ben Whitelaw: And so there is a, you know, basically a kind of iceberg above the water of, you know, material that is known, is in the database and therefore is matched.
• 17:52
  Ben Whitelaw: But there's this kind of vast amount underneath the surface that, you know.
• 17:56
  Ben Whitelaw: Databases haven't seen before.
• 17:58
  Ben Whitelaw: Therefore, platforms can't detect.
• 18:00
  Ben Whitelaw: Can you explain what that means in practice for platforms and, and how that maybe changes how trust and safety teams deal with it and, and even, you know, refer to it to law enforcement as well?
• 18:09
  George Vlasto (Resolver): Yes, so I'd agree completely with what you just said, which is that the proportion of known versus unknown CS a is sadly heavily weighted in favor of unknown CS A.
• 18:18
  George Vlasto (Resolver): So the vast majority of Cs A that exists online has previously never been seen before, or certainly not recorded in a hash database.
• 18:26
  George Vlasto (Resolver): So effectively you have a real constraint on your ability to detect child sexual abuse material, either because you've only got a limited number of people that you can deploy to human review.
• 18:35
  George Vlasto (Resolver): You know, you can't human review every single image or.
• 18:38
  George Vlasto (Resolver): item on your, platform at, tens of billion, hundreds of billions of items scale.
• 18:42
  George Vlasto (Resolver): and if you're just relying on known csam hash matching, you are only gonna be catching a very small proportion, as you say, that top of the iceberg that's been seen before.
• 18:49
  George Vlasto (Resolver): Mm-hmm.
• 18:50
  George Vlasto (Resolver): and even relatively small changes to those images, you know, the signature of those images can throw known csam detection as well.
• 18:56
  George Vlasto (Resolver): So, at the moment, or historically it's been a really significant challenge for platforms and for law enforcement, to detect novel CSA and also to be able to differentiate between the different grades or different levels of CSA, the most severe, which might indicate an imminent harm or ongoing harm to a child that therefore law enforcement may want to prioritize their efforts in, targeting or, or investigating.
• 19:19
  Ben Whitelaw: And just explain for listeners, George, if you, can, that those gradings, cause I was almost unaware that there were different types of child sexual abuse material in terms of severity.
• 19:28
  Ben Whitelaw: And, again, that's helpful I guess as a, a kind of.
• 19:31
  Ben Whitelaw: When we come to talk about the now
• 19:33
  George Vlasto (Resolver): Yeah.
• 19:33
  George Vlasto (Resolver): And I'll be a bit cautious about how I describe it because I don't want to, go sort of too far into the dark heart of this for your listeners, but effectively that in most jurisdictions, child sexual abuse material is broken down into different categories.
• 19:44
  George Vlasto (Resolver): in the United States category 1, 3 3 in the United Kingdom and other jurisdictions, it's a to c, C being the least.
• 19:52
  George Vlasto (Resolver): egregious category that's still sexual content involving a minor, but it's the least egregious category A is the most egregious.
• 20:00
  George Vlasto (Resolver): And I won't describe exactly what those categories contain, right?
• 20:03
  George Vlasto (Resolver): But I, imagine with a leap of imagination, your listeners can, uh, work that out.
• 20:07
  George Vlasto (Resolver): But, category A is, yeah, is generally speaking, the work that is, generally treated as the most egregious, and that tracks the most severe criminal penalties.
• 20:16
  George Vlasto (Resolver): if you are, Obviously creating it or sharing it or disseminating it in any way.
• 20:20
  Ben Whitelaw: so let's go onto Athena now.
• 20:22
  Ben Whitelaw: I mean, this is a new tool that Resolver, is bringing to the market.
• 20:26
  Ben Whitelaw: how did it come about?
• 20:27
  Ben Whitelaw: What does it do and how does it differ from those other kind of technologies that exists right now and which platforms utilize?
• 20:34
  George Vlasto (Resolver): so again, we.
• 20:35
  George Vlasto (Resolver): We thought about this from the perspective of the problem rather than the kind of product, as I was describing earlier.
• 20:40
  George Vlasto (Resolver): So the problem for anyone trying to develop an unknown CSA classification tool is that in order to develop a computer vision model, you ideally have access to a database of images.
• 20:52
  George Vlasto (Resolver): So if you are training a machine learning classifier to differentiate between different breeds of dogs, for example, you would feed the machine learning model.
• 20:59
  George Vlasto (Resolver): however many millions of pictures of a Labrador versus a, poodle until the model could determine between the two of them.
• 21:05
  George Vlasto (Resolver): it is illegal to hold a database of child sex abuse material.
• 21:10
  George Vlasto (Resolver): Therefore, you can't apply, in most cases, that methodology to training a c sound detection model.
• 21:14
  George Vlasto (Resolver): There are technical ways around it so you can, train a model.
• 21:19
  George Vlasto (Resolver): To recognize similar signatures of images, so you can train a model based on those hash databases I was talking about before, and look for what's called an embedding model.
• 21:27
  George Vlasto (Resolver): Look for kind of similarities in signature once the image has been hashed, but it's relatively imperfect that model.
• 21:33
  George Vlasto (Resolver): And it's not a true computer vision model.
• 21:35
  George Vlasto (Resolver): So we knew this was a huge and growing problem.
• 21:38
  George Vlasto (Resolver): The proliferation of CS a online, the technology that was being deployed against unknown CS a was, not as effective as it could be because of this kind of legal and ethical constraint around CSA database images.
• 21:50
  George Vlasto (Resolver): so we.
• 21:50
  George Vlasto (Resolver): Decided we were going to try and find a way to solve that problem.
• 21:54
  George Vlasto (Resolver): and we therefore partnered with, a company called R Manor Research, who are our partner in delivering this, who do a lot of work with, in this case, the UK government.
• 22:04
  George Vlasto (Resolver): and we discovered that the UK government held a database called Cade, the Child Abuse Imagery Database, to which law enforcement and organizations such as the IWF contribute images, and it's a off system.
• 22:16
  George Vlasto (Resolver): Database.
• 22:16
  George Vlasto (Resolver): It's not connected to the internet.
• 22:18
  George Vlasto (Resolver): You need to have a very high, level of security clearance to get anywhere near it.
• 22:21
  George Vlasto (Resolver): But we were able to work with Broke and the Home Office to get access to that database to train a true computer vision model on the world's most highly curated database of child sexual abuse imagery.
• 22:32
  George Vlasto (Resolver): Which seems like an odd thing to say, but it's, it's the sort of highest quality database that exists in the world because it's all been reviewed so many times.
• 22:40
  George Vlasto (Resolver): And as a result, we were able to, spend.
• 22:42
  George Vlasto (Resolver): Considerable amount of time and development effort to to train what is by a considerable margin, the most accurate detection system for unknown child sexual abuse because it's actually being trained on real images of child sexual abuse under a very strict ethical and legal.
• 22:59
  George Vlasto (Resolver): framework governed by the UK home office in this case.
• 23:02
  George Vlasto (Resolver): So the uk interior ministry for some of your listeners.
• 23:05
  Ben Whitelaw: Interesting.
• 23:06
  Ben Whitelaw: We'll, we'll come onto the testing that you've been doing with Athena, but I'm interested in that kind of security aspects.
• 23:12
  Ben Whitelaw: You know, access to Cade is not something that gets given to many partners.
• 23:17
  Ben Whitelaw: I'm sure.
• 23:17
  Ben Whitelaw: I'd, I'd love just to kind of know.
• 23:19
  Ben Whitelaw: How that went, how that came about, and also how you made sure to kind of protect your teams as they had access to this, from a kind of personal standpoint, but also obviously from a security perspective as well.
• 23:31
  George Vlasto (Resolver): Well, so, so those people who had access to Cade and retain access to Cade on an ongoing basis to make sure the model is retrained regularly, all have very high level of, government security clearances.
• 23:42
  George Vlasto (Resolver): So they have to have kind of passed what's called a vetting process to be allowed to go anywhere near that database.
• 23:47
  George Vlasto (Resolver): That's not everyone in Resolver, or at Rogue, but a small number of people are able to do that.
• 23:52
  George Vlasto (Resolver): but even those people don't.
• 23:54
  George Vlasto (Resolver): Have to look at the images.
• 23:56
  George Vlasto (Resolver): so they're able to access the data to train the machine learning model, but they don't have to look at the images because fortunately for us, and unfortunately, I guess for child protection, police officers and the IWF, other people who are, Extremely expert in this have already looked at those images and graded them.
• 24:13
  George Vlasto (Resolver): so our team don't actually need to view the images themselves.
• 24:17
  George Vlasto (Resolver): They just need to have access to the data to feed through the model.
• 24:19
  George Vlasto (Resolver): And it's actually only the model that looks at the image, not our staff.
• 24:23
  George Vlasto (Resolver): but it's a pretty complex.
• 24:24
  George Vlasto (Resolver): Ethical and legal, process to go through.
• 24:26
  George Vlasto (Resolver): And obviously it has to be done on a very, very strict, guidelines and supervision, to ensure that, you know, nobody is exposed to data that they don't need to see of the most egregious type, and that no one who shouldn't be seeing or having access to this data is, is allowed to, it's, yeah, it's probably the most complicated commercial.
• 24:42
  George Vlasto (Resolver): Pathway that I've ever had to sort of chart through.
• 24:45
  Ben Whitelaw: Yeah, I'm sure.
• 24:46
  Ben Whitelaw: many, many sign offs.
• 24:48
  Ben Whitelaw: so, so to kind of summarize, I guess what Athena does, it provides that categorization of CS a in a way that has previously been very difficult to do, with the data that's been available.
• 24:59
  Ben Whitelaw: What's your expectation of what that will allow platforms to do once they have access to that?
• 25:04
  George Vlasto (Resolver): so when we say Athena, so there's, there's two sort of parts to this.
• 25:08
  George Vlasto (Resolver): So there's Athena, as the unknown CSUN detection model, which is based on a machine learning model called Vigil, vigil ai for platforms that already have a sophisticated trust and safety workflow and known CS sound detection in place and human moderation in place, effectively, it becomes just a plugin in the middle of that.
• 25:23
  George Vlasto (Resolver): So you can run your data through a known CSA detection database, which is still valuable.
• 25:27
  George Vlasto (Resolver): And then anything which comes back, you can obviously take as read that it's CSA, or at least it's been flagged as Cs a.
• 25:32
  George Vlasto (Resolver): Anything that doesn't, appear on that database, you can run through, Athena, for an indication that it might contain unknown CSA.
• 25:39
  George Vlasto (Resolver): And then there's, potentially a human review element at the end of that and reporting requirements to EC and other, uh, reporting bodies.
• 25:46
  George Vlasto (Resolver): So that's for sort of sophisticated trust and safety teams.
• 25:48
  George Vlasto (Resolver): That's the kind of most likely use case, either through an API call to our infrastructure, or in certain cases we can deploy this on-premises for the kind of biggest, technology companies if that's what they require.
• 25:58
  George Vlasto (Resolver): For smaller outfits or, or people who, or companies that are thinking about this perhaps for the, for the first time, and don't have those sophisticated workflows.
• 26:07
  George Vlasto (Resolver): Athena as a whole can provide all three components of that.
• 26:10
  George Vlasto (Resolver): So effectively, if you have, if you are grappling with the challenge of Csam on your platform and you have no existing measures in place, we can provide for you in a single workflow, known cs, a detection, unknown csam detection, and even human review of that csam and reporting, at the end of it.
• 26:24
  George Vlasto (Resolver): So for a platform that.
• 26:25
  George Vlasto (Resolver): Wants to deal with the problem, but doesn't have the resources or potentially the kind of, the wherewithal to set up their own workflows, that they can essentially outsource it to us and we will take care of that entire problem for them to the highest possible standard that exists at, at the moment, you know, with the technology that's available to us.
• 26:42
  Ben Whitelaw: Yeah.
• 26:43
  Ben Whitelaw: Interesting.
• 26:44
  Ben Whitelaw: you mentioned, George, that you've been testing with a number of different kind of platforms so far.
• 26:48
  Ben Whitelaw: You, you talked about the accuracy of Athena, and the models that underpin it.
• 26:53
  Ben Whitelaw: Can you talk a bit about that accuracy relative to, other technologies that are available and also kind of what you've learned in deploying, the models in these new scenarios.
• 27:04
  George Vlasto (Resolver): Sure.
• 27:04
  George Vlasto (Resolver): the kind of detailed accuracy scores for vigil, are slightly sensitive.
• 27:09
  George Vlasto (Resolver): So if, any of your listeners are interested in finding out more about it, then please do get in touch and we can share those details under sort of legal protection.
• 27:16
  George Vlasto (Resolver): But in principle, when, Athena is deployed kind of in the wild or in a live environment with our, with our, customers and clients, it achieves a level of accuracy as close to.
• 27:26
  George Vlasto (Resolver): perfect as you can really achieve with like a machine learning model.
• 27:30
  George Vlasto (Resolver): However, your second part of your question was how do we, how have we learned that this works in a live environment?
• 27:35
  George Vlasto (Resolver): And it depends on what a platform or a technology service is looking to achieve.
• 27:40
  George Vlasto (Resolver): Because with any machine learning model, there are two variables.
• 27:43
  George Vlasto (Resolver): There are basically two dials that you can turn.
• 27:45
  George Vlasto (Resolver): One is called precision, and the other is recall.
• 27:48
  George Vlasto (Resolver): If you dial up precision to the highest level, you will get almost no false positives, but you may miss some edge cases, particularly AI generated imagery that might not be sort of normal imagery that may involves like non humanoid characters, for example, and things like that.
• 28:06
  George Vlasto (Resolver): Precision set at a very high level, you may miss some of those edge cases.
• 28:09
  George Vlasto (Resolver): But you will get pretty much no false positives.
• 28:12
  George Vlasto (Resolver): If you turn up recall, we'd likely to get higher number of false positives.
• 28:15
  George Vlasto (Resolver): Still not a huge number, but a higher number of false positives.
• 28:18
  George Vlasto (Resolver): But you will miss like none of those edge cases.
• 28:20
  George Vlasto (Resolver): So effectively what we say to our clients is like, what, what are you aiming for?
• 28:24
  George Vlasto (Resolver): You're aiming for incredibly high precision, perhaps'cause you've got vast quantities of data.
• 28:29
  George Vlasto (Resolver): I false positive rates cost you a huge amount as a business to review and to triage.
• 28:35
  George Vlasto (Resolver): So you turn up precision or are you looking to remove any hint of child sexual abuse material from your platform, in which case you prioritize recall.
• 28:42
  George Vlasto (Resolver): And then the other thing which we've learned is that that categorization, from A to B2C.
• 28:48
  George Vlasto (Resolver): is really important because it allows focused attention, referrals to law enforcement, alerting to NCMEC and other organizations to happen, to the most egregious harm really, really fast.
• 28:58
  George Vlasto (Resolver): And it enables our clients to root that kind of content to very highly trained and protected teams.
• 29:06
  George Vlasto (Resolver): 'cause one of the other problems is obviously Cs a being viewed by.
• 29:09
  George Vlasto (Resolver): Normal moderators who don't have psychological support around them is incredibly traumatic.
• 29:13
  George Vlasto (Resolver): So if you get a really clear indication at the beginning that this is a deeply egregious, image, you can route it in the right way.
• 29:19
  George Vlasto (Resolver): You can treat it with the urgency it requires and the sensitivity that it requires.
• 29:23
  George Vlasto (Resolver): so it's seen by the right person really quickly, and then action is taken as fast as possible.
• 29:28
  George Vlasto (Resolver): So those are the kind of principle things that we've learned as we've kind of rolled it out.
• 29:31
  George Vlasto (Resolver): And, and do you
• 29:32
  Ben Whitelaw: expect George, as maybe platforms increase the dial on recall, in order to comply with kind of regulations and, and obviously protect children and, users that are kind of vulnerable, do you expect that false positive, increase to be.
• 29:48
  Ben Whitelaw: Also an issue that platforms have to, deal with as well.
• 29:51
  Ben Whitelaw: 'cause that's something we see in other harms as, as an ongoing problem is, is that that spike and maybe cause positives.
• 29:58
  George Vlasto (Resolver): I actually think this is a question for regulators as well as for platforms, because most regulation, all the regulation is really predicated on the idea that platforms and technology service providers should take all reasonable efforts to find and mitigate harm.
• 30:12
  George Vlasto (Resolver): If by setting your recall aperture so wide, it's Athena or any other type of machine learning detection system, you are generating a huge amount of noise in your system and you are incurring unreasonable cost or unreasonable, interferences with user experience.
• 30:25
  George Vlasto (Resolver): Clearly that's not.
• 30:26
  George Vlasto (Resolver): That's not in the spirit of the regulation.
• 30:28
  George Vlasto (Resolver): So I think there's a dynamic conversation that needs to take place between technology companies themselves and the regulators about, you know, what is reasonable.
• 30:35
  George Vlasto (Resolver): And I think it'll be context dependent.
• 30:37
  George Vlasto (Resolver): And certainly, you know, when you listen to Ocom or you listen to the European Commission or the Australian Needs Safety Commission's Office or other global regulators, there is a real sense that, discretion is, is required.
• 30:47
  George Vlasto (Resolver): There's no one size fits all, you know, for a very, very large online platform or a technology service provider.
• 30:53
  George Vlasto (Resolver): Those thresholds might be different than for a kind of emerging technology platform that perhaps is catering ex, you know, explicitly for children, for example.
• 31:00
  George Vlasto (Resolver): So I think regulators will take a reasonable approach to it, and I think each platform will have to determine for themselves, you know, what reasonable looks like, in that context.
• 31:08
  Ben Whitelaw: Yeah, most, Um.
• 31:10
  Ben Whitelaw: George, it's fascinating to hear a bit about Athena.
• 31:12
  Ben Whitelaw: I I wanted to kind of broaden out before we wrap up today, around some of the in blog posts that Resolve has put out over the last few weeks as it's celebrated its 20th birthday.
• 31:22
  Ben Whitelaw: one of them talks about kind of predictive online safety technology, and I wondered whether you thought in the context of our discussion today, whether we're moving towards a future where models can actually anticipate harms before they occur.
• 31:35
  Ben Whitelaw: is there a world.
• 31:36
  Ben Whitelaw: you can imagine that happening.
• 31:38
  Ben Whitelaw: I,
• 31:38
  George Vlasto (Resolver): think in certain cases, maybe, I mean, I guess it depends on how you, like, how philosophical you get about what AI is designed to do.
• 31:46
  George Vlasto (Resolver): But I mean, especially even the most sophisticated AI models are statistical probability models.
• 31:52
  George Vlasto (Resolver): so they're, their expertise lies in predicting the most likely next thing at the end of the day, like at its most simple level.
• 31:59
  George Vlasto (Resolver): And obviously what we know from, online safety, but you know, actually outside of just kind of digital safety in the world in general, sometimes the most damaging things are those that are unpredicted, the kind of black swan events that, that come along, or the, the really unexpected, evolution in human behavior or interest or misuse for technology.
• 32:17
  George Vlasto (Resolver): So I do think that AI models will get better at, learning and classifiers, generally speaking, will, you know, be closer to the margin.
• 32:26
  George Vlasto (Resolver): Until we reach a, depending what, what your viewpoint is, a halian or terrifying future of general artificial intelligence.
• 32:33
  George Vlasto (Resolver): I think that that kind of human guidance of, where AI is focused, the training data sets that they're, predicated on and that kind of empathic like human understanding of how people are actually behaving in real life will remain a really, really important part of the mix.
• 32:49
  George Vlasto (Resolver): so.
• 32:49
  George Vlasto (Resolver): Obviously we'll see incremental gains in ai, but I don't think in any foreseeable, future, I mean unforeseeable in tech terms, it's like one year.
• 32:59
  George Vlasto (Resolver): Uh, I don't think AI will ever be a panacea for, trust and safety teams.
• 33:03
  Ben Whitelaw: Yeah.
• 33:03
  Ben Whitelaw: Fascinating.
• 33:04
  Ben Whitelaw: that's a fairly reassuring and hopeful note to end our conversation on.
• 33:08
  Ben Whitelaw: George.
• 33:09
  Ben Whitelaw: really enjoyed talking to you today.
• 33:11
  Ben Whitelaw: we'll share some of the resources and, and the link to Athena and also the blog post that Olba has put out over the last few weeks in the show notes of today's episode.
• 33:20
  Ben Whitelaw: just wanna say thanks so much for your, your time, today and for sharing a bit more about what Silva does and what Athena, is set up to do.
• 33:26
  Ben Whitelaw: Really excited to see how that goes.
• 33:28
  George Vlasto (Resolver): Thanks Ben.
• 33:28
  George Vlasto (Resolver): It's great to talk today and thank you for the opportunity to, to come on the podcast and talk a little bit about Resolver.
• 33:33
  Ben Whitelaw: Brilliant.
• 33:34
  Ben Whitelaw: Take care.
• 33:34
  Ben Whitelaw: Thanks.
• 33:37
  Announcer: Thanks for listening to Ctrl-Alt-Speech.
• 33:39
  Announcer: Subscribe now to get our weekly episodes as soon as they're released.
• 33:43
  Announcer: If your company or organization is interested in sponsoring the podcast, contact us by visiting ctrlaltspeech.com.
• 33:50
  Announcer: That's CT RL alt speech.com.