Latest / The Road to Accountable AI / Henry Ajder, Latent Space Advisory: Deepfakes and the Crisis of Digital Trust

Transcript

Henry Ajder, Latent Space Advisory: Deepfakes and the Crisis of Digital Trust · The Road to Accountable AI

0:09
Kevin: Hi, I'm Kevin Warbeck, professor of Legal Studies and
0:11
Business Ethics at the Wharton School of the University of Pennsylvania.
0:15
For decades, I've studied emerging technologies from broadband to blockchain.
0:19
Today, AI is promising to transform our world, but AI needs accountability
0:26
mechanisms to ensure it's developed and deployed in responsible,
0:29
safe, and trustworthy ways.
0:32
On this podcast, I speak with the experts leading the charge for accountable ai.
0:41
We are now beyond the point where you can be confident that any
0:44
content you see or hear is real.
0:47
How can governments and the private sector address the deepfake explosion?
0:52
My guest on this episode is Henry Eider, one of the world's foremost experts
0:55
on DeepFakes and synthetic media.
0:58
We discuss the evolution of deep fakes efforts to address harmful activity,
1:02
technical responses, and how organizations developing or deploying AI tools
1:06
should act in light of these dangers.
1:11
Henry, welcome.
1:12
Thank you so much for joining me on the Road to Accountable ai.
1:16
Adjer: Thank you, Kevin.
1:16
No, it's great to be with you today.
1:19
Kevin: Most people have only become aware of the magnitude of the deepfake
1:24
problem in recent years, but you've been studying this for a long time.
1:27
You published a report on DeepFakes in 2019, which was
1:30
years before chat GPT came along.
1:33
Can you give us a, just a general picture about how the, the landscape of DeepFakes
1:38
has evolved over that period of time?
1:41
Adjer: Yeah, sure.
1:41
I mean.
1:42
You know, I describe my work broadly as kind of doing deepfake and
1:46
synthetic media cartography, right?
1:48
I've kind of been mapping this landscape more or less since it first emerged.
1:51
As you mentioned, I published the first report really kind of providing
1:55
something, almost a deepfake census as to how this technology was evolving,
1:59
the quantity of, uh, deepfake content out there, who was targeting.
2:03
Um, and primarily that was really to try and actually kind
2:06
of cut through the speculation.
2:07
You know, in 2019, late 2018, there was a lot of speculations to the impact, but
2:12
there wasn't much in a way of evidence to back up that speculation of its impact
2:15
on elections or cybersecurity and so on.
2:18
So yes, done a lot of work over the years, kind of mapping that landscape
2:21
with, you know, different reports.
2:23
Um, which primarily actually pointed to the fact that image abuse against women
2:27
at the time was the biggest challenge.
2:30
Um.
2:31
What has changed and what has not?
2:33
I think this is one of the things that is really important for
2:36
people to grasp, which is that the way I talk about DeepFakes
2:40
is typically along three metrics.
2:42
You could technically push it to four, but it's realism of the
2:45
outputs that can be generated.
2:47
It's the efficiency of the models, both in terms of compute and data for generation
2:51
and its accessibility of the tools in terms of how easy it is to actually use
2:54
them and access them in the first place.
2:56
Functionality would be a technical force in terms of what the models can now
3:00
actually do, not just the realism of what they can generate as well, but, but
3:04
that has changed dramatically, right?
3:06
I mean, that has been.
3:07
A real, um, frenetic space, particularly in the last two, three years, say, um,
3:14
realism has gone to a point where in most contexts with state-of-the-art
3:18
models, it's basically a coin flip when it comes to human discrimination
3:22
of AI generated and authentic outputs.
3:24
That's across audio images and increasingly starting
3:27
to become video as well.
3:28
Um, the efficiency of these tools that used to be that you needed, you know,
3:32
hundreds of neatly aligned and cropped images to train a model, to create a
3:36
face swap that looks pretty nightmarish.
3:38
We're now talking about one image being used to drive an entire
3:41
AI generated avatar, which is incredibly realistic, right?
3:45
Um, and the computational resources likewise have decreased.
3:48
Um, a lot of these can now, um, a lot of these tools for creating
3:51
deface can now run on device or indeed on cloud for, you know, for.
3:56
You know, pennies or cents on the dollar.
3:58
Right.
3:58
Um, and then accessibility.
4:00
I think that's the biggest factor that's dramatically changed is tools
4:04
that, you know, six, seven years ago would've been the dream of Hollywood
4:08
Studios to even begin to have access to somebody's capabilities.
4:11
Right.
4:11
Are now available on the laptop, on your desk, or the smartphone in your pocket.
4:15
Um, so those areas have really dramatically changed.
4:17
Mm-hmm.
4:18
The incidence of DeepFakes being used maliciously as well has also changed,
4:23
um, in terms of the volume of attacks and the volume of, um, people being targeted.
4:29
Um, so it used to be that we were talking about maybe hundreds
4:32
of people, particularly in the fraud context and so on.
4:34
C-suite top executives, politicians, um, figures in government.
4:40
It's now targeting private individuals, everyday people.
4:43
Um, so the volume of attacks has grown and the efficacy of
4:46
the attacks has also changed.
4:47
It's no longer the case that it's quite as easy to catch them.
4:51
Um, the three main vectors that I talk about is deception, doubt, and de uh,
4:55
de degradation or, or degrading people.
4:57
Um, you know, the impact of that has.
5:00
What hasn't changed though, Kevin, is that fundamentally the dynamics by which
5:04
harm are caused has remained the same.
5:06
It is those three same categories, and it is still about scamming
5:09
people, humiliating people, scaring people, and deceiving people.
5:14
Um, that hasn't changed.
5:16
Nor has the dynamic for how we tend to try and address them through
5:20
things like detection, provenance of content, and watermarking.
5:25
So lots has changed at the same time that the, the kind of, the core dynamics
5:29
hasn't really changed that much.
5:31
Kevin: I'm curious about that fourth potential category that
5:34
you mentioned, functionality.
5:35
Um, what might be changing in terms of what the models can do
5:39
beyond just the caliber of the, the fidelity of the images and videos?
5:43
Adjer: Sure.
5:43
So I mean, if you go back to say, 2019.
5:47
The talk of the town was what were called style gans.
5:50
Generative adversarial networks, GAN, um, style.
5:53
GaN was an algorithm developed by Nvidia and open source so anyone
5:57
could functionally access it.
5:58
It was on a commercial license, but bad actors don't tend to really care
6:02
too much about commercial licenses.
6:03
And, um, what this, uh, tool style GaN allowed you to do was to generate.
6:08
Pretty realistic face images, portrait style images of non-existent people.
6:14
So some of your listeners may remember a website called this person does Not Exist.
6:18
They may also be familiar with the kind of floods of fake LinkedIn
6:21
profiles that kind of hit the platform at that time using these style GaN
6:25
images as their profile picture.
6:28
Um, I have many of them people saying, I'd love to join your network from
6:32
clearly people that didn't exist.
6:34
Um, likewise it also did creep into some of the kind of disinformation sphere.
6:38
Um, a, um, an account, um, uh, linked to the supposed Hunter Biden dossier.
6:44
The person who leaked that, that was a style GaN image that
6:47
was the picture of that person.
6:49
This was something that had impact, but critically it was something
6:53
that had a very narrow impact.
6:55
This tool, as it had been trained, could only generate
6:59
pictures of realistic human faces.
7:02
Now also, you, you get to see enough of them and immediately
7:05
like style gall image.
7:06
I know that look right.
7:07
Um, but it was very narrow.
7:09
Now we're talking about tools, uh, platforms like Hicks Field ai, for
7:15
example, that contain just a multitude of really fine tuneable, um, tools that allow
7:22
you to do things like, you know, change certain regions of images to animate
7:27
them, you know, lip synchronization, entire face swapping, entire video
7:33
generation character changing.
7:35
Driving images, um, or rather, sorry, driving videos based on real images.
7:39
But with your facial movements, I mean, the amount of different tools that are
7:43
being released, you know, weekly by Higgs field in particular is astonishing.
7:48
So the functionality has just massively broadened.
7:51
There is just so many more permutations of what you can now generate, um, compared to
7:55
where we were before where you could get okay results, but they were very narrow.
8:00
Um, has really changed the game in terms of what we also now think of as possible.
8:04
It's kind of bled out into many more aspects of media in the digital world.
8:07
Mm-hmm.
8:07
And
8:08
Kevin: given the way AI technology evolves, though it was a safe bet
8:12
back then that the performance of these, uh, deep fake engines,
8:17
um, and image generation engines and so forth would improve.
8:20
Uh, and these were risks that people, you know, including you were highlighting
8:23
years ago, wa was there ever a moment where there was a real opportunity to, uh.
8:31
Severely mitigate the, the, you know, inappropriate and, and illegal
8:37
and, and harmful uses of DeepFakes?
8:38
Or, or was this sort of inevitable we get to this point?
8:42
Adjer: Yeah, I, Kevin, I dunno what's kind of worse here?
8:44
It's either the fact that, you know, that, that there was and we
8:47
missed it, or that there never was.
8:49
And I think it's more the latter to be honest.
8:50
I think, um, the dynamics that are in play when it comes to
8:55
DeepFakes are really challenging.
8:58
So one of the reasons it's challenging is that a lot of the tools that
9:01
are used to create malicious DeepFakes are open source, right?
9:06
These are not proprietary closed tools, um, that are hosted via
9:10
formal business operations and, um, and kind of play by the rules.
9:15
These are tools like the original face swapping tool that.
9:18
Term deep actually came from, was cobbled together from an open source library,
9:23
from Google of software called TensorFlow.
9:25
Um, and it was a weaponization of kind of disparate parts put together, so to speak.
9:30
The problem with this, right, is that if you're saying, well, look, we
9:32
need to try and remove this content.
9:34
We need to try and stop this stuff being created.
9:36
It's a bit like saying we need to try and regulate mathematics or
9:38
math as, as you guys say, right?
9:40
Um, it's something where it's really, really hard to stop people from
9:44
re-implementing and rebuilding systems.
9:47
To basically say anything that could potentially be weaponized now must be
9:51
restricted for the open source community.
9:53
That's something that, you know, just isn't feasible.
9:55
It's something that the EU AI Act kind of attempted to try and bring
9:58
in and got immediately shot down.
10:00
So the open source component is really hard.
10:02
It's really hard to restrict and, um, and kind of govern that, that ecosystem.
10:08
The second part is the, the, the kind of the way that the harm
10:11
spreads, which is online, right?
10:13
Primarily, um.
10:15
Jurisdictions and kind of operating across, uh, international lines on the
10:20
internet doesn't really make sense.
10:22
It doesn't really work in the same way.
10:24
And so even if a country has really, really comprehensive and strict
10:28
legislation prohibiting the sharing of certain content, making sure that labeling
10:32
is put in place, for example, of anything to say AI generated, um, as is being
10:36
discussed right now in India, South Korea, and so on, that's all well and good.
10:40
But when content is being posted from a random account in
10:42
Bulgaria or the UK or Mexico.
10:45
You know, it is much, much harder to actually, uh, legislate for that.
10:49
Um, also the scale of content now being generated and shared means
10:53
that the social platforms are also really struggling with it.
10:55
So this is, this is my way of basically saying there hasn't really been a moment
10:58
in my time, well, since the beginning of my time in this space, which is
11:02
really the beginning of the space.
11:03
It hasn't really been a moment where I've been like, we really missed the shot.
11:06
Kevin: Mm-hmm.
11:07
Adjer: What we did do, as is always the case though, Kevin, is we acted too late.
11:12
There was more we could have done in anticipation.
11:15
I've been having conversations with international governments for years, and
11:18
it has only been recently that they've started to act when I've been kind of
11:22
trying to encourage 'em to do so, you know, for a long time before then.
11:25
So there is still frustration about what could have been done, but I
11:28
don't think what could have been done would've been, you know, definitive
11:32
in, in solving or addressing a problem.
11:35
Kevin: And you mentioned, uh, a minute ago that, uh, when you looked at the
11:38
space in 2019, uh, non-consensual pornography, other kinds of, uh, attacks
11:43
on women were the largest category.
11:46
I, I, I think is, is that still the case?
11:49
Adjer: So it's really hard to give an answer to this with the same confidence
11:54
that I did that research in 2019.
11:56
The reason being, Kevin, that the landscape has, in some respects,
12:00
in terms of the volume of content out there, is just, it is.
12:03
It is almost like the grain of sand on the beach of today
12:06
in terms of where things were.
12:07
Um, one of the key findings from my stated DeepFakes report 2019, which
12:11
covered 2018 to 2019 in terms of the change in the landscape, was yes, that
12:16
96% of DeepFakes at that time were non-consensual pornographic content.
12:21
Um, but another finding was that the, the nu um, sorry, the amount of
12:24
defects online had almost doubled.
12:26
Um, and that's like, wow.
12:27
Um, that doubling was from around seven and a half thousand to 15,000, right?
12:33
I mean, now it seems almost quaint.
12:35
Um, you know, people are generating that much content a day within a
12:39
small business in some cases, right?
12:41
If you're a content agency, I mean, um, the scale is just so different.
12:45
And so to say, I've exhaustedly or exhaustively mapped this landscape,
12:50
and I can give you a definitive answer.
12:51
Is basically impossible.
12:53
And I think anyone who tells you that they can do that and they're not
12:56
extrapolating from or approximating is, is not being honest with you.
13:00
Um, that said, it is still my.
13:04
My perspective or my belief, um, the, the, the highest, um, harm level in terms
13:10
of number of victims still remains in that non-consensual image view space.
13:13
Mm-hmm.
13:13
Part of the reason for that, Kevin, is that yes, a lot of
13:16
this is being posted publicly.
13:18
Obviously at the start of, uh, this year, 2026, we saw a scandal.
13:22
With Grok undressing, um, images of women on X. Again, I wrote the
13:26
first report re investigating the commodification of ification apps
13:30
on Telegram, on, on a similar kind of, um, platform, uh, back in 2020.
13:35
Um, but obviously we've seen this in the public sphere, but it also happens at
13:39
scale in the private world, it's about the people who are doing this to people
13:42
on their phone and not sharing it as well.
13:44
So it's still my deep belief that the, the vast, um, you know, the vast
13:49
majority of victims of deep fake abuse.
13:51
Are predominantly women and it's predominantly of a
13:54
non-consensual sexual nature.
13:57
Having said that, the amount of people that are being targeted by the, um,
14:02
fraud components or being targeted by, um, disinformation campaigns and are
14:07
falling for AI generated content on their timelines in their daily life.
14:11
Has radically changed as well.
14:13
So depending on how you profile a victim and what degree of kind of
14:18
engagement that they have with AI generated content, you know, that,
14:21
that, that does have an impact.
14:23
But I think we can confidently say that.
14:25
You know, the fake image abuse against women is still a huge challenge to date,
14:29
irrespective of what the actual number is.
14:31
Kevin: Mm-hmm.
14:31
Well, so tell us about the work that you do now at, uh, with Latent Space Advisory,
14:36
um, and more broadly in this space.
14:40
Adjer: Yeah, sure.
14:40
So, I mean, my organization, latent Space Advisory is, is functionally just me.
14:45
It is my vehicle for working with a variety of different stakeholders who are.
14:51
Trying to navigate and then respond to this kind of new synthetic
14:55
reality that we are all living in.
14:58
I say new, relatively new given the span of history.
15:01
Um, although in the tech space, it feels like it's been a long, long time.
15:04
I feel old given how, how, um, how these eight years have flown by.
15:08
Um.
15:09
I work with governments a lot, so I'm working a lot with organizations
15:13
within governments or departments who are looking at legislation, looking
15:17
at what enforcement actions against malicious uses might look like.
15:21
Um, looking at trying to get the balance right between understanding the.
15:25
These tools are here to stay.
15:26
And there's a lot of kind of exciting, creative applications that are
15:29
out there whilst also recognizing the challenges I mentioned around
15:33
weaponization and, um, and kind of good governance and, and responsible design.
15:38
Um, so a lot of work with, with, with, you know, the government side of things,
15:42
um, particularly here in the uk but also in the eu and, and to some extent
15:46
less of, of recent, but in the US too.
15:49
Um, I do a lot of work with some of the bigger tech companies out there who
15:52
are developing some of these systems, um, and trying to again, understand
15:56
how to, how to best govern them, how to design them in the first place, what
16:00
release strategies look like, what kinds of policies need to be put in place.
16:04
For example, if you're a platform as well as a provider of the tools, um.
16:09
Big, big questions right now about things such as AI disclosure.
16:13
How should you disclose or ought users, um, disclose content?
16:17
What is deemed sufficiently deceptive?
16:20
How do we think about satire and critical art in the context of these platforms?
16:24
And hyperrealistic AI generated content as well?
16:27
Um, as well as looking at what is acceptable levels of kind of risk.
16:32
And, um, and what are some of the kind of ethical gray areas that emerge?
16:36
Yeah.
16:37
Um, I think one of the really interesting things in this space, Kevin and I wrote
16:40
a piece on this back in 2019 with the pretty bad name, I called it, the rise
16:44
of gray fakes really wasn't very catchy.
16:46
I'm not surprised it didn't catch on, but, but the, the argument I was
16:49
making at the time was, well, look, we've got all of these deep fakes,
16:52
which are explicitly clearly bad.
16:54
We've got non-consensual image abuse of women.
16:56
We've got cybersecurity.
16:57
Ving attacks, you know, voice phishing, cloning voices, impersonating people.
17:02
We've got state sponsored disinformation as well as just a
17:05
general disinformation deception.
17:07
Um, we've got people having their likeness stolen to perform in adverts and
17:12
films and content they never wanted to.
17:14
Um, there's a lot of explicitly, quite bad.
17:16
I think we can all kind of agree that that's not desirable.
17:19
Then there's also some pro-social applications.
17:21
I'm talking about, for example, the use of synthetic content in accessibility.
17:26
For example, cloning voices of people who've lost the ability to speak, um,
17:29
avatars that can be generated in real time, that sign language, um, to help
17:34
people with hearing impairments, right?
17:35
Some stuff in there that's pretty, pretty uncontroversially good, but
17:39
in the middle there's this huge swath of, of kind of applications, which,
17:42
Kevin: yeah,
17:43
Adjer: I think they unsettle us and they still unsettle us all these years later.
17:48
And that unsettling feeling is often hard to understand.
17:51
Well, is this just future shock?
17:53
Is this just something that's new and I'm not familiar with it and I
17:56
don't like it because it's something different and it makes me uncomfortable?
17:59
Or is it making me uncomfortable?
18:02
'cause there's a deeper kind of ethical intuition that's being disturbed,
18:06
um, and that I'm realizing, no, this isn't just uncomfortable because it's
18:09
new, it's uncomfortable because it's wrong or it's not, it's not okay.
18:14
And there's a huge amount of applications that fall into there.
18:16
I'm thinking.
18:17
Of, you know, the use of, um, synthetically resurrected
18:21
people, kind of, um, ai, ai, um, uh, tere as I kind of call it.
18:26
Um, you're talking about, you know, the use of, of kind of
18:29
non-existent people, avatars in films.
18:32
You might have seen a kind of thing around this Tillian Norwood character.
18:35
I think it's a bit of kind of empty hype at the moment, but I think the direction
18:38
of travel is clear towards, you know, synthetic labor in audio visual contexts.
18:43
Um.
18:44
Again, the satire point.
18:46
You know, what is good faith and acceptable uses of this technology
18:50
to clone people without consent?
18:52
Um, you know, are there acceptable use cases?
18:54
So there's loads of these.
18:56
Situations where this technology has just opened up things that we just don't fully
19:00
understand, both the longer term impact and how as a society we, we ought to feel
19:04
about it or we, or do we do feel about it.
19:06
So a lot of my work kind of helps to, um, bring those issues to people who are
19:13
invested in them, whether that's from a policy perspective, whether that's from a
19:15
product perspective, whether that's from a media perspective, to help them better
19:19
understand it and then make decisions, um, based on that better understanding.
19:25
Kevin: Yeah, it's a really interesting point and you know, as you suggested,
19:28
people have different intuitions and, and companies may have different cultures and
19:32
policies and views on how to handle this even, even if we're just talking about.
19:36
Uh, just the image generation tools or the image modification tools and,
19:40
you know, different communities feel differently about, uh, if I use an
19:44
AI enhanced profile picture, uh, whether that, you know, it's not
19:47
illegal, but is that legitimate or not?
19:50
So given all of that range, uh, and, and, and the novelty of
19:54
all these kinds of use cases.
19:56
You know, what is the pathway or what are possible pathways to actually
20:01
coming up with some measure of, uh, if not consensus, then some confidence
20:06
of organizations to draw those lines.
20:10
Adjer: Yeah, it's a good question.
20:11
So I think the consensus point is one that for a long time has been quite evasive.
20:16
I think there's been a lot of hesitancy, understandably, from government in
20:21
regulating with a heavy hand perhaps technologies, which, you know, many people
20:26
believe will, will form the foundation of a new world order functionally, right?
20:32
Um, and so from an economic perspective, there's hesitancy, but also from a kind
20:36
of an overstepping perspective too.
20:39
Um, and likewise with the public, I mean, um, I'm sure Kevin, you, you find yourself
20:43
in similar circles to me where you can end up in kind of AI echo chambers where
20:47
everyone is, is just talking about how amazing this technology is and how, you
20:51
know, this is gonna change the world for the better and they're so excited.
20:54
Um.
20:55
But you know, if you spend enough time in other spaces, you'll see just utter
20:58
horror and um, and, uh, just disdain for what AI is doing to creative
21:05
industries, to economic industries, to kind of slop ation of content and the
21:09
way that we interact with each other.
21:11
Um, and so there, you know, lots of different viewpoints,
21:13
that's not gonna change.
21:14
There is not gonna be a point with this, um, where all of a sudden
21:18
everyone reaches pure consensus, right.
21:20
Um.
21:21
Having said that, I do feel that I've seen over the last 18 months in
21:26
particular, but particularly the last year, um, a real change in whose voices
21:33
are pot potentially cutting through more.
21:35
Um, and I think that's.
21:36
As some of the hype has started to fade, um, and as, um, discontent
21:42
has started to grow louder, both because of the harms such as the, the
21:46
use of gr to synthetically stripped women or the deep faking of, of
21:50
various people in scams and fraud.
21:53
Um, or just because people aren't happy with the fact that every email they read
21:57
is clearly written by, by ai and it just sort of leads to a, a less enjoyable,
22:02
um, and less kind of human experience of how you interact with each other.
22:06
Um, and the result of that I think, has been growing.
22:09
Public backlash.
22:10
Yes.
22:10
But also there is started to become a more unified front internationally around.
22:14
Um, policy and legislation.
22:16
So, um, this year we've seen already India, South Korea, I believe Vietnam
22:21
as well, and the UK has started hinting at compulsory Montessori labeling, um,
22:27
of AI generated content, um, with China.
22:31
China, well, China.
22:32
China were, yeah, exactly right.
22:33
China were actually the first to really do this.
22:35
Yeah.
22:35
Um, the deep synthesis law that they had.
22:38
Couple of years back, um, closer to three now actually.
22:41
Um, which actually the EU is, uh, the EU AI Act was fairly closely
22:45
aligned with, um, on those points, but yes, you're absolutely right.
22:49
So, um, you're starting to see, you know, more and more com uh, countries moving
22:55
towards this position of like, okay, kind of enough, enough, like we need to.
22:59
Create some structure and some order here.
23:01
Um, and for me, that's, that's critical.
23:03
That's how you maintain information integrity in the synthetic age.
23:06
You need to mandate these kinds of actions.
23:10
And then that leads to perhaps some of the technological solutions or solution
23:14
approaches, um, which have been, you know, getting momentum, but not perhaps
23:19
enough proportionate to the scale of the challenge that that leads to them.
23:23
Then getting more attention and more resources and more adoption.
23:26
So, you know, I, I respect the, that there's differing views on AI
23:30
regulation and, um, I'm certainly not one to say that it's a clear cut, you
23:34
know, regulat it into the ground or let you know, let it all run free.
23:37
Um, but I feel that these are issues which there's been enough case studies
23:41
now, there are enough people who have felt it in their daily lives,
23:44
uh, in terms of getting fooled.
23:46
The, um, the, the, the unified front on legislation.
23:50
Even if it's not the best for legislation, I think it's, you know,
23:53
it is part of the process of iterating and, and getting to a better place.
23:56
So yes, we are starting to see more consensus on, on key issues, likewise
24:01
with children and, you know, the use of, um, chatbots, um, as kind of
24:05
romantic partners or as companions.
24:08
Um, that's something that in the US for example, has had bipartisan support.
24:11
Likewise with non-consensual image abuse and non-consensual pornography stuff.
24:15
That was the bill that Melania Trump put her name to.
24:17
Right.
24:18
Um, but was also spearheaded by, uh, A OC Alexandria Ocasio-Cortez.
24:22
So, you know, there are key points where there is agreement, but um, there's a lot
24:27
more that disagreement, but it is shifting towards, um, some kind of unified front.
24:34
Kevin: Let's talk a little bit about those technological responses.
24:36
There.
24:36
There are a variety of standards and tools for content provenance and watermarking.
24:43
And, and the two arguments that I always hear in response are there,
24:47
there's no technology that is perfectly robust against someone removing
24:52
the watermark or, or, or otherwise.
24:54
And you know, at some point if people are too credulous or they want to believe what
24:58
they see or they don't trust anything.
25:00
It's all fake news.
25:01
It really doesn't matter whether that technological capability is there.
25:06
What's your sense, uh, and and and thoughts about that?
25:10
Adjer: Yeah, it's, it is, it is, um, something that's taken up a huge amount
25:12
of my, uh, my, uh, headspace at the moment and is one of the primary issues
25:16
I'm working with from a government perspective in particular is precisely
25:20
this question of, look where we are now.
25:24
We are in a world where you cannot.
25:28
Say that a piece of media confidently is AI generated, um, or, or authentic,
25:34
um, based on a naked eye or ear, particularly as a lay person.
25:37
But increasingly, it's also difficult for, for experts, I guess, such as
25:42
myself to do that kind of analysis.
25:45
A lot of people talk about media literacy in this space and say, well, don't worry.
25:49
Here's the top five things that you can look out for.
25:51
Like, look out for the blinking, look out for the fingers, look
25:53
out for text in the background.
25:54
Look out for.
25:55
Um, you look, you know, look for disparities in detail between foreground
25:59
and background, blah, blah, blah.
26:00
Um, this to me is not just, um, not helpful, but can be actively harmful
26:06
because it gives people the wrong impression about their capabilities
26:09
relative to the capabilities of the generating, uh, tools and those companies.
26:14
So in good faith, we cannot tell people that their senses are any longer a
26:19
reliable judge of whether content is.
26:24
What do we tell them then?
26:25
Because that's a really scary premise, right?
26:27
If I come to you and say, Hey, Kevin, I just wanted to let you
26:29
know, mate, you can't tell what's real or fake anymore at all.
26:32
Good luck.
26:33
Um, that's quite unsettling.
26:35
Kevin: Yeah.
26:35
But
26:36
Adjer: it's the truth.
26:37
It is the truth.
26:38
It is the truth.
26:39
Um, the consequences of that, and if this, if the conversation ended,
26:43
there are effectively what I refer to as kind of epistemic nihilism,
26:46
like just nihilism about what we can know and what we can't know.
26:49
And that quite rightly, as you said, I think leads to a sense of.
26:53
There's no way I can know I'm gonna go with my gut.
26:55
I don't like that politician.
26:56
My friend would never say that.
26:58
Oh, she definitely would post a video like that, et cetera, et cetera.
27:01
Right?
27:01
Um, it leads to the biases rising to the surface, even in those
27:05
people who really would like to know definitively what is real or not.
27:09
So in response to this quite unsettling, uh, premise, we have a couple of
27:14
things that we can say, well, we can say, well, look, you can't
27:16
trust your eyes and ears anymore.
27:17
But, but don't worry because we've got these technological
27:20
solutions that can help us.
27:21
We've got deep fake detection tools that are basically able to spot things,
27:26
um, based on artifacts that have been detected in AI generated versus authentic
27:30
content in a way that the human.
27:31
Eyes and ears cannot.
27:33
We've got watermarking.
27:34
So we've got kind of good faith red flags planted in all of this content that
27:37
kind of stick out and tell the detection systems that this is AI generated, or
27:41
we've got kind of provenance systems, what we can talk about as digital
27:45
nutrition labels, you know, the, um, the standard C two pa, um, which is a
27:50
technical open standard for watermark, I'm sorry, watermarking for, for
27:54
providing, uh, secure metadata about how a piece of media has been created
27:58
and continues to be edited over time.
27:59
Right, so we can look at these and say, okay, don't worry.
28:02
You may not be able to tell with your eyes and ears that these standards are
28:05
gonna help us, or these technologies are gonna help us few problems.
28:09
Detection whilst it can be useful is never going to be of the categorical confidence
28:16
and reliability that I think most people would deem to be acceptable to make.
28:23
CI if at commercial.
28:28
If you benchmark and get somewhere in the region of 90%,
28:30
that's a really good school.
28:31
That's a really good school.
28:33
It also means that one in 10 pieces of media in, in the wild
28:36
context might get flagged as a false positive or a false negative.
28:39
And if you think about social media platforms where there's hundreds of
28:42
thousands of pieces of content being uploaded, you know, potentially even
28:44
by the hour, um, that's a lot of pieces of content being incorrectly flagged
28:50
or allowed to slip through the net.
28:53
Um.
28:54
The other thing with detection is of course, that it's an
28:56
adversarial dynamic, right?
28:57
It's a cat and mouse game.
28:59
Um, and so it's not like when you benchmark your tool, you can pat
29:02
yourself on the back and go away for six months and say, good job.
29:04
Our tool's at 90%.
29:05
The next day something could be released, which technically
29:08
could completely, um, undermine.
29:11
Your detection system and the confidence that you've just
29:14
communicated to people in its ability.
29:15
Mm-hmm.
29:16
Um, I know, I, I, I advise a detection company who are doing really great
29:20
work, um, and have humility to what their tools kind and can't do.
29:24
They are useful, but they are useful in the hands of trained individuals.
29:28
When you have people on x taking screenshots of free online detection
29:33
systems, which spoiler alert don't work well 'cause they're free, um,
29:37
that does more harm than good because again, it gives false confidence.
29:40
So detection has limited, um, uh, usefulness when used by
29:46
layperson, but can actually be harmful if not understood properly.
29:49
Watermarking, you know, yes.
29:51
That's something that, for example, Google Syn ID has been really
29:54
helpful for fact-checkers and for people doing forensic analysis.
29:58
Um, those watermarks compared to some of the previous ones are much
30:01
more robust so they can survive high levels of compression post-production,
30:07
you know, cropping and changing of the size of the image, for example.
30:11
Um, but they're not perfect.
30:13
Um, and what they often can't do is tell you necessarily how a piece of
30:17
content might have been edited or generated, or if it has been generated.
30:21
Let's say I generate a picture of a burning building.
30:24
Um, and then in a very small part of the image I introduce what looks like
30:27
an Israeli fighter jet or an American fighter jet or an Iranian fighter jet.
30:31
Given the current ongoing situation, um, that we have in the Middle East, um, you
30:36
know, some, some watermarks will persist.
30:40
In a way where you can introduce elements which won't be captured, um, or will be
30:45
treated as authentic, even if it's only a small portion of the image being changed.
30:49
Um, so again, not perfect useful, but not perfect.
30:52
Also, at the moment, synth ID is only really being allowed to be used in Google
30:56
products, so it's not that wide to spread.
30:59
And then with content provenance, technically it's the most secure.
31:02
It's the one that is the most kind of, um, from an architecture perspective,
31:06
it's the best designed in my opinion.
31:08
Um, you know, if you try and change the metadata, you break the seal,
31:12
you lose your digital nutrition label, you lose your content
31:15
credentials, they're called, um.
31:18
Problems with provenance though is explainability.
31:20
So how can you meaningfully explain to someone how a piece of content has
31:24
been edited and changed over time?
31:26
Um, think about, for example, the famous image of the shark on the highway, um,
31:31
which is often posted in Flo around Florida hurricanes and stuff like this.
31:34
Um, that's a really old image at this point.
31:37
It's been shared reedited and changed a lot.
31:39
And so the ledger, so to speak, of that piece of media is gonna be huge.
31:44
How do you distill that in a way that allows people to get the essence of
31:47
what they need to know, um, in a moment?
31:50
And the same goes for AI generated content.
31:51
If I say, Kevin, this video you're watching of me right now is actually
31:55
a deep fake, is AI generated?
31:57
We might be saying, well, is it just the lip movements?
31:59
Is it the background?
32:00
Is it my hair?
32:01
You know, is it, is it my entire face?
32:03
Is it my entire body?
32:05
Um, it can be really hard to communicate that effectively.
32:08
So this is all to say that the solution approaches that we have in response to a
32:11
difficult question are not perfect, and they aren't at a level of categorical
32:17
trust that perhaps we are used to.
32:21
And so that means that there are lots of people who, including myself,
32:24
who don't feel that at the moment.
32:26
We really have that satisfactory answer to the epistemic nihilism, to the
32:30
uncertainty to provide reassurance.
32:33
So we're not in the best spot right now.
32:35
My hope is that as people become more aware of how challenged our information
32:41
systems now are, that there will be rising, uh, recognition of the need
32:46
for more work, and we'll get to a better place with those solutions.
32:49
Kevin: So that leads to my last question, given that we're not in an ideal spot,
32:53
uh, with regard to all those elements, as you said, there's been a lot of
32:57
activity by governments in around the world, but we, we still don't necessarily
33:02
have the kind of legal frameworks we need for someone who is, um, involved
33:07
in developing or deploying AI systems.
33:11
And let's, let's put aside the.
33:12
The frontier model companies or the companies that are actually
33:15
doing image generating tools.
33:16
But you know, if you're someone who is just, uh, you know, thinking
33:19
about the impact that this deepfake phenomenon has more broadly on you and
33:24
on people's perception of ai, what's, what's the best thing for someone
33:28
like that to do in this environ?
33:33
Adjer: It is a good question.
33:33
I mean, I think one thing I'd start by saying is that I think in this space
33:37
where there are so many people who, um, like to pretend that they're an expert on
33:41
everything, um, when, when it comes to ai, um, who were experts on NFTs and crypto.
33:47
Three years ago.
33:48
Um, it's important for me to say that my focus really is on audio visual.
33:51
I do cover some aspects of LLMs, um, um, and some aspects of age agentic systems.
33:56
That's a whole other issue, which we didn't get to around kind of,
33:59
you know, judging authenticity in the age agentic kind of age.
34:01
But, you know, I would focus on the, the audiovisual side of things.
34:04
That's where my, my research is really focused.
34:07
Um.
34:09
I think you can't understate the importance of doing
34:12
things such as red teaming.
34:14
You know, taking these tools and putting 'em in the hands of people
34:17
who are just going to be able to forecast and foresee, um, weaponization
34:22
approaches that you haven't imagined.
34:24
If you can't afford red teaming, go look at places like four chan.
34:28
Go look at places like Reddit in some communities.
34:32
Um, find some discord channels of people who are trying to jailbreak.
34:36
AI tools and look at the techniques they're using, look at the kinds of
34:39
content they're wanting to generate and do some reverse engineering
34:43
based on basically the open source intelligence that you can gather
34:45
without having to get someone professionally to red team your tools.
34:48
But I think having, having a sense of the vulnerabilities.
34:52
Um, and, um, and kind of what you need to, what you need to improve before you
34:56
launch, I think is really important.
34:58
Often it's not the case.
35:00
Um, and it really feeds into one of the most challenging dynamics, and
35:03
it's another one I'm working on with governments, uh, at the moment, which
35:06
is at the moment, Kevin, it's just not strategically smart to invest
35:11
in safety over speed of development.
35:13
Like from a, from an entrepreneurial perspective, it
35:15
just doesn't make sense, like, um.
35:18
If you are spending money and time waiting to release spending money that you could
35:22
be spending on engineers or developers, um, on, on safety, testing your systems,
35:28
um, you know, you might win the moral victory, but that's not ultimately the
35:32
one that's gonna fund your next round.
35:34
Right?
35:34
So there are big challenges.
35:35
Aaron, I, I want to, you know, I want to be, uh, show awareness of
35:38
that, that, that, this is a difficult one, but I think, you know, my
35:41
hope is that we'll get to a point.
35:43
Via carrot or stick, that, that kind of becomes a hard, a less
35:46
of a difficult trade off to make.
35:48
But I think doing that red team and doing that safety testing is really critical.
35:51
I think it sounds, it sounds almost obvious, but I think doing effectively,
35:57
ethnography, doing like customer research, like truly understanding what is it
36:02
the tool is for or who is it for and what is it trying to help them achieve.
36:07
Um, so much of the backlash that I see around AI tools is by people saying.
36:12
I don't want this, like, I didn't ask for this.
36:15
Um, and what's more, you're actually insulting my craft or my identity
36:19
essence of what I see myself as.
36:21
You know, we're all humans.
36:22
We all have jobs.
36:23
Our work is a big part of who we are.
36:25
Um, saying, you know, this is, this is, this is insulting, basically.
36:29
So I think having an understanding of why you are developing the tool and who you're
36:32
developing it for is, is critical and can often get, um, glazed over by excitement
36:37
and sort of again, echo chambers.
36:40
Um.
36:41
And then I think the final thing I'd, I'd say is just don't over, um, don't
36:46
underestimate how important authenticity now really is in a synthetic age, and
36:50
that that doesn't just mean authenticity in the kind of social media sense
36:55
of like, I had a really hard day and I just binge a, a whole packet
36:58
of cookies to try and deal with it.
36:59
Right?
37:00
Um, it's less to do with that.
37:01
It's more about like, what is it that, what is it that your customers
37:04
want to see and your, your users want to see from you in terms of.
37:08
Saying what you do and don't know about how these tools work, admitting
37:11
mistakes, um, just having that kind of honest perspective, I think with your
37:15
users and with the people that you're ultimately trying to engage with.
37:18
Kevin: Mm-hmm.
37:19
Adjer: Um.
37:20
And about where these tools are designed to be used.
37:22
You know, make it clear you're not trying to replace critical parts
37:25
of, you know, the human experience.
37:27
I think these are all really powerful things to get that balance between,
37:30
you know, safety market and, um, an audience I think is, you know, that,
37:36
that's kind of some of the framing that I, I try to, uh, think about
37:39
when I'm talking to these companies.
37:41
Kevin: Henry, thank you so much for the conversation.
37:43
Adjer: My pleasure.
37:44
That's been great fun.
37:44
Thanks.
37:46
Kevin: This has been the Road to Accountable ai.
37:48
If you like what you're hearing, please give us a good review and
37:51
check out my substack for more insights on AI accountability.
37:54
Thank you for listening.
37:59
If you want to go deeper on AI governance, trust and responsibility with me.
38:03
Another distinguished faculty of the world's top business school.
38:07
Sign up for the next cohort of Wharton's Strategies for Accountable
38:10
AI Online Executive education program, featuring live interaction
38:14
with faculty expert interviews and custom designed asynchronous content.
38:19
Join fellow business leaders to learn valuable skills you can
38:22
put to work in your organization.
38:24
Visit exec ed.warden.upen.edu/acai for full details.
38:31
I hope to see you there.