Latest / What's Your Problem? / Detecting Deepfakes With AI

Transcript

Detecting Deepfakes With AI · What's Your Problem?

0:15
Speaker 1: Pushkin. Earlier this year, an employee working in Hong Kong
0:23
Speaker 1: for an international company got a weird message from one
0:26
Speaker 1: of his colleagues. He was supposed to make a secret
0:29
Speaker 1: transfer of millions of dollars. It seems sketchy. It obviously
0:34
Speaker 1: seems sketchy, so he got on a video call with
0:36
Speaker 1: a bunch of people, including the company's CFO, the chief
0:40
Speaker 1: financial officer. The CFO said the request was legit, so
0:44
Speaker 1: the employee did what he was told. He transferred roughly
0:47
Speaker 1: twenty five million dollars to several bank accounts. As it
0:52
Speaker 1: turned out, the CFO on the video call was not
0:54
Speaker 1: really the CFO. It was a deep fake, an AI
0:58
Speaker 1: generated twin created from publicly available audio and video of
1:03
Speaker 1: the real CFO. By the time the company figured out
1:06
Speaker 1: what was going on, it was too late, the money
1:09
Speaker 1: was gone. I'm Jacob Goldstein and this is What's Your Problem,
1:18
Speaker 1: the show where I talk to people who are trying
1:20
Speaker 1: to make technological progress. My guest today is Ali Shahieri.
1:24
Speaker 1: He's the co founder and chief technology officer at the
1:28
Speaker 1: audaciously named Reality Defender. Ali's problem is this, how can
1:34
Speaker 1: you use AI to protect the world from AI? More specifically,
1:39
Speaker 1: how do you build a set of models to spot
1:41
Speaker 1: the difference between reality and AI generated deep fakes. How'd
1:48
Speaker 1: you get into the defending reality business?
1:51
Speaker 2: Yeah, so when I started, it was around actually generating
1:57
Speaker 2: videos and deep fikes.
2:00
Speaker 1: So you were attacking reality before you were defending it.
2:04
Speaker 2: I wouldn't said we were attacking anything, but we were
2:06
Speaker 2: definitely into looking into this technology. And it is way
2:10
Speaker 2: back before all this stuff kind of went crazy. This
2:14
Speaker 2: is back in like twenty nineteen around that time, So
2:17
Speaker 2: we were building digital twins and we're looking at how
2:21
Speaker 2: do you make it so that it looks realistic? Is
2:23
Speaker 2: it a cartoon looking thing? Is it like a unity
2:27
Speaker 2: three D thing? And then that's when we started to
2:29
Speaker 2: see like these early research papers where they were taking
2:32
Speaker 2: like someone's face and putting it on a video and
2:36
Speaker 2: blending it in and it looked really good, and we
2:39
Speaker 2: were like, oh, maybe we can do the digital twins
2:41
Speaker 2: that way. And while we were like in that business,
2:46
Speaker 2: we were like, you know, probably in a few years
2:49
Speaker 2: someone can download an app and just make anything very easily.
2:53
Speaker 2: And that's kind of the origins of how how we started.
2:57
Speaker 2: We're very mission driven. What we're trying to do here
3:00
Speaker 2: is really protect the world and people from the dangers
3:05
Speaker 2: of AI, but in a way where you know, we
3:08
Speaker 2: want people not to abuse the technology. We're very we
3:12
Speaker 2: love AI, we just don't want it to be abused.
3:16
Speaker 1: So let's talk about this sort of deep fake detection
3:20
Speaker 1: kind of you know, jen AI detection market more generally,
3:25
Speaker 1: like who's like, who's selling deep fake detection right now,
3:30
Speaker 1: and who's buying what's the what's the sort of market
3:32
Speaker 1: landscape look like.
3:34
Speaker 2: The type of clients that we have right now are banks.
3:39
Speaker 2: For example, we are currently live with one of the
3:42
Speaker 2: largest banks in the world. When you call that bank,
3:46
Speaker 2: the audio goes through our defake detection models and we're
3:50
Speaker 2: able to tell the call center this person might be
3:53
Speaker 2: a deep fake. And part of that is that's actually happened.
3:57
Speaker 2: Someone's called the bank and they've transferred money out and
4:01
Speaker 2: actually this this goes back to twenty nineteen, so the
4:05
Speaker 2: first incident of defake fraud actually and back in.
4:09
Speaker 1: Two in nineteen that we're aware of. Right, you're right exactly,
4:13
Speaker 1: So what happened in twenty nineteen.
4:16
Speaker 2: Yeah, so this is back where this is early and
4:18
Speaker 2: nobody really knew about this, and there was a CEO
4:22
Speaker 2: that called a smaller company that THEO was a parent
4:26
Speaker 2: company calling the child company. The CEO calling the other
4:29
Speaker 2: CEO and he wanted to transfer some money out and
4:33
Speaker 2: it sounded like him and the guy transferred I think
4:36
Speaker 2: it was in UK, about two hundred and three hundred
4:37
Speaker 2: thousand dollars out And that was like the first one
4:39
Speaker 2: of the first ones that we.
4:40
Speaker 1: Know of, and they got away with it, I believe.
4:44
Speaker 2: So. Yeah.
4:44
Speaker 1: And there was an instance earlier this year right where
4:47
Speaker 1: I think it was in Hong Kong and some employee
4:49
Speaker 1: was on a zoom call with the company's CFO and
4:51
Speaker 1: the CFO was like, you know, why are twenty five
4:54
Speaker 1: million dollars or something to some bank account? And then
4:57
Speaker 1: the employee did it and it turned out the CFO
4:59
Speaker 1: on the call was a deep fake, right.
5:01
Speaker 2: Yeah, So fast were they your client? They were not
5:05
Speaker 2: in our clients unfortunately. But this shows the how quickly
5:09
Speaker 2: the technology is evolving. You know, twenty nineteen audio fast
5:13
Speaker 2: forward a few years now, You've got a zoom call.
5:15
Speaker 2: There'd a bunch of people on it and they all
5:17
Speaker 2: look like people, you know, I know, they're all de fis.
5:21
Speaker 1: So you were starting to mention. Banks are some of
5:23
Speaker 1: your main clients. Who are some of your other main.
5:25
Speaker 2: Clients, media companies, I think think of some of the
5:28
Speaker 2: big ones, there is our product this year, especially with
5:31
Speaker 2: the election. You know, back twenty twenty, we thought it
5:35
Speaker 2: would be a problem. It wasn't. This year we think
5:37
Speaker 2: is a big problem. For sure. I think we were early,
5:40
Speaker 2: but it's already this is happening everywhere even this year.
5:45
Speaker 2: This year is the largest election year in the world.
5:47
Speaker 2: More than fifty percent of the people are voting, and
5:50
Speaker 2: we already have documented cases of election issues with the fix.
5:55
Speaker 1: Okay, media companies, banks, any other kind of big categories
5:59
Speaker 1: of clients.
6:00
Speaker 2: Yeah, so other ones are government agencies. But in the end,
6:07
Speaker 2: everyone we think, we believe everyone needs this Product's not
6:12
Speaker 2: It shouldn't be up to the people to decide or
6:14
Speaker 2: figure out if something's a deepic. If you're on the
6:16
Speaker 2: social media platform, you shouldn't have to figure out, hey,
6:19
Speaker 2: is this person real or not. It should just be
6:21
Speaker 2: built in and anyone should be able to use it.
6:24
Speaker 1: Well. Our social media companies either buying or building deep
6:29
Speaker 1: fake detection tools or do they want to like stay
6:31
Speaker 1: out of that business and be like no, we don't
6:33
Speaker 1: want to be in the business of saying yes, this
6:35
Speaker 1: is real, no, this isn't real.
6:37
Speaker 2: I can tell you we've been in contact and have
6:39
Speaker 2: talked to some social media platforms. I think one issue
6:44
Speaker 2: is they don't have to flag these things. It's up
6:49
Speaker 2: to them, right, there's not a lot of regulation, so
6:53
Speaker 2: I know they're thinking about it. We've chatted with some,
6:56
Speaker 2: but that's the extent of it.
6:58
Speaker 1: So okay, So let's talk about how it works. And
7:00
Speaker 1: there's two ways that I want to talk about how
7:02
Speaker 1: it works. So one is from the point of view
7:03
Speaker 1: of the user, whoever that may be, and then the
7:06
Speaker 1: other is sort of what's going on under the hood. Right,
7:08
Speaker 1: So let's start with the point of view of the user.
7:11
Speaker 1: If I'm a whatever, a bank, university, a media company
7:15
Speaker 1: who is paying for your service, how does it work
7:18
Speaker 1: for me?
7:19
Speaker 2: Depends on exactly the user and the use case. If
7:22
Speaker 2: let's say it's a media company, Uh, they're looking at
7:26
Speaker 2: maybe filtering through a lot of content, so content moderation.
7:30
Speaker 2: Actually that would be like a social media company. They're
7:32
Speaker 2: looking at content moderation. Maybe they want they're looking at
7:36
Speaker 2: millions of assets and they want to quickly flag those
7:39
Speaker 2: things if they were in that business. Uh, the bank
7:42
Speaker 2: there For the example I gave the issue, someone could
7:46
Speaker 2: call and biometrics fail. By the way, if you call
7:48
Speaker 2: a bank, some banks say repeat after me, your my
7:52
Speaker 2: voice is my passport? That actually fails. Now what do
7:54
Speaker 2: you think? So a bank wants to make sure the
7:57
Speaker 2: person calling in is actually that person. This is more
8:00
Speaker 2: relevant to more to private banking, where there's actually a
8:03
Speaker 2: one on one relationship between the client and the bank.
8:07
Speaker 1: And so in that case, So let's take that case.
8:09
Speaker 1: So in that case, someone calls in and talks to
8:12
Speaker 1: their banker. They're a rich person who has a private banker.
8:15
Speaker 1: Basically it's what you're talking about, right, So this rich
8:18
Speaker 1: person calls in and talks to their private banker, and
8:22
Speaker 1: it is the system just always running in the background
8:25
Speaker 1: in that case, And like, how does it work from
8:28
Speaker 1: the point of view of the of the private banker.
8:30
Speaker 2: Sure, and I have to be careful what I say here,
8:33
Speaker 2: But the high level is the models are listening and
8:38
Speaker 2: if they detect a potential deep fake, they will the
8:41
Speaker 2: call center. That person will get a notification so is
8:44
Speaker 2: integrated into their existing workflow. They'll get a notification that says, hey, this.
8:48
Speaker 1: Person get like a text or a slack or something
8:51
Speaker 1: they're using. You're talking to a deep fake.
8:54
Speaker 2: No, they're using software for the bank they're using they're
8:56
Speaker 2: still using a software and there's a dashboard. In that scenario,
9:00
Speaker 2: they do they ascalate, so they might say, let me
9:03
Speaker 2: ask you some more questions or let me call you back.
9:05
Speaker 1: Huh. Let me call you back is a super safe one,
9:07
Speaker 1: right because if they have a relationship, probably they know
9:09
Speaker 1: the number. They just call them back. Yeah, absolutely, okay,
9:13
Speaker 1: And then how does it work? How does it work
9:16
Speaker 1: for like when you say, like I presume by the
9:18
Speaker 1: way that you can't name your clients. You said a
9:21
Speaker 1: media company and a bank. It's it's secret that they're.
9:24
Speaker 2: Yeah, we're not allowed to okay.
9:26
Speaker 1: So let's say a media company. How's it work for
9:28
Speaker 1: a media company?
9:29
Speaker 2: Their their use case is slightly different, especially right now,
9:33
Speaker 2: as I mentioned around the election, So there there might
9:35
Speaker 2: be something that that's starting to go viral in the
9:38
Speaker 2: news and they want to check, hey, is this a
9:40
Speaker 2: real or not? I would like to say like something
9:44
Speaker 2: like this is usually when something goes viral, the damage
9:47
Speaker 2: is already ton.
9:48
Speaker 1: Yes, although if you're if you're whatever, the New York
9:51
Speaker 1: Times of the Wall Street Journal. You don't want to
9:52
Speaker 1: repeat the viral lie. Part of your business model is
9:55
Speaker 1: people are paying to subscribe to you because you are
9:59
Speaker 1: more reliable.
10:00
Speaker 2: Right exactly. So that's why they come to us. They
10:02
Speaker 2: upload the assets and are our web app returns the
10:05
Speaker 2: results I see.
10:06
Speaker 1: So it's just like you just go to whatever Real
10:09
Speaker 1: Defender dot whatever and you upload the viral video and
10:13
Speaker 1: your machine says it's a fake.
10:16
Speaker 2: Yeah, So we give results and probabilities that we don't
10:19
Speaker 2: have the ground truth, so we give a probability. There's
10:22
Speaker 2: several different models running, so we use an ensemble of models.
10:25
Speaker 2: We have different models looking at different things, and we
10:29
Speaker 2: give an overall score averaging those. In the case of
10:32
Speaker 2: a video, we actually highlight the areas of a defake.
10:36
Speaker 2: If the person is speaking and they're a fake, there'll
10:38
Speaker 2: be a red box around them. If there is a
10:39
Speaker 2: real they'll be a green box around it.
10:41
Speaker 1: And well, that latter part sounds more binary as opposed
10:46
Speaker 1: to probabilistic.
10:47
Speaker 2: We give both. So yeah, there's there was a probably
10:50
Speaker 2: score and there's just the visual.
10:52
Speaker 1: And so the probabilistic score is basically according to our model,
10:55
Speaker 1: there's a seventy percent chance that this is fake something
10:58
Speaker 1: of that nature.
10:59
Speaker 2: According to our ensemble of models.
11:01
Speaker 1: Yes, yeah, our model of models, our fund of funds
11:04
Speaker 1: of models exactly. So so okay, so you're actually looking
11:09
Speaker 1: us toward what's under the hood, right, I'm interested in
11:12
Speaker 1: discussing this on a few levels. Right, there is the
11:15
Speaker 1: sort of broad beyond reality defender. You know, what are
11:20
Speaker 1: the basic ways that the technology works, Like how does
11:23
Speaker 1: deepfake detection gen AI detection work? In a broad way?
11:26
Speaker 1: Like can you talk me through that?
11:27
Speaker 2: Absolutely? Yeah. There's currently two ways people are looking at
11:31
Speaker 2: this problem. Number one is prominence. For example, you water
11:35
Speaker 2: mark a media that you create, maybe you water market
11:40
Speaker 2: or you digitally sign it, maybe you put on a
11:42
Speaker 2: blockchain somewhere or something like that. But basically there's a
11:44
Speaker 2: source of true that this video is real. Yeah, and
11:47
Speaker 2: there's a water mark. That's number one.
11:50
Speaker 1: But we're concerned. We're concerned with instances where that is
11:52
Speaker 1: not the case. Right. Our world is full of videos
11:54
Speaker 1: today that are not clearly watermarked, blockchain whatever for prominence.
12:00
Speaker 1: So we have this problem. What are the ways people
12:02
Speaker 1: are solving it?
12:03
Speaker 2: Yeah? The second way is how we're solving it, which
12:05
Speaker 2: is basically we use AI to detect AI, so we
12:09
Speaker 2: which we call inference. So we train AI models, as
12:13
Speaker 2: I mentioned, a bunch of them to look at various
12:17
Speaker 2: various aspects of plus say video.
12:20
Speaker 1: So like, is it a sort of generative adversarial network
12:25
Speaker 1: the right term? I mean, it seems like you It
12:27
Speaker 1: seems like if I were making up how to do this,
12:29
Speaker 1: I'd be like, well, I'm gonna have one model that's
12:32
Speaker 1: like cranking out really good deep fikes, but I'll know
12:35
Speaker 1: which ones are the deep fis, and then I'm gonna
12:36
Speaker 1: feed the deep fis and the real ones to my
12:38
Speaker 1: other model, and I'll score it on how well it does,
12:41
Speaker 1: and it'll get really good at figuring out the difference.
12:43
Speaker 2: Yeah, that's actually exactly how a lot of these work.
12:46
Speaker 2: For if you go to there's a website you can
12:48
Speaker 2: go where it just generates a person every time you
12:51
Speaker 2: go to it a right, and that's actually using again
12:53
Speaker 2: to generate that person. So the way we detect and
12:56
Speaker 2: I can I can give a little bit more detail here.
12:58
Speaker 2: So for example, one of our models which we actually removed,
13:02
Speaker 2: was looking at blood flow. So yeah, so imagine actually
13:07
Speaker 2: in this video lighting and conditions are right, we can
13:11
Speaker 2: actually detect the heartbeat and the blood flow and the
13:14
Speaker 2: veins the way we're looking at each other.
13:16
Speaker 1: As I'm looking at my weirdly today, maybe because it's
13:19
Speaker 1: hot or because the light hair, I can actually see
13:21
Speaker 1: a vein bulging on my forehead. So, like you're saying,
13:24
Speaker 1: an AI could like measure my pulse from that or something.
13:27
Speaker 2: In the right conditions. Now, that model has a lot
13:30
Speaker 2: of limitations, and you need to have the right It's
13:36
Speaker 2: basically it has a lot of bias. Right, So we
13:39
Speaker 2: tossed that.
13:40
Speaker 1: Wait, you're saying it didn't work. You're saying it didn't work.
13:42
Speaker 2: It worked in the right conditions and the right skin tone,
13:46
Speaker 2: so yeah, so otherwise it was biased. So we this
13:49
Speaker 2: was experimental and we tossed it.
13:52
Speaker 1: A lot of things. It didn't work. So you tried
13:54
Speaker 1: it and in a broad way it didn't work. It
13:56
Speaker 1: worked in narrow conditions, but you need things that work
13:58
Speaker 1: more broadly. What's another thing you tried that didn't work?
14:02
Speaker 2: Well, I can tell you every month we may be
14:05
Speaker 2: throwing away models.
14:06
Speaker 1: Well, presumably there's things that work for a while and
14:09
Speaker 1: then they don't. Right, It's kind of like antibiotics versus bacteria, right,
14:13
Speaker 1: like your adversaries are getting better every day.
14:16
Speaker 2: Basically, what we use, what we like to use is
14:19
Speaker 2: we like to say we're like an anti virus company.
14:21
Speaker 2: So every time every month there's a new genera of technique,
14:25
Speaker 2: maybe we should go detective. But maybe it's something we
14:28
Speaker 2: don't anticipate and we don't detect, and so we have
14:30
Speaker 2: to make sure we quickly update our models. So and
14:33
Speaker 2: then a model that worked last year, it's completely irrelevant now.
14:37
Speaker 1: So what else, like, what else is happening technologically on
14:40
Speaker 1: the reality defense side, on the detection side.
14:43
Speaker 2: Okay, so the way, we have a few different products.
14:46
Speaker 2: One is, as I mentioned, real time audio like scanning
14:50
Speaker 2: and listening for telephone calls. The other one is a
14:53
Speaker 2: place where a journalist or any user can go and
14:55
Speaker 2: upload not just videos, but we also detect images. We
14:59
Speaker 2: also detect audio, We also detect texts like chat GPT,
15:03
Speaker 2: and these tools also explain to a user why something
15:07
Speaker 2: is a deep fake. We don't just give a score.
15:09
Speaker 2: Or for an image, we might put a heat map
15:11
Speaker 2: and see these are the areas that set the model off.
15:14
Speaker 2: For text, we might highlight areas and see these other
15:17
Speaker 2: areas that appear to be generated.
15:20
Speaker 1: There's a case study you have about a university that
15:22
Speaker 1: is a client of yours that, among other things, uses
15:28
Speaker 1: uses your service to tell when students are turning in
15:32
Speaker 1: papers written by chat GIPT. Basically as I read it, right, like,
15:36
Speaker 1: I just assume that, like everybody writes papers with chat
15:39
Speaker 1: GPT now and there's nothing anybody can do about it.
15:41
Speaker 1: But is that not true? Like if I like have
15:43
Speaker 1: GPT write my paper and then I like change a
15:46
Speaker 1: few words, does that sort of help get let me
15:50
Speaker 1: sail past your defense?
15:52
Speaker 2: It depends depends how much you change, Yeah, or if
15:55
Speaker 2: you change like over fifty percent, maybe maybe would So
15:59
Speaker 2: it depends.
16:00
Speaker 1: Over fifty percent is more than a few words. And
16:02
Speaker 1: so can you talk? I mean, I know you can't
16:05
Speaker 1: name the university, but in practice you know how they
16:07
Speaker 1: use it. So you know, somefess runs the student's papers
16:11
Speaker 1: through your software and it says of when student there's
16:14
Speaker 1: a whatever sixty percent chance that this was created using
16:18
Speaker 1: a large language model. I mean, do you know in practice?
16:22
Speaker 1: Obviously the professor could do whatever they want or the
16:24
Speaker 1: university could have whatever policy, but do you know in practice,
16:26
Speaker 1: what do they do with this information like that's that's
16:30
Speaker 1: in a way a harder one to figure out than
16:31
Speaker 1: the like banker who's like, oh, it might be a
16:34
Speaker 1: deep fake on the phone. I'll call you right back
16:36
Speaker 1: for security. Like if my I don't have a banker,
16:38
Speaker 1: but if I had a banker and they did that,
16:40
Speaker 1: I'd be like, oh, that's cool. I'm glad my bank
16:43
Speaker 1: is doing this thing. Whereas with like the professor and
16:45
Speaker 1: the student, that's a much more sort of fraud situation, right,
16:52
Speaker 1: and harder to think of how to deal with again
16:55
Speaker 1: the probabilistic nature of the output of the model.
16:59
Speaker 2: Yes, I think a couple more things here. First of all,
17:01
Speaker 2: I think even universities are trying to figure out this problem.
17:05
Speaker 2: How to you solve it? You know. But the second
17:08
Speaker 2: thing to note, most of our users are not interested
17:13
Speaker 2: in a text detector. That seems to be a much
17:15
Speaker 2: smaller market. The biggest one is actually audio. It's becoming
17:20
Speaker 2: imagine you get a call from a loved one and
17:22
Speaker 2: send me money, and you send money if you realize
17:24
Speaker 2: is not who it was a defate, right, That's actually
17:27
Speaker 2: a much widely used system.
17:31
Speaker 1: That's the big one in terms of the business it's interesting.
17:34
Speaker 1: I mean, I wonder if that's partly like relative we
17:36
Speaker 1: think about the video more, but is it partly because
17:39
Speaker 1: deep fake audio is now quite good and there are
17:41
Speaker 1: lots of instances where people will transfer lots of money
17:44
Speaker 1: based solely on audio.
17:47
Speaker 2: De fake audio is the best and it's getting better,
17:49
Speaker 2: right interested. I used to go to make your voice,
17:51
Speaker 2: maybe I need a minute. Now I need just a
17:54
Speaker 2: few seconds and I can make your voice. It's getting
17:57
Speaker 2: exponentially better. All of them are, but audio is definitely
18:00
Speaker 2: top of the list right now.
18:01
Speaker 1: Huh And how are you keeping up?
18:06
Speaker 2: Yeah? I mean, so when we detect audio, it's tricky.
18:09
Speaker 2: There's a lot of factors to think about a person's accent,
18:13
Speaker 2: right what is model biased? Does it not understand or
18:17
Speaker 2: is there an issue where it detects It detects one
18:20
Speaker 2: person with a certain type accent always as a d thing.
18:22
Speaker 2: There's also issues of like noise when when when there's
18:26
Speaker 2: a lot of background noise, the model could be impacted.
18:28
Speaker 2: When there's cosstop, multiple people speaking at the same time,
18:31
Speaker 2: that could impact the model. So there's a variety of factors.
18:35
Speaker 2: And the other thing you think about is our models
18:37
Speaker 2: are more they support multiple languages, so we don't just
18:40
Speaker 2: do English, and so all of these kind of make
18:43
Speaker 2: it very complicated. So when we detect something it's called
18:46
Speaker 2: pre processing, there's a whole bunch of steps to the
18:50
Speaker 2: audio before it actually goes to our AI models where
18:52
Speaker 2: we have to clean up the audio, do certain types
18:56
Speaker 2: of transformations before we push it to the models.
18:58
Speaker 1: And is that happening in real time with these companies?
19:01
Speaker 1: Huh huh? And and are you like, what is the
19:07
Speaker 1: frontier of preprocessing? Like is it is it an efficiency
19:10
Speaker 1: and speed problem because you're trying to do it in
19:12
Speaker 1: real time and so you're just trying to kind of
19:15
Speaker 1: make the sort of algorithmic part of it as fast
19:18
Speaker 1: and efficient as possible.
19:19
Speaker 2: Yeah, I mean this is a challenge. There's a lot
19:22
Speaker 2: to be done. So that's an ongoing research. How do
19:25
Speaker 2: we continue to speed up not just a preprocessing, but
19:28
Speaker 2: the inference. And there's a variety of one thing that's
19:33
Speaker 2: called a foundation model. I'm not sure if you heard
19:35
Speaker 2: what those are, but these are extremely large pre train
19:37
Speaker 2: model GPT is a foundation model is a pre train model.
19:40
Speaker 2: And so these models can be useful in some parts
19:43
Speaker 2: of the preprocessing where they can quickly extract certain features
19:47
Speaker 2: for us, and then we can use those two down
19:50
Speaker 2: the pipeline.
19:54
Speaker 1: Still to come on the show. The problems that Ali
19:56
Speaker 1: is trying to solve. Now, how good are you at
20:09
Speaker 1: detecting de fikes? Can you quantify how good you are?
20:13
Speaker 2: So the way they usually do this is they look
20:16
Speaker 2: at benchmarks. Right, there's public data sets which we can
20:19
Speaker 2: take and run and we're in the nineties and then
20:23
Speaker 2: but you know that's not the real world.
20:25
Speaker 1: When you say you're in the nineties, you mean you
20:29
Speaker 1: in a binary sense, you guess correctly ninety percent of
20:33
Speaker 1: the time.
20:35
Speaker 2: Yeah, So on a public benchmark, we're in the nineties.
20:37
Speaker 2: There's accuracy, precision and recall. Accuracy is how accurate are
20:41
Speaker 2: we Let's say there is one hundred sample set is
20:45
Speaker 2: one hundred, maybe fifty is fake, fifty is real? Right.
20:50
Speaker 2: The accuracy is you take, okay, how many of those
20:52
Speaker 2: did you get right? How many of the real I'm fake? Divided? Right,
20:55
Speaker 2: that's the that's the accuracy. The problem with that is
20:58
Speaker 2: like unbalanced data set, maybe maybe only two is fake
21:03
Speaker 2: and then the other ninety eight are real. So in
21:06
Speaker 2: that case, the accuracy. See if we had said that Okay,
21:11
Speaker 2: everything is real, we would be ninety eight percent. Right,
21:14
Speaker 2: that's not very useful because you missed the defix. So
21:17
Speaker 2: that's why precision and recall coming. They look specifically at
21:21
Speaker 2: how did you do on that specific like the fakes
21:24
Speaker 2: or the reals, So there's more than just accuracy. There's
21:28
Speaker 2: also other factors to look at.
21:30
Speaker 1: So there's it's kind of like the sort of false
21:33
Speaker 1: positive false negative challenge with medical tests, right you want
21:39
Speaker 1: to test that both says you have the thing, says
21:43
Speaker 1: you have the disease when you have the disease, and
21:45
Speaker 1: also says you don't have the disease when you don't
21:47
Speaker 1: have the disease, And that actually ends up being a
21:49
Speaker 1: really complicated problem given the nature of baselines, right like
21:54
Speaker 1: in your universe, certainly in the universe of people calling
21:57
Speaker 1: their banker. Almost everybody calling their banker is a real person, right,
22:02
Speaker 1: but there are these very high stakes, presumably very rare
22:06
Speaker 1: cases where it is a defake, and so that's like
22:08
Speaker 1: a complicated problem.
22:10
Speaker 2: It actually is, It absolutely is, and it's something as
22:14
Speaker 2: we work with each customer, we have to tweak those.
22:16
Speaker 2: Someonet higher false positives, someone higher false negatives. It depends
22:20
Speaker 2: on each use case, in the case of a bank,
22:23
Speaker 2: they want to be a bit more cautious. But that
22:25
Speaker 2: also causes a lot of It could cause a lot
22:28
Speaker 2: of pain depending on the volume.
22:29
Speaker 1: Right, because if every client it's like, oh sorry, I
22:32
Speaker 1: got to call you back to make sure you're not
22:34
Speaker 1: a deep fake, Like that's not great.
22:36
Speaker 2: Yeah, And if you have thousands of calls a day
22:38
Speaker 2: and even one percent is a false positive or negative,
22:42
Speaker 2: that that creates a lot of work, Yeah, because it
22:45
Speaker 2: adds up.
22:46
Speaker 1: How do you solve that? What do you do about that?
22:49
Speaker 2: So the way it works is all about adjusting. You
22:53
Speaker 2: can think of thresholds, right, We can adjust variety of
22:58
Speaker 2: parameters as the output for a model, not just the
23:02
Speaker 2: model itself, but the for example, in an audio as
23:08
Speaker 2: we speak, you know, we could look at okay, how
23:11
Speaker 2: long do you want to listen before you give an answer?
23:14
Speaker 2: You know, maybe maybe? And the longer you listen, the
23:17
Speaker 2: more the more confident, the more that's smart.
23:21
Speaker 1: That makes sense, right, because it's essentially more data for
23:24
Speaker 1: the model exactly. Yeah, what are you trying to figure
23:28
Speaker 1: out now? Like what is the frontier?
23:32
Speaker 2: What's really the latest now? And it's just amazing how quickly.
23:35
Speaker 2: It's going as videos. So the videos that we detect
23:38
Speaker 2: are like a face swap, Like you're sitting there speaking
23:41
Speaker 2: and another person's face is on there. That's a face swap.
23:44
Speaker 2: But now you can generate an entire video completely from scratch,
23:49
Speaker 2: and you just type in the description and the video
23:52
Speaker 2: comes out. You can take some you can I can
23:55
Speaker 2: take your voice, a few seconds of your voice. I
23:57
Speaker 2: can then have you say anything I want, which you
24:00
Speaker 2: can clearly see. The bad, bad person can misuse these tools.
24:03
Speaker 2: So the latest is these things are getting really good and.
24:06
Speaker 1: Over time, like with those videos, is your how is
24:09
Speaker 1: your reliability and accuracy changing? You're getting better or worse
24:13
Speaker 1: or staying the same as the technology to create the
24:16
Speaker 1: deep fix improves.
24:17
Speaker 2: So what's interesting is it has slowed down in terms
24:20
Speaker 2: of like the signatures, Like we don't need as much
24:23
Speaker 2: data as we used to. So of course there's still
24:27
Speaker 2: a lot of work and we're never going to stop,
24:29
Speaker 2: but it is stabilizing a little bit.
24:32
Speaker 1: When you say it, what is stabilizing a little bit,
24:36
Speaker 1: So like the.
24:37
Speaker 2: Defied signatures are stabilizing the way.
24:40
Speaker 1: The signatures, meaning the giveaways, the things that I can't see,
24:44
Speaker 1: but that your models can see that AI exactly.
24:47
Speaker 2: So our models going back and give them more detail.
24:51
Speaker 2: They're looking at different attributes of a piece of media,
24:54
Speaker 2: and they pull out those attributes and then they send
24:57
Speaker 2: those to our and house neural networks that steady those attributes.
25:01
Speaker 1: Like one that you have mentioned, that the company has
25:03
Speaker 1: mentioned publicly is the the sync of audio and video. Right, Yes,
25:09
Speaker 1: maybe that's one where it's gotten better and it doesn't
25:11
Speaker 1: matter anymore, but like it, from what I understand, from
25:15
Speaker 1: what I've read, there was at least a time when
25:17
Speaker 1: the sink of the audio and video tended to be
25:20
Speaker 1: off in deep fake videos. Right? Is that an example
25:25
Speaker 1: of a signature.
25:27
Speaker 2: So the way that works is we train the model.
25:29
Speaker 2: We say, hey, here's a bunch of people speaking, here's
25:33
Speaker 2: what they look like. Look at the sink. Here's a
25:35
Speaker 2: bunch of people like that or defikes, and look at
25:37
Speaker 2: the sink, and we tune the model so we can
25:40
Speaker 2: tell the difference. That's also happening to a video. By
25:42
Speaker 2: the way, if you look at Sora and some of
25:44
Speaker 2: these new models where someone's are walking, for example, their
25:49
Speaker 2: legs are not like you know, they're not really smooth,
25:54
Speaker 2: or they don't look right, So you can look at
25:56
Speaker 2: that as well. That's the temporal dynamics we call that.
25:59
Speaker 1: Uh Like temporal dynamics is basically are things proceeding in
26:03
Speaker 1: time in a natural.
26:04
Speaker 2: Way exactly how things change over time.
26:09
Speaker 1: So yeah, all of these seem like things that you
26:12
Speaker 1: can just that are going to be fleeting, right. Like
26:14
Speaker 1: my baseline assumption is it'll all get solved. Do you
26:21
Speaker 1: how long do you think you'll be able to defend
26:22
Speaker 1: reality for?
26:24
Speaker 2: You know, this question comes up all the time where
26:26
Speaker 2: there is always a giveaway or there is always a
26:29
Speaker 2: new way to look at the problem. We're not just
26:31
Speaker 2: looking always at the raw pixels, right, We could look
26:34
Speaker 2: at different aspects. We could look at the frequency. For example,
26:38
Speaker 2: if you look at an image, you can actually break
26:40
Speaker 2: it down into frequencies.
26:41
Speaker 1: When you say frequency, what do you mean when you
26:44
Speaker 1: say you can look at the frequency? What does that mean?
26:46
Speaker 2: So? For example, okay, so let's go with audio. You
26:49
Speaker 2: know you can use some of call four yer transformers
26:51
Speaker 2: to actually break up an audio into individual wavelengths science
26:54
Speaker 2: and co science that does a look. You can do
26:56
Speaker 2: the same with for an image, for example, you can
26:59
Speaker 2: break that.
26:59
Speaker 1: Up like like the analogy of a wave form of audio.
27:03
Speaker 2: Yeah, it can. It can be translated into a bunch
27:05
Speaker 2: of waves. So so there's multiples that we look at.
27:10
Speaker 2: There's and the AI there's always a giveaway, uh and
27:14
Speaker 2: and and again we're also thinking outside the box, right,
27:17
Speaker 2: like the blood flow for example. Right, But there's other
27:21
Speaker 2: kind of similar things we could think about.
27:22
Speaker 1: I mean, presumably you know there, you know, renaissance Renaissance Capital.
27:28
Speaker 1: The James Simons is one of the first quant hedge funds,
27:32
Speaker 1: and they made tons of money for a long time.
27:37
Speaker 1: They wildly outperformed the market. Clearly they had a technological advantage.
27:41
Speaker 1: And the thing Simon said, the founder of this math
27:44
Speaker 1: guy about about that company. One of the things he
27:47
Speaker 1: said was like, we actually don't want to hire like
27:51
Speaker 1: finance people who have some story about why a stock
27:54
Speaker 1: is going to outperform, because if there's a story about
27:56
Speaker 1: it that then then somebody else is going to know
27:59
Speaker 1: it already. Right. Their thing was just like, we just
28:03
Speaker 1: give the model all the data and let the model
28:06
Speaker 1: find these weird ass patterns that no human even understands.
28:11
Speaker 1: But they work more often than they don't work, and
28:14
Speaker 1: we make tons of money. And I would think that
28:17
Speaker 1: would be the case for you to some extent that
28:20
Speaker 1: if you could think of a thing like monitoring blood
28:22
Speaker 1: flow or whatever, then the bad guys or whatever, the
28:25
Speaker 1: people who want to make realistic Jenny I would also
28:29
Speaker 1: think of it. And the real kind of secret sauce
28:31
Speaker 1: would be in weird correlations that the model finds that
28:37
Speaker 1: we wouldn't even understand.
28:40
Speaker 2: Exactly. I mean, that is oftentimes what the model is
28:44
Speaker 2: training on, and the way it determines of something that
28:48
Speaker 2: you think looking at certain features, it is something that
28:52
Speaker 2: we don't even tell it, right, Yeah, it determines on
28:55
Speaker 2: its own.
28:56
Speaker 1: Like that's the beauty of this kind of new era
28:58
Speaker 1: of whatever, neural networks, machine learning. Right, it's just you
29:03
Speaker 1: throw everything at it and let the machine figure it out.
29:07
Speaker 2: We like to say we throw the kitchen sink at sometimes.
29:09
Speaker 1: Yes, yes, I mean, And so when you were talking
29:12
Speaker 1: before about explainability, right about sort of saying in your output,
29:17
Speaker 1: here's why we think it's fake. I feel like that
29:20
Speaker 1: kind of throw everything at it and let the machine
29:22
Speaker 1: figure it out makes it hard to like sometimes you
29:24
Speaker 1: don't know, right, it's just like, well, the machine is
29:27
Speaker 1: very smart in it says this is probably fake, Like yes,
29:31
Speaker 1: that is that intention that can happen.
29:33
Speaker 2: So you'll look at it. We'll show you an image
29:36
Speaker 2: and it'll say the model was looking at certain areas.
29:38
Speaker 2: And by the way, this also helps us with debugging
29:41
Speaker 2: him bias. Right, maybe it was like for some reason
29:44
Speaker 2: looking at an area of the face that we wouldn't tell.
29:49
Speaker 2: Why would that set off the model? And so in
29:51
Speaker 2: those scenario as we also investigate like why was this
29:54
Speaker 2: area flag? And it could be one hundred percent correct,
29:58
Speaker 2: it's just we do we do have to examine it further.
30:02
Speaker 1: Could you create a deep fake that would fool your
30:05
Speaker 1: deep fake detector? Yes, haha, Well if you could do it,
30:10
Speaker 1: somebody else could do it. Don't you think I could
30:13
Speaker 1: do it?
30:13
Speaker 2: Because I have access to a lot more knowledge, right, Like,
30:18
Speaker 2: you know I could if I was running an anti
30:20
Speaker 2: virus company, I could probably write a virus if I
30:23
Speaker 2: knew exactly what we're constantly actually trying to do that.
30:27
Speaker 1: By the way, yeah, I mean in a sense, that's
30:29
Speaker 1: the whole adversarial network thing, right, Like I guess you
30:33
Speaker 1: have to do that for your detection models or your
30:37
Speaker 1: suite of models to get better, right.
30:39
Speaker 2: Yeah, So we have what's called red teeming both black
30:41
Speaker 2: box and understanding of the codes. So we're trying to
30:44
Speaker 2: break the models. That's part of the what we do.
30:47
Speaker 1: Uh huh. And so are there like evil geniuses at
30:50
Speaker 1: your company who can make killer deep fakes?
30:53
Speaker 2: We definitely have geniuses one hundred percent, but we're in
30:57
Speaker 2: the business of detection, right, we don't. We don't try
31:00
Speaker 2: to generate too much other than just for training the models.
31:03
Speaker 1: I mean I have to think like, there are many
31:08
Speaker 1: people in the world world who want to make a
31:11
Speaker 1: deep fakes for many reasons, and they're at different levels
31:15
Speaker 1: of technological sophistication. Naively not knowing much about this, I
31:22
Speaker 1: would think you can catch most of them. But if
31:25
Speaker 1: you have people who can beat your models, I would
31:27
Speaker 1: imagine that, say, state actors, countries throwing billions of dollars
31:31
Speaker 1: at this probably also have people who could defeat your models.
31:36
Speaker 2: Yeah, I mean that's always a case with any cybersecurity company.
31:40
Speaker 2: We are a cybersecurity company. Every cyber security company does
31:44
Speaker 2: its best to defend right, but we did not promise
31:49
Speaker 2: one hundred percent. Our models are always a probability.
31:54
Speaker 1: Who's who's the best at making deep fikes that you're
31:57
Speaker 1: aware of.
31:58
Speaker 2: There's a few, right, there's like Sora from OpenAI. There's Runway,
32:01
Speaker 2: there's Synthesia, there's you.
32:03
Speaker 1: Better be able to catch right, anything I've heard of.
32:06
Speaker 1: You better be really good at the technic. Presumably it's
32:08
Speaker 1: like some like you know, Russian Genius Squad or I
32:13
Speaker 1: don't know, the North Koreans are some things. I would
32:15
Speaker 1: imagine it is some state funded actor, but.
32:17
Speaker 2: I would I would actually say, you know, we're in
32:20
Speaker 2: a place where this is a problem is getting bigger.
32:23
Speaker 2: But we're in a place where a lot of the
32:25
Speaker 2: defects coming out are actually fron entertainment and they're not
32:28
Speaker 2: like Youth for Evil. You know, you've seen the famous
32:31
Speaker 2: Tom kruzwe or or other actors running around and do things,
32:36
Speaker 2: and those are defakes, right, those are actually pretty good.
32:38
Speaker 2: We detect them, but they're actually very good.
32:40
Speaker 1: What are you thinking about in the context of the
32:42
Speaker 1: of the election in the US this year and do
32:44
Speaker 1: you have particular clients who are especially focused on election
32:48
Speaker 1: related deep fakes.
32:51
Speaker 2: Yeah, the media companies are the main ones, and we're ready.
32:56
Speaker 2: We detect the best, the best defakes, right, everything that's
33:02
Speaker 2: coming out we detect, So we're ready and we want
33:05
Speaker 2: to make sure we're there as one avenue of people
33:10
Speaker 2: verifying content. I believe late last year there was an
33:13
Speaker 2: election in Slovenia where there was an audio of one
33:17
Speaker 2: of the candidates saying he's gonna double the price of beer. Yeah,
33:21
Speaker 2: and that actually was a defake. It was caught, but
33:25
Speaker 2: it kind of costed some damage. So it's starting to
33:28
Speaker 2: happen now.
33:29
Speaker 1: It's an awesomely stupid deep fake. I mean, to me,
33:31
Speaker 1: the real risk of deep fakes is not people believing
33:35
Speaker 1: something that's false. It's people ceasing to believe anything, right.
33:39
Speaker 1: It's just saying, oh, that's probably just a deep fake,
33:43
Speaker 1: right like that. Actually, to me seems like the bigger
33:45
Speaker 1: risk is nothing is true anymore. Nobody cares about the
33:49
Speaker 1: truth anymore.
33:50
Speaker 2: That's definitely a problem as well. Now we're seeing people saying, oh,
33:56
Speaker 2: this is a defake. That's actually happened. There's a few.
33:59
Speaker 2: I believe it was a Cape Milton video if I'm correct,
34:03
Speaker 2: that was earlier this year, or everyone thought that was
34:04
Speaker 2: a defake and it wasn't. So this kind of problem
34:08
Speaker 2: is happening.
34:08
Speaker 1: Like that's because people people want to believe things that
34:12
Speaker 1: are consistent with their prior beliefs, and they don't want
34:14
Speaker 1: to believe things that call their prior beliefs into question, right,
34:18
Speaker 1: and so deep fakes in a way are an easy
34:20
Speaker 1: out where if you see something you like, you assume
34:23
Speaker 1: it's true. If you see something you don't like, you
34:25
Speaker 1: assume it's not true, or you assume everything's just kind
34:27
Speaker 1: of bullshit like that to me seems like a big
34:29
Speaker 1: quind of societal level risk of deep fakes.
34:32
Speaker 2: We'll never fix that. That's something that will never solve. Yeah,
34:36
Speaker 2: people have their own beliefs. You can show them anything,
34:40
Speaker 2: the facts, math, that's not going to fix it all. Yeah.
34:44
Speaker 1: No, I guess that's a human nature problem, if not
34:46
Speaker 1: an AI problem. We'll be back in a minute with
34:52
Speaker 1: the lighting round. Okay, let's close with a lightning round.
35:09
Speaker 2: Okay.
35:10
Speaker 1: How often do people applying to work at Reality Defender
35:13
Speaker 1: use generative AI to write cover letters?
35:16
Speaker 2: Oh, that's a good one. Not a lot of, but
35:19
Speaker 2: we've seen it for sure. I would say maybe about
35:21
Speaker 2: three percent.
35:22
Speaker 1: Okay. If I want to use generative AI to write
35:27
Speaker 1: a cover letter to apply to work at Reality Defender,
35:29
Speaker 1: but I don't want to get caught, what should I do.
35:33
Speaker 2: Change about seventy five percent.
35:34
Speaker 1: Of the words Okay, who is Gabe Reagan?
35:41
Speaker 2: Gabe was? I think it was our VP of public
35:45
Speaker 2: relations or something like that. Here's a dfake. We we
35:48
Speaker 2: created him as a as kind of a as kind
35:51
Speaker 2: of a fun joke. But obviously we tell everyone.
35:54
Speaker 1: Tell me, tell me a little bit more about that.
35:57
Speaker 2: If you go on certain websites where you put your
36:01
Speaker 2: photo and maybe your job experience, there's quite a large
36:05
Speaker 2: number of deficke profiles on these websites like LinkedIn.
36:11
Speaker 1: Yes, huh, why why people be doing that?
36:17
Speaker 2: Sorry scammers?
36:18
Speaker 1: I'm trying to think, how do you get money out
36:20
Speaker 1: of people by having a fake LinkedIn account?
36:22
Speaker 2: Oh? I can tell you. Let's say you start the
36:25
Speaker 2: most popular ones that I'm aware of, is like cryptocurrency.
36:28
Speaker 2: Maybe you create a coin and you're like, here's a
36:31
Speaker 2: CEO and here's this person and they have these great
36:33
Speaker 2: LinkedIn profiles. Here's their photo and they're not real, but
36:36
Speaker 2: it sells a story. Right.
36:41
Speaker 1: Is it right that you founded a clothing company?
36:46
Speaker 2: I did?
36:46
Speaker 1: Yes, what's one thing you learned about fashion from doing that?
36:50
Speaker 2: It's much different than software development.
36:54
Speaker 1: Sure, I don't think you needed to start a company
36:57
Speaker 1: to learn that. I mean, the marginal cost is not
37:00
Speaker 1: zero for one thing.
37:02
Speaker 2: Yeah, the software is easy, you write some It's not
37:05
Speaker 2: easy at all. But what I mean is you're writing
37:08
Speaker 2: some code and you ship it. Versus in fashion, you
37:11
Speaker 2: have to have like you got to source the fabric.
37:13
Speaker 2: You gotta you gotta design it, you gotta make the patterns,
37:16
Speaker 2: you gotta cut it, sew it, make sure it fits.
37:18
Speaker 2: It's a lot more work.
37:22
Speaker 1: What are the chances that we exist in a simulation?
37:25
Speaker 2: You know, I used to think this is kind of
37:27
Speaker 2: a joke, but I don't know. I'm seeing every every
37:30
Speaker 2: month it seems to get higher. From my perspective.
37:35
Speaker 1: Why do you say that.
37:37
Speaker 2: I'm seeing what's happening with tech and what we're building,
37:40
Speaker 2: and there's you can see there's there was one paper
37:43
Speaker 2: where they took a bunch of agents and they gave
37:46
Speaker 2: them all a job and they start to do it
37:47
Speaker 2: and they just started to like create their own kind
37:50
Speaker 2: of like work cloths. Right, I don't know, it shuld
37:53
Speaker 2: be getting there.
37:53
Speaker 1: So so it's like, well, if we can create a
37:56
Speaker 1: simulation that seems like reality, maybe someone created a simulation
38:00
Speaker 1: that is our reality exactly. Yeah, what do you wish
38:05
Speaker 1: more people understood about AI.
38:07
Speaker 2: I mean, it's a tool, and I don't think people
38:09
Speaker 2: should be afraid of it. They should embrace it. And
38:12
Speaker 2: you know there's people are just running away from it.
38:16
Speaker 2: It's fantastic, it's great. Embrace it. Just be careful. One
38:20
Speaker 2: thing I'd like to tell, like my friends and family,
38:23
Speaker 2: especially with the e fake audio, have a safe word.
38:25
Speaker 2: As somebody calls you and you're like that's weird, you know,
38:29
Speaker 2: call him back or ask for a safe word.
38:31
Speaker 1: What do you wish more people understood about reality reality?
38:39
Speaker 2: I would say, just be aware that you exist. And
38:43
Speaker 2: every day's a gift, So you should be excited that
38:45
Speaker 2: you hear. Like the chances of you existing it's like
38:48
Speaker 2: you've won the lottery a million times, So every day's
38:52
Speaker 2: a gift.
38:56
Speaker 1: Ali Shakiyari is the co founder and CTO at Reality Defender.
39:01
Speaker 1: Today's show was produced by Gabriel Hunter Chang. It was
39:05
Speaker 1: edited by Lyddy Jean Kott and engineered by Sarah Bruguer.
39:09
Speaker 1: You can email us at problem at Pushkin dot fm.
39:13
Speaker 1: I'm Jacob Goldstein and we'll be back next week with
39:15
Speaker 1: another episode of What's Your Problem