Latest / The Nation Podcasts / Elon Musk Profits Off Non-Consensual Deepfakes w/ Kat Tenbarge | Tech Won't Save Us

Transcript

Elon Musk Profits Off Non-Consensual Deepfakes w/ Kat Tenbarge | Tech Won't Save Us · The Nation Podcasts

• 0:00
  [SPEAKER_01]: Deepfakes are really kind of like a wide-scale war on women as I've come to see it, and with looking at how rock was being weaponized, this is a way for misogynists to be able to control and punish women who they otherwise would not have access to.
• 0:32
  [SPEAKER_00]: Hello and welcome to Tech won't save us made of partnership with the nation magazine.
• 0:35
  [SPEAKER_00]: I'm your host Paris Marks and this week my guest is Kat 10-Barch.
• 0:38
  [SPEAKER_00]: Kat is an independent journalist who writes the Spitfire news newsletter.
• 0:42
  [SPEAKER_00]: You're probably just as enraged as I was to see what was going on on Elon Musk's X platform over the past month or so.
• 0:49
  [SPEAKER_00]: Where the safeguards were rolled back on the Grok Chatbot to allow it to generate non-consensual explicit deepfakes of virtually anybody.
• 0:57
  [SPEAKER_00]: men, women, but also even children and how shockingly bad the response was to what was going on.
• 1:04
  [SPEAKER_00]: You know, you essentially have a platform here that was already turned into this kind of right-wing space because of the way that Elon Musk has governed it and has changed the way that algorithmic amplification works and you know, the user base of the platform overall, but then you have him wanting even more engagement and seeing that playing into
• 1:24
  [SPEAKER_00]: the perverts and the misogynists who are on the platform and who are big fans of his could use this image generator in ways that are truly shocking and really to get away with it, right?
• 1:36
  [SPEAKER_00]: You know, he received this pressure from various governments who were not happy with what was happening but very few of them actually moved forward to practical actions to protect their citizens who are being victimized in the moment, just continue to allow this to happen and talked about
• 1:52
  [SPEAKER_00]: You know, maybe having repercussions or doing investigations or that sort of a thing, it was really terrible to see just how little our government seemed to be able to act against Elon Musk and this platform he owns even when it goes so far as not just to create.
• 2:07
  [SPEAKER_00]: non-consensual images of, you know, women and men of adults, but even of children, which is something that is so clearly illegal.
• 2:15
  [SPEAKER_00]: And so naturally, I felt we had to do an episode on this, and I could think of no better guest to come on to talk about what happened and the implications of it than cat.
• 2:25
  [SPEAKER_00]: She has been reporting on deep fakes on AI-generated explicit images for quite a long time now, and of course not only had the kind of background context that was necessary as this scandal broke out, but was obviously just as enraged about it as all the rest of us.
• 2:42
  [SPEAKER_00]: So with that said, I think we should just get into this conversation with Kat, I hope you enjoy it and, you know, it's not too shocking for you.
• 2:49
  [SPEAKER_00]: If you do like this conversation, you can leave a five-star review on your podcast platform of choice.
• 2:54
  [SPEAKER_00]: You can share the show on social media or any friends or colleagues who you think would learn from it.
• 2:58
  [SPEAKER_00]: And if you do want to support the work that goes into making tech won't save us every single week.
• 3:01
  [SPEAKER_00]: So we can keep having these critical in-depth conversations, looking at the ways that Elon Musk and so many other of these
• 3:10
  [SPEAKER_00]: Every single day, you can join supporters like Pranchu from India, Shivam from Toronto and Jordan in Denver, Colorado by going to patreon.com slash tech won't save us where you can become a supporter as well.
• 3:21
  [SPEAKER_00]: Thanks so much and enjoy this week's conversation.
• 3:24
  [SPEAKER_00]: Cat, welcome back to Tech Won't Save Us.
• 3:26
  [SPEAKER_01]: Thanks so much for having me back.
• 3:27
  [SPEAKER_00]: Absolutely.
• 3:28
  [SPEAKER_00]: Really excited to chat.
• 3:29
  [SPEAKER_00]: Unfortunately, it seems you know, I have you on to talk about probably the worst possible topics that we can talk about.
• 3:35
  [SPEAKER_01]: that is where my coverage always goes.
• 3:37
  [SPEAKER_00]: Yeah, at least you're doing the hard work of looking into this for people and making sure they're informed.
• 3:43
  [SPEAKER_00]: What I wanted to have you back on the show to talk about was what we have been seeing play out with Elon Musk's X platform and the Grot Chatbot over the past number of weeks where, you know, we have basically had
• 3:54
  [SPEAKER_00]: the platform take these restrictions off of the chatbot to allow it or make it much easier to generate non-consensual explicit images, not just of women and adults, but of children as well.
• 4:06
  [SPEAKER_00]: Obviously, there has been a whole firestorm around this.
• 4:09
  [SPEAKER_00]: And so I want to dig into a ton of aspects of this with you.
• 4:12
  [SPEAKER_00]: But I think just generally, as you were seeing this story,
• 4:16
  [SPEAKER_00]: break in late December early January, what was your initial reaction to this?
• 4:21
  [SPEAKER_00]: Were you surprised or did it just seem like a matter of time before something like this happen?
• 4:25
  [SPEAKER_01]: I definitely wasn't surprised, but it did remind me sort of as I've covered deep fakes over the past few years, there are like these cultural moments that are just like these peak points when things are really starting to shift and you just see this like massive influx of typically image-based sexual abuse, like produced with AI.
• 4:47
  [SPEAKER_01]: And that's exactly what this was.
• 4:48
  [SPEAKER_01]: And the way that the pattern always has gone, at least that I've observed,
• 4:52
  [SPEAKER_01]: is like, there will be this thing that seems so insignificant.
• 4:56
  [SPEAKER_01]: In this case, it was a viral tweet that basically said, like, I looked at Grox image output and I saw that it was being used to undress pictures of women and girls and that one post.
• 5:08
  [SPEAKER_01]: you know, got enough traction that other people went and started seeing that this was happening.
• 5:13
  [SPEAKER_01]: And then the outrage started to foment and grow and grow and grow.
• 5:17
  [SPEAKER_01]: And unfortunately, as we always see, alongside like the awareness that this is happening, the growth just accelerates and to skyrocket to an unbelievable degree.
• 5:29
  [SPEAKER_01]: So now that the numbers are in over the past month, we can see that this was again another major tipping
• 5:36
  [SPEAKER_01]: A library that you can hardly quantify or imagine or wrap your head around in terms of how many of these images were produced and how many people were victimized.
• 5:46
  [SPEAKER_00]: Absolutely.
• 5:47
  [SPEAKER_00]: Like I think there is a New York Times report just today as we're talking that put it in the millions, right?
• 5:51
  [SPEAKER_00]: Of the number of images generated just over the course of like a couple weeks at the at the peak period where this was really happening, right?
• 5:58
  [SPEAKER_01]: Exactly.
• 5:59
  [SPEAKER_00]: Yeah.
• 5:59
  [SPEAKER_00]: And I found it interesting what you said there about the tweet going viral because it did really seem like the restrictions were taken off of the grock chap out rolled back.
• 6:09
  [SPEAKER_00]: And it did seem like it took like a few days to a week or so for people to really realize what was happening and to start like reckoning with what, you know, the X platform, what Elon Musk, what his team had had really unleashed here, right?
• 6:23
  [SPEAKER_01]: a hundred percent.
• 6:24
  [SPEAKER_01]: And it's like so unfortunate that you can see almost exactly how this happened because the potential for this to happen has been there and sort of this practice has been happening now for years.
• 6:37
  [SPEAKER_01]: But what was really different about this moment and with the X platform specifically is that Elon Musk and the team behind X AI with him as it's like prominent,
• 6:50
  [SPEAKER_01]: they encouraged this cultural shift on the platform by rolling out all of these like sexy, flirty, spicy, NSFW functions and Elon himself obviously has a really misogynistic attitude toward within in general, toward marginalized people,
• 7:09
  [SPEAKER_00]: Yeah, really putting it lightly even.
• 7:11
  [SPEAKER_01]: Yes, exactly, exactly, like looking at Elon Musk, there's no question why so many men on the platform who look up to him and idolize him were inspired to create this type of content.
• 7:24
  [SPEAKER_01]: And so it was kind of happening quietly in the background as they rolled out these features.
• 7:31
  [SPEAKER_01]: And then, of course, in the new year, all it took was someone with a large following, spotting it happening for everyone to realize this is now happening.
• 7:40
  [SPEAKER_00]: And then, of course, it inspires more and more of that behavior, which is, you know, not surprising, but also like disgusting and uphorent at the same time.
• 7:49
  [SPEAKER_00]: Can you talk to us a bit about how this actually works?
• 7:52
  [SPEAKER_00]: Like, what were we actually seeing occur on this platform?
• 7:55
  [SPEAKER_01]: Yeah.
• 7:56
  [SPEAKER_01]: So the most common sort of frequent phenomenon that was happening is people were responding to women's photographs.
• 8:06
  [SPEAKER_01]: That could be a woman posting a selfie on X.
• 8:08
  [SPEAKER_01]: That could be a user pulling her pictures from some other place on the internet and they would post it on X and then reply an at-rock.
• 8:17
  [SPEAKER_01]: and say something like put her in a bikini or cover her face in whipped cream or turn her around and grock would often respond and edit the image in that fashion.
• 8:28
  [SPEAKER_01]: So the bikini pictures are really what captured the most attention.
• 8:33
  [SPEAKER_01]: because that was something that they could get the bot to easily do.
• 8:37
  [SPEAKER_01]: What we typically see with user behavior in these cases is that trends will emerge.
• 8:42
  [SPEAKER_01]: Like specific prompts will create a successful image.
• 8:46
  [SPEAKER_01]: Though result in the user getting what they wanted, which is to humiliate this woman.
• 8:50
  [SPEAKER_01]: And so that a bunch of other users will do the same thing.
• 8:53
  [SPEAKER_01]: And that's how like the bikini trend emerged.
• 8:57
  [SPEAKER_01]: And by the end of the first couple of days when this was proliferating,
• 9:01
  [SPEAKER_01]: You know, researchers found that more than 7,700 harmful images of this variety were being created every hour with grock.
• 9:11
  [SPEAKER_01]: And these ranged on a scale because there were some things that grock is still doing that don't fall under the range of sexually explicit territory.
• 9:21
  [SPEAKER_01]: But then there also was a lot of material that was actually explicit.
• 9:25
  [SPEAKER_01]: I've had researchers send me examples of like nudity of very realistic looking, jettitalia.
• 9:31
  [SPEAKER_01]: I've seen some pretty shocking stuff on the X platform publicly, and at the same time, what a lot of people don't know.
• 9:39
  [SPEAKER_01]: is that the standalone grock app and grock imagine, which is like a website, as an app, that was making even more sexually explicit and violent material.
• 9:49
  [SPEAKER_01]: So all of grock in all of its various forms, but what the most viral output was was what was happening on X publicly.
• 9:58
  [SPEAKER_00]: Yeah, I think that's a really good point to make and something that I want to come back to, though, right, because you have what's on the platform, but then you have this much bigger picture that seems to be occurring outside of exit self, and so I want to return to that in just a little bit, but in terms of like looking at what is happening on X and what grock is able to do on that specific platform.
• 10:17
  [SPEAKER_00]: Obviously, they were concerns about what was happening on the platform well before the end of October and these changes.
• 10:24
  [SPEAKER_00]: Can you talk about what was occurring on there before and to what degree any of this was actually possible before these changes were made?
• 10:32
  [SPEAKER_01]: Yeah, so I started covering deep fakes pretty routinely in early 2023.
• 10:39
  [SPEAKER_01]: And by that time, X was already the premier social media platform for deep fakes to get traffic on.
• 10:46
  [SPEAKER_01]: So from almost the beginning of this practice, becoming mainstream, X was just like a real environment for this to flourish.
• 10:55
  [SPEAKER_01]: And what I used to see a lot of was I would search for example for example I would search TikTokers names on the platform and I would just look at the chronological results of imagery associated with their names, the most recent post containing images and the words like Addison Ray or Charlie Demilio.
• 11:14
  [SPEAKER_01]: and I found deep fakes just running rampant.
• 11:17
  [SPEAKER_01]: And this was obviously happening in plain sight, but a lot of them weren't viral.
• 11:21
  [SPEAKER_01]: It was like this kind of thriving, almost underground community of deep fake proliferation, and it largely seemed to be targeting public figures, which makes sense because when it comes to the community aspect of deep fakes and the monetary aspect of deep fakes, it functions a lot like mainstream pornography, just with no consent.
• 11:42
  [SPEAKER_01]: And so women who are already famous, there's a real market and appetite for that content, but as the technology evolves alongside that demand, you start to see people going, I want to make deep fakes of my cousin, or I want to make deep fakes of some girl I'm in class with, or just this random person.
• 12:01
  [SPEAKER_01]: And so I saw all of that happening on X over the past few years and as time went by the incidents started getting more viral.
• 12:11
  [SPEAKER_01]: A lot of people will probably remember at the beginning of 2024 the Taylor Swift deepfaked incident.
• 12:16
  [SPEAKER_01]: where there were those AI-generated pictures of her in a football field, and that was probably the most massive incident that ex was embroiled in, but it was far from the only one.
• 12:27
  [SPEAKER_01]: The Taylor Swift stuff ex actually responded.
• 12:30
  [SPEAKER_01]: It was too late, the images were already viral and are still proliferating today, but you know, ex made it impossible to search Taylor Swift for a while.
• 12:38
  [SPEAKER_01]: They don't do that for for the vast majority of people affected, and there was legislation inspired by the Taylor Swift
• 12:46
  [SPEAKER_01]: you know, in Congress, other times I've seen this happen, the response has been much more minimal.
• 12:52
  [SPEAKER_01]: One of the most like shocking examples of this was in 2024, there was a viral deep fake of the actor Jacob awardee, and it was not hard to find.
• 13:02
  [SPEAKER_01]: It was all over X, and this was Jacob awardee's face on the body of someone who appeared to be like a porn performer.
• 13:10
  [SPEAKER_01]: Um, and these face swapped videos really, really common, really popular phenomenon.
• 13:14
  [SPEAKER_01]: What I found was the person who said it was their body in the video.
• 13:19
  [SPEAKER_01]: He told me that he was 17 when he took the video.
• 13:23
  [SPEAKER_01]: So that was essentially like C-SAM, child sexual abuse material being spread on X.
• 13:30
  [SPEAKER_01]: as this viral celebrity deep fake.
• 13:32
  [SPEAKER_01]: And I remember reaching out to exit the time, they took down some of the posts when I sent them to them.
• 13:37
  [SPEAKER_01]: But otherwise, like this has never been something that ex has really worked to contain.
• 13:42
  [SPEAKER_01]: They are more responsive than they are proactive.
• 13:46
  [SPEAKER_01]: And their response is often so much less than where it needs to be if they want us to stop happening.
• 13:52
  [SPEAKER_01]: So the vibe that I always had even before Brock was ex doesn't really care if their platform is a rampant space for this type of abuse of material, they're not really doing the work to ensure that it isn't.
• 14:05
  [SPEAKER_01]: And by the time it goes viral, there attempts that sort of like solving the problem do really very very little to stem those problems.
• 14:13
  [SPEAKER_00]: Yeah, I think really well said, and there's a bunch of things in there in that answer that I want to kind of drill down on a little bit further, right?
• 14:20
  [SPEAKER_00]: I want to start with that notion of the safety team, right?
• 14:24
  [SPEAKER_00]: And what is actually going on at exit self?
• 14:27
  [SPEAKER_00]: We know that when Elon Musk came into Twitter, he kind of gutted a lot of different parts of the company significantly reduced the head count at that platform before renaming it X and doing all this.
• 14:38
  [SPEAKER_00]: And there was this reporting in CNN about kind of like meetings that occurred before these changes to GROC and what it was able to do.
• 14:47
  [SPEAKER_00]: And it kind of suggested that there were even a couple people on what remained of the safety team there that even left before these things happened.
• 14:55
  [SPEAKER_00]: So what do we know internally about what is going on there and how this platform is actually looking at, you know, the kind of harmful material that could that can spread on it?
• 15:04
  [SPEAKER_01]: Absolutely.
• 15:05
  [SPEAKER_01]: I mean, I think as you said, the manpower to address these issues has just been decimated ever since Elon Musk took over and that was very intentional and that was very specific.
• 15:16
  [SPEAKER_01]: Some of the boards he disbanded and segments of the, like some of the employees who he let go, they were specifically in these roles around things like child safety.
• 15:27
  [SPEAKER_01]: So he really took a sledgehammer to any of the infrastructure that could have been around to address this, or raise these questions, or make sure that this didn't happen in the first place, and then in addition to that, when we look at sort of the priorities of the X team as communicated through their public posts.
• 15:46
  [SPEAKER_01]: as communicated through the actions that we see taking place on the platform, safety hasn't just been sort of like minimized as a concern.
• 15:55
  [SPEAKER_01]: We're really kind of seeing the opposite play out.
• 15:58
  [SPEAKER_01]: Like we are seeing people at the wheel of X, who are embracing the chaos and driving it in this direction.
• 16:06
  [SPEAKER_01]: And they're a little more subtle about it than getting out and saying like we don't care.
• 16:10
  [SPEAKER_01]: It's a CCM free for all.
• 16:11
  [SPEAKER_01]: Like they're not going to say that.
• 16:15
  [SPEAKER_01]: We love these depictions of women.
• 16:18
  [SPEAKER_01]: We want to see more of the type of stuff.
• 16:20
  [SPEAKER_01]: Like Elon Musk was responding to bikini edits with laughing emojis.
• 16:24
  [SPEAKER_01]: He frequently, like daily as posting these like sensual AI generated images of young looking women made with rock imagines.
• 16:33
  [SPEAKER_01]: So that's like the tone being set on the platform.
• 16:37
  [SPEAKER_01]: And the rules and the safety team are following along with that vision.
• 16:42
  [SPEAKER_00]: Yeah, it really is disgusting right like you have this man who is in his 50s has all of these children and you know different women who he's had children with and like is regularly posting as you're saying earlier you know these advertisements for the chatbot that can like have a not say for work mode and and stuff like that and then
• 17:02
  [SPEAKER_00]: these AI-generated images of women in like suggestive positions or even with like captions on it that you know is I don't know I find it kind of disturbing just to see like what he's up to and then to see like the reaction to to this whole moment where he seems to feel like it's a big joke right.
• 17:22
  [SPEAKER_01]: 100% and it's it's really interesting and kind of unique the relationships that the perpetrators of image-based sexual abuse on the platform have to Elon Musk.
• 17:33
  [SPEAKER_01]: Like I have found deep fakes on every major social media platform and I found tons of bad real violating content on all of these platforms and to a degree like there's nothing you can do to ensure that nothing better harmful is
• 17:49
  [SPEAKER_01]: But one of the major differences is just the level of sort of idolatry when it comes to Elon Musk.
• 17:55
  [SPEAKER_01]: I was looking at the accounts of people creating this material of people prompting rock to put her in a bikini and some of them had Elon Musk tweets as their banner.
• 18:05
  [SPEAKER_01]: and some of them had like number one Elon Musk fan in their bio and things like that.
• 18:11
  [SPEAKER_01]: And you don't really see that dynamic with someone like Mark Zuckerberg or one of the other text CEOs like Elon has this cold personality that he has created very intentionally and he has weaponized it and this is a consequence of that.
• 18:26
  [SPEAKER_00]: Yeah, even when you mention Mark Zuckerberg, like sure he's made this kind of like manly pivot recently, but I can't see many people being like, yeah, my male icon like for working out or whatnot is Mark Zuckerberg, like, yeah, I'm like, I'm not surprised to hear you say that about these people and the way that they look up to Musk, but it is still like it still turns my stomach to hear it.
• 18:47
  [SPEAKER_00]: You've talked about the way that these people are approaching women, the way that they see women, you know, in a very misogynistic way, and they're getting that kind of passed down from on-high, from the platform owner, from the richest man in the world, from this guy who has been held up from by all these media institutions and powerful figures for decades, and now is doing all of this with the power that he has accrued.
• 19:09
  [SPEAKER_00]: But there's another kind of
• 19:11
  [SPEAKER_00]: Dimension to this story, especially in the early days, and I don't know how much that was actually addressed later, but how with the graph was also being used to create images of children.
• 19:21
  [SPEAKER_00]: And you've mentioned this with the child's sexual abuse material.
• 19:24
  [SPEAKER_00]: See Sam, can you talk about that dimension of this?
• 19:27
  [SPEAKER_00]: And whether that has been properly rained in because it felt like the first week, there was a lot of attention on this, and then it felt like that really dropped off, and it was more about how this tool was being used against women after that.
• 19:38
  [SPEAKER_01]: Absolutely.
• 19:39
  [SPEAKER_01]: Yeah, I thought it was really interesting how the phenomenon kind of grabbed a lot of attention as framing grock as like a sea of generating machine, which is not entirely unfair.
• 19:49
  [SPEAKER_01]: I found in sort of reviewing the output from rock at various times throughout this controversy.
• 19:56
  [SPEAKER_01]: it's really hard to distinguish sometimes like are you looking at a picture of a child or are you looking at a picture of a grown woman and I think that a lot of perpetrators purposefully blur those lines where it's like they oftentimes focus on very young looking women but absolutely I mean just you know a couple days ago I was sent an account
• 20:18
  [SPEAKER_01]: of this guy who was posting pictures that were absolutely schoolgirls, like they were school-age children and he was having growth like turn them around.
• 20:27
  [SPEAKER_01]: And this was after X, you know, put in new restrictions, like this behavior is still happening all over the platform.
• 20:34
  [SPEAKER_01]: And I think a lot of times violence against women and girls and women in children is very interlinked.
• 20:41
  [SPEAKER_01]: Because it falls under a lot of the same sort of social patterns and impulses and biases that we recognize in this field of understanding abuse.
• 20:49
  [SPEAKER_00]: it makes me so mad like to even hear you hear you describe it, right?
• 20:52
  [SPEAKER_00]: So I want to ask you about the response to this as you were saying, you know, when they started to rain things in, but before we do that, I did want to dig a little bit more into the history and the consequences here, right?
• 21:02
  [SPEAKER_00]: So you talked about the history of the platform, what we have been seeing since 2023, but more broadly with explicit deepfakes and non-consensual AI-generated images,
• 21:12
  [SPEAKER_00]: Obviously this existed before generative AI, but generative AI feels like it has really kind of fueled and given a lot of acceleration to this practice.
• 21:22
  [SPEAKER_00]: So can you talk to me about the evolution of this that we've seen over the past number of years?
• 21:27
  [SPEAKER_01]: Absolutely.
• 21:28
  [SPEAKER_01]: Yeah, one of the things that I always try to communicate through my work on AI and image-based sexual abuse and harassment is like the ideas and the ideologies and even a lot of the behaviors at play are not new whatsoever.
• 21:43
  [SPEAKER_01]: So there's a legal scholar named Mary Anne Franks.
• 21:46
  [SPEAKER_01]: who she has told me in the past that like before AI, one of the ways that people would make this face swap, sexually explicit material is they would take physical magazines and they would cut out women's pictures, like celebrity women in girls and they would physically paste them over, like a pornographic magazine and then they would cut that out and mail it to the celebrity.
• 22:11
  [SPEAKER_01]: Yes, yes, so this was like the pre-internet well-it-advance of AI version of Deep Faking.
• 22:17
  [SPEAKER_01]: And at the time, the response from, you know, authority figures from people in these women's lives was like, oh, it's not a big deal.
• 22:26
  [SPEAKER_01]: Like, it's not real, just ignore it.
• 22:28
  [SPEAKER_01]: Maybe take it as a compliment, even.
• 22:31
  [SPEAKER_01]: And we have seen a lot of that sentiment kind of evolve alongside the proliferation of defect technology because it isn't really taken seriously.
• 22:40
  [SPEAKER_01]: And the vast majority of perpetrators are not held accountable in any way.
• 22:44
  [SPEAKER_01]: But the major distinction with defect technology and with rocks specifically is looking at sort of like the speed and scale and availability of what is happening.
• 22:55
  [SPEAKER_01]: So in the early days of deep fakes, which go all the way back to 2018 and even before, there were already communities on like Reddit, making like Scarlett Johansson deep fakes.
• 23:07
  [SPEAKER_01]: But at that point in time, it took a lot of sophistication, a lot of tech knowledge, and a lot of time and efforts to make a realistic looking deep fake, and even then the deep fake still weren't that realistic.
• 23:19
  [SPEAKER_01]: Today, the technology has evolved so much that you don't need any technical knowledge whatsoever.
• 23:25
  [SPEAKER_01]: All you need to be able to do is add, cross, and get the bot to do it for you.
• 23:29
  [SPEAKER_01]: So that obviously makes it much more accessible to anyone.
• 23:33
  [SPEAKER_01]: It also amplifies the scale at which this is happening.
• 23:37
  [SPEAKER_01]: So, for example, Genevieve O, the researcher, who got a lot of this data for journalist.
• 23:42
  [SPEAKER_01]: So, over the past few weeks, she found that rock was producing like 25 times more material than the top five deep fake porn websites combined.
• 23:55
  [SPEAKER_01]: So just like that scale is enormous.
• 23:59
  [SPEAKER_01]: And we've seen this before.
• 24:00
  [SPEAKER_01]: Back when I started reporting on this in 2023, a lot of these deep-picked websites were already out there and had been for years.
• 24:07
  [SPEAKER_01]: But there were some cultural moments that year where just like the rate of adoption.
• 24:12
  [SPEAKER_01]: skyrocketed.
• 24:13
  [SPEAKER_01]: And when these things skyrocket, they often don't come back down.
• 24:18
  [SPEAKER_01]: A lot of times like once people have figured out how to do it and sort of gotten a taste for it, they will find other ways to go about doing this.
• 24:26
  [SPEAKER_01]: So even if someone can no longer use grock to do exactly what they want, there are so many other apps out there, notifying apps have been,
• 24:38
  [SPEAKER_01]: and even Google and Apple have done a poor job of enforcing rules on apps that do this type of stuff.
• 24:46
  [SPEAKER_00]: I know that they even still run ads for those kinds of apps, like a Facebook and things like that, right?
• 24:51
  [SPEAKER_00]: I've seen reporting on that recently, which is so frustrating, right?
• 24:56
  [SPEAKER_00]: It's another one of those things that adds to the frustration around this whole issue.
• 25:00
  [SPEAKER_00]: You talked earlier, and I remember last time you were on the show as well, we were talking about this documentary,
• 25:08
  [SPEAKER_00]: Deepfakes made of her, you know, and this was like pre-generative AI, right?
• 25:12
  [SPEAKER_00]: And was kind of advocating for legislation in order to try to rain these things in.
• 25:16
  [SPEAKER_00]: And of course, you mentioned the Taylor Swift moment in early 2024 and have that prompted conversations around legislation and what a legislative response should be.
• 25:25
  [SPEAKER_00]: Have we seen anything tangible really come of those efforts in the past?
• 25:30
  [SPEAKER_01]: So it's interesting when it comes to sort of legislation and enforcement around deep fakes.
• 25:36
  [SPEAKER_01]: There has been a lot of progress over the past couple of years and we've seen action taken at the federal level with the TakeIt Down Act, which is very problematic for a whole host of reasons.
• 25:47
  [SPEAKER_01]: We've also seen different states, past legislation around deep fakes and from an international perspective, like Australia, the EU, other places have worked to pass measures around deep fakes.
• 25:58
  [SPEAKER_01]: The problem really comes in at the enforcement level, and there's really only one example that I am aware of of someone posting a deep fake on X and getting in trouble for it.
• 26:09
  [SPEAKER_01]: And this is an interesting case, Megan D. Stallion was targeted with deep fakes back in I believe 2024.
• 26:16
  [SPEAKER_01]: And she actually sued a woman who had re-shared some of these deep fakes and directed her audience to the deep fakes and she won that Deformation lawsuit in recent months.
• 26:27
  [SPEAKER_01]: But what's key there is that Megan was she knew who the perpetrator was.
• 26:32
  [SPEAKER_01]: She knew exactly who this woman was, and therefore she was able to see her for a defamation, and she didn't involve any of these newer statutes created around deep fakes.
• 26:42
  [SPEAKER_01]: She went for a more traditional, like, reputational harm, legal argument, and this situation is so unique, because the vast majority of the proliferation of this content is coming from anonymous users who could be based anywhere, or who could be disguising
• 26:59
  [SPEAKER_01]: and there's just not prosecutorial effort to go after any of these people.
• 27:04
  [SPEAKER_01]: The FBI and the DOJ thanks to the TakeIt Down Act.
• 27:07
  [SPEAKER_01]: They could be going after people doing this on X right now.
• 27:10
  [SPEAKER_01]: They have yet to publicly announce or investigate any of these cases.
• 27:16
  [SPEAKER_01]: So my suspicion is that most people who have been targeted over the course of this month by crock, a lot of them probably don't even know this is happening.
• 27:24
  [SPEAKER_01]: Those who do know it's happening don't have a lot of great options.
• 27:28
  [SPEAKER_01]: The options that they do have would be very costly, probably prohibitive for the vast majority of people.
• 27:34
  [SPEAKER_01]: And to pursue a case like this, requires such an enormous toll.
• 27:39
  [SPEAKER_01]: You have to popularize the images.
• 27:41
  [SPEAKER_01]: You have to think about this
• 27:45
  [SPEAKER_01]: energy expenditure, time, money, all of these different things that you may end up suffering more from trying to get justice than if you just sort of let it go.
• 27:54
  [SPEAKER_01]: So that's the unfortunate reality of where we stand with kind of enforcement around these things.
• 28:01
  [SPEAKER_00]: Yeah, I feel like the extent that I have seen of people being held to account at least like reading new stories and stuff like that is more like high school kids or what not being kind of like disciplined within the school environment rather than like through the legal system for doing this to their peers and things right like I haven't seen very many actually like legal cases or something where where someone actually feels the effects of what they've done right.
• 28:24
  [SPEAKER_01]: 100% yeah and even in those small scale community settings like an elementary school, the response varies so much depending on what the individual administrators decide to do.
• 28:35
  [SPEAKER_01]: There have been some really heartening cases where school administrators have taken it.
• 28:39
  [SPEAKER_01]: super seriously expelled perpetrators like insured that real consequences happen, but there have been just as many if not more examples of school administrators blaming the victim.
• 28:50
  [SPEAKER_01]: There was a case recently where a victim actually got expelled after she reported the perpetrators and there was basically no investigation.
• 28:58
  [SPEAKER_01]: The school administrator just assumed that she was lying.
• 29:01
  [SPEAKER_01]: So we still see these types of patterns play out at every level and I think that in itself relates so strongly to gender based violence in general, whether it's with the use of AI or not.
• 29:15
  [SPEAKER_00]: that's so shocking to hear of it.
• 29:16
  [SPEAKER_00]: Unfortunately, not surprising that some cases would be good in other cases, you have administrators who really don't want to do anything positive here, right?
• 29:24
  [SPEAKER_00]: You mentioned there like the effects of gender-based violence.
• 29:27
  [SPEAKER_00]: And do we know like what the effects of deepfakes of non-consensual images like this are on the people who, you know, they're created of and kind of weaponized against?
• 29:39
  [SPEAKER_01]: Absolutely.
• 29:39
  [SPEAKER_01]: I mean, we've seen so many different types of consequences, both that individual levels and at sort of systemic levels.
• 29:47
  [SPEAKER_01]: So at the individual level, there are different risk factors for different victims of this.
• 29:52
  [SPEAKER_01]: There have been victims including very young victims as well as victims from marginalized communities who have, you know, committed suicide because of this.
• 30:01
  [SPEAKER_01]: And sometimes that's because the victim is young and young people can often be really manipulated into thinking like my life is over, but also in some communities where religious modesty and cultural modesty are really prized, that can be a huge consequence because even if the material is not quote-unquote real, it can have the exact same impact depending on sort of your community norms.
• 30:26
  [SPEAKER_01]: And also at the individual level, there's a real psychological consequence at play here.
• 30:32
  [SPEAKER_01]: And it's something that applies to a lot of different forms of online harassment, where kind of the common knowledge or assumption is like there's that famous meme, like how it's cyber bullying, real just close your eyes.
• 30:44
  [SPEAKER_01]: and like that kind of stuff is like it's funny, it's comedic, but the reality is so much darker because the reality of what we know about online harassment is that our brains perceive these things that happen to us online as physical threats.
• 30:59
  [SPEAKER_01]: So when you are being physically threatened online or when you see a fake of yourself, you will have a physical reaction.
• 31:05
  [SPEAKER_01]: Your palms will shake.
• 31:06
  [SPEAKER_01]: You'll feel scared.
• 31:08
  [SPEAKER_01]: Your fire flight response will kick in.
• 31:10
  [SPEAKER_01]: You may endure the same consequences of what we would think of as quote unquote
• 31:16
  [SPEAKER_01]: It's still real, and your body is like perceiving it as happening to you.
• 31:21
  [SPEAKER_01]: So there are a lot of consequences that happen at that level, and then at the more systemic level, I think what's really unfortunate is that through reading like deep fake forums and through pouring over sort of what the perpetrators talk about and how they refer to what they're doing, a common thread that has emerged is similar to other forms of patriarchal violence.
• 31:42
  [SPEAKER_01]: This sort of end goal here is to punish women at large as a group, make it harder to be a woman, and specifically to sort of push women out of the public eye by retaliating and making it so dangerous to exist in these online spaces.
• 31:58
  [SPEAKER_01]: And we can actually see that happening on X because so many women have left the platform.
• 32:03
  [SPEAKER_01]: I left the platform in response to, you know, gender based violence on the app when Trump got reelected.
• 32:09
  [SPEAKER_01]: When Nick Funt has was saying, like, your body, my choice, that for me was like the breaking point of, I don't want to be here anymore.
• 32:16
  [SPEAKER_01]: This environment is psychologically destructive to me as a woman.
• 32:20
  [SPEAKER_01]: And a lot of people have left social media because of this.
• 32:24
  [SPEAKER_01]: I've seen multiple examples of, you know, high profile within leading social media because of the rock trend, because they were victimized by it or because they're scared of it.
• 32:32
  [SPEAKER_01]: I've seen so many women say, like, I don't post pictures of myself online anymore.
• 32:36
  [SPEAKER_01]: I would never let my daughter post pictures of herself online.
• 32:40
  [SPEAKER_01]: And that may seem like good safety advice in theory.
• 32:43
  [SPEAKER_01]: What that really means is, like, women aren't living their lives fully or equally in these spaces that are so important.
• 32:53
  [SPEAKER_01]: Because these are democratic spaces.
• 32:54
  [SPEAKER_01]: These are places where we make our voices heard.
• 32:57
  [SPEAKER_01]: These are places where we build careers.
• 32:59
  [SPEAKER_01]: These are places where we share our perspectives and knowledge and insights with the world.
• 33:03
  [SPEAKER_01]: And so deep fakes are really kind of like a wide scale war on women as I've come to see it.
• 33:09
  [SPEAKER_01]: And with looking at how rock was being weaponized, one thing that really struck me was like, this is a way for misogynists to be able to control and punish women who they otherwise would not have access to.
• 33:22
  [SPEAKER_01]: like it's a way to sort of see that conservative ideology in action at scale.
• 33:28
  [SPEAKER_00]: Really well put and like we can have our criticisms of social media and how these platforms work and you know what these companies have done to like these spaces where we communicate with one another but at the same time right now is that like
• 33:42
  [SPEAKER_00]: there shouldn't be these kind of structural barriers that are created that are enabled by people like Elon Musk that exclude whole swans of people from these spaces because they don't feel safe to communicate in them, right?
• 33:54
  [SPEAKER_01]: Exactly.
• 33:55
  [SPEAKER_01]: It's like a it's the digital version of the political project to ultimately undermine women's autonomy to make it harder to run for office as a woman.
• 34:04
  [SPEAKER_01]: There's no, it's not a coincidence that Alexandria Ocasio Cortez is one of the highest profile victims of the baking and she's also one of the most public facing women in politics like these things are aligned.
• 34:18
  [SPEAKER_01]: These sort of like goals and missions are aligned and I really do think that Elon Musk very intentionally and with full knowledge of what would happen like introduced these features to the platform.
• 34:29
  [SPEAKER_00]: definitely.
• 34:29
  [SPEAKER_00]: I even think back to like the past few years right and how for a while there, a lot of people were really kind of worried and concerned about being open and like admitting the politics that Elon Musk held, right?
• 34:42
  [SPEAKER_00]: Like I remember even when he was like,
• 34:45
  [SPEAKER_00]: you know, very clearly supporting like abortion bands in Texas, but like wouldn't say it explicitly.
• 34:51
  [SPEAKER_00]: People were like, oh, you know, you don't know.
• 34:52
  [SPEAKER_00]: Maybe he doesn't like whatnot.
• 34:54
  [SPEAKER_00]: But it was like, no, you can tell like he's disowning his trans daughter.
• 34:58
  [SPEAKER_00]: He very clearly sees women as, you know, kind of vessels for his procreation.
• 35:03
  [SPEAKER_00]: Like, this is not a man that should be given the benefit of the doubt.
• 35:07
  [SPEAKER_00]: Yet, even to this day, it feels like with his political positions, that that still remains the case,
• 35:12
  [SPEAKER_01]: Totally totally and a lot of people have sort of defended all of this under the guise of like it's freedom of speech It's a freedom of speech issue and the US government like federally has kind of taken this position And I've seen a lot of like commentators take this position and the reality is that it is a free speech issue But the free speech that's being suppressed is the free speech of the women and girls who are being affected by this as well as women and girls in general as a group
• 35:39
  [SPEAKER_01]: because we're all threatened by the normalization and allowance of the type of behavior.
• 35:43
  [SPEAKER_00]: Yeah, creating non-consensual deep fake images in AI generated images of people is not your free speech.
• 35:49
  [SPEAKER_00]: You know, that's not something you have a right to.
• 35:52
  [SPEAKER_00]: So I want to ask you about the response to this as well, right?
• 35:54
  [SPEAKER_00]: I want to start with the platform itself, right?
• 35:56
  [SPEAKER_00]: So it started to get all of this criticism from governments, from, you know, the public generally.
• 36:01
  [SPEAKER_00]: I'm sure from a certain segment of users of the X platform as well.
• 36:05
  [SPEAKER_00]: How did X and X AI respond as the kind of concern in the scandal around this was escalating?
• 36:12
  [SPEAKER_01]: So the response from the company is that it was very delayed.
• 36:17
  [SPEAKER_01]: There were several days that went by before they really responded to anyone and meanwhile the XAI press account responds to every request within automated response that just says legacy media lies, which is a classic like Elon Musk move.
• 36:33
  [SPEAKER_00]: at least it's not like the poop emoji anymore, but like it's not any better.
• 36:37
  [SPEAKER_01]: Yeah.
• 36:37
  [SPEAKER_01]: Exactly.
• 36:37
  [SPEAKER_01]: Exactly.
• 36:38
  [SPEAKER_01]: And so, and one thing that people picked up on in the early days of this that was so annoying is there were some legacy media outlets that were asking crock for comment and then frinting it.
• 36:50
  [SPEAKER_01]: Oh my god.
• 36:52
  [SPEAKER_01]: Oh.
• 36:52
  [SPEAKER_00]: I was going to say, like, there were all of these headlines, right?
• 36:55
  [SPEAKER_00]: That were like, Grok says whatever or what?
• 36:58
  [SPEAKER_00]: As though, like, Grok itself was like impersonated, was thinking, was like acting on its own.
• 37:02
  [SPEAKER_00]: And it's like, no, there's like Elon Musk and all these little guys turning the dials behind the scenes.
• 37:08
  [SPEAKER_00]: That is resulting in this, right?
• 37:09
  [SPEAKER_00]: It's not like the chatbot itself has any agency here.
• 37:12
  [SPEAKER_01]: Exactly.
• 37:13
  [SPEAKER_01]: There's like this huge tech literacy gap around conversational AI, where it's like, even authority figures treat it like it is a conscious thing that is capable of forming response.
• 37:25
  [SPEAKER_01]: It is reflecting what you ask it.
• 37:27
  [SPEAKER_01]: Like when you prompted to do something, it's just trying to please you by like regurgitating what you said back to you.
• 37:33
  [SPEAKER_01]: So that being said, like that was
• 37:39
  [SPEAKER_01]: statement a few days afterwards and they basically say an Elon Musk says this too.
• 37:44
  [SPEAKER_01]: Anyone who posts anything illegal to the platform will be treated the same way regardless of its AI or not, which was kind of a true statement in that when you post illegal material on X there is no guarantee or likelihood that anything will happen to you either.
• 38:00
  [SPEAKER_01]: Like the platform has become known for videos depicting like extreme violence, extreme gore, things like bestiality, sexual abuse, and of course the aforementioned deep fakes.
• 38:11
  [SPEAKER_01]: So he wasn't lying when he said that they weren't going to treat the content any differently.
• 38:15
  [SPEAKER_01]: It's just that they don't do anything about these problems, so that was the initial response.
• 38:20
  [SPEAKER_01]: And then the backlash really started to build.
• 38:23
  [SPEAKER_01]: And we initially saw this in the form of other governments responding to what was happening on X like Malaysia and Indonesia, which were the first governments to actually temporarily ban the platform.
• 38:35
  [SPEAKER_01]: India also like a government minister there, you know, took action, lost an investigation.
• 38:41
  [SPEAKER_01]: We saw the UK's independent regulatory bodies start to investigate the EU, France, California as its own entities started to investigate X.
• 38:50
  [SPEAKER_01]: So you saw all of this growing international pressure mainly coming from other western governments and Southeast Asian governments.
• 38:58
  [SPEAKER_01]: And so alongside that, there's like the growing user backlash more and more people speaking out about this and finally in really in response to that pressure and I think specifically that regulatory pressure like the fear of those fines, the fear of the app being completely blocked in places like the UK.
• 39:15
  [SPEAKER_01]: That's when you know last week I believe is when they said we're.
• 39:20
  [SPEAKER_01]: Well, first they were strict in the ability to create or to use grock to edit images to just paying premium users, which didn't stop the problem at all because those paying premium users are often the Musk fanboys who were driving a lot of this behavior.
• 39:35
  [SPEAKER_00]: Exactly.
• 39:35
  [SPEAKER_00]: And I even saw tweets, like, you know, when people would do that at grock and, you know, ask it to undress somebody.
• 39:41
  [SPEAKER_00]: I don't know if it still does it, but there was a period of time where it would respond and say, like, you can only do this if you're a
• 39:49
  [SPEAKER_00]: an explicit upsell on your attempt to create non-consensual defects like, uh, yes, absolutely.
• 39:54
  [SPEAKER_01]: And it is still doing that.
• 39:56
  [SPEAKER_01]: I'm pretty sure.
• 39:56
  [SPEAKER_01]: And so yeah, they're basically just at that point monetizing the image-based sexual abuse.
• 40:02
  [SPEAKER_01]: So that was the first response.
• 40:04
  [SPEAKER_01]: And then after more sustained pressure, that's when they announced that they were rolling back the ability.
• 40:10
  [SPEAKER_01]: for Grok to put real photos of people in revealing clothes.
• 40:14
  [SPEAKER_01]: That was kind of the way that they phrased it.
• 40:16
  [SPEAKER_01]: And with these efforts put into place, we did see a decrease.
• 40:21
  [SPEAKER_01]: Because at its peak when anyone could do this to anyone at any time, there were over 7,700 images per hour being created.
• 40:29
  [SPEAKER_01]: After the changes, we were still seeing a ton of images, like we were still seeing thousands of images being created per hour, but there was a decrease.
• 40:38
  [SPEAKER_01]: And then after they rolled the restrictions back, seemingly entirely, but I'll get into that in a second.
• 40:42
  [SPEAKER_01]: When they restricted it even more, we again saw another decrease.
• 40:47
  [SPEAKER_01]: But the reality of the situation was,
• 40:49
  [SPEAKER_01]: A, they left a lot of the most harmful material up.
• 40:53
  [SPEAKER_01]: So like you can still go back and look at these pictures that were created in the first place and people very quickly found ways to get around these restrictions.
• 41:01
  [SPEAKER_01]: So like the researcher who I've been in communication with, she was immediately able to find numerous examples of really sexually explicit like bad stuff still being created.
• 41:12
  [SPEAKER_01]: uh, really realistic looking like genitalia, bikinis.
• 41:15
  [SPEAKER_01]: She's still was able to find those, like stuff like the little girls being told to turn around, things like that.
• 41:20
  [SPEAKER_01]: That was all still happening and something that I personally observed as a shift in sort of this culture is an emerging phenomenon that is now really, really rampant involving like commanding grock to take off religious items of clothing like the hijab or put on religious items of clothing.
• 41:40
  [SPEAKER_01]: So now a lot of the prompts that I'm seeing are rock put her in a sorry, like the Indian traditional cultural clothing and or like rock put her in a nun's habit or like rock like cover her up and it doesn't fall under the same legal definitions of what we think of as actually explicit because it's literally the opposite, but that impulse and that ideology is the same.
• 42:04
  [SPEAKER_01]: So the boundaries have just shifted and there's this gray area around what they're doing because a lot of what they're doing is still pretty nakedly abusive at harmful especially like at grock take off her job and show her with curly hair I see that all the time.
• 42:21
  [SPEAKER_01]: And like that is really harassing, but they know that they can get around sort of the restrictions that have been put in place.
• 42:28
  [SPEAKER_01]: And the root of this behavior is obviously the desire to control within, to take away their autonomy.
• 42:34
  [SPEAKER_01]: And you know, it's very sexist sort of ideology at play.
• 42:37
  [SPEAKER_00]: Yeah, and I feel like that example just like adds to what you're saying as well, right?
• 42:41
  [SPEAKER_00]: Where it's like, okay, it's not even just about taking the clothes off of all these women.
• 42:45
  [SPEAKER_00]: It's like even just changing them, putting, you know, again, taking a hijab off, where that's part of like your cultural or religious identity can still be like very invasive and, you know, very abusive and things like that.
• 42:59
  [SPEAKER_00]: It just makes it so bad to hear about all these things.
• 43:01
  [SPEAKER_00]: I want to ask you as well about like the media response, right?
• 43:04
  [SPEAKER_00]: Because we talked about how some of these headlines were reporting on grock talking about this in a certain way, but I also felt that as X was announcing these changes to the platform and how it was going to work, we very quickly got these headlines like, oh, the platform is raining this in.
• 43:21
  [SPEAKER_00]: You know, they're taking action to address this.
• 43:23
  [SPEAKER_00]: And then as you're saying, when researchers or journalists certain journalists started digging into what was actually happening, they quickly found that it doesn't sound exactly like what X suggested these things can still happen or they're just monetizing it.
• 43:37
  [SPEAKER_00]: I wonder what you make of, I guess, the way that the media reported on this and how it wasn't always completely accurate or reflective of what was actually happening.
• 43:45
  [SPEAKER_00]: In a case that was important to give the public accurate information here, right?
• 43:50
  [SPEAKER_01]: 100% and I think that over the past few decades we've seen some advances in how the media often covers gender based violence.
• 43:58
  [SPEAKER_01]: One thing that I compare it to just to show how stark that difference can be is when what we call oftentimes revenge porn, a colloquially.
• 44:07
  [SPEAKER_01]: When that started to become a really viral phenomenon in the like 2000s and the 2010s, a lot of the media framing is that it was the victims fault for taking these photos in the first place and like if you think back to how the media treated like prominent celebrity women who experienced that, totally it was extremely like victim playing.
• 44:26
  [SPEAKER_01]: And today things have gotten a little bit better, but as you're describing here, there is still way too much of a reliance on sort of like the authority figures in the equation.
• 44:36
  [SPEAKER_01]: And we see that with media coverage in lots of different respects.
• 44:40
  [SPEAKER_01]: You see it with how the media gives the police so much authority when they cover crime.
• 44:45
  [SPEAKER_01]: You can see it in how the media gives like the federal government so much authority when they cover politics and you can see it how in how media companies give tech companies the benefit of the doubt when we cover these patterns of abuses and the reality is just because the tech company says that it is fixing the problem as we should all know by now that does not mean they are fixing the problem.
• 45:08
  [SPEAKER_01]: And in fact, a lot of these tech companies, when AI was starting to evolve into what we think of it as now, when that was happening a few years back, a lot of these companies signed on two pledges around non-consensual intimate imagery where they pledged and promised that they would do everything in their power to stop this from becoming the crisis that it is today, and all the media outlets wrote that up feverishly.
• 45:32
  [SPEAKER_01]: Like all these, they're doing a great job
• 45:35
  [SPEAKER_01]: they are not doing a great job and yet still we return to this idea that when Elon Musk and his company say they're doing something that means it can be reported on as fact and I wish that as an industry we would stop taking these guys at their word when they've proven over and over again that we're foolish to do so.
• 45:54
  [SPEAKER_00]: Yeah, it even makes me think of like whenever Elon Musk says something that sounds like any bit like sci-fi that gets written up as like headlines Elon Musk says this is coming whatever, but when he posts his like white nation was tweets and whatnot That doesn't get the same degree of coverage and attention as the the nice like science fiction is is really coming But as you say that too like it even makes me think of how early after the Twitter acquisition Elon Musk was talking a lot about taking on child sexual abuse material like I can't remember the exact language that he was using in that moment
• 46:24
  [SPEAKER_00]: And even as researchers were saying, this is proliferating on the platform, and now, of course, we've seen his AI chatbot actually create child sexual abuse material images, just like, boggles of mind, have there's not more accountability on this man for what he's doing, right?
• 46:39
  [SPEAKER_01]: 100% and it is so reminiscent of how child safety and protecting children and stopping child trafficking and child abuse and all these other things it is so frequently like a pillar of conservative rhetoric around like this is why we need ice this is why we're doing all of these things but the reality is it's a total displacement of responsibility and it's essentially just like a bait and switch.
• 47:02
  [SPEAKER_01]: because the reality is that these authorities are often the ones perpetuating the problem and letting it happen.
• 47:08
  [SPEAKER_01]: They just use that rhetoric to gain more popularity and to get themselves into these positions of power.
• 47:13
  [SPEAKER_00]: Definitely.
• 47:14
  [SPEAKER_00]: And so I wanted to ask you about the government response, right?
• 47:17
  [SPEAKER_00]: You mentioned in particular seeing the South Asian and South Asian governments moving really quickly to try to rain the sand or even to temporarily ban the platform.
• 47:26
  [SPEAKER_00]: We heard a lot of kind of talk.
• 47:28
  [SPEAKER_00]: from European governments, but at least at this point as we talk, it doesn't seem to have amounted to much in terms of enforcement or anything.
• 47:36
  [SPEAKER_00]: So I wonder what you make of how they responded to this and what you think we really should be seeing as government see this happening as we see this AI generated kind of deep fake crisis continually seemingly getting worse, and it doesn't feel like they're doing enough.
• 47:50
  [SPEAKER_00]: Like what do you think they should be doing here
• 47:55
  [SPEAKER_01]: Yeah, it's really, it's a really interesting distinction because it's not bad that these governments, you know, made these threatening statements that they rushed to investigate that they called this out as wrong in theory and even in practice to some extent that was a good thing.
• 48:22
  [SPEAKER_01]: But I think when you look closer at sort of the underlying reasoning behind a lot of these government responses and also sort of the structure of a lot of legislation here in the US to one of the issues is that the focus is so frequently on the obscenity component.
• 48:40
  [SPEAKER_01]: And the focus here oftentimes is like, well, this is bad because it's sexually explicit or this is bad because it's pornographic and so therefore it becomes a very narrow definition of what exactly is bad and what exactly these governments care about stopping.
• 48:57
  [SPEAKER_01]: And so that is how we wind up in the situation where we are now.
• 49:01
  [SPEAKER_01]: where they have removed or at least limited the ability to put someone in a bikini or to create public graphic nudity, but meanwhile, behind closed doors if you have the grock app, you can still do all of that stuff and worse, and even publicly we've just seen it shift.
• 49:18
  [SPEAKER_01]: to, you know, put her in a job or take her a hijab off.
• 49:21
  [SPEAKER_01]: And because that doesn't fall under the boundaries of what's considered sexually explicit or what's considered pornographic or obscene, you're not going to see the same government response.
• 49:29
  [SPEAKER_01]: And so, you know, a lot of the scholars and advocates who I've spoken to about this have argued that we really need to sort of change how we think about these issues.
• 49:38
  [SPEAKER_01]: The problem isn't that all nudity is bad and in fact that line of thinking will often carry into censorship and often carry into threats to freedom of expression and censorship and discrimination against sex workers and things like that.
• 49:53
  [SPEAKER_01]: The reality is that the problem is the desire to control women and that is something that our legal field does not have the language for yet.
• 50:03
  [SPEAKER_01]: or does not have the desire to put in our existing framework of right and wrong.
• 50:09
  [SPEAKER_01]: And so I think that will require a much bigger cultural shift to understand that the root of the problem is not just like the notifying aspect of this, but that it's like the control aspect of this.
• 50:20
  [SPEAKER_01]: and it's the misogynistic aspect of this.
• 50:23
  [SPEAKER_01]: And that is really like the root of this and associated behaviors, not just like this actually explicit function, but because that is where so much of like the government focus is, you see things like the take it down act, which the take it down act is not working as intended.
• 50:40
  [SPEAKER_01]: The take it down act was supposed to ensure that something like this wouldn't happen and that if it did happen, you would see this prosecution at the highest level.
• 50:48
  [SPEAKER_01]: We're not getting that prosecution.
• 50:49
  [SPEAKER_01]: What a lot of people fear is going to happen with the Take It Down Act is that in just a few months, in May of this year, the second half of that legislation is going to come into effect platforms like X and also all the other major social media platforms are supposed to create these removal flag and removal systems where you or anyone can say like, here's a defect of me take it down and they have 48 hours to do that.
• 51:14
  [SPEAKER_01]: But the problem with these incoming systems is that they could so easily be weaponized and used for censorship and used for anything, because platforms don't actually have time to look at every single thing that's gonna get flagged.
• 51:26
  [SPEAKER_01]: A lot of them, as we already know, are gonna operate with an over-bundance of caution.
• 51:30
  [SPEAKER_01]: And when Trump signed to take it down act into law, he even said he pointed out Linda Yaccorino, who was then the CEO of X and was then in the audience, and he said, like,
• 51:44
  [SPEAKER_00]: Right.
• 51:45
  [SPEAKER_00]: Okay.
• 51:46
  [SPEAKER_01]: And then the other thing he said was like this is so great for me because no one's treated worse than I am on the internet.
• 51:54
  [SPEAKER_01]: So it's very clear how this could become a tool of federal censorship versus what it was supposed to do, which is help victims, but this White House administration is obviously not focused on anything to help victims of sexual violence.
• 52:08
  [SPEAKER_00]: Well, they're focused on the biggest victim of all, right, Donald Trump, he's the victim of that.
• 52:13
  [SPEAKER_01]: Exactly, exactly.
• 52:15
  [SPEAKER_00]: Oh my god, as you were describing that though, like I was immediately, and like, you know, the cultural differences and like the issues with nudity generally or even just sexual content, it brought me back to like the early days of the App Store and have Steve Jobs who's like, we can't have apps that have like nudity or anything like that, like we're not going to allow it on the App Store.
• 52:35
  [SPEAKER_00]: and the Europeans were like, but we don't have a problem with that.
• 52:37
  [SPEAKER_00]: Why are you sticking your kind of cultural norms on to us and the rest of the world?
• 52:42
  [SPEAKER_00]: Because this makes no sense, right?
• 52:44
  [SPEAKER_00]: So you can totally see how those views definitely shape legislation, especially in the United States, and then how that can filter out two other places as it's normalized in that market and the major market of the internet and whatnot.
• 52:56
  [SPEAKER_00]: and then it's adopted elsewhere.
• 52:58
  [SPEAKER_00]: I feel like one of the points that angered me or that really stood out to me is like, listen, I guess people can debate about when this is being done to adults.
• 53:07
  [SPEAKER_00]: And I think both of you and I would be on the side of like this is not acceptable when it's done to adults because it takes away power, it takes away agency, there should be accountability for something like this.
• 53:17
  [SPEAKER_00]: But they're like clear laws.
• 53:20
  [SPEAKER_00]: about this being done to children and about this kind of material being created of children.
• 53:26
  [SPEAKER_00]: And I found it really shocking that there was not a stronger response at least on that side of things and how quickly that seemed to fade from the discussion and I wonder what you make of that.
• 53:37
  [SPEAKER_01]: Yes, you're absolutely right and I've even interviewed a legal scholar who made the argument that this has been illegal since the 90s because back in the late 90s there were laws passed around how children could be depicted in CGI and sort of what how that applied to to see Sam and sort of like fake.
• 53:56
  [SPEAKER_01]: But realistic looking content.
• 53:59
  [SPEAKER_01]: So this has been illegal and prosecutable for a very, very, very long time.
• 54:03
  [SPEAKER_01]: But the reality is that it rarely is taken seriously and followed through.
• 54:08
  [SPEAKER_01]: And in sort of my experience reporting on gender based violence at large, that doesn't surprise me because I've seen this time and time again, where like,
• 54:18
  [SPEAKER_01]: it falls to the FBI it falls to local police it falls to people in these very jurisdictions and it just doesn't get done like the cases just get dropped things do not get followed up on I think what's even concerning here is sort of this idea that because the problem is so pervasive because there are a million of these images out there like we're not even gonna try
• 54:40
  [SPEAKER_01]: We're not even gonna instill the pretense that we're gonna go after this type of behavior, whether it's illegal or not.
• 54:46
  [SPEAKER_01]: And I think that is really discouraging for anyone sort of in this field, but it also sort of speaks to the way that rhetoric around caring about children is so easily weaponized, even when the people behind that rhetoric have no intention of following it through.
• 55:02
  [SPEAKER_01]: And right now, one of the major conversations happening is around children and online safety.
• 55:08
  [SPEAKER_01]: I mean, if so much legislative momentum in the US, but also all around the world, saying, like, we got to take kids off the internet, like, kids shouldn't be allowed on the internet.
• 55:18
  [SPEAKER_01]: That's how we're going to keep kids safe.
• 55:19
  [SPEAKER_01]: When the reality is, is that as adults, we can't keep kids safe on the internet.
• 55:24
  [SPEAKER_01]: Like, even if kids aren't allowed to use X, what we have here as a problem is that adults are posting their pictures, and like, there's nobody doing anything about that.
• 55:34
  [SPEAKER_01]: So I think that a lot of times when we look at the discourse, politically and culturally and otherwise around children and safety in the internet, it's like, we kind of have our targets all wrong.
• 55:45
  [SPEAKER_01]: It's not that the kids are posing a daydream to themselves necessarily.
• 55:49
  [SPEAKER_01]: It's more so that as adults we have yet to figure out a way to hold each other accountable for hurting children.
• 55:55
  [SPEAKER_01]: And that's gonna be a problem with AI.
• 55:58
  [SPEAKER_01]: It's already such a pervasive problem in every respect.
• 56:01
  [SPEAKER_00]: I think that this is such a good example like this whole kind of like scandal right around how, okay, listen, you can have a debate around raising an age limit on what social media should be or whatnot, but it shows very clearly that the problem isn't the age that you go on to social media or these platforms, but
• 56:20
  [SPEAKER_00]: the broader way that we have allowed these companies to create these platforms and govern these platforms.
• 56:25
  [SPEAKER_00]: And it feels like we need much more of like a rulebook or like expectations and standards on how those platforms should operate to make sure that we reduce the potential for harm for everybody, adult or child alike, rather than being like, okay, you're not a certain age, so you can't use it, but everyone else can still like be abused or have mental health impacts or whatever it is we want to talk about.
• 56:47
  [SPEAKER_00]: because we're not going to pay attention to how the platform operates in any other context, right?
• 56:52
  [SPEAKER_01]: A hundred percent.
• 56:53
  [SPEAKER_01]: And to me, this has so many parallels to the Epstein story as well, which was a case involving a lot of victims who were underage, as well as a lot of victims who were adults.
• 57:03
  [SPEAKER_01]: And now that the child victims are adults and still continuing to advocate for themselves, they're still not being taken seriously.
• 57:09
  [SPEAKER_01]: Like, you see how this is a problem that is not just
• 57:15
  [SPEAKER_01]: but rather is a game much further reaching pervasive like cultural attitude and the lack of follow-up with rock is then reflected in all of these other prominent cases.
• 57:25
  [SPEAKER_00]: a really great point.
• 57:26
  [SPEAKER_00]: I have a couple final questions for it before we wrap up.
• 57:30
  [SPEAKER_00]: The first is that, okay, we've been talking a lot about growth, right?
• 57:32
  [SPEAKER_00]: We've been talking about a lot about what has been happening on the X platform or even what you can do with growth outside of X.
• 57:38
  [SPEAKER_00]: And obviously what happens on X is important because of the virality component, right?
• 57:43
  [SPEAKER_00]: Because it can really take off a lot of people can see it.
• 57:46
  [SPEAKER_00]: But we know that there is this kind of much
• 57:48
  [SPEAKER_00]: bigger network of generative AI tools, open source models that are used for different things and what have you that are also used to victimize people, to victimize women in particular, but also children because of the lack of guardbrails here.
• 58:03
  [SPEAKER_00]: Do you have any idea of what we can actually do to try to tackle this much bigger issue just beyond this one social media platform
• 58:12
  [SPEAKER_01]: It's such a great question, and I think that a lot of times I'm more scared of what we're not seeing, then what we are actually seeing because it's so much harder to even get a grasp of what's happening behind the scenes with these essentially like closed door platforms, the graph app that applies to that, but it also applies to all of these other technologies, all these other applications.
• 58:35
  [SPEAKER_01]: And so, I think within the evolution of AI in order to create products that are actually safe for consumer use and don't perpetuate these problems, the act of resisting and opposing sexual abuse has to be an act of consideration throughout the entire process.
• 58:54
  [SPEAKER_01]: Because it starts with sort of the programming and the data sets on which these AI technologies are trained on.
• 59:00
  [SPEAKER_01]: Researchers at I believe Stanford found that in popular data sets there were authentic images of C-SAM that AI tools were being trained on so it is baked in at that level and I think a lot of people have sort of figured this out through casual use like
• 59:18
  [SPEAKER_01]: AI depictions of women, they tend to be white, they tend to be young, they tend to be conventionally attractive and they tend to be sexual or sensual.
• 59:29
  [SPEAKER_01]: And so that is a very clear bias.
• 59:31
  [SPEAKER_01]: In a lot of these AI technologies and that can't be solved on just the user end.
• 59:36
  [SPEAKER_01]: It has to start on the end of the people who are creating and producing these tools.
• 59:42
  [SPEAKER_01]: And so I think that's something that has to be addressed in this industry in order for any of this to get better.
• 59:47
  [SPEAKER_01]: And in my experience reporting on sort of AI consumer products at how they've been adopted and what this looks like.
• 59:55
  [SPEAKER_01]: I find that the industry is pretty rotten when it comes to this issue.
• 59:59
  [SPEAKER_01]: Like you look at some of the earliest iterations of this technology and the earliest applications of this technology.
• 1:00:06
  [SPEAKER_01]: And it's like,
• 1:00:07
  [SPEAKER_01]: Sexual abuse is so baked in.
• 1:00:09
  [SPEAKER_01]: You go back to 2018 and those videos of Scarlett Johansson and it's like before charging PT before OpenAI before any of the stuff reached the mainstream on this massive scale.
• 1:00:20
  [SPEAKER_01]: This was already what it was being used for.
• 1:00:22
  [SPEAKER_01]: And that's a story that's been echoed repeatedly throughout the tech industry's history.
• 1:00:27
  [SPEAKER_01]: We've seen it time and time and time again.
• 1:00:29
  [SPEAKER_01]: So right now, all of the incentives are the opposite of what they need to be.
• 1:00:33
  [SPEAKER_01]: We need like a complete reversal and to kind of like, I would argue like destroy and rebuild in order to actually get better.
• 1:00:42
  [SPEAKER_00]: Listen, I'll completely agree with you on that one.
• 1:00:45
  [SPEAKER_00]: I love that idea.
• 1:00:46
  [SPEAKER_00]: But even as you're saying that, like a brought to mind, I think it was the end of last year, when Sam Altman was basically like open about the fact that they are going to push more of like the app as a companion and like the kind of AI girlfriend kind of stuff because they recognize that that application got a lot more engagement.
• 1:01:02
  [SPEAKER_00]: regardless of the broader consequences that might come with it.
• 1:01:06
  [SPEAKER_00]: As we end off this conversation, I know that you wrote a piece recently about getting off of X and not using that platform anymore.
• 1:01:14
  [SPEAKER_00]: So I wanted to end with everything that we've been seeing with the Deepfake scandal, but also everything before, do you think people should be getting off this platform and abandoning Elon Musk's social media?
• 1:01:25
  [SPEAKER_01]: I would love if more people would take that leap.
• 1:01:28
  [SPEAKER_01]: I think at this stage, I mean, it's something that I think about all the time.
• 1:01:32
  [SPEAKER_01]: And it was a very personal decision for me, where I just reached a moment, where physically, individually, I was like, oh my god, I can't take it anymore.
• 1:01:41
  [SPEAKER_01]: Cause I didn't, I honestly didn't expect to leave X. I thought of myself as like a long hauler, who was like, you're not gonna take my platform away from the Elon Musk.
• 1:01:52
  [SPEAKER_00]: I remember when he took it over, I was like, I'll be here until it dies and then that changed.
• 1:01:57
  [SPEAKER_01]: Yeah, I mean, I think at this stage, it's just like the whole platform has been essentially murdered and then reincarnated as like the worst possible version of itself.
• 1:02:09
  [SPEAKER_01]: So I know for me as an individual, like there's no place for me on there.
• 1:02:13
  [SPEAKER_01]: And I still spend a lot of time on the platform as a reporter.
• 1:02:17
  [SPEAKER_01]: So, in some ways, that almost feels like cheating, because I'm like, oh, I can still see all like the good jokes, but I'm not actually using it, or at least I'm not like contributing to it.
• 1:02:30
  [SPEAKER_01]: But I can see for other people why this is a very fraught decision.
• 1:02:34
  [SPEAKER_01]: I have heard a lot of stories from a lot of people about why they keep using X, and I can really relate to that and understand that.
• 1:02:42
  [SPEAKER_01]: Like one reason that I've heard is because there are people who have built support networks on X, who get support for just everyday necessities.
• 1:02:51
  [SPEAKER_01]: They have a community there that helps them get through a foreign life, or perhaps that is where their business is located, that's something that I hear from.
• 1:02:59
  [SPEAKER_01]: Content creators a lot is like,
• 1:03:01
  [SPEAKER_01]: They feel like they can't leave because it's still such an important tool and platform for them.
• 1:03:05
  [SPEAKER_01]: And then, ideally, they wouldn't have to leave like this is a platform that has been co-opted and really like made worse for them in a lot of ways.
• 1:03:14
  [SPEAKER_01]: And that's why also, even if I'm not still on X, I find it so important to continue reporting on X because there are people who really kind of feel like they're stuck there and that their experience is being degraded and it's really being turned against them.
• 1:03:29
  [SPEAKER_01]: content creators and sex workers in particular, it's like they are sort of forced to live within the confines of this platform and the platform is becoming more and more violent towards them.
• 1:03:40
  [SPEAKER_01]: So they deserve better.
• 1:03:42
  [SPEAKER_01]: And so I do think that a lot of people, I would encourage more people to get off X because I think at the end of the day, like the complaints about blue sky are like, oh, there's not as much stuff over there.
• 1:03:52
  [SPEAKER_01]: Like, where am I going to get my discourse
• 1:03:56
  [SPEAKER_01]: And I'm like, well, if you posted it over there, then it wouldn't be over there.
• 1:04:01
  [SPEAKER_01]: And yeah, so I would encourage more people to try out the alternatives, because X is unique in its current positioning as like an enemy force.
• 1:04:12
  [SPEAKER_01]: Like, it is the only social media platform that has been directly conjoined with Trump's administration, where like it is
• 1:04:22
  [SPEAKER_01]: So I'm like, this makes it like uniquely bad, but I know why people struggle with it.
• 1:04:28
  [SPEAKER_00]: totally it as much as I like recognize and accept the reasons that you gave for why people still might be on there.
• 1:04:34
  [SPEAKER_00]: Still when I because I occasionally pop back on to see if I need to see like what some politician has posted or something like that and see the number of people who are still engaged on it.
• 1:04:43
  [SPEAKER_00]: I have a harder and harder time kind of like understanding why you would still be over there.
• 1:04:48
  [SPEAKER_00]: It's like come on at some point you and I felt like especially seeing this
• 1:04:57
  [SPEAKER_01]: totally, and one thing that I really wish would happen is I wish that these politicians would get off X.
• 1:05:03
  [SPEAKER_01]: What's so wild about that is they're not even getting any engagement.
• 1:05:06
  [SPEAKER_01]: For one of my rock stories, I reached out to all of the representatives who co-sponsored the Take It Down Act and the Kids Online Safety Act and some of these other bills, and they are all posting on X every single day.
• 1:05:17
  [SPEAKER_01]: Some of them are getting zero likes.
• 1:05:19
  [SPEAKER_01]: I'm like, why are you there?
• 1:05:21
  [SPEAKER_01]: Just get off.
• 1:05:22
  [SPEAKER_01]: It's no, it deserves no value to you.
• 1:05:24
  [SPEAKER_01]: It's bizarre to me.
• 1:05:25
  [SPEAKER_01]: So I do think that a lot of people in positions of power have no excuse to be on the next anymore.
• 1:05:31
  [SPEAKER_00]: Totally.
• 1:05:32
  [SPEAKER_00]: And government accounts and stuff like that too, right?
• 1:05:34
  [SPEAKER_00]: You see next to no engagement in local government and even some like state or provincial government accounts.
• 1:05:39
  [SPEAKER_00]: It's like, okay, this needs to change.
• 1:05:41
  [SPEAKER_00]: This needs to be like the moment where you all start to evacuate from X.
• 1:05:45
  [SPEAKER_00]: Kat, it's always great to get your insights.
• 1:05:47
  [SPEAKER_00]: This was so informative.
• 1:05:48
  [SPEAKER_00]: Thank you so much for taking the time.
• 1:05:50
  [SPEAKER_00]: And of course, I'd highly recommend people go check out Spitfire, where you're constantly writing really informative stories about these issues, and so much more.
• 1:05:57
  [SPEAKER_00]: Thanks so much.
• 1:05:58
  [SPEAKER_01]: Thanks for having me.
• 1:06:02
  [SPEAKER_00]: Cat Timberg is an independent journalist who writes the Spitfire News news article.
• 1:06:06
  [SPEAKER_00]: Tequon Sevas is made in partnership with the nation magazine and is hosted by me Paris Marks.
• 1:06:10
  [SPEAKER_00]: Production is by Kylie Houser.
• 1:06:11
  [SPEAKER_00]: Tequon Sevas relies on the support of listeners like you to keep providing critical perspectives on the tech industry.
• 1:06:16
  [SPEAKER_00]: You can join hundreds of other supporters by going to patreon.com slash Tequon Sevas and making a pledge of your own.
• 1:06:21
  [SPEAKER_00]: Thanks for listening and make sure to come back next week.