Latest / Ctrl-Alt-Speech / Live at TrustCon 2026

Transcript

Live at TrustCon 2026 · Ctrl-Alt-Speech

• 0:08
  Ben Whitelaw: Hello, and welcome to this very special Trust Con edition of Ctrl-Alt-Speech.
• 0:19
  Ben Whitelaw: You got the memo.
• 0:20
  Ben Whitelaw: Uh, Con- Ctrl-Alt-Speech is the podcast where we make sense of the major debates shaping online speech, content moderation, platform power, and the future of the internet.
• 0:30
  Ben Whitelaw: It is absolutely brilliant to see so many of you here back at Trust Con again.
• 0:34
  Ben Whitelaw: Um, you've had three very intense days of panel discussions, corridor conversations, and maintaining that ever depleting social battery, um, that, that many of you have.
• 0:44
  Ben Whitelaw: So it's fantastic that you've taken the time out here.
• 0:47
  Ben Whitelaw: Um, I'm very personally proud to be back at Trust Con.
• 0:49
  Ben Whitelaw: I wasn't here last year.
• 0:50
  Ben Whitelaw: I didn't get to be part of this.
• 0:51
  Ben Whitelaw: Uh, I had a little baby boy, and I'm...
• 0:54
  Ben Whitelaw: can confirm that he is okay.
• 0:56
  Ben Whitelaw: And yes, I'm still very tired.
• 0:58
  Ben Whitelaw: Uh..
• 1:02
  Ben Whitelaw: But yeah, for those who don't ma- know me, I'm Ben Whitelaw.
• 1:04
  Ben Whitelaw: I'm the founder and editor of Everything in Moderation, an independent media outlet focusing on trust and safety, online regulation, and online speech.
• 1:12
  Ben Whitelaw: Um, for this very special recording, we have a fantastic panel with us.
• 1:16
  Ben Whitelaw: The man at the end needs no introduction.
• 1:18
  Ben Whitelaw: He is the founder and editor of Techdirt.
• 1:20
  Ben Whitelaw: He is the co-host of Ctrl-Alt-Speech.
• 1:22
  Ben Whitelaw: He is the man whose articles we send to family and friends when we want something complex explained, and we, and we can't be bothered to do it.
• 1:29
  Ben Whitelaw: It is Mike Masnick, everyone.
• 1:36
  Ben Whitelaw: And the two women in the middle also barely need introduction.
• 1:39
  Ben Whitelaw: Um, they are kinda longstanding members of this community.
• 1:42
  Ben Whitelaw: Uh, Kat Duffy is senior fellow for digital and cyberspace policy, and director of the Lead AI at Council of Foreign Relations.
• 1:51
  Ben Whitelaw: Uh, she has spent more than two decades working across technology, democracy, corporate responsibility, and human rights, and she has one of the best Twitter handles known to man.
• 2:00
  Ben Whitelaw: Uh, go and look that up.
• 2:01
  Ben Whitelaw: Uh, give a warm welcome to Kat, please.
• 2:07
  Kat Duffy: To be, to be clear, it's a Bluesky handle, no?
• 2:09
  Ben Whitelaw: And a Mastodon one as well, yeah.
• 2:11
  Kat Duffy: And a Mastodon as well.
• 2:12
  Ben Whitelaw: But I found her on Twitter.
• 2:13
  Ben Whitelaw: Um, also to her left, we've got Zoe Darma, who is, uh, the director of trust, knowledge, and information products, which is a hell of a title, at Google.
• 2:23
  Ben Whitelaw: Um, and has previously worked at a plethora of organizations and institutions, including Microsoft, Meta, and the UN, no less.
• 2:30
  Ben Whitelaw: So please give Zoe a warm welcome to the podcast as well.
• 2:38
  Ben Whitelaw: I should say that when I have told people that Kat and Zoe will be on the podcast, they have done one of two things.
• 2:44
  Ben Whitelaw: Their eyes have widened, um, in excitement, or they have rubbed their hands with glee.
• 2:50
  Ben Whitelaw: So this could get very, very interesting and very fun very quickly.
• 2:53
  Ben Whitelaw: Um, yeah, so it's, it's amazing to be here, and we've got a fantastic panel, and, uh, we're looking to get started.
• 3:00
  Mike Masnick: Yeah, and, uh, before we start getting into the stories, I did wanna thank TSPA for, uh, allowing us to do this for the third year in a row.
• 3:08
  Mike Masnick: Um, we'll see if, how well we do to come back again in the future.
• 3:12
  Mike Masnick: But, uh, TSPA, again, I think everybody here recognizes they put on a really wonderful conference.
• 3:17
  Mike Masnick: Trust Con every year gets better and better, uh, and more exhausting for me personally, but I think it, it's been wonderful.
• 3:23
  Mike Masnick: So I just wanna start out by thanking all of the TSPA people, volunteers, um- And, and yeah, everyone involved with that.
• 3:34
  Mike Masnick: So thank you so much.
• 3:36
  Mike Masnick: Um, as I said, this is a live recording of a podcast.
• 3:40
  Mike Masnick: The actual recording should go out on the feed on Friday, uh, assuming everything goes well.
• 3:46
  Mike Masnick: So you can go back and re-listen to it and hear yourselves scream and clap and all that kind of fun stuff.
• 3:52
  Mike Masnick: Um, but, uh, if you're looking for something to listen to on the way home, if you traveled here, uh, today, just earlier today, we put out, uh, a spotlight, s- a sponsored spotlight episode, uh, with Dan Hayes, uh, from PWC talking about their new trust and safety outlook, talking about major themes related to trust and safety, what is happening today, and how trust and safety can be more strategic.
• 4:16
  Mike Masnick: Uh, it was a really interesting discussion that Ben had with Dan that we recorded yesterday?
• 4:21
  Mike Masnick: Monday?
• 4:21
  Mike Masnick: I can't remember.
• 4:22
  Mike Masnick: I don't know what day it is.
• 4:23
  Ben Whitelaw: It's all the
• 4:23
  Mike Masnick: same.
• 4:23
  Mike Masnick: Don't worry about it.
• 4:24
  Mike Masnick: Uh, also the other announcement before we get into stuff is that, uh, different from past years, we now have a Patreon.
• 4:31
  Mike Masnick: Uh, and you can support us on the Patreon if you go to patreon.com/ctrlspeech.
• 4:35
  Mike Masnick: Somebody cheered.
• 4:36
  Mike Masnick: That was great.
• 4:36
  Mike Masnick: We like it.
• 4:37
  Mike Masnick: Um, if you look on a bunch of seats, especially ones in the front, and I see there's still, on this side of the room, there are still some empty seats.
• 4:45
  Mike Masnick: There should be some cards, some of which give discounts.
• 4:50
  Mike Masnick: If you want to sign up for our Patreon, you can sign up and get a special discount.
• 4:53
  Mike Masnick: If you're not seated in a seat with a card, you can get up and find one, move forward a little bit, or you can find Ben or I after the podcast, and we will magically hand you a card that gives you a discount on the Patreon.
• 5:06
  Mike Masnick: If you subscribe to the Patreon, you get early access to our podcast, and you get the extended versions where we think nobody's listening, so we get really spicy.
• 5:15
  Mike Masnick: Uh I think that's it.
• 5:18
  Mike Masnick: I think let's get into the stories.
• 5:19
  Ben Whitelaw: Yeah, that's enough admin.
• 5:20
  Ben Whitelaw: Let's, let's do this.
• 5:21
  Ben Whitelaw: Um, you will get a chance to ask questions at the end of our stories that we'll run through, so, um, if you feel desperate, put a pin in it, we'll come to you for sure.
• 5:30
  Ben Whitelaw: Um, and yeah, let's kick off with a story that I've heard discussed quite extensively in corridors over the last few days.
• 5:35
  Ben Whitelaw: Um, one person that I walked past ex- literally exclaimed,"Holy shit," when, uh, when they heard about this.
• 5:41
  Ben Whitelaw: And this is the, the story that OpenAI's, one of O- OpenAI's models has gone rogue and hacked Hugging Face.
• 5:48
  Ben Whitelaw: Um, Mike, you're gonna lead us on this.
• 5:50
  Ben Whitelaw: What caught your eye and, and kind of give us the background to it, first of all.
• 5:53
  Mike Masnick: Yeah, so this, I think a lot of people have seen this and, and I keep talking to people and I keep hearing people like,"What, what?
• 5:58
  Mike Masnick: What happened?" And the...
• 6:00
  Mike Masnick: It's interesting to sort of follow the timeline of how this came out, which was that on Friday Hugging Face made this announcement,"Oh, we got hacked and it was clearly some sort of automated agent, and it did all these things and it kept escalating and getting around guardrails, and, you know, we, we think that it was, you know, some sort of, uh, very strong model, and w- our own models that were trying to stop it were unable to, or to recreate what happened, uh, and we don't know who did it." And the implication, sort of reading through it, or at least m- my read on it when I first saw it, was like, it feels like nation state hacked into the, this, you know, AI platform and was trying to figure out what was going on.
• 6:38
  Mike Masnick: Then two days go by, Monday, and there's an announcement from OpenAI, uh, where they say,"We have partnered with Hugging Face to explore, uh, this hacking and figure out what happened," because it turns out that it was actually OpenAI, that they were testing the, their new Sole model and another undisclosed model where they were trying to, um, test some of its cybersecurity capabilities, and they were using one that had basically the guardrails turned off, but in theory they thought was sandboxed, and that it wasn't supposed to be able to get out.
• 7:18
  Ben Whitelaw: It's a very evil laugh you've got, sir.
• 7:21
  Mike Masnick: I like it.
• 7:21
  Mike Masnick: No, there was- That's perfectly
• 7:22
  Ben Whitelaw: timed
• 7:22
  Mike Masnick: a great, great laugh from the audience.
• 7:24
  Mike Masnick: Um- And so what, what happened was it was trying to figure out effectively how to beat this specific evaluation, uh, and slowly but surely kind of kept escalating and trying different things and figured out how to get out of the sandbox, get out onto the internet, figured out that Hugging Face would have the details of the eval, figured out how to get stolen credentials to hack into Hugging Face, and then begin to go around and explore and exploit its access to Hugging Face.
• 8:00
  Mike Masnick: So there are a number of different strange things about this that have people freaking out.
• 8:05
  Mike Masnick: Um, there's, you know, just the fact that these models are getting so powerful.
• 8:10
  Mike Masnick: There's the fact that, um, uh, OpenAI was apparently not monitoring where this-- uh, what this model was doing over a course of a few days perhaps.
• 8:20
  Mike Masnick: Um, there's, you know, there's the cynical version of this that I heard from a couple people that, uh,"Oh, this is OpenAI who is upset about all the marketing that Anthropic got with Mythos," where they're saying,"Oh, this is too powerful.
• 8:35
  Mike Masnick: We can't release it." They said,"We need our own story of the too powerful model.
• 8:39
  Mike Masnick: How about the one that hacked H- uh, Hugging Face?" That's a very cynical take.
• 8:44
  Mike Masnick: But, uh, it does seem to show an escalation in a, in a different sort of way.
• 8:49
  Mike Masnick: When we, you know, normally are talking about You know, safety issues with AI, you know, people...
• 8:56
  Mike Masnick: Th- th- this was sort of the science fiction version of it, but now there's at least a, a somewhat real example of it sort of going rogue and, and being able to, to get out into the world.
• 9:07
  Ben Whitelaw: Yeah.
• 9:08
  Ben Whitelaw: Zoe, you've, you've almost stopped shaking your head.
• 9:12
  Ben Whitelaw: Give, give us your thoughts on, on this because, um, it, it's fairly unprecedented, isn't it?
• 9:16
  Zoe Darme: Yeah.
• 9:17
  Zoe Darme: The, the thing I first thought when I heard about this story was, gosh, the models are like the A students, right?
• 9:22
  Zoe Darme: They...
• 9:23
  Zoe Darme: You tell them what you want them to do, and they really, really, really, really wanna go and do that.
• 9:29
  Zoe Darme: And so, um, that was my first reaction to it.
• 9:31
  Zoe Darme: I think the other reaction that I had to it are just a couple of things.
• 9:35
  Zoe Darme: One is, um, reflecting on three days, it has been three days?
• 9:40
  Zoe Darme: I don't...
• 9:40
  Zoe Darme: It's like the days are all running together, on, um, on our profession.
• 9:45
  Zoe Darme: I did actually like how clear it was.
• 9:48
  Zoe Darme: You laughed and you were cynical, but, uh, the announcement that OpenAI and Hugging Face were working together on this because it really shows that, um, some of these safety and security issues shouldn't be solved in isolation.
• 10:01
  Zoe Darme: They probably can't be solved in isolation.
• 10:03
  Zoe Darme: So there will need to be that continuation of what we do as a profession best, which is set aside ki- kind of the competitive pressures and figure out, um, how to address some of these incidents or, uh, just hard problems fa- uh, facing our industry.
• 10:20
  Zoe Darme: And then the other thing I, I think about it is, um, you mentioned that it had, that, that the incident team had to turn to an open source model, uh, that had some of the, uh, training, uh, the gu- uh, safety guardrails, uh, turned off so that the team could do the analysis of the attack.
• 10:39
  Zoe Darme: Um, and I tho- I thought that was particularly interesting.
• 10:42
  Zoe Darme: Um, you know, I, I won't get into,"Oh my gosh, it's a Chinese model.
• 10:45
  Zoe Darme: What do we think about that?" Um, and all of that, but, uh, but I thought those types of things were...
• 10:51
  Zoe Darme: That's where my head went to.
• 10:52
  Zoe Darme: But Kat, I don't know, we talked about this at midnight last night, so maybe you have- That's true.
• 10:56
  Zoe Darme: That's true more coherent thoughts.
• 10:58
  Kat Duffy: Uh, uh, I mean, uh, cringing face.
• 11:00
  Kat Duffy: Like I, I, um, I like angry face.
• 11:03
  Kat Duffy: Like, I don't know.
• 11:04
  Kat Duffy: Um, you know that phrase, to, to your point on like the model is the A student.
• 11:09
  Kat Duffy: First of all, I feel like if there's anyone in here who hasn't seen the Reese Witherspoon movie from forever ago, Election, where she's Tracy Flick.
• 11:16
  Kat Duffy: Basically, Tracy Flick is n- like embodies what I feel like these models look like, and I highly recommend that everyone watch it this weekend.
• 11:25
  Kat Duffy: Um, that, and that phrase of like you never admit, like, or never attribute to malevolence that which you can attribute to incompetence.
• 11:32
  Kat Duffy: Uh, w- I don't know here if this is...
• 11:37
  Kat Duffy: W- we, we know it wasn't malevolent, thankfully.
• 11:40
  Kat Duffy: Right.
• 11:40
  Kat Duffy: Because that would be e- even scarier if it had been.
• 11:44
  Kat Duffy: But also like- I don't know if we can attribute this to incompetence of the team that built this sandbox and was running this agent because they had pulled down some barriers because its job was to be testing for software vulnerabilities, which it certainly found.
• 11:59
  Kat Duffy: Um, or on the other hand, like now it's gonna be never attribute to malevolence what you can attribute to breathtaking, unexpected, unprecedented competence, which to some degree is how this agent operated.
• 12:10
  Kat Duffy: I think the takeaway is there's, you know, some major takeaways for me.
• 12:13
  Kat Duffy: One is that when you see this type of reporting coming out, it, it clarifies again that there is no responsibility for any company to disclose any of this right now.
• 12:25
  Kat Duffy: And I wrote about this a few months ago when we were looking at the Chinese incursion into Anthropic, and Anthropic took a really, like a significant political risk in disclosing that that incursion had occurred.
• 12:35
  Kat Duffy: People liked it.
• 12:36
  Kat Duffy: They didn't like that they disclosed it.
• 12:37
  Kat Duffy: They didn't like how much they disclosed it, yada, yada, yada.
• 12:39
  Kat Duffy: But they did not have to disclose anything, and they did.
• 12:43
  Kat Duffy: There is no, there is no ground right now.
• 12:46
  Kat Duffy: There is no floor in terms of what needs to be disclosed with the frontier systems, with agents, how they're operating, what breaches are occurring.
• 12:56
  Kat Duffy: More importantly, there is no space, especially a legally protected space, for those companies and d- and the deployers to be able to share information about the breaches that they're seeing, about the incursions that they're seeing.
• 13:10
  Kat Duffy: And so we know anecdotally that there's sort of bilat sharing that will happen sort of between the frontier companies, right?
• 13:17
  Kat Duffy: Or between a company, like a deployer and a frontier company.
• 13:20
  Kat Duffy: But there is no legally created, protected, safe space in which these disclosures can occur.
• 13:28
  Kat Duffy: And that is incredibly important because we are not going to get an attack into one model or one company.
• 13:36
  Kat Duffy: That is not the way this is gonna look.
• 13:38
  Kat Duffy: Attacks are gonna go across platforms.
• 13:39
  Kat Duffy: They're gonna go across different companies.
• 13:41
  Kat Duffy: They're gonna go across different models.
• 13:42
  Kat Duffy: They're gonna stack different models.
• 13:44
  Kat Duffy: And this is where I come back to we've, we've had a fundamental breakdown as well in the firmament of our cybersecurity reporting and vulnerability infrastructure with the demolition of CISA, with the defunding of the multi-state ISAC.
• 13:57
  Kat Duffy: We're gonna see that same thing now with the defunding of the elections ISAC.
• 14:01
  Kat Duffy: And so Right, we, right now what we've had in the federal government's move to pull back on a lot of governance has also left a massive convening and authoritative hole where we could have built for this type of disclosure, for this type of protection, and for this type of independent reporting and analysis that we don't even-- It's not just that we need this as a community, the entire world needs this from the United States because we are at the forefront of these models.
• 14:33
  Kat Duffy: And so these are all for me at a systems level things that I really have in mind.
• 14:37
  Mike Masnick: Do, do you think...
• 14:37
  Mike Masnick: I mean, the trust and safety world has often come up with different systems and organizations and setups for signal sharing around different things, whether it's terrorist content or CSAM or, or whatnot, um, mostly not intermediated by government or setting up, you know, spaces that involve the government.
• 15:01
  Mike Masnick: Is that the solution that these companies-- I mean, I, I, I wasn't being, I, I-- The cynical part was, was the way that OpenAI sort of presented as,"Oh, we've partnered with," as opposed to, you know.
• 15:14
  Mike Masnick: Uh, but, like, I do think it's good that, that Hugging Face and OpenAI are working together to sort of figure this out.
• 15:20
  Mike Masnick: Um, but is that, uh, the kind of thing that we should hope to see or expect to see organizational infrastructure where the companies get together and set up systems for sharing, you know, early alerts or, or ways to notify each other like,"Hey, our bot went rogue and, and, and attacked you"?
• 15:39
  Mike Masnick: Though I should note, by the way, somebody-- I saw somebody earlier complaining about people saying,"Oh, the bot went rogue." But as both of you were saying, like, it didn't go rogue, it just did what they t-they, what it told you to do and, and, you know, figured out the best way to do that, which just happened to be, you know, getting out of the sandbox and hacking.
• 15:56
  Mike Masnick: Um-
• 15:56
  Kat Duffy: Yeah, it went Tracy Flick.
• 15:58
  Mike Masnick: Yes.
• 15:59
  Mike Masnick: To, to, to, yes, to the extreme digital level.
• 16:02
  Mike Masnick: But, um, do you think that, that it's a place where, you know, the, the industry side of it should be putting together the frameworks and structures i-in order to do some kind of signal alert, uh, sharing for these kinds of situations, especially with the government sort of stepping back from it?
• 16:22
  Kat Duffy: I mean, to some degree, some of these already exist.
• 16:24
  Kat Duffy: So you have things like Cyber Threat Alliance, right?
• 16:26
  Kat Duffy: You have a couple of different cybersecurity infrastructural spaces in which some reporting can be done.
• 16:31
  Kat Duffy: What we're seeing in that space is the same thing that I think we're seeing actually in the child safety space.
• 16:35
  Kat Duffy: The pipes are getting flooded because there are so many vulnerabilities that are being found, right?
• 16:41
  Kat Duffy: Because people are running models to find vulnerabilities.
• 16:44
  Kat Duffy: So now, like, the reporting mechanisms for vulnerabilities, and it's not clear if there was an incursion or not.
• 16:49
  Kat Duffy: So in the same way that we're getting tons of reporting on, like, is it synthetic media?
• 16:52
  Kat Duffy: Is it true-- Like, is it synthetic CSAM?
• 16:54
  Kat Duffy: Is it real CSAM?
• 16:56
  Kat Duffy: How-- You know, uh, we are essentially Even in the cyber reporting models right now, it's almost like a DDoS in terms of the existing pipelines and mechanisms that we have to take it in.
• 17:07
  Kat Duffy: But the other thing is that this cannot be industry-funded and industry-led and industry-driven.
• 17:13
  Kat Duffy: Like at some point, especially in a CapEx race as the bloodbath to the degree that we are currently in, the companies should not have to weigh whether or not they are taking on an economic or political risk by voluntarily disclosing the mere basics when you could level the play fie- playing field by saying everybody has to disclose this certain amount.
• 17:38
  Kat Duffy: There's mechanisms that you can create to incentivize that.
• 17:41
  Kat Duffy: You wanna be able to give companies air cover, for example, to be able to disclose that this happened without necessarily saying it happened to them.
• 17:47
  Kat Duffy: So it can be good to have an intermediary, but you do not wanna have an intermediary that is completely funded by industry, because if industry is funding the intermediary and the interme- intermediary becomes, uh, annoying for industry, it's also then very easy for industry to walk away and the funding falls down.
• 18:06
  Mike Masnick: I mean, in, in this case, obviously, the company's publicly reported it.
• 18:10
  Mike Masnick: There have been other breaches before that the companies have publicly reported.
• 18:12
  Mike Masnick: Do you think that there's an issue with companies not reporting these kinds of breaches?
• 18:16
  Mike Masnick: Like, how is...
• 18:18
  Mike Masnick: I, I mean, I don't know, but, like, is-- do you think that's happening, where they're similar, maybe not as egregious and not as, as big a deal, but that there are these kinds of security breaches that are happening that are sort of being swept under the rug?
• 18:30
  Mike Masnick: I,
• 18:31
  Kat Duffy: I don't, I don't even know that they would be being swept under the rug.
• 18:33
  Kat Duffy: I mean, it's-- we don't even know which ones are being identified and which aren't, right?
• 18:38
  Kat Duffy: I mean, it took an AI system for Hugging Face to figure out that this was even happening.
• 18:43
  Kat Duffy: And I think the, for me, the bigger question is, um, we would have absolutely no idea of knowing that.
• 18:49
  Kat Duffy: None at all.
• 18:49
  Kat Duffy: Like, I wouldn't, I wouldn't want to speculate because it's possible that it's, this is not really a thing.
• 18:54
  Kat Duffy: Like, that it's not really a bad practice and people are, companies are being very forthcoming.
• 18:58
  Kat Duffy: It's possible you've got one very bad actor in the mix who's not disclosing anything, and you've got some other better faith actors that are trying to disclose things.
• 19:05
  Kat Duffy: I have more faith, frankly, in the more mature companies as well because they've built muscle memory around things like disclosures and how you navigate that and the importance of, of working on those issues.
• 19:16
  Kat Duffy: Um, but the, to me, the, the fundamental question there is how would any of us know?
• 19:22
  Kat Duffy: And we can't con- that's unconscionable.
• 19:23
  Kat Duffy: We cannot continue to live in that level of ignorance.
• 19:27
  Ben Whitelaw: Yeah.
• 19:28
  Ben Whitelaw: I mean, I'm, I'm really interested in, in...
• 19:29
  Ben Whitelaw: I mean, that's a, that's a great point.
• 19:30
  Ben Whitelaw: I'm really interested in the way this story might be used by various groups in the conversation about how to regulate AI.
• 19:36
  Ben Whitelaw: You know, the AI company said,"Actually, you know, the problem with this incident was that there was, you know, the guardrails that we're forced to put in place stopped us investigating this." Uh, so, you know, hand, you know, hands off.
• 19:47
  Ben Whitelaw: Don't regulate what we do.
• 19:49
  Ben Whitelaw: Obviously, politicians are saying,"This is the incident that means we have to go faster in terms of regulating." What kind of effect do you think this particular story and incident will have in both of those a- potential outcomes?
• 20:01
  Ben Whitelaw: Maybe Zoe, you wanna chime in first?
• 20:03
  Zoe Darme: Yeah.
• 20:04
  Zoe Darme: I don't actually know a great answer for this because I don't know how, um, uh, how much our policymakers really are, uh, are really in tuned with the risks, benefits, and, um, all the kind of technical details of what agentic AI is.
• 20:23
  Zoe Darme: You know, there...
• 20:24
  Zoe Darme: Uh, I was joking with my team when we were submitting these, um, proposals.
• 20:28
  Zoe Darme: Charlotte, I hope you're not listening.
• 20:30
  Zoe Darme: Um, but we, we were saying,"Oh, c-" Apparently
• 20:32
  Kat Duffy: she went to the other session.
• 20:33
  Zoe Darme: Yeah, she went to the other session- So- so that's her, that's her
• 20:35
  Ben Whitelaw: problem.
• 20:35
  Ben Whitelaw: Do, do not get me started on that.
• 20:37
  Zoe Darme: But, um- I was saying,"Oh, you know, just it, it's not blockchain this year, it's agentic.
• 20:42
  Zoe Darme: Make it about agentic AI, you know, that'll, that'll catch people's attention." Um, and I actually think, you know, this is showing that we do need more people, more, um, really smart, great, uh, pr- um, you know, seasoned trust and safety professionals kind of really thinking about agentic AI, de- defining what agentic AI is.
• 21:06
  Zoe Darme: Um, and we're kind of, kind of in this position where, like, we, we know it when we see it type of thing.
• 21:11
  Zoe Darme: And I think if that's where we are, how do you regulate that, you know?
• 21:16
  Zoe Darme: Um, and how do you regulate that in a future-proof way?
• 21:19
  Zoe Darme: I do think it's incidents like these that really drive the political pressure to do so.
• 21:24
  Zoe Darme: Um, and I hope that we can just have enough space to breathe to really figure out what needs to get regulated in what way, and how we can enforce those things when not every agent is going to be run by a giant American tech company.
• 21:41
  Zoe Darme: Mm.
• 21:42
  Mike Masnick: Yeah, I mean, the, the, the thing that strikes me about that, sort of building on what Zoe was just saying, is that this story is getting a lot of attention because it seems crazy and it seems cinematic in some way, right?
• 21:54
  Mike Masnick: You know, agent goes crazy and hacks by itself.
• 21:57
  Mike Masnick: That's the kind of thing that, you know, I think leads to, to bad regulations and bad law because people will over-index on it because it's sort of...
• 22:07
  Mike Masnick: It's so memorable and it stands out.
• 22:09
  Mike Masnick: Oh my gosh, you know, this agentic thing, you know, went out and hacked stuff.
• 22:14
  Mike Masnick: When that is not necessarily a real world thing that we should expect to happen, like me running Claude Code, I'm not gonna be able to, like, have it go try and, you know, build me a little tool to check the news, and it's not gonna go hack Hugging Face or something else, right?
• 22:28
  Mike Masnick: Um, but from the, the regulator side and the policymaker side, they're gonna say,"Wait, oh my gosh, these things are out of control.
• 22:37
  Mike Masnick: We have to stop them." And that's how you get bad laws generally, when you have sort of a big disaster like this, and then the regulation is not sort of thoughtful, how do we approach this in a thoughtful way?
• 22:47
  Mike Masnick: But, like, we have to do something.
• 22:49
  Mike Masnick: Really bad stuff happened, crazy agents, you know?
• 22:52
  Mike Masnick: So, um, that's my worry about kind of where it goes.
• 22:55
  Kat Duffy: But I would say you've seen in the last, I don't know, six weeks maybe, five weeks, we've seen a proposal from Dario.
• 23:02
  Kat Duffy: Uh, like you've seen a proposal from Anthropic for essentially a governance structure that is I, I, if I remember correctly, sort of FDA based.
• 23:08
  Kat Duffy: And then you have Sam Altman coming out and yet again sort of pushing to nuclear governance, which spoiler alert, if you know anyone in nuclear governance, they will all be like,"Do not, do not go for nuclear governance.
• 23:19
  Kat Duffy: Did not work the way you think it worked." Uh, and then you had Demis Hassabis just recently put out his proposal, which is based more on FINRA, which is, um, self-regulation of, um, in the finance industry.
• 23:32
  Kat Duffy: And there is also now apparently a proposal that's floating around in the White House that came from Secretary Treasury, uh, Treasury Secretary, uh, Bessent, which is being also sort of aligned with a FINRA-esque model but is slightly different than the one that, uh, Demis Hassabis proposed.
• 23:49
  Kat Duffy: And so what I think is interesting about this is that you've had, um, on one, on one hand a very Anti-regulation, governance is the inherent enemy of innovation.
• 24:02
  Kat Duffy: Our industry's ability to proceed completely unfettered is a national security and strategic competitive advantage, and it's an imperative.
• 24:09
  Kat Duffy: And that has more or less been the narration of the last eighteen months.
• 24:12
  Kat Duffy: And now what you have seen is all three major frontier companies come in with governance proposals, as well as a proposal floating around in the White House.
• 24:20
  Kat Duffy: Um, this is coming out on Friday.
• 24:22
  Kat Duffy: So anyway, tomorrow, you guys will see an article, uh, from me and my colleagues Vinh Nguyen and Elham Tabassi at Brookings, where we go through this approach, and we talk about the different ingredients that we think a model-- that a, a governance model would need in this space in order to have political legitimacy and credibility.
• 24:41
  Kat Duffy: Um, and so I hope everybody will check that out.
• 24:45
  Kat Duffy: But we are interesting at-- an-- We have entered an interesting moment of governance discourse because the ad hoc interventions we've seen from the federal government since February in terms of the Pentagon, DOD, you know, Mythos closing, you know, Fable, Glasswing, then limiting Glasswing, all of that has created such phenomenal unpredictability that I think now what you're seeing understandably is the leading companies saying,"We don't want you to, we don't want you to regulate us so much that we can't move forward, that, you know, we can't progress.
• 25:21
  Kat Duffy: But also, we cannot operate in a business environment with this amount of instability and this total lack of predictability." And so those two things are converging right now, and I would say, like, watch, watch this space because they're converging quickly, and the power dynamics have shifted.
• 25:36
  Ben Whitelaw: And you'll have all the answers in that Saturday piece
• 25:39
  Kat Duffy: Hundred percent.
• 25:40
  Kat Duffy: Great.
• 25:40
  Kat Duffy: Absolutely.
• 25:41
  Kat Duffy: All you guys needed was a, with, a think piece- From, from a think tank- Yeah and we're, we're sorted.
• 25:47
  Kat Duffy: Everyone just really over-admires the problem, you know?
• 25:50
  Ben Whitelaw: Also, I can't gloss over the fact that there was a tiny but very audible cheer for FINRA in the audience, which was not on my bingo card for today's episode of Ctrl Alt Speech, um, which, which is a, a nice way to kind of round up that story.
• 26:01
  Ben Whitelaw: Thank you guys for, for your thoughts there.
• 26:03
  Ben Whitelaw: Um, the chances of doing a kind of Ctrl Alt Speech episode without some sort of social media ban chat was, uh, I think close to zero, and our good friends in France have, uh, have done their duty and given us a, a story to talk about this week, which I'm gonna kind of talk through.
• 26:18
  Ben Whitelaw: So on Tuesday, uh, France kind of became the first European country to ban social media for teens and, you know, announced a block for new accounts starting in September to coincide with the start of the school year and, uh, kind of it would apply subsequently to existing accounts from January.
• 26:37
  Ben Whitelaw: So in the next six months, kind of all under 15s would, uh, essentially be banned from, from social media.
• 26:43
  Ben Whitelaw: As with other bans, there's a big focus on age assurance technology in order to, to kind of make that happen, and we can talk about the kind of, uh, you know, some of the privacy and surveillance concerns associated with that.
• 26:54
  Ben Whitelaw: Um, it does coincide quite int- interestingly with a cell phone ban in high schools as well, which was, uh, part of the legislation, and that extends, uh, a ban that happens in middle schools and primary schools in France.
• 27:06
  Ben Whitelaw: So again, there's an, an interesting kind of, you know, link between social media and smartphones, even though as many people in the room will know, those things aren't, are not always, uh, fully linked.
• 27:16
  Ben Whitelaw: And, and, you know, President Macron was very effusive in his, uh, praise for himself and his government- uh, a- as you might expect, for, uh, for getting this done and for France kinda leading the way in protecting teens.
• 27:30
  Ben Whitelaw: I- interestingly, it comes a week after President Ursula von der Leyen came up with, uh, Europe's, Europe's own plan for r- for regulating social media for teens in, in Europe.
• 27:41
  Ben Whitelaw: But the fact that he's come out now, I think suggests that he doesn't believe that that process is happening fast enough, and, and it could take kind of between a year and two years, depending on who you speak to.
• 27:50
  Ben Whitelaw: So this is France kind of taking the bull by the horns and saying,"No.
• 27:54
  Ben Whitelaw: We, we're gonna do it by, by our timelines." Um, I don't really wanna get into the kind of, you know, effectiveness of bans here'cause we, we only-
• 28:03
  Mike Masnick: We, we have done that so many times.
• 28:04
  Mike Masnick: You can go back and listen to many discussions- Yeah on the effectiveness of these bans.
• 28:09
  Ben Whitelaw: Exactly.
• 28:09
  Ben Whitelaw: And, and, you know, we- Or
• 28:10
  Mike Masnick: lack thereof, I should say.
• 28:11
  Ben Whitelaw: Yeah.
• 28:11
  Ben Whitelaw: And, um, and I'm sure there'll be questions from the audience as well.
• 28:14
  Ben Whitelaw: I think there's, there's still a debate to be had, but we, we only have 50 minutes.
• 28:18
  Ben Whitelaw: Um, I, I'm kind of interested in what your guys think about how policymakers in the trust and safety field should respond to this clear public and political need for something to be done and the fact that bans have become such a kind of powerful force.
• 28:35
  Ben Whitelaw: You know, h- how do we respond other than pushing back on the idea of bans?
• 28:40
  Zoe Darme: Well, I'm gonna start this by saying I have worked on child safety, and you can hear the kind of sigh in my voice now for, um, I don't know, six years across two companies.
• 28:52
  Zoe Darme: And this, uh, uh, and it's, it's-- I won't claim to be a child development expert.
• 28:57
  Zoe Darme: I won't claim to say I know what's best for kids and that sort of thing.
• 29:01
  Zoe Darme: What I will say is, having been in all of these conversations for so many years, there is no other policy issue that starts with, um,"Well, as a parent, I think..." You know?
• 29:16
  Zoe Darme: And we don't start conversations about hugging face and OMNA as,"Well, as a victim of a cyberattack, I think..." Um, and there's- Well,
• 29:24
  Kat Duffy: as an agent.
• 29:26
  Zoe Darme: Exactly.
• 29:27
  Zoe Darme: Um, and so I think it is so hard to check your priors at the door when we're talking about these types of issues.
• 29:33
  Zoe Darme: And, um, and I do it myself.
• 29:35
  Zoe Darme: I catch myself starting with,"Oh, you know, well, as a parent of a 12-year-old..." You know, my experience as a parent of a 12-year-old in a very privileged environment in a coastal city is almost irrelevant, you know, to when we're thinking about what the impacts are, um, across all of our users, not just our child users.
• 29:54
  Zoe Darme: Um, and you know, there's a 230 expert out there who many of us know and love, and she was being questioned about tech law, because this issue has to do with tech law.
• 30:05
  Zoe Darme: And, um, and she was asked, uh,"Yeah, but are you a mother?" As if that's, like, a disqualifying- Wow question, right?
• 30:14
  Zoe Darme: Or,"Are you a parent?" Or something like that.
• 30:16
  Zoe Darme: And you know, I really think that the question should be, um,"Have, have you read the law?
• 30:23
  Zoe Darme: Have you read the law?
• 30:24
  Zoe Darme: Have you read the legislative text?" Right?
• 30:26
  Zoe Darme: Um, because I have been in so many of these conversations, and people have so many opinions.
• 30:34
  Zoe Darme: And you know, the classic, the classic thing we say to all these hard problems,"Oh, well, the devil's in the details." I'm like, how many of us have read the details?
• 30:42
  Zoe Darme: You know?
• 30:42
  Zoe Darme: Do we know what the scoping definition is of social media in France and Australia and wherever and wherever?
• 30:48
  Zoe Darme: Um, w-why are we choosing U15 here?
• 30:51
  Zoe Darme: Why are we choosing U16 here?
• 30:52
  Zoe Darme: Is that based in evidence in, in child development, brain development, any things like that?
• 30:56
  Zoe Darme: Um, is it banning kids from social media?
• 30:59
  Zoe Darme: Because that's what gets the headline.
• 31:00
  Zoe Darme: Or is it banning from new accounts?
• 31:01
  Zoe Darme: Which, Ben, thank you for being clear about what, what the France law says.
• 31:06
  Zoe Darme: Um, and then, you know, just in child safety laws generally, if we're thinking about app stores too, you know, it's, um, uh, what's the knowledge standard?
• 31:15
  Zoe Darme: Uh, what do you do about a conflict in age?
• 31:18
  Zoe Darme: Are you supposed to take the highest confidence signal?
• 31:20
  Zoe Darme: Are you supposed to take, uh, the lowest age?
• 31:23
  Zoe Darme: These are all details that really matter.
• 31:25
  Zoe Darme: I am not a lawyer, I will say that very happily.
• 31:29
  Zoe Darme: With all due respect.
• 31:31
  Zoe Darme: I am not a lawyer, but as someone who works on designing, uh, the product, these are implementation details that matter, not just for technical complexity, but also the impacts to our users.
• 31:45
  Zoe Darme: And so the next time somebody starts a child safety conversation with you, I really hope you say,"But did you read the law?" Like that.
• 31:51
  Ben Whitelaw: Yeah.
• 31:51
  Zoe Darme: And then we can talk about your kids.
• 31:52
  Zoe Darme: I love to hear about all your children.
• 31:55
  Ben Whitelaw: Yeah.
• 31:57
  Ben Whitelaw: I mean, I put a photo of mine up on the screen last year, so- I, I'm maybe not a good person to speak to.
• 32:02
  Ben Whitelaw: But, um, I, I, actually am being a little bit facetious here, but I think actually being a parent is a hindrance to the way that a lot of people talk about, uh, social media bans.
• 32:12
  Ben Whitelaw: Because it obviously has...
• 32:14
  Ben Whitelaw: it brings a lot of emotion and a lot of kind of tension up.
• 32:17
  Ben Whitelaw: And, and I've, I've actually kind of written and spoken about how these social media bans are almost a kind of referendum on parenting, as much as they are a kind of vote on, on how people think about big tech and social media.
• 32:28
  Ben Whitelaw: Clearly there is, there is, uh, uh, an ov- an overlap there.
• 32:32
  Ben Whitelaw: But, you know, parenting is one thing and, and as you say, the law is another thing, and we shouldn't kind of get the two mixed up.
• 32:39
  Ben Whitelaw: Um, Kat, what did you think about the kind of France proposal and, and how it relates to other bans that we're seeing in the UK
• 32:47
  Kat Duffy: and elsewhere?
• 32:47
  Kat Duffy: I mean, my, my immediate...
• 32:49
  Kat Duffy: I saw the headlines, and my reaction in my head was basically like,"Bad man".
• 32:55
  Kat Duffy: You know?
• 32:55
  Kat Duffy: Uh, like-
• 32:58
  Ben Whitelaw: That was on my bingo card, actually.
• 33:00
  Ben Whitelaw: Um...
• 33:01
  Ben Whitelaw: I knew that was gonna happen.
• 33:02
  Kat Duffy: Oh, and then I just went and stress ate a baguette.
• 33:04
  Kat Duffy: Um- So, so a couple of things.
• 33:08
  Kat Duffy: O- one, on the, on the question around, like, policymakers, right?
• 33:12
  Kat Duffy: Um, so many of the policymakers and s- and so many of the parents as well, and, and frankly our kids, are getting their information about this and are getting their understanding about this from how the media is reporting on it.
• 33:26
  Kat Duffy: And if you read the media reporting that has come out, it is so shoddy.
• 33:31
  Kat Duffy: It is, it is not at all a deep dive into what the true intent of the law would be, how it would pan out.
• 33:41
  Kat Duffy: I mean, I've only read an English translation of it, but from what I can tell...
• 33:46
  Kat Duffy: I mean, again, it's about accounts.
• 33:47
  Kat Duffy: It's not about social media.
• 33:49
  Kat Duffy: And then the definition of where you can't have accounts is so vast that I, I, I don't think a kid could use Scratch.
• 33:56
  Kat Duffy: I don't think they would be able to use Duolingo.
• 33:58
  Kat Duffy: I don't think...
• 33:59
  Kat Duffy: All of these things that many people would say are a good and healthy use of a digital tool or app for a kid, I, I think under this law, my plain, again, my just plain reading of it would suggest that those are not allowable.
• 34:15
  Kat Duffy: And how many people are thinking about Scratch, like, you know, as social media?
• 34:20
  Kat Duffy: Like, hopefully no one.
• 34:22
  Kat Duffy: Um, so A, there's that.
• 34:23
  Kat Duffy: I really think that we need, um...
• 34:27
  Kat Duffy: I, I think we need more intellectual honesty and rigor from those who are reporting specifically on things like social media bans because they are...
• 34:36
  Kat Duffy: they capture the attention, the concern, and the passion of so many people, and it's important to then be specific about what is actually being proposed.
• 34:44
  Kat Duffy: The s- the second thing I would say is I, I could not agree more With Zoe.
• 34:49
  Kat Duffy: Like, I'm a mom, I've got two kids.
• 34:51
  Kat Duffy: That makes me sort of an expert maybe on any given day in maybe one of my kids.
• 34:59
  Kat Duffy: That's the entirety of it, right?
• 35:01
  Kat Duffy: I am not an expert in child safety, and I don't have to have kids to be an expert in child safety, and anyone who would suggest that you can't be an expert or a champion for child safety because you somehow, like, haven't decided to, you know, you haven't yet or haven't or don't want to or whatever it might be parent, like, a- step off.
• 35:24
  Kat Duffy: And so this is where I think this community is so important.
• 35:28
  Kat Duffy: I, I can't overestimate how important I think it is for the expertise that sits in this community in terms of that attention to detail, in terms of having a pragmatic understanding of how a particular rule or policy may or may not play out, in terms of building out open source tooling, right?
• 35:45
  Kat Duffy: And better interoperability.
• 35:47
  Kat Duffy: These are such important steps, and I'm just very grateful for this community, and I'm sorry, can I get on my last hobby horse?
• 35:53
  Kat Duffy: Please.
• 35:54
  Kat Duffy: I wanna see the ban instead on parents making boatloads of money by using their children's images and using videos of their children- in order to fuel a creator economy There's no informed consent- Mm for your four-year-old that you allow to become a meme.
• 36:13
  Kat Duffy: There's no informed consent for your family videos.
• 36:16
  Kat Duffy: Like, no.
• 36:17
  Kat Duffy: I- Leah Plunkett has done beautiful work on sharenting that I think is so great, and when I think about child safety, as the brilliant Riana Pfefferkorn, who is sitting right here, knows because we wrote a paper on this together several years ago, child safety is one thing children should have.
• 36:34
  Kat Duffy: It is one of many rights that children are guaranteed both through international law and treaty, through domestic and state law in the United States, through state laws in some places, and certainly through domestic law in other nations.
• 36:50
  Kat Duffy: They have all sorts of other rights beyond safety.
• 36:53
  Kat Duffy: Safety is one of them, and the more that we conscribe or limit what they are allowed to being safe...
• 37:03
  Kat Duffy: I mean, how many of us grew from being fully safe all the time, and who thinks that that is a reality?
• 37:09
  Kat Duffy: And so this is the other thing where, like, I thi- I would rather see a child be safe from sharenting than see a child be safe from scratch.
• 37:17
  Ben Whitelaw: Mm.
• 37:17
  Ben Whitelaw: Yeah.
• 37:18
  Ben Whitelaw: I mean, to, to what extent are we...
• 37:21
  Ben Whitelaw: You mentioned about the kind of media reporting on this and, and the kind of lack of understanding that that breeds maybe among the general public.
• 37:27
  Ben Whitelaw: To what extent is this about the lack of detail around the policy announcement in the first place?
• 37:33
  Ben Whitelaw: We saw it in the UK where there was a glitzy announcement, and the prime minister got up and said,"You know, this is my legacy, and, you know, here's, here's the ban." Th- three months later, there's a bit more detail.
• 37:46
  Ben Whitelaw: Maybe three months later, there'll be more detail.
• 37:48
  Ben Whitelaw: Does it...
• 37:48
  Ben Whitelaw: It makes it hard actually to, to report on it if there's very little substance.
• 37:53
  Ben Whitelaw: Is that not part of the problem, Mike?
• 37:55
  Mike Masnick: Yeah, I mean, I think that's...
• 37:56
  Mike Masnick: My reaction to this is that so much of these bans are more for politicians to say,"S- something must be done.
• 38:03
  Mike Masnick: This is something.
• 38:04
  Mike Masnick: We'll do it," sort of the classic political fallacy, um, that, you know, the details are always somewhat different, and each one matters in, in what they are.
• 38:14
  Mike Masnick: But it's more about getting the headlines and saying,"We're doing something," than actually figuring out, is this something that will actually keep children safer or respect their rights or, or things of that nature?
• 38:26
  Mike Masnick: There's, there's just this, um, you know, pull for action whether or not it's actually helpful.
• 38:33
  Ben Whitelaw: Yeah, and, and, and just to go back to the point you made earlier, Kat, about parents not being policy experts.
• 38:38
  Ben Whitelaw: I spoke to the founder of one very large, one very influential parenting campaigning group in the UK who was largely responsible for the number of consultation responses in the UK social media ban consultation and who was there at the launch of the, the kind of announcement of the ban and, and they admitted to me,"We don't know anything about policy." Um, which is a very kind of scary place to be when, when a group that powerful doesn't necessarily know the implications of what they're advocating for.
• 39:07
  Ben Whitelaw: Um-
• 39:07
  Kat Duffy: I also just really wish we would ask- Ask kids
• 39:10
  Ben Whitelaw: Yeah, yeah.
• 39:11
  Ben Whitelaw: For sure
• 39:12
  Kat Duffy: Like, just bring them into the conversation.
• 39:14
  Kat Duffy: Children are mono- like, they're not a monolith.
• 39:16
  Kat Duffy: They have a voice.
• 39:16
  Ben Whitelaw: Yeah And there's also been lots of good actually, uh, sessions into this week, this year's Trust Con about people trying to do that at platforms, um, with some difficulty, but-
• 39:27
  Mike Masnick: There, there was also just a study that came out that surveyed a bunch of kids.
• 39:30
  Mike Masnick: Um, so we're starting to see that, but it doesn't, doesn't get as into the conversation nearly as much.
• 39:37
  Ben Whitelaw: Yeah.
• 39:37
  Ben Whitelaw: Maybe a little bit too late.
• 39:38
  Ben Whitelaw: Um, in terms of other stories we wanna talk about today, you know, it'd be remiss not to have a live recording without mentioning Elon Musk or one of his companies.
• 39:48
  Ben Whitelaw: This, this is essentially- This is essentially a greatest hits of, uh, stories we've talked about, Mike.
• 39:54
  Ben Whitelaw: But y- you found a kind of very interesting, quite niche story, um, that you wanted to bring to listeners today.
• 40:00
  Mike Masnick: Yeah.
• 40:00
  Mike Masnick: Well, I mean, there was...
• 40:01
  Mike Masnick: There's the larger story that I think a lot of people obviously...
• 40:04
  Mike Masnick: I mean, you'd have to be living under a rock to not, not have heard the story from late last year and earlier this year, where Grok, which is the, uh, AI component of X, which is owned by xAI, which is owned by SpaceX, which whatever, uh, where it was, um, you know, producing, uh, declothed pictures of individuals, where you could ask Grok to put, uh, various people into bikinis was tended to be the, uh, the prompt.
• 40:35
  Mike Masnick: Uh, and Elon Musk encouraged this and even posted a picture that Grok had made of himself, um, wearing a bathing suit, um, and sort of laughed about it as other people were doing this, even as it was clear that it was also being used at that time to produce CSAM or CSAM-adjacent stuff.
• 40:53
  Mike Masnick: Certainly, uh, horrifying in many, many ways.
• 40:57
  Mike Masnick: Uh, that story has continued to evolve in all different ways.
• 40:59
  Mike Masnick: A bunch of people have sued, um, variations of Elon's companies for this.
• 41:06
  Mike Masnick: Um, there have also been some people who were arrested for, um, producing CSAM with Grok.
• 41:14
  Mike Masnick: So the thing that happened that caught my attention last week was that, um, X or xAI, I forget which entity of the entities was the actual party in the lawsuit, sued one of the individuals who had been arrested for producing CSAM.
• 41:30
  Mike Masnick: Um, and this is the first time that I can remember, and I could be wrong, and I could miss a lawsuit somewhere, of a platform suing one of their users for, as they said in the thing, violating a terms of service in some form or another.
• 41:45
  Mike Masnick: Um, you know, usually if you violate the terms of service, there can be actions taken against you.
• 41:50
  Mike Masnick: Usually, that is, you know, either limiting your access or kicking you off the platform entirely.
• 41:55
  Mike Masnick: Suing them, filing a civil suit against them, strikes me as very different and an escalation in all sorts of ways.
• 42:02
  Mike Masnick: There's also the questions of then like,"Wait, what?" Like,"Why?
• 42:05
  Mike Masnick: How?" Like...
• 42:07
  Mike Masnick: And so there are a number of different possible answers to what...
• 42:10
  Mike Masnick: Like, why?
• 42:11
  Mike Masnick: I think one of it is, like- Uh, the company realizing that it is coming out of this looking very badly for a variety of potentially very good reasons, um, but also deciding like,"Oh, maybe we can change the story if we're suing the user," and say,"We really..." I mean, like the first ha- it's a very short complaint, and the first half of the complaint is basically,"Here's all the stuff we do to...
• 42:32
  Mike Masnick: 'Cause we, we take CSAM really, really seriously," never mentioning the fact that their, you know, CEO was out there promoting this, this functionality, uh, of the, of, of Grok.
• 42:44
  Mike Masnick: Um, and saying, you know,"We-- This guy was arrested because we reported it to NCMEC, and we do all this NCMEC re- reporting, and we take all these things seriously," and therefore we're relying on the indemnification clause, which every one of these platforms has an indemnification clause basically saying that, you know, you will hold them harmless or if you do something that causes a lawsuit, you will agree to pay all their legal bills.
• 43:09
  Mike Masnick: They're relying on that indemnification clause to say,"Well, we can sue this guy," though it's unclear if the victims of this particular person is suing them, that there...
• 43:19
  Mike Masnick: Is there like a specific lawsuit?
• 43:22
  Mike Masnick: But it's, it just felt like a, a massive escalation and change, and there was no limiting factor in the lawsuit that wouldn't necessarily apply to any other platform saying,"You violated our terms of service.
• 43:37
  Mike Masnick: Now we're going to sue you, leaning on our indemnification clause.
• 43:40
  Mike Masnick: Because your violation of our service caused us potential legal problems, we're gonna sue you and demand money from you." That struck me as different and a little scary.
• 43:50
  Ben Whitelaw: Yeah.
• 43:50
  Ben Whitelaw: May- maybe this is the thing that will finally get users to read terms of service.
• 43:54
  Kat Duffy: I mean, this is-- I have two quick takes on this, right?
• 43:56
  Kat Duffy: One is I, uh, I- I'm ki- uh, uh, I don't hate the idea of racking and stacking accountability 12 ways to Sunday for CSAM production.
• 44:06
  Kat Duffy: Like the, go after many which way we can, right?
• 44:09
  Kat Duffy: A.
• 44:10
  Kat Duffy: B, I know the SpaceX IPO did not go well, but this feels like a really nickel and dime approach to try to build your revenue back up.
• 44:19
  Kat Duffy: I, I, like- I just don't know that it's gonna play out for them from a cost effectiveness standpoint.
• 44:23
  Ben Whitelaw: Every little helps, Cat.
• 44:27
  Ben Whitelaw: Zai, what did you think about that story?
• 44:29
  Zoe Darme: Um, yeah, I agree with Kat that, you know, I, I think it's worth ensuring that individuals are held responsible for CSAM generation.
• 44:37
  Zoe Darme: Um, I don't, I don't know if I have an opinion on whether suing your users is the way to do that.
• 44:43
  Zoe Darme: Um, I do think that whenever we take a new precedential step, uh, it often happens with CSAM.
• 44:50
  Zoe Darme: Um, and, uh, and I think there's just an interesting set of, uh, trade-offs, uh, when you're deciding to do actor-based enforcement, when you're deciding to do content-based en- enforcement.
• 45:02
  Zoe Darme: We've always been more comfortable with actor-based enforcement with CSAM, not least because of the legal reporting requirements.
• 45:10
  Zoe Darme: Um, and, um, but I, I do always think that when, when we are breaking new ground with CSAM, there's always pressure to say,"Okay, well, you do it for CSAM, so not-- why not XYZ?" I get that for every violation type, every...
• 45:30
  Zoe Darme: "Well, you did this for CSAM, how come you can't just do it for, uh, NCI?
• 45:36
  Zoe Darme: How come you can't just do it for hate speech?
• 45:39
  Zoe Darme: How come you can't just do it for suicide and self-harm?" And, um, it's not the same.
• 45:44
  Zoe Darme: It's just not the same.
• 45:46
  Zoe Darme: And one of the reasons it's not the same is because CSAM is, in some ways, a uniquely good machine learning problem, right?
• 45:55
  Zoe Darme: You can classify, and thank you, Griffin Hunt, for teaching me this, by the way.
• 45:59
  Zoe Darme: Before I go on, can I just say, raise your hand if you work on CSAM Turn and thank these people
• 46:09
  Ben Whitelaw: Yeah.
• 46:09
  Ben Whitelaw: Yeah.
• 46:17
  Zoe Darme: Um, the best PMs I've worked with have worked on CSAM.
• 46:22
  Zoe Darme: The best eng, the best lawyers work on it because you have to be a special type of person.
• 46:27
  Mike Masnick: Mm.
• 46:28
  Zoe Darme: Uh, a really committed type of person.
• 46:30
  Zoe Darme: Uh, and those people have taught me that one of the reasons CSAM is different, not because of the severity, not because of the reporting requirements, not just because it's a, one of those harm types that's almost universally illegal, but also because you can teach a classifier, is this explicitness?
• 46:48
  Zoe Darme: Yes, no.
• 46:49
  Zoe Darme: Is this potential underage-ness?
• 46:51
  Zoe Darme: Yes, no.
• 46:52
  Zoe Darme: That's a classic machine learning problem- Mm in a way that a lot of the other harm types aren't, where there's more subjectivity.
• 46:58
  Zoe Darme: I don't know where Dave Wilner is, probably at the other thing that Charlotte told everyone to go to.
• 47:03
  Zoe Darme: What
• 47:03
  Ben Whitelaw: a, what a scab.
• 47:04
  Zoe Darme: So he- He might disagree with me and say,"No, you can teach the magic ronets to do all sorts of labeling." But I actually think, you know, there is something a little bit different about CSAM, and so that's where even if it, it does turn out to be a good idea to, to, to, to sue people to stop CSAM generation, um, I would always caution us to pause and think what about CSAM makes the response unique to CSAM.
• 47:32
  Ben Whitelaw: Yeah, I think that's a good call-out.
• 47:33
  Ben Whitelaw: I'm, I'm gonna-
• 47:34
  Kat Duffy: And we should also clarify that we love Charlotte and Dave.
• 47:36
  Kat Duffy: We do, yeah.
• 47:36
  Kat Duffy: Oodles and noodles and boodles.
• 47:38
  Kat Duffy: We're just, we're just
• 47:38
  Zoe Darme: trolling them.
• 47:39
  Ben Whitelaw: That, that was a joke.
• 47:40
  Ben Whitelaw: That was my British humor.
• 47:41
  Ben Whitelaw: Um, my, my one job here was to keep time, and I've failed to do that in great detail, so I'm looking pleadingly at our room host to see if we can have maybe a couple of questions from the audience.
• 47:52
  Ben Whitelaw: We can try.
• 47:52
  Ben Whitelaw: Amazing.
• 47:53
  Ben Whitelaw: Uh, that was kind of a yes.
• 47:54
  Ben Whitelaw: So do
• 47:54
  Mike Masnick: we
• 47:54
  Ben Whitelaw: have- Um, so two, two lucky people perhaps can ask questions of our, of our great panel.
• 48:02
  Ben Whitelaw: Um, this gentleman...
• 48:03
  Ben Whitelaw: Actually, Ian, you had a question two years ago, I'm afraid.
• 48:06
  Ben Whitelaw: I can't have you...
• 48:07
  Ben Whitelaw: Yeah, I'm sorry, but you can't have a question again.
• 48:10
  Ben Whitelaw: Um.
• 48:10
  Ben Whitelaw: Oh, yeah.
• 48:12
  Ben Whitelaw: You, you were up fast though
• 48:23
  Speaker 6: So there have been a few lawsuits before where platforms have sued their users.
• 48:27
  Speaker 6: Twitch has done so at least twice.
• 48:30
  Speaker 6: Uh, once against hate raid, uh, s- bot software makers, once previously for follower count inflate bot makers.
• 48:38
  Speaker 6: In that previous lawsuit, they get, uh, uh, the original lawsuit 10 years ago, they got a$1.3 million judgment against those guys.
• 48:44
  Speaker 6: Surely judgment-proof.
• 48:45
  Speaker 6: And then also in a very long ago past life, I helped Twitter to sue software makers for spamming the platform.
• 48:52
  Speaker 6: So it is an antecedent for that.
• 48:54
  Speaker 6: Elon Musk, of course, has also sued places like the Center for Countering Digital Hate and other organizations for the same TOS violations, for trying to show, uh, things like there's Nazi speech alongside ads on the platform.
• 49:09
  Ben Whitelaw: That was, that was the best comment as a question that I've ever- Yes, exa- ever had.
• 49:12
  Ben Whitelaw: Um- If you're
• 49:13
  Kat Duffy: gonna get a comment as a question, you want it to be from someone as expert as Riana.
• 49:16
  Ben Whitelaw: Yeah, for sure.
• 49:17
  Ben Whitelaw: I think that means we still get two questions, right?
• 49:19
  Ben Whitelaw: Um.
• 49:23
  Mike Masnick: Do we...
• 49:23
  Mike Masnick: I, I saw a hand over here somewhere.
• 49:25
  Ben Whitelaw: Jeff, I think that's you
• 49:30
  Speaker 7: Hi.
• 49:31
  Speaker 7: Um, I'm wondering, how do we move past the kind of paradigm that we're in with regulation, where regulators are like,"Hey, your platforms kinda suck, and we're seeing it hurt our citizens"?
• 49:42
  Speaker 7: And then platforms are like,"Hey, your regulation sucks, and it's, like, off target." Um, and, and, and, and it's like c- my honest take is I kind of agree with both, at least when it comes to, like, age, age gating and age assurance.
• 49:54
  Speaker 7: Um, and, you know, there's another comment in a different session that was like,"Oh, we really wish that regulators had more data and more understanding, you know, of how the platforms worked." And it's like, well, platforms are now required to put out risk assessments.
• 50:07
  Speaker 7: Those risk assessments are required to include, you know, how the platforms deal with child safety.
• 50:12
  Speaker 7: Uh, I've read more of those risk assessment than I ever fricking wanted to, um, and they all kinda are terrible.
• 50:18
  Speaker 7: And if I was a policymaker, I would have no idea what regulation would make platforms safer after reading the transparency reports that the platforms currently put out about, you know, how they keep children safe.
• 50:27
  Speaker 7: So how do we get beyond this kind of back and forth?
• 50:30
  Speaker 7: Yeah.
• 50:31
  Ben Whitelaw: Maybe one, one intervention, one kind of razor-sharp intervention that might, might solve that issue that Jeff talks about.
• 50:39
  Kat Duffy: Through this community.
• 50:41
  Ben Whitelaw: Hmm.
• 50:42
  Kat Duffy: Y- The, the expertise that sits in this committee es- uh, community, especially people who are really good, like PMs or frontline engineers, like, that...
• 50:50
  Kat Duffy: The, the ability to translate an idea, a, an aspiration into operational or eng language and back is a superpower, and I think that we have, um, underemployed it in this space and in these debates, and I wish that I saw a greater emphasis there.
• 51:11
  Ben Whitelaw: Great.
• 51:11
  Zoe Darme: Yeah.
• 51:12
  Zoe Darme: I would say, uh, that, um, uh, regulators and policymakers really need to hire more experts, and I'm not quite sure how to do that, um, considering the, let's say- Remuneration imbalance, having been a civil servant myself.
• 51:33
  Zoe Darme: But I think our...
• 51:35
  Zoe Darme: Really having great edge, really having great former product managers in there, data scientists, would be helpful.
• 51:43
  Zoe Darme: Mm.
• 51:43
  Zoe Darme: Uh, and so I'm hoping that, you know, some of us in our next pivot are willing to do one more thing, and that one more thing should be, um, figuring out ways to navigate between just regulate and don't do that, that's obviously a stupid idea.
• 52:02
  Zoe Darme: Um, there is a h- there is some pressure that can be helpful.
• 52:06
  Zoe Darme: Um, and I don't...
• 52:07
  Zoe Darme: I think, you know, um, uh, uh, zero knowledge proof age assurance, uh, technology, I think is one of those things.
• 52:16
  Zoe Darme: Without that pressure, you know, would we be talking ab- Would so many people be learning about zero knowledge proof, um, age signal sharing?
• 52:25
  Zoe Darme: Um, I think that's, I think that's really helpful.
• 52:27
  Zoe Darme: Not that that technical solution is like the silver bullet- Mm but it is a helpful prompt for us to get creative.
• 52:34
  Ben Whitelaw: And there's, there has been a kind of an open door between, you know, policymakers from p- policy professionals from platforms to regulators.
• 52:41
  Ben Whitelaw: Certainly in the UK, a lot of folks were, were nabbed from big platforms, but less so engineering products, which I think is interesting.
• 52:47
  Ben Whitelaw: Mike, what would you do?
• 52:48
  Ben Whitelaw: Y-
• 52:49
  Mike Masnick: yeah, I mean, the one thing I would say is just, you know, a, a little humility on, on everybody's part in terms of dealing with these things.
• 52:56
  Mike Masnick: And, and you know, there's...
• 52:58
  Mike Masnick: P- part of lawmaking is often,"Okay, we have this problem.
• 53:01
  Mike Masnick: We're gonna write a law.
• 53:02
  Mike Masnick: We p- we've passed this law.
• 53:04
  Mike Masnick: We've solved it.
• 53:05
  Mike Masnick: Move on.
• 53:05
  Mike Masnick: Forget about it." Um, and these are issues where, as Charlotte famously says, everything is trade-offs, uh, when you're dealing with safety issues.
• 53:15
  Ben Whitelaw: And sadness.
• 53:16
  Mike Masnick: And sadness.
• 53:17
  Mike Masnick: I left out the sadness part- Don't forget the sadness because we're short on time.
• 53:19
  Mike Masnick: But the, um, but the regulatory process doesn't take that into account, that these systems need to be constantly tweaked and monitored.
• 53:29
  Mike Masnick: It would be great if we had a system where there was more iteration, um, and more collaboration in terms of figuring out,"Okay, we're, we're, we're gonna do this intervention.
• 53:39
  Mike Masnick: Um, does it work?
• 53:40
  Mike Masnick: Can we iterate?" The tech world is sort of, you know, constantly iterating.
• 53:43
  Mike Masnick: Like,"Test this.
• 53:44
  Mike Masnick: We're gonna roll this out, see if it works, and, and change and fix." The regulatory process isn't at all.
• 53:50
  Mike Masnick: And so it would be nice if there was some way to figure out a middle ground that would say,"Okay, y- you guys wanna roll out this under 15 thing in France.
• 53:59
  Mike Masnick: Like, what are you gonna do to measure it?
• 54:01
  Mike Masnick: Did it actually work?
• 54:02
  Mike Masnick: How are you gonna change it if it doesn't work?" I would love to see more of that.
• 54:05
  Ben Whitelaw: Yeah.
• 54:05
  Ben Whitelaw: I have a question.
• 54:06
  Ben Whitelaw: How many people in the room are regulators?
• 54:10
  Mike Masnick: No, we got a few
• 54:11
  Ben Whitelaw: Not many
• 54:12
  Mike Masnick: There's a few
• 54:13
  Kat Duffy: Thank them, too
• 54:13
  Ben Whitelaw: Yeah.
• 54:15
  Ben Whitelaw: But, but I rest my case.
• 54:16
  Ben Whitelaw: Um, do we have...
• 54:18
  Ben Whitelaw: No.
• 54:21
  Mike Masnick: We're being shut down
• 54:22
  Ben Whitelaw: That
• 54:22
  Mike Masnick: i- that-
• 54:22
  Kat Duffy: Wait.
• 54:23
  Kat Duffy: Okay, but so then can we, can I do a pr- can we close do- it's, it's Trust Con's fifth birthday-
• 54:29
  Ben Whitelaw: Yeah
• 54:29
  Kat Duffy: you guys.
• 54:29
  Kat Duffy: It's a really, for those of us who are parents, this is something we know, it's a really big birthday.
• 54:34
  Kat Duffy: Could we, could we close out by singing Happy Birthday to Trust Con?
• 54:37
  Kat Duffy: Yeah,
• 54:38
  Ben Whitelaw: I wanna hear them sing.
• 54:39
  Ben Whitelaw: You gotta lead, Kat.
• 54:40
  Kat Duffy: All right.
• 54:40
  Kat Duffy: You gotta lead.
• 54:41
  Kat Duffy: Are we get- c- w- but you guys have to be loud w-'cause I've got a mic.
• 54:44
  Kat Duffy: Yeah?
• 54:44
  Kat Duffy: Someone
• 54:44
  Mike Masnick: has to take video.
• 54:44
  Mike Masnick: And
• 54:46
  Kat Duffy: Oh, oh, yeah, Harbotov, of, of course she's on it.
• 54:48
  Kat Duffy: We're
• 54:48
  Mike Masnick: getting, we're getting video set up here.
• 54:50
  Kat Duffy: Because-
• 54:51
  Ben Whitelaw: This was not planned, by the way.
• 54:52
  Ben Whitelaw: This is so impromptu.
• 54:53
  Ben Whitelaw: Yeah, this is-
• 54:53
  Kat Duffy: No, this is just, this is my lack of impulse control This is Kat taking charge because TSPA, that team has worked so hard for so long and done such an incredible job for so many years, as has TSF, and I just think we should close this Trust Con by celebrating them and all
• 55:09
  Mike Masnick: of
• 55:09
  Kat Duffy: that
• 55:09
  Mike Masnick: work.
• 55:09
  Mike Masnick: And, and, and, and thankfully, Happy Birthday is now in the public domain, so we can do this- There it is without, without getting shut down.
• 55:13
  Mike Masnick: There it is.
• 55:14
  Mike Masnick: We have to be- it all comes back to copyright in the end, in some sense.
• 55:16
  Mike Masnick: It's all
• 55:17
  Zoe Darme: copyright.
• 55:17
  Zoe Darme: All roads lead
• 55:18
  Kat Duffy: to copyright.
• 55:18
  Kat Duffy: All right, here we go.
• 55:19
  Kat Duffy: Happy birthday to you.
• 55:22
  Kat Duffy: Happy birthday to you.
• 55:25
  Kat Duffy: Happy birthday, dear Trust Con.
• 55:30
  Kat Duffy: Happy birthday to you.
• 55:42
  Ben Whitelaw: Guys, don't stop clapping.
• 55:43
  Ben Whitelaw: Keep clapping for our brilliant panelists Thank you m- thank you very much for joining us, everyone.
• 55:55
  Ben Whitelaw: Uh, we hope to see you next year.
• 55:56
  Ben Whitelaw: Have a great, uh, rest of Trust Con and safe journey home
• 56:01
  Announcer: Thanks for listening to Ctrl-Alt-Speech.
• 56:03
  Announcer: Subscribe now to get our weekly episodes as soon as they're released.
• 56:07
  Announcer: If your company or organization is interested in sponsoring the podcast, contact us by visiting ctrlaltspeech.com.
• 56:14
  Announcer: That's CT RL alt speech.com.