Latest / Ctrl-Alt-Speech / ChatGPT Told Us Not to Say This, but YOLO

Transcript

ChatGPT Told Us Not to Say This, but YOLO · Ctrl-Alt-Speech

• 0:00
  Mike Masnick: So Daphne, we start off each week, with a prompt from social media, and I know that you are a big LinkedIn user, and one of the prompts that they use there, which is a little weird, but I'm going to run past you, is try writing with AI.
• 0:16
  Mike Masnick: So would you like to do that this week?
• 0:19
  Daphne Keller: Why, yes, as it happens, I did try writing with AI this week and I got some really interesting answers from chat GPT about what some laws like KOSA in the Senate and the Digital Services Act in Europe actually require.
• 0:35
  Daphne Keller: So I'm pretty into writing with AI this week.
• 0:38
  Mike Masnick: All right, well, let's explore that.
• 0:47
  Mike Masnick: Hello and welcome to Ctrl-Alt-Speech, your weekly roundup of the major stories about online speech, content moderation, and internet regulation.
• 0:57
  Mike Masnick: This week's episode is brought to you with financial support from the Future of Online Trust and Safety Fund.
• 1:02
  Mike Masnick: I am Mike Masnick, the founder and editor of TechDirt.
• 1:06
  Mike Masnick: And as we mentioned at the end of last week, Ben is off this week and instead we are joined by Daphne Keller.
• 1:13
  Mike Masnick: The director of the program on platform regulation at Stanford's Cyber Policy Center, and also former associate general counsel at Google.
• 1:23
  Mike Masnick: So I am hoping that we can explore some platform regulation issues this week, since you are the expert.
• 1:31
  Daphne Keller: And there are so many platform regulation issues this week.
• 1:34
  Mike Masnick: Yes, in prepping for this, we were having trouble cutting down on stories because there are so many and we're going to bounce around the world a little bit.
• 1:45
  Mike Masnick: But welcome to the podcast.
• 1:47
  Mike Masnick: Thank you for stepping in to Ben's Shoes.
• 1:50
  Mike Masnick: You don't have an English accent, but I think your expertise will fill in where, where, where we are lacking the British accent.
• 1:59
  Daphne Keller: I can't even fake one.
• 2:00
  Daphne Keller: Sorry.
• 2:00
  Mike Masnick: Uh, so I want to start with one that we mentioned right at the end of last week's podcast, because it came out while we were recording, which was the Ninth Circuit's ruling in, Net choice versus Bonta, or is it Bonta versus net choice?
• 2:16
  Mike Masnick: I always get confused which direction those go in.
• 2:18
  Mike Masnick: I guess they usually change at the Supreme Court.
• 2:19
  Mike Masnick: So I think it was still net choice versus Bonta.
• 2:22
  Mike Masnick: and that was looking over California's age appropriate design code.
• 2:27
  Mike Masnick: And I had spent a lot of time writing about and warning about this law.
• 2:33
  Mike Masnick: And California Politicians ignored me, as they are known to do.
• 2:39
  Mike Masnick: And then Netchoice challenged the law, and it was thrown out by the district court, and then it went to the Ninth Circuit on appeal.
• 2:45
  Mike Masnick: And so, last week at this time, the Ninth Circuit came out with their decision.
• 2:49
  Mike Masnick: And it's very interesting, and I think it could have pretty wide ranging impact.
• 2:54
  Mike Masnick: But Daphne, do you want to sort of summarize what the Ninth Circuit said, and kind of, get into what it means?
• 3:01
  Daphne Keller: Sure.
• 3:01
  Daphne Keller: So first, thank you for being such a canary in the coal mine on this one.
• 3:05
  Daphne Keller: You and Eric Goldman were pointing out the problems with this law way back.
• 3:09
  Daphne Keller: also to contextualize, this is one of almost half a dozen Cases going on around the country, almost all of which are all of which were brought by the Trade Association, NetChoice, that are challenging child safety laws passed by states.
• 3:24
  Daphne Keller: And most of these laws have, I think, some pretty obvious constitutional thoughts.
• 3:29
  Daphne Keller: but NetChoice v.
• 3:30
  Daphne Keller: Bonta, which is the California case, is the first one to have reached a court of appeals.
• 3:35
  Daphne Keller: So this is a really important ruling and hopefully it will affect the other cases.
• 3:39
  Daphne Keller: So the, the law The California age appropriate design code is this big sprawling thing with a lot of pieces and some of the pieces might be totally fine and innocuous.
• 3:51
  Daphne Keller: They look to me like normal privacy requirements to, you know, not collect more data than you need or, have a disclosure about your privacy policy that that kind of thing.
• 4:02
  Daphne Keller: But then mixed together with that are a whole lot of things that raise complicated constitutional questions.
• 4:08
  Daphne Keller: And then something that I think raises an uncomplicated constitutional question, which is what the court ruled on.
• 4:13
  Daphne Keller: So the law says that platforms that might be accessed by children have to make.
• 4:19
  Daphne Keller: Data Protection Impact Assessments, or DPIAs, and the DPIAs are supposed to assess anything the platform does that might create a risk of harm to children, including, risk of Things that are detrimental to their physical health or mental health or well being.
• 4:41
  Daphne Keller: And then after writing up this assessment of the risks to children, the platform also has to create a timed plan to mitigate or eliminate those risks.
• 4:52
  Daphne Keller: So, it's, platforms talking about the kinds of risks that are created by online content, like content that makes kids feel bad about themselves, or that scares them, or that encourages eating disorders.
• 5:06
  Daphne Keller: This is bad stuff, but it's definitely about content, and it's definitely about content posted by kids.
• 5:12
  Daphne Keller: Users and it seems to me that it's also about content that the state doesn't have the authority to regulate even though it's bad.
• 5:20
  Daphne Keller: It's about lawful but awful content.
• 5:22
  Daphne Keller: And so, a law requiring platforms to assess and mitigate the harms of that kind of speech seems to me obviously unconstitutional and, the Ninth Circuit.
• 5:34
  Daphne Keller: Agreed in this case, they said that this was subject to strict scrutiny and they said that it, quote, deputizes private actors into censoring speech based on its content, unquote.
• 5:47
  Daphne Keller: They were quite firm in saying that this part of the law, is unconstitutional and they were not at all.
• 5:54
  Daphne Keller: fooled or distracted by this formulation the lawmakers tried to make, about how, well, we're just asking platforms to assess risks that arise from their data management practices.
• 6:07
  Daphne Keller: Well, I mean, that means risks that arise from everything about platform operation because everything platforms do involves data management.
• 6:15
  Daphne Keller: So, you know, the, the poor lawyer for the AG's office, for the California AG's office defending the law before the Ninth Circuit just had a terrible time because they kept pressing her on how can this not be a law regulating content?
• 6:29
  Daphne Keller: And she really didn't have an answer.
• 6:31
  Daphne Keller: So the ruling focuses on that, says that's unconstitutional.
• 6:35
  Daphne Keller: It remands a bunch of other pieces for reconsideration.
• 6:39
  Daphne Keller: But the most important part of the ruling, I think, is It's super straightforward and should tell us a lot about the constitutionality of things like the COSA draft law in the Senate or a lot of these other state laws.
• 6:52
  Mike Masnick: Yeah, and I think that's important because there's this sort of dance that I think a lot of, regulators and, and lawmakers have been doing over the last couple of years in which they are trying to frame laws that are clearly intended to regulate content and speech, but they frame it as in, this is just about.
• 7:14
  Mike Masnick: data management or privacy or, process, you know, we, we just want best practices or things like that, but the underlying issue is always really goes right back to what you really want is to regulate content.
• 7:28
  Mike Masnick: You just know you can't say that directly.
• 7:30
  Mike Masnick: And I think the nice thing here, the lower court, I think.
• 7:33
  Mike Masnick: saw this as well, and then the Ninth Circuit says like, no, like, come on.
• 7:37
  Mike Masnick: The only way that you can really enforce this is to, take down certain kinds of content.
• 7:42
  Daphne Keller: So the lower court, I think, said some things that alarmed privacy advocates for good reason.
• 7:50
  Daphne Keller: And I'm happy that the Ninth Circuit did not replicate the lower court's work in this respect.
• 7:57
  Daphne Keller: Because the lower court kind of went down this road of saying, Oh, well, maybe all data is information and all data is speech.
• 8:04
  Daphne Keller: And therefore, a law that regulates how you use data, like.
• 8:08
  Daphne Keller: You know, private information you have in your logs from users usage practices, maybe that violates the first amendment in invoking a Supreme Court case called Sorrell.
• 8:17
  Daphne Keller: And so I, I think there's a reason that so many amicus briefs came in from academics and from privacy organizations saying.
• 8:27
  Daphne Keller: No, hey, wait a minute, you know, the basic privacy regulation parts of this are okay.
• 8:32
  Daphne Keller: And that's pretty much what the briefs from the ACLU and CDT and EFF said also was like, look, privacy regulation is okay.
• 8:41
  Daphne Keller: But a big chunk of this law is something else.
• 8:43
  Daphne Keller: It's obviously speech regulation and it's unconstitutional.
• 8:47
  Mike Masnick: Yeah.
• 8:48
  Mike Masnick: I mean, it was funny because I actually attended the oral arguments at the district court.
• 8:51
  Mike Masnick: I watched the ones for the Ninth Circuit, but the district court, the judge had just recently before that had been reversed by the Ninth Circuit.
• 9:00
  Mike Masnick: And she said a few times during the oral arguments, I don't want to be reversed.
• 9:04
  Daphne Keller: Oh,
• 9:07
  Mike Masnick: And, and so like the Sorrell part of it.
• 9:10
  Mike Masnick: Was she really focused on Sorrel and that was a whole case around like medical information and whether or not, it could be regulated in terms of how that was shared and, and stuff like that.
• 9:20
  Mike Masnick: We don't need to, go down that path, but I think she was really influenced by that case.
• 9:24
  Mike Masnick: So it is interesting that she was sort of partially reversed here, not, you know, remanded back to the court.
• 9:30
  Mike Masnick: But, part of that also is, this is, coming off of the, also Net Choice, Supreme Court cases, which now people refer to as Moody, which was the Florida and Texas law, where the Supreme Court sent those cases back, but basically did so because they said Net Choice brought these as a facial challenge and there are different requirements for a facial challenge, which were not met in those cases.
• 9:54
  Mike Masnick: And the Ninth Circuit kind of does the same thing here.
• 9:57
  Mike Masnick: Is that a fair statement?
• 9:59
  Mike Masnick: Right.
• 10:00
  Daphne Keller: think this case is very much influenced by Moody, by that part of Moody that said, When the state passes a big sprawling law, you have to look at all the little pieces of it.
• 10:11
  Daphne Keller: Well, maybe you have to look at all the pieces in isolation and look at how each of them might apply in different circumstances, because it might be that there are enough legitimate applications of.
• 10:21
  Daphne Keller: You know, some are all of the law, that it shouldn't be struck down, it's in its entirety, that it can't be subject to a facial challenge, which is the kind of challenge that Net Choice brought both against the Texas and Florida laws and here in, Bonta.
• 10:35
  Daphne Keller: And in oral arguments at the Ninth Circuit, the, Judges were really interested in just like the mechanics of how to do what Moody is telling them to do.
• 10:43
  Daphne Keller: Like how many isolated questions do you have to assess separately?
• 10:49
  Daphne Keller: And I think they actually did a pretty nice job of kind of bucketing the pieces that obviously could be dealt with.
• 10:57
  Daphne Keller: Right away, struck down, facially were done.
• 11:00
  Daphne Keller: and the, the parts that needed further attention from, from the lower court.
• 11:06
  Daphne Keller: But on the other hand, like, because it's, this law has so many different pieces in it, like there's an implicit age verification requirement.
• 11:14
  Daphne Keller: There are some disclosure requirements that are a lot like the transparency mandates that are being litigated elsewhere in the country.
• 11:21
  Daphne Keller: There's just a lot there.
• 11:23
  Daphne Keller: And it's expensive for NetChoice, but also for the state to have to litigate all of those different things, like, we should not be grateful to our legislators for passing a big, messy, confusing law that has so many different constitutional questions embedded in it.
• 11:42
  Mike Masnick: Yeah.
• 11:42
  Mike Masnick: It struck me that also the judge in the district court had asked both Metrois and California to focus on the DPIA requirement, this, you know, explaining each of your features and how you're going to mitigate any potential harm.
• 12:00
  Mike Masnick: And there had been a discussion.
• 12:02
  Mike Masnick: At the district court on severability.
• 12:04
  Mike Masnick: Like, could we cut out this part of the law and say, that goes away, but the rest of it can go into effect, which is slightly different than the issues with the facial challenge, as I understand it.
• 12:14
  Mike Masnick: But that seems to get really deep into the legal weeds that I have no interest in ever learning about.
• 12:19
  Daphne Keller: Yeah, one of the questions is federal and the other one is state
• 12:23
  Mike Masnick: Oh God.
• 12:23
  Mike Masnick: Okay.
• 12:24
  Mike Masnick: Let's, let's stop before, before we go any further on this, because I don't want to know.
• 12:30
  Mike Masnick: But.
• 12:31
  Mike Masnick: there are these questions.
• 12:31
  Mike Masnick: And that choice argued that the whole law is sort of tied up in the DPIA stuff.
• 12:35
  Mike Masnick: I think you're right.
• 12:36
  Mike Masnick: There are some of these other things.
• 12:37
  Mike Masnick: I mean, the age verification stuff really just didn't come up that much in the lower court.
• 12:42
  Mike Masnick: But I, kind of feel that, on remand feels like that might also get thrown out on First Amendment grounds once it's actually briefed and once they're actually discussed.
• 12:51
  Daphne Keller: but, but there's an age verification case before the Supreme Court right now about the FSC case about Texas's age verification law for porn sites or sites with more than a third pornography.
• 13:03
  Daphne Keller: So if I were a lower court,
• 13:05
  Mike Masnick: Right.
• 13:05
  Daphne Keller: would wait and see what the Supreme Court says there.
• 13:08
  Mike Masnick: Yes.
• 13:09
  Mike Masnick: Well, we just had in the seventh circuit, that Indiana had a similar law to Texas's law and net choice again, not net choice, uh, FSC, FSC had the free speech coalition had challenged that law and the lower court had thrown it out.
• 13:25
  Mike Masnick: It's obviously unconstitutional because of the age verification stuff.
• 13:29
  Mike Masnick: And then the seventh circuit just said, well.
• 13:32
  Mike Masnick: The Supreme Court is looking over the Texas law, and while they're looking it over, they did not put an injunction on the law.
• 13:40
  Mike Masnick: So the Texas law went into effect.
• 13:42
  Mike Masnick: And in fact, Paxton is already suing using that law.
• 13:45
  Daphne Keller: Yes, and the Seventh Circuit's like, yeah, okay, we'll let ours come into
• 13:48
  Mike Masnick: yes, yes.
• 13:50
  Mike Masnick: Yeah.
• 13:50
  Mike Masnick: Like, all right.
• 13:51
  Daphne Keller: that one, I think?
• 13:52
  Mike Masnick: yeah, That's the Indiana
• 13:53
  Daphne Keller: and, so Pornhub, I think, pulled out of Texas completely when the law came into effect, and I assume they'll now do the same,
• 14:01
  Mike Masnick: They they've, I think they're now pulled out of 17 States.
• 14:04
  Mike Masnick: I think the, I mean, it's just a block where if they recognize your IP address as being in one of the 17 States that has passed a similar law, will get blocked from accessing it.
• 14:15
  Daphne Keller: yeah.
• 14:15
  Daphne Keller: And one of their arguments is, if we're not there, people are definitely still going to find porn.
• 14:21
  Daphne Keller: It's just going to be from offshore sites that are even worse.
• 14:25
  Daphne Keller: Uh, which, they're not wrong.
• 14:27
  Mike Masnick: Yeah.
• 14:27
  Mike Masnick: I mean, there's, there's a whole, yeah, issue to go down there where it's like, yes, every other site, uh, porn site tends to not pay attention to these laws and that's in part because they have nothing in the U.
• 14:39
  Mike Masnick: S.
• 14:39
  Mike Masnick: that could be gone after.
• 14:41
  Mike Masnick: We're getting a little further away, but yes, all of these things are connected, and it's kind of an interesting time because of that, um, but so, the big part of the law, the injunction stays in place, the DPIA requirement, is gone.
• 14:57
  Mike Masnick: The rest of it gets sent back to the lower court.
• 15:01
  Mike Masnick: one thing that frustrated me, and I wrote about this this morning, was that California Governor Gavin Newsom and Attorney General Rob Bonta put out a statement claiming victory, of this.
• 15:14
  Mike Masnick: And I found it remarkably similar.
• 15:17
  Mike Masnick: To when the Moody decision came out in the Supreme Court and Ashley Moody, who is the Moody in, in, in the Moody case declared victory because she said, you know, it threw out the 11th circuit ruling.
• 15:31
  Mike Masnick: So we won, which is not.
• 15:33
  Mike Masnick: how any normal human being would read the Supreme Court's ruling in that, which basically said, the law is almost clearly unconstitutional.
• 15:41
  Mike Masnick: There are all sorts of free speech problems with it, but the lower courts just did it wrong because they didn't do it as a full facial challenge, as we sort of discussed.
• 15:49
  Mike Masnick: And so to see Newsom and Bonta basically do the same thing, they declare victory because they send back the case.
• 15:56
  Mike Masnick: They completely ignore it.
• 15:58
  Mike Masnick: The fact that And it's interesting because, the key part of this law, the DPIA section, the court was like, this is clearly unconstitutional.
• 16:06
  Mike Masnick: This is clearly an attack on free speech.
• 16:08
  Mike Masnick: and so having politicians and attorneys general claiming victory over cases they clearly lose, frustrates me.
• 16:16
  Mike Masnick: feels like either they don't understand it, which can't be true, or they assume the public is too stupid to be told the truth.
• 16:24
  Mike Masnick: And I don't, think that's a good look for, anyone.
• 16:27
  Mike Masnick: And, mimicking the way Florida does anything is probably a bad thing for California to be doing.
• 16:34
  Mike Masnick: Uh, and so that was frustrating to
• 16:36
  Daphne Keller: And it's just, I agree.
• 16:38
  Daphne Keller: Very frustrating.
• 16:39
  Daphne Keller: And it's, so, Just dishonest and keeps us from having a real conversation about these laws.
• 16:45
  Daphne Keller: And it's almost like this echo of how dishonest the laws in the first place are, you know, where there's this sort of pretense of regulating privacy as a feint in order to create content restriction rules, you know, we'd be much better off if we could just.
• 17:02
  Daphne Keller: Speak clearly about what rules we're trying to create.
• 17:05
  Daphne Keller: Um, but that's not what's going on.
• 17:08
  Mike Masnick: Yes, but that sets me up perfectly to segue to the next thing that we wanted to cover, which was something that you did this week with ChatGPT to explore some of these laws and figure out which parts are hidden and which are not.
• 17:22
  Mike Masnick: And so this is something that you wrote up on the Stanford Cyber Law blog, around ChatGPT saying the quiet part out loud.
• 17:30
  Mike Masnick: So do you want to describe what it was that you found and what you did?
• 17:34
  Daphne Keller: Sure.
• 17:35
  Daphne Keller: So this was so like fun and exciting for me.
• 17:40
  Daphne Keller: I was really:surprised, by the answers I was getting from chat GPT and I wound up.
• 17:45
  Daphne Keller: You know, staying up late, playing with it.
• 17:47
  Daphne Keller: So I started with, um, a version of ChatGPT that was put together by, someone who was at a trust and safety vendor called the Trust and Safety Regulation Expert, ChatGPT.
• 18:02
  Daphne Keller: And what I think that means is that in addition to having whatever the background training materials are that ChatGPT has, there was also additional information entered as, prompts.
• 18:11
  Daphne Keller: To sort of set up this one to have more or better answers than regular chat GPT.
• 18:17
  Daphne Keller: so I asked it questions like.
• 18:21
  Daphne Keller: What must I do in order to comply with the California age appropriate design code, the draft of COSA that's in the Senate right now, Europe's digital services act.
• 18:35
  Daphne Keller: And the answers should be, if you listen to the proponents of all of those laws, the answers should be like, Build good systems and risk mitigation practices, you know, things that are kind of hand wavy.
• 18:49
  Daphne Keller: but this is, this is an instance of ChatGPT that's trying to tell you how to do compliance.
• 18:54
  Daphne Keller: It's aimed at the people doing the work and they need something more concrete than that.
• 18:57
  Daphne Keller: And it turns out that something more concrete than that is ChatGPT says, take down the following categories of content.
• 19:05
  Daphne Keller: And so, for the U.
• 19:06
  Daphne Keller: S.
• 19:07
  Daphne Keller: ones, so it lists sexual content, which is the one kind of content that the Supreme Court has consistently said, can be legally restricted for children, and you can have reasonable laws trying to achieve that, and then we can debate what a reasonable law is.
• 19:22
  Daphne Keller: But then it also lists violent content, which we know from the violent video games case is something California has failed to restrict from children.
• 19:31
  Daphne Keller: Before dangerous challenges, cyber bullying, self harm and suicide, eating disorders, content, hate speech.
• 19:39
  Daphne Keller: Also something the Supreme court has recently emphasized is generally constitutionally protected speech.
• 19:45
  Daphne Keller: So it lists these things that like.
• 19:47
  Daphne Keller: Yeah, those are terrible content.
• 19:49
  Daphne Keller: You know, like, I, don't particularly want kids seeing that stuff, or at least the bad version of that stuff.
• 19:55
  Daphne Keller: Of course, if you read the list literally, it would exclude all kinds of educational material, classic literature, things that are taught in high school, comic strips, you know, like, lots and lots of things.
• 20:07
  Daphne Keller: but, but, but, What was striking is just that it, it is a literal list of legal kinds of speech that CHAT GPT thinks you have to take down or you have to stop kids from seeing in order to be compliant with these laws.
• 20:21
  Daphne Keller: And that's exactly what the AG's office in Bonto was trying to insist the law didn't mean.
• 20:28
  Daphne Keller: And you'd get, you know, uh, ChatGPT I think arrived at the same answers that anyone trying to comply with this law would probably arrive at.
• 20:38
  Daphne Keller: This isn't some kind of hallucination.
• 20:40
  Daphne Keller: This is just being realistic about what these laws actually require and probably reflecting what a bunch of, uh, online coverage or recommendations of best practices for vendors or, you know, whatever, what they say you're supposed to do to comply.
• 20:54
  Daphne Keller: So I think it really is saying the quiet part out loud and it's refreshingly blunt and straightforward.
• 20:59
  Mike Masnick: Yeah.
• 21:00
  Mike Masnick: And this is, you know, one of the things that comes up in these discussions is, how would you comply with these laws?
• 21:06
  Mike Masnick: And as much as the politicians and regulators say that it wouldn't require removal of content.
• 21:12
  Mike Masnick: The reality is that a lot of companies are going to read these laws this way, in part just.
• 21:18
  Mike Masnick: To avoid even having to litigate it, right?
• 21:21
  Mike Masnick: The problem is that so many of these things are vague and vaguely written, and, you don't want to have to be in court defending this, because that would be crazy expensive.
• 21:30
  Mike Masnick: And the way to avoid that is just to remove all this content.
• 21:33
  Mike Masnick: And then you get to this point where, effectively, the law is requiring you take down protected speech.
• 21:38
  Mike Masnick: And that's what the Ninth Circuit saw.
• 21:41
  Mike Masnick: It was interesting to me also, because you did.
• 21:43
  Mike Masnick: Sort of two separate questions on this.
• 21:45
  Mike Masnick: You talked about the ADC and, COSA, but you also did it with, uh, Europe and the DSA, where, as everyone in Europe always reminds me every time I talk about European regulation of the Internet, there is no First Amendment.
• 22:00
  Mike Masnick: And they have different standards, but there are lots of European officials who have insisted to my face at times that the DSA is not intended to regulate speech and is not intended to require sites to take down speech, yet, as we have talked about, Terry Breton, who is, uh, effectively in charge of the DSA, seems to think otherwise, and ChatGPT seems to agree with him?
• 22:26
  Daphne Keller: Yep.
• 22:27
  Daphne Keller: Yeah.
• 22:27
  Daphne Keller: So in Europe's defense, you know, they have human rights instruments that protect free expression.
• 22:33
  Daphne Keller: They just draw the line between what's protected and what can be prohibited under law in a different place than the U S would.
• 22:40
  Daphne Keller: and so I think the fundamental question is identical in Europe.
• 22:45
  Daphne Keller: Versus the U.
• 22:46
  Daphne Keller: S.
• 22:46
  Daphne Keller: It's for, whatever that, slice of speech is that's lawful, but awful, you know, that's not prohibited by law, but that lawmakers wish they could prohibit or that, you know, most people find offensive or harmful or, you know, morally abhorrent for that slice of content.
• 23:02
  Daphne Keller: What can laws get away with doing?
• 23:05
  Daphne Keller: And the part of the DSA that.
• 23:09
  Daphne Keller: I think that comes closest to being like the AADC or COSA and being like, Oh, here's a sneaky way of getting platforms to take down lawful but harmful content.
• 23:19
  Daphne Keller: That part is the risk assessment and risk mitigation rules for the very large online platforms in articles 34 and 35 of the DSA.
• 23:30
  Daphne Keller: Those say, you know, You have to do an assessment of the risks that your platform poses to this whole list of really important societal values, safety of children, privacy, free expression, violence, civil unrest, you know, et cetera, and having assessed the risk of.
• 23:47
  Daphne Keller: Everything that might go wrong in a society, then you say how you're going to mitigate that risk.
• 23:54
  Daphne Keller: And is the answer that you as a platform have to mitigate that risk by taking down lawful speech?
• 24:01
  Daphne Keller: Well, Terry Breton seems to think so.
• 24:05
  Daphne Keller: Um, I was recently at a multi day event in Brussels about this specific issue, like just these articles at the DSA and a bunch of experts there were like, No, actually, obviously, like they didn't even think it was a question, obviously, that you're supposed to take down this harmful but lawful speech, but that is absolutely not what the commission was saying the law was supposed to mean when it was being drafted.
• 24:29
  Daphne Keller: I have a really strong quote on that in my blog post of how they were framing it.
• 24:32
  Daphne Keller: And I think sincerely framing it, I think they truly wanted this to be a law that is about systems, but that didn't give government the authority to push for removal of specific kinds of harmful, but lawful speech.
• 24:45
  Daphne Keller: Well, turns out chat GPT, like Commissioner Breton, uh, thinks that is what the DSA requires.
• 24:55
  Daphne Keller: And so, you know, it told me to remove, quote unquote, a harmful speech, which is the sort of has been the buzzword for this lawful but awful category in the DSA discussion.
• 25:06
  Daphne Keller: And it also told me to do something that crosses an enormous, important, bright line in EU law, which is, it's been a legislative rule for a very long time that governments can't compel platforms to do quote unquote general monitoring of users communications.
• 25:27
  Daphne Keller: You can't make them go out and search for every time a user says something that violates the law.
• 25:33
  Mike Masnick: hmm.
• 25:34
  Daphne Keller: and the, highest court of the EU has said repeatedly, this is not just a legislative rule.
• 25:40
  Daphne Keller: It also protects users free expression rights from the over removal that comes from filtering speech.
• 25:46
  Daphne Keller: And it protects their privacy and data protection rights.
• 25:48
  Daphne Keller: This is a really important rule.
• 25:49
  Daphne Keller: You can't make platforms engage in general monitoring.
• 25:52
  Daphne Keller: Now.
• 25:53
  Daphne Keller: There are some holes in that rule, which we can talk about another time.
• 25:57
  Daphne Keller: But ChadGBT just blows right past that and says, go out, do proactive monitoring, you know, use the tools vendors want to offer you, um, find the bad stuff and take it down proactively.
• 26:09
  Daphne Keller: So it, it's really going against, it's passing two really important red lines.
• 26:14
  Daphne Keller: one is telling platforms to take down lawful, but harmful speech.
• 26:17
  Daphne Keller: And the other is telling them to do general monitoring.
• 26:21
  Mike Masnick: Yeah, and, there is a response to some people can have for this which is like, well, this is Chet GPT.
• 26:25
  Mike Masnick: It's not a lawyer.
• 26:27
  Mike Masnick: Chachapiti is prone to hallucinations, but I think the important point here is that like, this is a fairly standard way that these laws will all be, and in some cases already are, being interpreted.
• 26:39
  Mike Masnick: And the fact that Chachapiti always seemed to to that same conclusion with all of these laws, and these are the ways that people, warned about that the laws could be interpreted, and just the fact that, you know, this sort of gets left out of the debate, but I think is an important.
• 26:54
  Mike Masnick: Point, which is that, companies are liability averse, right?
• 26:59
  Mike Masnick: They don't want to risk this stuff.
• 27:01
  Mike Masnick: And so if you give them something where they might face liability and you give them an easy out to avoid that liability, they are probably going to take it.
• 27:11
  Mike Masnick: we had a post, uh, about a month ago about COSA in particular by Matt Lane, who was talking about like the laziness of companies in terms of how they will deal with these laws.
• 27:22
  Mike Masnick: And it is like.
• 27:23
  Mike Masnick: They don't want to fight it out, so they're just going to remove the content because that is the easiest and that avoids even having a lot of these questions.
• 27:31
  Daphne Keller: Absolutely.
• 27:31
  Daphne Keller: And there are so many shortcuts like that.
• 27:33
  Daphne Keller: You know, for example, if you are a company and you don't want to deal with the slight differences in hate speech law or privacy law between France versus Sweden versus Romania, you just expand your terms of service to prohibit all of it.
• 27:50
  Mike Masnick: Right.
• 27:50
  Daphne Keller: And then that's, you know, much cleaner and easier.
• 27:53
  Daphne Keller: And that's the same kind of shortcut that we should expect in, in the face of an ambiguous legal mandate.
• 27:59
  Daphne Keller: I also think the biggest platforms, the very biggest ones, have lawyers who can parse this carefully and who know what the commission has been saying this means, et cetera.
• 28:10
  Daphne Keller: Anyone else looking for advice on this, they're going to look at whatever the secondary literature is on the web, and that's the same secondary literature I think ChatGPT must have mostly trained on, and it says these sloppy things about taking down harmful speech.
• 28:26
  Mike Masnick: Yeah.
• 28:27
  Mike Masnick: And, and just to note, because I think there might be some people listening to this who are like, well, what's the problem here?
• 28:32
  Mike Masnick: If we're talking about harmful content, why not just remove it?
• 28:34
  Mike Masnick: and I think I'll just point out that that turns out to be a lot trickier of a problem than people make it out to be because, what counts as harmful content and harmful to whom and harmful how, and, do you include Content that might be, LGBTQ useful content that some people consider harmful and some people consider life saving.
• 28:58
  Mike Masnick: determining these things are really difficult.
• 29:00
  Mike Masnick: I talk about the eating disorder content issue quite a bit because that comes up as one of the examples harmful content that people talk about a lot.
• 29:08
  Mike Masnick: And yet, many of the studies have found that When that content is removed from platforms, it actually makes the problem worse, not better, because people continue to seek it out, and they tend to seek it out in deeper and darker places that have fewer protections and, importantly, fewer mitigations.
• 29:25
  Mike Masnick: When that content is found on platforms like Instagram or TikTok, they will often include some sort of mitigating content.
• 29:31
  Mike Masnick: Content saying, you know, warning people or, you know, pushing them towards resources for recovery, whereas the deeper, darker places of the web don't do that.
• 29:41
  Mike Masnick: And so, I always like to raise that because sometimes you get into this discussion and some people I know are listening saying like, but no, like that's bad content.
• 29:48
  Mike Masnick: It shouldn't be allowed at all.
• 29:50
  Mike Masnick: And I just want to remind people that it's, it is often trickier than people think it is.
• 29:54
  Daphne Keller: It's tricky and it's so inextricable from this sort of culture wars backdrop.
• 29:59
  Daphne Keller: You know, these are laws that have a vague concept of child safety, and then someone has to be the enforcer to decide what content is or is not safe for children.
• 30:11
  Daphne Keller: And so the concerns that LGBTQ groups have been raising I think are incredibly compelling about conservative AGs saying what child safety means or conservative FTCs under a Trump administration saying, you know, what's harmful to children is information about, reproductive.
• 30:30
  Daphne Keller: choice or information about transgender identity.
• 30:33
  Daphne Keller: And Marsha Blackburn very much said the quiet part out loud on this, such as saying that we need to protect children from quote unquote, the transgender.
• 30:42
  Daphne Keller: And so the, you know, parts of the left that are paying attention have been rightly very alarmed by these laws.
• 30:47
  Daphne Keller: But recently we've seen parts of the right appropriately get alarmed in You know, the exact same way.
• 30:54
  Daphne Keller: We have Rick Santorum and pro life organizations saying, wait, what if a liberal attorney general or enforcer wants to override our values and preferences about what our children should see?
• 31:04
  Daphne Keller: And yeah, they should worry about that.
• 31:06
  Mike Masnick: Yeah.
• 31:07
  Daphne Keller: I'll say one more thing about the, the speech at issue, moving away from the kind of fraught cultural work questions.
• 31:13
  Daphne Keller: There's this big, Beautiful brief that I just love, filed in the lower court as an amicus brief in Bonta by the New York Times and the Student Press Law Center.
• 31:22
  Daphne Keller: I link from it in, in my blog post.
• 31:25
  Daphne Keller: And the second half of the brief is just like moving stories about student activism and journalism and how important it is for Teenagers, especially, to like, see the upsetting things that are going on in the world on the news and not be protected from that.
• 31:41
  Daphne Keller: and I just think it's such an important point and it's so impossible to isolate the good and productive upsetting information in the world that kids need to see from whatever it is that these love is what platforms taking down.
• 31:56
  Mike Masnick: Yeah.
• 31:57
  Mike Masnick: And I'll use that as a transition point to our next story, which is also somewhat related to all of this, which is that it was announced in the past week that X, the platform formerly known as Twitter, was shutting down.
• 32:11
  Mike Masnick: all of its operations in Brazil in response to an ongoing battle that is happening in Brazil that started earlier this year where there is a Brazilian judge and I'm going to try and fail to pronounce the name properly, Alexandre de Mores, something along those lines.
• 32:31
  Mike Masnick: I apologize if I got it wrong, um,
• 32:34
  Daphne Keller: But I think he's on the Supreme Court and maybe is the equivalent of the Chief Justice.
• 32:37
  Daphne Keller: Like, this is a very powerful person.
• 32:39
  Daphne Keller: It's
• 32:39
  Mike Masnick: a very powerful person who had ordered, Twitter to, I'm just going to call it Twitter, to remove certain content earlier this year, and Elon Musk sort of, went very public with it, opposing it, and saying that he would not do that, and then there were threats that were passed back and forth, and then like two days later, very quietly, Twitter said that it would, be removed.
• 33:02
  Mike Masnick: Enforce those.
• 33:03
  Mike Masnick: And it would pull down that content.
• 33:05
  Mike Masnick: And now this is months later and apparently, that content wasn't removed, even though they said it was.
• 33:10
  Mike Masnick: And so there were new threats and there were threats about.
• 33:13
  Mike Masnick: daily fines and potentially arrests of ex employees in Brazil.
• 33:20
  Mike Masnick: And so Elon just said, screw it.
• 33:22
  Mike Masnick: we're going to shut down all of our operations in Brazil so that there won't be someone there to arrest and fine.
• 33:29
  Mike Masnick: And I thought this was interesting.
• 33:32
  Mike Masnick: This is not unprecedented.
• 33:34
  Mike Masnick: We've had Companies leave other countries in the past.
• 33:38
  Mike Masnick: Usually those have been authoritarian countries.
• 33:42
  Mike Masnick: You know, I know Google shut down its operations in Russia at one point, going way back at one point, Yahoo shut down operations in China over similar concerns, there have been other examples of companies pulling out.
• 33:56
  Mike Masnick: This still struck me as kind of interesting.
• 33:58
  Mike Masnick: Kind of newsworthy.
• 34:00
  Mike Masnick: I mean, the battle between Brazil and Elon struck me as interesting in a few ways.
• 34:06
  Mike Masnick: And I've spoken about this before, where Elon has claimed, though he doesn't live up to this, I mean, like, lots of things that he claims he doesn't live up to, that, he believed that free speech means that which the law allows, which is a very strange interpretation of free speech.
• 34:23
  Mike Masnick: Uh, Daphne just rolled her eyes, but I want to make sure that is on the record for the podcast.
• 34:27
  Mike Masnick: Um, And so if that were true, if he believed it was that which the law allows, then he would obey the ruling in Brazil.
• 34:38
  Mike Masnick: And yet he is protesting it.
• 34:39
  Mike Masnick: I think he's right to protest it because I think that some of What the judge in Brazil was ordering seems really questionable.
• 34:48
  Mike Masnick: It did strike me as interesting that he chooses to only fight or appears to only fight, that which the law allows when doesn't agree with it.
• 34:58
  Mike Masnick: Uh, that country's political leadership, whereas in other cases, such as in India and in Turkey, where he is much more aligned with them, ideologically, he seems to have no problem taking down the content that is demanded of him.
• 35:11
  Mike Masnick: But here he's taking another step, not just fighting it that way, but actually shutting down operations.
• 35:15
  Mike Masnick: So did you have any thoughts on, Musk making this decision?
• 35:20
  Daphne Keller: I mean, so it's, it's interesting that it's happening in Brazil specifically because Brazil does have a history of threatening to arrest, maybe actually arresting?
• 35:28
  Daphne Keller: I can't remember.
• 35:29
  Daphne Keller: Uh, the sort of in country executives, uh, once it was from Google, but more, more high profile, was about WhatsApp.
• 35:37
  Daphne Keller: They wanted WhatsApp to, turn over information about users to the police that WhatsApp didn't want to turn over.
• 35:42
  Daphne Keller: So that one was a fight about surveillance.
• 35:44
  Daphne Keller: This one is a fight about speech and what content gets taken down.
• 35:48
  Daphne Keller: But the sort of Brazil being The country that's willing to really threaten to put people in jail is, is a recurring pattern.
• 35:56
  Daphne Keller: And I would think if this were old Twitter, I would think this was the biggest and most interesting story of the week.
• 36:02
  Daphne Keller: But what, what's going on in India?
• 36:05
  Daphne Keller: You know, like, I just can't imagine that there have not been situations in India that could, and maybe should have led to the same kind of standoff.
• 36:15
  Daphne Keller: Um, and yet we don't see Musk and X picking fights with the Modi government in India.
• 36:22
  Daphne Keller: So you know, I think there's a piece here that is about, Musk's political preferences and which speech he values.
• 36:28
  Daphne Keller: There's a piece here surely that is about markets and commercial interests of both X and other countries.
• 36:36
  Daphne Keller: And so, tragically, this just feels like another front in the war of, well, Musk against other.
• 36:44
  Daphne Keller: Big important men.
• 36:46
  Daphne Keller: He does seem to like to have these public standoffs.
• 36:51
  Daphne Keller: Yeah, but you know, I feel terrible for people in Brazil who use Twitter and have relied on Twitter and presumably if Twitter pulls out it, you know, the service will still be available until Brazilian ISPs are ordered to block the site and the app, which seems like a pretty plausible thing that could happen if, Twitter's refusing to comply with the law there.
• 37:11
  Daphne Keller: So, you know, as somebody who's seen.
• 37:14
  Daphne Keller: Twitter go from being something incredibly valuable to me down to something that I reluctantly still use for my 30, 000 followers, all of whom have bikinis in their profile pictures.
• 37:30
  Daphne Keller: You know, it's tragic to me to see that demise.
• 37:32
  Daphne Keller: And I think to all of us, but to have it just like poof, it's gone overnight, for people in Brazil, that, that will be very unfortunate.
• 37:39
  Mike Masnick: Yeah.
• 37:40
  Mike Masnick: you know, it strikes me as interesting too, because Brazil, it feels like there's sort of a pendulum where they, Push for and sometimes pass laws that are very supportive of open internet and civil liberties and civil rights kinds of concepts.
• 37:55
  Mike Masnick: And then they do things like this,
• 37:59
  Daphne Keller: Yeah.
• 37:59
  Mike Masnick: they're threatening to arrest executives for not pulling down speech.
• 38:03
  Daphne Keller: Yeah.
• 38:03
  Daphne Keller: I mean, Brazil has some of the most sophisticated thinkers and advocates on intermediary liability for platforms in, in the world.
• 38:10
  Daphne Keller: Um, their law, the Marcos Civil is one of the most pro, free expression, pro privacy, anti surveillance, Laws out there, but it's under a constant attack.
• 38:23
  Daphne Keller: There are constant threats to change it.
• 38:25
  Daphne Keller: Um, it is not necessarily being applied in lower courts.
• 38:28
  Daphne Keller: Like the reality on the ground and the likely future in Brazil is not what it might be if they took advantage of, again, this just like tremendous rich body of expertise that they have in country.
• 38:42
  Mike Masnick: Yeah.
• 38:43
  Mike Masnick: Well, that actually is a good segue to our next story, which is also about Brazil, but in a different context.
• 38:48
  Mike Masnick: And this was a story from Rest of World, which was talking about indigenous creators in Brazil who are having trouble with both YouTube and Instagram, and that they are often being moderated or banned.
• 39:02
  Mike Masnick: Due to in theory, violating sensitive content or nudity rules or things of that nature.
• 39:10
  Mike Masnick: So do you want to talk a little bit about that story?
• 39:13
  Daphne Keller: Yeah.
• 39:14
  Daphne Keller: So this is a pattern of indigenous content being taken down, as you say, probably for violating nudity rules.
• 39:22
  Daphne Keller: And it jumped out at me in part because you know, you know that I follow so called must carry cases, the cases where somebody tries to force a platform to reinstate content.
• 39:31
  Daphne Keller: One of the earliest ones that I came across probably like eight years ago now was this exact same issue in Brazil with an indigenous woman asserting a right under the constitution To, it's like protection of cultural identity or something like that, that she was taking to some, a human rights body to get an image.
• 39:50
  Daphne Keller: I think it's a topless woman in traditional attire in the Amazon to get an image reinstated on Facebook and that case kind of fizzled out.
• 39:57
  Daphne Keller: And I don't think anything came of it, but this is a longstanding issue and it's an issue that's important and it's an issue that has.
• 40:05
  Daphne Keller: Some analogs, I think, in the U.
• 40:07
  Daphne Keller: S.
• 40:08
  Daphne Keller: and around the world in this question of like, well, the government can't, shouldn't, I think, come along and tell platforms how to moderate speech online generally, but if the claim is this moderation is racially discriminatory or is discriminatory in other ways that we do have other laws for, I actually think that's kind of, you know, Different, there's plenty of precedent for the idea that a law that's there to combat discrimination can have some incidental on effect on speech.
• 40:41
  Daphne Keller: And the classic example is, you can't have a job sign in the window that says only white people need apply, you know, there's some.
• 40:51
  Daphne Keller: Things that work differently when discrimination is at issue, and I'm, by no means saying I think there's some valid discrimination claim that could be brought, but the ways this has come up in the U.
• 41:02
  Daphne Keller: S.
• 41:02
  Daphne Keller: so far, including some kind of like junky and poorly substantiated cases against YouTube by Black and LGBTQ plus content creators saying they'd been discriminated against, they just didn't tee up the question very well, You know, it's a question I think governments around the world need to be thinking about a little bit differently than the questions that are about, can you silence someone based on their advocacy or their political perspective?
• 41:26
  Mike Masnick: Right, right.
• 41:28
  Mike Masnick: Yeah, the other thing that I thought was interesting about this particular story was that it, discussed how, Indigenous creators are effectively self censoring themselves, and they're sort of, because of these rules, they know that they can't show certain content, or they can't talk about certain content, and so they're changing the way they present themselves.
• 41:47
  Mike Masnick: And that is something that is interesting and we're thinking about in one that isn't as widely discussed.
• 41:53
  Mike Masnick: How these rules change the way people actually act online.
• 41:57
  Mike Masnick: and seeing that struck me as very interesting.
• 42:00
  Daphne Keller: Yeah, I mean, in a way it's analogous to the ways that, you know, sex workers have changed how they talk about themselves online.
• 42:06
  Daphne Keller: There's a whole body of sexual content where people are self censoring in various ways, but it usually doesn't have this overlay with, and this is how we are perpetuating our cultural tradition that is gravely threatened by the modern world.
• 42:20
  Daphne Keller: You know, that, that is a whole different, category of harm that, that's, happening.
• 42:25
  Daphne Keller: Sad and troubling.
• 42:27
  Mike Masnick: Yeah.
• 42:27
  Mike Masnick: So, and then I want to move on to, to,, uh, Other stories that I'm sort of going to merge together because I think they're a little different but semi related and has to do with people making money in different ways online having to do with their content that I actually think is interesting.
• 42:44
  Mike Masnick: one is also from rest of world, which is a great source if you don't follow it.
• 42:48
  Mike Masnick: Talking about teenagers who are making thousands of dollars by debating on TikTok.
• 42:54
  Mike Masnick: And TikTok has this feature, which I did not realize until I read this article, though it's been around for a little while, apparently, where you can sort of live stream a debate for five minutes against one another.
• 43:05
  Mike Masnick: And.
• 43:06
  Mike Masnick: A lot of people are using it for political fights.
• 43:09
  Mike Masnick: So there's like Trump versus Harris, where one person, you know, argues in favor of Trump and one in favor of Harris and all sorts of other things.
• 43:18
  Mike Masnick: And the way it works is that it's, there's a five minute timer that counts down and people watching effectively vote on.
• 43:26
  Mike Masnick: Which side they prefer, but the voting is giving virtual gifts, which have cash value.
• 43:33
  Mike Masnick: So the story is about kids making thousands of dollars a month by debating in these live streams and making money from it.
• 43:42
  Mike Masnick: And this struck me as just fascinating for a variety of reasons, but.
• 43:46
  Mike Masnick: The sort of trust and safety element of it all struck me as like, if I were a trust and safety person and someone showed me this feature, I would freak out because it raises like every, alarm bell related to trust and safety.
• 44:00
  Mike Masnick: There's live streaming, there is political content fighting, arguing, there is sort of just like, Disinformation, there is the monetary component, because as soon as you put money in, you get money involved in any of this, then it leads to questions of fraud and other issues, and you have people donating lots of money, and how do you track that?
• 44:23
  Mike Masnick: There are a couple stories in this, this article of people spending.
• 44:26
  Mike Masnick: Thousands of dollars supporting the TikTok creators that they like.
• 44:30
  Mike Masnick: There is one story, which is a little weakly sourced about somebody who, who stole 1.
• 44:35
  Mike Masnick: 2 million where apparently some of it was to support creators on TikTok.
• 44:40
  Mike Masnick: so all sorts of stuff that just struck me as there All of these crazy questions that come up around trust and safety with a tool like this, that I'm surprised this tool ever got released in the first place to some extent, but it's also kind of a neat idea to have people debate in these ways and get people conversing in this
• 45:00
  Daphne Keller: Yeah.
• 45:01
  Daphne Keller: So I, I mean, my first response was, couldn't this be a rap battle?
• 45:06
  Daphne Keller: I want this to be a rap battle, but like, hear me out.
• 45:09
  Daphne Keller: I think that would help because there's presumably these teenagers pretending to be Trump and Harris are going to spew a whole bunch of misinformation.
• 45:20
  Daphne Keller: Because they're just trying to be entertaining.
• 45:23
  Daphne Keller: And like, that's bad.
• 45:24
  Daphne Keller: We don't like misinformation.
• 45:25
  Daphne Keller: On the other hand, Saturday Night Live sketches and political cartoons are also full of misinformation that are part of the like, commentary and entertainment content.
• 45:35
  Daphne Keller: So if it were a rap battle, Mike, then it would be clearer to everyone that this is like an artistic take and you shouldn't see it as the news.
• 45:44
  Mike Masnick: Yeah.
• 45:45
  Mike Masnick: Okay.
• 45:45
  Mike Masnick: I like it.
• 45:46
  Mike Masnick: I like it.
• 45:47
  Mike Masnick: We'll suggest it to Tik TOK, force them to rap.
• 45:50
  Mike Masnick: All right.
• 45:52
  Mike Masnick: The other one that, that has to do with monetary support for people is the story about Patreon and Apple and Patreon recently announced that they're changing some of their policies in terms of how their subscriptions work.
• 46:07
  Mike Masnick: Okay.
• 46:07
  Mike Masnick: In response to Apple's demands on Patreon, and there is an element of this that a lot of people talked about that is to me, actually somewhat less interesting, which is that Apple demands that if someone is paying an app through iOS, that there's a 30 percent discount.
• 46:23
  Mike Masnick: Charge that has to go to Apple.
• 46:25
  Mike Masnick: And so they've pressured different companies.
• 46:27
  Mike Masnick: There've been legal battles over this Spotify and Apple thought about it, Epic and Apple thought about it.
• 46:31
  Mike Masnick: So there were a bunch of different back and forths with that.
• 46:34
  Mike Masnick: And here Apple is forcing Patreon to do it.
• 46:37
  Mike Masnick: Patreon is telling creators that they have to choose whether or not they bump up their price.
• 46:43
  Mike Masnick: By the like 3% or, or 30%, or if the creator themselves will eat the 30%.
• 46:51
  Mike Masnick: and so that, got some attention.
• 46:52
  Mike Masnick: But someone pointed out, which I thought was really interesting, was that part of this is that Patreon now has to match the way that Apple does subscription payments, and that means that they are getting rid of the ability to pay per creation.
• 47:08
  Mike Masnick: And this struck me as a big deal because that was how Patreon started.
• 47:12
  Mike Masnick: The whole idea that Jack Conte, who started Patreon, had this idea.
• 47:16
  Mike Masnick: He made all these like really elaborate YouTube videos.
• 47:20
  Mike Masnick: And so he set it up where people would commit to pay a certain amount, a dollar, 5, 10, whatever it might be per YouTube video that he released of a new song.
• 47:31
  Mike Masnick: And that was the beginning of Patreon.
• 47:33
  Mike Masnick: I thought that was a really brilliant.
• 47:35
  Mike Masnick: business model.
• 47:36
  Mike Masnick: If you're a content creator, and you only create every so often, but you have people who want to support those creations, if you get them to pre back each creation, that's fascinating and a really unique and interesting business model.
• 47:47
  Mike Masnick: I recognize that Patreon has mostly moved to a pre back.
• 47:51
  Mike Masnick: Per month subscription model, which is what a lot of creators wanted.
• 47:55
  Mike Masnick: But now the fact that they're effectively getting rid of this per creation model, not because they want to, or not because users don't like it, but because Apple's forcing them to, in order to meet the requirements of being in the app store, that just struck me as an interesting change and one worth noting.
• 48:13
  Daphne Keller: Yeah, absolutely.
• 48:14
  Daphne Keller: I mean, you're more of a content creator than I am.
• 48:16
  Daphne Keller: So you probably see this more clearly, but that just seems like it rewards a certain mode of cultural creation that is routinized, that can come out the same way every month.
• 48:28
  Daphne Keller: Maybe that depends on having a bigger organization.
• 48:31
  Daphne Keller: And it harms other modes of more, as my colleague Evelyn Dueck would say, stochastically produced content, podcasts, et cetera, you know, that seems to just really flatten out what it's possible to do with Patreon and what speech and communication and projects you can do.
• 48:49
  Mike Masnick: Yeah.
• 48:49
  Mike Masnick: I mean, it also gets a little bit at, you know, the impact that a company like Apple has on other areas of the creator space, just by the nature of their policies for the app store, which, which is worth thinking about.
• 49:01
  Mike Masnick: But on that, I wanted to go to our final story, which is we started this podcast out talking about a Ninth Circuit ruling in the Bonta case, and we're going back to the Ninth Circuit for another ruling that I think may be somewhat impactful, which.
• 49:15
  Mike Masnick: is about YOLO, which is perhaps either the best or worst name.
• 49:21
  Mike Masnick: Uh, in this case, there was a company called YOLO Technologies that seems like an awful company.
• 49:26
  Mike Masnick: And I'll state upfront that I, everything about this company appears to be awful, but they built some sort of add on to Snapchat.
• 49:33
  Mike Masnick: There were already concerns about how Snapchat is used among kids, but they created this add on that allowed you to post, questions or polls on Snapchat where people could respond anonymously.
• 49:46
  Mike Masnick: There's some areas where they could choose to reveal themselves, but basically they could respond anonymously.
• 49:52
  Mike Masnick: Not surprisingly, some people used it and then were harassed and bullied.
• 49:57
  Mike Masnick: in one tragic case, at least, someone appears to have died by suicide after being harassed and bullied via the app.
• 50:06
  Mike Masnick: so some of the, people who were harassed and bullied, including the family of the child who died, they sued YOLO.
• 50:14
  Mike Masnick: and made a bunch of claims.
• 50:15
  Mike Masnick: Some of them were product liability claims, which have been very popular over the last few years, for apps that a lot of kids use and occasionally abuse.
• 50:24
  Mike Masnick: but part of the claims were that the site itself, YOLO, the service said that the way that they dealt with bullying and harassment was that if they caught you harassing someone, they would.
• 50:37
  Mike Masnick: Reveal who you were.
• 50:39
  Mike Masnick: So you would no longer be anonymous if you were bullied and harassed.
• 50:42
  Mike Masnick: And the way the story plays out here is that these children who were bullied and harassed then reached out to YOLO for that purpose to try and unmask who was harassing them.
• 50:52
  Mike Masnick: And they got no response, like literally no response.
• 50:55
  Mike Masnick: In one case where the legal, contact was So basically, it appears that YOLO had no intention whatsoever of ever living up to this stated, anti bullying, feature and service.
• 51:12
  Mike Masnick: and so the lower court had dismissed the case and now the appeals court let some of it back in.
• 51:17
  Mike Masnick: So, Daphne, do you want to sort of summarize the finding of the Ninth Circuit?
• 51:22
  Daphne Keller: Yeah, so that the Ninth Circuit says the plaintiffs can't proceed on their products liability theories.
• 51:30
  Daphne Keller: Um, and so that makes this one of so many different cases going on right now, where there's some products liability theory probing the edges of Section 230 and, and in, um, child safety cases, including the multi district litigation consolidated in the Northern District of California courts have actually said, yeah, there's some things like infinite scroll, for example, that that's not user speech exactly.
• 51:53
  Daphne Keller: And, you know, we can debate that, but they're kind of finding edges.
• 51:56
  Daphne Keller: Um, but this, this court did not find that kind of an edge.
• 52:01
  Daphne Keller: Instead, it embraced this idea that, YOLO had made a promise to users and then it had breached that promise, you know, basically a breach of contract, and That's not immunized by section 230 because the cause of action arises from their promise.
• 52:20
  Daphne Keller: The court uses the word promise a lot rather than arising from the user content and I want to say, I think there's kind of a continuum of cases here that I'm more and less, Sympathetic to kind of stepping away from the law a little bit from the specific doctrine in these cases, section two 30 was supposed to allow platforms to moderate content.
• 52:42
  Daphne Keller: And not get in trouble if they made mistakes in moderating content.
• 52:47
  Daphne Keller: So like we want them in section two 30, wants them to moderate content.
• 52:51
  Daphne Keller: We also generally want them to disclose what the rules are.
• 52:55
  Daphne Keller: And often those rules are mentioned in their contracts.
• 52:57
  Daphne Keller: And so if the idea is every time a platform says something about content moderation, they expose themselves to a contract claim.
• 53:07
  Daphne Keller: That's just guts 230, right?
• 53:09
  Daphne Keller: Like it's ridiculous to say that that kind of case should be allowed to go forward.
• 53:14
  Daphne Keller: And the Ninth Circuit did allow a claim kind of like that to go forward in a case called Calise, which I think is actually a very problematic ruling.
• 53:24
  Daphne Keller: And I'll, come back to why in a minute.
• 53:27
  Daphne Keller: but this case Yolo feels a little different to me because If the facts alleged are true, you know, when you sign up for the product, there's a big notification saying, Hey, don't harass or bully people.
• 53:39
  Daphne Keller: If you do, we're going to disclose who you are and we're going to block you.
• 53:43
  Daphne Keller: I think also, you know, it's this sort of big, noisy representation about what the service is.
• 53:49
  Daphne Keller: And then they apparently had a staff of 10 people and there was no way they could possibly do this.
• 53:52
  Daphne Keller: And they didn't respond to inquiries, et cetera.
• 53:55
  Daphne Keller: And that.
• 53:57
  Daphne Keller: Again, not as a matter of legal doctrine, but hypothetical better legal doctrine.
• 54:01
  Daphne Keller: That feels almost more like fraud, right?
• 54:03
  Daphne Keller: Like they're, they're selling their service as being a thing that it just isn't.
• 54:09
  Daphne Keller: and I would like to think that there can be a line between those two kinds of cases where something with sort of an extreme commercial representation that's just can, can itself be, could, could stand outside section 230.
• 54:23
  Mike Masnick: But I think that the fear that I have, and I agree with all of that, the fear though, is that companies make marketing promises all the time, or they'll say, like, we don't allow bullying or something like that.
• 54:34
  Mike Masnick: And now does this open up the possibility that people could sue over things like that, or, you know, the sort of classic example I'm thinking of, where Twitter.
• 54:43
  Mike Masnick: Had executives claim that we are the free speech wing of the free speech party, and we're not going to take down content.
• 54:49
  Mike Masnick: And then they did because you have to, would that open up a lawsuit for saying, well, you said you were never going to take down this content.
• 54:56
  Mike Masnick: And then you did.
• 54:57
  Daphne Keller: Yeah, so I think there would have to be a lot of limits on what kinds of representations could be actionable.
• 55:03
  Daphne Keller: I do think that companies should be careful and say, we try hard to do X rather than sounding like they're guaranteeing they're going to take down all of a certain kind of content.
• 55:14
  Daphne Keller: Cause they're, they're definitely not going to succeed.
• 55:17
  Daphne Keller: I mean, this kind of gets to one of my worries.
• 55:20
  Daphne Keller: So the, The lawsuit that was most focused on the Twitter representation about being the free speech wing of the Free Speech Party was Taylor versus Twitter.
• 55:28
  Daphne Keller: And that was brought by Jared Taylor, who's a white nationalist.
• 55:33
  Daphne Keller: His brief was really interesting.
• 55:35
  Daphne Keller: It was basically like, I'm not Rude to anyone.
• 55:39
  Daphne Keller: I'm, I'm a gentleman racist.
• 55:40
  Daphne Keller: I'm just presenting my scientific theories.
• 55:42
  Daphne Keller: They shouldn't have banned me.
• 55:44
  Daphne Keller: But also he said Twitter, this was a bait and switch.
• 55:47
  Daphne Keller: They said they were a free speech platform and then they changed their rules to prohibit more content.
• 55:52
  Daphne Keller: And that made my business and you know, my use of Twitter no longer viable.
• 55:56
  Daphne Keller: And the district court was kind of sympathetic to that at first.
• 55:59
  Daphne Keller: Like the idea that there's a bait and switch in the commercial offering doesn't really sit right with judges.
• 56:06
  Daphne Keller: So I don't know exactly where the line should stand, but just to come back to Khalees, this, um, ruling that I think is so problematic from the Ninth Circuit earlier this year, I think that, you know, that court's reasoning was if you were harmed by content, or in this case harmed by ads for defective products, and you think that that was a breach of the contract, then you can sue on breach of contract, Has a mirror image, which is if you're Jared Taylor and you think that you had a right to share your white nationalist content online and that taking it down was a breach of contract, probably you can sue under Khalees also, you know, just opened the door to contract claims about every kind of content moderation dispute, including the must carry variety.
• 56:55
  Daphne Keller: So I find that ruling kind of troubling.
• 56:58
  Mike Masnick: Yeah, I think it's a concern.
• 57:00
  Mike Masnick: I mean, I think a lot of this goes back to, there's a case that, both of these cases cite, which is the Barnes case, Barnes versus Yahoo.
• 57:08
  Mike Masnick: I always keep trying to say YouTube, because I think with a Y, but it's Yahoo, because it's an old case.
• 57:13
  Mike Masnick: It goes all the way back to 2009.
• 57:16
  Mike Masnick: And that was one where there was, Content that somebody didn't want and they called up somebody at Yahoo and that person told them directly.
• 57:24
  Mike Masnick: I will get this taken down and In that case, which is mostly a good section 230 case I think actually has a lot of good section 230 language in that case.
• 57:33
  Mike Masnick: They said that that part There's the promissory estoppel, because this person directly promised to take care of it and didn't, that is outside of section 230 protections.
• 57:44
  Mike Masnick: and so like when that part came out, like I was worried about, and I'd written something saying like, this is kind of scary.
• 57:51
  Mike Masnick: But that hadn't really come up.
• 57:52
  Mike Masnick: We'd seen people sort of try and use Barnes in other cases and not successfully.
• 57:57
  Mike Masnick: And now in Kelis and now in this Yolo case this week, it seems that suddenly judges are using that.
• 58:04
  Mike Masnick: And so I sort of, you know, feels like that Barnes case from 2009 was like this ticking time bomb that was always sitting out there and now judges that don't like section two 30.
• 58:13
  Mike Masnick: And in the, Kelis case in particular, it The judge makes it clear that he's no fan of section 230 and would like to
• 58:21
  Daphne Keller: Quotes Clarence Thomas on the subject.
• 58:23
  Mike Masnick: quotes Clarence Thomas and it's pretty clear that Just think section 230 is bad and they're now sort of using this element of like the promise and turning it into a duty or sort of a contractual requirement.
• 58:37
  Mike Masnick: If you say something, you must do it and just opens up this wide area that I think a lot of lawyers are going to, make use of.
• 58:47
  Mike Masnick: in making claims in and around the ninth circuit and say, we can get around section two 30 because someone at this company or some marketing material from this company promised X and they didn't deliver on it.
• 58:59
  Mike Masnick: And so that, worries me at least.
• 59:01
  Daphne Keller: Yeah.
• 59:01
  Daphne Keller: I mean, so I think Barnes in principle should be distinguishable because it was this, one to one communication with a specific promise about a specific piece of content.
• 59:11
  Daphne Keller: I assume that plaintiffs have tried to use Barnes this way a lot and that courts have found a way to distinguish it.
• 59:20
  Daphne Keller: And I assume it's something like that.
• 59:21
  Daphne Keller: But to be honest, I don't remember.
• 59:23
  Daphne Keller: The, the main, when I used to teach Barnes, uh, in.
• 59:26
  Daphne Keller: Berkeley back in 2012, I did a very practice oriented version of my class.
• 59:32
  Daphne Keller: and I would point out that the person who made this representation was Yahoo's director of communications.
• 59:39
  Daphne Keller: And so the lesson from Barnes was don't let your PR people do trust and safety work, those need to be separate undertakings.
• 59:49
  Mike Masnick: Yeah, but, you know, it'll be interesting.
• 59:51
  Mike Masnick: I think that these cases, the Calise and the YOLO case are going to, they're going to lead to a flood of, new cases.
• 59:56
  Mike Masnick: There is good language, I think, in the YOLO case about the product liability side, which I think people have been very aggressive in making these product liability claims and YOLO makes it clear that those are, in this case, certainly are blocked by 230.
• 1:00:09
  Mike Masnick: but I, I worry about where these cases go and we're going to see a lot more of these in the future.
• 1:00:14
  Mike Masnick: So something to, pay attention to.
• 1:00:17
  Mike Masnick: But I think that is it for this episode of Control Alt Speech.
• 1:00:22
  Mike Masnick: So Daphne, thank you very much for stepping in and filling in for Ben this week and doing such an excellent job with so much interesting and relevant expertise and information for us.
• 1:00:34
  Mike Masnick: Cool,
• 1:00:36
  Daphne Keller: I hope you're having a good vacation.
• 1:00:37
  Daphne Keller: If you're listening, I hope you're not listening.
• 1:00:40
  Daphne Keller: Uh, but this was a lot of fun.
• 1:00:41
  Daphne Keller: Thanks for having me on.
• 1:00:44
  Mike Masnick: will be back with another episode next week.
• 1:00:50
  Announcer: Thanks for listening to Ctrl-Alt-Speech.
• 1:00:52
  Announcer: Subscribe now to get our weekly episodes as soon as they're released.
• 1:00:56
  Announcer: If your company or organization is interested in sponsoring the podcast, contact us by visiting ctrlaltspeech.Com.
• 1:01:03
  Announcer: That's C T R L Alt Speech.
• 1:01:05
  Announcer: com.
• 1:01:07
  Announcer: This podcast is produced with financial support from the Future of Online Trust and Safety Fund, a fiscally sponsored multi donor fund at Global Impact that supports charitable activities to build a more robust, capable, and inclusive trust and safety ecosystem.