Latest / Ctrl-Alt-Speech / A Tale of Two Internets

Transcript

A Tale of Two Internets · Ctrl-Alt-Speech

• 0:00
  Ben Whitelaw: Mike, if I asked you to spill some tea,
• 0:02
  Mike Masnick: no.
• 0:03
  Ben Whitelaw: what would you do?
• 0:06
  Mike Masnick: Uh, considering I am not a, a Gen Z person, spill some tea is like, you know, talking about like gossip and rumors, right?
• 0:18
  Ben Whitelaw: Yeah.
• 0:19
  Ben Whitelaw: Okay.
• 0:19
  Ben Whitelaw: So yeah, you're, you've got the lingo enough to understand what you're being
• 0:22
  Mike Masnick: I have kids.
• 0:24
  Ben Whitelaw: yeah.
• 0:25
  Ben Whitelaw: I, I Would've thought it was a, a kind of incident with a beverage.
• 0:30
  Ben Whitelaw: And, uh, and so I'm, glad your, one half of control or speech knows what they're talking about.
• 0:35
  Ben Whitelaw: But basically there's, there is a plethora of apps where you can, you know, spill your tea.
• 0:40
  Ben Whitelaw: And one of them unsurprisingly, is called spill,
• 0:43
  Mike Masnick: uh,
• 0:43
  Ben Whitelaw: right?
• 0:43
  Ben Whitelaw: Which helps you find the, the hottest spills and the most popular topics on the internet.
• 0:48
  Ben Whitelaw: it's great for kind of discovery.
• 0:50
  Ben Whitelaw: It's not got the best app reviews, I will say that.
• 0:52
  Ben Whitelaw: But nonetheless, we're using it today on control speech.
• 0:55
  Ben Whitelaw: its user prompt is not spill, which is, I think is a missed
• 1:00
  Mike Masnick: Yeah.
• 1:00
  Mike Masnick: Really?
• 1:02
  Mike Masnick: Or what's the T or or something.
• 1:03
  Mike Masnick: There's like a whole bunch of opportunities there.
• 1:06
  Ben Whitelaw: right,
• 1:07
  Mike Masnick: They should hire us for marketing.
• 1:10
  Ben Whitelaw: free product feedback right here.
• 1:12
  Ben Whitelaw: Um.
• 1:13
  Ben Whitelaw: But instead it asks its users to comment and, share what you are interested in.
• 1:19
  Ben Whitelaw: So I'm gonna give you the very generic, question today.
• 1:22
  Ben Whitelaw: Have what you're interested in.
• 1:23
  Mike Masnick: Oh, interesting.
• 1:24
  Mike Masnick: well, this week I was really interested in the sort of confirmation of the idea that trust and safety people are the ones who raise their hands, the ones who see a problem and rush towards it.
• 1:36
  Mike Masnick: And, and there have been various discussions on this, like, you know, startup companies that don't have trust and safety, something goes wrong.
• 1:43
  Mike Masnick: And someone who, it's not their job, but they say, Ooh, this is a problem.
• 1:46
  Mike Masnick: I need to solve it.
• 1:46
  Mike Masnick: I need to step up.
• 1:48
  Mike Masnick: And we saw that in partly horrifying, but also partly amazing way last week at a Wikipedia conference where unfortunately, a very troubled individual showed up with a gun, climbed on stage, and basically threatened to take his own life.
• 2:08
  Mike Masnick: Two people from the Wikipedia community stepped up and ran towards him, literally, and, got him, you know, someone sort of gra held him and they got him to calm down and, you know, got the authorities involved.
• 2:22
  Mike Masnick: But Showing that, you know, there, there are people out there who see a problem and rush towards it as opposed to run away from it.
• 2:30
  Mike Masnick: And I believe some of these people were involved in trust and safety with Wikipedia.
• 2:35
  Mike Masnick: And so it fits that stereotype of the trust and safety people are the ones who rush towards the problems and say, how can I help?
• 2:45
  Mike Masnick: And it's a crazy, crazy story, but I thought it was really.
• 2:49
  Mike Masnick: Interesting.
• 2:50
  Mike Masnick: A really interesting demonstration of that concept of the trust and safety people are the ones who, who rush towards the problem to try and do something.
• 2:58
  Ben Whitelaw: very much so.
• 2:58
  Ben Whitelaw: Internet heroes and Real Life heroes at the same
• 3:01
  Mike Masnick: Yeah.
• 3:02
  Mike Masnick: So, Ben, what are you interested in?
• 3:03
  Ben Whitelaw: Well, I, I'm interested, Mike, in whether I can remember anything from my history GCSE class, which is gonna be very important for today.
• 3:14
  Ben Whitelaw: if we are going to get through today's episode, I already am worried about what my history teacher would think about my knowledge of the Victorian era, which I'm going to try and delve into my brain and, and, regurgitate some of today, but.
• 3:28
  Ben Whitelaw: That gives you a little hint at what we're gonna be talking about in today's episode.
• 3:39
  Ben Whitelaw: Hello and welcome to Control Alt Speech, your weekly roundup of the major stories about online speech, content moderation, and internet regulation.
• 3:47
  Ben Whitelaw: It's October the 23rd, and this week's episode is brought to you by Clavata, a first of its kind automated content safety platform that allows you to go from defining a policy to enforcement in minutes.
• 3:58
  Mike Masnick: Yeah, I, I stay tuned for the, end of today's episode because I had a fascinating conversation with Clavata's founder Brett Levinson on how to make trust and safety more consistent and explainable.
• 4:11
  Mike Masnick: he's really focused on this concept of policy as code, which.
• 4:15
  Mike Masnick: Absolutely fascinating, and he and I had a really, really interesting discussion of what that means and how it works.
• 4:22
  Mike Masnick: So, definitely stay tuned for.
• 4:25
  Ben Whitelaw: Perfect.
• 4:25
  Ben Whitelaw: That sounds great.
• 4:26
  Ben Whitelaw: that'll be at the end of today's episode, but before then, we're gonna be talking about, whole raft of stories that have come out this week, including, a really interesting op-ed about Victorian morals and whether we're entering into a kind of Charles Dickens version of the internet.
• 4:39
  Ben Whitelaw: Disinformation as a mindset, which I think some of my, uh, family members know a bit about.
• 4:44
  Ben Whitelaw: And, uh, the constant battle between humans and ai.
• 4:48
  Ben Whitelaw: so look forward to that.
• 4:50
  Ben Whitelaw: Mike, before we crack on on today's stories, in last week's episode, we ask our listeners to, rate and review us, as we always do.
• 4:57
  Ben Whitelaw: we challenge listeners to leave a spooky review, where.
• 5:03
  Ben Whitelaw: They mentioned Halloween or some sort of Halloween reference, and tis the season after all.
• 5:09
  Ben Whitelaw: Um, we, we had a very good one, which I think you'll like, this is from a user in Germany and I won't expose them because you'll see why in a second.
• 5:18
  Ben Whitelaw: Um, they said that this Halloween I'll be dressing as EU chat control because there's nothing scarier.
• 5:25
  Mike Masnick: There we go.
• 5:26
  Ben Whitelaw: Which is a link to last week's, uh, episode for those who haven't listened, and this use.
• 5:31
  Ben Whitelaw: So thanks Mike and Ben for the inspiration.
• 5:33
  Ben Whitelaw: So not only are we getting reviews about Halloween, we are the inspiration for some people's Halloween celebrations.
• 5:40
  Mike Masnick: how one would dress up as chat control, but I love it.
• 5:44
  Mike Masnick: It does seem very, very scary.
• 5:46
  Mike Masnick: I appreciate it.
• 5:47
  Mike Masnick: That's.
• 5:48
  Ben Whitelaw: this user, wants to send us in a picture of themselves, a chat control, in a couple of weeks time when they're going to a friend's party, we would love that.
• 5:57
  Ben Whitelaw: We would absolutely love that.
• 5:58
  Ben Whitelaw: but yeah, we won't be telling, listeners who they are in case they're going to the same party you are.
• 6:03
  Mike Masnick: Consider it encrypted.
• 6:05
  Ben Whitelaw: Consider it encrypted.
• 6:06
  Ben Whitelaw: Indeed.
• 6:07
  Ben Whitelaw: Um, we also had another very nice, review from somebody who's a regular listener.
• 6:11
  Ben Whitelaw: I also went share this person's, username or who they are because they've dubbed themselves a non-techie.
• 6:18
  Ben Whitelaw: this review says, um, very accessible for non-super techies.
• 6:21
  Ben Whitelaw: These guys are smart and fun to listen to, and they cover a very important topic that is accessible to a Jen Dial phone.
• 6:27
  Ben Whitelaw: Like me, keep podcasting and I'll keep listening.
• 6:30
  Ben Whitelaw: Is a Gen, is that a kind of a Jen dial up phone.
• 6:33
  Ben Whitelaw: Are you familiar
• 6:34
  Mike Masnick: I, I've not heard that term before, but,
• 6:36
  Ben Whitelaw: somebody who's of the generation where Dial Up Internet existed is presumably or was the common
• 6:42
  Mike Masnick: Well, I, I, well, I was wondering, is it, it was a dial up internet or was it like from the generation where phones actually had a dial on it?
• 6:49
  Mike Masnick: Right.
• 6:49
  Mike Masnick: Which is what I had as a, as a kid was, you know, like dialing a, a number was, you actually had to turn the dial.
• 6:56
  Mike Masnick: and if you are of that generation, and this person may well be, there are some really fun YouTube videos, which I have watched in the past of handing kids today, an actual dial phone.
• 7:08
  Mike Masnick: And asking them how to call a number and they have no idea.
• 7:13
  Mike Masnick: And it is, it is both amusing and infuriating because like some of us grew up with actually having to dial a phone number.
• 7:21
  Mike Masnick: Now I'm older than you.
• 7:22
  Mike Masnick: So did you ever have to do that
• 7:25
  Ben Whitelaw: We didn't have one of those.
• 7:26
  Ben Whitelaw: I'm a, I'm a kind of button
• 7:28
  Mike Masnick: push button?
• 7:29
  Ben Whitelaw: yeah, I'm from that generation, so I, I would know how to use it now, but, um, back when I was younger, I would struggle, I have to say, so.
• 7:35
  Ben Whitelaw: You know, it's good to hear we have listeners of, of all kinds, of all shapes and sizes.
• 7:40
  Ben Whitelaw: and, uh, yeah, appreciate those listeners who, left us a review in the last week.
• 7:44
  Ben Whitelaw: Please, listeners, if you're a, regular listener of controlled speech, drop us a review.
• 7:48
  Ben Whitelaw: you don't have to do it.
• 7:50
  Ben Whitelaw: on your keyboard.
• 7:50
  Ben Whitelaw: You can do it via a, a gen dial phone.
• 7:53
  Ben Whitelaw: Why not just, you
• 7:55
  Mike Masnick: note, I will note really, really briefly, I got scolded by someone you know, who you are, for saying like, we promised to, to read any reviews that mentioned Halloween.
• 8:05
  Mike Masnick: And I was told that, we were opening ourselves up to, trolling by some people.
• 8:10
  Mike Masnick: And, you know, as a trust and safety related podcast, we should have thought through the, the possible, what we were putting ourselves into in this user generated content world, of, of promising something that maybe we wouldn't be able to deliver on.
• 8:26
  Ben Whitelaw: it's a fair call out.
• 8:27
  Ben Whitelaw: It's a fair call out.
• 8:28
  Ben Whitelaw: You know, we are very judicious with, uh, with not only the reviews, but people's information and obviously if you say something really terrible about us, it'll never see the light of day.
• 8:38
  Ben Whitelaw: Because
• 8:39
  Mike Masnick: We, we have the power.
• 8:40
  Ben Whitelaw: that.
• 8:42
  Ben Whitelaw: Exactly.
• 8:43
  Ben Whitelaw: Exactly.
• 8:43
  Ben Whitelaw: So, good reason to review us wherever you get your podcasts, does really help us get, discovered, increases our listenership, and all those good things.
• 8:52
  Ben Whitelaw: okay, so we, have got lots to get through, including that brilliant bonus chat with CTAs founder Mike.
• 8:58
  Ben Whitelaw: So, so we're gonna jump in, very quickly and talk a bit about, You know, we, we spoke last week about Sam Altman, right?
• 9:04
  Ben Whitelaw: We talked about AI and, open AI's policies and the way that, you know, these large language models are kind of going out and crawling all of this information, creating these huge databases.
• 9:15
  Ben Whitelaw: And here this week we're seeing the kind of other end.
• 9:18
  Ben Whitelaw: Of that and two platforms that I would probably call some of the, like most creative parts of the internet.
• 9:25
  Ben Whitelaw: Certainly some of the kind of parts that I frequent the most, are really struggling with AI bots and everything that comes with generative ai.
• 9:33
  Mike Masnick: Yeah, so two that you're talking about, I think are Wikipedia and Reddit.
• 9:37
  Mike Masnick: and so there were a couple different stories here.
• 9:39
  Mike Masnick: So we'll start with, there was an article in 4 0 4 Media, which.
• 9:44
  Mike Masnick: Is about how Wikipedia says that AI is causing a dangerous decline in human visitors.
• 9:50
  Mike Masnick: And this was based on a, a blog post, from Wikipedia sort of talking about this.
• 9:54
  Mike Masnick: And then 4 0 4 spoke to, the folks at, at Wikipedia and got a little bit more details on it.
• 10:00
  Mike Masnick: Basically they're saying, you know, this isn't too surprising.
• 10:03
  Mike Masnick: And, and the Wikipedia people don't sound too surprised by it, that, based on the fact that, AI tools are giving answers, sort of giving complete answers.
• 10:12
  Mike Masnick: It's often leading to fewer people actually going and visiting Wikipedia.
• 10:17
  Mike Masnick: even though much of the responses from AI systems were probably, effectively sourced from Wikipedia, Wikipedia is a hugely important database that goes into the training of the different AI tools.
• 10:31
  Mike Masnick: And so.
• 10:31
  Mike Masnick: they're seeing this decline.
• 10:33
  Mike Masnick: And the thing that struck me as interesting about it was that the folks at Wikipedia note that this is a concern and it's potentially a problem, but they're not freaking out about it.
• 10:44
  Mike Masnick: Right.
• 10:44
  Mike Masnick: I think, you know, whenever we've seen there have been similar stories about other sites.
• 10:49
  Mike Masnick: That feel like, oh, we're losing out on traffic from AI summaries or, or whatever.
• 10:54
  Mike Masnick: And usually they're really, really angry about it.
• 10:56
  Mike Masnick: And Wikipedia, not surprisingly, I would say, is sort of taking a more kind of academic observer approach to this and saying, you know, this is potentially concerning and we should think about it and we should have a discussion about it.
• 11:08
  Mike Masnick: But it's understandable why it's happening and people want.
• 11:12
  Mike Masnick: Answers and they're sort of using AI tools as answers and they sort of suggest like, maybe there's a better, you know, maybe we should be thinking about how do we encourage people to dig deeper?
• 11:23
  Mike Masnick: How do you check your sources, go for citations and.
• 11:27
  Mike Masnick: Know, it's interesting to me because it reminds me a little bit of the way that, people reacted to Wikipedia itself when it first came around and you know, there was a good decade where Wikipedia was seen as like a threat to, good information.
• 11:45
  Mike Masnick: And schools were saying never use Wikipedia.
• 11:48
  Mike Masnick: And students were told to stay away from it entirely.
• 11:51
  Mike Masnick: And now I think, As time has gone on and people have learned to use it properly, one of the things that people say is like Wikipedia is often a really good starting place to do research because it has all these citations and you shouldn't rely on it alone, but you should use it to explore the citations.
• 12:08
  Mike Masnick: And I think there's an argument that Wikipedia is making here, which is sort of similar, which is like.
• 12:13
  Mike Masnick: AI is a good starting source, but you shouldn't rely on it as the final thing.
• 12:17
  Mike Masnick: And more and more AI tools and most of the ones that people are using these days have gotten much better about adding links.
• 12:25
  Mike Masnick: I know Chat, GBT, which is the most popular one now, really does try to add links and sources.
• 12:31
  Mike Masnick: Google Gemini does that.
• 12:32
  Mike Masnick: Perplexity does that.
• 12:34
  Mike Masnick: They're all sort of trying to add sources.
• 12:36
  Mike Masnick: That allow you to go and investigate, and maybe it's a cultural thing, maybe it's a, like, I'm an old man kind of thing, but like, you know, to me, like I am always the kind of person who tries to read as much as I can about something.
• 12:49
  Mike Masnick: I mean, it obviously depends on the subject you know, if I'm just looking for like the time that the shop closes or whatever, I don't have to go investigate too deeply.
• 12:55
  Mike Masnick: But like there are other things where it's like, I want more sources to sort of really triangulate how I think about things.
• 13:01
  Mike Masnick: And so.
• 13:02
  Mike Masnick: Wikipedia for me has always been useful, but I've also found AI tools to be useful in opening up more sources and allowing me to, look at it, which is why, as we were just talking about before we started recording, I tend to have like 500 tabs open at once, but part of it is because I do these kinds.
• 13:21
  Mike Masnick: I do these kinds of searches and it leads me to more and more, and I'll just open up all the tabs and kind of start going through them.
• 13:27
  Mike Masnick: And so I think part of the argument from Wikipedia here is like, this AI thing is, is creating a challenge, but it's something that we should think through.
• 13:35
  Mike Masnick: Like how do we encourage people to be better information consumers in this age of ai?
• 13:41
  Mike Masnick: And I think that's.
• 13:42
  Mike Masnick: That's a valid point and a valid way of thinking about it.
• 13:46
  Mike Masnick: and so I appreciated sort of the way that they looked at it as well.
• 13:49
  Ben Whitelaw: Yeah.
• 13:49
  Ben Whitelaw: I mean, the other part of, of the kind of comments from Wikipedia that this 4 0 4 media story talks about is how you sustain.
• 13:58
  Ben Whitelaw: Financially a, a kind of initiative like Wikipedia.
• 14:00
  Ben Whitelaw: Right.
• 14:01
  Ben Whitelaw: And, and that's sustained in both Yeah.
• 14:03
  Ben Whitelaw: Financial sense and also the kind of contributor sense what AI agents don't do or bots do or scrapers don't do is, you know, update Wikipedia pages yet, or, or add their knowledge that they've gleaned from.
• 14:18
  Ben Whitelaw: parts of the web as they've gone about providing answers to questions like yours and mine is, you know, they didn't go and re-update those pages and they certainly don't dip into their imaginary wallets and, you know, donate to Wikipedia's annual campaign to keep its lights on.
• 14:34
  Ben Whitelaw: so I think there's, what I got from this is there's this kind of asymmetric relationship.
• 14:39
  Ben Whitelaw: Between, kind of AI companies and the bots and, you know, Wikipedia and other.
• 14:45
  Ben Whitelaw: Data sources that have been used to train LLMs.
• 14:48
  Ben Whitelaw: and yes, that Wikipedia isn't sounding the alarm too much, but it does feel like a, a fundamental question, right?
• 14:54
  Ben Whitelaw: And we see this with other, other news organizations as well who produce original content whose content is scraped by ai.
• 15:00
  Ben Whitelaw: And then, producing AI overviews or responses.
• 15:04
  Ben Whitelaw: how do we sustain?
• 15:05
  Ben Whitelaw: Kind of original producers of information is, is a question I came back to you.
• 15:09
  Mike Masnick: And I think, I think it's important, and I, again, I sort of appreciated the way that Wikipedia is sort of raising this question and saying like, we should discuss this now, you know, before it's, it's a true emergency where like the signs are concerning and the trends are concerning and let's, have that conversation.
• 15:25
  Mike Masnick: Whether or not the problem is like people tend not to take these things seriously until it's an emergency.
• 15:31
  Mike Masnick: and that may be true here, but I do appreciate kind of the way that they're approaching it.
• 15:35
  Mike Masnick: And I do think, you know, I mean, it's kind of funny because as you were saying, like, and as was raised by Wikipedia.
• 15:41
  Mike Masnick: That, obviously they don't, donate money to, to the campaign, but also they don't donate, information and knowledge back to it.
• 15:48
  Mike Masnick: both of those are not definite, right?
• 15:51
  Mike Masnick: Like you could change that, but I could see a pretty quick, like people freaking out.
• 15:56
  Mike Masnick: If AI tools started editing Wikipedia, can you imagine the freakout that would occur?
• 16:03
  Mike Masnick: Right.
• 16:03
  Mike Masnick: Like, and so even as you raise that, I'm just like, huh, that's interesting.
• 16:07
  Mike Masnick: But you know, they could, you know, like what if they, they had, you know, a citation that, debunks something that is in Wikipedia or adds to something that is in Wikipedia.
• 16:18
  Mike Masnick: They could take on the role of a Wikipedia editor as long as they live up to the rules of Wikipedia in terms of.
• 16:25
  Mike Masnick: You know, having the citation, having it be neutral, all of these things, and, this is still early, but like, there is talk about the sort of agentic AI systems where you could give them a credit card and allow them to act on things.
• 16:41
  Mike Masnick: So you could imagine a world in which, an AI agent is donating to Wikipedia.
• 16:47
  Mike Masnick: But you know, I think that's, probably leaping a little far ahead into the, the science fiction future, but.
• 16:53
  Mike Masnick: It's kind of interesting.
• 16:55
  Mike Masnick: I mean, the other thing that I was wondering, you know, there is a little bit in here about the sort of cost of bots and the bots scraping Wikipedia constantly and how that is a real cost.
• 17:05
  Mike Masnick: But then there's the fact that there are fewer human people coming to, to Wikipedia.
• 17:10
  Mike Masnick: But you know, because Wikipedia doesn't rely on ads.
• 17:14
  Mike Masnick: In some ways, and I'm, this is not an endorsement.
• 17:16
  Mike Masnick: I should be really clear on this, like in some ways, right, fewer human people.
• 17:20
  Mike Masnick: Visiting means less traffic, which means less cost.
• 17:23
  Mike Masnick: A little bit to Wikipedia.
• 17:25
  Ben Whitelaw: Yeah.
• 17:25
  Mike Masnick: You know, I'm, I'm just sort of trying to balance out all the different, elements here.
• 17:30
  Mike Masnick: So it's not like, I mean, other sites will complain like, oh, we lost all these views that then help pay for things because the ads pay for things.
• 17:36
  Mike Masnick: But Wikipedia doesn't have that issue.
• 17:38
  Mike Masnick: Exactly.
• 17:39
  Mike Masnick: So it's a little bit different in terms of how you think about it.
• 17:42
  Mike Masnick: but it is still.
• 17:44
  Mike Masnick: I think it's the kind of thing that, lots of different websites that are informational websites that you know, are sort of grew up in the age of an open web are starting to deal with, which is this.
• 17:57
  Mike Masnick: idea of like, well, the open web was all about humans, and now we have the machines as well.
• 18:02
  Mike Masnick: and how does that play into it?
• 18:03
  Mike Masnick: Which I think leads into the second part of the, this story that we wanted to talk about, which was the other area of the internet that is often talked about as sort of being creative and user generated and, community focused, which is Reddit.
• 18:15
  Mike Masnick: and Reddit.
• 18:16
  Mike Masnick: You know, there's a lot of controversy around Reddit.
• 18:18
  Mike Masnick: Both, you know, there's plenty of good stuff and plenty of problematic stuff.
• 18:21
  Mike Masnick: but there was a new study that came out.
• 18:25
  Mike Masnick: Looking at how Reddit moderators, because again, like just in there must be some people who don't know how Reddit works, right?
• 18:32
  Mike Masnick: You have like all these subreddits that are community moderated and it's volunteers and they set their own rules and they moderate the moderate the way they do.
• 18:42
  Mike Masnick: some of those rules include no AI generated content.
• 18:45
  Mike Masnick: Some are much more open to ai, some are entirely about AI generated content, but every, every moderator gets to set their own rules.
• 18:52
  Mike Masnick: some folks at Cornell, my alma mater, always happy to see Cornell folks doing good stuff.
• 18:57
  Mike Masnick: did a study where they spoke to a bunch of Reddit moderators to try and figure out how they're dealing with the sort of influx of AI content and it's.
• 19:08
  Mike Masnick: Apparently quite a challenge, not surprisingly for some of them, because even if you have a rule that says no, AI generated content, as lots of forums do, subreddits do.
• 19:19
  Mike Masnick: How, how do you determine that?
• 19:21
  Mike Masnick: Right now, like we have a problem where like some content is obviously AI generated, but lots of content.
• 19:27
  Mike Masnick: It's not clear at all if it's AI generated.
• 19:30
  Mike Masnick: and there are AI generator checkers and they tend to be terrible.
• 19:34
  Mike Masnick: They're not good.
• 19:35
  Mike Masnick: They make errors in both directions, thinking AI stuff is not, and thinking non-AI stuff is.
• 19:40
  Mike Masnick: And so if they can't figure it out.
• 19:42
  Mike Masnick: How are the moderators figuring out?
• 19:44
  Mike Masnick: And it's a real struggle and they don't know.
• 19:46
  Mike Masnick: And then you have like, there's a gray area between like, is the content AI generated or is it, created by a human, but then modified by ai?
• 19:56
  Mike Masnick: Where is the line?
• 19:57
  Mike Masnick: What do you count?
• 19:58
  Mike Masnick: what determines it?
• 19:59
  Mike Masnick: And so, you know, I don't think the study has any answers, but it does really clearly call out the challenge That subreddit moderators are now facing and they're all sort of trying to puzzle their way through it.
• 20:12
  Ben Whitelaw: Yeah, I, I really like this piece of research and, and the way that the concerns that the moderators shared with the researchers kind of boiled down to, into three areas.
• 20:21
  Ben Whitelaw: Right.
• 20:21
  Ben Whitelaw: And I, I think they are my concerns about AI as a user as well.
• 20:26
  Ben Whitelaw: Right.
• 20:26
  Ben Whitelaw: You, the moderators talked about content quality, kind of social dynamics on the subreddit, exchanging and a kind of difficulty to kind of govern, I guess because.
• 20:36
  Ben Whitelaw: it's their platform.
• 20:37
  Ben Whitelaw: It's, they're in charge and.
• 20:39
  Ben Whitelaw: I think they're really helpful buckets to kind of almost place our concerns about generative ai.
• 20:44
  Ben Whitelaw: you know, some of the headlines we see each week, work slop, this new term that's been defined, where a worker is using generative AI to produce a, a, piece of work or to do a task, and is almost kind of creating more, more issues by using generative I to.
• 21:00
  Ben Whitelaw: do that.
• 21:00
  Ben Whitelaw: And it's been called by, I think some Stanford researchers work slop that fits into the content quality bucket, right?
• 21:07
  Ben Whitelaw: And so these, buckets are kind of helpful for us to kind of think about the, downsides and the trade-offs of using generative ai.
• 21:15
  Ben Whitelaw: I just love the fact that it's kind of originated from a group of moderators, 15 moderators who look after some of the most, Engaged communities on the internet.
• 21:24
  Ben Whitelaw: As you say, like I think moderators have so much to be learned from, and are often, as you said at the top of the, the episodes, you know, the people who kinda stick their hands up first, who are the first people to see something happen.
• 21:34
  Ben Whitelaw: And I think that's what we're this kind of research represents in, in a way.
• 21:37
  Ben Whitelaw: I had a question for you about Wikipedia, Mike.
• 21:39
  Ben Whitelaw: what would you do if you were Wikipedia.
• 21:43
  Ben Whitelaw: How would you, this is not a pitch to become their, you know, CEO or to,
• 21:50
  Mike Masnick: They do have an open CEO search right now.
• 21:53
  Ben Whitelaw: I, I noted this is an inadvertent job interview that, but like, what would you do?
• 21:58
  Ben Whitelaw: You know?
• 21:58
  Ben Whitelaw: cause Reddit, made a deal with, at least one big AI platform to sell its content.
• 22:04
  Ben Whitelaw: and you, you know.
• 22:05
  Ben Whitelaw: You could imagine, just like other news orgs have done, potentially Wikipedia doing something similar, if it didn't have this kind of nonprofit open web mission and ethos, do you think it would ever change that?
• 22:16
  Ben Whitelaw: Where would you go?
• 22:16
  Ben Whitelaw: What direction would you take it?
• 22:18
  Mike Masnick: Yeah, I mean, I think it's a challenge.
• 22:21
  Mike Masnick: I think, the concept of the open web is important.
• 22:24
  Mike Masnick: It's certainly important to the Wikipedia people.
• 22:26
  Mike Masnick: I think that they would not do it in that way.
• 22:30
  Mike Masnick: I do think there's an argument on the reverse side, which is that for the AI companies that really.
• 22:38
  Mike Masnick: Benefits so much from Wikipedia, they should be stepping up and voluntarily donating significantly.
• 22:46
  Mike Masnick: Um, and just saying like, the future of our business as a AI providers relies on good, high quality factual content, such as the kind that is produced by Wikipedia, and therefore it's actually in our interest to donate a significant sum of money to Wikipedia to keep it going for the sake of.
• 23:07
  Mike Masnick: Our own, you know, goodness, but not just like a license for us, but effectively just encouraging Wikipedia to be able to continue to do what it has done all along.
• 23:18
  Mike Masnick: And therefore, it is, you know, for the good of the world, not for the good of us as a company.
• 23:22
  Mike Masnick: This is somewhat similar to.
• 23:25
  Mike Masnick: An argument that I put out, uh, it's a while ago now, almost two years ago I think, where I said, for all these debates about copyright and scraping for journalists against AI companies, you know, all the journalism companies and the, all these lawsuits that are still going on the New York Times against open AI and whatnot.
• 23:43
  Mike Masnick: I think copyright is the wrong tool for that, even though that is where all the lawsuits are.
• 23:48
  Mike Masnick: And my argument was.
• 23:49
  Mike Masnick: The AI companies need more and more good high quality content Journalists produce good high quality content.
• 23:58
  Mike Masnick: I think the AI companies should be giving lots of money to journalism outfits not.
• 24:04
  Mike Masnick: For copyright, which to me is for paying for old content, but rather for the production of new content.
• 24:11
  Mike Masnick: So fund journalists, fund authors, fund content creators to create new content that goes out into the world that more people can enjoy, that is open.
• 24:21
  Mike Masnick: That is not just for the AI companies, not just for the, you know, to build into more slop, but.
• 24:27
  Mike Masnick: Give it to the, creators to, build more.
• 24:30
  Mike Masnick: And there are all different mechanisms for how that could work, but give it to them for the production of future content, not for paying for the access to old content, which is the copyright model.
• 24:39
  Mike Masnick: and I think there's an argument for that in Wikipedia as well.
• 24:43
  Ben Whitelaw: Yeah, I think it's interesting.
• 24:44
  Ben Whitelaw: I mean, other tech platforms in the past have done versions of this, right?
• 24:48
  Ben Whitelaw: So, you know, like Facebook.
• 24:50
  Ben Whitelaw: Used to pay news organizations to create video because video was the thing that kept users on platforms.
• 24:55
  Ben Whitelaw: and my issue is that what happens when you, when the well runs dry or there's a change of strategy, and all of a sudden you have that kind of money ripped away.
• 25:03
  Ben Whitelaw: But maybe that's a question we, we answered down the
• 25:06
  Mike Masnick: it's a different kind of challenge too, and, and I would differentiate it slightly in that Facebook paid people to do video on Facebook, right?
• 25:13
  Mike Masnick: It was, it was come into our walled garden.
• 25:16
  Mike Masnick: Come into our silo, and then as happened.
• 25:21
  Mike Masnick: We, changed the rules and realized that we've been, measuring video views all wrong.
• 25:26
  Mike Masnick: And it turns out nobody's actually watching your videos.
• 25:29
  Mike Masnick: Even though we told you plenty of people were, then we pull, the rug away.
• 25:33
  Mike Masnick: I think that's a problem, and that's why I'm sort of encouraging this concept and it feels, you know, like too philanthropic to ever actually succeed.
• 25:43
  Mike Masnick: But this idea of like, just.
• 25:45
  Mike Masnick: Give people, give creators money to create and don't limit them and say, it has to be on my platform.
• 25:52
  Mike Masnick: But just say, create stuff that is out and available in the world.
• 25:55
  Mike Masnick: And yes, it should be available for our.
• 25:58
  Mike Masnick: AI models to train on, but also for others, it is part of contributing to the open internet that has enabled AI models to exist at all, right?
• 26:08
  Mike Masnick: They're all trained on what is available on the internet, pay it back, pay it back to, to create more content for the open internet that is available for all, as opposed to locking it up.
• 26:19
  Mike Masnick: And so that's, you know.
• 26:20
  Mike Masnick: it's a little bit of a soapbox and maybe it's, foolhardy and, and nobody will ever do that, but I am going to keep beating that drum as long as I can.
• 26:29
  Ben Whitelaw: And, you know, I'm not sure if that'll get you the, the CEO role at Wikipedia, but it's a nice idea.
• 26:34
  Ben Whitelaw: And Sam Altman, if you're interested, you know, we've got a ready made, blueprint, ready to
• 26:39
  Mike Masnick: Yeah,
• 26:39
  Ben Whitelaw: uh,
• 26:40
  Mike Masnick: You could And, and pay us to create more content.
• 26:43
  Ben Whitelaw: Yeah, more episodes of Controlled speech.
• 26:46
  Ben Whitelaw: Um, cool.
• 26:47
  Ben Whitelaw: So that's a really interesting story.
• 26:48
  Ben Whitelaw: I feel like we could go deeper.
• 26:50
  Ben Whitelaw: we, wanna kind of keep going with today's stories I'm going to talk a bit about, a really interesting.
• 26:56
  Ben Whitelaw: Op-ed that was in the conversation this week.
• 26:59
  Ben Whitelaw: The conversation being the, again, openly published, free academic website.
• 27:05
  Ben Whitelaw: contributed to by lots of professors and researchers and academics.
• 27:08
  Ben Whitelaw: Really amazing
• 27:09
  Mike Masnick: Everything under Creative Commons license is I've republished conversation pieces.
• 27:13
  Mike Masnick: It's, it's great.
• 27:14
  Ben Whitelaw: Yeah.
• 27:14
  Ben Whitelaw: Fantastic.
• 27:15
  Ben Whitelaw: And, the piece that caught my eye was by, an academic called Alex Beatie.
• 27:19
  Ben Whitelaw: And he asked a really fascinating question, which is the thing that's gonna test my, sophomore or freshman, history knowledge as, for our US listeners is are we entering internet in the Victorian era?
• 27:32
  Ben Whitelaw: is this the Internet's Victorian era?
• 27:34
  Ben Whitelaw: And, I love this because the Victorian era was one of the.
• 27:37
  Ben Whitelaw: pieces of history that I loved talking about most and learning about most, right?
• 27:40
  Ben Whitelaw: The industrialization of, Britain and the world.
• 27:43
  Ben Whitelaw: the kind of really interesting technology shifts, the creation of the steam train, the light bulb, really interesting morality, aspects of the Victorian era where, you know, you're meant to be very, kind of demure and, and pure and, there's this emphasis on sanctity and following authority and anyway.
• 28:00
  Ben Whitelaw: Alex asked the question, are we entering this kind of era of the internet?
• 28:05
  Ben Whitelaw: And he uses, three things to kind of ask that question.
• 28:08
  Ben Whitelaw: He, he looks at age verification, which we've talked about at length on the podcast.
• 28:12
  Ben Whitelaw: He looks at the kind of social media bands and, not only the one that we've seen go through, parliament in Australia, but also the one that's making its way through.
• 28:22
  Ben Whitelaw: Legislation in, Denmark and also apparently there's one in New Zealand where, Alex is from and also the, the smartphone free childhood campaign, which is something we've touched on as well.
• 28:31
  Ben Whitelaw: Those three things make him wonder if we're seeing kind of resurrection of these Victorian moral ideas.
• 28:39
  Ben Whitelaw: But for a, a kind of 21st century internet age.
• 28:42
  Ben Whitelaw: and it's kind of fascinating to, you know, thought about it in this way, but.
• 28:46
  Ben Whitelaw: he points out that there's a, a link between, and I'd never thought about it like this, the idea of digital wellbeing and kind of an abstinence from technology and virtue.
• 28:56
  Ben Whitelaw: Right.
• 28:57
  Ben Whitelaw: So the idea that if you kind of reject technology, if you turn off notifications, if you get rid of apps on your phone or you stop using Instagram, if you set kind of, I dunno, particular restrictions for yourself, then that's a good thing.
• 29:11
  Ben Whitelaw: and I dunno about you, but we, I see this all the time.
• 29:14
  Ben Whitelaw: I do it myself.
• 29:16
  Ben Whitelaw: I've talked at length to my friends about how, you know, great I am, because I've set up an app that delays my, itch to go and use certain apps.
• 29:25
  Ben Whitelaw: my wife, I'm gonna shame her here, um, because she never listens to the podcast.
• 29:28
  Ben Whitelaw: She has, she has talked at length about getting rid of Instagram and how much better that makes her feel, which may be true, but there is a, there's a kind of virtue signaling element to it.
• 29:38
  Ben Whitelaw: and that idea of kind of anti-technology in virtue something that is a really interesting one.
• 29:42
  Ben Whitelaw: And you'll be pleased to know, Mike, that the person who kind of, I guess, uh, characterizes this most in this op-ed is our good friend, Jonathan Hyde.
• 29:52
  Ben Whitelaw: Uh, who, who will always give you, a chance to kind of rant about, um, So we will get into kind of why maybe height's, ideas.
• 30:01
  Ben Whitelaw: Beyond the anxious generation book that we talked about and, and maybe some of these kind of ideas about morality, perhaps have shaped the way we think about technology now.
• 30:09
  Ben Whitelaw: But I just wondered what you thought about the OPED and and what you made a.
• 30:12
  Mike Masnick: Yeah, it's really interesting and it's, it's sort of an interesting thought exercise and I don't know if I fully agree, but I think it's, it's worth exploring.
• 30:18
  Mike Masnick: I do think that there is the sort of aspects of morality play and virtue that have gone into a lot of the discussion.
• 30:26
  Mike Masnick: and I mean, I have seen some of that and it reminds me a little bit of the concept of the Baptist and the bootlegger.
• 30:33
  Mike Masnick: Do you know this, this concept or I, I don't know if I've brought it up before.
• 30:37
  Mike Masnick: It's pretty common in, in, in some like economic circles, which is that when regulations come around, you have sort of two groups that will team up and we'll coordinate in support of them.
• 30:50
  Mike Masnick: And, the Baptist and the bootlegger concept is with the issue of alcohol prohibition, where you have two groups who find it.
• 30:56
  Mike Masnick: Like, oh, prohibition of alcohol is good.
• 30:59
  Mike Masnick: One is the Baptist who are doing it for moral and virtuous reasons, and the other is the bootlegger who says, okay, if we outlaw alcohol, I'm going to get very, very wealthy.
• 31:08
  Mike Masnick: and you definitely have some elements of that within the sort of like, anti-tech, tech is bad, apps are bad.
• 31:17
  Mike Masnick: Kids on apps are bad.
• 31:18
  Mike Masnick: You have the people who are moralizing, who are, making a very moral campaign about it.
• 31:23
  Mike Masnick: And then you have the, groups who are like, oh, there's an economic advantage to me here.
• 31:27
  Mike Masnick: And this is something that just came out yesterday, which I sort of ranted about a little bit on Blue Sky, that is sort of somewhat connected to this, Is there's a, Supreme Court case going on right now.
• 31:38
  Mike Masnick: I'm not gonna go into the details of a Cox versus Sony, which is sort of technically a copyright case.
• 31:44
  Mike Masnick: But the, amicus briefs friend of the court briefs in favor of Sony came out yesterday and there was one from, NCOs, which is the National Center on, sexual exploitation.
• 31:57
  Ben Whitelaw: We mentioned them last week, right?
• 31:58
  Ben Whitelaw: Yeah,
• 31:58
  Mike Masnick: did I mention them last week?
• 31:59
  Mike Masnick: I don't even
• 32:00
  Ben Whitelaw: Yeah.
• 32:01
  Mike Masnick: And they're, yes.
• 32:02
  Mike Masnick: They're like this very moralizing group.
• 32:05
  Mike Masnick: they started That's right.
• 32:06
  Mike Masnick: I did mention them.
• 32:06
  Mike Masnick: They, they started out as,
• 32:08
  Ben Whitelaw: See, I was
• 32:09
  Mike Masnick: Yeah, and I apparently forgot, but, but, um, so they came out with a brief that they said was funded in part by the recording Industry of America, and that struck me as really, really surprising.
• 32:23
  Mike Masnick: Now, the ra, aa.
• 32:26
  Mike Masnick: Has an economic stake in this case, but they're trying to get someone who will make the moral argument for them.
• 32:33
  Mike Masnick: And it's a very baptist and bootlegger situation where, when you start to think about these things, what is the real incentive behind it?
• 32:42
  Mike Masnick: And is it really virtue or is it, there's an economic incentive behind the virtue?
• 32:47
  Mike Masnick: And I, and I think it's sort of a useful framing for thinking about this argument as well, because.
• 32:53
  Mike Masnick: I think there are some people who are jumping into these things with moral arguments and some not so much.
• 32:58
  Mike Masnick: I think there are some people who are thinking through these things in terms of like, well, this is new technology and we're learning how to use it.
• 33:05
  Mike Masnick: And some of learning how to use it is learning, when not to use it and how to balance your life.
• 33:10
  Mike Masnick: And I think those are valuable things to think about and sometimes that is presented under this sort of moral virtuous.
• 33:19
  Mike Masnick: Guys, because it sort of is a way of telling a narrative that convinces people to do it, right?
• 33:24
  Mike Masnick: This is some of the, practice of religion and some of the concepts in religion are really that, right?
• 33:29
  Mike Masnick: They're sort of telling people how to live a healthy life, but put into moral and, ethical terms you know, as opposed to like, this is why it's good for you.
• 33:39
  Mike Masnick: And so that always kind of happens in some form.
• 33:42
  Mike Masnick: I do think that this is.
• 33:43
  Mike Masnick: Interesting framing in that we should think through, like what happens if we're sort of overtaken by moral arguments as opposed to reasoned arguments and where does that lead us and what are the limitations of that?
• 33:55
  Mike Masnick: And I think this piece does it very well and sort of, comes to the conclusion like, we should be a little concerned about like what we're doing if we're overly focusing on the moral arguments, rather than thinking through like how do we, Get kids to use these tools properly.
• 34:10
  Mike Masnick: I mean, the same thing that we've talked about, uh, millions of times, and I will get on a soapbox about if given the, the chance.
• 34:16
  Ben Whitelaw: Yeah, which is what this podcast is about.
• 34:19
  Mike Masnick: But I think, I think it's a really interesting framing and I really liked, I hadn't thought about it this way and I, I like, I like the argument.
• 34:27
  Ben Whitelaw: Yeah.
• 34:27
  Ben Whitelaw: And I think I, I kind of was thinking a bit about the role of height again, and I don't want, I don't want this to become a
• 34:34
  Mike Masnick: I didn't even mention him.
• 34:35
  Mike Masnick: Okay.
• 34:35
  Ben Whitelaw: or I know, I know, I know he is got in my, he's got in my brain like a kind of worm.
• 34:39
  Ben Whitelaw: But, um, I went back to kind of like his, kind of moral.
• 34:43
  Ben Whitelaw: Foundation's theory that he proposed in the Righteous Mind, which is his kind of big book before yt, anxious Generation.
• 34:49
  Ben Whitelaw: And it, it was very interesting to me to see that kind of three of the five moral foundations looked very similar to the kind of Victorian ideals that I was talking about.
• 34:59
  Ben Whitelaw: Right.
• 34:59
  Ben Whitelaw: the idea of kind of sanctity and degradation authority versus subversion loyalty versus betrayal.
• 35:05
  Ben Whitelaw: they're kind of, they are.
• 35:07
  Ben Whitelaw: You could claim that they're not rooted in that particular era per se, but they, that is an era where they were particularly strong.
• 35:15
  Ben Whitelaw: And it's interesting that it's not that long ago, and it's interesting that he's, kind of landed there and so it kind of helped me realize that, okay, why are we seeing all these.
• 35:23
  Ben Whitelaw: ideas become so popular now?
• 35:26
  Ben Whitelaw: You know, height is one of those voices in the discussion, and a lot of what he's saying is rooted in this kind of Victorian
• 35:33
  Mike Masnick: and, even,
• 35:34
  Ben Whitelaw: I think is, is helpful for me at least.
• 35:36
  Mike Masnick: the anxious generation.
• 35:37
  Mike Masnick: There's this weird chapter in the middle, which is basically like we need to bring back spirituality, which really felt extremely outta place and reads very strange.
• 35:46
  Mike Masnick: But is a purely sort of moralistic, argument in the middle of the anxious generation, and which I never quite understood why it's there, but it sort of, it made me very uncomfortable reading it, like, why are you making this suggestion?
• 36:01
  Mike Masnick: and what is the basis for it?
• 36:03
  Ben Whitelaw: and you know, who as a parent or otherwise, doesn't want to feel like they're virtuous or that they're moral, right?
• 36:10
  Ben Whitelaw: So, so it is interesting.
• 36:12
  Ben Whitelaw: It is interesting just to kinda think about how maybe he's used some of these ideas and applied them to technology, smartphone, social media.
• 36:20
  Ben Whitelaw: We, we've talked about how the data's not very robust in, in the actual generation, or at least some academics believe It's not very robust, so.
• 36:27
  Ben Whitelaw: a great addition to, I think, a, a longstanding topic.
• 36:29
  Ben Whitelaw: We've, discussed.
• 36:30
  Ben Whitelaw: We will, I think, turn to a few other stories now, Mike, that we will quickly run through, before passing'em to, to yours, discussion with the GLO founder.
• 36:40
  Ben Whitelaw: and so you take us through what you were also, are you reading this week?
• 36:44
  Mike Masnick: Yeah, there are a, a bunch of interesting stories, but one that, that really caught my eye is that Tinder, has launched this, facial verification tool.
• 36:53
  Mike Masnick: and this was really interesting and obviously trust and safety at Match Group is run by former.
• 37:01
  Mike Masnick: control alt speech, guest host Yoel Roth.
• 37:03
  Mike Masnick: and he has been out in the media sort of talking about this, where they're having a system that will sort of prove that the, person opening an account is a human being that sort of matches the pictures that they're putting online.
• 37:19
  Mike Masnick: It's done in a very thoughtful way, which is not surprising given that Yoel is there and Yoel is a very thoughtful trust and safety person.
• 37:26
  Mike Masnick: it's not age verification.
• 37:28
  Mike Masnick: It's not collecting all sorts of private.
• 37:31
  Mike Masnick: Hidden data on you.
• 37:32
  Mike Masnick: It's just using a system that, basically, as I understand it, at least I haven't tried it obviously, but like, makes you prove that you're a human being where they can see a, a head and, you know, makes you move and do certain things.
• 37:46
  Mike Masnick: in other contexts there have been like age verification providers who talk about that, like proof of lifeness, which seems scary and I'm always just more concerned in the age verification realm.
• 37:57
  Mike Masnick: But this seems fairly clever and minimizing the risk factors, the threats that I'm always concerned with, with age verification, what kind of data you're storing.
• 38:07
  Mike Masnick: Part of that also is just the context of dating apps as opposed to social apps where there's just people talking with dating apps.
• 38:14
  Mike Masnick: There is in theory an eventual actual human to human meetup, where the security concerns are much greater.
• 38:21
  Mike Masnick: And obviously there've been all sorts of stories, you know, sort of.
• 38:23
  Mike Masnick: Generally referred to as catfishing stories of fake people on dating sites, tricking other people into meeting them where they can then be attacked, robbed or worse.
• 38:33
  Mike Masnick: There's all sorts of things along those lines, and so being able to actually prove that someone is who they say they are.
• 38:39
  Mike Masnick: and is, the human being that they say they are actually does make sense.
• 38:43
  Mike Masnick: and there have been similar tools on various dating sites in the past.
• 38:47
  Mike Masnick: According to the, articles, though, most of them have been voluntary.
• 38:50
  Mike Masnick: And the big change here is that they're going to mandatory.
• 38:53
  Mike Masnick: I just thought it was a really interesting story of like within a specific context, a certain type of app that is embracing a new technology to deal with a specific problem that contextually.
• 39:04
  Mike Masnick: is an issue, but doing it in a way that is thoughtful and probably, that the downside risks to it are not.
• 39:11
  Mike Masnick: as great as some other proposals that are out there.
• 39:14
  Mike Masnick: So I thought it was a really interesting and thoughtful approach to this from, Tinder that, is unique and is a sign of why it's nice to have, trust and safety that can experiment and find the right solutions to various problems within their context, as opposed to sort of like the top down one size fits all, Every site must do X, which is what you know a lot of the regulations are right now.
• 39:35
  Mike Masnick: Here's a situation where Tinder and match groups are said like, we have certain problems.
• 39:40
  Mike Masnick: We have people trying to fake accounts or whatever.
• 39:43
  Mike Masnick: Here is a clever way that hopefully solves that, or at least tackle some of that problem in a way that is not particularly harmful to our users and probably is really helpful.
• 39:53
  Ben Whitelaw: Yeah, it's always, you never want to believe, Without kind of critical thinking, the press release of a, of a, platform when they, when they release a, a safety, uh, a safety feature like this.
• 40:03
  Ben Whitelaw: But the numbers in this one from, Tinder are very chunky, right?
• 40:06
  Ben Whitelaw: So it's saying 60% decrease in exposure to bad actors based upon a, a kind of weighted sample that they did a 40% decrease in bad actor reports.
• 40:15
  Ben Whitelaw: You know, these are, Big numbers and, that's why UL says like, this is maybe the most impactful trust and safety feature I've seen in my 15 year career.
• 40:24
  Ben Whitelaw: Like, that's pretty notable.
• 40:25
  Ben Whitelaw: So, this idea of having verified badges.
• 40:28
  Ben Whitelaw: Also something we're seeing a lot more of, in various guys is LinkedIn do something similar.
• 40:32
  Ben Whitelaw: You have to upload an ID in that case.
• 40:34
  Ben Whitelaw: But again, we're back in a, we're seeing much more of the kind of trust, of the trust and safety dynamic being leveraged by platforms now, which I think is, is fascinating.
• 40:44
  Ben Whitelaw: great.
• 40:45
  Ben Whitelaw: I will quickly round up with a, very interesting piece of academic research.
• 40:48
  Ben Whitelaw: Mike that we spotted in our reading this week.
• 40:51
  Ben Whitelaw: this is a, a, great, study about how people who are most liable to believe misinformation, are most commonly not receptive to the influence of authority.
• 41:04
  Ben Whitelaw: and I'll explain a bit more about what that is.
• 41:06
  Ben Whitelaw: Basically, a bunch of kind of social psychologists surveyed people during the pandemic to ask them about why they believed or didn't believe about COVID misinformation.
• 41:16
  Ben Whitelaw: you know, some of the kind of famous claims that we saw in the news all the time, 5G Networks, causing the mind virus and that kind of stuff.
• 41:23
  Ben Whitelaw: And of all the questions that.
• 41:25
  Ben Whitelaw: Was posed to these, participants, Mike, the thing that they found as to be the kind of driving factor of why they believed or didn't believe something was whether they saw it as a kind of win to, you know, have their view as if they saw it as a strength to counter, what is a received wisdom or a, a widely held view, or, you know, and very often in that case was that was kind of government or authority view.
• 41:51
  Ben Whitelaw: And it was kind of, it really shows the problem and the challenge of, missing disinformation because it means that essentially before you can provide any reason or argument, the person who already has a view about how they see themselves in the world as, as a kind of contrarian, perhaps they're not They're not gonna be convinced otherwise.
• 42:10
  Ben Whitelaw: Right.
• 42:10
  Ben Whitelaw: They, it is gonna be very, very hard to come up with a perfect argument to convince that person because they see themselves in a particular light in a particular way.
• 42:17
  Ben Whitelaw: And it just goes to show you, I think why, the issue of misinformation, disinformation is such a kind of difficult challenge to crack and why so many platforms, academics, and the media have really struggled to get its kind of arms around it.
• 42:30
  Ben Whitelaw: this feels like a, a reason why.
• 42:32
  Mike Masnick: Yeah, I think that, you know, I, I've always said that a lot of the disinformation problem is really a confirmation bias problem.
• 42:37
  Mike Masnick: People are sort of predisposed to certain ideas and they're looking for confirmation of it, and I think this is some element of it, but I think.
• 42:43
  Mike Masnick: what this adds and makes it really interesting is how much of it is this sort of belief in themselves as contrarians and which allows them to discount any source that has evidence against them.
• 42:54
  Mike Masnick: and I, I'm sort of wondering to some extent where this came from because I think, you know, we've taught people for a while, which is good, which is like, you know, you should be skeptical of information that is provided to you.
• 43:05
  Mike Masnick: That has led some people into this, you know, like self identity as a contrarian.
• 43:10
  Mike Masnick: And I don't think like, being skeptical or being contrarian is necessarily bad, but like the issue is when that is your entire identity, which means you will reject anything that disagrees with you,
• 43:21
  Ben Whitelaw: Yeah.
• 43:22
  Mike Masnick: is not, that's not being skeptical.
• 43:23
  Mike Masnick: It's not really being skeptical, but you've, you've created this identity yourself.
• 43:26
  Mike Masnick: It's like, well, I'm a skeptic, which means I can just reject anything I don't like.
• 43:31
  Ben Whitelaw: Yeah.
• 43:32
  Mike Masnick: I think we, we should maybe recalibrate how we think about talking about skepticism and doing your own research, right?
• 43:38
  Mike Masnick: Like, you know, you have all these conspiracy theories who always talk about doing your own research, which is good in, theory, right?
• 43:44
  Mike Masnick: The idea of doing your own research and understanding these things.
• 43:46
  Mike Masnick: And you know, as I said earlier on the podcast, looking at, you know, exploring different sources and all that kinda stuff.
• 43:51
  Mike Masnick: But that's not what is really happening.
• 43:53
  Mike Masnick: What they're really doing is looking for confirmation bias that makes them feel better.
• 43:56
  Mike Masnick: Like, oh yeah.
• 43:57
  Mike Masnick: All you sheep are believing this, but I'm the contrarian and therefore, you know, like I know better.
• 44:04
  Mike Masnick: And that's kind of what this study is getting at is this sort of weird self identity of like, I don't believe those guys.
• 44:11
  Mike Masnick: I will find my own sources that, you know, promote how I am the skeptic in the contrarian.
• 44:17
  Ben Whitelaw: Yeah.
• 44:17
  Ben Whitelaw: You know what I thought about this research, Mike.
• 44:19
  Ben Whitelaw: I think it's the perfect Thanksgiving dinner table fodder.
• 44:25
  Ben Whitelaw: It's, it's the perfect research to get out when your like strange Auntie Anne says, you know, I'm skeptical about this thing that is, uh, verifiably true.
• 44:34
  Ben Whitelaw: And then you can say, auntie Anne, look, you know, I'm never gonna convince you.
• 44:38
  Ben Whitelaw: You are a self, you know, self-diagnosed contrarian.
• 44:42
  Ben Whitelaw: This research proves that.
• 44:44
  Ben Whitelaw: Enjoy your Turkey.
• 44:47
  Ben Whitelaw: Um, yeah.
• 44:49
  Ben Whitelaw: Um, and on that note, Other holidays are available.
• 44:52
  Ben Whitelaw: By the way, I'm just very conscious that Thanksgiving is, is coming down the track.
• 44:56
  Ben Whitelaw: Um, on that note, let's, go now into our bonus chat.
• 44:58
  Ben Whitelaw: Mike, give our listeners the, pitch as to why they should stick around and listen to this great conversation.
• 45:03
  Mike Masnick: this was a great conversation as Brett Levinson, who's always really interesting.
• 45:07
  Mike Masnick: I always enjoy talking to him.
• 45:08
  Mike Masnick: Very, very thoughtful and creative mind and thinking about trust and safety and challenges and thinking about, how do we make trust and safety more consistent and explainable the things that everyone always talks about in the trust and safety space.
• 45:23
  Mike Masnick: And he has this idea of policy as code and sort of what does that mean and how do you think about.
• 45:30
  Mike Masnick: As like subjective decision making and how do we make it a little bit more consistent and explainable?
• 45:37
  Mike Masnick: And so it was a really, really fun and fascinating discussion and I think we'll just jump right to that right now.
• 45:52
  Mike Masnick: Welcome to Control Alt Speech.
• 45:54
  Mike Masnick: Brett, let's dive right in here.
• 45:57
  Mike Masnick: I know you've been thinking a lot about this concept of policy as code, and even partially how it might challenge my mass Nixon impossibility theory.
• 46:06
  Mike Masnick: So can you possibly, so can you explain what, what do you mean by policy as code?
• 46:13
  Brett Levenson: Sure.
• 46:14
  Brett Levenson: Yeah, of course.
• 46:15
  Brett Levenson: so I guess I would start with, so I'm an engineer my whole life.
• 46:19
  Brett Levenson: and there are certain things that I think are, like people just think of code as like coders and maybe like that's, you know, a thing that engineers deal with.
• 46:27
  Brett Levenson: But I think like when we talk about code, there are some things about it.
• 46:32
  Brett Levenson: That are true, whether it's a very complex programming language or a very simple programming language, like I, I would guess I would almost say like there's a bar for what the point at which I would consider something to be truly like code basically.
• 46:43
  Brett Levenson: Dykstra himself actually sort of had, had talked about this many years ago and, talked about the folly of natural language programming specifically, because natural language is inherently imprecise.
• 46:54
  Brett Levenson: And so to me, like the first thing about code is that.
• 46:58
  Brett Levenson: It is inherently precise, right?
• 47:01
  Brett Levenson: In every programming language that we've ever built.
• 47:04
  Brett Levenson: The idea is like, you can write a line of code and the results are sort of deterministic.
• 47:09
  Brett Levenson: You know, that if, if I do this, the outcome will be this.
• 47:13
  Brett Levenson: and so I think like that's the first thing about code, right?
• 47:15
  Brett Levenson: It, it creates a much more deterministic, tructure.
• 47:19
  Brett Levenson: Environment.
• 47:20
  Brett Levenson: this may not have always been true, certainly in the days of punch cards and things like that, but generally speaking, code is also testable.
• 47:27
  Brett Levenson: That's sort of like the second, primitive, I guess I would say around code is like, it's a testable object again, because, you know, like.
• 47:34
  Brett Levenson: If I put these inputs in and I run this code, I expect to get this outcome.
• 47:40
  Brett Levenson: and so for something in my mind to be code, it should be testable.
• 47:44
  Brett Levenson: You should be able to effectively predict, you know, we'll get into this sort of details of, of policy as code a little bit, but you should be able to, predict within some range of possible outcomes what the outputs might look like.
• 47:56
  Brett Levenson: and then I think.
• 47:57
  Brett Levenson: To be useful code.
• 47:58
  Brett Levenson: I guess I would add this little detail.
• 48:00
  Brett Levenson: the code should be debug able, in other words, if it's not working the way you intend for it to, it should be possible to sort of introspect into the code and go, okay, well.
• 48:11
  Brett Levenson: Lemme see what's the state at this point, and at this point, and at this point, and through that exploration, understand, well, how might I change the code to get the results, that I want?
• 48:21
  Brett Levenson: and so I think like, you know, I won't go on forever.
• 48:24
  Brett Levenson: There's a whole bunch more things I, I could talk about.
• 48:26
  Brett Levenson: But I think, like when I talk about policy as code, it's really something I came up with when I was working for Meta, um, at, you know, in Integrity, the.
• 48:35
  Brett Levenson: basic idea that was in my head was like, God, everything we're being asked to do is so subjective.
• 48:42
  Brett Levenson: There's all this subjectivity.
• 48:44
  Brett Levenson: That's just like a starting place.
• 48:45
  Brett Levenson: We have to deal with whatever we're trying to moderate.
• 48:48
  Brett Levenson: Can we at least find a way to describe a policy?
• 48:53
  Brett Levenson: Is more precise, that is more structured, that is testable, that is debu.
• 48:57
  Brett Levenson: so I think like those were the original ideas that I had behind, policy as code and, and ultimately I think certainly contributed to a lot of what we've built at ccle.
• 49:07
  Mike Masnick: And, and I think that's important because I think one of the things that people struggle with in the trust and safety space is you have these policies and people look at it and constantly say.
• 49:20
  Mike Masnick: Like some people will disagree with the policies, but rarely is that the issue.
• 49:24
  Mike Masnick: The issue is almost always that they disagree with the, implementation or the execution of the policies.
• 49:29
  Mike Masnick: And they say, well, This person violated your policy based on their interpretation.
• 49:34
  Mike Masnick: And other people say, well, but no, it's not like we interpret things this way.
• 49:38
  Mike Masnick: Or people say, you know, this person didn't, you know, you punished this person.
• 49:41
  Mike Masnick: I actually just got this very long email from someone claiming like, how dare.
• 49:46
  Mike Masnick: Such and such website have banned me because I didn't violate their policy.
• 49:50
  Mike Masnick: I only did X, Y, and Z.
• 49:52
  Mike Masnick: And it's like, well, I could see how that might, but there's the interpretation side of it.
• 49:56
  Mike Masnick: And so the thing that I think you're getting at, which is interesting is that because everyone, you know, people have struggled with this idea that policy has been.
• 50:08
  Mike Masnick: subjective.
• 50:09
  Mike Masnick: And so is the goal here thinking of policy as code is like, can we get to a point where policy is less subjective and that there's some sort of more objective component to it, or is it something different?
• 50:22
  Brett Levenson: No, I think you're, you're hitting on exactly, the system and the framework and ultimately the language that, that really we're trying, you know, at colada to create.
• 50:30
  Brett Levenson: uh, I guess what I would say is.
• 50:32
  Brett Levenson: Once you start thinking deeply about policy as code, you start to realize it's not just an, it's not enough to just go like, oh, well, we'll make it code.
• 50:38
  Brett Levenson: Um, you actually, you actually kind of have to, you have to think about like, what are the right constructs, what's the right set of logical operators, what's the right, set of functions that you want to give people access to so that they're able to remove some of the interpretation and subjectivity.
• 50:56
  Brett Levenson: For those who, uh, hopefully many people are familiar with the idea of a decision tree.
• 51:01
  Brett Levenson: and I think largely what in my mind, policy as code should try to create and try to enable users to create is, a well-defined decision tree.
• 51:13
  Brett Levenson: and what we actually see from our users is that very often that decision tree starts out kind of shallow.
• 51:19
  Brett Levenson: but because it's testable.
• 51:21
  Brett Levenson: Because you can see how it's being interpreted over time.
• 51:25
  Brett Levenson: You start to realize like, you know what?
• 51:26
  Brett Levenson: There's a little too much room for interpretation in this rule.
• 51:31
  Brett Levenson: I wrote The good news then is, okay, you know what?
• 51:34
  Brett Levenson: I can actually break this rule down further.
• 51:36
  Brett Levenson: the example I, I always give people, came from, something I saw during my career, which was, subject, everybody loves a policy about pornography, right?
• 51:46
  Brett Levenson: And one of the exceptions and anybody who's sort of worked in safety knows, like this is one of the things that makes it hard is that there are exceptions to every imaginable rule you could come up with.
• 51:55
  Brett Levenson: One of the exceptions was, well, if it's art, it's not pornography.
• 51:59
  Brett Levenson: Right?
• 52:00
  Brett Levenson: Okay, what's art?
• 52:01
  Brett Levenson: Now we have to interpret and define art.
• 52:04
  Brett Levenson: Right?
• 52:05
  Brett Levenson: But you can interpret and define art.
• 52:07
  Brett Levenson: Maybe not.
• 52:08
  Brett Levenson: And, and you know, I know you and I have talked about this before, like maybe not.
• 52:12
  Brett Levenson: To a definition that will satisfy everyone, but actually given how hard it is to do moderation effectively, I'm sort of a proponent of the idea that like I'm willing to sacrifice a little bit of what I would call subjective accuracy.
• 52:26
  Brett Levenson: Like how happy is everybody gonna be with the definition?
• 52:29
  Brett Levenson: For consistency in decision making and transparency in decision making, right?
• 52:35
  Brett Levenson: Like, can we make the rules precise enough and if they're not yet precise enough, can we continue to iterate on them and make them more and more precise, and continue to sort of increase the depth of that tree, until hopefully there's l little room left for misinterpretation of the rules.
• 52:54
  Brett Levenson: I think that's largely one of the goals of structuring policy as code is to give people a way to build that decision tree.
• 53:01
  Brett Levenson: Easily extend it as of course the world throws new and unexpected examples at us.
• 53:07
  Brett Levenson: and of course, quickly.
• 53:09
  Brett Levenson: tie that iteration loop together, right?
• 53:10
  Brett Levenson: We write the policy, we test the policy, we see the results, and if they're not what we want, we go back and we write the policy again.
• 53:17
  Brett Levenson: And, and, and hopefully if we've, executed all of this correctly, that's minutes, not, weeks or months, as is often the case.
• 53:23
  Mike Masnick: Right, right there.
• 53:24
  Mike Masnick: There's a famous radio lab episode, which is now very, very out of date where they were exploring and they actually got like to go inside the room.
• 53:33
  Mike Masnick: I don't know if you.
• 53:33
  Mike Masnick: There at Meta or Facebook at the time where they were like determining some of these things.
• 53:38
  Mike Masnick: And the point they kept making is like, okay, well we set a rule and then it's like suddenly you have this situation, which is like, where does it fit the rule?
• 53:44
  Mike Masnick: Okay, well we write this sub clause to the rule and then you have this, another situation.
• 53:48
  Mike Masnick: You're like, Ooh, we have to write another.
• 53:49
  Mike Masnick: And they sort of talked about how the rules just expanded and expanded and expanded.
• 53:53
  Mike Masnick: And then, then you sort of have the related problem of like, how do you have someone who understands this massive book of rules and can then, do stuff with it.
• 54:02
  Mike Masnick: Consistently and it becomes its own issue.
• 54:05
  Mike Masnick: the other issue that comes up and I'm sort of curious, your take on this, and this is always my concern with a lot of these is, how context plays into this.
• 54:12
  Mike Masnick: Because there are often cases where there are moderation decisions that if you strip it of all context.
• 54:20
  Mike Masnick: You can say, oh, this situation is just like that situation.
• 54:23
  Mike Masnick: But if you add back in the context, you quickly realize like, oh wait, these are wholly different situations.
• 54:28
  Mike Masnick: And people who are claiming they're the same, it's often bad faith where it's like you're, you're stripping away the important context.
• 54:35
  Mike Masnick: So I'm curious, as you're thinking about policy as code, how do you build in the ability to handle context and how does that work?
• 54:43
  Brett Levenson: We actually have a context operator, like we, we realized very early on, how important context actually is, to the decision making.
• 54:52
  Brett Levenson: I guess a little history, when I, first conceptualized this, like LLMs were not, Everywhere yet.
• 54:58
  Brett Levenson: we knew about them Facebook AI research and like there was early research going on, but they certainly weren't publicly available.
• 55:04
  Brett Levenson: so the original conception of policy as code was like, well, can we break what is effectively, complex policy that we're gonna ask somebody to make a decision about down into little bits of information?
• 55:15
  Brett Levenson: What we ended up calling info first, basically, And then ask each of those questions to our human reviewers to make the decisions, hopefully less judgment heavy, let's say.
• 55:26
  Brett Levenson: Right?
• 55:26
  Brett Levenson: But we knew even then that like the context was going to be important.
• 55:31
  Brett Levenson: And so clearly, like there's two requirements here, right?
• 55:34
  Brett Levenson: One is the context has to be available.
• 55:36
  Brett Levenson: the classic example is like a social media post and there's an image and a piece of text, and individually they're fine, but together they're potentially really offensive or really problematic.
• 55:48
  Brett Levenson: I, someone just cited a, a great example of this to me the other day and I was like, I can't repeat it here, let's just say, but, um,
• 55:56
  Mike Masnick: Okay.
• 55:57
  Brett Levenson: but, um.
• 55:58
  Mike Masnick: It, it, this is a, a hazard of the, of being in this space is that you often come across stuff that you cannot repeat publicly.
• 56:06
  Brett Levenson: but nonetheless, like one, you must have the context available.
• 56:09
  Brett Levenson: And I do think this is, you know, this is why we've strived to make sure we can do multimodal evaluation.
• 56:13
  Brett Levenson: And I honestly think.
• 56:14
  Brett Levenson: Anybody who's serious about moderation, like in today particularly with the availability of the technology we have, should be looking to do multimodal evaluation and to make sure that that context can be included.
• 56:25
  Brett Levenson: But then as we've designed effectively our language for policy as code, we have a context operator so that we can essentially say, Hey, these rules must be true.
• 56:36
  Brett Levenson: But they're only true if the context is X, y, or Z.
• 56:40
  Brett Levenson: Basically, when the context is not x, y, or Z, this is not true.
• 56:44
  Brett Levenson: Or of course, vice versa as well.
• 56:46
  Brett Levenson: You could say if the context is not this, then the rules are true.
• 56:49
  Brett Levenson: but I think to your point, right, like there's plenty of examples of of cases, you know, that I think the classic one that always comes to mind for me is like.
• 56:57
  Brett Levenson: We, and I used to hear about this from friends when I did work at Meta Integrity, and they were like, well, I was, I made a post and I was speaking out against racism.
• 57:05
  Brett Levenson: Why did my post get taken down?
• 57:07
  Brett Levenson: Right?
• 57:07
  Brett Levenson: Uh, and I was like, uh, God, the models just aren't good enough to understand, you know, the, the context of what you were saying.
• 57:14
  Brett Levenson: They, they simply understood that you were referencing something that that can create toxicity basically.
• 57:19
  Mike Masnick: right.
• 57:20
  Mike Masnick: Yeah.
• 57:20
  Brett Levenson: you know, I think.
• 57:21
  Brett Levenson: this is why to me, actually designing policy as code has been a really interesting exercise because it, it, it bears similarities to other kinds of programming languages, sort of.
• 57:31
  Brett Levenson: but I've never seen a programming language that has a context operator.
• 57:34
  Brett Levenson: And, you know, so we've had to definitely like invent some paradigms that that never existed before.
• 57:40
  Mike Masnick: Right.
• 57:40
  Mike Masnick: Yeah, no, it's, it's really fascinating.
• 57:43
  Mike Masnick: is there anything else, I mean there, there's obviously a lot we could go into, but, in the interest of time, is there anything else specifically that you think our listeners should be thinking about as regards to this concept of policy as code?
• 57:58
  Brett Levenson: I mean, I think the other thing that I would maybe call out about policy as code, there's really two things.
• 58:03
  Brett Levenson: So one, I, I think it.
• 58:05
  Brett Levenson: One of the things I love about it and that I wished I had when I was at Meta is that it actually connects the subject matter experts, you know, the people who are responsible for maintaining the policies, for updating the policies, for thinking deeply about what we want on our social platforms and, you know.
• 58:23
  Brett Levenson: This ties to the second thing, like more importantly now, perhaps how we want ouris behaving.
• 58:28
  Brett Levenson: What, what behaviors we want them to engage in.
• 58:31
  Brett Levenson: you know, when should we be worried about them leaking information?
• 58:33
  Brett Levenson: When should we be worried about hallucinations and truth?
• 58:36
  Brett Levenson: one of the things I really like about it is that those people, it gives them pretty much direct control.
• 58:42
  Brett Levenson: Direct impact when it comes to the outcomes, which is like something that at Meta was simply not possible.
• 58:49
  Brett Levenson: You know, the policies got written, they kind of got thrown over the wall and operations did their thing, and the policy got translated into it, into something that we could train human reviewers on.
• 58:58
  Brett Levenson: And then the human reviewers got trained and then they labeled and the labels came to me.
• 59:02
  Brett Levenson: And so basically it was like the labels were the representation of policy, and I just kind of had to hope.
• 59:07
  Brett Levenson: That it matched up with what the policy author said, and often it didn't.
• 59:11
  Brett Levenson: But that's a story for a different time.
• 59:13
  Brett Levenson: so I think like one of the reasons I would encourage people to look at, you know, however, implemented this idea of policy as code is that I think it can significantly, incentivize our subject matter experts, the people who are responsible for this.
• 59:26
  Brett Levenson: To sort of have agency in the process, right?
• 59:29
  Brett Levenson: Like I make a change.
• 59:30
  Brett Levenson: That change is, impacts the platform.
• 59:32
  Brett Levenson: And if it's not having the impact I want, well, I, it's just another change away basically.
• 59:37
  Brett Levenson: and then two, what we are seeing at least is that it's been incredibly useful.
• 59:42
  Brett Levenson: For latency sensitive applications, which these days more and more means AI agents.
• 59:48
  Brett Levenson: Right.
• 59:48
  Brett Levenson: Um, and there's so many, I guess interesting use cases I, could talk about, but, you know, we work with customer service agents and retail sales agents and internal ais that have access to databases.
• 59:59
  Brett Levenson: and it, you know, in these use cases, it's like if you're gonna have any kind of decent user experience.
• 1:00:04
  Brett Levenson: the moderation kind of has to be invisible to some degree and for it to be invisible.
• 1:00:10
  Brett Levenson: Yeah, and I guess this is what I'll say is like.
• 1:00:13
  Brett Levenson: One of the cool things about policy as code that we have discovered is that it allows us to get pretty tricky about how we actually go about doing the moderation.
• 1:00:23
  Brett Levenson: No longer is this just one big decision we have to render.
• 1:00:26
  Brett Levenson: It's lots of little decisions.
• 1:00:28
  Brett Levenson: We're trying to, you know, fit together into a puzzle.
• 1:00:31
  Brett Levenson: and that means this may get a little too techy, but like that means we can parallelize the work.
• 1:00:36
  Brett Levenson: That means if we discover very early in the process that there is a violation, maybe we just don't finish the rest of the process.
• 1:00:42
  Brett Levenson: 'cause we already know the answer basically.
• 1:00:44
  Brett Levenson: and so I think.
• 1:00:46
  Brett Levenson: Thinking ahead to the AI driven world that I think is coming our way.
• 1:00:50
  Brett Levenson: personally I think policy as code is going to be a really powerful tool in the toolkit, to make sure that all theis that are coming our way, stay within the bounds of behavior that we want them to, you know, maybe adhere to brand and tone style guides.
• 1:01:04
  Brett Levenson: and even, you know, certainly don't do worse things than that, you know, obviously are possible.
• 1:01:09
  Brett Levenson: So, yeah, I, I think, um, that's one other thing I would add there.
• 1:01:13
  Mike Masnick: That's great.
• 1:01:14
  Mike Masnick: Great.
• 1:01:14
  Mike Masnick: Well, once again, Brett Levinson from cda.
• 1:01:17
  Mike Masnick: Thanks so much for having this discussion.
• 1:01:19
  Mike Masnick: It's really interesting.
• 1:01:20
  Mike Masnick: I think it's lots of stuff to think about, and I'm sure we'll continue to discuss this idea of policy as code and sort of as the, the space continues to develop.
• 1:01:30
  Mike Masnick: It's a really interesting concept, so thanks for coming on board.
• 1:01:32
  Mike Masnick: Okay.
• 1:01:33
  Brett Levenson: Thanks so much, Mike.
• 1:01:33
  Brett Levenson: I really appreciate the time.
• 1:01:37
  Announcer: Thanks for listening to Ctrl-Alt-Speech.
• 1:01:39
  Announcer: Subscribe now to get our weekly episodes as soon as they're released.
• 1:01:43
  Announcer: If your company or organization is interested in sponsoring the podcast, contact us by visiting ctrlaltspeech.com.
• 1:01:50
  Announcer: That's CT RL alt speech.com.