Latest / Logbuch:Netzpolitik / LNP504 The politics of intellectual shame

Transcript

LNP504 The politics of intellectual shame · Logbuch:Netzpolitik

• 0:00
  Tim Pritlove: Good morning, Linus.
• 0:01
  Linus Neumann: Good morning, Tim.
• 0:02
  Tim Pritlove: Linus, what watch?
• 0:04
  Linus Neumann: Ten watch.
• 0:05
  Tim Pritlove: Such much?
• 0:06
  Meredith Whittaker: You'll get along beautifully in America.
• 0:30
  Tim Pritlove: Logbuch Netzpolitik Nummer 503 vom 16.
• 0:34
  Tim Pritlove: Oktober 2024 und ihr werdet es schon gemerkt haben.
• 0:41
  Linus Neumann: Wir sprechen eine andere Sprache.
• 0:43
  Tim Pritlove: Wir sprechen eine andere Sprache heute, weil wir haben heute einen Gast und
• 0:47
  Tim Pritlove: das ist dann ein Gespräch, was wir heute auf Englisch führen wollen.
• 0:52
  Linus Neumann: Bevor wir aber kurz unsere Episodennummer korrigieren, ist nämlich 504.
• 0:56
  Tim Pritlove: Was habe ich gesagt?
• 0:57
  Linus Neumann: 503.
• 0:58
  Tim Pritlove: Oh.
• 1:01
  Tim Pritlove: Ja, ausnahmsweise lebe ich mal nicht in der Zukunft.
• 1:03
  Linus Neumann: Man merkt, wir sind aufgeregt.
• 1:06
  Linus Neumann: Tim hat ja einen britischen Pass und muss jetzt mal seine Britischkenntnisse unter Beweis stellen.
• 1:11
  Tim Pritlove: Was habe ich einen britischen Pass? Ich habe gar keinen britischen Pass, ich bin Deutscher.
• 1:14
  Linus Neumann: Okay, aber du hattest ja nicht einen britischen Pass.
• 1:16
  Tim Pritlove: Das ist aber abgelaufen.
• 1:17
  Linus Neumann: Achso, der ist abgelaufen. Okay, dann das erklärt einige.
• 1:20
  Tim Pritlove: Anyway, we say hello to Meredith. Meredith Whitaker. Hello, welcome to our podcast.
• 1:26
  Meredith Whittaker: I'm so happy to be here. Thank you for having me.
• 1:29
  Linus Neumann: Meredith, you are, in case somebody doesn't know you, the president of the Signal Foundation.
• 1:37
  Meredith Whittaker: I am. I am.
• 1:38
  Linus Neumann: And Signal is a messenger application?
• 1:42
  Meredith Whittaker: It is the world's most widely used, truly private messaging application.
• 1:49
  Meredith Whittaker: I always have to put the full sentence in there. A very special messaging application.
• 1:55
  Linus Neumann: What makes it special?
• 1:55
  Meredith Whittaker: Um we build against
• 1:58
  Meredith Whittaker: the grain to put it in kind of flowery language we go out of our way to build
• 2:04
  Meredith Whittaker: for privacy and to get as close to collecting no data at all as possible which
• 2:11
  Meredith Whittaker: as you and i assume many in your audience know is a lot more work.
• 2:15
  Linus Neumann: Than simply encrypting.
• 2:18
  Meredith Whittaker: Then well encrypting is part of that but then you need to make sure the encryption
• 2:22
  Meredith Whittaker: works and then encryption of message content is Das ist nicht möglich,
• 2:26
  Meredith Whittaker: weil du also hast Meta-Data. Du hast auch libraries, du hast du in.
• 2:31
  Meredith Whittaker: Du hast Core Services, du hast du mit. So, da ist viel ingenuity required zu
• 2:36
  Meredith Whittaker: actually create a system das ist rejecting den normative assumptions der Tech-Industrie heute.
• 2:43
  Meredith Whittaker: Wir alle wollen ein ton of Daten, und wir alle wollen monetize das Daten.
• 2:47
  Meredith Whittaker: Wir wollen Sie ads mit dem Daten. Wir wollen das zu anderen Kunden.
• 2:52
  Meredith Whittaker: Wir wollen das Daten-Broker. Wir wollen das AI-Modeln mit dem Daten.
• 2:56
  Meredith Whittaker: Most of the infrastructure in the tech ecosystem now assumes that as a given.
• 3:02
  Meredith Whittaker: And then we have to rewrite things too.
• 3:04
  Linus Neumann: And you negate all these assumptions and try to build a truly private messaging
• 3:10
  Linus Neumann: platform in a world that has a completely different business model.
• 3:14
  Meredith Whittaker: Yeah. Well, I'm going to just flip that to be a little bit rhetorical and say
• 3:17
  Meredith Whittaker: those assumptions negate the human right to private intimate communication.
• 3:23
  Meredith Whittaker: And we are trying to rebuild a tech ecosystem that actually honors those.
• 3:29
  Linus Neumann: That's an even higher goal.
• 3:31
  Meredith Whittaker: Yeah.
• 3:32
  Tim Pritlove: Before we dig more into the Signal project, which in itself I think is quite
• 3:38
  Tim Pritlove: interesting, we'd like to understand how did you get there, where you are right now.
• 3:45
  Tim Pritlove: So what was your introduction into this digital world? When did it start?
• 3:53
  Meredith Whittaker: I don't have one of those stories where I'm like an Atari. I was four. I'm a hacker.
• 4:01
  Meredith Whittaker: I didn't care about tech, qua tech. I wasn't interested.
• 4:05
  Meredith Whittaker: You remember back in school, there were two kinds of nerds.
• 4:10
  Meredith Whittaker: There were the math nerds and the book nerds.
• 4:13
  Linus Neumann: You were the book nerd?
• 4:14
  Meredith Whittaker: I was the book nerd. But in the story we tell now, because nerddom has been
• 4:18
  Meredith Whittaker: overlapped with monetary success and sort of a career in math and science,
• 4:22
  Meredith Whittaker: I think we forget about the book nerds. But hello, I'm here to remind you.
• 4:26
  Meredith Whittaker: We exist. We're here. And that's who I was.
• 4:30
  Meredith Whittaker: And so I studied literature and rhetoric at Berkeley, which I just thought were,
• 4:34
  Meredith Whittaker: I mean, they're still beguiling.
• 4:36
  Meredith Whittaker: Like being able to read and write is pretty fundamental for anything.
• 4:39
  Meredith Whittaker: Und dann ich war poor so ich habe einen Job bei Google weil sie waren die ersten
• 4:43
  Meredith Whittaker: Menschen zu offeren mich ein Job und dann habe ich sehr fasziniert mit was auf
• 4:47
  Meredith Whittaker: der Erde was going on When war das?
• 4:50
  Linus Neumann: Wenn warst du bei Google als jemand der Studierende Literatur und Rhetorik?
• 4:54
  Meredith Whittaker: July 10, 2006 Ich habe mich über Berkeley und ich musste Geld und ich musste
• 5:00
  Meredith Whittaker: und ich put mein Resumé auf Monster.com which ist ein precursor zu LinkedIn,
• 5:06
  Meredith Whittaker: und ich habe mich überlegt out.
• 5:07
  Meredith Whittaker: And then I talked to my friend Michelangelo, because at the time you still needed
• 5:11
  Meredith Whittaker: to invite to join Gmail, as I recall.
• 5:14
  Meredith Whittaker: And I was like, hey, Michelangelo, can you get me an invite so I can make a
• 5:17
  Meredith Whittaker: Gmail address so I can appeal to this recruiter?
• 5:21
  Meredith Whittaker: And now my spam-filled Gmail address dates from that moment.
• 5:25
  Meredith Whittaker: So I was hired as something called a customer operations associate,
• 5:29
  Meredith Whittaker: and I didn't know what that was.
• 5:32
  Meredith Whittaker: No one knows what that is. It's just a bunch of words, But it sounded,
• 5:34
  Meredith Whittaker: I was like, that's a business job. It sounds like a business title.
• 5:38
  Meredith Whittaker: I had no idea. It was basically a customer support person who wrote technical
• 5:42
  Meredith Whittaker: documentation, who are kind of user documentation, some technical documentation,
• 5:46
  Meredith Whittaker: and answered inquiries about Google's free products.
• 5:50
  Meredith Whittaker: And I was doing that for Rightly, which was an acquisition that Google made
• 5:57
  Meredith Whittaker: that then they rebuilt to become Google Docs. Okay.
• 6:01
  Meredith Whittaker: Und das ist an der Zeit, die ganze Zeit, die Google-Businessen war, was nur Search.
• 6:06
  Meredith Whittaker: So Search war all auf der Main Campus. Es war wo die Money kam,
• 6:10
  Meredith Whittaker: und dann war das Dinky-little-Building, wo alles, Gmail, Blogger, Rightly, Reader,
• 6:19
  Meredith Whittaker: all of them sat und sie waren experimental projects. Und das war called Apps.
• 6:23
  Tim Pritlove: Did you ever have exposure to actual customers or did you just write documentation?
• 6:28
  Meredith Whittaker: Well, they would email in. I only did this for a second because I kind of figured
• 6:34
  Meredith Whittaker: out I wanted to do other things. And that's part of the story.
• 6:38
  Meredith Whittaker: There's a part of the story, actually. So I did answer inquiries sometimes.
• 6:42
  Meredith Whittaker: And there were auto responses, like hotkeys you'd use in this really janky ticketing system.
• 6:47
  Meredith Whittaker: And this is back. We didn't have laptops. There's no such thing as a smartphone.
• 6:51
  Meredith Whittaker: Our desktops were chained to the desk. Und wenn du nach Hause bist,
• 6:54
  Meredith Whittaker: wenn du nach Hause bist, wenn du nach Hause bist, dann musst du nachdenken.
• 6:57
  Meredith Whittaker: Du hast du nachdenken, das klunky Lenovo oder so.
• 6:59
  Meredith Whittaker: Ich weiß nicht genau, aber es war...
• 7:02
  Meredith Whittaker: Ich war mir, okay, ich werde rewarded in meinem Job, based auf wie viele Tickets ich close bin.
• 7:08
  Meredith Whittaker: Aber ich weiß nicht die Engenheer, die haben zu close diese Tickets,
• 7:10
  Meredith Whittaker: weil ich mich reporting bugs, und ich muss sie fix.
• 7:13
  Meredith Whittaker: Und wenn sie nicht fix die Bugs sind, dann bin ich nicht rewarded.
• 7:15
  Meredith Whittaker: Und ich war mir, das ist wirklich silly.
• 7:18
  Meredith Whittaker: So ich war über die Apps-Büte, weil ich in einer anderen Bühne bin bin.
• 7:21
  Meredith Whittaker: Und ich mit der engineering team und ich war, hey, ich bin das Person,
• 7:24
  Meredith Whittaker: die gibt es diese E-Mails, die Sie anwenden Sie.
• 7:27
  Meredith Whittaker: Und ich war able, zu convince einem einen zu gehen, zu Costco,
• 7:31
  Meredith Whittaker: die ist dieses große Warehouse-Store, ein Walmart-E-Store in den USA,
• 7:35
  Meredith Whittaker: und put ein giantes Kuchen auf seine eigene Karte, und bring es in den Apps-Bilden,
• 7:41
  Meredith Whittaker: so ich konnte sit mit dem.
• 7:42
  Meredith Whittaker: Weil wenn ich mit dem satte, dann könnte ich mit dem, und dann würde ich mich
• 7:47
  Meredith Whittaker: mehr möglich zu fixen, meine Bugs.
• 7:48
  Meredith Whittaker: Und so all my bugs started getting fixed but then my manager got really,
• 7:53
  Meredith Whittaker: upset because it was like obviously I had breached the hierarchy and I was like
• 7:57
  Meredith Whittaker: and so this meeting appeared on my calendar that was basically about me being
• 8:03
  Meredith Whittaker: insubordinate but I'm also like I didn't come from that class I had no I was
• 8:06
  Meredith Whittaker: like what the fuck did I do wrong I don't know,
• 8:08
  Meredith Whittaker: and I was like oh god I like I did something and then.
• 8:13
  Meredith Whittaker: The day before I think that meeting was going to happen, this email hit,
• 8:17
  Meredith Whittaker: we had this all, like some group, I don't know, it was like the group to the
• 8:22
  Meredith Whittaker: consumer operations team.
• 8:24
  Meredith Whittaker: And one of the engineering directors in Apps must have had like a beer or something
• 8:28
  Meredith Whittaker: during one of the many, many, many drinking parties that happened during the
• 8:31
  Meredith Whittaker: day at Google in that era.
• 8:33
  Meredith Whittaker: And sent some email that was like, Meredith, this couch is a model for collaboration.
• 8:40
  Linus Neumann: I was going to say, that's why you hired somebody from Berkeley.
• 8:44
  Linus Neumann: To write your tickets because they come up with ideas like that.
• 8:47
  Meredith Whittaker: And then that meeting just disappeared and I was like, well shit,
• 8:51
  Meredith Whittaker: I need another job because I just burned someone who's going to figure out how
• 8:54
  Meredith Whittaker: to get back at me and that's where my street sense comes in.
• 8:56
  Meredith Whittaker: So I took a bike around campus for a couple months and asked every VP during
• 9:02
  Meredith Whittaker: their office hours, which is a thing they used to have like, hey, I want another job.
• 9:05
  Meredith Whittaker: And I would just drop in and be like, hi, you don't know me. I'd like a job.
• 9:09
  Meredith Whittaker: And then I got a job doing basically Standards work, like trying to push document interop standards.
• 9:17
  Meredith Whittaker: And so I started in standards and then standards parlays into measurement pretty
• 9:21
  Meredith Whittaker: quickly because measurement is ultimately the methodological standard.
• 9:25
  Meredith Whittaker: And co-founded MLab and then that became the nucleus around which the research
• 9:30
  Meredith Whittaker: group I founded existed and then that was the...
• 9:35
  Meredith Whittaker: That was when the fresh air of sort of these political and social complexities
• 9:38
  Meredith Whittaker: started hitting my technical work. And I was like, oh, standardization is power.
• 9:43
  Meredith Whittaker: Oh, creating data means you own the narrative. Oh, shit, none of this is neutral.
• 9:47
  Meredith Whittaker: All of this is contingent.
• 9:49
  Meredith Whittaker: I started seeing the balance sheet. I started seeing the capital that was involved
• 9:52
  Meredith Whittaker: in running infrastructure.
• 9:54
  Meredith Whittaker: And I remember around the time I met you, Linus, like maybe over a decade ago.
• 9:59
  Linus Neumann: Yeah, it must have been about a decade or longer. Yeah, probably a bit longer even.
• 10:02
  Meredith Whittaker: Yeah, aging is weird.
• 10:06
  Meredith Whittaker: Aber ich habe diese schluss-Ballanz-Ballanz-Sheets. Ich habe aus Google gepritten,
• 10:12
  Meredith Whittaker: die gezeigt, wie viel die Uplink-Bandwidth und die Infrastruktur und Power-Costs für MLAB war.
• 10:19
  Meredith Whittaker: Und ich würde das zeigen, das zu all diese Civil Society funders,
• 10:22
  Meredith Whittaker: wie gesagt, Sie können 250k verdienen.
• 10:25
  Meredith Whittaker: Es kostet 40 Millionen a Jahr in Bandwidth, nur zu MLAB.
• 10:29
  Meredith Whittaker: Du wirst nicht verstehen, die economics of this, denn wir sind in der Era der
• 10:32
  Meredith Whittaker: Civic-Technik, All we need is a good idea, right?
• 10:36
  Meredith Whittaker: And I think I was really lucky to get sensitized to the political economy and
• 10:41
  Meredith Whittaker: the fact that we're talking about infrastructure,
• 10:43
  Meredith Whittaker: capital, network effects, economies of scale and not some kind of brilliant
• 10:49
  Meredith Whittaker: idea that just ephemerally transformed our world,
• 10:54
  Meredith Whittaker: and that our side just needs to wait to have one of those to get our turn.
• 10:58
  Tim Pritlove: That sounds like a quick upgrade from a customer support person.
• 11:02
  Tim Pritlove: Yeah, yeah, yeah. So, you mentioned MLab, which stands for Measurement Lab.
• 11:08
  Meredith Whittaker: It does, yeah.
• 11:09
  Tim Pritlove: Can you explain what this is all about and how it came alive and why you were involved?
• 11:13
  Meredith Whittaker: Yeah. It was a project that was Derek Slater, Vince Cerf, Stephen Stewart,
• 11:22
  Meredith Whittaker: Sasha Meinrath at Open Tech Institute.
• 11:29
  Tim Pritlove: The elders of the internet.
• 11:30
  Meredith Whittaker: Yeah, some old internet guys and me.
• 11:35
  Linus Neumann: Let's say the elders of the internet.
• 11:38
  Tim Pritlove: You're one of them now.
• 11:41
  Meredith Whittaker: And the lady with the couch. And the conceit there, which felt really simple
• 11:47
  Meredith Whittaker: to me at the time, is everyone is buzzing about net neutrality.
• 11:51
  Meredith Whittaker: And I was a Kool-Aid drinker. I still think the value underlying that kind of
• 11:56
  Meredith Whittaker: mythology, let's say, like, yes, we should not have one gatekeeper deciding
• 12:02
  Meredith Whittaker: which news source, right? This is old school common sense.
• 12:06
  Meredith Whittaker: It goes back to Western Union, who there's a little, you know,
• 12:09
  Meredith Whittaker: in the US would refuse to carry telegrams from political candidates the company didn't support, right?
• 12:15
  Meredith Whittaker: So there's a real bedrock precedent here.
• 12:19
  Meredith Whittaker: And I was like, yeah, of course we need net neutrality.
• 12:23
  Meredith Whittaker: Neutrality itself is a really wafty, it's a loose concept.
• 12:27
  Meredith Whittaker: It needs to be augured in some type of benchmark against which we can assess is this neutral or not.
• 12:32
  Tim Pritlove: You need to quantify it.
• 12:33
  Meredith Whittaker: Yes. And then you just flung into the abyss of philosophy the second you try to do that.
• 12:39
  Meredith Whittaker: But we started and I would say it was not necessarily built to succeed.
• 12:45
  Meredith Whittaker: It was built as a sort of hypothesis project where
• 12:49
  Meredith Whittaker: we stood up three servers that hosted
• 12:52
  Meredith Whittaker: open source measurement clients and the thing
• 12:56
  Meredith Whittaker: that we were doing was putting servers that were all configured the same that
• 13:02
  Meredith Whittaker: gave each client a dedicated slice of resources and then way over-provisioned
• 13:08
  Meredith Whittaker: the uplink between the server and the switch so that we could all,
• 13:15
  Meredith Whittaker: for all intents and purposes, guarantee that any artifacts that were detected
• 13:20
  Meredith Whittaker: through the measurement methodology like some TCP,
• 13:24
  Meredith Whittaker: RTT or something, were not interfered with by our infrastructure and weren't
• 13:29
  Meredith Whittaker: suffering for bandwidth.
• 13:31
  Tim Pritlove: So for practical reasons you had unlimited bandwidth and zero latency to everywhere.
• 13:36
  Meredith Whittaker: Yeah, and let's, you know, and that's very expensive. But then,
• 13:42
  Meredith Whittaker: we immediately DDoS the servers or like, oh shoot, because Vint's put a blog
• 13:47
  Meredith Whittaker: post out and he was you know he's a big name,
• 13:50
  Meredith Whittaker: and then it was like this job of years and years of getting servers in different
• 13:55
  Meredith Whittaker: interconnection points because we wanted to measure across consumer you know
• 13:59
  Meredith Whittaker: we're not just measuring to a server in the network exactly and we need to cross
• 14:02
  Meredith Whittaker: interconnection points to do that because that's where you begin to see interesting
• 14:06
  Meredith Whittaker: business relationships and kind of feuds,
• 14:09
  Meredith Whittaker: and it was just it was very it taught me about like how difficult and contingent and,
• 14:15
  Meredith Whittaker: ultimately subjective, creating data is, and the political process of then sort
• 14:23
  Meredith Whittaker: of defending that data as a reliable proxy for reality.
• 14:27
  Meredith Whittaker: Because I would be at the Federal Communications Commission and I had Comcast
• 14:31
  Meredith Whittaker: across from me being like, no, we want to measure it multi-threaded TCP versus
• 14:37
  Meredith Whittaker: single-threaded because you get a, let's just say, higher number that way.
• 14:41
  Meredith Whittaker: It's more forgiving. and we would be sort of defending the methodology of our
• 14:45
  Meredith Whittaker: tests and the openness principles.
• 14:48
  Meredith Whittaker: So it was open source code, the database architecture was open.
• 14:53
  Meredith Whittaker: I don't remember all the server architecture, it was old school,
• 14:59
  Meredith Whittaker: old school ideological.
• 15:00
  Tim Pritlove: You really don't sound like a book nerd now. You've become a real network nerd.
• 15:05
  Meredith Whittaker: Let's just say book nerds read and network stuff is written down.
• 15:13
  Meredith Whittaker: I read the fucking menu of it. I'm like, no one told me I was allowed to not
• 15:19
  Meredith Whittaker: read it and still have an opinion.
• 15:24
  Tim Pritlove: I love networking.
• 15:26
  Meredith Whittaker: It taught me everything about so much because you get down to the network layer
• 15:31
  Meredith Whittaker: and you're like, fuck, everything is broken. It's all tied together with string.
• 15:35
  Meredith Whittaker: What's going on? Okay, so your promises in the pitch meeting are guaranteed by this.
• 15:42
  Meredith Whittaker: This crusty old library that's maintained by a guy on a boat with a BitLocker
• 15:47
  Meredith Whittaker: account. Like, what is going on?
• 15:51
  Tim Pritlove: It's scary, isn't it?
• 15:53
  Meredith Whittaker: It is. I mean, it's a catalyst into today for my work.
• 15:59
  Linus Neumann: You did that for, I think, a couple of years, the MLab stuff.
• 16:03
  Linus Neumann: And you also, at this point in time, I think, began, you know, the ties to the,
• 16:09
  Linus Neumann: Let's say broader internet freedom community. I remember Uniprobe at the time, I think, was another.
• 16:17
  Linus Neumann: They still exist, right? They do.
• 16:19
  Meredith Whittaker: And we actually talked to them at Signal. They'll detect signal blocking.
• 16:24
  Meredith Whittaker: And I believe, so one of the things MLab did was provide backend infrastructure
• 16:29
  Meredith Whittaker: for projects like Uniprobe.
• 16:31
  Meredith Whittaker: And I don't actually know what that relationship is today.
• 16:35
  Meredith Whittaker: So academics and hackers and developers could write a test or kind of a measurement
• 16:42
  Meredith Whittaker: methodology deploy a client,
• 16:45
  Meredith Whittaker: to consumers like you test from your laptop or whatever and then we would be
• 16:49
  Meredith Whittaker: the ones paying for the bandwidth and infrastructure that would allow that to scale Uni.
• 16:54
  Linus Neumann: Is the Open Observatory of Network Interference.
• 16:58
  Meredith Whittaker: If I remember the acronym Hi Arturo So...
• 17:04
  Linus Neumann: Disclosing a whole social network here voluntarily. So, but eventually something
• 17:10
  Linus Neumann: happened that is known as the Google Walkouts.
• 17:14
  Linus Neumann: Was that following your MLab activities or during your MLab activities or did
• 17:19
  Linus Neumann: you rise even higher in the Google hierarchies before that happened?
• 17:25
  Meredith Whittaker: Well, I mean, there are many. I think, so...
• 17:30
  Meredith Whittaker: I was not working on MLAB by that point.
• 17:34
  Meredith Whittaker: MLAB, there's a part of this story that I didn't cover, which is around,
• 17:40
  Meredith Whittaker: I don't know, 2014, 2015,
• 17:45
  Meredith Whittaker: we started seeing really interesting artifacts in the data,
• 17:48
  Meredith Whittaker: which showed essentially that at particular interconnection points between particular
• 17:55
  Meredith Whittaker: telcos, we were seeing drastic drops in performance.
• 18:00
  Meredith Whittaker: And what we were able to do was look at, you know, every intersection of Telco
• 18:06
  Meredith Whittaker: 1 and Telco 2, we see this drop.
• 18:10
  Meredith Whittaker: Intersections of Telco 3 and Telco 2 don't see this drop.
• 18:14
  Meredith Whittaker: And so what we're able to do is say, like, there's actually a business feud
• 18:17
  Meredith Whittaker: going on. And the interconnection point is the locus of that feud.
• 18:22
  Meredith Whittaker: And using traceroute data, we were able to say, like, look, these aren't,
• 18:24
  Meredith Whittaker: it's not just that they're sharing a path in some other region of the network that's slowing it down.
• 18:30
  Linus Neumann: They're actively throttling.
• 18:32
  Meredith Whittaker: They're actively throttling. And we put together a report on that.
• 18:37
  Meredith Whittaker: And this is where I was sort of, I think, kind of sideloading myself into some
• 18:42
  Meredith Whittaker: academic research work. I was like, okay, how do we do this?
• 18:45
  Meredith Whittaker: Document our methodology. Document sort of everything was open.
• 18:49
  Meredith Whittaker: So we pulled that together.
• 18:51
  Meredith Whittaker: And it showed, I don't know if you remember, there was a sort of Comcast Cogent Netflix.
• 18:56
  Linus Neumann: Oh, yeah.
• 18:57
  Meredith Whittaker: Netflix was shoving all its traffic through Cogent. What we had done is expose
• 19:01
  Meredith Whittaker: that and exposing that it exposed the principle that you have to if you wanted
• 19:05
  Meredith Whittaker: to ensure net neutrality, you have to take the interconnection points and the
• 19:08
  Meredith Whittaker: interconnection agreements into account.
• 19:11
  Meredith Whittaker: And that led Obama to add interconnection to the reclassification under Title
• 19:16
  Meredith Whittaker: II of these, you know, which kind of moved toward net neutrality that was nullified.
• 19:21
  Meredith Whittaker: But, you know, that was kind of the swan song, let's say, for my MLab time,
• 19:27
  Meredith Whittaker: because, of course, that was a huge deal, right?
• 19:29
  Meredith Whittaker: Like, that's where the business model rubber hits the road.
• 19:33
  Meredith Whittaker: And, you know, I'll just shorthand it. Like, that bought me a lot of capital at Google.
• 19:37
  Meredith Whittaker: And with that, I was, you know, I was already interested in a lot of things.
• 19:41
  Meredith Whittaker: MLab was sort of humming along and, like, it had grown from a hypothesis project
• 19:44
  Meredith Whittaker: to, like, a global thing that was working and doing stuff.
• 19:48
  Meredith Whittaker: And I started getting agitated by AI and a lot of these privacy and security
• 19:53
  Meredith Whittaker: concerns, being part of the community that you were part of,
• 19:57
  Meredith Whittaker: you know, kind of thinking around like
• 19:58
  Meredith Whittaker: tech alternatives and getting less comfortable with the business model.
• 20:03
  Meredith Whittaker: And from there I always had like eight different projects going on but I went
• 20:07
  Meredith Whittaker: on to found, to co-found the AI Now Institute which was really trying to bring the conversation,
• 20:14
  Meredith Whittaker: on what we called machine learning back then but like AI was like a flashier
• 20:18
  Meredith Whittaker: term, like bring it down to the ground a little bit and stop talking about superintelligence
• 20:23
  Meredith Whittaker: and stop talking about, you know,
• 20:25
  Meredith Whittaker: Political economy, what are these technologies, how are they being deployed,
• 20:30
  Meredith Whittaker: how do you oversee them, who uses them on whom, and what are the social and
• 20:34
  Meredith Whittaker: political dynamics of that.
• 20:36
  Linus Neumann: But the AI Now Institute is not hosted at Google, is it? No.
• 20:41
  Linus Neumann: Which university is it again?
• 20:43
  Meredith Whittaker: It was at NYU, and it's moved out of NYU for reasons of that being easier.
• 20:48
  Linus Neumann: Okay.
• 20:49
  Meredith Whittaker: And cheaper, because they take like 40%, those universities.
• 20:53
  Linus Neumann: Okay.
• 20:54
  Meredith Whittaker: Word to the wise, your money.
• 20:59
  Linus Neumann: What do you think?
• 21:00
  Tim Pritlove: Like whatever donations you get. Okay.
• 21:03
  Meredith Whittaker: Yeah. So it's an expensive brand.
• 21:10
  Linus Neumann: Well, for a reason, right? I mean, NYU Institute, that's something to have, I think, co-founded.
• 21:17
  Meredith Whittaker: Co-founded, yeah. I mean, the work was really good. And this was,
• 21:22
  Meredith Whittaker: again, gaining the capital at Google, cementing a reputation,
• 21:26
  Meredith Whittaker: being able to get to a level where I had a budget.
• 21:28
  Meredith Whittaker: And then part of what I was always trying to do is how much can I pull out of
• 21:33
  Meredith Whittaker: Google and get into the hands of people doing work that I think is cool.
• 21:38
  Meredith Whittaker: How do we carve tributaries in the massive river of this huge,
• 21:42
  Meredith Whittaker: rich company and get it out?
• 21:46
  Tim Pritlove: So you started that institute at the end of 2017.
• 21:51
  Meredith Whittaker: Well, beginnings are sometimes hard to date.
• 21:55
  Meredith Whittaker: It was born out of a request from the Obama administration to be the host of one of his AI summits.
• 22:03
  Meredith Whittaker: I don't remember exactly the contours there, but it was in 2016.
• 22:07
  Meredith Whittaker: And so it started as like, like the idea then was like, let's do this and let's do it big.
• 22:13
  Meredith Whittaker: Like, let's make this the most polished, flashiest, like hard to ignore kind
• 22:18
  Meredith Whittaker: of spectacle using all the tools we can get from like hiring an events agency,
• 22:23
  Meredith Whittaker: doing good press, all of that.
• 22:24
  Meredith Whittaker: But let's also make this the one that is the most engaged with these political topics,
• 22:31
  Meredith Whittaker: that's actually forcing the debate in that direction and kind of making it face
• 22:36
  Meredith Whittaker: these questions that are much more grounded and, you know,
• 22:40
  Meredith Whittaker: hopefully much more healthy for our population.
• 22:44
  Meredith Whittaker: So it was very successful and from there we got offers of funding and a lot
• 22:50
  Meredith Whittaker: of encouragement and the work just kept going.
• 22:53
  Tim Pritlove: Well, we in Europe, we are used to be a bit behind, but if I can recall 2016,
• 23:00
  Tim Pritlove: there was no discussion about AI in Europe at all.
• 23:04
  Tim Pritlove: And so, it's quite interesting to see that Obama has actually,
• 23:10
  Tim Pritlove: you know, decided that it's finally time to do something about this topic nobody
• 23:16
  Tim Pritlove: has ever really talked about.
• 23:18
  Linus Neumann: Germans are currently making up their mind whether that is a topic.
• 23:21
  Tim Pritlove: Ja, wir sprechen viel über Europa heute.
• 23:25
  Tim Pritlove: Aber mein Frage ist, kann man describe was kind of discussions going on 10 Jahre
• 23:33
  Tim Pritlove: ago in American society that this has come up to be something of a topic for the near future,
• 23:41
  Tim Pritlove: which actually became a big one.
• 23:45
  Tim Pritlove: Where did this discussion take place?
• 23:49
  Meredith Whittaker: Yeah, to answer this question, I'm going to be drawing on a lot of the research
• 23:54
  Meredith Whittaker: and the historical work, the work I've done since then.
• 23:58
  Meredith Whittaker: Because when this dawned in my life, when it started being a thing,
• 24:03
  Meredith Whittaker: I had basically that same question. What is this stuff?
• 24:07
  Meredith Whittaker: Why is it kind of... At Google, you would see...
• 24:12
  Meredith Whittaker: A shift toward a new paradigm or a new trend by the incentives that were structured
• 24:18
  Meredith Whittaker: into the OKRs, the quarterly goals,
• 24:21
  Meredith Whittaker: that kind of, you know, there'd be all these training modules that would pop
• 24:23
  Meredith Whittaker: up and it's like make your software engineer into an AI developer,
• 24:28
  Meredith Whittaker: you know, and you'd be like, there's an incentive here.
• 24:30
  Linus Neumann: Seems like they weren't too successful though, right? I mean,
• 24:32
  Linus Neumann: that's not like the biggest AI wave at Google.
• 24:37
  Meredith Whittaker: Well, they were, that was DeepMind era. So, okay, okay, good.
• 24:40
  Linus Neumann: Sorry.
• 24:40
  Meredith Whittaker: I think they were ahead. It was them and Meta for a long time. And the history...
• 24:46
  Linus Neumann: Didn't make it to the business though at the time, right?
• 24:48
  Meredith Whittaker: No, but they're chaotic. It's a kind of court in decline. So that,
• 24:52
  Meredith Whittaker: you know, actually where the... Actually the business model part has never been
• 24:56
  Meredith Whittaker: their strong suit beyond search.
• 24:58
  Meredith Whittaker: To be real, you know, like cloud is like the best technology presented confusingly
• 25:03
  Meredith Whittaker: with 18 versions all deprecated, right? Like that's... Um,
• 25:09
  Meredith Whittaker: um, but this, the, the AI stuff was, so if you look at the, if you look at the
• 25:16
  Meredith Whittaker: sort of recent history, which is something I'm spending, I've spent a lot of time on.
• 25:19
  Meredith Whittaker: I spend a lot of time on because I think we, you know, that gives us a really
• 25:22
  Meredith Whittaker: different picture than the Elon Musk narrative or the kind of popular narrative.
• 25:28
  Meredith Whittaker: There's a very important paper that was published in 2012 that introduced the
• 25:33
  Meredith Whittaker: AlexNet algorithm. und das war Jeff Hinton und his students, you may recognize.
• 25:39
  Tim Pritlove: Who just got the Nobel Prize. Physics, interestingly enough.
• 25:43
  Linus Neumann: Yeah, because there is no AI.
• 25:45
  Tim Pritlove: Yeah, because there is no computer Nobel Prize.
• 25:48
  Meredith Whittaker: Well, if you claim that your technology is everything, then you can get a prize in anything.
• 25:56
  Tim Pritlove: Which is a proven point now, yeah.
• 25:59
  Meredith Whittaker: And it was Ilya Stutzkever and then Alex. I'm sorry, Alex, I'm not grabbing
• 26:05
  Meredith Whittaker: your last name from the ether right now.
• 26:08
  Meredith Whittaker: But nonetheless, this was a paper that kind of pulled together key ingredients
• 26:12
  Meredith Whittaker: that became the foundation of the AI boom now.
• 26:16
  Meredith Whittaker: So this is deep learning algorithms, which is the paradigm we're still in.
• 26:20
  Meredith Whittaker: It doesn't matter. There's architectural sort of rejiggering,
• 26:23
  Meredith Whittaker: but nonetheless, it's deep learning.
• 26:26
  Meredith Whittaker: Huge amounts of data. So what I've called the derivatives of the surveillance business model.
• 26:31
  Tim Pritlove: He found all the cats on YouTube.
• 26:34
  Meredith Whittaker: I mean, yeah, that was Jeff Dean, I think.
• 26:38
  Meredith Whittaker: And then, you know, powerful compute, right?
• 26:41
  Meredith Whittaker: And they showed that sort of using gaming chips and a lot of data,
• 26:45
  Meredith Whittaker: you could beat the benchmark.
• 26:47
  Meredith Whittaker: So score much better against standard evaluations than past models and thus
• 26:53
  Meredith Whittaker: sort of catalyzed the industrial industry interest in AI and why were they interested?
• 26:59
  Meredith Whittaker: I think this is a key point because these optimist-seeking algorithms are really
• 27:06
  Meredith Whittaker: good at curating news feeds.
• 27:09
  Meredith Whittaker: They're really good at figuring out algorithms, right?
• 27:14
  Meredith Whittaker: I don't think it's an accident that Jeff was immediately hired,
• 27:20
  Meredith Whittaker: dass Jan LeCun, die die deep-learning-Algorithmen, die das Seed der Moment in
• 27:28
  Meredith Whittaker: AI in den 80ern, in den letzten 80ern, war er mit Meta.
• 27:32
  Meredith Whittaker: Und es war die Plattform-Companien mit der realen Investition in squeezing mehr
• 27:37
  Meredith Whittaker: Ad-Dollars aus der Daten, better-serving, all of das.
• 27:45
  Meredith Whittaker: Das ist Google mit DeepMind. Du sehen Meta und Google die leaders in das.
• 27:52
  Meredith Whittaker: Als measured by different evaluation standards, the measurement question hier
• 27:57
  Meredith Whittaker: ist eigentlich wirklich interesting und troubling.
• 27:59
  Meredith Whittaker: Until this generative moment where I think the ChatGPT Microsoft products shifted
• 28:06
  Meredith Whittaker: people's perception of AI and what it can do and just rearrange the leaderboard.
• 28:12
  Meredith Whittaker: But the paradigm is still the same And the paradigm is still that AI is applying
• 28:16
  Meredith Whittaker: old algorithms on top of the sort of massive platform monopoly business model,
• 28:22
  Meredith Whittaker: availing itself of huge amounts of data, which is produced via this surveillance
• 28:27
  Meredith Whittaker: business model, and really powerful compute that was designed,
• 28:32
  Meredith Whittaker: built up, you know, I would say, like, consolidated in the hands of these platform
• 28:38
  Meredith Whittaker: companies via the imperatives of the surveillance business model, right?
• 28:43
  Linus Neumann: Would you say that, so clearly, I mean, you say,
• 28:48
  Linus Neumann: so here are Google and Meta that have these massive amounts of data,
• 28:53
  Linus Neumann: like larger amounts of data than probably ever existed or were in the hands
• 28:57
  Linus Neumann: of anybody or any organization.
• 28:59
  Linus Neumann: Would you say that they had amassed all this data and eventually learned,
• 29:07
  Linus Neumann: okay, we probably can't handle this anymore, so we're interested in this new
• 29:11
  Linus Neumann: paradigm to even monetize this data any further?
• 29:14
  Linus Neumann: Or was it rather like, oh, we have all this data, we're monetizing it well,
• 29:18
  Linus Neumann: here's another way to monetize it on top of that. because clearly these deep
• 29:25
  Linus Neumann: learning algorithms are not of much use if you don't have large data.
• 29:29
  Meredith Whittaker: Set and large compute for both training and inference but is.
• 29:34
  Linus Neumann: It the hen egg what's the hen what's the egg there.
• 29:37
  Meredith Whittaker: I think this
• 29:40
  Meredith Whittaker: is almost a perpetual motion machine right like
• 29:43
  Meredith Whittaker: every quarter you have to report progress you have to report growth the logic
• 29:49
  Meredith Whittaker: is metastasis and so you know you're trying to squeeze more out of what you
• 29:55
  Meredith Whittaker: have and you're trying to get more of what you have so you can squeeze more
• 29:57
  Meredith Whittaker: right there's also these sort of laws of scale remember big data we used to call it that,
• 30:04
  Meredith Whittaker: and so I don't I don't actually know the answer to your question but I think it,
• 30:10
  Meredith Whittaker: Which came first? But, well, I mean, the business model came first, right?
• 30:17
  Meredith Whittaker: You had to have the ingredients to know what they did together.
• 30:20
  Meredith Whittaker: And I think it was deep learning and AI had sort of languished in the backwater
• 30:25
  Meredith Whittaker: with some interesting experiments through the 2000s because its history is always
• 30:33
  Meredith Whittaker: promising too much and disappointing since the mid-50s when it was invented.
• 30:38
  Meredith Whittaker: And I think the goal was really to sort of supercharge their existing business model.
• 30:45
  Tim Pritlove: I think deep learning was just the technology that perfectly served their current
• 30:50
  Tim Pritlove: beliefs in that they have to work on the data,
• 30:53
  Tim Pritlove: that they have to build up algorithms to somehow predict your personal future
• 30:58
  Tim Pritlove: and be there with an ad before you even know it and we've seen this,
• 31:05
  Tim Pritlove: as an ad everywhere and we've also seen it in political influence as we've seen in the Brexit,
• 31:13
  Tim Pritlove: decision and also in the elections where we heard there's going to be one in
• 31:20
  Tim Pritlove: the near future as well in the US that might be influenced as well.
• 31:24
  Meredith Whittaker: Let me check my calendar.
• 31:27
  Meredith Whittaker: I mean I think we can like peel back also just this concept of like what is
• 31:32
  Meredith Whittaker: an advertisement right? It's an influence.
• 31:35
  Meredith Whittaker: Is it trying to get you to buy something? Is it trying to get you to like something?
• 31:38
  Meredith Whittaker: Is it trying to get you to believe something? Is it trying to get you to vote a certain way?
• 31:42
  Meredith Whittaker: And I think that the term advertisement is usefully deconstructed when we start
• 31:48
  Meredith Whittaker: to think about the connection between all of those.
• 31:52
  Linus Neumann: The term advertising is one of the best tricks the devil ever pulled.
• 31:58
  Linus Neumann: Because it's behavior manipulation. It's a stated goal. But I remember when.
• 32:02
  Linus Neumann: The first debates came up about, for example, the Cambridge Analytica scan.
• 32:08
  Linus Neumann: It's like, they used Facebook data to manipulate voters.
• 32:12
  Linus Neumann: And it's like, no, there's a red line over here. You can't manipulate behavior.
• 32:17
  Linus Neumann: I mean, Facebook and Google were basically built to manipulate users' behavior
• 32:23
  Linus Neumann: and ideally capture their attention to change their behavior.
• 32:28
  Linus Neumann: Wir haben wahrscheinlich dieses shirt oder das oder das oder das oder das oder
• 32:31
  Linus Neumann: das oder das oder das oder das oder das oder das oder das oder das oder das oder das ist.
• 32:35
  Linus Neumann: Wie können wir das so etwas machen? Das ist unheard of.
• 32:40
  Linus Neumann: So viewing advertising as like, oh, this is just an offer.
• 32:45
  Linus Neumann: This is, oh, we're just making our product. No, no, I have a limited amount of attention per day.
• 32:49
  Linus Neumann: You're capturing it and you're doing it with one simple reason.
• 32:53
  Linus Neumann: And that is changing my behavior in the sense that you aim for.
• 32:59
  Linus Neumann: And I guess eventually it sounds like,
• 33:03
  Linus Neumann: eventually you questioned your Google career, right? Because you were,
• 33:06
  Linus Neumann: I mean, clearly you had an impressive career there in just a few years.
• 33:10
  Linus Neumann: Um, but you began to question not only your, like Google and the company culture,
• 33:18
  Linus Neumann: but apparently also a little bit your,
• 33:20
  Linus Neumann: your, the way you want to continue with your, with your, you know,
• 33:29
  Linus Neumann: making use of the influence and the knowledge you have.
• 33:32
  Meredith Whittaker: Yeah. I think.
• 33:34
  Linus Neumann: Is that the walkouts?
• 33:35
  Meredith Whittaker: I mean, it's all interlinked and kind of periodizing your own consciousness is hard.
• 33:46
  Meredith Whittaker: But I think I'm pretty earnest and I also don't come from that world,
• 33:52
  Meredith Whittaker: I don't come from that class so there are often places where I just didn't,
• 33:55
  Meredith Whittaker: you know, I would take things sincerely or be like really committed and then only realize like
• 34:01
  Meredith Whittaker: two thirds of the way through whatever it was like oh no one else really cares
• 34:04
  Meredith Whittaker: about this they're just networking or you know whatever it was so I think there
• 34:08
  Meredith Whittaker: was an element there where like you know when I was doing MLab I was like I
• 34:11
  Meredith Whittaker: really want to win net neutrality Und dann we won net neutrality.
• 34:15
  Meredith Whittaker: Aber dann ist das, dass ich das nicht der Fall war.
• 34:18
  Meredith Whittaker: Google hat eine größere Netzwerke als Comcast. Das ist nicht die Gatekeeper.
• 34:22
  Meredith Whittaker: Aber das war eine Sincere Sache.
• 34:25
  Meredith Whittaker: Und dann war ich, okay, ich kann mich,
• 34:27
  Meredith Whittaker: um, Geld zu all diese coolen Privatsch-Hacker-Projekte. Das war Sincere.
• 34:33
  Meredith Whittaker: Und dann wurde ich in AI und ich war, okay, und ich denke, das ist etwas,
• 34:37
  Meredith Whittaker: das hat sich verändert für mich.
• 34:40
  Meredith Whittaker: I think I used to have a lot, lot more faith in the power of ideas to influence real change, right?
• 34:49
  Meredith Whittaker: And I still think, you know, I spend a lot of time in kind of thinking through
• 34:52
  Meredith Whittaker: discourses, how do we shape them?
• 34:54
  Meredith Whittaker: Like, how do we kindly walk people into understanding things that,
• 34:59
  Meredith Whittaker: you know, they may have an interest in not understanding or they may have been,
• 35:03
  Meredith Whittaker: you know, misinformed about or what have you.
• 35:06
  Meredith Whittaker: But I began you know I began around the time I was looking at AI and sort of making all these cases,
• 35:13
  Meredith Whittaker: that everyone loved right like I was out there giving talks that were completely
• 35:18
  Meredith Whittaker: against the Google party line and I was getting like applauses,
• 35:20
  Meredith Whittaker: I was getting promoted like I was like this is a this is a perfect job.
• 35:29
  Meredith Whittaker: And then you know like I was I envied you not only once I'm the house troll,
• 35:39
  Meredith Whittaker: but then there was I was getting more influence so I was becoming known outside
• 35:42
  Meredith Whittaker: and inside I was like the person you'd call into your team when it was like
• 35:46
  Meredith Whittaker: oh we want to implement this is there an ethical way to do it,
• 35:50
  Meredith Whittaker: and you would say no I would, I would be like my dear friends let us sit down,
• 36:00
  Meredith Whittaker: and then I was getting, I don't know, that was sort of my life and we kind of
• 36:08
  Meredith Whittaker: took the AI Now Institute and really did a lot to reshape the debate.
• 36:13
  Meredith Whittaker: I was very focused on that discursive intervention and how do we begin to talk
• 36:16
  Meredith Whittaker: about AI in a more realistic way and that was working outside of Google but
• 36:21
  Meredith Whittaker: it wasn't really influencing core decisions at Google and that was kind of the
• 36:26
  Meredith Whittaker: thing I kept hitting up against more and more strongly until I got a signal message in...
• 36:33
  Meredith Whittaker: Late 2017.
• 36:34
  Linus Neumann: From Moxie?
• 36:36
  Meredith Whittaker: No, I mean, I probably did get one from Moxie at that time, but this one was not that one.
• 36:41
  Meredith Whittaker: It was from a friend of mine at Google who said, yo, there's a really disturbing
• 36:46
  Meredith Whittaker: project that is hidden that I'm very close to.
• 36:50
  Meredith Whittaker: And you should know about it because you're the AI person who cares about this
• 36:55
  Meredith Whittaker: and you have standing at the company around it.
• 36:57
  Meredith Whittaker: And this was the secretive contract that Google signed with the Department of
• 37:01
  Meredith Whittaker: Defense to build AI drone targeting and surveillance systems for the drone war.
• 37:07
  Meredith Whittaker: And of course, like I was politicized post 9-11, post Snowden.
• 37:11
  Meredith Whittaker: Like this was, you know, the drone war and the signature strike and all of that
• 37:15
  Meredith Whittaker: were really core in kind of my, you know, things that I ideologically rejected
• 37:20
  Meredith Whittaker: and, you know, felt like we needed to disarm, not supercharge.
• 37:26
  Meredith Whittaker: And I had like a, you know, like a righteous anger. I was just like, fuck this.
• 37:32
  Linus Neumann: Because you're there running around, you know, trying to shape the discourse on AI.
• 37:38
  Meredith Whittaker: And making them look good, right? Because they get to be like,
• 37:41
  Meredith Whittaker: look, we platform such heterodox voices. We're surely benevolent, right?
• 37:46
  Meredith Whittaker: And then I'm like, okay, and then you're inking this deal with the DoD behind
• 37:50
  Meredith Whittaker: the scenes for technology that one we know doesn't sort of work for the purpose, right?
• 37:55
  Meredith Whittaker: Like, you know, it's not going to better identify a worthy
• 37:58
  Meredith Whittaker: target of death oder whatever the fuck it's you know it's not you
• 38:01
  Meredith Whittaker: know we know this is bullshit but like and you
• 38:05
  Meredith Whittaker: know this is a multinational company like more than
• 38:08
  Meredith Whittaker: half the employees are outside the US there is an issue with yoking yourself
• 38:11
  Meredith Whittaker: to you know not that you have one nation's government not that many tech companies
• 38:17
  Meredith Whittaker: care that much about that and then there was just the you know what is the structural
• 38:22
  Meredith Whittaker: danger which is deeply acute and,
• 38:26
  Meredith Whittaker: of a massive surveillance company with more data than the world has ever seen,
• 38:31
  Meredith Whittaker: more compromised than you can imagine, like yoking their fortunes and a key
• 38:35
  Meredith Whittaker: dependency to the US military, right?
• 38:40
  Meredith Whittaker: And, you know, we know from Snowden that that's already like,
• 38:43
  Meredith Whittaker: you know, seen as, you know, as long as it's a corporation gathering it, it's not, you know.
• 38:47
  Tim Pritlove: Was Google still running under the motto of don't be evil at the time?
• 38:51
  Meredith Whittaker: They were, they were. And that was, we marshaled that actually.
• 38:54
  Tim Pritlove: Kind of like the end of it, wasn't it?
• 38:58
  Meredith Whittaker: They quietly, like the lawyers removed that from the Google Manifesto.
• 39:04
  Meredith Whittaker: It was slowly fading. Yeah, it was like, they were like, just don't open that closet.
• 39:08
  Tim Pritlove: Which motto? I don't recall any. It's like, do you know something about this?
• 39:13
  Meredith Whittaker: Try not to be bad is the new motto.
• 39:18
  Tim Pritlove: Don't be as bad as that.
• 39:20
  Meredith Whittaker: Yeah, like lay off the evil.
• 39:25
  Meredith Whittaker: Ja, so I mean, and this was, there were like a lot of old school people there
• 39:29
  Meredith Whittaker: at that time who really did drink the Kool-Aid and so it was you know,
• 39:33
  Meredith Whittaker: I just put my energy into organizing against that and that was when I turned
• 39:37
  Meredith Whittaker: toward labor organizing and thinking through traditional,
• 39:42
  Meredith Whittaker: methods and approaches to combating that type of corporate power or you know
• 39:48
  Meredith Whittaker: industrial power and we, you know, and that That was the on-ramp to the walkout.
• 39:54
  Meredith Whittaker: So the walkout was like a big, that was like a rupture, like a manifestation.
• 39:58
  Meredith Whittaker: It got a lot of press and it was the biggest labor action in tech.
• 40:02
  Tim Pritlove: So just to make that clear, the walkouts were actually people walking outside
• 40:07
  Tim Pritlove: of the Google buildings for, I don't know, how long?
• 40:12
  Meredith Whittaker: I think it was November 11th, 2018.
• 40:18
  Meredith Whittaker: Und everyone walked out for 20 minutes at 11.11am in their local time so we
• 40:25
  Meredith Whittaker: called it Rolling Thunder and it started in the Singapore office as I was going
• 40:30
  Meredith Whittaker: to bed in New York and I was seeing the photos,
• 40:34
  Meredith Whittaker: and this was chaos I hadn't slept in days there's so many meetings there's so
• 40:38
  Meredith Whittaker: many tears it's hard to organize something like that,
• 40:42
  Meredith Whittaker: and I remember going to bed and seeing the images from Singapore mit ein paar
• 40:48
  Meredith Whittaker: hundert Leuten in Singapur und ich war,
• 40:54
  Meredith Whittaker: das ist großartig.
• 40:56
  Meredith Whittaker: Und dann habe ich New York time gewohnt, um 5 a.m. zu gehen in New York zu unserer
• 40:59
  Meredith Whittaker: location und prepen es, um, dass wir uns da sind.
• 41:06
  Meredith Whittaker: And then I just remember seeing, like, there was this little park near the New
• 41:08
  Meredith Whittaker: York office, and then it just, like, grew outside the park.
• 41:12
  Meredith Whittaker: And then no one could get into the park. And then I was looking at my home,
• 41:15
  Meredith Whittaker: and there's my phone, and there's live helicopter feeds.
• 41:18
  Meredith Whittaker: And we don't have bullhorns, because I'm one of the speakers.
• 41:21
  Meredith Whittaker: There's, like, speakers standing on chairs to address the crowd.
• 41:25
  Meredith Whittaker: And then this guy, like, some, you know, there's this sort of type.
• 41:29
  Meredith Whittaker: I don't know if in Germany you have this type, but they're, like,
• 41:31
  Meredith Whittaker: kind of, like, the leftist at every protest.
• 41:34
  Meredith Whittaker: And like some guy had like found out about it and came in and I just remember this like,
• 41:41
  Meredith Whittaker: like this man I'd never seen like handing me a bullhound from below and I picked
• 41:46
  Meredith Whittaker: it up and like we were like bullhound like a megaphone like a megaphone yeah yeah yeah.
• 41:52
  Meredith Whittaker: And then it was you know and then we walked over
• 41:56
  Meredith Whittaker: to a Mexican restaurant and sat at the table and had a press operation
• 41:59
  Meredith Whittaker: where like that we were running and what was cool was like everyone organizing
• 42:04
  Meredith Whittaker: that was kind of a professional most of them were femme and most of them had
• 42:08
  Meredith Whittaker: jobs at Google that were like organizing the company so organizing their comms
• 42:12
  Meredith Whittaker: being an administrator for like 13 different directors,
• 42:17
  Meredith Whittaker: this particular type of hyper competence at coordinating activity across a number
• 42:22
  Meredith Whittaker: of people who you may not have direct power over and like for that much more.
• 42:27
  Tim Pritlove: Than developers in that.
• 42:29
  Meredith Whittaker: Respect I mean let's just say Yeah, yes. You know, you can, like somebody who
• 42:34
  Meredith Whittaker: can write an email and get someone to respond to that email by doing a thing is kind of a witch.
• 42:41
  Meredith Whittaker: And it was all these like these preternaturally competent femmes just turning
• 42:46
  Meredith Whittaker: that energy toward organizing across the company,
• 42:49
  Meredith Whittaker: which was building on this sort of base of meetings and locals and all of this
• 42:54
  Meredith Whittaker: sort of work that we'd done to kind of form a discursive environment inside
• 43:00
  Meredith Whittaker: Google where we're like weekly meetings to discuss the news,
• 43:03
  Meredith Whittaker: what Google could do, what campaigns we could do.
• 43:05
  Meredith Whittaker: So we had this sort of energy and solidarity already at that point.
• 43:10
  Tim Pritlove: How many percent of the Google employees would you say have at one point taken part in this?
• 43:17
  Meredith Whittaker: I don't know. we were really careful
• 43:19
  Meredith Whittaker: not to keep lists yeah I mean you can dream up a number now you know I.
• 43:27
  Meredith Whittaker: 20,000 was the number estimated from the Sky Photos and the local reports.
• 43:33
  Meredith Whittaker: We had local leads at every office.
• 43:36
  Meredith Whittaker: We sent them zip files of the kit for handing out the flyers,
• 43:41
  Meredith Whittaker: the talking points, how to treat media, all of that.
• 43:43
  Meredith Whittaker: They all had that, and then they organized their local. And then they had reporting
• 43:47
  Meredith Whittaker: back in from press, reporting back into the central organizers around numbers.
• 43:51
  Meredith Whittaker: And then we were issuing the press releases. um but
• 43:54
  Meredith Whittaker: then you know there's employees of google and
• 43:58
  Meredith Whittaker: then there's contractors of google which more than double employees so
• 44:01
  Meredith Whittaker: it you know that number is and and then there's people who
• 44:04
  Meredith Whittaker: couldn't participate but were really supportive because they
• 44:07
  Meredith Whittaker: have you know need their health insurance or they'll die or they are on a visa
• 44:11
  Meredith Whittaker: and so it wasn't um i think there was a huge amount of support uh i know that
• 44:17
  Meredith Whittaker: we were able to get google to drop their military contract because there was
• 44:22
  Meredith Whittaker: enough support and there was you know there was like a spreadsheet of people who are quitting,
• 44:26
  Meredith Whittaker: conscientious objectors we had you know like every week at the all-town meeting
• 44:30
  Meredith Whittaker: there was a table we would set up with like banners and something like we would
• 44:34
  Meredith Whittaker: had questions like it was a very,
• 44:36
  Meredith Whittaker: it was a very like rigorous campaign and it kind of laid the laid the groundwork
• 44:43
  Meredith Whittaker: and i think it built some muscle that people are probably almost are definitely
• 44:48
  Meredith Whittaker: using now even if they don't call at organizing.
• 44:51
  Meredith Whittaker: It's like, how do we marshal the resources from this?
• 44:54
  Tim Pritlove: So did Google then still love you at that point?
• 44:57
  Meredith Whittaker: Well, I think Google does still love me, but it doesn't love itself enough to admit it.
• 45:10
  Tim Pritlove: They have issues, I think.
• 45:16
  Meredith Whittaker: I do think that. Come on, come on. Which side do you want to be on?
• 45:19
  Meredith Whittaker: Which party do you want to go to? Let's be real.
• 45:23
  Tim Pritlove: I mean, it leads to an interesting point because I would say,
• 45:28
  Tim Pritlove: looking from the other side of the ocean, I think to us, this whole tech scene,
• 45:37
  Tim Pritlove: The startups, the new stuff, the internet, everything that has developed in
• 45:42
  Tim Pritlove: the last 20 years or so, always had this liberal touch to it.
• 45:49
  Tim Pritlove: It felt as if it was mostly about an open, world-loving agenda and it's good for everybody.
• 45:58
  Tim Pritlove: And Google kind of tuned in with their motto and some other companies did as well, some not so much.
• 46:06
  Tim Pritlove: But it was always this feeling that liberal thinking is at the core of everything
• 46:14
  Tim Pritlove: that is driving the internet forward.
• 46:17
  Tim Pritlove: And I think we stopped thinking that now because it looks totally different right now.
• 46:28
  Tim Pritlove: Currently, we have more the feeling that it's turned into a total right-wing apocalypse somehow.
• 46:39
  Tim Pritlove: And I haven't really seen this coming.
• 46:43
  Tim Pritlove: Can you explain what happened to this tech scene? What happened?
• 46:50
  Meredith Whittaker: I, well, you know, I think about post 9-11 and a lot of the fights over surveillance and tech.
• 47:03
  Meredith Whittaker: And there's a talk that, who was, Frank Rieger and Rupert Gronkripp?
• 47:11
  Tim Pritlove: Rob Gronkripp, Rob Gronkripp.
• 47:12
  Meredith Whittaker: Sorry, I don't, that's my American, you'll do beautifully in America.
• 47:17
  Tim Pritlove: It's complicated for us.
• 47:17
  Meredith Whittaker: Ja, aber ich entschuldige für nicht das richtig.
• 47:21
  Tim Pritlove: Roppenfrank.
• 47:21
  Meredith Whittaker: Ja, ich habe diese Talk, We Lost the War, ich glaube es war 1994. Nein, 2004.
• 47:27
  Tim Pritlove: Ja.
• 47:28
  Meredith Whittaker: Ich habe die gleiche Sache gemacht.
• 47:34
  Meredith Whittaker: Und nicht jeder, ich glaube, war equating die growth und monetary success of
• 47:44
  Meredith Whittaker: the US-based tech industry with sort of, you know, values of social progress.
• 47:50
  Meredith Whittaker: I mean, I think we can, yeah, yes, it was liberal and then we can get back to
• 47:52
  Meredith Whittaker: a critique of liberalism and what have you, but it was, I think there were people
• 47:56
  Meredith Whittaker: who were looking at the infrastructure,
• 47:58
  Meredith Whittaker: who were looking at its capabilities, who were looking at the gap between the
• 48:02
  Meredith Whittaker: promises and the reality of what this tech did and calling that out.
• 48:07
  Meredith Whittaker: And I feel like, you know, when I entered into this kind of,
• 48:13
  Meredith Whittaker: Privacy, security, hacker development scene.
• 48:19
  Meredith Whittaker: There was a lot of that skepticism there around Google.
• 48:22
  Meredith Whittaker: That educated me a lot. There was a lot of skepticism around surveillance.
• 48:26
  Meredith Whittaker: I immediately recognized, yes, we need privacy because it doesn't matter if
• 48:31
  Meredith Whittaker: these people are good or benevolent.
• 48:33
  Meredith Whittaker: What we're doing is setting up an infrastructure that could be turned over at
• 48:36
  Meredith Whittaker: any moment to another regime. Logically, all of that made sense.
• 48:41
  Meredith Whittaker: But I don't feel that until the Snowden revelations, any of that was anywhere
• 48:46
  Meredith Whittaker: near in the nervous system of a kind of tech consciousness.
• 48:50
  Meredith Whittaker: And a lot of the work I've spent in the 1990s trying to sift through the crypto
• 48:56
  Meredith Whittaker: wars and sift through what happened with tech regulation to set up these surveillance
• 49:02
  Meredith Whittaker: giants and to permit this monopoly platform business model,
• 49:06
  Meredith Whittaker: has kind of looked at that gap between the rhetoric of liberal rights preserving.
• 49:13
  Meredith Whittaker: Open, free tech, and what was actually being built, right?
• 49:17
  Meredith Whittaker: And one of the things, if you look at, there's a scholar named Katharina Ryder,
• 49:22
  Meredith Whittaker: who I would really suggest, for show notes, I can send some of these links.
• 49:27
  Meredith Whittaker: But she did her dissertation looking at some of the negotiations in the crypto wars.
• 49:32
  Meredith Whittaker: And what you begin to see is that, yeah, we, you know, and this is a thesis
• 49:36
  Meredith Whittaker: sort of I build on top of in some of my work.
• 49:39
  Meredith Whittaker: Yeah, we won liberalized encryption, right?
• 49:42
  Meredith Whittaker: By 1999 in the US, it was finally legal to build, share, implement strong cryptosystems
• 49:51
  Meredith Whittaker: without approval from the government, without some threshold that made them useless.
• 49:57
  Meredith Whittaker: But the agreement there was basically, yeah, you can have encryption,
• 50:01
  Meredith Whittaker: but we're going to permit mass surveillance by companies.
• 50:06
  Meredith Whittaker: Und so you don't actually, like, you can just get the, you know,
• 50:10
  Meredith Whittaker: we're going to permit, we're going to endorse the advertising business model.
• 50:14
  Meredith Whittaker: We're going to endorse, and I can actually, I can start this point over, actually.
• 50:19
  Linus Neumann: Would you say those two decisions were...
• 50:25
  Linus Neumann: Strategically interlinked or were they just coinciding?
• 50:30
  Meredith Whittaker: I can't say that there was a conspiracy. What I can say is that Katarina's work
• 50:35
  Meredith Whittaker: shows that Microsoft saying liberalize encryption, don't worry we're not going
• 50:41
  Meredith Whittaker: to encrypt all the data, we need it.
• 50:44
  Meredith Whittaker: Just come to us quietly and we'll give it to you.
• 50:48
  Meredith Whittaker: Instead of fighting over a backdoor, instead of doing this in sort of the public
• 50:54
  Meredith Whittaker: domain where we're kind of losing the fight on technical and other grounds.
• 50:59
  Meredith Whittaker: Allow companies free reign to surveil because that allows us to implement this
• 51:05
  Meredith Whittaker: ad-supported business model.
• 51:07
  Meredith Whittaker: And then the data agreements can happen behind the scene.
• 51:11
  Meredith Whittaker: Now, I've like completely compressed a very complex history into basically a meme.
• 51:17
  Meredith Whittaker: But I think the purpose is there is that there was always that gap between these
• 51:22
  Meredith Whittaker: sort of rhetoric and what was actually going on.
• 51:25
  Meredith Whittaker: And I think the, I don't know, like there's like a kind of internet people, right?
• 51:31
  Meredith Whittaker: This type maybe misunderstood exactly like how this, you know,
• 51:36
  Meredith Whittaker: who would have power over this technology, right?
• 51:38
  Meredith Whittaker: Like encryption is liberalized, but it's not gonna be applied to protect personal
• 51:43
  Meredith Whittaker: communication. It's applied to protect transactions, right?
• 51:46
  Meredith Whittaker: It's not, you know, like the people who get to choose whether or not it's used aren't us.
• 51:51
  Meredith Whittaker: In terms of, you know, actually this sort of mass infrastructure in terms of
• 51:55
  Meredith Whittaker: the tech ecosystem that's being built by the actors who are calibrating their
• 51:59
  Meredith Whittaker: decisions based on the surveillance business model that has been instantiated right,
• 52:04
  Meredith Whittaker: you know, through regulatory decisions made by the Clinton administration.
• 52:07
  Linus Neumann: So then there were a couple of hundreds of thousands of nerds worldwide that used PGP encryption,
• 52:16
  Linus Neumann: using additional software and plugins to send an email with nonsense information
• 52:22
  Linus Neumann: to avoid government or private sector surveillance.
• 52:27
  Linus Neumann: But it was probably never really a significant number of individuals that...
• 52:36
  Linus Neumann: That got to the point of, you know, having mass encryption out there.
• 52:40
  Meredith Whittaker: Yeah, well, I tried. It's just that my friend group didn't overlap exactly with
• 52:45
  Meredith Whittaker: those couple of thousand nerds. So what am I going to do, right?
• 52:49
  Meredith Whittaker: And I think that's, you know, this is the network effect. This is why it's actually
• 52:52
  Meredith Whittaker: very difficult to do that.
• 52:53
  Meredith Whittaker: And this is why if one of those actors that controls these infrastructures doesn't
• 52:57
  Meredith Whittaker: make the choice for us, it's really difficult to make that choice.
• 53:01
  Linus Neumann: And none of these actors voluntarily got the idea, right? I think it's part
• 53:06
  Linus Neumann: of the founding myth of Signal that Moxie at the time wanted to implement the,
• 53:13
  Linus Neumann: it's now called the Signal Protocol.
• 53:14
  Linus Neumann: I still remember it as being Axolotl or whatever it was called.
• 53:17
  Meredith Whittaker: It used to be Axolotl. And then it was when Signal was launched,
• 53:21
  Meredith Whittaker: which was the Red Phone Tech Secure integrated into one app for iOS in 2013. It was changed.
• 53:28
  Linus Neumann: Right. But initially, I believe Moxie wanted to implement at Crypto for Twitter,
• 53:35
  Linus Neumann: direct messages, if I'm not mistaken. I think it was at Twitter before.
• 53:39
  Linus Neumann: And so his idea to roll out Twitter,
• 53:44
  Linus Neumann: Definitely, he devoted a few years of his life to implementing and rolling out
• 53:50
  Linus Neumann: mass end-to-end encryption, right?
• 53:53
  Linus Neumann: But I believe it was he wanted to do it at Twitter at the time.
• 53:56
  Linus Neumann: And maybe I'm wrong here, then our listeners will correct me two minutes into the show.
• 54:04
  Linus Neumann: But clearly, none of the large...
• 54:06
  Meredith Whittaker: And I don't actually know for sure that story, so don't correct me.
• 54:11
  Linus Neumann: It's my mistake.
• 54:14
  Tim Pritlove: Ich habe einen anderen Punkt, aber ich denke, dass Twitter nicht sich als eine
• 54:21
  Tim Pritlove: Messergebung und eine Messergebung und eine Messergebung was,
• 54:22
  Tim Pritlove: was wahrscheinlich ein Mistake von der Ich meine.
• 54:25
  Meredith Whittaker: Und der Regulatory Issues. Ja. Ich meine, ich habe mich mit der Security Team
• 54:29
  Meredith Whittaker: und da waren immer diese Ideen,
• 54:30
  Meredith Whittaker: ich habe immer ein Projekt wo wir uns an die Keystore für andere Services und
• 54:38
  Meredith Whittaker: du hast diese wirklich exciting conversations mit den Security Guys and then
• 54:42
  Meredith Whittaker: it wouldn't go anywhere because then, you know, the other guys would get involved.
• 54:47
  Linus Neumann: Point I was trying to make is none of the big platforms ever rolled out mass
• 54:51
  Linus Neumann: scale end-to-end encryption until eventually WhatsApp did.
• 54:54
  Linus Neumann: And it took an Edward Snowden revelations and an ongoing scanner for I think
• 55:01
  Linus Neumann: roughly one year until Facebook at the time said, okay, we'll implement this.
• 55:08
  Meredith Whittaker: Well, actually, I mean, that's, So Signal existed before the WhatsApp integration.
• 55:13
  Linus Neumann: Sure.
• 55:14
  Meredith Whittaker: And the WhatsApp integration was driven by Brian and Jan, who are the co-founders.
• 55:19
  Meredith Whittaker: And my understanding there is that they were rushing to get that done before
• 55:23
  Meredith Whittaker: the Facebook integration, to make sure that they weren't selling something that
• 55:27
  Meredith Whittaker: would violate their principles.
• 55:30
  Meredith Whittaker: And I know Moxie was working on that. I remember that period of time.
• 55:34
  Meredith Whittaker: Und dann Facebook bought it.
• 55:39
  Meredith Whittaker: Und ich weiß nicht, dass die deal da ist, aber es bleibt, mit der Signals-Protokolle.
• 55:51
  Meredith Whittaker: Es war ein Post-Snowden Moment.
• 55:55
  Meredith Whittaker: Du sahst Android und iOS implementiert Full Disk Encryption.
• 55:59
  Meredith Whittaker: Du sahst Google encrypting. its.
• 56:06
  Meredith Whittaker: HTPS for its networks and I think a lot of this was just like we need to distance
• 56:12
  Meredith Whittaker: ourselves from bad government spying by adding encryption that proves that we're
• 56:17
  Meredith Whittaker: not actually part of the problem that they have just sort of,
• 56:21
  Meredith Whittaker: the bad government has attacked us as taxpaying corporations,
• 56:26
  Meredith Whittaker: taken this data that we really want to protect but god how could we have known
• 56:30
  Meredith Whittaker: and so now we're encrypting things and,
• 56:34
  Meredith Whittaker: ultimately good, right? But it was a way of not looking at the full story of
• 56:39
  Meredith Whittaker: like, why is that there to begin with?
• 56:42
  Meredith Whittaker: And what else is not being encrypted? What other data is being given over?
• 56:47
  Meredith Whittaker: And why do you have the choice to do that to begin with instead of a more socially
• 56:53
  Meredith Whittaker: beneficial, democratic process
• 56:55
  Meredith Whittaker: of determining how we're comfortable with technology entering our lives?
• 56:58
  Linus Neumann: I probably would have even had an even better or even stronger perspective on
• 57:04
  Linus Neumann: it seems like the governments didn't really attack these corporations.
• 57:09
  Linus Neumann: The governments made their business model look bad. So now they needed to change
• 57:14
  Linus Neumann: something to convince people no, no, no, your data is safe here.
• 57:17
  Linus Neumann: We'll encrypt something and moving on, moving on.
• 57:21
  Meredith Whittaker: I was, for the gentle listener, I was kind of joking because that rhetoric around
• 57:27
  Meredith Whittaker: being attacked and being like, oh my gosh, was Very much the mood at that time.
• 57:34
  Meredith Whittaker: I was at Google when Snowden dropped and I remember just things popped off and
• 57:39
  Meredith Whittaker: I actually had to get, I got on a plane.
• 57:42
  Meredith Whittaker: I don't know, you all probably have a memory similar to me of like when the
• 57:46
  Meredith Whittaker: Guardian stories with the Verizon water, like the Glenn Greenwald stories.
• 57:50
  Meredith Whittaker: It was night in New York. It was probably morning the next day you guys saw
• 57:54
  Meredith Whittaker: it. I remember sitting on my couch and being like, holy fuck.
• 57:57
  Meredith Whittaker: And realizing just how big that was because it was the kind of thing we'd been
• 58:01
  Meredith Whittaker: talking about, speculating about in the rooms that you and I were in, Linus.
• 58:05
  Meredith Whittaker: And then it was like, oh, receipts, shit.
• 58:08
  Meredith Whittaker: And there was a lot of unclarity. There was that Prism slide where it was like,
• 58:12
  Meredith Whittaker: is Google just giving them full access? That people were rioting inside.
• 58:16
  Meredith Whittaker: Security engineers were threatening to quit.
• 58:18
  Meredith Whittaker: And then that morning I got on a plane to TorDev.
• 58:25
  Meredith Whittaker: And so, yeah, in Berlin, actually.
• 58:29
  Linus Neumann: I remember that this time,
• 58:34
  Linus Neumann: and we at CCC really had to bite our tongue and write a media communications
• 58:40
  Linus Neumann: strategy and said, nobody says I told you so.
• 58:45
  Linus Neumann: We need to at least act surprised as well, right? And not say,
• 58:52
  Linus Neumann: well, we told you all along.
• 58:55
  Meredith Whittaker: I give you all a pass, because you did tell us all along.
• 58:59
  Linus Neumann: But of course it gave a lot of international media attention to our cause.
• 59:05
  Linus Neumann: And we needed to play that public attention wisely.
• 59:11
  Meredith Whittaker: Exactly.
• 59:12
  Linus Neumann: I think we maybe did.
• 59:15
  Tim Pritlove: I find it quite interesting how encryption as a topic has changed over time. It's just...
• 59:23
  Tim Pritlove: More or less 10 years ago that Facebook actually changed to HTTPS on their website by default.
• 59:29
  Tim Pritlove: And so there was a time not so long ago, you know, where most of the data was
• 59:34
  Tim Pritlove: flowing around on the Internet, mostly unencrypted.
• 59:38
  Tim Pritlove: And that although there was these already mentioned crypto wars,
• 59:43
  Tim Pritlove: you know, about general encryption, but it was also always for nerds and for
• 59:47
  Tim Pritlove: specific applications.
• 59:49
  Tim Pritlove: Then it also got this nice paint with this whole cryptocurrency craze going
• 59:56
  Tim Pritlove: on, which made it somehow popular and almost took the word away.
• 1:00:02
  Meredith Whittaker: We're so wrestling.
• 1:00:05
  Tim Pritlove: And it was also the rise of encrypted messaging that was really giving a new fuel.
• 1:00:14
  Tim Pritlove: So Signal was in the middle of all of this, as we already heard.
• 1:00:17
  Tim Pritlove: So, I'd like to focus on Signal for a moment as an organization that you now head.
• 1:00:27
  Tim Pritlove: What's your understanding of
• 1:00:29
  Tim Pritlove: what Signal is and what it's not and how the organization deals with it?
• 1:00:37
  Meredith Whittaker: Well, I love Signal and I'm really...
• 1:00:44
  Meredith Whittaker: Yeah, it's the only cool tech company in my view. Und ich denke...
• 1:00:51
  Meredith Whittaker: So boiling down, like what is Signal in one word is a little,
• 1:00:55
  Meredith Whittaker: you know, I'll just, I'll start somewhere and we'll end another place because
• 1:00:58
  Meredith Whittaker: I think it's actually, it's a number of things and kind of represents even more.
• 1:01:03
  Meredith Whittaker: You know, Signal started back, you know, the late 2000s, right?
• 1:01:09
  Meredith Whittaker: And we can, you know, we can date it to whenever, right?
• 1:01:11
  Meredith Whittaker: Signal as the integrated app was 2013, but your Red Phone, TextSecure predated that.
• 1:01:17
  Meredith Whittaker: And this is, you know, there's no iPhone.
• 1:01:22
  Meredith Whittaker: Jabber is the competition, right? It's like web client-based chat.
• 1:01:27
  Meredith Whittaker: There's no, you know, people aren't carrying smartphones. There isn't,
• 1:01:32
  Meredith Whittaker: WhatsApp doesn't exist.
• 1:01:33
  Meredith Whittaker: IMessage doesn't exist. You have a very, very different marketplace.
• 1:01:38
  Tim Pritlove: Were there still ICQ and AIM, I think, at the time?
• 1:01:42
  Meredith Whittaker: Yeah, yeah. I mean, I remember using, like, I don't remember,
• 1:01:48
  Meredith Whittaker: I'd like send text messages sometimes, but they were expensive on my BlackBerry, maybe.
• 1:01:55
  Meredith Whittaker: But we're talking about a drastically different tech ecosystem.
• 1:01:59
  Meredith Whittaker: And this is particularly important in the context of messaging and communications apps.
• 1:02:03
  Meredith Whittaker: Because, of course, you need a network effect for those to work.
• 1:02:07
  Meredith Whittaker: No one buys the first telephone because you can't use one telephone.
• 1:02:12
  Meredith Whittaker: Your friend has to have a telephone.
• 1:02:14
  Meredith Whittaker: All your friends have to have an app if you're going to use it.
• 1:02:16
  Meredith Whittaker: Particularly, it takes two to encrypt. Group encryption, it takes everyone in the group.
• 1:02:21
  Meredith Whittaker: And if everyone isn't using your app for communication, it's very different
• 1:02:29
  Meredith Whittaker: in a saturated marketplace where you have WhatsApp,
• 1:02:32
  Meredith Whittaker: where you have iMessage, where you have these normative models that people go
• 1:02:35
  Meredith Whittaker: to just to have their regular communication to introduce a new platform.
• 1:02:42
  Meredith Whittaker: For secure messaging or insecure messaging, right?
• 1:02:45
  Meredith Whittaker: Because people don't switch outside of the network unless their friends switch
• 1:02:49
  Meredith Whittaker: outside of the network and there's a collective action problem there and an
• 1:02:52
  Meredith Whittaker: inertia problem there and all of that, right?
• 1:02:54
  Meredith Whittaker: So when I think about, there's many things that are very precious about Signal,
• 1:02:59
  Meredith Whittaker: but the fact that Moxie carried it on his back for that decade and was actually
• 1:03:05
  Meredith Whittaker: able to keep it going and surviving without selling out,
• 1:03:09
  Meredith Whittaker: without selling data and actually creating something that is now able to scale
• 1:03:14
  Meredith Whittaker: to hundreds of millions of people.
• 1:03:19
  Meredith Whittaker: Means that Signal actually has a position in this ecosystem that makes it useful to people.
• 1:03:26
  Meredith Whittaker: That means that it's actually providing encrypted communication to people all
• 1:03:32
  Meredith Whittaker: over the globe because their friends are using it.
• 1:03:35
  Meredith Whittaker: And my contention here, and I'm willing to discuss this, I don't think we can
• 1:03:40
  Meredith Whittaker: recreate Signal, right?
• 1:03:42
  Meredith Whittaker: You could shift, you know, because it has that user base, right?
• 1:03:46
  Meredith Whittaker: You can introduce a new app, but how do you get people to use it without an
• 1:03:50
  Meredith Whittaker: OEM, without an existing installed user base, without some way of,
• 1:03:54
  Meredith Whittaker: you know, kind of making it useful to people?
• 1:03:57
  Meredith Whittaker: Because again, it's, you know, one telephone or, you know, a couple thousand
• 1:04:01
  Meredith Whittaker: hackers who all use PGP, but they can't talk to their dad on PGP, right?
• 1:04:05
  Meredith Whittaker: They can't talk to anyone outside of themselves on PGP. So Signal has both sort
• 1:04:10
  Meredith Whittaker: of kept this form that is very heterodox in tech.
• 1:04:15
  Meredith Whittaker: It's a nonprofit, so I couldn't really sell out profitably.
• 1:04:20
  Meredith Whittaker: I could try to sell out, but I'm not going to get any money from it.
• 1:04:22
  Meredith Whittaker: It has to be reinvested in the mission, and there are certain rules around mission alignment there.
• 1:04:27
  Meredith Whittaker: So we're not pressured to monetize surveillance or to collect any data.
• 1:04:33
  Meredith Whittaker: We're really able to stay focused on that mission and in an ecosystem where.
• 1:04:39
  Meredith Whittaker: The diametric opposite is the norm. That's really, really important,
• 1:04:45
  Meredith Whittaker: irrespective about the flaws of the nonprofit model more generally.
• 1:04:49
  Meredith Whittaker: So it's achieved this pretty rare, like it's the only thing like it in the ecosystem.
• 1:04:54
  Meredith Whittaker: And I think it also serves as a model for how we could think about building tech differently.
• 1:05:04
  Meredith Whittaker: Like, how do we disarm, deconstruct the massive centralized power of a handful
• 1:05:11
  Meredith Whittaker: of platform companies that basically control most of the infrastructure and
• 1:05:15
  Meredith Whittaker: information ecosystem in our world and our jurisdiction in the US?
• 1:05:18
  Meredith Whittaker: And how do we build other models that may be interoperable, that are more open,
• 1:05:24
  Meredith Whittaker: that are more rights preserving, and that aren't subject to the pressures of
• 1:05:28
  Meredith Whittaker: and incentives of the surveillance business model?
• 1:05:30
  Linus Neumann: Now, I find it interesting, you mentioned the network effect that for a long
• 1:05:35
  Linus Neumann: time worked against you, right? You said, okay, it took Moxie a decade.
• 1:05:42
  Linus Neumann: Now, that network effect, you know, the EU regulatory bodies have an idea on
• 1:05:48
  Linus Neumann: how to weaken it by enabling messenger interoperability,
• 1:05:54
  Linus Neumann: Thus, pretty much trying to force the large messenger operators,
• 1:06:01
  Linus Neumann: be it WhatsApp or whatever else people use, to offer an interoperability interface.
• 1:06:08
  Linus Neumann: So any new messenger would have it easier than Signal did to reach the critical,
• 1:06:18
  Linus Neumann: amount of users to actually have the network effect.
• 1:06:23
  Linus Neumann: Now I know Signal's position strongly opposes this idea or you decided not to
• 1:06:30
  Linus Neumann: participate in this interoperability or making use of it.
• 1:06:36
  Linus Neumann: There are two.
• 1:06:40
  Linus Neumann: Forces or two goals of Signal, I guess, that contradict each other here,
• 1:06:45
  Linus Neumann: and that is its security and building open communication systems.
• 1:06:51
  Linus Neumann: Maybe you can explain a little bit why, after, you know, having understood how
• 1:06:56
  Linus Neumann: hard it is to build Signal and, you know, against a market, why you would still
• 1:07:02
  Linus Neumann: oppose messenger interoperability.
• 1:07:05
  Meredith Whittaker: Yeah, I like this question because I'm going to, I want to clarify our position,
• 1:07:12
  Meredith Whittaker: which is a little bit of nuance.
• 1:07:19
  Meredith Whittaker: I don't oppose interoperability in principle.
• 1:07:24
  Meredith Whittaker: Like if the interoperability mandate of the DSA were, you all have to,
• 1:07:31
  Meredith Whittaker: you know, interoperability needs to be an option and it has to happen at this rigorous security bar.
• 1:07:38
  Meredith Whittaker: You have to make sure that you're implementing metadata security, sealed sender.
• 1:07:46
  Meredith Whittaker: Basically, you're adopting the
• 1:07:48
  Meredith Whittaker: signal privacy and security bar as the conditions for interoperability.
• 1:07:56
  Meredith Whittaker: That would be really cool, right? And I think that's the issue here.
• 1:08:00
  Meredith Whittaker: I mean, and I want to put an asterisk here for a moment saying,
• 1:08:03
  Meredith Whittaker: like, there are a lot of other complexities around policy.
• 1:08:06
  Meredith Whittaker: Like, you know, how do you take in a, you know, who deals with a law enforcement request, right?
• 1:08:13
  Meredith Whittaker: Even if you have no data to give them, you can't just ignore that.
• 1:08:16
  Meredith Whittaker: You know, like, if a user has a complaint, who do they write in?
• 1:08:21
  Meredith Whittaker: If you're interoperating with a platform for communication that also has a social
• 1:08:25
  Meredith Whittaker: media arm, there's a totally different regulatory environment for Telegram or
• 1:08:29
  Meredith Whittaker: WhatsApp with channels than for a signal.
• 1:08:32
  Meredith Whittaker: How does that sort of work?
• 1:08:35
  Meredith Whittaker: So I want to say this isn't simple.
• 1:08:38
  Linus Neumann: It's a whole can of worms.
• 1:08:39
  Meredith Whittaker: There's a massive can of worms as the EU often opens. Um, but the,
• 1:08:45
  Meredith Whittaker: you know, like the conditions of interoperability are actually really,
• 1:08:49
  Meredith Whittaker: you know, they're really political here. Right.
• 1:08:51
  Meredith Whittaker: So in order to interoperate with WhatsApp, am I going to be giving signal user data to Meta?
• 1:08:59
  Meredith Whittaker: Well, that would violate the entire premise of what I'm spending my life's energy
• 1:09:04
  Meredith Whittaker: doing and Signal, right?
• 1:09:06
  Meredith Whittaker: Is Meta going to decide to, you know, cut off the account of one of our users?
• 1:09:11
  Meredith Whittaker: Who gets to decide that, right? Are they, you know, are they collecting,
• 1:09:15
  Meredith Whittaker: you know, other data because they aren't implementing some of our libraries or whatever?
• 1:09:20
  Meredith Whittaker: And so I think, you know, that's where the rubber meets the road.
• 1:09:23
  Meredith Whittaker: And we have to have a duty of care to the people who rely on Signal.
• 1:09:27
  Meredith Whittaker: Das heißt, wir sind absolut nicht going zu verabschieden, dass wir wissen,
• 1:09:32
  Meredith Whittaker: wenn wir real sind, dass es eine Frau in der Jäule in den USA ist,
• 1:09:39
  Meredith Whittaker: weil Meta in den Facebook-Messagen hat, zwischen ihr und ihr daughter,
• 1:09:44
  Meredith Whittaker: die sie überprüfen hat, dass sie sie erhalten,
• 1:09:47
  Meredith Whittaker: in der Stadt von Nebraska, nach dem Dobbs-Dobbs-Dobbs-Dobbs-Dobbs.
• 1:09:52
  Meredith Whittaker: That's the stakes of this conversation, even when we're talking about the technical
• 1:09:56
  Meredith Whittaker: details of interoperability, right?
• 1:09:59
  Linus Neumann: Dobbs' decision is reverting Roe v. Wade, right?
• 1:10:03
  Meredith Whittaker: Yeah.
• 1:10:04
  Linus Neumann: Okay, so just for some of the listeners that are not that much into the US policy.
• 1:10:09
  Tim Pritlove: Basically outlawing abortion.
• 1:10:11
  Linus Neumann: Allowing states to outlaw abortion.
• 1:10:14
  Meredith Whittaker: Access to life-saving healthcare that more than half the population may need.
• 1:10:21
  Linus Neumann: So your answer in short is interoperability is fine,
• 1:10:26
  Linus Neumann: but you do not see any path currently being debated that would result in upholding
• 1:10:33
  Linus Neumann: the security promises that Signal has worked long for to be able to make to its user base.
• 1:10:41
  Meredith Whittaker: We will continue to advocate for a path that raises that bar,
• 1:10:46
  Meredith Whittaker: meets or exceeds Signal's bar. And if it succeeds, I'm like,
• 1:10:50
  Meredith Whittaker: yeah, I want to talk about that.
• 1:10:53
  Meredith Whittaker: But, you know, those are the conditions under which we would interoperate.
• 1:10:57
  Meredith Whittaker: So we're not, you know, we don't take a stand against it.
• 1:10:59
  Meredith Whittaker: You know, we just say like, look, these are the complexities and this is,
• 1:11:04
  Meredith Whittaker: you know, Signal stands with the people who rely on Signal, not with a sort
• 1:11:07
  Meredith Whittaker: of, you know, vision for some muddy middle where we're all interoperating,
• 1:11:11
  Meredith Whittaker: but we've sort of, you know, sold people out and made them susceptible to,
• 1:11:14
  Meredith Whittaker: you know, what we describe with Meta.
• 1:11:17
  Tim Pritlove: Do people in the EU understand what you're talking about if you're offering
• 1:11:22
  Tim Pritlove: these technical explanations why it's complicated?
• 1:11:25
  Tim Pritlove: Because we have the impression that they don't really get it.
• 1:11:28
  Meredith Whittaker: Is this mean or median, people?
• 1:11:32
  Meredith Whittaker: I mean, I don't think most people don't understand this at all because they
• 1:11:35
  Meredith Whittaker: got laundry to do and this isn't their area, right?
• 1:11:38
  Meredith Whittaker: I think some of the politicians I've talked to seem to get it, but it's not...
• 1:11:47
  Meredith Whittaker: Ich denke, dass es eine Inertia über das Prozess ist, das bedeutet,
• 1:11:52
  Meredith Whittaker: dass es nicht klar, wie weit diese Punkte haben, die Bedrock zu haben, so zu sprechen.
• 1:11:59
  Meredith Whittaker: Ich würde auch, als Amerikaner, ich bin hier, für heute.
• 1:12:04
  Meredith Whittaker: Aber ich also habe, ich würde sagen, mein Instinkt auf die Generalentwicklung
• 1:12:12
  Meredith Whittaker: der EU ist nicht was ich, was ich rely auf.
• 1:12:14
  Tim Pritlove: Which brings me to an interesting point because we are actually very interested
• 1:12:20
  Tim Pritlove: in your view as an American knowing how things work on that continent,
• 1:12:28
  Tim Pritlove: What's your impression of how Europe deals with tech these new technologies
• 1:12:34
  Tim Pritlove: coming up and how it impacts society Can you just give me a feeling for how this is to you,
• 1:12:47
  Tim Pritlove: in a good way, in a bad way, whatever,
• 1:12:50
  Tim Pritlove: you feel just to...
• 1:12:52
  Meredith Whittaker: One meta-observation, having been here for a bit of duration now, is,
• 1:13:00
  Meredith Whittaker: And I've thought this before, but it's interesting to think of Europe as one thing.
• 1:13:07
  Meredith Whittaker: When you go to different countries and meet different people and you're like,
• 1:13:10
  Meredith Whittaker: wow, this is not one thing.
• 1:13:14
  Linus Neumann: I mean, that's the same in the US, right?
• 1:13:16
  Meredith Whittaker: It is in the US, yeah. I mean, you know, it's big.
• 1:13:19
  Tim Pritlove: But there is a common theme somehow. I'm just focusing on what you can probably
• 1:13:25
  Tim Pritlove: match to Europe in general. or at least to the kind of discussions you have
• 1:13:29
  Tim Pritlove: on a political level when you face EU institutions.
• 1:13:33
  Meredith Whittaker: It's almost, it's split in an interesting way.
• 1:13:37
  Meredith Whittaker: Because on one part, if you go to kind of the startup ecosystem,
• 1:13:43
  Meredith Whittaker: the VC ecosystem, like that world, there's a real,
• 1:13:47
  Meredith Whittaker: you know, there's a lot of smart people and cool people doing cool things.
• 1:13:52
  Meredith Whittaker: And there's sometimes a bit of magical thinking that I see, which is really like,
• 1:13:58
  Meredith Whittaker: you know, if we wish hard enough, if we're able to figure it out,
• 1:14:02
  Meredith Whittaker: we're going to be able to create competitors to the US incumbents,
• 1:14:06
  Meredith Whittaker: right? And we're going to have our own thing.
• 1:14:09
  Tim Pritlove: Own search engine.
• 1:14:12
  Meredith Whittaker: Which often just, you know, sometimes I'm just like, okay, that's a money play, right?
• 1:14:16
  Meredith Whittaker: Like, you get enough in, you know, your Series A, Series B, and then you'll
• 1:14:19
  Meredith Whittaker: get acquired, and no one will do anything with it, or you'll get rich, or whatever.
• 1:14:22
  Meredith Whittaker: You know, like, you may not necessarily believe that. Like, markets float on Hype.
• 1:14:27
  Meredith Whittaker: So, okay. But there is this thread where it's almost a willful misunderstanding
• 1:14:34
  Meredith Whittaker: of the reality of incumbent platforms, of the history that accrued that type
• 1:14:40
  Meredith Whittaker: of power to US companies,
• 1:14:42
  Meredith Whittaker: and of the dependencies that Europe and most of the rest of the world have on these companies.
• 1:14:49
  Meredith Whittaker: The three cloud companies based in the US have 70, 70% of the global market.
• 1:14:55
  Meredith Whittaker: You have five major social media platforms.
• 1:14:58
  Tim Pritlove: Amazon, Google and Microsoft.
• 1:14:59
  Meredith Whittaker: Yeah, AWS, Azure, GCP. And then I think the other percentage are made up by
• 1:15:07
  Meredith Whittaker: US companies as well. And then there's some Chinese companies.
• 1:15:10
  Meredith Whittaker: And then you have five platforms that effectively shape our global information
• 1:15:15
  Meredith Whittaker: ecosystem, like our perception of reality.
• 1:15:18
  Meredith Whittaker: The four biggest jurisdictions in the US, right?
• 1:15:22
  Tim Pritlove: Which five platforms?
• 1:15:26
  Meredith Whittaker: Das ist, was ich werde, ich werde, ich werde, vier haben, denn da ist immer eine Last in der Liste.
• 1:15:30
  Meredith Whittaker: So TikTok ist eine Non-U.S. one, und das ist die eine die Leute freigte aus, um, recently.
• 1:15:37
  Linus Neumann: Weil es ist Non-U.S.?
• 1:15:38
  Meredith Whittaker: Ja, ich denke, so flatly, ist es.
• 1:15:43
  Meredith Whittaker: Facebook, Instagram, X.
• 1:15:48
  Meredith Whittaker: Und dann, es ist nicht Twitch, aber irgendwie, da ist eine andere one auf YouTube.
• 1:15:55
  Meredith Whittaker: Und dann ist Twitch, too.
• 1:15:59
  Meredith Whittaker: Für all intents und purposes, das ist ein huge amount of concentrated power
• 1:16:02
  Meredith Whittaker: that, again, relies on network effects, relies on economies of scale,
• 1:16:06
  Meredith Whittaker: relies on all kinds of global infrastructure.
• 1:16:09
  Meredith Whittaker: It's trillions of dollars that can't just be interrupted by investment. This is a kind of...
• 1:16:17
  Tim Pritlove: This is social media companies.
• 1:16:19
  Meredith Whittaker: Social media and platform companies, right?
• 1:16:21
  Tim Pritlove: And isn't Telegram one of them?
• 1:16:23
  Meredith Whittaker: Well, Telegram doesn't, I think, run most of their own infrastructure.
• 1:16:26
  Meredith Whittaker: They don't have a cloud business model.
• 1:16:29
  Meredith Whittaker: And they also don't really have a business model. It seems like they have this crypto play.
• 1:16:34
  Meredith Whittaker: But it's not clear how that money moves. There's a lot of UAE investment.
• 1:16:40
  Meredith Whittaker: So you're not talking about these big tech...
• 1:16:42
  Tim Pritlove: Okay, you're focusing on cloud, not so much on the social media aspect.
• 1:16:45
  Meredith Whittaker: And what is it difficult? Where is the normative shape of the tech industry coming from, right?
• 1:16:54
  Meredith Whittaker: Like if the cloud companies all of a sudden decided to cut off half their APIs
• 1:16:57
  Meredith Whittaker: and change their infrastructure, there's like most startups in the entire world,
• 1:17:02
  Meredith Whittaker: including organizations like Signal, Telegram, whoever's writing on top,
• 1:17:08
  Meredith Whittaker: all their engineers' pagers go off.
• 1:17:10
  Meredith Whittaker: They got to respond to that, right? It's unidirectional that way.
• 1:17:14
  Linus Neumann: Right?
• 1:17:15
  Meredith Whittaker: If sanctions go up, let's do a wild dictator gets elected in the US and decides
• 1:17:23
  Meredith Whittaker: that Europe is now sanctioned and that Amazon can't do business with Europe. What happens?
• 1:17:29
  Tim Pritlove: Data in the US, that would never happen.
• 1:17:31
  Meredith Whittaker: I make things up as a creative person.
• 1:17:37
  Meredith Whittaker: Remember, I come from literature and rhetoric.
• 1:17:41
  Tim Pritlove: Just stories in your head. Ja.
• 1:17:43
  Meredith Whittaker: Und dann sieht man die Social Media Platforms, die wir wissen,
• 1:17:47
  Meredith Whittaker: dass es ein far-right ist,
• 1:17:50
  Meredith Whittaker: das wirklich hat die Intelligenz und Interessenzierung auf die Alternative Media
• 1:17:57
  Meredith Whittaker: Ecosystems über die Plattformen und die Affordanzer von Surveillance-Advertising-Driven
• 1:18:04
  Meredith Whittaker: Media Plattformen zu schaffen.
• 1:18:08
  Tim Pritlove: Okay, back to the magical thinking of Europe.
• 1:18:12
  Meredith Whittaker: Yeah. I'd like to understand how you see.
• 1:18:16
  Tim Pritlove: So in this world.
• 1:18:19
  Linus Neumann: There are like a few Europeans that say, here's 50 million, let's compete with these guys.
• 1:18:23
  Meredith Whittaker: Yeah, yeah, yeah.
• 1:18:23
  Tim Pritlove: And that's probably what the European search engine is.
• 1:18:25
  Meredith Whittaker: It's like 500 million dollars European sovereign AI fund.
• 1:18:30
  Meredith Whittaker: And then you're like, but that's half a training run. Like, what are you buying with that?
• 1:18:37
  Meredith Whittaker: Which is like which is disturbing because it's like okay well that's a lot of
• 1:18:41
  Meredith Whittaker: money also let's not be flip about it and it could be going to really good things
• 1:18:45
  Meredith Whittaker: it could be supporting you know interesting open projects it could be supporting
• 1:18:49
  Meredith Whittaker: you know alternative interoperable alternatives are like you know smaller clouds for more,
• 1:18:56
  Meredith Whittaker: heterodox open source project like there's really cool stuff that is languishing
• 1:19:03
  Meredith Whittaker: without that money and I think it's where is that money going?
• 1:19:08
  Meredith Whittaker: Well if you're talking about going into AI it's going to one of those three
• 1:19:11
  Meredith Whittaker: cloud companies it's renting infrastructure from Microsoft Amazon or Google
• 1:19:16
  Meredith Whittaker: for model development or for deployment which is inference and inference is
• 1:19:21
  Meredith Whittaker: really expensive you don't just train once,
• 1:19:24
  Meredith Whittaker: you use a model using it is way way more expensive than normal information retrieval
• 1:19:29
  Meredith Whittaker: and so you're also it's just this massive computationally expensive,
• 1:19:34
  Meredith Whittaker: Und du hast nicht die Europäische Sovereinheit, du hast eine Gefühl, die...
• 1:19:41
  Meredith Whittaker: I don't know, like, not behind, which, like, you know, and feeling of not being
• 1:19:46
  Meredith Whittaker: ashamed by being technologically...
• 1:19:49
  Tim Pritlove: But apart from the magical thinking, is there anything else you would stick to in Europe?
• 1:19:54
  Meredith Whittaker: And then there's the other side, which is I often find a much more sophisticated
• 1:19:58
  Meredith Whittaker: and clear-eyed view of these problems, right?
• 1:20:01
  Meredith Whittaker: Like, having this discussion about that concentrated power in the hands of infrastructure
• 1:20:04
  Meredith Whittaker: and media ecosystem, way easier in Europe.
• 1:20:08
  Meredith Whittaker: I mean, people feel it, right? They see it. And there's been a history of pushing
• 1:20:13
  Meredith Whittaker: back against the encroachment of US
• 1:20:15
  Meredith Whittaker: tech, both effectively and often very ineffectively, that I really enjoy.
• 1:20:21
  Meredith Whittaker: And particularly in Germany, there's a very high sensitivity to privacy,
• 1:20:26
  Meredith Whittaker: very often clear-eyed view on some of these debates, which doesn't always translate into policy.
• 1:20:31
  Meredith Whittaker: But there's at least, like I find the intellectual environment around this stuff,
• 1:20:36
  Meredith Whittaker: when you talk to people who are knowledgeable and have thought about it,
• 1:20:38
  Meredith Whittaker: to be very, I don't know, to teach me a lot and be really sophisticated.
• 1:20:43
  Tim Pritlove: Yeah, the GDPR is probably a German thing somehow in its core, for sure.
• 1:20:50
  Tim Pritlove: So how does this affect the talk to European politicians and how do you see
• 1:20:56
  Tim Pritlove: the trends in regulations and trying to apply new laws and regulations towards
• 1:21:05
  Tim Pritlove: this whole tech industry?
• 1:21:07
  Meredith Whittaker: Ja, ich meine, es gibt beide threads.
• 1:21:14
  Meredith Whittaker: Du hast zwei Wolves inside European Politik.
• 1:21:19
  Meredith Whittaker: Der eine will seine eigene Tech-Industrie und der eine will sie nicht zu machen.
• 1:21:23
  Meredith Whittaker: Das ist nicht zu US-Tech-Colonialismus.
• 1:21:27
  Meredith Whittaker: Und ich denke, du hast du some weirden Laws.
• 1:21:30
  Meredith Whittaker: Du hast die AI-Act, die aufwärtigte, die aufwärtigte, kind of had this last-minute
• 1:21:35
  Meredith Whittaker: brinksmanship around whether foundation models.
• 1:21:41
  Meredith Whittaker: These big LLMs that are now the trendy kind should be included or not.
• 1:21:46
  Meredith Whittaker: And you often see bold regulatory attempts that then get kind of shaped in odd
• 1:21:53
  Meredith Whittaker: ways, trying to have it both ways, right?
• 1:21:56
  Meredith Whittaker: Like how do we regulate the Americans away and get our own, right?
• 1:22:00
  Meredith Whittaker: But how do we do that in a way that it is reflected in principles,
• 1:22:04
  Meredith Whittaker: not in actually declaring that as an intent.
• 1:22:07
  Meredith Whittaker: And I think that is, I think you're seeing a huge amount of money be spent by
• 1:22:12
  Meredith Whittaker: the US companies in Brussels right now, which is also influencing things in interesting ways.
• 1:22:18
  Meredith Whittaker: And then this is something I'm theorizing a lot in my intellectual work,
• 1:22:23
  Meredith Whittaker: or just, and I think is really important.
• 1:22:27
  Meredith Whittaker: You also see what I'm calling the politics of intellectual shame.
• 1:22:31
  Meredith Whittaker: Be really pervasive in this conversation and this is not just Europe this is across the board.
• 1:22:37
  Meredith Whittaker: I mean that there is a real.
• 1:22:40
  Meredith Whittaker: A real fear among a lot of people who are in decision-making positions,
• 1:22:44
  Meredith Whittaker: politicians or academics or whoever,
• 1:22:47
  Meredith Whittaker: and not even in decision-making positions, but it matters when it's them,
• 1:22:52
  Meredith Whittaker: of being stupid about tech,
• 1:22:56
  Meredith Whittaker: of being behind the ball on tech.
• 1:22:59
  Meredith Whittaker: And this plays right into patriarchal dynamics.
• 1:23:02
  Meredith Whittaker: Men hate when someone else knows something more than them, in particular if that's a small woman.
• 1:23:08
  Linus Neumann: I think we enjoy it right now.
• 1:23:09
  Meredith Whittaker: Ja, ja, ja. Totally.
• 1:23:11
  Linus Neumann: Wir haben eine gute Zeit.
• 1:23:12
  Meredith Whittaker: Ja. Well, you all are, generally, and I don't want to gender this in such a
• 1:23:19
  Meredith Whittaker: schematic, but there is an ego that can be very, very fragile here.
• 1:23:25
  Meredith Whittaker: And the way I've put it before is it kind of turns uncertain men into yes men.
• 1:23:29
  Meredith Whittaker: Like, they don't want to ask the dumb question. They don't want to be like,
• 1:23:32
  Meredith Whittaker: what's an LLM? What's a server? Like, how does that work?
• 1:23:35
  Meredith Whittaker: And that type of insecurity, The fear of being behind, the fear of being called
• 1:23:40
  Meredith Whittaker: technically unsophisticated or hampering progress or putting your finger on
• 1:23:46
  Meredith Whittaker: the scales of science, look the Nobels, how could you stand in the way of all this progress?
• 1:23:50
  Meredith Whittaker: I think really gives the upper hand to the companies and those who have an interest
• 1:23:57
  Meredith Whittaker: in creating products and growth and domination via these technologies.
• 1:24:04
  Meredith Whittaker: Because people really don't want to challenge them because challenging their
• 1:24:09
  Meredith Whittaker: dominance or their plans gets conflated with somehow being anti-science or being
• 1:24:14
  Meredith Whittaker: stupid about tech or not being smart enough to have a position on a topic.
• 1:24:19
  Meredith Whittaker: And I think that's something because I kind of came up through Google asking
• 1:24:22
  Meredith Whittaker: every dumb question in the book because I had no I didn't come from that world.
• 1:24:26
  Meredith Whittaker: Right. So I had to ask, like, how does a computer work? Right.
• 1:24:29
  Meredith Whittaker: I'm like, I'm like, can someone diagram what a function is? I don't know any of this stuff. Right.
• 1:24:35
  Meredith Whittaker: But I kind of I think I have a sensitivity to that because I also I remember feeling it.
• 1:24:40
  Meredith Whittaker: I remember people being mean about it. Like if I didn't, you know,
• 1:24:43
  Meredith Whittaker: like back in the day when I was trying to learn this stuff.
• 1:24:46
  Meredith Whittaker: And I think that, you know, a discourse that collapses scientific progress into kind of.
• 1:24:54
  Meredith Whittaker: The success of a handful of tech companies preys on that type of insecurity
• 1:25:00
  Meredith Whittaker: and has created an environment in which people have no idea what AI is and are
• 1:25:04
  Meredith Whittaker: still professing boldly on how to regulate it.
• 1:25:10
  Tim Pritlove: So I read it as you think that the European positions might be slightly under-informed
• 1:25:17
  Tim Pritlove: and probably not well thought out in the current situation.
• 1:25:22
  Meredith Whittaker: I should be clear. I don't think that what I was describing and the politics
• 1:25:26
  Meredith Whittaker: of intellectual shame are not unique to Europe, but I think are in Europe as well.
• 1:25:32
  Meredith Whittaker: And particularly folks who feel like, you know, the Americans beat us, we got to get ahead.
• 1:25:37
  Meredith Whittaker: Right. I think where I see the European position being most, let's say,
• 1:25:44
  Meredith Whittaker: under-informed or perhaps just in some cases pernicious is in the chat controls
• 1:25:50
  Meredith Whittaker: regulation and the desire,
• 1:25:53
  Meredith Whittaker: the apex of magical thinking, Which is, you know, let's rename a backdoor client-side
• 1:25:59
  Meredith Whittaker: scanning and then let's mandate scanning everyone's private messages,
• 1:26:04
  Meredith Whittaker: comparing what's in those messages against some database of permissible or impermissible
• 1:26:08
  Meredith Whittaker: content and then, you know, taking action on those, you know,
• 1:26:12
  Meredith Whittaker: in the name of protecting children,
• 1:26:14
  Meredith Whittaker: which is the justification during this instantiation of the crypto wars.
• 1:26:20
  Tim Pritlove: Let's stick to this topic, because it's still an ongoing battle right now.
• 1:26:26
  Tim Pritlove: We are more or less talking about this in every of our shows.
• 1:26:30
  Tim Pritlove: And yeah, it's still totally unclear what's going to come out of this.
• 1:26:35
  Tim Pritlove: How do you see this discussion evolve?
• 1:26:38
  Meredith Whittaker: Well, I don't, you know, I see this as an ongoing power struggle, right?
• 1:26:44
  Meredith Whittaker: Between who? Well, between, like this is not a misunderstanding.
• 1:26:51
  Meredith Whittaker: I think a lot of the people pushing for this understand that backdoors are dangerous
• 1:26:56
  Meredith Whittaker: and understand that, you know, the pretext is flimsy, but that, you know.
• 1:27:03
  Meredith Whittaker: Asymmetric power constitutes itself in part through information asymmetry.
• 1:27:09
  Meredith Whittaker: And there's a deep discomfort that dates back to 1976 when Diffie Hellman were
• 1:27:14
  Meredith Whittaker: trying to publish their paper introducing public cryptography and the U.S.
• 1:27:18
  Meredith Whittaker: Government was trying to suppress it, trying to say, don't publish this, right?
• 1:27:22
  Meredith Whittaker: And then, you know, but databases weren't quite big enough, networks weren't
• 1:27:25
  Meredith Whittaker: quite big enough or ubiquitous enough for it to matter, matter.
• 1:27:28
  Meredith Whittaker: But they were already looking at like, oh, shit, we don't want this in the public,
• 1:27:32
  Meredith Whittaker: right? And then you go through the 90s and there's the Clipper chip and key
• 1:27:36
  Meredith Whittaker: escrow and you have Stuart Baker writing in Wired magazine, like PGP is just
• 1:27:41
  Meredith Whittaker: for terrorists. We have proof.
• 1:27:43
  Meredith Whittaker: No, PGP is for pedophiles, right? Which really echoes what we're hearing now.
• 1:27:47
  Meredith Whittaker: Right? Like who even has a computer in 1994, I believe, when this op-ed is written?
• 1:27:52
  Meredith Whittaker: And then we have Post 9-11 and then it's like, actually, PGP is for terrorists, right?
• 1:27:56
  Meredith Whittaker: And encryption is for terrorists. All the while, our dependency on digital infrastructures
• 1:28:01
  Meredith Whittaker: for communications is growing and growing and growing.
• 1:28:03
  Meredith Whittaker: Our dependency on digital infrastructures generally is growing.
• 1:28:06
  Meredith Whittaker: And the need for encryption to protect commerce becomes existential to the Internet industry.
• 1:28:12
  Meredith Whittaker: And then what do you do about communications, right? And I think this has been
• 1:28:18
  Meredith Whittaker: an anxiety that is pervasive among those who, you know, law enforcement,
• 1:28:22
  Meredith Whittaker: governments, whoever, who feel that they need to constitute their power via
• 1:28:26
  Meredith Whittaker: information asymmetry.
• 1:28:28
  Meredith Whittaker: And any encryption that protects people, not just commerce, is a threat to that, right?
• 1:28:33
  Meredith Whittaker: And so what I don't see is that we're going to win an argument,
• 1:28:36
  Meredith Whittaker: right, or that we're going to win this via strength of argument.
• 1:28:40
  Meredith Whittaker: I do think we can fight and we, you know, I think, I think we're in a position
• 1:28:43
  Meredith Whittaker: now where we're seeing, we're seeing chat controls, you know,
• 1:28:47
  Meredith Whittaker: I believe Hungary just tried to raise it and didn't get the support.
• 1:28:51
  Meredith Whittaker: There was the Belgian proposal a few months ago, also didn't get the support at the last minute.
• 1:28:57
  Meredith Whittaker: And we just had the Dutch law enforcement authorities writing a memo to the
• 1:29:01
  Meredith Whittaker: government there saying, yo, don't support this.
• 1:29:04
  Meredith Whittaker: You're talking about a very dangerous backdoor that would undermine Dutch cybersecurity,
• 1:29:09
  Meredith Whittaker: right? At the same time, we have reporting in the Wall Street Journal...
• 1:29:13
  Meredith Whittaker: Es gibt einen Receit für was, was all of us should have suspected all along,
• 1:29:17
  Meredith Whittaker: ist, dass die Backdoors, die in den US-Telecommunications Infrastruktur,
• 1:29:22
  Meredith Whittaker: für Government Intercept, haben wir von China-Intelligence und vielleicht anderen.
• 1:29:29
  Meredith Whittaker: Ich denke, an dieser Moment, wir haben ein vieles...
• 1:29:34
  Meredith Whittaker: Die Facts sind auf unserer Seite, und die Facts sind auf der Seite,
• 1:29:39
  Meredith Whittaker: ist permeatet in diese Diskussion, and making it harder and harder for them
• 1:29:43
  Meredith Whittaker: to push it forward in the European Commission.
• 1:29:46
  Linus Neumann: But that usually means they just do another attempt next year.
• 1:29:49
  Meredith Whittaker: Exactly. And that's why I think we're not going to win. There's going to be
• 1:29:53
  Meredith Whittaker: another pretext if we win this one, right?
• 1:29:56
  Meredith Whittaker: There's going to be another angle if we win this one.
• 1:29:59
  Meredith Whittaker: We just have to keep building our muscle to sustain this fight probably forever,
• 1:30:03
  Meredith Whittaker: because I don't think the will to power is going away.
• 1:30:06
  Meredith Whittaker: I think they're just going to keep trying to rearrange the reasoning.
• 1:30:10
  Tim Pritlove: Aber wie geht es? Wenn Sie sagen, dass die Strecke der Argument nicht genug ist, ist es nicht genug?
• 1:30:15
  Meredith Whittaker: Ich mache Yoga every day.
• 1:30:16
  Tim Pritlove: Does es helfen?
• 1:30:17
  Meredith Whittaker: Ich meine, yes, es hilft.
• 1:30:18
  Tim Pritlove: Ich meine, in terms of politische Diskussionen.
• 1:30:24
  Meredith Whittaker: In terms of politische Diskussionen, es hilft, dass wir richtig sind.
• 1:30:29
  Meredith Whittaker: We bring in, there's a huge amount of evidence that a lot of people haven't
• 1:30:33
  Meredith Whittaker: seen in these political discussions.
• 1:30:36
  Meredith Whittaker: I think we're on the, our side has been on the back foot for a while.
• 1:30:41
  Meredith Whittaker: There has been just in civil society, there's been cutting funding to privacy
• 1:30:46
  Meredith Whittaker: advocacy has happened since around, there's a sort of a history here.
• 1:30:53
  Meredith Whittaker: I think there's a move toward tech accountability happened after the 2016 election.
• 1:30:59
  Meredith Whittaker: There's the Cambridge Analytica scandal. There's all of this.
• 1:31:02
  Meredith Whittaker: And it's like, okay, we need to hold tech accountable.
• 1:31:05
  Meredith Whittaker: And then there are a number of – the way to hold tech accountable is to attack
• 1:31:10
  Meredith Whittaker: the business model, is my view.
• 1:31:12
  Meredith Whittaker: But there aren't that many pieces of legislation or proposals that actually do that.
• 1:31:17
  Meredith Whittaker: Many of them sort of use the wrapping and the language of accountability.
• 1:31:22
  Meredith Whittaker: But are actually just expanding surveillance, right?
• 1:31:26
  Meredith Whittaker: It's like, we're going to hold them accountable, so we need a database,
• 1:31:28
  Meredith Whittaker: so we need to know who's logging into websites so we can find the bad guy.
• 1:31:34
  Meredith Whittaker: We need to know what's in your messaging so that we can make sure that these
• 1:31:40
  Meredith Whittaker: tech companies aren't allowing crime on their platforms, et cetera, et cetera.
• 1:31:43
  Meredith Whittaker: So it was basically a hijacking of this, in many cases, kind of righteous moment
• 1:31:48
  Meredith Whittaker: where people recognize that this business model was pretty harmful,
• 1:31:51
  Meredith Whittaker: to fulfill the wishes that have been pervasive since well before then.
• 1:31:56
  Meredith Whittaker: At the same time that we're seeing privacy advocacy and a lot of those,
• 1:32:00
  Meredith Whittaker: you know, a lot of the things Linus, you and I had been doing for a long time,
• 1:32:04
  Meredith Whittaker: receiving less and less support and sort of, you know, out of the limelight.
• 1:32:08
  Meredith Whittaker: And so I think it was in that environment that things like chat controls,
• 1:32:13
  Meredith Whittaker: that things like the online safety bill and other, you know,
• 1:32:17
  Meredith Whittaker: paradigmatic examples of this, you client-side scanning to save children meme,
• 1:32:24
  Meredith Whittaker: grew up and then one of the reasons I was,
• 1:32:28
  Meredith Whittaker: There are many, many reasons I
• 1:32:29
  Meredith Whittaker: decided to move from being on the board of Signal to full-time at Signal.
• 1:32:34
  Meredith Whittaker: One of them was that I saw this moment and I realized there weren't that many
• 1:32:41
  Meredith Whittaker: people fighting it and that one of the things that I could bring was a staunch
• 1:32:48
  Meredith Whittaker: willingness to fight it.
• 1:32:50
  Tim Pritlove: And how do you do that? I mean, can you walk us through a day?
• 1:32:55
  Meredith Whittaker: I open up my laptop. Ja, um, um, well, we, you know, it's not,
• 1:33:02
  Meredith Whittaker: obviously, it's not just me, nothing like this is a singular thing.
• 1:33:06
  Meredith Whittaker: We work with a pretty broad coalition of folks. I'm sure, you know,
• 1:33:09
  Meredith Whittaker: many of your friends, many listeners, perhaps, are part of that.
• 1:33:13
  Meredith Whittaker: Um, Signal doesn't have a policy arm. It's a very kind of lean,
• 1:33:18
  Meredith Whittaker: targeted, pretty senior organization.
• 1:33:21
  Meredith Whittaker: But we do work with people around the globe EDRI and the EU a number of other
• 1:33:26
  Meredith Whittaker: organizations to keep tabs on what's happening we also are in a good position.
• 1:33:33
  Meredith Whittaker: We're a non-profit we are very committed to rigorous communication so we don't
• 1:33:42
  Meredith Whittaker: have a history of hyper-marketing we don't do hyper-marketing now and so we're
• 1:33:47
  Meredith Whittaker: very careful when we make a claim When we make a statement,
• 1:33:51
  Meredith Whittaker: we're backing that with citations. It's accurate.
• 1:33:53
  Meredith Whittaker: We're really marshalling the technical knowledge and prowess that we have.
• 1:33:58
  Meredith Whittaker: I almost think of it as clarifying the record. If there's a report that says
• 1:34:04
  Meredith Whittaker: client-side scanning is actually safe, we know it's not safe.
• 1:34:08
  Meredith Whittaker: Okay, well, there's an academic coalition that has written this letter.
• 1:34:11
  Meredith Whittaker: Signal can write a letter. We can begin to put a bit more weight on the scales
• 1:34:15
  Meredith Whittaker: that have been fairly light, given the dynamics I just outlined.
• 1:34:22
  Meredith Whittaker: And then I do media, I do public speaking.
• 1:34:25
  Meredith Whittaker: I think a lot about how to tell this story in a way that isn't boring or alienating
• 1:34:33
  Meredith Whittaker: for regular people, particularly because the story on the other side is so arresting.
• 1:34:37
  Meredith Whittaker: It's like we have to save children from abuse.
• 1:34:41
  Meredith Whittaker: And that every one of us it like hits you in the heart, right?
• 1:34:45
  Meredith Whittaker: Like myself, right? Like my amygdala is activated.
• 1:34:49
  Meredith Whittaker: I, you know, suddenly I just want to do something. I want to help,
• 1:34:53
  Meredith Whittaker: you know, give me the thing to do. How do we do that, right?
• 1:34:56
  Meredith Whittaker: And then sitting across from that and being like, well, let me tell you about
• 1:35:00
  Meredith Whittaker: a one-way function, right? Like you can't, that's not.
• 1:35:05
  Meredith Whittaker: That's not going to work, right? And so like, how do you,
• 1:35:10
  Meredith Whittaker: How do you enter into that debate in a way that isn't dismissing the very grim
• 1:35:15
  Meredith Whittaker: and real problem that is being evoked and make it clear that the solution to
• 1:35:21
  Meredith Whittaker: that problem that is being presented will not solve that problem, one,
• 1:35:24
  Meredith Whittaker: and two, will cause drastically worse problems for many people around the world?
• 1:35:30
  Meredith Whittaker: And that's the task at hand right now.
• 1:35:32
  Tim Pritlove: So basically the discussion is
• 1:35:35
  Tim Pritlove: led by pointing the other side to like the infeasibility of the approach.
• 1:35:41
  Meredith Whittaker: Well, the infeasibility, the danger of the approach,
• 1:35:45
  Meredith Whittaker: a lot of evidence around the infeasibility of the approach that is either kind
• 1:35:50
  Meredith Whittaker: of willfully ignored or just not understood,
• 1:35:53
  Meredith Whittaker: and then figuring out how we explain that without being either accidentally
• 1:36:02
  Meredith Whittaker: or genuinely callous about the concerns that have brought people to the table.
• 1:36:05
  Tim Pritlove: Then they will say, but we have to do something.
• 1:36:08
  Meredith Whittaker: Well, how about funding social services?
• 1:36:12
  Meredith Whittaker: How about, you know, what do you do? And I mean, like, if we're going to go
• 1:36:17
  Meredith Whittaker: there, we're going to go there.
• 1:36:19
  Meredith Whittaker: Prince Andrew's walking around.
• 1:36:20
  Tim Pritlove: No, you're suggesting to make the world better.
• 1:36:23
  Meredith Whittaker: Jimmy Savile's walking around. You know, like what are the infrastructures in
• 1:36:27
  Meredith Whittaker: place to make sure that when children are going through this,
• 1:36:29
  Meredith Whittaker: they're believed, they're protected.
• 1:36:32
  Meredith Whittaker: You know, what happens when it's your priest? What happens when it's your teacher?
• 1:36:35
  Meredith Whittaker: What happens when it's your brother, right? Like these are the questions that
• 1:36:38
  Meredith Whittaker: are really hard to look in the face because they implicate social pathologies
• 1:36:43
  Meredith Whittaker: and interpersonal relationships and power dynamics that are really,
• 1:36:47
  Meredith Whittaker: really difficult and often relate to emotionally challenging factors outside
• 1:36:54
  Meredith Whittaker: of that or people's past experiences or what have you.
• 1:36:57
  Meredith Whittaker: So you're going right into very traumatic subjects.
• 1:37:02
  Meredith Whittaker: But I don't think we can have that conversation without having a real conversation.
• 1:37:06
  Meredith Whittaker: And then when you begin to pull back the layers there, you say,
• 1:37:10
  Meredith Whittaker: oh, well, the UK has been pushing for client-side scanning as a remediation to child abuse.
• 1:37:18
  Meredith Whittaker: But the UK government in 2023 funded social services at 7% of the amount recommended.
• 1:37:28
  Meredith Whittaker: Right? So like there's, you know, the roofs on the schools in the UKs are collapsing.
• 1:37:32
  Meredith Whittaker: Like, you know, there isn't support for this.
• 1:37:35
  Meredith Whittaker: And then if you, you know, if you look at, and I don't have public numbers to
• 1:37:39
  Meredith Whittaker: share, but I've had a number of personal conversations.
• 1:37:42
  Meredith Whittaker: Okay, well, how many law enforcement people, how many people are tasked with
• 1:37:47
  Meredith Whittaker: actually sort of pursuing the,
• 1:37:49
  Meredith Whittaker: you know, like the criminality that may be reported via online imagery?
• 1:37:55
  Meredith Whittaker: In some cases, it's two. In one case, it's two in one country. Two people, right?
• 1:38:02
  Meredith Whittaker: So, like, you're not actually, like, if you begin to map this,
• 1:38:05
  Meredith Whittaker: what you see is a story that does not add up.
• 1:38:08
  Meredith Whittaker: And you see, like, what, you know, and this is where I get enraged.
• 1:38:12
  Meredith Whittaker: Because I'm like, you are fucking trading...
• 1:38:15
  Meredith Whittaker: On children's pain, to get your backdoor, whatever the fuck you want,
• 1:38:21
  Meredith Whittaker: pretending that you're solving it, so taking up the space for actual solutions
• 1:38:24
  Meredith Whittaker: that could actually help real children who are suffering now,
• 1:38:29
  Meredith Whittaker: and turning no attention to every glaring problem in this massive list,
• 1:38:35
  Meredith Whittaker: which is pretty obvious, even for me, and I'm not an expert here,
• 1:38:41
  Meredith Whittaker: I've just sort of sifted through this.
• 1:38:42
  Meredith Whittaker: So I think that's the dynamic we're walking into.
• 1:38:47
  Tim Pritlove: They might only have two people, but at some point in time they might have doubled,
• 1:38:54
  Tim Pritlove: by adding another one.
• 1:38:55
  Linus Neumann: I agree that, I mean, it is quite telling how much emphasis is being laid on,
• 1:39:01
  Linus Neumann: hey, we really need client-side scanning and then the world is going to be safe.
• 1:39:06
  Linus Neumann: And, you know, if you say, well, how about we, you know, we fund support or
• 1:39:10
  Linus Neumann: any kind of prevention, preventive activities in um social care it's like yeah.
• 1:39:16
  Meredith Whittaker: Yeah yeah well sorry we use the prefix online so that's not our,
• 1:39:23
  Meredith Whittaker: um and i think it's also like there's something people gravitate to the abstraction
• 1:39:30
  Meredith Whittaker: right if this is online child abuse then we don't have to deal with it in our
• 1:39:34
  Meredith Whittaker: real lives it becomes an abstraction that we can almost blame on the same platforms
• 1:39:38
  Meredith Whittaker: that have been so unaccountable we can blame it as an internet phenomenon, not a phenomenon.
• 1:39:43
  Meredith Whittaker: And like, oh wait, our church doesn't have the infrastructure to actually deal
• 1:39:46
  Meredith Whittaker: with this in a humane way, right? And I think that's a dynamic that we're also seeing here.
• 1:39:50
  Linus Neumann: This is, by the way, one thing I find so interesting about Signal as a secure messenger.
• 1:39:58
  Linus Neumann: Well, it has become mainstream, but it has also managed to maintain a reputation
• 1:40:05
  Linus Neumann: of goodness, right? I mean, saying like, okay, text me on signals.
• 1:40:12
  Linus Neumann: Oh yeah, that's the secure messenger, blue symbol, looks nice,
• 1:40:15
  Linus Neumann: you know, very friendly user interface.
• 1:40:18
  Linus Neumann: Or it would be like, let's text on 3MAR, be like, oh, that's the complicated black one.
• 1:40:26
  Linus Neumann: Well, how about Telegram? And that's like, okay, that's a completely different end of the internet.
• 1:40:32
  Linus Neumann: And that makes me think of the curious case of Pavel Durov being detained in
• 1:40:40
  Linus Neumann: France and apparently at least charged because they refused to cooperate in numerous cases.
• 1:40:53
  Linus Neumann: Why do you dare to come to Europe, Meredith?
• 1:40:57
  Meredith Whittaker: Well, I'm a brave person, Linus. Aber, zu sagen, ich denke,
• 1:41:04
  Meredith Whittaker: This is one of the places where more public education is necessary because Telegram
• 1:41:10
  Meredith Whittaker: is actually, you said the other end of the internet, it is very,
• 1:41:14
  Meredith Whittaker: very, very different from Signal.
• 1:41:16
  Meredith Whittaker: So Telegram is a social media platform. It allows mass broadcasts to millions
• 1:41:21
  Meredith Whittaker: of people. You can go viral on Telegram.
• 1:41:23
  Meredith Whittaker: You can find strangers on Telegram via directories.
• 1:41:26
  Meredith Whittaker: There's a near-me feature that will geolocate things happening near you,
• 1:41:32
  Meredith Whittaker: all sorts of things that are not private, are not secure,
• 1:41:37
  Meredith Whittaker: are regulated completely differently from private and secure communications
• 1:41:42
  Meredith Whittaker: like Signal, which is solely a private and secure interpersonal communications app.
• 1:41:48
  Linus Neumann: What about Signal Stories, though?
• 1:41:50
  Meredith Whittaker: Signal Stories don't go viral. If I sent all the people in my contact list one
• 1:41:56
  Meredith Whittaker: by one a photo or something...
• 1:41:59
  Linus Neumann: I don't get your stories.
• 1:42:00
  Meredith Whittaker: They're so cute, Linus. I wish, like, it's, you know.
• 1:42:04
  Tim Pritlove: I hate it.
• 1:42:04
  Linus Neumann: I mean, that feature was implemented. I was like, okay, where can I turn it off?
• 1:42:07
  Meredith Whittaker: I'm sorry. Well, you're all missing my stories is all I'm saying,
• 1:42:10
  Meredith Whittaker: and they are pretty good.
• 1:42:12
  Tim Pritlove: Have you sent us some? No.
• 1:42:14
  Meredith Whittaker: Well, have you activated them to check? But we do let you deactivate them forever
• 1:42:21
  Meredith Whittaker: and never bother you about them, which is, you know, part of the way that I
• 1:42:24
  Meredith Whittaker: think we maintain our reputation for not being shitty is we try to literally not be shitty.
• 1:42:29
  Tim Pritlove: Yeah, thanks.
• 1:42:29
  Meredith Whittaker: Right? And when we're designing Signal, we're actually very,
• 1:42:35
  Meredith Whittaker: very careful not to be a social media platform.
• 1:42:37
  Meredith Whittaker: We think about that in the design phase so everything we do can be as encrypted as possible.
• 1:42:43
  Meredith Whittaker: So that we don't know anything about you or as close to zero about the people
• 1:42:49
  Meredith Whittaker: who use Signal as possible.
• 1:42:50
  Meredith Whittaker: And what we do know is we can say, yes, this phone number did sign up for a
• 1:42:55
  Meredith Whittaker: Signal account. We know when that phone number signed up for a signal count
• 1:42:58
  Meredith Whittaker: and we know last time they accessed it.
• 1:43:02
  Meredith Whittaker: Um, but we don't, you know, we, we would like to even not know that if it were possible.
• 1:43:08
  Meredith Whittaker: On the other hand, Telegram is a social media platform, which retains huge amounts of data,
• 1:43:14
  Meredith Whittaker: has a duty to under law to cooperate in turning over that data and was search
• 1:43:20
  Meredith Whittaker: functions that has search functions, has, you know, directories so you can find new things.
• 1:43:25
  Meredith Whittaker: So it's a, it's a very different beast. And I think, one, because Durov has, you know, been very.
• 1:43:33
  Meredith Whittaker: I'm trying for a diplomatic word, like, has made statements that are not supported
• 1:43:39
  Meredith Whittaker: by fact around Telegram being private and secure and kind of taken on this,
• 1:43:44
  Meredith Whittaker: you know, like yeoman's defender of free speech and privacy position.
• 1:43:48
  Meredith Whittaker: People often think Telegram is private and secure because it has a DMs feature, right?
• 1:43:53
  Meredith Whittaker: But, you know, Signal is just private and secure. So the TLDR on that is,
• 1:43:59
  Meredith Whittaker: there's really no danger for Signal here because we are very,
• 1:44:03
  Meredith Whittaker: very far away from Telegram.
• 1:44:05
  Meredith Whittaker: And we have set ourselves up so that one, such cooperation isn't required.
• 1:44:10
  Meredith Whittaker: And two, such cooperation is not possible.
• 1:44:13
  Meredith Whittaker: Because we literally, like you could put a gun to my head, I don't have that
• 1:44:16
  Meredith Whittaker: data. Whereas Telegram has servers and servers and servers full of that data.
• 1:44:22
  Tim Pritlove: Leaving out Signal completely now, what do you think happened?
• 1:44:26
  Tim Pritlove: Pawelldorf was put into custody. He's now free on bail.
• 1:44:31
  Tim Pritlove: And France talked to him.
• 1:44:35
  Meredith Whittaker: I have no idea. I mean, this is like...
• 1:44:38
  Linus Neumann: Didn't he send you like a telegram message?
• 1:44:40
  Meredith Whittaker: I haven't checked.
• 1:44:43
  Tim Pritlove: You missed his story.
• 1:44:44
  Linus Neumann: Yeah.
• 1:44:45
  Meredith Whittaker: I mean, this is like overlaid the French legal system.
• 1:44:48
  Meredith Whittaker: I am not a lawyer, especially not in France. Some of the vagaries of their legal
• 1:44:54
  Meredith Whittaker: system in which, you know,
• 1:44:56
  Meredith Whittaker: any judge can open an investigation and the basis for the charges will not be
• 1:45:01
  Meredith Whittaker: known until, you know, trial.
• 1:45:03
  Meredith Whittaker: And we're looking at years and years until then, with me not speaking French
• 1:45:08
  Meredith Whittaker: well at all with like weird translation, you know, like, so I want to stay away
• 1:45:13
  Meredith Whittaker: from speculating there.
• 1:45:14
  Meredith Whittaker: But what it looks like based on the charges that were released in the press
• 1:45:18
  Meredith Whittaker: release is that it was, you know, failure to comply with requests for data and
• 1:45:23
  Meredith Whittaker: then, you know, kind of a handful of other charges added on that aren't as severe as those.
• 1:45:28
  Tim Pritlove: So does Signal get these requests too?
• 1:45:31
  Meredith Whittaker: You can go to Signal.org slash Big Brother and every request that we have been
• 1:45:38
  Meredith Whittaker: forced to comply with because we fight them and have unsealed are posted there
• 1:45:43
  Meredith Whittaker: showing exactly how close to no data we are able to turn over and showing,
• 1:45:50
  Meredith Whittaker: and I think this is interesting for some of your listeners probably,
• 1:45:53
  Meredith Whittaker: You see what the law enforcement agencies in these requests are requesting,
• 1:45:58
  Meredith Whittaker: and it's often huge lists,
• 1:46:01
  Meredith Whittaker: massive amounts of data, which gives you a sense of just how much data surveillance,
• 1:46:07
  Meredith Whittaker: like a telegram or another platform,
• 1:46:11
  Meredith Whittaker: is commonly able to provide that Signal is not.
• 1:46:15
  Linus Neumann: Oh, so it's actually every single request. I thought it would maybe be aggregated.
• 1:46:24
  Meredith Whittaker: Yeah, no, it's PDFs of the request. It's not a transparency report, it's transparency.
• 1:46:35
  Tim Pritlove: I like that.
• 1:46:38
  Meredith Whittaker: Although it's just the ones that we can unseal, so we do have to go through that.
• 1:46:41
  Linus Neumann: I would assume there are probably some with gag orders, right? be able to say?
• 1:46:46
  Meredith Whittaker: There's no gag order on me being able to say.
• 1:46:49
  Meredith Whittaker: And there are some, but that's the fight to unseal it. And what I don't recall
• 1:46:54
  Meredith Whittaker: the answer for right now is whether we're in one of those fights right now or not.
• 1:46:58
  Meredith Whittaker: But I'd have to check with my friends at the ACLU.
• 1:47:05
  Tim Pritlove: So Signal as a company, how does it work?
• 1:47:10
  Meredith Whittaker: It's a non-profit. So we're funded by donations only
• 1:47:13
  Meredith Whittaker: only yeah um and we
• 1:47:16
  Meredith Whittaker: are thinking about in the future maybe having a
• 1:47:19
  Meredith Whittaker: paid tier for some features something like backups encrypted
• 1:47:22
  Meredith Whittaker: backups which we're you know building right now um you know could we charge
• 1:47:28
  Meredith Whittaker: people for media storage or other expensive features but um you know that would
• 1:47:33
  Meredith Whittaker: be in addition to donations and squarely within the non-profit structure that
• 1:47:38
  Meredith Whittaker: keeps us safe from pressure to surveil.
• 1:47:42
  Tim Pritlove: And how do you get your talent? How do you get people to work for Signal?
• 1:47:48
  Meredith Whittaker: We pay very well. And we are, I mean, it's a really cool mission, right?
• 1:47:53
  Meredith Whittaker: So imagine the jobs in tech are kind of depressing in many cases.
• 1:47:59
  Meredith Whittaker: Not everyone wants to go optimize an ad server and then Signal,
• 1:48:04
  Meredith Whittaker: you know, Signal you can work for core infrastructure for dissent and human
• 1:48:10
  Meredith Whittaker: rights work and journalism around the world that, you know, without which a
• 1:48:15
  Meredith Whittaker: lot of those things would be deeply imperiled.
• 1:48:17
  Meredith Whittaker: Like it's a real cool thing to get to do and support. And we pay well.
• 1:48:23
  Linus Neumann: So not only do you get respected for your six-digit salary, but for doing good
• 1:48:29
  Linus Neumann: thing, earning that money.
• 1:48:30
  Meredith Whittaker: Es ist die original Silicon Valley dream.
• 1:48:34
  Tim Pritlove: So ist es der letzte Platz in tech where people are actually happy?
• 1:48:40
  Meredith Whittaker: Well, I would never presume to speak to the consciousness of another person without there.
• 1:48:47
  Meredith Whittaker: But I think, yeah, I am very happy. I think a lot of the people who are at Signal are very happy.
• 1:48:53
  Meredith Whittaker: And I think it's also like we're kind of part of a project. And this is...
• 1:48:59
  Meredith Whittaker: Shows that what we have in tech, what's built in the tech industry is not inevitable.
• 1:49:05
  Meredith Whittaker: There's a series of choices, a series of incentives, a business model that has
• 1:49:09
  Meredith Whittaker: shaped tech into the form we have now, but it does not have to be that way, right?
• 1:49:13
  Meredith Whittaker: Like we can rewrite the stack, we can build alternatives.
• 1:49:16
  Meredith Whittaker: Nonprofits can work, right? We need capital, we need will, we need talent,
• 1:49:21
  Meredith Whittaker: we need all of those things.
• 1:49:22
  Meredith Whittaker: But the thing we have now is not inevitable. And I think of Signal as like a
• 1:49:26
  Meredith Whittaker: keystone species in the ecosystem, Kind of like, you know, like setting the
• 1:49:32
  Meredith Whittaker: bar, kind of regulating the rest, right? Like, you know, you can have privacy.
• 1:49:36
  Meredith Whittaker: You can have, you know, the right to private communications.
• 1:49:39
  Meredith Whittaker: You can subsist outside of this paradigm.
• 1:49:43
  Meredith Whittaker: And I think the future I want is that it's not just Signal, right?
• 1:49:47
  Meredith Whittaker: There are many, many other organizations and efforts sort of doing it differently,
• 1:49:53
  Meredith Whittaker: rejecting that paradigm, you know, drawing in capital there and,
• 1:49:57
  Meredith Whittaker: you know, away from the other place.
• 1:49:59
  Meredith Whittaker: Und beginnt zu marscheln die type of political will that is often very shallow
• 1:50:04
  Meredith Whittaker: like the 500 million AI fund but marscheln it for something that is actually
• 1:50:09
  Meredith Whittaker: substantive and is actually,
• 1:50:12
  Meredith Whittaker: making the kind of change to the tech ecosystem that I think we need to have a livable world.
• 1:50:18
  Tim Pritlove: You mean not only a model for other communication companies but also a model
• 1:50:23
  Tim Pritlove: for any kind of technology company?
• 1:50:28
  Meredith Whittaker: Ja, well, I mean, we build a communications app, but we rely on telecom networks.
• 1:50:37
  Meredith Whittaker: We rely on server infrastructure. We rely on core libraries.
• 1:50:43
  Tim Pritlove: That's not what I meant. I mean, I understood you that you think that the modus
• 1:50:49
  Tim Pritlove: operandi of Signal as a company might be something that other companies could also leverage and do.
• 1:51:00
  Tim Pritlove: It's not only limited to some much-needed device.
• 1:51:04
  Meredith Whittaker: I do think that there's a model there. I think I'm interested right now in researching
• 1:51:11
  Meredith Whittaker: hybrid structures and tandem structures.
• 1:51:13
  Meredith Whittaker: Are there sort of for-profit,
• 1:51:17
  Meredith Whittaker: Are there areas of tech that aren't driven by surveillance?
• 1:51:26
  Meredith Whittaker: Are there ways you could fund nonprofits, fund some of this core infrastructure,
• 1:51:31
  Meredith Whittaker: libraries and other things that have been languishing for decades?
• 1:51:35
  Meredith Whittaker: How do you revitalize that? And then are there, you know, are there ways to
• 1:51:41
  Meredith Whittaker: build truly independent infrastructure outside of the, you know,
• 1:51:44
  Meredith Whittaker: three companies, five platforms model that, like, I think it's just clearly
• 1:51:49
  Meredith Whittaker: critically dangerous at this point.
• 1:51:53
  Linus Neumann: So, when it's about building infrastructures in our hands, that's not going
• 1:51:59
  Linus Neumann: to be easier in an AI world, right?
• 1:52:03
  Linus Neumann: Where it's model data that we need, huge investments into these models just
• 1:52:09
  Linus Neumann: for training, then for operating them.
• 1:52:12
  Linus Neumann: Do you see any future for this whole AI thing in users' hands,
• 1:52:19
  Linus Neumann: in our hands, serving our actual privacy needs and let's say private and business needs?
• 1:52:27
  Meredith Whittaker: Well, I think my answer to that is that that future will rely on laying an independent
• 1:52:35
  Meredith Whittaker: infrastructural bedrock and actually transforming some of the way we govern
• 1:52:40
  Meredith Whittaker: and think about digital technology generally,
• 1:52:44
  Meredith Whittaker: Including being really attentive to things like how is data created?
• 1:52:50
  Meredith Whittaker: Who gets to decide what data we use to reflect our complex lives and realities?
• 1:52:56
  Meredith Whittaker: Who gets to decide how patterns in that data are made sense of?
• 1:53:01
  Meredith Whittaker: What analysis is done to that data? And then what we do with the sense we make
• 1:53:06
  Meredith Whittaker: of it, right? What decisions we make, right? And so...
• 1:53:10
  Meredith Whittaker: We do all that, we transform what AI is and what it means.
• 1:53:14
  Meredith Whittaker: Because you're no longer just scraping all the detritus off the stupid web,
• 1:53:19
  Meredith Whittaker: which was deposited or created via this surveillance business model,
• 1:53:23
  Meredith Whittaker: packaging that in an LLM and calling that intelligence, right?
• 1:53:26
  Meredith Whittaker: You're actually having to grapple with the epistemic process by which data becomes
• 1:53:32
  Meredith Whittaker: a proxy for reality, and that proxy shapes our lives and institutions.
• 1:53:38
  Meredith Whittaker: And so I think AI itself, right now we're talking about these massive models, right?
• 1:53:43
  Meredith Whittaker: This laws of scale, this sort of like big American guy dream of the,
• 1:53:49
  Meredith Whittaker: you know, the largest in the world.
• 1:53:52
  Meredith Whittaker: But AI is a, you know, it's a very slippery term. It's not a technical term
• 1:53:56
  Meredith Whittaker: of art. It can apply to many, many different things. And there are small models.
• 1:54:00
  Meredith Whittaker: There are sort of, you know, heterodox approaches. There are expert systems,
• 1:54:03
  Meredith Whittaker: which they're now trying to bolt onto the side of generative systems.
• 1:54:06
  Meredith Whittaker: Because weight, probabilistic answers aren't true, so we need to bolt truth
• 1:54:10
  Meredith Whittaker: back on and we're kind of repeating a lot of the history of AI,
• 1:54:16
  Meredith Whittaker: kind of speedrunning it in the search for a business model.
• 1:54:19
  Meredith Whittaker: So my answer there is that a lot of the things that need to be done to simply
• 1:54:23
  Meredith Whittaker: disarm and draw down the centralized power of these surveillance and infrastructure companies
• 1:54:29
  Meredith Whittaker: are the same things that would need to be done to redefine, in a sense,
• 1:54:34
  Meredith Whittaker: what AI is and our relationship to how truth, how, you know,
• 1:54:42
  Meredith Whittaker: decisions, how, you know, I don't want to use the word truth actually,
• 1:54:45
  Meredith Whittaker: but like how information is made via analyzing data and who gets to control that.
• 1:54:51
  Meredith Whittaker: And I think, you know, my sensitivity to data in that answer comes directly
• 1:54:54
  Meredith Whittaker: from my measurement experience, right?
• 1:54:56
  Meredith Whittaker: Like where you, you know, one upgrade to the Linux kernel across our server
• 1:55:02
  Meredith Whittaker: fleet fundamentally changed.
• 1:55:05
  Meredith Whittaker: The kind of data we were able to create, like how it populated the schema.
• 1:55:10
  Meredith Whittaker: And meant that that data wasn't necessarily fungible with the data collected
• 1:55:15
  Meredith Whittaker: on the older version of the kernel, right?
• 1:55:19
  Meredith Whittaker: And in order to solve that problem, I had to get a guy to go sit with the kernel
• 1:55:23
  Meredith Whittaker: maintainers for like two years to make sure that the update wasn't going to
• 1:55:27
  Meredith Whittaker: fuck up the way we got TCB DOMS, basically.
• 1:55:30
  Meredith Whittaker: Um so like that you know that's and
• 1:55:34
  Meredith Whittaker: that's that's then think about social
• 1:55:37
  Meredith Whittaker: data then think about data that reflects like who gets
• 1:55:39
  Meredith Whittaker: access to resources then think about all of the other things and um
• 1:55:42
  Meredith Whittaker: i think i think it's actually an exciting idea to think on like you know how
• 1:55:48
  Meredith Whittaker: do we how do we create systems where we're much more attentive to that and recognize
• 1:55:52
  Meredith Whittaker: that there is a you know it really matters how those choices are made how those
• 1:55:56
  Meredith Whittaker: you know how data is created who gets a und wie es zu sagen ist, und wie es zu sagen ist.
• 1:56:01
  Linus Neumann: Would you say in that future, so there is these large gen AI models and whatnot,
• 1:56:10
  Linus Neumann: and from other discussions we've had, I know that you probably believe there
• 1:56:15
  Linus Neumann: is a stronger future for specialized models, expert models, not the generative ones.
• 1:56:24
  Linus Neumann: Would that be a prediction of how this whole AI thing is going to evolve?
• 1:56:30
  Linus Neumann: Not from the business model perspective or from the political perspective but
• 1:56:35
  Linus Neumann: from the actual technology and research perspective.
• 1:56:40
  Linus Neumann: Do you think there is still going to be exponential improvement in the Gen AI
• 1:56:46
  Linus Neumann: world or do you think it's now the time for the smaller speedboats?
• 1:56:50
  Meredith Whittaker: Well, I think it's definitely time for smaller speedboats and I want to index
• 1:56:55
  Meredith Whittaker: on that word improvement because if we scratch the surface on some of these large models,
• 1:56:59
  Meredith Whittaker: some of which are generative you begin to realize that a lot of the claims to
• 1:57:04
  Meredith Whittaker: improvement and accuracy are based on really narrow benchmarks and evaluations
• 1:57:11
  Meredith Whittaker: that don't reflect the performance of these models That's.
• 1:57:15
  Linus Neumann: How Germans optimize their diesel engines.
• 1:57:19
  Meredith Whittaker: Exactly So it's there
• 1:57:23
  Meredith Whittaker: are things that Gen AI models can do I don't see them realistically going away
• 1:57:28
  Meredith Whittaker: But I do see the struggle for a market fit that can produce the kinds of returns
• 1:57:35
  Meredith Whittaker: necessary to prop up a massively energy intensive,
• 1:57:42
  Meredith Whittaker: massively infrastructurally intensive, extraordinarily capital intensive industry, right?
• 1:57:49
  Meredith Whittaker: So you have billions of dollars for a training run, just huge amounts of energy
• 1:57:54
  Meredith Whittaker: and effort needed to create a model.
• 1:57:56
  Meredith Whittaker: But okay, who's going to keep paying for a chatbot that's wrong?
• 1:58:02
  Meredith Whittaker: And so I think there is a struggle for market fit.
• 1:58:05
  Meredith Whittaker: I think you see this with things like Microsoft Recall where they pushed to
• 1:58:09
  Meredith Whittaker: implement this. I don't know.
• 1:58:13
  Meredith Whittaker: Microsoft Recall was supposed to ship with Windows 11.
• 1:58:18
  Tim Pritlove: Es ist aber nicht auf der Default.
• 1:58:21
  Meredith Whittaker: Ja, wir haben das kleine Sache. Aber es ist ein AI-System, whose value proposition
• 1:58:26
  Meredith Whittaker: ist, dass es will remember everything you were doing on your device for the last N months.
• 1:58:32
  Linus Neumann: Das ist ein nice value proposition.
• 1:58:34
  Meredith Whittaker: Ich weiß, value to your boss, maybe. Und so you can type, oh,
• 1:58:40
  Meredith Whittaker: ich will find this browser, ich will find this thing, und es will...
• 1:58:45
  Meredith Whittaker: I get it for you. I obviously don't use it. And how does it remember is really the key here.
• 1:58:52
  Meredith Whittaker: It remembers because it's taking screenshots of your device every five seconds,
• 1:58:57
  Meredith Whittaker: creating a library of those screenshots and accessing those as the data on which
• 1:59:02
  Meredith Whittaker: it is able to claim intelligent memory.
• 1:59:05
  Linus Neumann: That is not an efficient way to approach the problem.
• 1:59:10
  Meredith Whittaker: And I don't need to know that I was doomscrolling. Like, you know,
• 1:59:13
  Meredith Whittaker: that's not a proud moment of memory for me.
• 1:59:16
  Meredith Whittaker: So, yeah, I mean, and to me what that says, like, that's not a very useful purpose.
• 1:59:21
  Meredith Whittaker: It's probably going to be marketed to enterprises for, you know,
• 1:59:24
  Meredith Whittaker: worker surveillance is my guess.
• 1:59:26
  Meredith Whittaker: But it shows that Microsoft is really trying to find a market for this, right?
• 1:59:31
  Meredith Whittaker: Because they clearly circumvented their QA process.
• 1:59:34
  Meredith Whittaker: They clearly circumvented their security evaluation. There was a lot of things
• 1:59:37
  Meredith Whittaker: that clearly didn't happen and didn't happen at a company that is actively...
• 1:59:44
  Meredith Whittaker: Die haben OpenAI, sie investieren eine große Menge in Azure,
• 1:59:48
  Meredith Whittaker: sie sind in der Infrastruktur überall, all of sie sind.
• 1:59:50
  Meredith Whittaker: Aber Microsoft ist joked zu OpenAI, so sie hat die Leader Position für ein Moment.
• 1:59:56
  Meredith Whittaker: Und ich denke, also, dass Microsoft
• 2:00:00
  Meredith Whittaker: ist wirklich zu regieren eine gute Reputation in der Security-World,
• 2:00:05
  Meredith Whittaker: es ist indicative of how desperate but that rush to market fit and the AI exceptionalism
• 2:00:12
  Meredith Whittaker: that is driving it is that they just mess that up so egregiously.
• 2:00:17
  Meredith Whittaker: And I can hear some hacker in a Microsoft hallway being like,
• 2:00:20
  Meredith Whittaker: I don't fucking know. I just left that meeting.
• 2:00:24
  Meredith Whittaker: Because you can kind of sense how those things happen.
• 2:00:27
  Tim Pritlove: What would you say is the dividing line right now between useful applications
• 2:00:32
  Tim Pritlove: for machine learning, expert systems, AI stuff, and the hype?
• 2:00:40
  Meredith Whittaker: Well, I mean, the question to dig into is useful to whom?
• 2:00:46
  Meredith Whittaker: Because hype is useful to investors, right? You know, yada yada.
• 2:00:51
  Tim Pritlove: I'm not talking about, yes.
• 2:00:52
  Meredith Whittaker: I mean, I think AI is...
• 2:00:56
  Tim Pritlove: In a computer science-y way. I mean, really like making applications possible
• 2:01:02
  Tim Pritlove: that haven't been before, that actually do useful stuff to people or society.
• 2:01:07
  Meredith Whittaker: Yeah, I mean, one use I think about a lot that is definitely,
• 2:01:10
  Meredith Whittaker: I'm sure, useful to intelligence services is, you know, I'm sure we all assume
• 2:01:15
  Meredith Whittaker: that POTS telephony data is being collected en masse by every intelligence service,
• 2:01:22
  Meredith Whittaker: who can and has been for many, many, many years,
• 2:01:25
  Meredith Whittaker: and that data was probably not that useful for a long time because,
• 2:01:29
  Meredith Whittaker: you know, you're going to have to know, you're going to have to have a human
• 2:01:31
  Meredith Whittaker: review it or, you know, something like,
• 2:01:34
  Meredith Whittaker: It's probably a lot more useful now that you can quickly transcribe that with
• 2:01:38
  Meredith Whittaker: AI and sort of synthesize and search using these generative systems, right?
• 2:01:42
  Meredith Whittaker: So that's one example where I think it's probably almost certainly very,
• 2:01:46
  Meredith Whittaker: very useful and changes the calculus on like how dangerous this surveillance
• 2:01:50
  Meredith Whittaker: business model is as well.
• 2:01:52
  Meredith Whittaker: But who is that useful to? It's not me and you.
• 2:01:58
  Tim Pritlove: Yeah, you're going right into surveillance capitalism again.
• 2:02:02
  Meredith Whittaker: No, well, that's my, I'm really good at making that turn.
• 2:02:07
  Tim Pritlove: I'm looking for a rosy outlook into the future, like hope.
• 2:02:12
  Meredith Whittaker: Yeah, well, I mean.
• 2:02:13
  Tim Pritlove: Anything in store?
• 2:02:15
  Meredith Whittaker: I refuse to plant my hope in delusion.
• 2:02:22
  Linus Neumann: Okay, thanks, thanks.
• 2:02:25
  Meredith Whittaker: And I am probably one of the more optimistic people you'll meet.
• 2:02:32
  Tim Pritlove: All right.
• 2:02:33
  Linus Neumann: Meredith, it was a pleasure.
• 2:02:34
  Meredith Whittaker: So fun. Always so fun. Thank you for having me. It's been a really,
• 2:02:38
  Meredith Whittaker: really delightful conversation with you all.
• 2:02:41
  Tim Pritlove: All right. Thank you. Thanks for listening and see you next week. Goodbye.
• 2:02:47
  Linus Neumann: Bye-bye.