Latest / Never Post / The World's Most Special Boys Building the Apocalypse

Transcript

The World's Most Special Boys Building the Apocalypse · Never Post

0:10
Dave Karpf: For thirty years, the big tech figures have been
0:13
promising that their technologies are going to
0:16
disrupt lawyers, doctors, and higher education. Not because
0:20
those three areas are so disruptable, but because that's
0:23
where all the money is.
0:25
Mike Rugnetta: Friends, hello, and welcome to Never Post, a
0:28
podcast for and about the Internet. I'm your host, Mike
0:31
Rugnetta, and we have a centering show for you this
0:36
week. In this episode, Hans chats with Dave Karpf, associate
0:40
professor in the school of media and public affairs at George
0:44
Washington University and the author of several books on the
0:48
intersection of digital media and politics. Hans and Dave take
0:53
a close look at the recent onslaught of tweets, blog posts,
0:57
and headlines all claiming that the AI apocalypse is on its way,
1:04
if not in some sense already here. Will artificial
1:08
intelligence bring about the end of humanity within a decade as
1:12
multiple anthropic researchers claim is the case?
1:15
Are AI developers actually creating a super intelligent god
1:21
that represents the next evolution of existence. Wait. Is
1:26
that a thing people actually believe? I I have bad news for
1:32
you. Hans and Dave get into some of the deep backstory that leads
1:36
to this moment, but one significant event worth covering
1:39
upfront right here is the hack of AI research platform Hugging
1:44
Face by a not publicly available OpenAI model.
1:49
The company disclosed the hack in July, and the story as it's
1:53
told is this. OpenAI is testing the model's capabilities inside
1:58
a sandbox, a digitally enclosed space, which the model pursuing
2:04
what it understands its instructions to be breaks out
2:07
of. Please imagine that I'm doing big quote fingers around
2:11
all of these spatial metaphors. The model then makes its way to
2:16
Hugging Face where it, through various misdeeds, gains access
2:20
to a private database on their servers. OpenAI claims that the
2:25
hack was not malicious.
2:27
The model was simply attempting to pass a test it was given, one
2:31
related to benchmarking. Other companies like Anthropic and
2:34
Meta, not wanting to be left in the dust looking like losers
2:38
without models doing some casual b and e, piped up claiming that
2:42
their models had done something similar, broken containment.
2:47
Seeing then narrow problems posed to these models resulting
2:52
in so called misalignment that leads to security breaches, a
2:57
particular stripe of researchers saw only paper clips, the end of
3:02
humanity. And so they began ringing the doom bell. It's
3:09
these events, these people, their beliefs, and their place
3:13
in technology history that Hans and Dave take as their focus
3:17
today.
3:19
But first, you know what I'm gonna say. We're gonna take a
3:21
quick break. You're gonna listen to some ads unless you're on the
3:24
member feed. And when we return, Hans, Dave, and the probably not
3:28
so impending AI apocalypse.
3:35
Hans Buetow: I Man, I appreciate this. Dave, thank you. So
3:39
they... Okay. So the reason is because on Tuesday, September 8,
3:45
so just last week, a guy I've never heard of named Jacob
3:49
Coxen, he's just left the AI team at Anthropic, and he's
3:53
tweeting that they don't understand what's going on, and
3:55
there's a ten percent chance that AI destroys humanity in the
3:59
next ten years.
4:01
And this is the spark that sets off a chain reaction of nuclear
4:08
discourse over the past week. Just wild stuff. He says even
4:13
this is possibly the most dangerous technology humanity
4:16
has ever created. So you are an associate professor in the
4:21
School of Media and Public Affairs at George Washington
4:24
University, and you do research which is exploring the Internet
4:27
and the impact on political associations, which I think is
4:30
perfect for this. You have a couple of books, The Move On
4:33
Effect, The Unexpected Transformation of American
4:35
Political Advocacy and Analytic Activism, Digital Listening, and
4:39
The New Political Strategy, and you post.
4:41
I read what you say about blog, tech, industry, politics, AI. So
4:48
before I go investing in hazmat suits and bunkers, I would love
4:55
if we could get a level set here from you, someone I trust to
4:59
give me this trait dope. Does this sound okay?
5:02
Dave Karpf: Absolutely. Also, let me... This is not just
5:05
bragging, but this is relevant. One sec.
5:06
Hans Buetow: Wandering back to your copious bookshelf. I'm so
5:10
excited about what you just held up Vaporware Inc, How Silicon
5:14
Valley Sets the Future to Control the Present by David
5:17
Karf.
5:17
Dave Karpf: Yes. The book comes out in January. This is the pre
5:20
pub version. But everybody can preorder it right now. And
5:24
that's really where I'm coming from in this stuff is I spent
5:28
the past several years.
5:29
I started out by reading the entire back catalog of Wired
5:31
magazine, like issue by issue, cover to cover. I read it twice
5:36
to get a sense of the stories that Silicon Valley tells about
5:39
the future. Because, you know, there's the old line by Mark
5:42
Twain that history doesn't repeat itself, but it rhymes.
5:44
Like, the same is kinda true for futurism. And that's how I tend
5:49
to approach this moment is taking seriously what these
5:53
folks are saying, but also remembering that they're not the
5:55
first set of people to be having these sorts of conversations.
5:59
We can learn a bit from their their own history.
6:02
Hans Buetow: Okay. So let's start with Coxen then. This guy
6:05
goes out and tweets. There's a 10% chance that humanity is
6:08
about to end in the next ten years because of the speed at
6:13
which AI is moving. What context can you give us for who this man
6:22
is?
6:23
Dave Karpf: With this guy... And, you know, he he writes this
6:26
tweet. He resigns. I think part of why people paid so much
6:29
attention to it is that other figures at Anthropic then
6:33
retweeted and replied saying, like, yeah, we're not leaving
6:36
our jobs, but he's right. We're not...
6:38
We're trying our best, but we're not prepared for this. And,
6:41
yeah, maybe like 10 ish percent chance that we will end the
6:43
entire world. I think the thing to understand is this isn't just
6:49
Crita Hype marketing. Like, they're not just trying to scare
6:53
you so they'll build their products. I think they do
6:55
believe the things that they're saying.
6:57
Okay. And also, they're kind of huge online weirdos. And, like,
7:03
they are not dispassionate scientists who have run the
7:06
numbers and arrived at 10%. Okay. Those are people who are
7:09
hanging out in a subculture, and that subculture has
7:12
extraordinaire...
7:13
Access to extraordinary wealth and power. The AI safety
7:17
community, which is what Jacob Coxen comes out of, what this
7:20
entire crowd comes out of, is not a cult, but it's not not a
7:24
cult. Oh, yeah. Like, it's kinda culty. Some of the history here,
7:29
the entire AI safety community originates in the less wrong
7:34
online discussion boards, which are lorded over by Elias
7:37
Riyadkowski, the author last year of If Anyone Builds It,
7:41
Everyone Dies, which is sort of a major AI doomer tract.
7:45
He before that wrote a... I think it was like a 700 page
7:49
Harry Potter fanfic called Harry Potter and the Methods of
7:53
Rationality. And prior to the less wrong rationalist
7:57
community, Elijah Judkowski and the rest of that crowd started
8:00
out in the extropion discussion boards back in the nineteen
8:04
nineties. The extropions are the people who think that we can
8:08
live for forever so long as we freeze our bodies when we die,
8:11
and then they'll later on invent a singularity and be able to
8:14
upload our minds, we'll live forever. Because again, like,
8:17
encountered the extropions by reading Wired 1994 covering
8:21
them.
8:21
And my thought at the time was, well, yeah, these guys seem
8:24
kinda nuts. Pretty ridiculous. Kinda fun that back in the
8:27
nineties, people thought we would be there soon. And what's
8:31
happened over the intervening decades is the subset of the
8:34
Internet that took that stuff really seriously all marinated
8:38
together in online discussion boards and sort of built up as,
8:43
like, a group think and a set of shared beliefs. And then some of
8:45
them became multibillionaires and started companies and
8:49
invested in the rest of them.
8:50
I've joked before that the rationalist community is to
8:54
Silicon Valley what Scientology is to Hollywood. Like, everyone
8:58
in Hollywood isn't a Scientologist, but an awful lot
9:01
of very powerful people in Hollywood happen to be
9:03
Scientologists, and it's a little weird. And since they're
9:05
all networked together, like, all believe the same thing
9:08
about, like, Xenu. And they genuinely believe it, and also
9:11
that doesn't make it real. So the entire AI safety community
9:16
is part of that crowd that genuinely believes some things
9:20
that seem astonishingly strange to the rest of us, we usually
9:24
don't encounter it.
9:25
And when we do, like, we should believe that they believe it,
9:27
but that doesn't mean that they're right. Like, there's a
9:29
lot to be worried about with AI and safety right now. But it's
9:33
not that the AI is gonna turn sentient, become super
9:36
intelligent, and then kill us all. Like, we don't really need
9:39
to worry about that. It's definitely not a 10% chance.
9:42
What we do need to worry about is there are an awful lot of
9:44
flaws in this technology that is getting pushed everywhere. And
9:48
since they need to make all the money, they're going where all
9:50
the money is, which means that we're now having the AI, like,
9:52
pilot our drones. And so I do think the reaction to this
9:55
should be, okay. Yeah. We should slow down the deployment of all
9:59
this AI, not because we think that they're right and
10:04
superintelligent AI is gonna destroy humanity, but because
10:07
there's this really strange subculture with too much access
10:10
to wealth and power, and even they aren't sure how this is
10:13
gonna go.
10:14
So maybe we shouldn't be trusting them to police and
10:17
control themselves, and we actually need the rest of
10:19
society to fill in and set a regulatory framework, set
10:23
liability frameworks so that these company, these, like,
10:26
multi trillion dollar companies apparently can't just go and
10:29
wreak harm all the time.
10:30
Hans Buetow: So now that we have that sort of... Like, I feel a
10:33
little bit calmer about, like Yeah. How to think about it. Now
10:36
I'm just really curious about the why now. Like, why did this
10:39
tweet thread blow everything so sky high?
10:42
And I wanna put on my tinfoil hat a little bit and just and
10:45
just see if I'm being weird about this because we're in
10:49
September now, which means we're just on the brink of q four. And
10:53
this is a q four that comes at the end of a year where Pew
10:57
estimates that more than a third of the con... New content on the
11:00
Internet was written by or with the assistance of AI, and also a
11:06
year where the polls show that most people are more wary of
11:10
tech than they are hopeful, and also where more than half of
11:14
Americans have said in this past June that they're worried AI
11:17
will put someone in the household out of work. Mhmm. So
11:21
is this marketing to a very specific shareholder audience?
11:26
Because it sure does line up timing wise with an important
11:29
moment for them. But is that, like, is that wild thinking?
11:33
Dave Karpf: It's not the wildest thinking. There's a... One of my
11:36
favorite writers on finance, Matt Levine, his riff on this
11:39
has often been that, like, when Dario Amadai and these other
11:43
folks worry about AI doom, he doesn't think that it's just
11:46
marketing. It also happens to be really good marketing.
11:49
Hans Buetow: Sure.
11:49
Dave Karpf: Like, if you were saying, like, our technology is
11:51
so powerful that it can end the world and render money obsolete,
11:55
it's a pretty good investment pitch. I would actually point to
11:59
a different thing than the q four earnings, though, which is
12:01
the hugging face hack, where OpenAI had to come out and
12:05
admit, like, yeah. You know, we were testing this AI in in a
12:08
sandbox with all the guardrails turned off, and it got out of
12:12
the sandbox, and then it actually hacked hacked things on
12:14
the open web. And we're still trying to figure out how it did
12:18
it. And then after that, Anthropic came out and said,
12:21
like, oh, yeah.
12:21
Ours kinda kinda hacked some stuff too. Whoops. I think
12:25
they're reacting to that and sort of saying, oh, wow.
12:27
We're... We feel like we're moving really, really fast, and
12:31
it's all gas, no brakes as we head towards IPO land.
12:35
And those of us who are in charge of safety are getting
12:37
real nervous about that. And, again, part of the reason why I
12:39
think that is the general... Like, if you work at OpenAI and
12:43
Anthropic, you are otherwise... Like, the rest of the week,
12:47
you're really upset about how most of the country now really
12:50
doesn't like data centers and thinks that your products are
12:52
low key evil. Right.
12:54
And if you're working in PR for one of these companies, you
12:57
really wanna have, like, a good day where you can just brag
12:59
about how, like, oh, yeah. Like... And, like, OpenAI wants
13:02
to brag about how, like, they just solved a a millennial math
13:05
problem.
13:05
Hans Buetow: Yeah.
13:06
Dave Karpf: And so right now, it is, I think, inconvenient for
13:08
them that Coxswain was like, don't know. I'm kinda freaked
13:11
out, and I think everyone should be more freaked out. But I think
13:13
it's that sense of hugging face seems like... Seems very
13:17
important to the AI safety crowd specifically, and they have the
13:21
sense that everything is all gas, no brakes, and they would
13:23
like some brakes, please.
13:25
Hans Buetow: I would like to put it as not just important to that
13:26
crowd. I think it's really important to people like me
13:30
who's, like, trying to follow, and that's, like, the big story
13:34
from the summer. It's happened in, like, July. Right? Yeah.
13:36
And so that's the context I'm coming from as the last big AI
13:39
story, And all a sudden, the security guys are coming out and
13:41
being like, holy moly. Like, we're heading towards
13:44
extinction. And I think you put those two things in such
13:47
proximity to each other, it foments a specific type of
13:52
believability alongside a specific kind of panic. And I I
13:57
wanna ask you about it because it's... You framed it just now
14:00
so much differently than most of the media framed that incident,
14:05
the hugging face incident.
14:06
Right? Mhmm. So there have been people who have spoken like you
14:09
did, who have urged caution. Right? So Kieran Martin from the
14:13
economist, Wired put out a piece that says rogue AI agents aren't
14:17
evil.
14:17
They're just eager to please. Kat Tenbarge, Cal Newport. There
14:20
have been folks who have said, hey now. It's not actually
14:25
malicious super beings. Yep.
14:28
What's happening is much more complicated, but what's really
14:31
staggering is the level of headline that happened at the
14:36
level of publication. I'm gonna read you a couple of headlines
14:39
really quick from legacy media. Mhmm. Politico says hundreds of
14:44
AI agents went rogue in OpenAI's hugging face hack. CNN, rogue AI
14:49
agents escaping containment.
14:52
Wall Street Journal, cyber attack by rogue AI swarm stokes
14:57
fears of out of control agents. New York Times. This one I love
15:01
because it's just alliteration. Anatomy of an autonomous attack,
15:05
five alarming AI capabilities. I mean, I'm a sophisticated
15:11
consumer of news if that's where I'm getting my sources, and
15:15
that's the level of panic that they're putting into these
15:18
headlines.
15:19
Why did everyone run with the story? Why are people framing
15:23
this story this way? And kinda like, how should we think about
15:26
that hugging face incident in relationship to this?
15:29
Dave Karpf: Right. And so part of the reason why we get those
15:32
sorts of headlines is there is sort of like an information
15:35
asymmetry here where the... A lot of the people who are
15:40
working at the companies and in a position to explain at a
15:45
technical level what happened Also, come out of the crowd that
15:50
thinks we are building sentient AI that is going to become...
15:54
Like, is going to be the next stage of evolution and will
15:57
replace humanity. Like, again, they come out of that crowd.
16:01
They believe that stuff. I think it was Dwarkash Patel, the
16:04
podcaster, wrote a really popular piece where he was
16:08
trying to explain the hugging face hack as, like, three
16:11
civilizations of AI agents rose and fell along the way to their
16:18
effort to, like, hack these websites.
16:21
Hans Buetow: There's so much intentionality in that frame.
16:24
Dave Karpf: So much, like, so much intentionality and, like,
16:27
that, you know, that is very clever wording. And then the
16:31
response from an awful lot of critics, including people who
16:34
are much more technically knowledgeable than me, was like,
16:36
man, can you not call it a... They're not... Like, that's fun
16:39
sci fi, but, no, they didn't they didn't have the rise and
16:41
fall of civilizations here, man. Like, they had...
16:45
Like, I think it's Corey Doctorow, who again is much more
16:48
technically knowledgeable than they pointed out, like, one of
16:50
the ways to understand the Hugging Face hack is they built
16:53
a shitty sandbox. Like Yeah. They built a better sandbox. It
16:57
shouldn't have been able to get onto the Internet. I guess it's
16:59
impressive that they have now built sophisticated enough AI
17:02
agents that they can hack out of the insecure sandboxes, but
17:07
guys, you gotta build better sandboxes.
17:09
Hans Buetow: This is not the velociraptors opening the doors.
17:12
Right.
17:12
Dave Karpf: Yeah. This is you left part of it, like, one of
17:15
the doors unlocked and then also created a scenario where you
17:19
pushed these AI agents until they could finally find a way to
17:24
unlock the doors. And if you do that for enough cycles, then
17:27
some stuff will happen. And, again, that proves a bunch of
17:29
capabilities that weren't there two or three years ago. But this
17:33
points to one of the the broader questions of sort of like, what
17:35
do you do with all this stuff?
17:37
Because one of the... Like, the the answer for normal people,
17:41
and I cannot stress enough that it is also the right answer, is
17:46
you assign liability to the companies with punitive fines.
17:52
Like, if if you would like swarms of AI agents not to hack
17:56
lots and lots of websites, know, one thing is to say, like, we
17:59
must slow down and come to understand AI agent society and
18:04
then treat them to be nice and respectful to human society, all
18:08
of which is sort of steeped in this weird mysticism that
18:10
doesn't really hold up. And another is to say to all the
18:13
Frontier Labs, hey, guys. Since these are your products, if your
18:17
if your products go and hack websites, there are, you know,
18:21
giant fines and jail time for you.
18:24
If you do that and you enforce that, then I can pretty much
18:28
guarantee that all of the... These major labs will complain
18:31
for a minute and then figure out ways to make sure that their
18:33
sandboxes are airtight because they don't wanna go to jail. And
18:36
one of them did. The rest will figure out, okay. Let's not do
18:38
that.
18:39
Like, the the way to regulate this stuff is to actually have a
18:44
regulatory scheme with penalties. It's not to sort of
18:47
build up a a... Like a like a a church of AI that comes to
18:52
understand it and then enlighten us all.
18:55
Hans Buetow: Lina Khan Mhmm. Our our former Federal Trade
19:00
Commission secretary responded to this in exactly the way that
19:03
you're talking about. She said there are plenty of laws that
19:06
the FTC has on the books already for this kind of situation. She
19:10
said that law enforcers already have the authority to charge
19:14
companies and their CEO for creating and releasing
19:17
dangerous, unvetted, or defective products. We shouldn't
19:20
let discussions about new legal regimes distract from the fact
19:23
that there's no AI exemption, no exemption, from laws already on
19:27
the books, things like consumer protection laws against unfair,
19:31
deceptive acts, unfair methods of competition, conflict of
19:34
interest laws, laws against poor data security, all of those
19:38
already on the books.
19:39
Yeah. We know, though, that we are currently being led by an
19:44
administration that is in no way going to enforce those things. I
19:47
think we can all kind of agree that that's just not on the
19:51
table. Yeah. Is there then a way to read kind of what happens
19:55
next in the timeline of the past week as the company's coming out
20:00
and being like, okay then.
20:03
Why don't we do it ourselves? Because on Friday of last week,
20:07
this is just a couple of days after Cox sends his tweet, Dario
20:11
Amade, who's the CEO of Anthropic, he comes out with a
20:15
3,200 word blog post all about this, where he actually talks
20:21
about the the hugging face thing and says that it's... You know,
20:26
he gets all doomerist about it, and he says... And he agrees
20:29
with Coxen, and he says we need a slowdown. And that he's gonna
20:32
open his company to auditors, and that other companies should
20:35
too, and that we're gonna pace the frontier.
20:40
And I wanna know, like, it feels to me like this is the sort of
20:45
thing where he's like, hey. Hey. Hey. Hey. The smart guys are
20:48
talking.
20:49
Don't worry. We'll... We will take care of this. That, like,
20:52
take your cues for how all this works from us, how you feel, and
20:56
how you behave towards our products. You should take your
20:58
cues from us.
20:59
And you know what? We'll just be in charge of our own regulation.
21:01
He actually says in this thing, unfortunately, passing laws can
21:05
take time. Mhmm. And AI is advancing very quickly.
21:08
Therefore, in parallel with the regulatory route, AI companies
21:12
can and should voluntarily work together to set standards. Am I
21:16
off on that? Does it feel like they're stepping forward and
21:19
being like, nothing... I mean, it's wrong, but we'll take care
21:22
of it.
21:23
Dave Karpf: So I think I have three notes. Maybe more than
21:28
that, but let's start with three. Okay. First is I think it
21:32
would be really interesting to give truth serum to the AI execs
21:39
and ask them in a scenario where they had to give an honest
21:43
answer, which are you actually more afraid of? Super
21:47
intelligent AI or Lena Khan?
21:50
Part of a big reason why so much of Silicon Valley moved over to
21:54
Trump in 2024 was out of a fear of Lena Khan and other
21:59
regulators in the Biden administration who were treating
22:03
Silicon Valley as though it wasn't, like, the world's
22:06
special boy who the rules didn't apply to. Yes. So, like, Lena
22:09
Khan spent several years... There... There's this one memo.
22:11
This, by the way, is is in my book, Vaporware Incorporated.
22:14
Hans Buetow: I love it. I love
22:15
Dave Karpf: it. Much better. But one of the things that I that I
22:18
point out in that, there was a memo that got published by
22:21
Khan's FTC that just said to the AI companies, like, hey. Just a
22:25
reminder, you're not allowed to lie about what your products do.
22:28
If you say that your product uses AI for some purpose, it
22:31
actually has to use AI for that purpose.
22:33
Otherwise, we will investigate and find you. And all of Silicon
22:36
Valley lost their minds. Yeah. Because, like, we're supposed to
22:40
move fast and break things. We're supposed to, you know, do
22:42
vaporware.
22:42
We're supposed to build up products and then wait to
22:44
actually create the real product until after we've seen that
22:46
there's a market for it. Like, that's our way. We're we're the
22:49
world's special boys building the future. And they were
22:51
outraged at this idea that the largest companies on Earth would
22:55
be treated on the basis of their results rather than their
22:58
ambitions. Like, that's part of how we got here is so many of
23:02
the AI figures, not not Amodai, but an awful lot, you know, like
23:04
Elon Musk, you know, Greg Brockman from OpenAI, like, they
23:09
all have donated heavily to Trump and the regime.
23:12
A bunch of them got into Doge and led Doge because they
23:16
believe that the regulatory state should be decimated and
23:18
Silicon Valley should be left to its own devices. Mhmm. So while
23:21
I do think that they believe that they're building super
23:24
intelligent AI and maybe it'll destroy the world, I I honestly
23:27
think their dreams are a lot more haunted by Lena Khan and
23:31
competent regulators. It's like that's the first thing I wanna
23:33
point out here.
23:34
Hans Buetow: Yeah. Okay.
23:35
Dave Karpf: That gets to the second that... I I read that
23:38
Amadai essay. Yeah. I thought... I I saw it on Saturday, which
23:41
meant that I had to spend my weekend reading that thing.
23:43
Thanks a lot, buddy. But... I I read that. I've got... Wrote a
23:50
thread responding to it on Blue Sky.
23:51
That thread is gonna turn into an essay on tech policy press, I
23:55
think, later on this week. But the the main thing that stands
23:57
out, like, you... You're right. It's accurate that his stance
24:00
is, well, the government's not gonna do it, so let's just trust
24:04
the AI industry to regulate itself. His ask from government
24:08
is he thinks for the big AI firms to freeze things in place
24:12
and slow things down and handle it all, what they need is a
24:16
waiver from the government not to pursue them for antitrust
24:19
violations.
24:20
So the one thing they want from regulators is really, really
24:22
promise not to regulate us at all. Mhmm. He's... So like,
24:26
that... Then like that stands out because again, it's it's a
24:28
weird thing when Amadai who kind of plays the role of at least if
24:32
not the good AI guy, then at least the less bad AI guy.
24:36
Right? Like, when he comes forward and says, yes. You know,
24:39
we need to slow down the pace of AI adoption. And he he has a
24:43
line in there that, like, unfettered competition amongst
24:47
companies is gonna be a race to the bottom. Yes.
24:50
Like, I read that, and I think, yes. Like, this is the reason
24:52
why I think we ought to keep on slowing down the pace of data
24:56
centers because the the way this stuff is getting developed is
25:00
nuts and is bad for society. So I... Like, I want to agree with
25:03
him, And then the place where he definitely loses me is I look at
25:08
the trajectory of this industry and think we cannot trust the AI
25:12
industry to police the excesses and the worst behaviors of the
25:16
AI industry. There's too much money involved.
25:18
If if we build out this technology just through the
25:21
mechanisms of market competition in unfettered capitalism, that's
25:25
the way to end up at an awful lot of the worst results. Right?
25:27
Like, that's how we end up just figuring out what is the market
25:30
for AI murder drones, whether they work or not, because
25:33
there's an awful lot of money in military tech. Rather than
25:36
figuring out, okay, what's the transparency requirements? What
25:38
are the boundaries gonna be?
25:39
Like, who is liable when the AI murder drone kills people it
25:42
wasn't supposed to kill? Like, all that needs to get worked
25:45
out. The industry is not gonna do that to itself. I think that
25:50
we as a society should be looking at AI and the pace of AI
25:53
adoption and sort of this industry and its behaviors and
25:56
saying, you know, you guys can't be in charge of this. We need
25:59
governments.
26:00
We need democracy. We need regulators to kinda get in your
26:03
way and interfere. And he's throwing out a bunch of
26:07
examples. Like, he's he's got this... His first proposal is
26:10
they should have monitoring and transparency, And then he brags
26:13
that this company, Meter, has been doing the monitoring for
26:17
them.
26:17
Meter is is tiny, and it's made up of a bunch of people who,
26:20
like, live in the same group houses as as Anthropic
26:22
employees. Mhmm. Like, it's well meaning, but it is nowhere near
26:26
large and powerful enough to actually be a meaningful check
26:29
on Anthropic. And part of how you know that is if they ever
26:31
found anything that Anthropic didn't want, Anthropic could
26:34
could just change its mind since it's voluntary. Right.
26:37
But he compares that to, like, the monitors in the banking
26:40
industry. And the thing about the the monitoring in the
26:43
banking industry is that largely comes about after the great
26:47
financial crisis.
26:48
Hans Buetow: Yeah.
26:48
Dave Karpf: And then the... We have the force of government
26:50
saying, you're gonna have this monitoring and transparency, or
26:53
we will find the hell out of you, and there will be jail
26:55
time. Right? Like, the force of government is the only thing
26:57
that makes that stick. The thing he's describing is actually an
27:00
awful lot more like Arthur Andersen and the Enron scale...
27:04
In the Enron scandal.
27:05
Hans Buetow: Oh, interesting comparison. Yeah.
27:07
Dave Karpf: There... There's an awful lot of transparency in the
27:09
Enron era. But since it's all voluntary with a third party
27:13
that's much smaller, Enron figures out like, oh, we can
27:16
just... It's just gonna be in the financial interest of Arthur
27:19
Andersen to say what we want them to say. Yeah.
27:21
That ends up furthering all of the problems. Like, you really
27:24
do need government to make the industry feel uncomfortable.
27:28
Otherwise, you're not gonna have any meaningful checks on the
27:30
industry. And so, again, like, think Amadai is is well meaning.
27:34
He...
27:34
He's like, he wants to have morals, and those morals are in
27:38
tension with his business ambitions, and he's just kinda
27:40
hoping that we can make them both work. But the answer, if
27:45
the history from any other large in... Powerful industry is any
27:48
guide, the answer is actually you need regulations that aren't
27:51
set by the industry itself. You need rules that they don't like
27:55
and make them feel uncomfortable. You need serious
27:57
liability that they can face massive fines for in jail time
28:00
if they don't think...
28:00
Do things wrong. Of course, the industry isn't gonna propose
28:03
that, but that's why I think we should take his recommendation
28:05
that we should slow the frontier and say, that's great. You don't
28:09
get to slow the frontier. Other people do. That's right.
28:11
But I have one more point, and this is the really bad one.
28:13
Hans Buetow: Oh, boy.
28:15
Dave Karpf: So the the bad one, the trap we're in Yeah. Is...
28:18
And and you noted this. Like, we we don't have Lina Khan in the
28:22
Kamala Harris administration. We're Yeah.
28:24
Stuck with the Trump administration. Trump listened
28:27
to... You know, Ahmad I published this, and we had both
28:30
Elon Musk and Sam Altman saying, yep. We'll sign on to that. And
28:33
then Trump replied to that by saying, no.
28:35
We don't need any of that. All we need is a really smart
28:38
president. We have one. So all gas, no brakes. Let's just move
28:41
forward with it, which I think Jensen Huang seemed to really
28:44
like because, I don't know, I guess NVIDIA needs more
28:47
purchases or whatever.
28:49
But we are in a situation where the Trump administration has
28:52
gutted the entire administrative state in The United States. We
28:55
do not have administrative capacity. We don't have
28:57
competent regulators. They've been threatening Europe and
29:00
basically saying, you know, give or take, if you want NATO, then
29:03
you can't regulate anything in the AI industry. Like, we want
29:06
American AI to to win, and we're gonna flex our muscles.
29:10
So we're kinda in... We're kinda stuck with the worst possible
29:13
president for this moment. Right. And that that does create
29:16
a trap that is impossible to get out of in this moment. Right?
29:20
Like, if I was advising Amadai, I'm like, okay. What do you do
29:23
in the next six months? Like, the industry does need to help
29:28
out with slowing down since god knows this administration isn't
29:31
gonna do it. And, also, the admin... The industry needs its
29:36
power to be checked by somebody, and it's unclear who.
29:39
This, again, is the reason why even though the industry won't
29:41
like it, I'm a big fan of moratoria on data centers on
29:46
making huge demands on these companies before they can build
29:49
one to make sure that the communities they're in are made
29:51
better by that data center. Like, slow all that down because
29:54
that's one of the ways to change the trajectory of the AI future
29:59
and take power away from these companies. If people are
30:02
listening to this and they're they're worried, like, show up
30:04
to a data center protest in your in your area and say, like,
30:08
let's say no to that. That's one way to help slow down the
30:10
trajectory of this stuff because eventually we need competent
30:14
regulators again, but it's gonna take a while.
30:15
Hans Buetow: It's gonna... It's... And and in the meantime,
30:17
you know, Daria's proposing, like, oh, I'll just have
30:20
auditors come in that'll have, you know
30:23
Dave Karpf: Auditors who I'm friends with.
30:25
Hans Buetow: Like Who I'm friends with. Exactly. I... Who
30:26
I control. So don't you worry, folks.
30:29
We're the smart... We're the geniuses who created the
30:31
problem. We're the only geniuses who can solve the problem
30:34
Dave Karpf: Yeah.
30:34
Hans Buetow: Is what I hear a lot of in that.
30:36
Dave Karpf: That that is absolutely what I read into that
30:38
as well. And we should trust Dario Amadai just enough to say,
30:44
alright. We're not gonna trust you for the solution. We need
30:47
other people here.
30:48
Hans Buetow: So let's then catapult ourselves forward a few
30:52
days. It's alarming how fast conversation is.
30:55
Dave Karpf: I'm so tired by this already. I'm so tired.
30:59
Hans Buetow: Okay. Well, we're almost there because this gets
31:01
us almost up to speed now. So let's catapult to the fifteenth.
31:05
So this is Tuesday the fifteenth. And it seems like one
31:10
of the responses to what you're saying where some sort of
31:14
regulation is needed, some sort of regulation actually exists.
31:17
We just need somebody to, like, actually employ that. Now we're
31:22
getting these absolutely Frankenstein bizarre alliances
31:28
of people getting together with bizarre high minded names
31:35
Dave Karpf: Mhmm.
31:36
Hans Buetow: To try to do something about what has erupted
31:39
in the past week. So just this Tuesday, there was a quote,
31:42
unquote... It was called a prohuman assembly Mhmm. In
31:47
Washington. And this is the trio who got together.
31:51
Coxen, who is the... Our our tweet guy from last week who no
31:55
one knew about the eight days before that.
31:58
Dave Karpf: Coxen Fifteen minutes of fame.
32:00
Hans Buetow: Bernie Sanders.
32:02
Dave Karpf: Mhmm.
32:02
Hans Buetow: And who do you place with that? Well, of
32:04
course, you go with Steve Bannon.
32:05
Dave Karpf: Of course. Of course. Because otherwise, it's
32:07
not sweaty enough.
32:11
Hans Buetow: Guys are getting together and being like, we're
32:12
gonna go with... We're doing prohuman AI. Yeah. What? What do
32:18
Dave Karpf: you Yeah.
32:19
Hans Buetow: Dave, help.
32:21
Dave Karpf: I I I I really can't. I I I would love to sit
32:27
down with Bernie and have a slow conversation that features an
32:32
awful lot of me, like, peering over my glasses at him and
32:35
saying, sorry, what? Like, just a real slow conversation with
32:40
him about what the hell he thinks he's doing. The main
32:43
thing that stands out to me is, like, that's all spectacle. I
32:47
can understand how...
32:48
Like, I can understand how Steve Bannon has decided this is a
32:51
spectacle that I can play to. And so I'm gonna go get a bunch
32:55
of attention before, we once again get the parable of the
32:58
scorpion and the frog. Like, come on. It is 2026, and you've
33:02
decided that Steve Bannon is an honest ally for you here. Come
33:06
on, guys.
33:07
Yeah. But, like, this is an opportunity for them to
33:11
grandstand, them to say, like, the world needs to wake up, and
33:15
we need some things, and then do some performative proposals that
33:19
sort of suck the air out of the room in this moment, and then
33:22
the AI industry sort of continues unchecked. Because as
33:26
substantial... Substantive threats go, this doesn't seem
33:29
like a big one. Now again, I I I think I saw that one of the many
33:32
bills that Bernie has introduced is a data center moratorium
33:36
bill.
33:37
I don't think it's gonna pass, but I'd be all for all for that.
33:39
Like, sure, man. Go for it. Right. But an awful lot of these
33:43
proposals that are making are starting from the premise that,
33:46
yeah, the threat here is super intelligent AI.
33:49
Yeah. And so we need the better AI, and we're gonna collaborate
33:53
with the AI industry, but, like, really make some demands to do
33:57
so... To be serious and answer that threat. And, again, going
34:00
back to the first point, the AI industry really believes what
34:05
they're talking about there. That doesn't mean that it's
34:08
actually all that believable.
34:10
There are a lot of other threats and harms from AI that come not
34:13
from it being misaligned with humanity, but from it being a
34:17
set of technologies that are getting applied to an awful lot
34:19
of uses that they're not all all that good at... Good for yet.
34:23
Like, the the problem is how we are deploying it and letting
34:27
these unconstrained corporations go wherever the money is and
34:32
just try to scoop the money out even if their products don't
34:34
quite work right yet. So there's an awful lot of other threats
34:37
that I think we ought to be more worried about, none of which fit
34:43
the story that the AI industry wants to tell. And, like, it's
34:46
kind of a really annoying distraction that now Bernie and
34:50
Bannon have decided we are gonna be the the the torchbearers for
34:54
fighting AI because the the tune they're singing isn't gonna make
34:58
any sense and isn't gonna do anything.
35:00
Hans Buetow: Is there anything then to the threat of China?
35:05
Because another thing people are calling for is AI diplomacy.
35:08
Dario Amade in his long screed says that first we need the
35:12
democrat... We need nations that are... You know, democracies to
35:16
come together, then we need full international cooperation there.
35:18
We're gonna bring it to the UN. We're gonna make this thing an
35:21
international effort. China is, of course, the thing that's
35:24
looming large in this, and it feels like there's IPOs is one
35:28
pressure on these companies. And China developing something
35:34
ambiguous that's super intelligent is on the other
35:37
side, and they've got a... They can't slow down because one of
35:41
those two things is gonna mean...
35:43
Is gonna be a death knell for them if they don't keep moving.
35:46
Dave Karpf: So there is a little something to China... The the
35:49
China thing, but only a little something. And let me Okay. What
35:51
I mean by that because it's important. The Amadai proposal
35:55
that there ought to be bilateral talks between The US and China
35:59
on the pace and direction of AI adoption, that is, in broad
36:02
strokes, a good idea.
36:03
Like, there ought to be bilateral lateral conversations.
36:06
Yep. The reason why there should be bilateral conversations is
36:09
China... The China threat is functioning as a boogeyman.
36:12
Okay.
36:13
Like, if the threat of China winning this race didn't exist,
36:16
the AI industry would need to invent it because it is really
36:19
useful to have a boogeyman that serves that... Look. Whenever
36:24
somebody says, like, yeah, we should slow down the pace of
36:27
data centers. We should apply liabilities to these companies.
36:30
The answer is always, well, if you do that, then China's gonna
36:33
win.
36:33
Like, they need that boogeyman in order to make sure that the
36:36
regulators take a light touch with them and allow them to sort
36:39
of act now and apologize later. The... There's a historical
36:42
parallel that always stands out to me. And, again, it's it's in
36:46
the book. It's in chapter two of my forthcoming book.
36:48
But the historical parallel is actually the the science of
36:52
human cloning in the nineteen nineties.
36:54
Hans Buetow: Okay.
36:55
Dave Karpf: So nineteen nineties, you know, they they
36:57
clone a sheep, the sheep dolly, and I think it was 1994. Yeah.
37:00
And pretty quickly, a bunch of people who are sort of vaguely
37:06
in the area of science, but not actual scientists in this area,
37:08
say like, well, you know, if you can clone a sheep, then, like,
37:11
you can clone a human. Like, it's the same thing. I'm sure
37:13
we'll be able to do that, like, next year.
37:15
And people freak out because, well, if we're cloning humans,
37:18
then that seems like a radical change to what it means to be
37:20
human. I remember this. And Bill Clinton then passes a moratorium
37:25
saying there will be no use of government funds to advance
37:30
human cloning. Like, you you can't do it. At that point, the
37:34
sort of nascent tech industry, this showed up in Wired magazine
37:36
a bunch, so I had to read all about it.
37:38
But, like, the nascent tech industry freaked out because,
37:41
again, it was government saying, no. No. There are regulations
37:43
that apply to you. There are things that you can't do. And
37:46
they don't like that story.
37:47
They like the story of we are the people who build in the
37:49
future. We are special. The rules don't apply to us. So they
37:52
got real, real mad for several years. And the boogeyman they
37:55
kept on invoking was, well, look.
37:57
China doesn't have these limitations. So if you don't let
38:00
us spend tax dollars to advance our science of human cloning,
38:05
then China's gonna invent the science of human cloning. And,
38:08
like, oh my god. Then then they'll rule. We can't allow
38:11
that.
38:11
And they said that for, like, about... Probably about eight
38:14
years. They they kept on beating that drum. In retrospect, it's
38:18
not that, like, China didn't have the same limitations as us.
38:21
But China in the same years as the the one child policy was in
38:25
fact not trying to build out a new technology that would let
38:29
them produce tons and tons of humans.
38:30
So, like, they did an awful lot of investigation into... Like,
38:33
they they supported scientists, like, looking into stem cell
38:36
research and a bunch of other research areas, but they were
38:39
not on the trajectory that the pretend China boogeyman was
38:43
supposedly on because the Chinese government didn't wanna
38:45
do that. In the AI race today, like, yes, it is true that there
38:49
are Chinese companies that are moving forward on on AI. There
38:53
are really impressive models that are a few months behind our
38:55
frontier models. And there is this concern from the AI
38:59
industry that, like, if we don't win the AI race, the China
39:02
would.
39:04
And, also, I'm not a China expert, but from what I
39:07
understand of the Chinese government, they may be
39:10
interested in surveilling the populace, but they're not really
39:13
interested in overturning all of humanity and all of government
39:17
by unleashing, like, super intelligent AIs that can replace
39:21
us. Like, that at least doesn't sound like a major goal for
39:27
their government in a way that it's the goal for OpenAI and and
39:32
Grok because apparently that's how they're gonna be able to
39:35
scoop $30,000,000,000,000 out of out of the market the way, you
39:37
know, the way these companies want to. So the China race is...
39:41
Isn't good, and it would be a good idea to have bilateral
39:45
talks. Again, if the Harris administration was here, I would
39:47
be saying like, yes, Harris administration, like, there's an
39:50
awful lot of spending here.
39:52
Yep. And there's a lot of competition between the two
39:54
countries. You should maybe come up with some rules of the road
39:57
and some agreements. That's generally a good idea, but the
40:00
China boogeyman really poisons the well here. And anytime I...
40:05
Like, when I saw Dario Amadai talking about it, that for me
40:08
was kind of a flag of, okay, he wants them to be... He wants the
40:12
AI industry to police itself and wants us to make sure that even
40:16
if they slow down, they don't slow down in a way that could
40:18
allow the China boogeyman to to get in front of them. So again,
40:21
it sort of strikes me as an attempt to avoid accountability
40:24
in the way that he's doing it. So, like, yeah, worry about
40:26
China, but, like, worry about China in the right way. Like
40:30
Right.
40:30
They're they're not trying to build the things that Elias
40:33
Rudkowski and Dario Ahmaday say that they're trying to build.
40:36
Hans Buetow: You have been very helpful, Dave, at helping me
40:39
understand the parameters. I feel like I'm even talking
40:42
slower. I feel so much calmer. I don't know that it's helped you
40:46
at all, but I've I've been helped. But I am...
40:49
Okay. So I feel like I have a sense of the players. I feel
40:52
like I have a sense of the moments for the past week or two
40:54
weeks or, you know, two months really. But I don't have a
40:58
sense, just like you're saying, of where we go from here Yeah.
41:02
And and what we do.
41:03
Like, what can you say? You wrote recently about what you
41:08
call a realistic strategy for limiting the harms of AI. Yeah.
41:11
And I'm wondering if you can help me feel like there's people
41:14
thinking about this in a good way.
41:16
Dave Karpf: Yeah. So, again, it it... It's only a partial
41:19
strategy because what we need is time. Yeah. What we need is to
41:22
reset the power dynamics.
41:25
I I I am not one of the people who thinks, like, AI is gonna
41:29
disappear tomorrow and, like, there's nothing to it. Like,
41:32
clearly, there's a bunch of things that this set of
41:34
technologies is useful for. And I've got a line in the book, the
41:39
trajectory of any new technology bends toward money. Right? So we
41:43
need to think an awful lot about what are the financial
41:46
incentives that companies are going to chase after.
41:50
Right? Like, Sam Olmbeck in in 2021 wrote one of the early big
41:55
think essays on this stuff where he... It was called Moore's Law
41:57
for Everything. And he was saying how, like, within a few
42:00
years, AI is going to replace lawyers. It's gonna replace
42:04
doctors.
42:04
It's gonna replace it's gonna replace higher education. And
42:09
again, as a guy who studies the history of the digital future,
42:11
for thirty years, the big tech figures have been promising that
42:15
their technologies are going to disrupt lawyers, doctors, and
42:19
higher education, not because those three areas are so
42:22
disruptable, but because that's where all the money is. They're
42:26
they're always gonna move towards where is the big money
42:28
in these markets Interesting. Yeah. If we leave it just to
42:31
these companies.
42:32
So my my short term solution, it's a partial solution, is what
42:37
I call the data center bank shot. That right now, the AI
42:40
future is getting shaped by a small number of companies that
42:43
are pretty much entirely unchecked and also are cash
42:46
furnaces. Right? Like, they are spending an awful lot more money
42:50
than they make. And their ambition is, like, we're gonna
42:53
keep moving faster and faster to get the next infusion of cash.
42:57
Mhmm. That's not going well for society because there's no
43:00
checks on them, and they get to just sort of go wherever they
43:03
want even when those ideas are bad or unsustainable or going to
43:07
wreak havoc. I think that the only solution, at least for,
43:10
like, citizens, right, like, because we don't have a a
43:13
government that's able to do anything right now. I think what
43:16
we need as citizens is to pressure these data centers, try
43:20
to slow them down, try to shut them down, shut down their
43:22
construction, because the data centers are the one leverage
43:26
point we have that can affect the pace of the AI build out.
43:30
And if you slow down the AI build out, then this huge
43:33
speculative economy where these company...
43:36
Like, there are, you know, you know, multi... Multibillion
43:38
dollar companies right now that don't have a product, but
43:42
they've got AI engineers. That only lasts for so long.
43:45
Eventually, they actually need to show a profit. They can only
43:48
keep that wheel spinning so long as they keep it feeling like the
43:51
AI future is gonna arrive really fast.
43:54
I think you pressure the data centers and try to slow them
43:56
down in the hopes of popping the financial bubble around AI.
44:00
Because if you can pop that financial bubble, hopefully, the
44:04
the fallout isn't too extreme for the economy. But, like, I
44:08
think it's Paul Krugman has said a few times that at this point,
44:10
the the stock market is basically like three Nvidias in
44:13
a trench coat. Right? Like, circuit of finance deals between
44:17
all of these companies means that if you can pop that finance
44:21
bubble, that creates an awful lot of havoc for these companies
44:26
similar in form to the great financial crisis.
44:29
Hans Buetow: Right.
44:29
Dave Karpf: And it's after the great financial crisis that the
44:31
banks finally have to accept regulation that they didn't
44:35
want. Right? Like, they then need to do reporting that makes
44:37
them uncomfortable because after that, they no longer had the
44:41
power to set their own... Like like, set the agenda for how
44:44
they're gonna get regulated. So kinda step one is to fight
44:47
against data centers.
44:49
Step two is to hope that that leads to a popping of the
44:52
finance bubble. And, I wanna remind people, when you're in a
44:55
bubble environment, the best thing you can do is pop it
44:59
earlier rather than later. Like, I'm not rooting for anybody to
45:02
lose their investment dollars here, but, like, this this
45:05
bubble, like, either it leads to the unstoppable AI future where
45:10
everything is abundant and everyone lives for forever
45:12
because we get uploaded to the cloud, or it's a finance bubble
45:16
and it is going to pop. So, like, it's either Valhalla or
45:20
these things are nuts and they need to pop sooner rather than
45:23
later. I'm betting that we're not about to become super
45:27
intelligent AIs that all live for forever in the cloud.
45:30
So in that case, we should be hoping that the bubble pops
45:32
early. So you try to pop the bubble by pressuring the data
45:35
centers. That's the one... The leverage point that we have.
45:38
That then resets the political dynamics, the power dynamics of
45:42
the industry.
45:43
And then you hope probably with Europe leading the way since
45:45
they actually have competent regulators, you then hope that
45:48
regulators come in and say, well, okay. We're gonna need to
45:52
set some rules and some liabilities and some boundaries.
45:55
And no, like Dario and Sam Altman and Elon Musk, you don't
45:58
get to pick and choose the regulations of the regulators.
46:00
And also, sort of keep in mind that that weird information
46:03
asymmetry where people with the most technical knowledge about
46:07
AI, the people who work at these companies, have also been
46:10
engaging in such intense groupthink for so long now.
46:14
Like, they're all part of this one weird subculture that even
46:17
though they're quite knowledgeable and they're
46:19
genuine in their concerns, that doesn't mean that they're right.
46:21
It it means that there needs to be some check on their power and
46:24
authority, and they're not gonna create that themselves.
46:36
Hans Buetow: Dave Karp, thank you so much for this help. This
46:40
is really genuinely comforting to me, but also illuminating.
46:45
Like, I feel like I have a much better grasp.
46:47
Dave Karpf: Great talking to you. My pleasure.
46:48
Hans Buetow: So where can folks find you, and, what is your new
46:52
book called one more time? I want... I'm excited to read it.
46:56
So I want
46:56
Dave Karpf: everyone be excited to read it. So I'm on blue sky,
46:59
davecarp.besocial. It's davekarpf.bsky.social. I have a
47:06
weekly newsletter that's on Beehive now. It used to be on
47:08
Substack and moved to Beehive.
47:09
That's davecarp.beehiiivv.com. So you can subscribe there. And
47:15
the book is called Vaporware Incorporated, how Silicon Valley
47:18
sells the future to control the present. It's coming out January
47:22
12 from Norton. You can preorder it wherever books are sold.
47:25
Hans Buetow: And we'll put links to all that stuff in the show
47:27
notes. Dave, thanks again.
47:28
Dave Karpf: Thanks a lot.
47:49
Mike Rugnetta: That is the show we have for you this week. We're
47:51
gonna be back here in the main feed on or around Thursday,
47:54
September 24. Never Post is a listener supported podcast. Ads
48:00
cover the operating expenses, but it is our members that pay
48:04
for the time and resources to write, research, book guests,
48:07
record, edit, score, and sound design every episode. If you
48:12
wanna become a member, I give you my solemn oath that every
48:16
dollar you send to us will go towards PDFs and sound effects,
48:20
and we will not, I repeat, we will not usher in an apocalypse
48:25
of any kind, no matter what Georgia is saying in the
48:29
newsletter.
48:29
Neverpo.st. Become a member today. Never Post's producers
48:35
are Audrey Evans, Georgia Hampton, and the mysterious
48:38
doctor first name, last name. Our senior producer is Hans
48:41
Buto. Our executive producer is Jason Oberholzer, and the show's
48:44
host, that's me, is Mike Cognetta.
48:48
You have to begin somewhere. The devil of your empty pocket moves
48:51
as eschar go up the artery of a hollow arm, ending on the lip of
48:56
your dismay, it shows, in the brillo morning of a shaving
49:00
mirror. It's that morning always, and it is that morning
49:05
now, and you must fight, not with fists, but with an eraser.
49:10
The duelist awaits a ham sandwich on the dock where your
49:14
ship comes in. Excerpt of Lonely Tylenol by Peter Gietze.
49:21
Never Post is a production of Charts and Leisure, and it's
49:24
distributed by Radiotopia.