Latest / Trumponomics / Cory Doctorow Says Be Skeptical of the AI Sales Pitch

Transcript

Cory Doctorow Says Be Skeptical of the AI Sales Pitch · Trumponomics

0:02
Speaker 1: Bloomberg Audio Studios.
0:04
Speaker 2: Podcasts. Radio. News. Let's just have enough object permanence to
0:10
Speaker 2: win a game of peekaboo and not assume that when
0:14
Speaker 2: these people who are barreling towards a flotation, who are
0:17
Speaker 2: losing money at a rate never seen before, who are
0:20
Speaker 2: reliant on normal investors bailing out their insiders as a
0:25
Speaker 2: way of walking away with their pocketbooks intact, let alone
0:28
Speaker 2: having the technology continue. that their statements just shouldn't be
0:31
Speaker 2: taken at face value. There should be a certain degree
0:33
Speaker 2: of healthy skepticism when they talk about this stuff.
0:46
Speaker 1: I'm Stephanie Flanders, head of government and economics at Bloomberg,
0:49
Speaker 1: and this is Trumponomics, the podcast that looks at pretty
0:51
Speaker 1: much everything in the economic world of Donald Trump. This week,
0:56
Speaker 1: we have a summer conversation with Cory Doctorow, blogger, journalist,
1:00
Speaker 1: science fiction writer, activist in support of online rights. He
1:04
Speaker 1: was the author, you might remember, of Enshitification. But we're
1:08
Speaker 1: talking today about his latest book, The Reverse Centaur's Guide
1:12
Speaker 1: to Life After AI, How to Think About Artificial Intelligence Today.
1:16
Speaker 1: before it's too late. I read this actually just on
1:20
Speaker 1: a few days vacation and immediately thought of bringing Corey
1:23
Speaker 1: onto Trumponomics because although he takes seriously some of the
1:28
Speaker 1: concerns and debates about life after AI that we often
1:31
Speaker 1: discuss on the show, and he offers this reverse centaur
1:35
Speaker 1: idea that we'll get into as a way of thinking
1:38
Speaker 1: about it, but he also thinks fundamentally and fervently that
1:42
Speaker 1: the AI companies are full of it. Most of what
1:45
Speaker 1: they say about AI's capabilities is either hugely optimistic or
1:49
Speaker 1: just not possible in any world any of us is
1:52
Speaker 1: likely to live in. And with all that, the book
1:55
Speaker 1: is also well-written, funny, and short.
1:57
Speaker 2: Hooray!
1:58
Speaker 1: Corey Doctorow, welcome to Trumponomics.
2:00
Speaker 2: Well, thank you very much. Pleasure to be on. Thank you.
2:08
Speaker 1: I was thinking about it because although it is quite
2:09
Speaker 1: a short book, which is always welcome on holiday and
2:11
Speaker 1: when you're reading an awful lot about AI in other contexts,
2:14
Speaker 1: there is many sides to it. On the one hand,
2:17
Speaker 1: it is a guide to thinking about AI. And in
2:20
Speaker 1: some ways, I would say to listeners, this is a
2:22
Speaker 1: kind of companion discussion to the one we had with
2:25
Speaker 1: Sarah O'Connor about her book, We Are Not Machines, back
2:28
Speaker 1: in June. But you're also calling on many of the
2:34
Speaker 1: grander claims about AI that animate a lot of that
2:38
Speaker 1: discussion and concern. So I want to get into all
2:42
Speaker 1: of that. But first, I guess we should explain your
2:45
Speaker 1: reverse centaur idea. I don't know if it's going to
2:48
Speaker 1: quite catch on in the same way that gentrification did,
2:50
Speaker 1: but we could do our best.
2:52
Speaker 2: Sure. Well, it won't surprise you or your listeners to
2:55
Speaker 2: know that this isn't the first time we've had a
2:56
Speaker 2: conflict between automation and capital and labor. And so there's
3:00
Speaker 2: pretty good literature describing what happens when automation is taken
3:03
Speaker 2: up either by workers or by their bosses. And broadly,
3:06
Speaker 2: we find that when workers are in charge of automation adoption,
3:10
Speaker 2: they tend to use new tools in order to improve
3:12
Speaker 2: the quality of their outputs. Whereas capital, because they want
3:16
Speaker 2: to maximize a resource that's depreciating off their balance sheet
3:19
Speaker 2: or a subscription they're paying as a fixed cost monthly,
3:21
Speaker 2: wants to improve throughput, realize a good profit from that
3:24
Speaker 2: at the expense of your suppliers and your customers. And
3:28
Speaker 2: so this brings us to AI and Centaurs and reverse Centaurs.
3:31
Speaker 2: In automation theory, a centaur is someone who's assisted by
3:35
Speaker 2: a machine. So whether using a spell checker or riding
3:37
Speaker 2: a bicycle, you are the human head. You have the discernment.
3:41
Speaker 2: You have the judgment. You exercise will. And then there's
3:45
Speaker 2: a machine below you, you know, the human head on
3:47
Speaker 2: the horse's body. And it's stronger than you and it's
3:50
Speaker 2: faster than you. It's able to go longer than you can.
3:53
Speaker 2: But it's not bossing you around. And the corollary, of course,
3:56
Speaker 2: is that a reverse centaur is the reverse. It's the
3:59
Speaker 2: machine head on the human body, the person who's being
4:01
Speaker 2: directed by the machine. And, you know, where you have
4:04
Speaker 2: a system where you're trying to increase throughput because you've automated,
4:08
Speaker 2: you are always going to be running the machine at
4:10
Speaker 2: the limit of the weakest, slowest part of the system,
4:14
Speaker 2: which is the person. We see this with Lucy and
4:15
Speaker 2: Ethel at the end of the conveyor belt, trying to
4:17
Speaker 2: get as many bonbons as possible into the chocolate box
4:19
Speaker 2: or Charlie Chaplin being sucked into the gears of the
4:22
Speaker 2: machine in modern times. The reason that the motif of
4:25
Speaker 2: automation is the speed up is that when you're a
4:29
Speaker 2: peripheral for a machine doing the part of the job
4:31
Speaker 2: the machine can't do when you're only there because your
4:34
Speaker 2: boss put seven figures into the Amazon warehouse and wants
4:38
Speaker 2: to realize the maximum dividend from it, that machine's going
4:40
Speaker 2: to run as fast as it can, which is really
4:42
Speaker 2: as fast as you can. And as long as you
4:45
Speaker 2: can go, and we all know if you work as
4:47
Speaker 2: fast and as long as you can, you will make
4:48
Speaker 2: a mistake.
4:49
Speaker 1: We actually did have a discussion around the latest round
4:52
Speaker 1: of Amazon automation in the UK a few weeks ago.
4:57
Speaker 1: And I think some of these absolutely sort of what
4:59
Speaker 1: they would call the cutting edge ones have less injury,
5:03
Speaker 1: but the jobs themselves have become less. if anything, even
5:08
Speaker 1: more soulless than they were and even more kind of
5:11
Speaker 1: demotivating than in the past. People aren't running around these
5:14
Speaker 1: warehouses anymore. All these robots are coming to them, but
5:17
Speaker 1: they are definitely still probably suffering more from the psychological
5:21
Speaker 1: problems of having a completely soulless job, which, as you say,
5:24
Speaker 1: is defined by what the machines can't do. The way
5:27
Speaker 1: you started reminded me that you have what some people
5:29
Speaker 1: would say was a kind of Marxist tinge to the
5:32
Speaker 1: way you think about the world. And particularly when you
5:36
Speaker 1: focus on the sort of power relations in AI, you
5:41
Speaker 1: say we should look at not what the AI tool does,
5:44
Speaker 1: but who to and who for.
5:47
Speaker 2: Yeah, I don't know if that's quite Marxist, but certainly
5:50
Speaker 2: I do think that doing a neoclassical econ degree inflicts
5:54
Speaker 2: a kind of very specific neurological injury that makes you
5:56
Speaker 2: incapable of perceiving or reasoning about power and leads you
5:59
Speaker 2: to engage in fantasies about things like reveal preferences, where
6:03
Speaker 2: you're You know, you observe someone who's flogged their kidney
6:05
Speaker 2: to make the rent and go, look at that person
6:06
Speaker 2: with the revealed preference for one kidney rather than thinking
6:10
Speaker 2: at all about power.
6:11
Speaker 1: Well, as someone who studied Marxism alongside neoclassical economics, I
6:15
Speaker 1: don't know whether that's just kind.
6:16
Speaker 2: Of termly confused.
6:19
Speaker 1: You're speaking to something that a lot of people say
6:21
Speaker 1: in terms of having this desire to have a more
6:25
Speaker 1: human-centered approach to AI, to have AI be supporting people.
6:29
Speaker 1: humans in their work as opposed to humans kind of
6:33
Speaker 1: running around trying to fill the holes left over by AI.
6:36
Speaker 1: But that doesn't feel like the path we're on when
6:38
Speaker 1: you look at the way these tech companies are structured
6:42
Speaker 1: and how much money they have invested on mass automation.
6:46
Speaker 2: Well, I would say that that human centricity is not
6:49
Speaker 2: just about being squishy and big hearted and wanting people
6:52
Speaker 2: to have a good working life. It's also about the
6:55
Speaker 2: kinds of outputs that AIs can produce and And the
6:59
Speaker 2: limitations on producing high quality output with AI. So one
7:03
Speaker 2: of the things about the centaur versus centaur framing is
7:06
Speaker 2: that it resolves a seeming paradox, which is that you encounter,
7:09
Speaker 2: especially like skilled white collar workers who are quite senior
7:13
Speaker 2: and are historically good, reliable narrators of their own experience
7:16
Speaker 2: who say, well, I've just started using AI to write software.
7:19
Speaker 2: I write software. I'm good at it. And I can't
7:21
Speaker 2: tell you how great the software I'm writing is today.
7:23
Speaker 2: It's so wonderful. I've become just like an incredible, like
7:28
Speaker 2: Louis Pasteur of app making. And then on the other hand,
7:32
Speaker 2: you have equally reliable narrators of their experience who are
7:35
Speaker 2: equally senior who say like, I work in aviation systems.
7:39
Speaker 2: No one should get on an airplane ever again because
7:41
Speaker 2: we're shoveling tech debt into the sky with the help
7:44
Speaker 2: of AI at a rate never seen. And that will
7:46
Speaker 2: never be mitigated, right? And so the question is how
7:48
Speaker 2: do we resolve the seeming paradox of two groups of people,
7:52
Speaker 2: very similar, seem to be doing the same thing, but
7:53
Speaker 2: having a different experience. Well, once you realize that one
7:56
Speaker 2: group are the centaurs and the other group are the
7:58
Speaker 2: reverse centaurs, you realize they're not doing the same thing
8:01
Speaker 2: at all, right? There's just some superficial commonalities that when
8:04
Speaker 2: a skilled worker can direct their adoption of a tool broadly,
8:08
Speaker 2: I think that that is good for the quality of
8:10
Speaker 2: their outputs because that's what it means to be skilled
8:13
Speaker 2: and have discernment. But specifically in the case of AI,
8:16
Speaker 2: these defects that are intrinsic to AI systems that they
8:19
Speaker 2: whimsically call hallucinations, which I think sounds better than we
8:23
Speaker 2: have a defective product, These defects can only be mitigated
8:27
Speaker 2: by people who can distinguish a valid output from an
8:30
Speaker 2: invalid output. When we talk about using chatbots to teach kids,
8:34
Speaker 2: by definition, kids don't understand the subject the chatbot is
8:38
Speaker 2: trying to teach them. They are incapable of knowing when
8:41
Speaker 2: the chatbot is quote-unquote hallucinating, experiencing a defect, which means
8:45
Speaker 2: that you really can only get reliable outputs out of
8:51
Speaker 2: an AI if if you operate it at the speed
8:55
Speaker 2: that corresponds to the ability of a skilled practitioner to
8:58
Speaker 2: carefully examine its outputs, and when you increase that speed,
9:02
Speaker 2: even if you have a very skilled worker in the
9:05
Speaker 2: human in the loop, they develop what is called automation blindness.
9:09
Speaker 2: And anyone who's ever gone through an airport checkpoint understands this.
9:12
Speaker 2: You cannot remain vigilant for things that occur very infrequently,
9:15
Speaker 2: which is how we've produced in the form of the
9:17
Speaker 2: TSA the world's water bottle spotting as mother * * * * * *
9:21
Speaker 2: history has ever seen, who 95% of the time fail
9:25
Speaker 2: to spot the guns that red teams smuggle through the checkpoints, right?
9:28
Speaker 2: And so we see this with white shoe law firms
9:31
Speaker 2: that have these skilled lawyers who get the chatbots to
9:34
Speaker 2: help them with their briefs and they run down the
9:37
Speaker 2: slippery slope. And the next thing you know, there's a
9:39
Speaker 2: federal judge tearing three strips out of them and saying,
9:42
Speaker 2: I catch you with one more hallucinated citation, I'm referring
9:45
Speaker 2: you to the bar, and I'm gonna make your client
9:48
Speaker 2: pay your opposing counsel's legal bills for reviewing your defective draft.
9:53
Speaker 1: The classic things in medicine, people say, where you'll be
9:56
Speaker 1: able to use the AI for scanning all these radiology
9:59
Speaker 1: and other things, And you point out, again, this kind
10:01
Speaker 1: of automation blindness actually makes the doctors less good at
10:04
Speaker 1: spotting things with that.
10:06
Speaker 2: Well, I guess it really depends on the configuration. I
10:08
Speaker 2: actually do think AI can produce higher quality outputs. But
10:11
Speaker 2: the difference is, I think that those higher quality outputs
10:14
Speaker 2: don't realize a cost savings. I think that if you
10:17
Speaker 2: want to fire 99 of your programmers and have the
10:20
Speaker 2: remaining one mark the AI's homework, you're going to get
10:22
Speaker 2: a lot of bad software. If you keep all the
10:25
Speaker 2: programmers on the strength and add an AI that they
10:28
Speaker 2: use or don't use as they see fit with the
10:31
Speaker 2: token budget to go with it, some of them will
10:33
Speaker 2: use it to produce better software, right? But that's not
10:36
Speaker 2: the same as a wage savings. And while I think
10:39
Speaker 2: there is a market for tools that improve the quality
10:41
Speaker 2: of outputs, no one who's investing in Sam Altman's money
10:45
Speaker 2: furnace is saying Let's see how many sales calls you
10:47
Speaker 2: can have where your salespeople go out to a manager
10:49
Speaker 2: and say, how can I increase your cost today? Right.
10:52
Speaker 2: That's just not the return that they're anticipating. So you
10:55
Speaker 2: mentioned radiology. I've got a personal experience with this. About
10:59
Speaker 2: seven weeks ago now, a radiologist at the Kaiser Hospital
11:02
Speaker 2: in Los Angeles told me that I'm cancer free. It's
11:05
Speaker 2: very nice. It was always considered a very treatable form
11:07
Speaker 2: of cancer. But you worry and I've spent a lot
11:09
Speaker 2: of time. talking to radiologists during my course of treatment
11:12
Speaker 2: and also reading the literature on radiology and AI. And
11:16
Speaker 2: it seems very clear that there are some solid mass
11:18
Speaker 2: tumors that some chatbots can spot that some radiologists miss.
11:23
Speaker 2: And that Kaiser Hospital, it's right next to the big
11:26
Speaker 2: church of Scientology because that's how we roll in Los Angeles.
11:29
Speaker 2: And if there was like an AI salesman like, deking
11:32
Speaker 2: out a Scientologist who wanted to give him a personality
11:35
Speaker 2: test to meet with the CEO of the Kaiser Hospital
11:37
Speaker 2: and say, look, here's the plan. You've got the 10
11:40
Speaker 2: radiologists you employ at $ 300, 000 a year each. That's a
11:43
Speaker 2: $ 3 million wage bill. Each of them runs 100 x-rays
11:47
Speaker 2: a day. Give me another million dollars. And we're going
11:51
Speaker 2: to stick a chat bot behind them that a couple
11:52
Speaker 2: of times a day taps them on the shoulder and says,
11:54
Speaker 2: take another look at that one. So in addition to
11:56
Speaker 2: the extra million, you're probably going to have to hire
11:58
Speaker 2: another part-time radiologist to take up the slack. but fewer
12:01
Speaker 2: people are going to die of cancer. As a cancer survivor,
12:04
Speaker 2: I'm totally all over that. That's not the sales call, right?
12:07
Speaker 2: The sales call is you go past the Scientologist, avoid
12:10
Speaker 2: this personality test, get up to the CEO's office, say
12:13
Speaker 2: fire nine of your 10 radiologists, that's 2.7 million a
12:17
Speaker 2: year in wage savings, split half of it between you
12:19
Speaker 2: and Sam Altman, have the remaining radiologists mark the AI's homework,
12:23
Speaker 2: turn them into what Dan Davies calls the accountability sink.
12:26
Speaker 2: They're the one who takes the blame when someone dies
12:28
Speaker 2: of cancer. And I think that's like the sales pitch
12:31
Speaker 2: everyone's counting on. That's why we see this, I think, frankly,
12:35
Speaker 2: very irrational exuberance to invest in AI companies. And it's
12:40
Speaker 2: why I think this sector does not have a future
12:44
Speaker 2: as a sector dominated by extremely money-hungry firms that replace
12:50
Speaker 2: every trivial task with heavily subsidized AI. It's just not
12:54
Speaker 2: going to happen. I guess.
12:56
Speaker 1: They may not be very good at saving costs for companies,
13:00
Speaker 1: but they could be very good at selling to your
13:03
Speaker 1: employer that they can save costs. So the risk is,
13:07
Speaker 1: and you identify this in the book, you could still
13:10
Speaker 1: get all the disruption that people are worried about, even
13:13
Speaker 1: if ultimately the savings aren't there or certainly the quality
13:17
Speaker 1: of the work is not there when you replace most
13:19
Speaker 1: people with AI. There's one piece of this future that's
13:23
Speaker 1: been very prominent. There was the Citrini kind of analysis,
13:27
Speaker 1: probably on the far end of, this is going to
13:30
Speaker 1: happen much faster than people think. And you're probably on
13:33
Speaker 1: the other end of the spectrum. There was a recent
13:35
Speaker 1: Economist magazine cover that was basically suggesting that agents were
13:38
Speaker 1: going to cause total chaos for government departments all over
13:42
Speaker 1: the world because everyone was suddenly going to be able
13:44
Speaker 1: to put in every claim and resist every parking ticket,
13:49
Speaker 1: whatever it is, use all their rights to the absolute
13:52
Speaker 1: fullest without needing to pay for lawyers. But that world
13:55
Speaker 1: where also we're all sort of able to do enormous
13:59
Speaker 1: amounts of comparison shopping for our car hire or any
14:02
Speaker 1: of these other things, you claim relies on companies and
14:06
Speaker 1: websites all over the world agreeing to consistently label things
14:11
Speaker 1: and organize themselves in a way that agents can easily
14:15
Speaker 1: navigate and as you point out they don't necessarily have
14:19
Speaker 1: that much interest in doing that so you basically say
14:22
Speaker 1: that's not going to happen or at least not to
14:24
Speaker 1: the degree that people are saying and that just seemed
14:27
Speaker 1: to me is that if true that just seemed to
14:29
Speaker 1: be a massive flaw in this whole view of the
14:33
Speaker 1: world Yeah, so.
14:34
Speaker 2: The first generation of agentic AI really failed badly. So
14:38
Speaker 2: this was where you tried to create a single agent
14:40
Speaker 2: and say to it, figure out how to use a
14:43
Speaker 2: search engine, then find the travel websites, then figure out
14:46
Speaker 2: how to book a flight, then go all the way
14:48
Speaker 2: through the workflow, then tell me how much the cheapest
14:50
Speaker 2: flight is that satisfies the following requirements, right? And that
14:54
Speaker 2: was just catastrophic. Those tasks are so specialized that no
14:58
Speaker 2: one could get an agent to really do them. So
15:00
Speaker 2: now what the agentic workflow looks like is decomposing the
15:03
Speaker 2: task to a bunch of specialist bots. So you have
15:05
Speaker 2: the web searching bot, and you have the thing that
15:08
Speaker 2: knows what airport codes are, and you have the thing
15:10
Speaker 2: that can parse out the junk fees and whatever. And
15:12
Speaker 2: so you're breaking this down to a series of specialist agents,
15:15
Speaker 2: each of which hands off to the next one. And
15:18
Speaker 2: the claim of the agentic firms, so let's just sort
15:21
Speaker 2: of steel man their claims here, right? The claim of
15:23
Speaker 2: the agentic firms is that they can get something like 95%
15:27
Speaker 2: accuracy out of these highly specialized bots. Well, the thing
15:33
Speaker 2: is that if you decompose a project into like 20, 95%
15:37
Speaker 2: accurate steps, you get error rates, right? They're multiplicative, multiplicative
15:42
Speaker 2: error rates. A 95% accurate task performance multiplied by another 95%
15:47
Speaker 2: task performance. Once you get about 14 steps in, you're
15:49
Speaker 2: under 4% reliability. So that's looking like it's not going
15:54
Speaker 2: to be hugely reliable either, except for some pretty trivial instances.
15:59
Speaker 2: And not only that, but it's very expensive. because each
16:01
Speaker 2: of these chatbots is burning its own set of tokens, right?
16:04
Speaker 2: So decomposing the task at the expense of doing more compute.
16:07
Speaker 2: And so then you have the big idea that everyone's
16:10
Speaker 2: got now, which is agentic workflow, which is that you
16:13
Speaker 2: get everyone in the world who has a website that
16:15
Speaker 2: might be addressed by an agent to redesign that website
16:18
Speaker 2: for machine-to-machine communication, where the agent says, what is the
16:22
Speaker 2: true price of a flight from New York to London
16:25
Speaker 2: next Wednesday? And it gives a faithful response. Now, websites
16:31
Speaker 2: go to enormous lengths to hide this ball, right? To
16:35
Speaker 2: stop you from actually knowing what these prices are. This
16:38
Speaker 2: is how we get surveillance pricing, right? The entire direction
16:42
Speaker 2: travel runs against this right now. And I'm not saying
16:44
Speaker 2: that's good.
16:44
Speaker 1: You should explain what it is. I mean, although it's
16:46
Speaker 1: sort of an element of like basic economics segmenting your market,
16:49
Speaker 1: but it's sort of on steroids.
16:51
Speaker 2: Yeah. So the idea here is that whatever AI is
16:55
Speaker 2: bad or good at, one thing it's really good at
16:58
Speaker 2: is multivariate statistical analysis and automatic experimentation. So you have
17:02
Speaker 2: these massive surveillance dossiers on everyone who's buying and selling
17:06
Speaker 2: in the world, whether that's workers selling their labor or
17:09
Speaker 2: consumers buying the products of labor. Largely, that's because the U.S.
17:12
Speaker 2: hasn't updated its privacy law since 1988 when it became
17:16
Speaker 2: illegal for video store clerks to disclose your VHS rental history.
17:19
Speaker 2: And every other form of consumer privacy invasion remains legal
17:22
Speaker 2: in the United States because they just can't pass a
17:25
Speaker 2: privacy law. And in Europe, they do have a privacy law,
17:28
Speaker 2: but all the tech companies pretend they're Irish so it
17:30
Speaker 2: doesn't get enforced. So you have this kind of privacy
17:32
Speaker 2: vacuum in global law. So you've got these very detailed
17:35
Speaker 2: dossiers on everyone. And so you ask the AI not
17:39
Speaker 2: to do what they were doing in the Don Draper era,
17:41
Speaker 2: which is like dreaming up demographic categories and going, oh,
17:45
Speaker 2: you're a depression-haunted can stalker. You're a high school dropout
17:49
Speaker 2: from the Midwest. You're, you know, whatever. And then like
17:51
Speaker 2: running focus groups to find out about you. Instead, you
17:53
Speaker 2: say naively, take all these people... Based on their behavioral
17:57
Speaker 2: data and just naively put them in different buckets and
18:00
Speaker 2: now start experimenting on them. So when a consumer comes
18:03
Speaker 2: to you before eight in the morning, that's a variable
18:06
Speaker 2: you can control. You might discover without ever planning to
18:09
Speaker 2: that parents before eight in the morning, when they're trying
18:12
Speaker 2: to get the kids at the door, we'll take a 15%
18:14
Speaker 2: charge increase without ever even having to plan it. Same
18:18
Speaker 2: thing happens with wages. AI is actually remarkably good at
18:20
Speaker 2: this is just basic multivariate analysis. with a lot of
18:23
Speaker 2: automated A-B testing. And it's the kind of thing AI
18:26
Speaker 2: is great at. And so what we see is more
18:28
Speaker 2: and more labor and consumer surplus being harvested by capital.
18:32
Speaker 2: This is a thing that cuts against the Hayekian idea
18:35
Speaker 2: of markets as accumulators of information, right? Because you're not
18:39
Speaker 2: getting these demand and supply signals. It's effectively a command
18:43
Speaker 2: economy being run by people at the center of the
18:47
Speaker 2: economy who are acting as economic planners on the basis
18:50
Speaker 2: of giant amounts of surveillance data. Thank you.
19:05
Speaker 1: You mentioned the cost of the credits, the tokens that
19:07
Speaker 1: are being used by the AI. I just wonder whether
19:11
Speaker 1: that will end up putting a sum constraint on this
19:14
Speaker 1: because it goes to the sort of economics, what you
19:17
Speaker 1: would say was the bad economics of AI, that every
19:19
Speaker 1: model has cost more to produce, which is exactly opposite
19:22
Speaker 1: of what we've seen with many other technologies. And we
19:25
Speaker 1: see that also in terms of the users as well.
19:28
Speaker 1: You know, famously, people having been madly encouraged by their
19:31
Speaker 1: employers to use AI and now having this desperate reverse ferret,
19:34
Speaker 1: as we would say in the UK, where it's like, no, no,
19:36
Speaker 1: you have to be limiting your tokens. You have to
19:38
Speaker 1: show the value. Are costs going to be a constraint
19:41
Speaker 1: sooner than some might think? Or does it just produce
19:44
Speaker 1: more stratification between the sort of have AIs and the
19:47
Speaker 1: have-nots or the have-shoddy AIs?
19:50
Speaker 2: Well, I think the costs are so high that the
19:53
Speaker 2: haves and the have-nots don't really apply here. I think
19:55
Speaker 2: that there is a kind of trap where you're amortizing
19:58
Speaker 2: your fixed costs across a lot of customers. At the
20:01
Speaker 2: same time, you're subsidizing them a great deal. Effectively, the
20:04
Speaker 2: AI companies are selling $ 100 bills for $ 1 apiece. And
20:08
Speaker 2: when they started cleaning out their balance sheets preparatory to
20:11
Speaker 2: their flotations, they were like, maybe we can charge $ 5
20:13
Speaker 2: for these $ 100 bills. And as you say, all the
20:16
Speaker 2: CEOs who've been saying, if I catch you not using AI,
20:18
Speaker 2: you're fired. Switch from saying, if I catch you using AI,
20:21
Speaker 2: you're fired. Because those $ 100 bills are not worth $ 5 each.
20:24
Speaker 2: And the thing is that if you switch everyone in
20:28
Speaker 2: a $ 200 plan to a $ 2, 000 plan and 90% of
20:28
Speaker 2: them nope out, then like everyone else has to pay more. Right.
20:37
Speaker 2: And so more of them are going to know about, too.
20:39
Speaker 2: So you're spreading the same fixed costs among a diminishing pool.
20:42
Speaker 2: So I don't know that there is a future at
20:45
Speaker 2: all for like this as presently constituted form of business.
20:49
Speaker 2: Statistical extrapolation has got a future, obviously.
20:53
Speaker 1: Right.
20:53
Speaker 2: But but like the kind of business that we have
20:56
Speaker 2: now doesn't. And as you say, their unit economics suck.
20:59
Speaker 2: Every user they add costs the money. Every use costs
21:02
Speaker 2: the money. Every generation is more expensive than the last.
21:05
Speaker 2: This isn't like the web where the business press was
21:07
Speaker 2: full of audit editorials about how bosses are going to
21:09
Speaker 2: cope with the incoming cohort of young workers who demand
21:12
Speaker 2: the web at the office. Now it's full of like,
21:14
Speaker 2: how will bosses cope with the incoming group of Gen
21:17
Speaker 2: Zs who think AI is garbage and don't want to
21:20
Speaker 2: use it at all.
21:21
Speaker 1: Right.
21:21
Speaker 2: So like, it's just not the same story. And then
21:23
Speaker 2: you have the amortization schedules where these data centers, they're
21:26
Speaker 2: depreciating them on a five-year schedule with the GPUs in them.
21:30
Speaker 2: But in actual fact, they're actually like turning them over
21:33
Speaker 2: every two to three years and And then a lot
21:35
Speaker 2: of people, I think, assume that when you throw away
21:37
Speaker 2: the GPUs, you can keep the data center. And one
21:40
Speaker 2: of the ways that NVIDIA is getting so much new
21:43
Speaker 2: performance out of each generation of GPU in order to
21:46
Speaker 2: convince people that they need this year's chips in preference
21:48
Speaker 2: to last year's chips is by jettisoning the normal bedrock
21:52
Speaker 2: of product design, which is to maintain as much backwards
21:54
Speaker 2: compatibility as possible. So these new generations of chips... often
21:58
Speaker 2: have radically different power consumption, heat dissipation, networking requirements. And
22:03
Speaker 2: the retrofit, it's like you could do it, but it
22:05
Speaker 2: might be cheaper to scrape the data center to the
22:07
Speaker 2: foundations lab and start over. So, you know, these are
22:10
Speaker 2: not durable assets where you do it once and then
22:13
Speaker 2: you get to run it forever. And then even if
22:15
Speaker 2: they solve all of that, this claim that, okay, we'll
22:19
Speaker 2: make the model work and then we'll amortize it over
22:22
Speaker 2: the life of the model by getting people to use
22:25
Speaker 2: it with low cost operating expenditures. What we saw with
22:29
Speaker 2: the new generation of Claude is that it was because
22:31
Speaker 2: it was visibly better than ChatGPT and because the switching
22:34
Speaker 2: costs are so low, everyone decamped from OpenAI to Anthropic. Well,
22:39
Speaker 2: the implication here is that OpenAI now has to train
22:42
Speaker 2: a new foundation model. And when they ship it, if
22:44
Speaker 2: it's better than Claude, everyone's going to leave Anthropic and
22:47
Speaker 2: go to OpenAI. And then Anthropic is going to have
22:49
Speaker 2: to do the capital expenditure again. So there's no sign
22:53
Speaker 2: that this business ever stops being a Red Queens race
22:55
Speaker 2: where they lose ground every day.
22:57
Speaker 1: And I guess some sharp listeners will note that your
22:59
Speaker 1: references to the things that AI can do, you point
23:02
Speaker 1: out many times in the book, the things that you
23:03
Speaker 1: can use AI in a lucrative way for are generally
23:06
Speaker 1: things now that people don't like.
23:09
Speaker 2: Or are cheap, right? That's the other thing. It's like,
23:12
Speaker 2: I can't figure out the mania for robo-taxis. What language
23:15
Speaker 2: contains the phrase, as rich as a taxi driver? We
23:19
Speaker 2: are talking about displacing some of the lowest wage, most
23:22
Speaker 2: precaritized workers, workers who are providing their own capital. What
23:26
Speaker 2: an amazing deal Uber has. You have a worker who
23:30
Speaker 2: you can treat as a contractor who shows up with
23:32
Speaker 2: their own capital, who doesn't even know how to do
23:34
Speaker 2: the amortization schedule. So they're not even depreciating their car
23:38
Speaker 2: when they factor in whether it's worth doing a job.
23:40
Speaker 2: At best, they're thinking about petrol and insurance. Why would
23:43
Speaker 2: you replace that with a robot that you own and
23:45
Speaker 2: internalize your capital costs?
23:47
Speaker 1: To your point, I guess it takes us right back
23:49
Speaker 1: to the beginning about power relations. They can focus on
23:52
Speaker 1: those because they are precisely the weaker end. I think
23:56
Speaker 1: one could argue that the sort of less developed bit
23:59
Speaker 1: of the book is the sort of guide to life
24:01
Speaker 1: after bit. But one big argument that you point to
24:05
Speaker 1: that's kind of a source of a positive future is
24:08
Speaker 1: that AI works can't be copyrighted and that this could
24:11
Speaker 1: give a financial reason for companies to not want to
24:14
Speaker 1: just get rid of humans altogether because You need the
24:17
Speaker 1: human component in order to have any kind of ability
24:20
Speaker 1: to copyright and presumably to make money out of any
24:22
Speaker 1: of these things.
24:23
Speaker 2: Yeah. So I've spent the last 25 years working for
24:26
Speaker 2: the Electronic Frontier Foundation, which is, I think, the most
24:29
Speaker 2: important and most effective digital rights group in the world.
24:31
Speaker 2: I've been an activist with them. And one of the
24:34
Speaker 2: great frustrations of having done this for 25 years, although
24:36
Speaker 2: it's very rewarding work, is that you see people making
24:40
Speaker 2: the same mistakes over and over again. And so I
24:42
Speaker 2: have watched my colleagues in the media industry brief for
24:46
Speaker 2: more copyright as kind of human shields for their bosses
24:51
Speaker 2: for like a quarter of a century. And indeed, since
24:53
Speaker 2: the 70s, since the 1976 Copyright Act in the U.S.,
24:57
Speaker 2: copyright has monotonically expanded, not just in America, everywhere, because
25:00
Speaker 2: the U.S. trade rep made this a condition of their
25:03
Speaker 2: deals with their trading partners. So it lasts longer, covers
25:06
Speaker 2: more kinds of works. The statutory damages are amazing, right?
25:10
Speaker 2: $ 150, 000 per infringement in the US for copyright infringement. It
25:14
Speaker 2: restricts more activities. The media companies are more valuable than
25:17
Speaker 2: they've ever been, and they have higher profits than they've
25:19
Speaker 2: ever had. And the share going to the labor force
25:23
Speaker 2: is lower than it's ever been. We make more, both
25:25
Speaker 2: in real terms and as a proportion of those increased profits.
25:28
Speaker 2: And the paradox, again, reveals itself to be completely self-explanatory
25:33
Speaker 2: once you understand that we are bargaining with five publishers
25:37
Speaker 2: and four studios and three labels and two companies that
25:39
Speaker 2: control the ad tech and one company that controls audio
25:42
Speaker 2: books and eBooks and giving us more to bargain with
25:44
Speaker 2: is giving us more to bargain away. It's like, while
25:46
Speaker 2: it's true, they are suing the AI companies for taking
25:49
Speaker 2: the training data without asking. It's not because they want
25:52
Speaker 2: to stop models from being trained and used to erode
25:55
Speaker 2: the wages of creative workers. They're very clear that they
25:58
Speaker 2: view having to pay creative workers as a bug, not
26:00
Speaker 2: a feature. The US Copyright Office and I are, for
26:02
Speaker 2: a rare moment, entirely in accord because they hold correctly
26:06
Speaker 2: that copyright accrues only to works of human creativity. So
26:09
Speaker 2: I used to be a delegate to the United Nations
26:11
Speaker 2: World Intellectual Property Organization. This is built into global copyright treaties.
26:15
Speaker 2: This is how it works. And so what they've said
26:18
Speaker 2: is if an algorithm makes it, you can't copyright it.
26:21
Speaker 1: I think a lot of people would listen to this.
26:23
Speaker 1: And in fact, some of the people that I've described
26:25
Speaker 1: your book to have sort of said, oh, great. So
26:27
Speaker 1: we don't have to worry so much about AI.
26:31
Speaker 2: Taking over the world.
26:33
Speaker 1: Then they will laugh at your examples, this sort of
26:35
Speaker 1: ludicrous number of times that tech people, Elon Musk and others,
26:39
Speaker 1: have sort of made great claims for AI, even using,
26:42
Speaker 1: you know, the dancing robot who dances better than any
26:44
Speaker 1: other robots ever danced as a sign of the future
26:47
Speaker 1: of the humanoids. And that turns out to be a
26:49
Speaker 1: little person in a robot. So you've got various examples
26:52
Speaker 1: like that and all the jokes about ChatGPT, you know,
26:55
Speaker 1: Gujarati people typing. So you might feel relieved. But then
26:59
Speaker 1: you look at the paper and you see the latest
27:01
Speaker 1: round of stories about AI's breaking out of their sandboxes
27:07
Speaker 1: and hacking places they shouldn't hack. It sort of feels
27:09
Speaker 1: like it may be that AI can't do all the
27:12
Speaker 1: things that are supposed to save companies money, but it
27:15
Speaker 1: feels like it could still pose a pretty existential threat
27:19
Speaker 1: if we don't keep control of some of these bots.
27:22
Speaker 2: Well, this is one of my wheelhouses. I speak most
27:24
Speaker 2: years at DEF CON, the big hacker conference in Las Vegas,
27:27
Speaker 2: and I've been actually gutted because it was last week
27:29
Speaker 2: and I didn't go, and there were lots of talks
27:31
Speaker 2: about this stuff. And here's a thing, because security practitioners
27:35
Speaker 2: are like, their arts are not widely understood outside the field.
27:39
Speaker 2: It's a little insular. But it is not uncommon for
27:43
Speaker 2: automated hacking tools to escape containment. And when they do,
27:48
Speaker 2: we turn to the security researchers who built that tool
27:51
Speaker 2: and we say, why are you so bad at making sandboxes?
27:54
Speaker 2: Why are you so bad at containing your tool? Not,
27:58
Speaker 2: my God, you are a wizard. You've made the tool
28:00
Speaker 2: that can defeat a sandbox. And I think that the
28:04
Speaker 2: credulity of the reports about these security tools that, quote,
28:09
Speaker 2: escape containment, not one of which I've seen is questioned
28:13
Speaker 2: whether or not, except outside the specialist security press, has said, like,
28:17
Speaker 2: why are these companies with all this money playing with
28:20
Speaker 2: these very advanced tools so crummy and honestly lax and
28:25
Speaker 2: irresponsible in building their security sandboxes for testing this stuff?
28:29
Speaker 2: And they should be better. I think that's the more
28:31
Speaker 2: parsimonious explanation. Now, that's not to say that finding defects
28:35
Speaker 2: in software is an unimportant thing. And I think the
28:38
Speaker 2: security tools that we've seen for automatically finding defects have
28:41
Speaker 2: been very good. It's exciting to see because our software sucks, right?
28:46
Speaker 2: One of the things that at the Electronic Frontier Foundation,
28:49
Speaker 2: we spent 25 years sounding the alarm about is the
28:51
Speaker 2: extremely low quality of software being produced by commercial entities.
28:55
Speaker 2: which especially when coupled with highly surveillance practices where you
28:58
Speaker 2: have these large corpuses of data being amassed by firms
29:01
Speaker 2: that then fail to secure them well, you have this
29:04
Speaker 2: pipeline of like things that Facebook knows about you that
29:07
Speaker 2: become things that are for sale on the dark net
29:09
Speaker 2: that become things that are used to steal your identity
29:12
Speaker 2: or do a business account takeover and empty out your
29:14
Speaker 2: corporate treasury. This is a very dangerous thing. Any tools
29:16
Speaker 2: we have that improve that are great. And I think
29:19
Speaker 2: the AI tools are improving it just like many generations
29:22
Speaker 2: of tools before have improved it. It's wonderful to see.
29:26
Speaker 2: We should be happy about it. I think sometimes what
29:28
Speaker 2: we're seeing is that there's a combination of hype and misdirection,
29:33
Speaker 2: as with this sandbox thing, or, you know, Anthropic turning
29:36
Speaker 2: around one day and saying, we have an IPO coming
29:39
Speaker 2: and there's two things you need to know this week
29:41
Speaker 2: in our news cycle. The first one is that we're profitable,
29:43
Speaker 2: but not according to generally accepted accounting practices. We're actually
29:47
Speaker 2: using a different math because we're too cool for regular accounting.
29:50
Speaker 2: And we can't explain how it works to you because
29:52
Speaker 2: it would break your brain. And then also, we've made
29:54
Speaker 2: this tool called Mythos that's a really good hacking tool.
29:57
Speaker 2: But if we show you how it works, the world
29:59
Speaker 2: will end. And so you just have to take our
30:01
Speaker 2: word for it. Did we mention we have an IPO coming? Now,
30:04
Speaker 2: maybe that tool is very good. I spoke to one
30:06
Speaker 2: security researcher who said, yeah, based on what I know
30:09
Speaker 2: about them, what I think they've done. is they've built
30:11
Speaker 2: a tool that'll do $ 20, 000 worth of security consulting for
30:15
Speaker 2: $ 50, 000 worth of tokens, which would be a really good
30:17
Speaker 2: reason not to let other people play with it. So again,
30:20
Speaker 2: let's just have enough object permanence to win a game
30:24
Speaker 2: of peekaboo and not assume that when these people who
30:27
Speaker 2: are barreling towards a flotation, who are losing money at
30:31
Speaker 2: a rate never seen before, who are reliant on normal
30:34
Speaker 2: investors bailing out their insiders... As a way of walking
30:39
Speaker 2: away with their pocketbooks intact, let alone having the technology continue.
30:43
Speaker 2: Their statements just shouldn't be taken at face value. There
30:45
Speaker 2: should be a certain degree of healthy skepticism when they
30:49
Speaker 2: talk about this stuff.
30:50
Speaker 1: Skepticism is, I'm sure, warranted and welcome. And people listening
30:54
Speaker 1: to this will enjoy the tone of your approach to this.
30:58
Speaker 1: Just to push back on some of these examples, I'm
31:01
Speaker 1: sure they are reported with breathless credulity. I'm sure that
31:05
Speaker 1: we should be turning our gaze to the people who
31:08
Speaker 1: are creating these sandboxes. But there is a sort of
31:11
Speaker 1: quite lazy thing, which actually neoclassical economists tend to do,
31:15
Speaker 1: which is, you know, they might want to talk up
31:17
Speaker 1: the risks, but if they are known for creating chaos
31:21
Speaker 1: and bringing down financial systems, or at least allowing their
31:25
Speaker 1: bots to steal people's money. That's not going to be
31:28
Speaker 1: in their commercial interest. So we should just assume that
31:30
Speaker 1: they will sort this out. I mean, we've made similar
31:33
Speaker 1: arguments before the financial crisis. You know, it's in the
31:36
Speaker 1: bank's interest to understand their investments and whether they can
31:38
Speaker 1: blow themselves up. And that hasn't always worked out so well.
31:42
Speaker 1: So is there a little bit of a complacency running
31:45
Speaker 1: through some of your view on that? Can we count
31:48
Speaker 1: on them really to be able to build these better sandboxes,
31:51
Speaker 1: for example?
31:52
Speaker 2: I do think we'll have to make sandboxes better as
31:54
Speaker 2: well as tools better and so on. I'm not saying
31:56
Speaker 2: there's never going to be another instance in which a
31:58
Speaker 2: security tool malfunctions in a way that's dramatic and bad.
32:02
Speaker 2: And obviously that's going to happen. When I think about
32:05
Speaker 2: whether the practices will go on or won't go on
32:06
Speaker 2: and so on, and this mantra, incentives matter, I think
32:10
Speaker 2: about the possibility that you can get out, especially by
32:16
Speaker 2: doing flotations directly into the indexes, without having to pay
32:20
Speaker 2: any consequences if the actual thing doesn't work, right? If
32:24
Speaker 2: you say to an investor, it doesn't matter whether it works,
32:27
Speaker 2: you make money either way. Why does the investor care
32:30
Speaker 2: so much if it works? They're not there to make
32:33
Speaker 2: a better widget. They're there to make a better alpha.
32:36
Speaker 1: Last question, taking it back to the kind of worldview
32:40
Speaker 1: that we often are adopting, or at least thinking about
32:43
Speaker 1: when we do this show. We have voters in America
32:47
Speaker 1: who UK, other places, feeling pretty angry or at least
32:53
Speaker 1: concerned about AI. We see on the ground all the
32:56
Speaker 1: people complaining about the processing centers, but also people worried
32:58
Speaker 1: about their kids' jobs, all of those things. At the
33:01
Speaker 1: same time, to your point, a huge amount of money
33:03
Speaker 1: in their 401ks or their ISAs or whatever, their investments,
33:06
Speaker 1: now tied up with these companies who have invested enormous
33:09
Speaker 1: amount of physical resource of the world in building these products.
33:16
Speaker 1: If you're right... in your basic assessment of how AI's
33:21
Speaker 1: ability to really save companies money or do some of
33:23
Speaker 1: the things that it says it can do, is that
33:26
Speaker 1: good news or bad news for those voters?
33:28
Speaker 2: Well, I think it's bad news. I mean, let's not
33:30
Speaker 2: overestimate the likelihood that the median voter has got money
33:32
Speaker 2: in the stock market. The median American worker has $ 955
33:35
Speaker 2: saved for retirement.
33:37
Speaker 1: But they are invested in the economy, which could be
33:39
Speaker 1: seriously affected.
33:40
Speaker 2: That's what you have to worry about. And so seven companies,
33:43
Speaker 2: the Magnificent Seven, are 35% of the S & P 500.
33:46
Speaker 2: NVIDIA is loaning them money to buy its products. This
33:49
Speaker 2: is never a good sign. They're all passing around the
33:51
Speaker 2: same $ 100 billion IOU and pretending it's in all their
33:53
Speaker 2: bank accounts at once. You know, I think there's a
33:56
Speaker 2: chance we vaporize a third of the American stock market.
33:59
Speaker 2: And if we respond to that the way that we've
34:01
Speaker 2: responded to every crisis this century, except to a limited
34:04
Speaker 2: extent COVID, with austerity, We are going to drive more
34:08
Speaker 2: people into the arms of fascists. I think that is
34:11
Speaker 2: our big worry. It's not that we're going to teach
34:13
Speaker 2: too many words to the word-guessing machine that's going to
34:15
Speaker 2: wake up, become God, and turn us into paperclips. It's
34:17
Speaker 2: like worrying that if you breed horses to run faster
34:20
Speaker 2: and faster, eventually they're going to give birth to a
34:22
Speaker 2: world-destroying fleet of locomotives. We're not word-guessing machines that know
34:26
Speaker 2: more words than the computer. We're different. It's not to
34:29
Speaker 2: say that someday someone might not make an artificial consciousness,
34:32
Speaker 2: just not by guessing words. But I do worry... that
34:35
Speaker 2: history will repeat itself. And the thing that it's going
34:38
Speaker 2: to repeat is that financial engineers are going to suck
34:41
Speaker 2: up a ton of the real economy into a totally
34:45
Speaker 2: unproductive piece of financial gamesmanship that is going to destroy
34:48
Speaker 2: the economy. We're going to have to hold the bag
34:50
Speaker 2: for it. We're going to have to clean up for it.
34:52
Speaker 2: And that it's going to further erode our politics and
34:56
Speaker 2: our civilization. And I'm scared as hell about that. Corey, Dr. Ong.
35:01
Speaker 1: So that is The Reverse Centaur's Guide to Life After AI,
35:04
Speaker 1: a bracing 222 pages for your summer reading list. It's
35:10
Speaker 1: definitely a different take than we've heard, but it also,
35:14
Speaker 1: I think, will resonate with a lot of people listening.
35:16
Speaker 1: So thanks again.
35:17
Speaker 2: Oh, it's absolutely my pleasure. Thank you.
35:23
Speaker 1: Thanks for listening to Trumponomics from Bloomberg. It was hosted
35:26
Speaker 1: by me, Stephanie Flanders. And I was joined by the blogger, journalist,
35:30
Speaker 1: science fiction writer and author, Corrie Doctorow. Trumponomics was produced
35:35
Speaker 1: by Moses Andam and Sam Asadi with help from Amy Key.
35:39
Speaker 1: And sound design, as ever, is by Blake Maples. Cheryl
35:42
Speaker 1: Bromley is now Bloomberg's head of podcasts. To help others
35:46
Speaker 1: find us, please rate and review us highly wherever you listen.