Latest / The Overlap / AI: What They're Not Really Saying

Transcript

AI: What They're Not Really Saying · The Overlap

0:00
Sixteen developers. That's it.
0:02
Sixteen. Every one of them experienced.
0:05
Average five years working on their own open source projects.
0:10
These aren't tourists. These are people who know their code bases the
0:15
way you know the layout of your own kitchen in the dark.
0:19
Code bases averaging over a million lines of code.
0:23
Meter paid them a hundred and fifty dollars.
0:27
dollars an hour to do something very simple.
0:30
Do your real work. 246 real tasks.
0:36
Half of them use the AI tools.
0:39
The good ones. Cursor Pro running Claude 3.5 and 3.7 Sonnet.
0:44
The state of the art. Half of them,
0:46
no AI. And here's the thing. Before they started,
0:51
they were asked to predict how much faster will AI make you.
0:55
They said
0:56
24% faster. The economists who study this predicted
1:01
39% time savings. The machine learning experts predicted 38%.
1:07
Everyone agreed. The AI would make them faster.
1:11
Then they actually measured it.
1:14
Like actually measured it. Not a survey,
1:17
not a vibe, a gold standard randomized controlled trial.
1:23
We call that science.
1:25
The developers using AI took nineteen percent longer,
1:30
slower, not by a little, nineteen percent slower doing their own work with
1:36
the best tools money could buy at the time.
1:40
Then, after they finished, after they were objectively measurably
1:46
slower, they were asked again,
1:48
how did the AI do? And they said it made them twenty percent faster.
1:54
They were slower. They felt faster.
1:57
They could not tell the difference between the two.
2:00
Sixteen people who write software for a living on their own coat could
2:05
not perceive that a tool was slowing them down.
2:09
They were paid to notice. They were experts.
2:13
And their own senses lied to them in the same direction at the same time.
2:18
Now hold that thought, because right now across this country and across
2:24
the country I'm doing.
2:25
currently recording this, companies are laying off tens of thousands of people.
2:30
And the reason they give is that the AI makes everyone faster.
2:35
They can feel it. Stick with me.
2:53
This is The Overlap, the show about systems of power,
2:55
labor, and American injustice.
2:57
I'm Joshua, and I'm very glad you're here.
3:00
I am ~ actually recording this in Canada right now.
3:05
Don't worry, I haven't moved yet.
3:07
I don't think I'm going to, but I am recording this ~ remotely.
3:13
So the audio might sound a little bit different today.
3:15
The ~ content, however, is not going to be.
3:19
Because we still talk about systems of power and we talk about labor
3:22
and we talk about money and politics and how it relates to you.
3:25
Quick word for Will. He's still not with us.
3:27
We're sending our love and hope and all of those things to him out in the universe,
3:32
putting out some positive vibes.
3:34
I hope he's drinking a cup of coffee right now.
3:35
And listen to this. You earned it,
3:37
buddy. We love ya. So here's where we are.
3:39
The topic today is AI. Or more precisely,
3:44
it's the gap.
3:44
The gap between what we're told artificial intelligence is doing to
3:48
the economy and what we can actually measure it doing.
3:52
And I want to be clear from the top.
3:55
This is not an episode about whether technology is cool.
3:58
We all know it's cool. Some of it is really,
4:02
really impressive. This is an episode about proof.
4:05
It's about measurement. It's about the difference between a claim and a fact.
4:10
Because here's the thing that made me want to do this this week.
4:14
So this is July 2026. It's not one story.
4:18
It's never one story, right? It's a collision of two stories.
4:22
On one side, you've got the money.
4:25
For incredibly large companies,
4:27
Microsoft, Amazon, Google, Meta.
4:31
They are projected to spend $670 billion,
4:36
ba-ba-ba-ba-ba-billion, on AI infrastructure this year alone.
4:42
Six hundred and seventy billion dollars.
4:45
That's a sixty-two percent jump from last year.
4:48
And they have burned through their free cash flow.
4:51
They're actually issuing debt to keep building what they're building because they
4:56
think the value is going to eventually outweigh the detriment and what it's costing
5:01
them. Data center debt doubled to $182 billion last year.
5:07
On the other side of this, you have Goldman Sachs.
5:10
Not some, you know, skeptical blog.
5:12
it's not an Alex Jones scenario,
5:15
although the Onion has taken over ~ Alex Jones' old show,
5:19
and it's hilarious, you have to check it out.
5:21
So this is Goldman Sachs, right?
5:23
If you haven't heard of them, crawl out from under the rock you've been hiding
5:27
under. But their own senior economist said in March of this year,
5:31
we will not find a meaningful relationship between productivity and AI adoption
5:38
At the economy wide level. You didn't hear that wrong.
5:41
No meaningful relationship after all of that money.
5:46
Now the reason both the history and the moment matter here is that we have seen this
5:52
exact movie before, more than once.
5:55
And every single time the people telling you the revolution is here
5:59
and it's measurable, they were the people getting paid for you to believe it.
6:04
So we're gonna measure.
6:05
That's the whole game today. We're gonna do the one thing that the
6:09
the AI hype artists don't want you doing.
6:11
We're gonna ask quietly, patiently,
6:14
okay, show me where's the proof.
6:18
And if the claims were true, we'd have the proof by now.
6:21
But we don't. I want you to hold on to that phrase throughout this whole episode
6:25
because it's gonna keep coming back and it's gonna feel different every time.
6:29
If the claims were true, we'd have proof by now.
6:33
Not a forecast, not a survey, not a feeling.
6:36
Proof. So let's go back. Because to understand why this pattern keeps working,
6:41
you have to understand that it is a pattern,
6:43
and that has a shape. And the shape is older than most of the people building these
6:49
models. Nineteen eighty seven,
6:51
an economist named Robert Solaw,
6:53
Nobel laureate, MIT, one of the most respected people in this entire field,
6:59
wrote a short observation to
7:01
In the New York Review of Books.
7:03
And it becomes one of the most quoted lines in the history of economics.
7:08
He said, You can see the computer age everywhere,
7:12
except in productivity statistics.
7:15
Sit with that for a second. 1987,
7:19
I was three years old. By then,
7:21
American business had spent well over a decade pouring money into computers.
7:26
the nineteen seventies, the nineteen eighties,
7:28
offices are filling up with these machines that basically generate heat.
7:34
Everyone could see them, everyone believed they were changing everything.
7:40
The revolution was really visible on every desk.
7:43
But the productivity numbers flat.
7:45
Nothing. You could not find the computer revolution in the one place that
7:51
it was sold, that it was supposed to show up in the aggregate data.
7:56
The bottom line for the whole economy.
7:59
They actually called it the Sola Paradox,
8:02
and I want to be really careful here because the hype artists around
8:07
AI really love this story. They love it because they think it ends well for them.
8:12
They say see, the gains were real,
8:14
they just came late. Be patient,
8:17
the gains from AI are coming too.
8:19
And here's the thing that's half true.
8:22
And half that's true is the half that actually damns them.
8:27
Because yes, eventually the computer revolution did show up
8:31
in the productivity numbers in the late 1990s.
8:34
But I need you to listen to how it happened.
8:38
It took roughly a decade longer than the boosters of the eighties promised.
8:43
And it didn't happen because the machines got magic.
8:46
It happened because organizations spent years,
8:50
a decade, two decades restructuring how they actually
8:56
did work around the machine. New processes,
9:00
new training, new job descriptions,
9:02
new management layers built specifically to extract value from the
9:08
out of the technology that they had purchased.
9:10
The technology alone really did nothing.
9:12
The technology plus a decade of expensive,
9:16
painful, human organizational change actually did something.
9:21
That's the real solo paradox, right?
9:24
It's not be patient, the magic is coming,
9:27
it's the technology is the easy part and the productivity gain is the hard part
9:33
and it takes 10 years longer than anyone selling it.
9:36
is willing to admit. And importantly,
9:38
the companies that actually got the productivity gain weren't the ones that bought
9:43
the most machines. They were the ones that changed how people worked around
9:48
the machines. The hardware was necessary,
9:50
but not really sufficient. The human restructure,
9:53
that was the variable that really mattered,
9:56
and nobody selling hardware ever led with that,
9:59
because frankly that's expensive.
10:01
Now fast forward.
10:02
The 1990s, we probably remember this.
10:06
This is the dot-com boom. And ~ the claims.
10:09
The internet was going to permanently accelerate GDP growth.
10:13
Analysts predicted productivity gains of three to four percent every year forever.
10:20
Some of them said the business cycle itself was over.
10:23
Recessions, a thing of the past.
10:26
We had transcended economics. We were
10:28
We were post a post economic society.
10:31
There was a famous business week cover,
10:34
and there there were wired manuf manifesto,
10:39
serious, credentialed people with a lot of letters behind their name,
10:43
stood at podiums all over the world and said that the old rules no longer applied.
10:49
That the the internet had changed the underlying physics of growth.
10:55
The NASDAQ peaked in March of
10:57
two thousand
10:58
By October of two thousand two,
11:00
it had lost seventy eight percent of its value.
11:03
Five trillion dollars in market value were gone overnight.
11:08
Vaporized. Not over years, over months.
11:12
And the companies that disappeared weren't small bets.
11:16
Pets dot com WebVans
11:21
E toys.
11:21
These were businesses that had raised hundreds of millions of dollars
11:26
on the strength story, on the feeling that the internet changed everything.
11:30
So the old rules about revenue and profit and unit economics didn't really apply
11:37
to them anymore. And the productivity gains and the the the hypers promise
11:42
was you know, they came eventually,
11:45
sort of, but a decade later than it was actually advertised.
11:50
And nowhere near three to four percent forever.
11:52
The internet was real. It's still very real.
11:55
You're listening to this podcast over the internet.
11:57
The internet was transformative.
12:00
I'm not denying that. The internet is is ~ an infrastructure that's
12:05
now so incredibly foundational that we'd now kind of forget to notice it.
12:10
And the people who told you exactly how much and exactly when were wrong
12:16
by an order of magnitude and
12:19
And a decade. The technology is real.
12:22
It was real. But the timeline and the magnitude completely fabricated.
12:27
Confly stated, completely fabricated,
12:31
right? Same shape every time. So let me give you a third one because this one's
12:37
good, and I mean that in the the darkest dystopian way possible.
12:41
IBM's Watson. Between twenty eleven and twenty sixteen,
12:46
IBM
12:48
marketed Watson as this revolutionary AI that would transform medicine
12:52
and law and and finance. They spent billions of dollars promoting Watson health.
12:58
And here was the pitch. This thing can diagnose cancer better than your doctor,
13:04
better than a human oncologist,
13:05
the future of medicine running on a mainframe.
13:08
Indy Anderson Cancer Center, one of the best arguably cancer hospitals on the planet
13:15
Bought in all the way. They spent sixty two million dollars on
13:20
a Watson oncology project. Sixty two million to a research hospital.
13:26
That's not a rounding error. That's years of clinical trials.
13:30
That's research positions and equipment and opportunity costs that don't just come
13:35
back. They actually canceled it in twenty seventeen because it failed
13:39
to produce usable results.
13:42
sixty two million dollars for a system that couldn't do the one thing it
13:47
was sold to do. It's like if you spent sixty two million dollars on
13:51
a blender that didn't make smoothies.
13:54
And the Watson Health Division,
13:55
the flagship, you know, the future of medicine,
13:58
the thing that was going to cure cancer.
13:59
IBM actually quietly sold it off in 2022 for a fraction of what they they believed
14:06
it to be worth and hyped it to be worth.
14:07
There was not a press conference,
14:09
there was no, you know, Mia Culpa,
14:11
no accountability whatsoever. Just a quiet sale and a hope that you'd forgotten
14:16
the diagnosed cancer better than doctors' part.
14:19
The press releases from 2020.
14:22
13 or actually still on the internet.
14:24
~ I pulled a couple of them and you can read them too.
14:26
The gap between what was claimed and what was delivered is right there.
14:31
Archived and it's big. And it's kind of sad.
14:36
Did they know it might not work when they were selling it?
14:39
I I can't I mean I honestly I can't tell you what was in their hearts.
14:42
I can't tell you what was in their heads,
14:43
but I know the money moved before the proof did,
14:46
every single time.
14:48
The money moves first. The proof,
14:50
if it comes at all, comes years later.
14:53
And by then, the people who made the money are gone,
14:57
and the people who lost the sixty-two million dollars are writing it off as well,
15:01
it's a lesson learned. It's just make sure not to throw good money after bad.
15:06
One more, the self-driving car.
15:07
Between 2016 and 2018, Waymo,
15:11
Uber, Tesla, all of them repeat.
15:13
Repeatedly telling you full autonomy was two years away.
15:17
Two years. Always two years. Two weeks.
15:21
Travis Kalimek, the CEO of Uber,
15:24
predicted in twenty sixteen that self driving cars would replace every single Uber
15:31
driver within a decade. Elon Musk said in twenty nineteen that Tesla would have
15:36
a million robotaxis on the road by twenty twenty.
15:39
A million. By twenty twenty.
15:42
That decade is basically now. And as of twenty twenty six,
15:45
full level five autonomy, the car that drives everywhere,
15:50
anywhere, any conditions with no human,
15:53
it's not commercially available at scale.
15:56
It doesn't exist as a product you can buy.
15:59
Now Waymo operates you know in a handful of highly geofenced cities.
16:04
And it's pretty cool if you've ever had the opportunity to ride in one.
16:08
And
16:10
Outside of them, in the rain, in a construction zone,
16:14
in a small town in rural Oklahoma with no HD map data,
16:19
just doesn't work. Uber's self driving division actually got sold off
16:23
to Aurora in twenty twenty after one of their test vehicles killed
16:27
a woman in Arizona. You might remember that from the news.
16:30
The technology moved forward, but the promises from twenty sixteen,
16:34
they've yet to materialize.
16:36
Billions of dollars on the promise,
16:38
on two years away, and the investors who funded these promises just moved
16:44
on to the next story, and the drivers who were told their jobs would
16:47
be automated out of existence,
16:49
they're still driving. Now with the added anxiety of having been told
16:54
for a decade that they're just temporary,
16:56
they're a contract, they're 1099.
16:58
So here's the pattern. And let's name it clean before we walk into the present.
17:03
Because you're about to see it wearing a completely brand new suit.
17:07
The pattern is this a genuinely interesting technology shows up.
17:11
The people positioned to profit from it make enormous,
17:16
specific, confident claims about how much it will change the economy and how fast.
17:22
Those claims drive a flood of money.
17:25
And look, the money creates its own momentum.
17:28
You have to invest because everyone else is investing because
17:32
What if it's real?
17:33
And the actual measured results when they finally arrive are smaller
17:37
and slower and messier than anyone promised.
17:41
Computers in nineteen eighty seven,
17:43
the internet in two thousand, Watson in twenty seventeen,
17:48
self driving cars. We're still waiting on all of it.
17:51
Solo told us nineteen eighty seven,
17:53
and you can see it everywhere,
17:55
except the statistics. And now there's a machine that talks back.
18:00
And the claims are bigger than they have ever been.
18:02
So let me ask the question I I keep asking myself.
18:06
Why does this still work? Why,
18:09
after computers and the internet and Watson and the self-driving
18:12
car that's always two years away,
18:14
why does anyone still fall for this specific confident number thing?
18:18
And the oddest answer is because falling for it is extremely profitable
18:23
for the people telling you to fall for it.
18:25
That's it. It's the whole engine.
18:28
Let's follow the money because it's not subtle.
18:30
So Jensen Wong, the CEO of NVIDIA.
18:32
Now, if you don't know, if you're not involved in all of that,
18:36
you are hiding under a rock. NVIDIA makes the chips that everybody needs
18:40
to build these AI systems. And since ChatGPT launched,
18:44
NVIDIA's net income has grown roughly 20 times over.
18:49
That's a 20x return. Their market cap hit around $3 trillion.
18:56
And Jensen Wong is, without exaggeration,
18:58
the single biggest financial winner of the entire AI boom.
19:02
He is the arms dealer in a gold rush,
19:06
and he has been richly historically rewarded for it.
19:11
And here's the thing that he said that you should be sitting with
19:14
at night when you're waiting on your popcorn to finish.
19:18
Wong called the CEOs who blame AI for layoffs lazy.
19:23
Lazy. He said it doesn't even make sense that companies are already using
19:26
AI enough to be replacing people.
19:29
Play that back if you need to.
19:30
The man selling the shovels is telling you the gold isn't really there yet.
19:36
Meaning the person who profits the most from this AI race is saying the thing
19:42
the AI race is justified by just hasn't happened.
19:46
And he's still selling shovels as fast as he can make them.
19:50
He's even got some pre-sold for the future.
19:53
2026 to try to buy a chip is impossible because they're sold all the
19:57
way out until 2028.
19:59
That's why you see Anthropic and and Grok are renting out their servers
20:05
and and you know putting leases on Google Gemini to make it worse for everybody.
20:09
Is because they don't have the chips to to purchase.
20:13
They have to rent them from other people.
20:15
And look, Jensen Wong is not contradicting himself.
20:18
He is being perfectly consistent from the perspective of someone whose business
20:23
model is infrastructure, not the output.
20:26
He makes money whether or not
20:28
The productivity gain ever materializes.
20:30
Hy makes money on the belief.
20:32
And so he can afford to be honest about the belief's limits because
20:35
his balance sheet doesn't depend on the belief being true.
20:39
It depends on you buying the chips while you still believe.
20:43
And that's not a contradiction he's embarrassed by.
20:45
That's the business model. Sell the infrastructure for a revolution that
20:49
has yet to be proven. And if anyone get hurt gets hurt buying into it,
20:52
you just call them lazy. Now, the four big buyers are the hyperscalers,
20:57
right? Microsoft, Amazon, Google,
20:59
Meta. Six hundred and sixty-seven billion dollars in projected capex this year.
21:06
And I told you.
21:07
They've burned through their operating cash flow and started borrowing.
21:10
Data center debt doubled to $182 billion last year.
21:15
Why? Why keep spending when your own bank,
21:18
Goldman Sachs, tells you there's no measurable economy-wide return?
21:23
Goldman's James Cavello had the answer,
21:26
and it's pretty brutal. He diagnosed the whole thing as FOMO.
21:30
For those of you who don't keep up with internet slang,
21:33
FOMO means the fear of missing out.
21:34
He said, and I quote, FOMO has proven a stronger incentive than poor stock
21:40
performance. Think about what that means.
21:43
These are the four most sophisticated companies in the world.
21:47
They have armies of economists,
21:49
risk analysts, strategy teams,
21:52
and the thing driving six hundred and sixty-seven billion dollars in spending.
21:59
Is the same thing that makes a teenager buy a stock because their group chat won't
22:04
shut up about it. What if that's real?
22:05
What if the other guy figures it out first?
22:08
I can't be the one who missed it.
22:09
The CEO of a company with a trillion dollar market cap is,
22:13
at the moment of this decision,
22:14
operating on the same psychology as someone panic buying during
22:19
the GameStop GME rut.
22:22
That's not a strategy, right? That's panic.
22:25
Dressed up as a keynote. And then the third leg of the stool,
22:28
the consultants. Mmm, everybody's favorite.
22:31
McKenzie Global Institute. McKenzie projected that AI could contribute four point
22:37
four trillion dollars in productivity gains annually.
22:40
Trillion, that's with a T, you didn't hear me wrong.
22:43
Annually.
22:44
They anticipated three to four percentage points of additional GDP growth every year
22:50
from now to twenty forty. That report got cited in boardrooms everywhere.
22:55
It got cited by executives who had not read it to justify decisions that they
23:01
had already made. It was the permission slip.
23:04
four point four trillion dollars per year.
23:08
Hold that number.
23:10
Next to what an actual sober Nobel adjacent economist said.
23:14
Because in 2024, Darren Asamoglu,
23:19
MIT Nobel laureate, published a paper called The Simple Macroeconomics of AI.
23:25
In his estimate, AI would produce somewhere between one point one and
23:30
one point six percent of GDP growth over ten years.
23:34
Not per year, total.
23:37
over ten years. McKinsey said three to four points per year.
23:41
Asa Moblu said one to one and a half points over a decade.
23:47
That's not a disagreement or a rounding error.
23:50
That's two different worlds. McKinsey's number is roughly twenty
23:54
to thirty times bigger than than Asimoblu,
23:59
measured over the same horizon.
24:01
One of those two estimates is going to look very,
24:04
very wrong.
24:05
10 years. And I'll give you one guess which one was produced by
24:09
the firm getting paid to help companies make the spending decision.
24:13
And who pays McKinsey? The companies making the AI decisions.
24:17
That's who. Average monthly enterprise AI spend hit almost $63,000
24:24
in 2024, projected to rise to over $85,000 in 2025,
24:29
a 36% jump, driven in part by
24:33
By consulting recommendations.
24:34
Mackenzie helps produce the number that justifies the spending.
24:37
And then gets paid to help you do the spending,
24:40
and nobody, nobody, has a contract clause that says if the four point four trillion
24:46
doesn't materialize, we give your fee back.
24:48
The accountability runs in one direction.
24:50
That direction is the invoice.
24:53
Look, nobody ever goes back and checks Mackenzie's four point four trillion dollar
24:56
number against reality. You make the projection,
24:59
you cash the check, and by the time the number's proven wrong,
25:02
the engagement is over and there's a new number for a new technology.
25:05
The thing that holds this whole machine together makes me angry in
25:09
a very specific way. So in March of this year,
25:11
Goldman found that seventy seventy give or take percent of S
25:16
P five hundred management teams talked about AI on their earnings calls.
25:20
Seventy percent.
25:21
That means practically everyone's talking about it,
25:24
but only 10% quantified its impact on any specific use case.
25:30
And only 1%, 1% quantified its impact on earnings.
25:35
1% of people showed their work,
25:38
right? Showed the math. 70% of people are talking about it.
25:41
One percent can actually point to a number.
25:43
And the census data. So fewer than than twenty percent of US establishments
25:48
are even using AI for any business function at all.
25:51
Fewer than one in five businesses is using it.
25:55
But a major component of the biggest companies are are on stage talking about
26:00
it like it's already rebuilt the world,
26:02
like the restructuring is done,
26:04
like the productivity is already in the bag.
26:06
And that's the machine, right?
26:09
That's that's the thing.
26:10
It runs on talk, it runs on the number nobody checks,
26:13
and the FOMO nobody admits, and the earnings call where you have to say
26:17
the word AI enough times to keep the stock up,
26:20
whether or not you can actually point to a single dollar it's earned.
26:23
six hundred and sixty seven billion dollars and one percent can prove
26:28
it earned anything. If the claims were true,
26:32
we'd have proof by now.
26:33
We don't. So let's talk about Clairna because this is where it starts getting to,
26:38
you know, like real people and real things.
26:42
Because the AI hype artists are showing Clarna to everybody because they're
26:49
holding it up like a trophy. So Clairna's a Swedish fintech company,
26:53
financial technologies, buy now,
26:55
pay later. And they deployed an open AI powered customer service assistant,
27:01
Chatchi.
27:02
And then they went out and told the world about it.
27:04
And the numbers were incredible.
27:06
The AI handled 2.3 million conversations in its first month.
27:11
Clarina said it did the work of 853 full time human agents.
27:17
That means 853 people were essentially replaced by a chatbot.
27:22
They said the resolution time dropped from 11 minutes to under 2.
27:28
They said they saved sixty million dollars of twenty twenty five,
27:32
and they posted a hundred and fifty two percent increase in revenue per employee.
27:37
That was the case study globally.
27:39
All of these AI hype artists, they pointed at Clarina and said,
27:42
Hey look, here it is. The proof you keep asking for.
27:45
Here's a real company, real money,
27:47
real people replaced, real savings.
27:50
Stop being a skeptic. And look,
27:52
honestly, that's a really cool story.
27:53
If you only hear the first half.
27:55
Because the second half, which came in late 2025,
27:58
Clarin's own CEO, Sebastian I don't know,
28:01
Simiatikowski, admitted the company had,
28:05
in his word, over pivoted on AI.
28:08
Quality problems, the service got worse,
28:10
customers were having bad experiences.
28:13
Tickets that should have been closed in two minutes were getting mishandled
28:16
in ninety seconds instead. Is it faster?
28:19
Sure. Is it correct? No, it's not.
28:22
And they had to bring the human agents back to Clara.
28:25
Bring them back, like 853 people.
28:28
The trophy case study for AI replacing humans became a case study
28:33
in bringing the humans back because the AI started to falter.
28:37
So, because I mean this is kind of like the meter study all over again,
28:42
just at a scale of a whole company.
28:44
So Clarina felt the productivity gain.
28:46
They announced it, they put it in a press release.
28:48
They believed it's hard enough to restructure their entire customer service
28:52
operation around it. And then when the actual quality got measured
28:57
by their own customers having actual bad experiences,
29:00
reality decided to step in. This gain that they felt just evaporated,
29:05
and the humans had to come back.
29:07
The gap between what they believed and what was true cost real people their real
29:13
jobs. Temporarily, maybe, but you tell me.
29:16
You tell the customer service worker who got laid off in the trophy announcement
29:19
and quietly rehired in the walk back.
29:22
You tell them that that gap was harmless.
29:24
You tell them the months of unemployment and uncertainty while a company figured
29:29
out the AI wasn't actually doing the job that they built the AI for.
29:34
Tell the you tell them that it was a ~ it's a minor inconvenience and a story with
29:39
a happy ending. Fifty five percent of companies that cut jobs in AI
29:44
In 2024 and in 2025, they now regret it.
29:47
That's from HR Dive actually, 55% regret.
29:51
More than half. They fired real people based on what felt like a game,
29:57
and then they wish they hadn't.
29:59
And the the regret doesn't rehire anybody.
30:01
It's just an admission. It's a Mia culpa in the aggregate.
30:05
But the feeling wasn't proof.
30:08
Now let me tell you about a case where nobody gets to be rehired,
30:11
where the stakes aren't a customer service ticket.
30:13
The Centers for Medical Medicare and Medicaid Services,
30:18
CMS, in January of this year launched a program called Wiser.
30:22
And I I have to tell you the full name because the full name is kind
30:26
of a confession. Wasteful and inappropriate service reduction.
30:31
Reduction. That's the goal. Right there in the name.
30:35
Not better medicine, not more accurate decisions,
30:38
reduction. The metric baked into the program's identity is denying more things,
30:44
not getting more things right.
30:46
Wiser uses AI to evaluate and effectively to automate the denial
30:51
of requests for medical care for people on Medicaid.
30:55
Poor people, sick people, people asking for care they and their doctors believe they
31:02
need.
31:02
And what should make your skin crawl is that the program was deployed with
31:07
no public transparency about how the AI made its decisions.
31:11
No published error rate, no independent audit before launch,
31:15
no clear, accessible path for a patient to appeal an algorithmic denial.
31:21
Nothing. A machine is deciding whether a poor sick person gets care,
31:26
and the public, the people who essentially pay for it.
31:30
Is not allowed to know how well the machine works.
31:33
We don't know how often it's wrong.
31:35
We don't know what it optimizes for.
31:37
We don't know whether it was validated against actual patient outcomes
31:42
or just against prior authorization approval rates.
31:45
We know the name of a program,
31:47
and we know it denies things, that's about it.
31:49
Did you see it? Do you see how the hype connects to this?
31:54
The whole reason.
31:55
A government agency can deploy a black box to deny medical care,
31:59
is that we've spent three years being told these systems work,
32:02
that they're smart, that they're better than the humans.
32:06
Nobody demanded the error rate because the culture had already assumed
32:10
the machine was competent. The hype did that.
32:13
The hype built the permission structure for a machine to deny your grandmother's
32:19
care without ever having to prove that it can tell a valid claim from an invalid.
32:24
And the hypers primed the audience.
32:25
And CMS walked through the door and the hypers opened it.
32:31
The Electronic Frontier Foundation,
32:33
the EFF. If you haven't heard of these guys,
32:35
you should really check them out.
32:36
They do some interesting work.
32:37
They filed a FOIA request, Freedom of Information Act lawsuit,
32:41
against CMS to try to get the records to find out how this thing actually decides.
32:47
They just wanted to see the algorithm,
32:48
they wanted to see the training data,
32:50
the validation results, the error rates,
32:52
the basic information you would want before trusting a system with that
32:56
big and that consequential of a decision.
32:59
And as of this month, July of twenty twenty six,
33:02
the lawsuit is still pending, still.
33:04
They're still fighting just to see the numbers and how it was trained
33:08
and whether there are even numbers at all.
33:10
And that's the through line made physical.
33:12
The one percent problem. Remember,
33:14
only one percent of companies can actually quantify AI's impact on earnings?
33:18
Well, CMS can't. Well, maybe they won't.
33:22
Quantify Weiser's impact on patients.
33:25
Same darkness. Same refusal to measure.
33:28
Except now the thing being measured isn't a stock price.
33:32
It's whether a human being gets the care that keeps them alive.
33:36
Remember the 16 developers from the cold open?
33:38
Experts, paid to notice, who couldn't feel that a tool was slowing them down.
33:44
The perception gap wasn't malice.
33:47
It was just human. People feel the tool working,
33:51
they feel the responses generating,
33:54
they feel productive. And the gap between the feeling and the measurement was 19%.
33:59
Now imagine.
34:00
That same perception gap, that same confident wrongness sitting inside
34:07
a government system that denies medical care to people who could
34:11
not otherwise afford medical care.
34:13
Nobody at CMS is going home feeling like they made wrong decisions.
34:17
The machine says no, it feels efficient,
34:20
it feels like fiscal responsibility,
34:22
which feels like progress. And there's no meter study for.
34:27
People wiser denies. There's no gold standard trial,
34:30
no, you know, $150 an hour to sit down and measure whether the machine is right.
34:35
There's just a denial and a sick person and a lawsuit still pending.
34:39
If the claims were true, we'd have proof by now.
34:42
They're denying care with a machine that they won't let anybody measure.
34:45
So let me pull this all the way back and I will tell you what I think
34:49
of this and what it actually is.
34:50
Because I actually don't think it's about AI.
34:53
AI is just the current, you know,
34:56
costume, it's the flavor of the week.
34:58
The pattern is the perception reality gap.
35:01
And it runs in the exact same direction every single time.
35:04
What people believe about the productivity gain is always,
35:07
always bigger than what you can measure.
35:11
And the industry, this is the the the
35:14
important part. The industry systematically avoids funding
35:18
the measurement that would close the gap.
35:21
Think about the the meter study again.
35:22
It's the first gold standard randomized controlled trial of AI
35:28
on real developer work. The first one.
35:30
And it came out in July of twenty twenty five.
35:32
That's why if you're following all the the you know foundational models,
35:36
it used models that are now deprecated and older.
35:40
But
35:41
ChatGPT launched in November of 2022.
35:44
Before that, we didn't have anything like this.
35:46
That's more than two and a half years of the biggest technology story on Earth.
35:52
Hundreds of billions of dollars.
35:55
Entire companies restructured before anyone ran
36:00
one rigorously controlled experiment on whether the flagship use case actually makes
36:05
people faster. Two and a half years.
36:08
And the gap wasn't filled by the companies building the tools.
36:11
GitHub's own study, the vendor study,
36:14
claimed a 55% productivity boost.
36:16
55%. Now that was based on self-reported feelings
36:21
at narrow synthetic tasks that looked nothing like real development
36:26
and production work. That number was cited everywhere,
36:30
like in earnings calls and board presentations and layoff announcements.
36:33
For years it was the number.
36:37
The vendor's feeling based number,
36:39
right? The one before the independent trial existed.
36:41
And when they finally ran the independent trial,
36:43
the answer was 19% slower. Why do you think it took so long to measure that?
36:48
Why do you think the measurement that existed was a vendor survey
36:53
and not a controlled trial? Because the measurement was the enemy in this scenario.
36:58
The gap is the product.
37:01
As long as everyone's operating on vibes,
37:03
the perception always favors the sale.
37:06
The moment you measure, the moment you run the controlled trial,
37:09
the moment you you know FOIA the algorithm,
37:11
that gap starts to close. And the closing always goes the same direction,
37:16
down, toward reality, toward smaller and slower and messier than promised.
37:23
Solov told us in 1987,
37:26
everywhere except the statistics.
37:29
And here's the human cost. And this is where it stops being
37:31
an interesting economic story and starts being an injustice,
37:35
which is what this show tries to point out for everyone.
37:39
The Harvard Business Review surveyed over a thousand global executives,
37:44
and the finding was devastating in how plainly they said it.
37:48
Companies are cutting jobs because of AI's anticipated future potential.
37:54
Not because AI has demonstrated it can do the displaced workers' job anticipated.
38:00
Future potential. These are not words that describe a thing that's been proven.
38:04
Those are words that describe a bet.
38:07
And the people losing their jobs aren't losing them because the machine
38:11
has proven it can do the work.
38:12
They're losing their jobs because a board decided the machine might do it someday.
38:17
And someday is cheap, and people are expensive.
38:21
Nearly a hundred thousand jobs since late twenty twenty three were blamed on AI.
38:27
fifty five thousand in twenty twenty five alone.
38:29
By May of this year, over eighty seven thousand layoffs blamed on AI just this year.
38:35
Tech sector unemployment climbed to five point eight percent.
38:39
That's the highest tech unemployment has been since the dot com bust.
38:44
The last time we did exactly this thing.
38:46
The last time the feeling outran the reality and regular people paid
38:50
the price while the infrastructure sellers cashed out.
38:53
An anthropic. Anthropic is an AI company doing the research.
38:58
They found that job finding rates for workers aged 22 to
39:03
25 entering AI exposed jobs fell
39:08
14% since ChatGPT launched.
39:11
Young people trying to start a career locked out of the first rung of the ladder.
39:17
Not because a machine is doing the job really well.
39:20
But because a hiring manager has a feeling that a machine might do
39:24
it really well eventually, the cost of the hype is landing on the on
39:29
the people who can least afford to absorb it.
39:32
Not the executives deciding to cut,
39:34
not the investors funding the infrastructure,
39:36
the twenty three year old who can't get their first interview.
39:40
That's the injustice.
39:41
The gains are theoretical, the layoffs they're real.
39:43
The $4.4 trillion is a McKinsey slide in their latest,
39:49
you know, keynote. The 87,000 people are human beings with rent
39:55
and student loans and families who were counting on a job because they were promised
40:00
that if they went into this field and got this degree and went into this debt,
40:04
that there would be something they're waiting for them.
40:07
We are transferring wealth and security away from workers toward NVIDIA
40:10
and hyperscalers and consultants,
40:12
and we're doing it on the strength of a of a vibe.
40:16
A vibe that sixteen experts couldn't even trust with their own work.
40:21
A feeling Clarina believed until their own customers proved it wrong.
40:25
A feeling CMS won't even let us look at while it denies people health care.
40:31
The perception reality gap isn't a quirk.
40:33
It's not an accident. It's a mechanism for moving money.
40:37
And money always moves in a very specific direction,
40:40
we've said that before, away from people who work.
40:42
Word people who own. And the people it moves money away from,
40:46
they don't get to file a lawsuit and wait.
40:48
They don't get a FOIA request,
40:50
they just lose the job. And then they watch the earnings call where the
40:53
CEO says the word AI 14 times and the stock goes up.
40:57
So here's what I want you to watch more closely.
41:01
Not the claims, the measurement.
41:03
Or, in this case, the absence of it.
41:05
Every time you hear a number about what AI is doing to productivity or jobs
41:09
or the economy, I want you to ask one question.
41:12
Who measured this? And can I see it now or am I going to see it later?
41:17
If the answer is a company survey or a self reported feeling
41:21
or a McKenzie projection about twenty forty,
41:24
that's not proof. That's marketing.
41:27
What I want you to carry out here is this too.
41:29
Seventy percent of the biggest companies in America talk about AI on earnings costs.
41:33
One percent can point to what it did to their earnings.
41:37
Seventy talking, one numbing. Hold those numbers next to each other for
41:42
the rest of this year, and you'll see the whole thing pretty clearly.
41:45
And then Jensen Wong, the man who's made more money off of this boom than anyone
41:50
alive. Even, yes, your beloved and beloved to be hated Elon Musk,
41:56
whose net income grew 20 times over selling picks and shovels.
42:01
He's the one telling you the gold isn't there.
42:04
He called the CEOs blaming AI for layoffs lazy.
42:08
He said it doesn't make sense that AI is replacing people already.
42:11
The biggest beneficiary of the hype is on record saying the hype's central claim
42:16
is false. And he said it while selling more shovels.
42:19
That's not a slip. That's just like the entire logic naked.
42:23
You can profit enormously from a revolution you don't even believe has come,
42:28
as long as everyone else is too afraid to miss out to stop and measure.
42:32
So two things.
42:33
radical over a drink things. One,
42:35
if you work somewhere that's making AI decisions,
42:38
ask the right questions for measurement.
42:41
Not like a vendor deck, the internal number.
42:43
What did we measure and how? And then watch what happens.
42:47
In a lot of places, a question is going to be the most disruptive thing that
42:52
you do because making the Emperor's tailor put something in writing,
42:56
the paper trail matters, you know,
42:58
later. And two, if you're not if you're hiring,
43:00
Or you know someone else who is,
43:02
go and hire the twenty two to twenty five year olds getting locked
43:05
out of this economy right now.
43:07
The ones down fourteen percent on job finding because a machine might someday maybe
43:12
be able to do their job adequately.
43:14
That's the most ~ underpriced talent in the market.
43:18
So everybody's too spooked by a feeling to grab it.
43:21
Be the person who measures instead of a person who feels.
43:25
If the claims were true, we'd have proof by now we don't.
43:28
So stop accepting that vibe and start asking for the number.
43:32
This has been the overlap. You can find us at FOF.foundation.
43:36
We are available on Apple Podcasts and Spotify,
43:40
and you know, wherever you like to get your podcasts.
43:43
Will, I hope you're enjoying yourself.
43:45
To the rest of you, go ask somebody to pull out a ruler and measure something.
43:49
Take care of one another. Bye.