Latest / AI Fire Daily / #611 Neil: AI 2027 Paper Got More Right Than Most People Expected

Transcript

#611 Neil: AI 2027 Paper Got More Right Than Most People Expected · AI Fire Daily

0:00
You know, back in 2025, reading the AI 2027 paper
0:03
was a truly wild experience. Oh, completely.
0:08
It felt totally disconnected from reality. Yeah,
0:10
it read exactly like a sci -fi movie script,
0:13
except this particular script was attached to
0:15
a dense Excel spreadsheet. Right. And I mean,
0:18
people were openly laughing at that timeline.
0:19
They really were. But today is Thursday, September
0:23
3, 2026. And nobody is laughing anymore. No,
0:27
not at all. The hard data is aggressively catching
0:29
up. Welcome to this deep dive. If you're listening
0:32
right now, you probably feel the ground shifting
0:34
in your own industry. It's happening everywhere.
0:37
So today we're taking a hard look at those famous
0:39
predictions. We are grading the AI 2027 paper
0:42
against hard data from 2026. Yeah, we need to
0:45
see what it actually got right. Exactly. What
0:47
is lagging behind? And most importantly, what
0:50
is the one massive assumption dictating our immediate
0:53
future? Well, we really need to set a baseline
0:55
first. Right. The paper didn't just predict smarter
0:58
chat bots. We already had those. It predicted
1:00
AI doing real, tangible economic work. Which
1:03
is a huge leap. It is. It also predicted AI speeding
1:07
up its own development. And all of this was supposed
1:09
to happen alongside massive unprecedented growth
1:13
in computing power. So it predicted a fundamental
1:16
shift from simple conversation to autonomous
1:19
action. Exactly. So before we judge the extreme
1:22
sci -fi scenarios, let's look at that baseline.
1:25
We need to examine the evolution from chatbot
1:27
to digital worker. The paper called this specific
1:30
phase stumbling agents. Stumbling agents. I like
1:34
that term. Yeah, it predicted agents would begin
1:36
handling real world tasks, things like ordering
1:39
food or, you know, managing complex spreadsheets.
1:42
But it explicitly expected them to be unreliable
1:45
for long drawn out projects. That feels incredibly
1:49
accurate to where we are sitting today. It's
1:50
remarkably spot on. Just look back at July 2025.
1:53
Right. The chat GPT agent update. Exactly. It
1:56
could suddenly browse websites. It filled out
2:00
digital forms, it edited spreadsheets, and ran
2:02
its own code. It was completing multi -step tasks
2:05
from start to finish and without human prompting.
2:08
Yeah, but the unreliability part was a massive
2:10
bottleneck. Oh, for sure. I remember testing
2:13
that. It would just break down halfway through.
2:15
Right. And mechanically, we need to understand
2:16
why they fail. That's where a research group
2:19
called METR comes in. METR, yeah. They run rigorous
2:23
evaluations on these frontier models. Precisely.
2:26
And their latest research focuses heavily on
2:28
something called time horizons. Let's define
2:31
that for the listener really quick. Time horizons
2:33
basically means measuring how long AI stays on
2:36
track without failing. That's a perfect definition.
2:39
So METR measures both 50 % and 80 % time horizons.
2:43
OK. And they found a massive gap in reliability.
2:47
Completing a long task occasionally is very different
2:50
from completing it reliably every single time.
2:52
It makes complete sense mechanically. If you
2:55
give an AI a 50 -step process, it has 50 chances
2:58
to hallucinate. Right. And if an agent fails
3:01
just one step in that chain, the whole task crumbles.
3:04
METR explicitly warns us about this dynamic,
3:07
don't they? They do. Just because an agent has
3:09
a long time horizon on one benchmark, don't assume
3:12
it handles every real -world task perfectly.
3:15
The specific context matters deeply. Let's unpack
3:17
this with a real -world scenario. Let's say you
3:20
ask an agent to plan a complex business trip.
3:23
Okay, a classic multi -step problem. Right. It
3:25
has to book flights, reserve a hotel, schedule
3:28
client dinners. It might nail the flights in
3:30
the hotel beautifully. But then it gets to the
3:33
restaurant reservations. It mysteries a calendar
3:36
invite. It books a dinner for the wrong time
3:39
zone. And suddenly, the entire itinerary is functionally
3:43
useless. That's exactly how cascading errors
3:46
work. The models lack persistent context. They
3:48
need that for complex, multi -day reasoning.
3:51
They just can't easily catch their own subtle
3:53
mistakes, not without human course correction.
3:55
Notely, yeah. It's like having an incredibly
3:57
eager, highly caffeinated intern. They can do
4:00
amazing things. They work incredibly fast. Yeah,
4:02
but you wouldn't hand them a messy week -long
4:04
project and expect perfection by morning. Exactly.
4:07
You have to constantly check their work. You
4:09
have to manage their output step by step. They
4:11
are undeniably useful, but they stumble repeatedly,
4:15
hence stumbling agents. They need structural
4:17
gargoyles and active supervision. So we have
4:20
utility, but are we just hitting a wall on true
4:23
reliability? For the immediate moment, yes. The
4:26
architecture still struggles to maintain deep
4:28
logical consistency over extended periods. Right.
4:31
They complete short tasks beautifully, but still
4:34
struggle with long -term focus. Exactly. But
4:37
this leads us to a fascinating divergence. OK.
4:40
Because if agents are still stumbling on messy,
4:44
general purpose projects, there is one highly
4:47
structured area where they are absolutely sprinting.
4:49
You're talking about writing software. Yes, writing
4:52
software. This is where the paper's predictions
4:55
are hitting our reality the hardest. The AI 2027
4:58
paper predicted coding systems would fundamentally
5:01
evolve. They'd stop being simple autocomplete
5:04
tools. They wouldn't just predict the next line
5:06
of text anymore. They would act as highly autonomous
5:09
agents. Wait, autonomous agents? Because in my
5:12
experience, they still need a human hand constantly
5:15
holding the steering wheel. Are they really making
5:17
major changes on their own? They are making massive
5:20
repository -wide changes now. They are saving
5:23
developers hours, sometimes entire days of manual
5:26
work. And we finally have the hard numbers to
5:29
back this up completely. I hear that claim a
5:31
lot. But I need to see the actual hard data.
5:34
What does adoption actually look like on the
5:36
ground today? We'll take Claude Code as our first
5:39
example. Anthropic recently reported their internal
5:41
metrics. Coding agent activity across GitHub
5:45
projects has more than doubled since late 2025.
5:48
That is a staggering adoption rate for a single
5:51
year. It gets even crazier. Developers are now
5:53
spending about 20 hours per week actively using
5:56
Cloud Code. That's fully half of a standard corporate
5:59
work week. They're essentially pair programming
6:01
with the AI for half their job. That's wild.
6:04
Then look at OpenAI's Codex ecosystem. By June
6:07
2026, they hit over five million weekly active
6:11
users. Five million developers actively integrating
6:14
this. Right. And then there's Cursor. It's a
6:16
wildly popular AI native code editor. Right now,
6:19
more than 70 % of Fortune 500 companies actively
6:23
use Cursin. 70%. Yeah, they use it to deploy
6:26
and manage coding agents across their massive
6:28
engineering teams. That proves it isn't just
6:31
a shiny Silicon Valley gimmick. Not at all. Fortune
6:33
500 companies do not adopt experimental technology
6:37
at that massive scale unless the return on investment
6:40
is immediate and measurable. The ROI is undeniably
6:44
there. Curtin agents have moved way past the
6:47
simple function stage. Developers are handing
6:49
off real structural parts of their daily jobs
6:52
to these systems. Why is coding working so well
6:57
when planning a business trip fails? It comes
6:59
down to closed feedback loops. Code compiles
7:02
or it crashes. The compiler gives the AI instant
7:06
objective feedback. Oh, I see. Yeah, if it makes
7:08
a mistake, the terminal throws an error. The
7:10
AI can read that error and fix itself. Ah, so
7:13
it doesn't have to guess if the restaurant reservation
7:15
was right. The compiler acts as a ruthless, instantaneous
7:19
supervisor. Exactly. The environment provides
7:21
its own rigorous guardrails, but it's still isn't
7:24
a completely frictionless process. No, it's not.
7:28
I have to admit something vulnerable here. I
7:30
still wrestle with prompt drift myself when trying
7:34
to get AI to handle my own technical workflows.
7:37
It is incredibly frustrating. We should probably
7:39
clarify that term for everyone listening. Sure.
7:41
Prompt drift is basically AI slowly forgetting
7:44
your original instructions over time. That's
7:47
a perfect, concise definition. It happens to
7:49
literally everyone. You ask an agent to build
7:53
a specific feature. Five prompts later, the AI
7:56
has rewritten half your core script into something
7:59
completely unrecognizable. Exactly. It completely
8:02
loses the structural plot. Yeah. So given that
8:05
drift, are developers just handing over the keys
8:07
completely at this point? Not entirely. They
8:09
are radically elevating their professional roles.
8:12
They spend their time reviewing high -level architecture,
8:15
while the AI meticulously grinds out the actual
8:18
syntax. Yeah. Developers aren't replaced. They're
8:19
just becoming managers of their own AI teams.
8:22
That is exactly how the industry is adapting.
8:23
adapting, and this structural shift exposes the
8:28
massive fragile assumption holding up the entire
8:31
2027 paper. You're talking about the self -improvement
8:34
feedback loop. Yes, the feedback loop. Think
8:38
about the logical progression here. If AI is
8:40
successfully writing complex code for Fortune
8:42
500 companies, it makes total sense that AI labs
8:45
are using it too. They are using AI to write
8:49
the code that builds the next generation of AI.
8:52
Let's explore the mechanics of that. How does
8:53
an AI... build an AI. It analyzes its own training
8:57
algorithms. It looks at how data is processed
9:00
through its neural network. OK. Then it suggests
9:03
and writes optimizations to make those specific
9:06
processes run faster and more efficiently. And
9:09
doing that at a frontier level requires a mind
9:12
-bending amount of physical hardware. It requires
9:14
an industrial scale we have never historically
9:16
seen. The 2027 paper predicted something highly
9:20
specific here. What was it? It expected AI assistance
9:23
to organically speed up algorithmic progress
9:25
by about 50 percent. So roughly a 1 .5x multiplier
9:29
on raw human engineering intellect. Basically.
9:32
Now proving that exact 50 percent number publicly
9:34
is quite difficult. Yeah. Labs keep their internal
9:37
metrics highly classified. Right. But the underlying
9:40
trend is absolutely undeniable. As of May 2026,
9:44
Claude wrote more than 80 percent of all the
9:46
code merged into Anthropic's own core code base.
9:49
80 % of the code building the AI was written
9:52
by the AI. Right. And Anthropic publicly stated
9:54
that Claude now matches or actively outperforms
9:57
skilled human engineers in some well -defined
10:00
research experiments. So the system is literally
10:02
bootstrapping its own future intelligence. It
10:04
is designing its own biological successor, in
10:07
a sense. And to support that massive computation,
10:10
the global infrastructure forecast is playing
10:12
out exactly as the paper predicted. The paper
10:15
projected $600 billion in AI data center spending
10:18
by 2026. It sounded absurd at the time, but look
10:22
at Alphabet alone today. They currently expect
10:24
their 2026 capital expenditures to land between
10:27
$195 billion and $205 billion. That is just one
10:32
single company. One company massively expanding
10:35
physical capacity for AI and cloud demand. Two
10:37
secs silence. Whoa. Imagine authorizing a $200
10:41
billion budget just to build enough data centers
10:43
for these models. It's genuinely hard to even
10:46
picture what $200 billion of physical server
10:49
racks looks like in reality. Think about the
10:52
physical mechanics of that spin. This isn't just
10:54
software living in the cloud. It's pouring millions
10:57
of tons of concrete. It's securing massive water
11:01
rights for cooling systems. It's physical reality
11:04
violently bending to support a digital mind.
11:07
And they are building these digital cities because
11:09
because they fundamentally believe the 50 % algorithmic
11:12
speedup is real. Even if the public can't see
11:14
all the internal efficiency metrics yet? Exactly.
11:17
Let me push back on that logic. If the 50 % speedup
11:20
isn't totally proven yet, isn't that infrastructure
11:23
spend a massive gamble? It is arguably the biggest
11:26
financial gamble in modern corporate history.
11:29
They are trusting the theoretical trajectory
11:31
of the self -improvement loop above all else.
11:34
They're spending billions today. betting entirely
11:37
on that future self -improvement loop. They absolutely
11:39
are. They're burning cash today to buy tomorrow's
11:42
intelligence. But that massive speculative gamble
11:45
brings us directly to the undeniable cracks in
11:48
the paper's predictions. So things are not arriving
11:51
perfectly on schedule. No, they're not. The future
11:53
is definitely arriving, but the timeline is surprisingly
11:56
bumpy. OK. The biggest missing piece in the ecosystem
11:59
right now is the R &D acceleration. You mean
12:02
open -ended scientific research. Exactly. As
12:05
we discussed, Coding is one thing. You have clear
12:07
right and wrong answers. A compiler gives you
12:09
a closed feedback loop. Right. But scientific
12:12
research is deeply different. Research requires
12:15
wandering in the dark. It requires forming hypotheses
12:18
about things that do not exist yet. Yes. There
12:21
was a highly revealing study published in July
12:24
2026. Researchers gave several frontier AI agents
12:29
up to six full days to work on open -ended AI
12:32
research problems. Six days of continuous compute
12:35
is an absolute lifetime for an AI agent. It really
12:38
is. It's an eternity of processing time. And
12:40
the results were surprisingly mixed. What happened?
12:43
Well, the agents successfully handled a massive
12:45
ton of the grunt engineering work. They flawlessly
12:48
set up testing environments. They ran basic repetitive
12:51
tests, but they completely fail to make major
12:55
progress on the core research questions. Why
12:57
did they fail mechanically? Because they lack
12:59
an intuitive model of the physical world. They
13:02
can synthesize existing data perfectly, but they
13:05
cannot easily make an intuitive, creative leap
13:08
into completely unmapped conceptual territory.
13:11
They can't invent something truly novel without
13:14
heavy human guidance. No. So we cannot give the
13:17
AI 2027 paper full credit here. We can't. The
13:22
models heavily assist with known experiments,
13:24
but they are not autonomously speeding up the
13:27
whole scientific discovery process yet. Not yet.
13:30
It's like having a race car that's incredibly
13:32
fast on the straightaways of writing syntax,
13:34
but it just can't navigate the complex hairpin
13:37
turns of open -ended scientific discovery. That
13:40
is a brilliant analogy. The engine redlines perfectly,
13:43
but it hits a concrete wall when the path isn't
13:45
strictly defined by existing data. And there
13:48
are other crucial areas where the paper missed
13:49
the mark, right? especially regarding how this
13:52
affects the listener's daily life. Yes, mostly
13:54
on the broader macroeconomic impact. The paper
13:57
made some very specific, highly aggressive financial
14:00
predictions. Like what? It expected the global
14:02
stock market to rise about 30 % in 2026 alone,
14:06
driven entirely by AI productivity gains. 30
14:09
% in a single year is a massive, historic bull
14:13
run. But as of September 2, 2026, the S &P 500
14:17
is only up about 12 % for the year. Which is
14:20
a perfectly good year, but definitely not a paradigm
14:23
shifting year. Exactly. 2026 obviously isn't
14:27
over yet, but this data shows the paper might
14:30
have been significantly too optimistic about
14:33
how fast the real world economy actually absorbs
14:36
these technological shocks. Why is the economy
14:38
so slow to absorb it? If the tools are this good,
14:41
shouldn't productivity be skyrocketing everywhere?
14:43
Because enterprise adoption is inherently sluggish.
14:46
If you are listening to this on your commute,
14:48
you know exactly how corporate software works.
14:50
Oh, yeah. You have intense legal reviews. You
14:52
have ancient legacy systems. A Fortune 500 company
14:56
cannot just instantly rip out its entire software
14:58
stack overnight. So are these lagging indicators
15:00
a sign the paper is fundamentally wrong or just
15:03
early? I strongly believe it is just early. The
15:06
massive structural changes are definitely happening,
15:10
just slightly slower than the most aggressive
15:12
mathematical models suggested. Right. The paper
15:14
is better at predicting the destination than
15:17
the exact arrival time. That's exactly the right
15:20
takeaway. And understanding that slight delay
15:23
brings us to the most important part of this
15:25
whole deep dive. The ultimate bottleneck. The
15:27
big assumption. The big assumption. Everything
15:29
we just spent time dissecting. The stumbling
15:31
agents, the coding acceleration, the massive
15:33
data centers. That's all just the required setup
15:36
phase. It's just laying the foundation. Right.
15:39
But the paper's most extreme, world -altering
15:42
predictions all firmly hinge on one specific
15:45
event. The takeoff. Yes, the takeoff. The self
15:48
-improving intelligence loop. The paper outlines
15:51
very specific demanding requirements for this
15:54
takeoff to actually happen. Where are they? First,
15:56
general AI research needs to become two to four
15:59
times faster across the board. Not just the 50
16:02
% we discussed earlier, up to 400 % faster. Second,
16:06
coding needs to become almost entirely automated,
16:10
minimal to zero human intervention required.
16:12
Wow. Third, they boldly predict the arrival of
16:16
something called Agent 3. Let's define that term
16:19
for clarity. Agent 3 is basically and AI outperforming
16:24
the best human software engineers alive. Precisely.
16:27
A truly superhuman coding entity. And what is
16:30
the fourth and final requirement? Autonomous
16:33
self -maintenance. Wait, self -maintenance? Do
16:35
you mean like physical error correction? Because
16:37
they obviously can't plug in their own server
16:39
racks. Right. Not physical maintenance. We are
16:41
talking about... digital resource management.
16:43
The agents need to act like highly autonomous
16:46
project managers. If they hit a computational
16:49
bottleneck, they need the agency to automatically
16:52
rent more cloud compute. They need to spin up
16:54
new digital copies of themselves, divide the
16:57
labor, and continue working without waiting for
16:59
a human to type approve. That sounds like a terrifyingly
17:02
fast way to accidentally burn through a corporate
17:04
credit card. Exactly, because the scale the paper
17:07
predicts is terrifyingly large. It describes
17:10
around two 200 ,000 independent copies of Agent
17:13
3 working in parallel. 200 ,000 superhuman coders
17:17
operating simultaneously. Working 24 hours a
17:19
day, never sleeping, never taking a break. Just
17:22
endlessly iterating and aggressively improving
17:25
the core AI architecture. That sheer volume is
17:28
genuinely hard to wrap your head around. But
17:30
mathematically speaking, are we seeing any early
17:33
signs of this kind of swarm intelligence today?
17:36
We actually are seeing the early prototypes.
17:39
Back in March 2026, a researcher named Marinka
17:42
Zitnik shared a truly mind -blowing experiment.
17:45
She had 15 separate AI agents work collaboratively
17:49
together on a single complex research problem.
17:51
Just 15 agents? Just 15. They worked continuously
17:54
for 48 straight hours. In that short time, they
17:58
collaborated, debated, and made over 574 distinct
18:01
edits to a shared research board. How did they
18:03
actually interact? Were they just blindly overriding
18:06
each other's work? No, they were actively coordinating.
18:09
They proposed ideas. They systematically verified
18:11
each other's logical steps. That's fascinating.
18:14
And around that exact same time, another platform
18:16
called Skypilot ran a totally different structural
18:20
test. They gave Claude code direct access to
18:23
16 powerful GPUs. What did Claude do with that
18:27
massive compute power? It autonomously submitted
18:30
about 910 distinct experiments in just eight
18:33
hours. Whoa! How does it physically do that so
18:36
fast? By parallelizing its thought process, humans
18:39
think sequentially. We try idea A. If it fails,
18:42
we try idea B. An AI can theoretically fork its
18:45
own context state. It spawns multiple instances
18:48
of itself. Instance one tests A. Instance two
18:50
tests B. They run completely concurrently. It
18:53
compresses months of sequential human trial and
18:56
error into hours. Exactly. Researchers carefully
18:59
estimated that SkyPilot's run was roughly nine
19:01
times faster than a standard sequential human
19:04
setup. Now, these examples are undeniably impressive.
19:08
But 15 agents is a very long way from 200 ,000
19:11
parallel agents. Oh, definitely. It is a fascinating
19:15
proof of concept, but it is certainly not a massive
19:18
production run. Right. We have not seen a proven,
19:23
sustained 2x to 4x speed up across an entire
19:27
Frontier AI lab yet. No, but the mechanics are
19:30
firmly in place. If 15 agents can pull off hundreds
19:34
of edits, what happens when a lab flips the switch
19:38
on 200 ,000 of them? The math gets overwhelmingly
19:41
exponential very quickly. That scale is the exact
19:45
moment the curve goes totally vertical. That
19:48
incredible scale is exactly what triggers the
19:50
massive intelligence takeoff they predicted.
19:52
It is the specific spark that ignites the whole
19:55
self -improving loop. We're going to take a quick
19:57
break, but when we come back, we'll wrap up what
19:59
all this means for you. Stay tuned. Insert sponsor
20:02
read here. And we are back. We've covered a massive
20:04
amount of conceptual ground today. We really
20:06
have, from those early stumbling conversational
20:09
agents to $200 billion physical data centers.
20:13
Let's synthesize this journey for everyone listening.
20:15
The main takeaway is that the core thesis of
20:17
the AI 2027 paper absolutely should not be ignored.
20:20
It successfully got the big structural directions
20:23
right. They clearly saw the massive trends before
20:26
they were obvious to the general public. The
20:28
setup says is undeniably here today. Autonomous
20:31
agents are out there doing real economic work
20:34
every single day. AI is actively writing the
20:38
foundational code for the next generation of
20:40
AI. And the global compute infrastructure race
20:43
is simply astronomical. But we firmly have to
20:47
stay grounded in the present data. The crucial
20:49
takeoff phase is still completely unproven. Right.
20:51
That fast, autonomous, self -improving loop driving
20:55
open -ended scientific R &D, it just hasn't clicked
20:58
into place yet. It is the ultimate missing piece
21:01
of the puzzle. The setup is literally sitting
21:04
right on our desks today, but the intelligence
21:06
explosion hasn't functionally happened. It's
21:09
a tense waiting game now, but all the necessary
21:11
pieces are actively on the board. Thank you for
21:14
taking this deep dive with us today. I want to
21:16
highly encourage you to look closely at your
21:17
own daily workflows this week. Yeah, try to spot
21:21
where this foundational setup is already happening
21:23
around you. Exactly. Where exactly are the stumbling
21:27
agents in your own professional life? They are
21:29
probably much closer than you think. They usually
21:31
are. I want to leave you with one final slightly
21:33
provocative thought to mull over. Let's hear
21:35
it. We spent a lot of time talking about how
21:38
AI is doing real work. We talked about it writing
21:40
complex code and autonomously managing spreadsheets.
21:44
But if the 2027 setup is already sitting right
21:47
on our desks, what happens to our very definition
21:49
of valuable work? The exact moment that self
21:52
-improving loop finally clicks into place. OTRO
21:55
music.