Latest / AI Fire Daily / #558 Neil: Coding Agent Skills That Make Senior Engineers Stronger

Transcript

#558 Neil: Coding Agent Skills That Make Senior Engineers Stronger · AI Fire Daily

0:00
Imagine a software engineer at Coinbase on a
0:04
Monday morning. They have this brilliant idea
0:06
for a new feature. Now two years ago that idea
0:09
set in a really sluggish pipeline. Oh, absolutely.
0:12
The endless Jared ticket updates. Exactly. It
0:15
required all these planning meetings. It took
0:17
eight full days just to become a poll request
0:20
today. That exact same process takes under 30
0:24
minutes. It is wild. It really is. It is staggering
0:28
to think about. But here is where it gets highly
0:30
interesting. The secret isn't just the sheer
0:33
power of the AI itself. Not at all. It actually
0:36
requires the exact legacy engineering skills
0:40
we thought were dead. Right, skills we assumed
0:42
AI was going to completely replace. Welcome to
0:45
the deep dive. Today, our mission is exploring
0:48
autonomous coding agents. We are looking at tools
0:51
like Cloud Code, Codex, and Cursor, and we will
0:54
see how they actually make traditional software
0:56
engineering skills more valuable than ever before.
0:58
The shift is truly incredible to witness, honestly.
1:01
We are rapidly moving away from humans just manually
1:04
typing out code. Right. Now, humans are building
1:06
these massive systems that manage the AI. Which
1:09
is a whole different ballgame. Exactly. Yeah.
1:12
We will look at real enterprise setups today
1:14
from OpenAI and Stripe. We will break down exactly
1:17
how to automate away recurring AI mistakes. Yeah.
1:21
Ultimately, we're going to completely redefine.
1:24
what a senior engineer actually does. Okay, let's
1:27
unpack this core paradox first. Yeah. Because
1:29
before these coding agents arrived, development
1:32
was, well, very isolated. It's isolated, yeah.
1:35
An environment supported one single human developer.
1:38
You wrote your code, you ran your own tests locally.
1:41
It was a very simple one -to -one relationship.
1:43
Yeah. Your local machine was set up perfectly
1:44
for your specific kind of idiosyncratic habits.
1:47
Right. But now... That exact same setup has to
1:51
guide many autonomous agents. And they are all
1:54
working simultaneously across an entire massive
1:57
code base. Which introduces absolute chaos if
1:59
you lack strict boundaries. Oh, total chaos.
2:01
Before agents, one bad commit may be ruined your
2:03
afternoon. Now, ten agents running parallel tasks
2:07
could theoretically corrupt an entire architecture
2:09
in seconds. Exactly. And this is why traditional
2:12
system rules are absolutely foundational now.
2:14
They are literally the only way to allow AI to
2:18
function safely at scale. This brings us to the
2:20
OpenAI codex case study. Right. They build a
2:23
complex product utilizing codex internally, but
2:27
they didn't just unleash the AI and hope for
2:29
the best. No, they created a tightly controlled
2:32
environment first. It was highly rigid. What
2:35
is fascinating here is how they enforced architectural
2:38
rules mechanically. They didn't just write a
2:42
polite prompt asking the AI to behave nicely.
2:45
No, polite prompts don't scale. They built custom
2:48
linters. And for those unfamiliar, linters are
2:51
tools that scan code to automatically block formatting
2:55
or logic errors. Yes. Think of them as automated
2:58
tripwires. They are hidden right there in the
2:59
deployment pipeline. So they just catch the bad
3:01
code immediately. Exactly. They instantly detect
3:04
and block it before it merges into the main branch.
3:06
They hard -coded their architectural boundaries
3:08
directly into the working environment. And they
3:10
paired these linters with incredibly robust CI
3:14
workflows. And CI workflows are automated systems
3:17
that test and build code before merging changes.
3:20
They checked deep dependency rules automatically
3:23
on every single agent commit. It was rigorous.
3:26
Wow. They checked for structured logging protocols.
3:29
They strictly enforced variable naming conventions.
3:32
They even had hard file size limits. So how does
3:35
this feedback loop actually work under the hood?
3:38
I'm trying to visualize the mechanical interaction
3:40
here. Sure. If the AI breaks a rule, how does
3:43
it know why it failed? Does a human engineer
3:45
have to step in and translate the error code?
3:48
That is the genius of their automated setup.
3:51
No human engineer ever steps in to explain the
3:54
failure. Really? Never. When codecs broke an
3:57
architectural rule, the linter didn't just flash
4:00
red and stop. Okay. The lint message actively
4:02
rejected the bad code, sure. But it also did
4:05
something else. It explained the exact structural
4:07
problem to the agent. Yes. It appended detailed
4:11
fixing instructions directly back into the agent's
4:13
context window. So the AI learned how to fix
4:15
it immediately from the system itself. I picture
4:18
this dynamic kind of like bowling with bumper
4:20
lanes. Oh, I like that. The AI is the bowling
4:23
ball rolling down the lane. It desperately wants
4:26
to knock down the pins and build the feature.
4:27
Right. It has a goal. But the LINT rules and
4:30
CI tests are the physical bumpers. They keep
4:33
the ball completely out of the gutter. We do
4:35
not have to watch every single roll manually.
4:37
Because the bumpers handle all the micromanagement
4:40
for us. Yes. And once those physical bumpers
4:43
are locked into the code base, they reliably
4:46
guide every future code change made by any agent.
4:49
So no engineer has to endlessly repeat instructions
4:53
in every single prompt. Exactly. It lets the
4:56
entire team move incredibly fast. Because the
4:59
underlying architecture stays perfectly consistent
5:01
despite the massive volume of generated code.
5:04
Yep. Every coding agent gets that instantaneous,
5:08
perfectly accurate feedback. You get exponentially
5:11
more code without checking every tiny logic change
5:14
manually. And the overall software quality doesn't
5:17
degrade over time. Not at all. I am struggling
5:19
with the implications of the feedback loop here,
5:21
though. Does this mean manual code review is
5:24
basically dead? That is a great question. It
5:26
definitely shifts our daily focus away from the
5:28
mundane stuff. We absolutely stop reviewing basic
5:32
syntax errors or missing brackets. Instead, we
5:35
review the underlying integrity of the system
5:37
rules themselves. We audit the bumpers. Got it.
5:40
We review the system's guardrails, not just the
5:44
raw code. Yes. And that is a massive conceptual
5:47
leap. You are managing the overarching system
5:50
that directly manages the AI. So we know these
5:54
system rules are entirely non -negotiable now.
5:56
They keep the autonomous agents from silently
5:58
destroying the code base. Right. But OpenAI's
6:01
approach works for a tightly controlled custom
6:03
build environment. What happens in the messy
6:06
reality of the trenches? Oh, it gets complicated.
6:09
Because I still wrestle with prompt drift myself.
6:11
We all do. It is incredibly frustrating. The
6:14
agent fixes a complex issue perfectly on Tuesday.
6:17
Yeah. But then on Thursday, it completely forgets
6:19
the broader context. It makes the exact same
6:22
architectural mistake all over again. You are
6:24
definitely not alone in facing that specific
6:27
friction. who created Claude Code, has a brilliant
6:32
insight on this exact problem. Oh, really? He
6:35
argues that manually fixing the same issue repeatedly
6:38
is a dangerous trap. Well, it wastes expensive
6:41
computing tokens for one thing. It does. Yeah.
6:44
But worse, it severely risks future architectural
6:47
misses. Right. Eventually the human reviewer
6:50
gets tired. You might miss that same hallucinated
6:53
import on the tenth time it happens. Humans get
6:55
fatigued. Exactly. So the overarching goal is
6:58
to turn that temporary prompt solution into a
7:00
permanent boundary. We need to systematically
7:02
automate the automator. Bingo. But how do we
7:05
build these rules? When the AI keeps repeating
7:09
very specific errors, let's break down the practical
7:12
workflow from our sources. OK. Imagine Claude
7:15
imports a module from the wrong architecture
7:17
layer. Let's say it grabs a database driver for
7:19
a front -end component. Which is a classic AI
7:22
hallucination. It just grabs whatever looks plausible
7:24
to save time. It happens constantly. Now, your
7:26
first instinct is usually just to type, hey,
7:29
fix this import path. You have to resist that
7:31
urge immediately. Do not just ask it to fix the
7:35
raw code. Because that doesn't solve the root
7:37
issue. Exactly. You have to force the agent to
7:40
understand the underlying pattern. You give it
7:42
a very specific, deliberate sequence of steps.
7:45
Okay. Step one, ask it to show hard evidence
7:49
of this repeating issue. So you make the agent
7:52
dig through the commit history and find the pattern
7:55
itself. Yes. Step two, ask it to explain the
7:58
fundamental root cause. Why is it consistently
8:01
misunderstanding this specific architectural
8:04
boundary? And step three is where the real magic
8:07
happens, I assume. You ask it to proactively
8:10
choose a long -term solution. Like writing a
8:13
new custom lint rule or generating a new automated
8:15
test. Yes. And finally, step four, you force
8:18
the agent to build the solution and confirm it
8:20
actually detects the problem. It has to mathematically
8:23
prove the new bumper works. Let's ground this
8:25
in a specific concrete example from the source
8:27
material to make it real. It really highlights
8:29
the power of this methodology, honestly. So two
8:31
front -end UI files completely bypass the backend
8:35
service layer. Yeah. They were querying the database
8:38
directly, and the code base's existing automated
8:41
tests completely missed this severe violation.
8:45
They missed it because traditional tests usually
8:47
only check the final user output. Right, they
8:50
just verify if the button turns blue and clicks.
8:53
Exactly. They rarely check the hidden architectural
8:56
path the data took to render that button. So
8:59
the engineering team used this exact prompt workflow
9:02
we just talked about. Claude Code analyzed the
9:05
critical gap in the testing suite. It realized
9:08
why the UI was able to fetch data directly without
9:10
triggering alarms. And then it generated a highly
9:13
reusable structural check. It added that new
9:16
check directly into the CI Workhole pipeline.
9:18
So the new test failed when the bad direct query
9:21
code was present. Yep. And it passed successfully
9:24
when the correct architecture was restored. It
9:27
removed the manual work we would normally repeat
9:29
through endless exhausting prompts. Every single
9:32
automation you build like this permanently hard
9:34
codes your team's specific standards. It's beautiful.
9:37
But here's where I get slightly skeptical about
9:39
this entire process. Wow. Isn't asking an AI
9:41
to write its own governing rules slightly risky?
9:44
Yeah. It feels a bit like building a flimsy house
9:47
of cards. out of AI -generated LINT rules. That
9:50
is a very varied operational fear, which is precisely
9:54
why this specific prompt workflow demands hard
9:58
historical evidence. Ah. The AI cannot just invent
10:01
a restrictive rule based on a momentary hallucination.
10:05
It must prove the repeating issue exists in the
10:08
actual code base history before proposing any
10:11
structural change. Makes sense. Require hard
10:13
historical proof before changing the actual system
10:16
rule. Exactly. You never just let it guess at
10:18
your architecture. You make it prove the fundamental
10:20
problem exists first. But catching errors like
10:23
that is still purely reactive, though. We are
10:26
building the bumpers after the ball already went
10:28
into the gutter. We need to shift our thinking
10:30
to the proactive side now. How do we give these
10:32
agents the hidden team knowledge up front? The
10:36
unspoken context. Yes, the stuff that usually
10:38
only lives inside a senior developer's head.
10:41
And that hidden context is absolutely everything.
10:44
I mean, AI doesn't automatically know why your
10:47
specific team chose a relational database over
10:50
a document store. Right. It doesn't inherently
10:53
know your bespoke historical naming conventions.
10:55
And when it lacks that crucial context, it just
10:58
guesses. It does. It hallucinates a convention
11:01
that looks plausible, but is fundamentally incompatible
11:05
with your stack. It is kind of like stacking
11:06
Lego blocks of data using the wrong size pieces.
11:09
That's a great way to put it. And this specific
11:12
problem scales exponentially as your team grows.
11:15
I can imagine. Stripe faced exactly this terrifying
11:19
chaos when they evolve their workflow. They hand
11:22
a cursor out to over 3 ,000 internal developers.
11:25
3 ,000 developers using AI agents simultaneously.
11:28
Yeah. Think about the sheer volume of code being
11:31
generated every hour. If every agent is blindly
11:34
guessing at the internal rules, it would be absolute
11:37
anarchy. The merge conflicts alone would crash
11:39
the entire system. Wow. So to proactively prevent
11:42
this, Stripe utilized something called cursor
11:45
rules. This provided deep bespoke code -based
11:49
context right from the very start. Let's unpack
11:51
the actual mechanics of that system. How do cursor
11:55
rules actually interface with the IDE's context
11:58
window? Good question. Because it doesn't just
12:01
force the AI to read the entire massive repository
12:04
every time you type. No, no. That would fry the
12:06
context window immediately. Right. Under the
12:08
hood, it injects only the rules relevant to the
12:11
specific domain the developer is currently touching.
12:14
So a front -end developer automatically sees
12:16
the front -end guardrails. Exactly. And a back
12:18
-end engineer automatically gets the database
12:20
rules. The AI already knows the... intricate
12:23
rules of the road before the developer even types
12:26
their very first prompt. They do not have to
12:28
explain team conventions ever again. You can
12:31
structure this exact same proactive knowledge
12:33
yourself too. If you're using Claude code, the
12:35
mechanism is a file simply named ClaudeE .md.
12:39
It sits right in the root directory of your project.
12:42
But the underlying strategy for writing this
12:43
file is crucial. It must be brutally ruthlessly
12:46
concise. Too sex silence. It acts as a lightweight
12:49
map pointing to deeper external documentation.
12:53
It is not a giant sprawling dump of every single
12:57
project rule. Okay, but say you have a massive
12:59
undocumented legacy project. A nightmare scenario,
13:02
sure. How do you actually build this map without
13:04
spending three agonizing weeks writing it manually?
13:07
There is a brilliant prompt strategy specifically
13:10
for this scenario. You leverage the agent to
13:14
explore and map its own environment. Oh, that's
13:16
smart. You ask the AI to thoroughly study the
13:20
existing code base. So you have it trace the
13:22
underlying architecture and the complex data
13:24
flow. Yeah. You have it document your repeatable
13:26
terminal commands and build workflows. Then you
13:29
ask it to draft short punchy instructions for
13:31
that clay .md file. It can even propose specialized,
13:35
deeper docs for highly complex functional areas.
13:38
But here's the critical catch in the workflow.
13:41
You mandate a hard pause for human approval.
13:43
This is non -negotiable. The AI is strictly forbidden
13:46
from modifying any actual files during this discovery
13:48
phase. You review the drafted map first. You
13:52
verify it accurately reflects the unspoken ghost
13:55
in the machine of your team's actual culture.
13:58
And only then do you let it commit the documentation.
14:02
Exactly. I have to challenge this slightly though.
14:04
Hit me. Why not just cram every single team rule
14:07
into one giant prompt file? Wouldn't giving the
14:11
AI more context always yield better, more accurate
14:14
results? It actually has the exact opposite effect,
14:17
surprisingly. Really? Yeah. A massive bloated
14:20
file completely overwhelms the agent's limited
14:23
context window. It heavily dilutes the critical
14:26
information. It becomes mathematically impossible
14:29
for the AI to figure out which rules actually
14:31
matter most for the immediate task at hand. Right.
14:34
Too much context just dilutes the most critical
14:37
rules. It is exactly like handing a new hire
14:39
a thousand page manual on their very first day.
14:42
They won't remember the truly important parts.
14:44
You need a quick scannable reference guide instead.
14:46
OK, we are going to take a quick break. But when
14:48
we come back, we'll look at what this all means
14:49
for the actual humans running the floor. Sounds
14:52
good, sponsor. All right, we are back. So, we
14:56
have encoded the hidden team knowledge cleanly.
14:59
We have. We have automated our architectural
15:02
guardrails with custom linters. The bumpers are
15:04
firmly up. The entire system is humming autonomously.
15:09
If you are managing an engineering team right
15:11
now, this is where your workflow fundamentally
15:14
changes. Radically changes. What does this actually
15:16
mean for the human engineers running the floor?
15:19
It demands a complete reevaluation of our daily
15:21
jobs. As the AI agents increasingly handle the
15:25
raw code implementation, the human role completely
15:28
changes. We elevate from typists to systems designers.
15:33
That's exactly it. Let's revisit that incredible
15:35
Coinbase example we mentioned at the start. They
15:37
didn't just casually slap AI onto their existing
15:40
agile process. No, they recognized the paradigm
15:43
shift. They rebuilt their entire operational
15:45
process from the ground up around Cursor. Teams
15:48
now write product requirements specifically formatted
15:50
so autonomous agents can easily read them. Right,
15:53
and they lean heavily into a feature called plan
15:55
mode first. Let's explain what that actually
15:58
does mechanically before any code is written.
16:01
Sure. Plan mode forces the AI to stop and think
16:04
systematically. Instead of immediately vomiting
16:06
raw code, the agent reads the requirements and
16:09
generates a comprehensive dependency graph. So
16:12
it outlines every single file it intends to modify.
16:15
It proposes the specific architectural path it
16:18
wants to take. And the human engineer carefully
16:21
reviews this logical plan. They don't just blindly
16:24
accept it. No, they tweet the logic. identify
16:27
edge cases, and only then approve the actual
16:29
code generation. And the resulting metrics from
16:32
this workflow are just mind -blowing. Let's deliver
16:34
those numbers again, because they really reframe
16:36
everything. The total time from a raw idea to
16:39
the first pull request dropped from 8 days to
16:42
under 30 minutes. Insane! And the time from an
16:45
idea all the way to a live production environment.
16:48
That plummeted from 20 full days down to 1 .8
16:51
days. Whoa! Imagine scaling to that level of
16:54
speed, taking a 20 -day product cycle down to
16:57
under two days. That fundamentally changes reality.
17:00
The compound interest on that velocity is almost
17:02
unfathomable. It really is. It shatters our traditional
17:05
understanding of software delivery. But we have
17:09
to emphasize it only works because of the senior
17:11
engineers continuously managing those automated
17:14
guardrails. Exactly. And this brings us to the
17:18
ultimate redefinition of seniority in this new
17:20
era. It is a massive structural paradigm shift.
17:24
Because being a senior developer is no longer
17:27
just about fixing a junior's architecture mistake
17:29
in a manual pull request. Right. It is about
17:32
creating a structural rule that stops every future
17:35
coding agent from ever making that mistake again.
17:38
You solve the deeply complex problem once, then
17:41
you systematically automate the prevention. Your
17:44
hard -earned experience literally becomes part
17:47
of the automated workflow. Yeah. You are coding
17:49
your own expertise directly into the environment.
17:51
You are building the automated factory itself.
17:54
You are no longer just hammering the individual
17:56
widgets on the assembly line. But if senior devs
17:58
are just building the factory guardrails now,
18:01
how do junior developers ever learn the ropes?
18:03
That's a huge concern for a lot of people. Don't
18:05
they need to struggle through writing that raw
18:07
code to truly understand the underlying systems?
18:11
Not necessarily. They actually learn by interacting
18:14
directly with the automated feedback loops. Oh,
18:16
interesting. When a junior developer or their
18:19
AI assistant makes a structural mistake, the
18:22
system instantly catches it. It doesn't wait
18:24
for human review. Right. It explains the failure
18:27
and provides the fix in real time. Fascinating.
18:30
The system itself becomes an interactive mentor
18:33
for the junior developers. Yes, they get instant,
18:36
perfectly accurate feedback on every single commit
18:38
they make. They do not have to wait three days
18:41
for a senior engineer to review their cloud.
18:43
It is actually a vastly superior, psychologically
18:46
safer learning environment. Okay, let's pull
18:48
all of these threads together for you. Let's
18:50
do it. The great paradox of the coding agent
18:52
era is truly fascinating. The more code the AI
18:55
writes autonomously, the more critical your human
18:58
design system rules become. The robust automated
19:01
tests, the intricate custom lint rules, the continuous
19:05
integration checks. Exactly. Clear, hyper concise
19:09
documentation is more vital than ever before.
19:11
It really is the bedrock. You solve a complex
19:14
architectural problem once, you build a system
19:16
that flawlessly remembers it forever. The old
19:21
legacy skills of system architecture and test
19:24
-driven development are actually the master keys
19:26
unlocking this new world. It has been amazing
19:29
exploring the mechanics of this with you. Thank
19:31
you for joining us on this deep dive. But before
19:34
we sign off, I want to leave you with a thought.
19:36
We are building these automated checks so that
19:38
AI can safely update the code base today. Right.
19:41
But what happens when the AI starts independently
19:44
identifying massive workflow gaps. Oh boy. What
19:48
happens when it starts writing its own governing
19:50
rules without asking for human permission first?
19:53
Will our code bases eventually become completely
19:55
self -governing entities? Something for you to
19:58
chew on until next time.