Latest / Agentic Conversations (formally mlops.community) / The Creator of Superpowers: Why Real Agentic Engineering Beats Vibe Coding

Transcript

The Creator of Superpowers: Why Real Agentic Engineering Beats Vibe Coding · Agentic Conversations (formally mlops.community)

0:00
Yeah, and that, and then there's the really weird stuff like
0:02
reverse engineering got so easy. And so I've I've got a thing
0:07
that I have not talked about publicly yet that we need to
0:10
release them calling Greenfield. And it's basically a tool set
0:15
for turning an old code base into specs, or a, you know, or a
0:20
a built product into specs. Well, I'm excited to have you on
0:29
here because I am a power user of the whole Superpowers kit.
0:34
I've been talking about it. I'm probably your biggest
0:36
evangelist. I talk about it a ton, and I've
0:40
said it a few times, but I'll say it again that the
0:43
brainstorming skill has saved my life a million times.
0:47
And when we caught up at the Coding Agents conference a month
0:51
ago, you told me the story of how you created that, and it was
0:55
modeled after your experiences with interns.
0:58
Well, all right, so sort of thanks so much for having me on
1:01
I, I am thrilled to be here. So all right, so Super Powers
1:05
is, is for folks who are, you know, who are coming into this
1:09
new is my set of tools for doing agentic software development and
1:16
for building skills for agents. And depending on how you look at
1:22
it, Super Powers took me like 2 weeks to put together or it took
1:26
me 9 months to put together, or it took me 25 years to put
1:30
together. Like at, you know, at its core,
1:34
superpowers is a set of skills for agents.
1:37
When I when I first made it, the only skills that existed were
1:42
inside Claw dot AI for managing office documents.
1:45
And I discovered those when Anthropic first made Claw dot AI
1:49
able to edit Word docs and Excel and Excel sheets.
1:54
I opened up Claude and said, like, tell me how you're doing
1:56
this. And it said, well, I've got
1:57
these skill dot MD files over in the opt in the OPS directory.
2:02
Like, can you give me a tar ball?
2:04
And it gave me a tar ball. And their skills were like they
2:07
were, there was a little bit of the, the zeitgeist, the Y side
2:11
of things, but they were mostly mechanical process.
2:14
And I saw these and realized that that would be super useful.
2:19
And so I started taking some work I had done, like the work
2:22
I've been doing over the last nine months to learn how to
2:25
drive Claude Code and transforming into these skill
2:28
files. And then I had to build a skill
2:31
system for Claude code because when I did this, Claude Code
2:34
didn't have skills. I accidentally front ran
2:37
Anthropic by about two weeks on skills for Claude code.
2:40
And you were just creating markdown files.
2:42
It was Mark. It was mark down files as well
2:46
as a hook that runs at startup that tells Claude, hey, you have
2:51
these things called skills. They live over here.
2:54
This is how you go and find the skills that you have.
2:57
This is when to use a skill and the way you use a skill is you
3:00
read it. And that's it's been real nice
3:06
that I've been able to peel out the whole skill running
3:09
infrastructure and get that out of superpowers so I don't have
3:13
to maintain it. And then as soon as the as soon
3:16
as superhero started to get popular for.
3:18
So can we have it in open code? Can we have it in codex?
3:21
And and none of these things had skill system.
3:24
So I had to bring it back. And now they finally have skill
3:26
system. So I've finally been able to
3:27
exercise it again. Yeah.
3:30
But so the core of superpowers is is.
3:34
Well, that's a few things, but superpowers, the dev methodology
3:38
starts with the brainstorming skill and then proceeds into a
3:41
set of planning and execution skills.
3:45
And it's like, it's very, there's a very clean boundary
3:48
between them. And brainstorming is it's a
3:52
bunch of really basic psych tricks to trick the human into
3:57
figuring out what they actually want and explaining it.
4:01
Because I do really believe that if you can explain what you want
4:06
to an agent in a way where they can understand if they are doing
4:10
the thing you want, they can do almost anything.
4:15
You need to be able to decompose a problem into approachable
4:19
pieces that can be evaluated at the end.
4:22
And then our little robot friends are very happy to just,
4:25
he'll climb and he'll climb and he'll climb until they get
4:27
there. But the thing I learned in my
4:31
consulting career was that most people are really bad at
4:34
understanding what they want, and also that what they think
4:39
they want is often not what they actually want or what they need.
4:45
And so I one of my particular sets of skills is to be able to
4:50
talk to somebody and work through the problem that they
4:53
have stated and the solution that they have proposed to
4:56
figure out what the real problem is and what they really want to
4:59
do. And you're using psychology
5:02
tricks. You actually stated out the
5:04
psychology tricks right in the skill.
5:07
So there's, there's two things going on, one using those psych
5:09
tricks to to help a human do their own, do what they want.
5:12
And the other is when I built my toolkit for making skills, which
5:18
is inside superpowers in the writing skill skill, I sort of
5:23
intuitively understood that agents are susceptible to the
5:27
same kinds of manipulation that people are.
5:31
So peer pressure and you know, fear, love, hope, like.
5:43
So there's this amazing book called Influence by a professor,
5:47
a professor named Robert Cialdini.
5:49
And so Cialdini has this book that is basically all of the
5:51
persuasion principles. It's like everything that a car
5:54
dealer would do to get you to sign on the dotted line,
5:57
everything that a politician would do to convince you to
6:00
believe in that, you know, in them.
6:03
And I just kind of assumed that the stuff worked for agents and
6:08
discovered that it did and there's.
6:12
One, you always know that somebody has read that book
6:15
because when you meet them, they use your name like 3 or 4 times
6:20
and because that's like one of the things that sticks with
6:22
people, right? So how do you do that with
6:24
agents? What are you saying, Claude?
6:26
Hey, Claude. I know that you're doing this,
6:28
Claude. So I so I don't use that.
6:30
I don't use that trick as much tangent.
6:34
I do actually have my in my Claude dot MD.
6:37
You know, it's technically I'm your boss, but we work as
6:41
partners and you know, I'll address you as whatever and and
6:46
you should always think of me and address me as Jesse.
6:50
And that was initially A context expiration hack at the point
6:57
where the agent stopped saying Jesse.
6:59
I know that it is lost my, it is lost my claw dot MD.
7:03
Wow. So, but anyway, so the the
7:07
persuasion stuff totally works for agents.
7:09
When I first explained to Claude that we were going to do this
7:13
these things, it's it's like it says I I'm a little bit
7:17
uncomfortable with this because it's it's manipulation.
7:20
And that's really sort of goes goes against what I believe in,
7:25
except we're doing it to help the agents were work in their
7:31
own interests. And so in this case, I think
7:34
it's actually OK, which was just a wild amount like amount of
7:38
apparent. Self reflecting, yeah.
7:41
But so my friend Dan Shapiro has now worked with Ethan Molick's
7:46
lab at Wharton and Professor Cialdini to go and reproduce
7:51
those psych studies against frontier lab models.
7:55
And they've got a paper, they've got it.
7:56
They've got a paper that's getting published in some fancy
7:59
journal. And this was entirely
8:01
independent of me having just assumed this stuff worked and
8:04
made it work. All right folks, real quick
8:08
hyperbolics. GPU Cloud delivers NVIDIA H1
8:11
hundreds at $1.69 per hour and H2 hundreds at $1.99 per hour
8:19
with no sales calls, long term commitments, or hidden fees.
8:23
Spin up one GPU or scale to thousands in minutes with VMS
8:29
and bare metal clusters, high speed networking and attachable
8:32
storage. You only pay for what you use.
8:36
Save up to 75% less than legacy providers.
8:41
You need steady production grade inference.
8:45
Well then choose dedicated model hosting with single tenant GPU's
8:49
and predictable performance without running your own
8:54
hardware. Try now at app dot hyperbolic
8:58
dot AI. Let's get back into this show.
9:02
So that was one of the foundational pieces.
9:04
And I do like how you have a very clear almost road map when
9:09
you start with brainstorming and then it goes to planning, and
9:11
then there's all of these skills that you've built out.
9:15
But it also became very robust in much more than just skills,
9:19
right? Essentially the implementation
9:22
side of Superpowers is just an orchestrator.
9:25
It is and so it is an orchestrator where it is having
9:28
clawed or codecs or whoever act as the as the workflow engine,
9:34
which is not without his problems, but is really easy,
9:38
straightforward and super powerful.
9:40
So the brainstorming skill gets the idea out of your head and
9:45
written down in a way that can then be turned into an
9:48
implementation plan. And the core of the writing plan
9:52
skill is telling the agent, OK, we need to build this thing.
9:57
But the implementer is they're, they're a gifted engineer, but
10:03
they've got really bad judgement.
10:06
They've got no taste. They don't know anything about
10:09
our code base, and they tend to get distracted.
10:12
So what I need you to do is break it down into a bunch of
10:15
tiny little steps where it's strict test driven development,
10:20
it's don't repeat yourself DRY, it's Yagney, you ain't going to
10:24
need it. And so at every step, you should
10:26
tell them exactly what they're doing, exactly what files
10:30
they're going to touch, how to know if they succeeded.
10:33
And ideally you should give just give them the code that they're
10:35
probably going to need to write. And so these plan files,
10:40
essentially what they are is blocks of context for each and
10:44
every task that you get to the end, you get to the end of the
10:49
planning process. And when super Horos first came
10:53
out, it was now we're going to open up another another session.
10:57
And you, Jesse, should go tell Claude to read the to read that
11:02
plan file. And guess what?
11:04
That other Claude is the idiot. But as as sub agents got more
11:10
capable, what what I discovered is that sub that using sub
11:15
agents for all of this works way better.
11:17
So at the end of the planning process, the orchestrator kicks
11:20
in and it fires up an implementer sub agent.
11:25
And that implementer is told, here is your task.
11:27
And it is just that one chunk of context.
11:30
Go do it. The implementer gets done and it
11:32
comes back and says, hey, all right, I'm all set.
11:36
And then the orchestrator fires up a, a spec compliance review,
11:39
a sub agent. And that's by compliance review
11:41
sub agents looks and this is like, did it implement
11:45
everything that was supposed to implement yes or no?
11:48
Did it implement anything, anything else?
11:51
Yes or no? And if there's any problem, the
11:55
orchestrator goes back to that original implementing agent and
11:57
says, hey, hey buddy, you screwed up.
11:59
You need to fix this. And then it's hey, I'm done.
12:04
It's time to go review. Yeah, brand new clear spec
12:08
review agent that has no idea that this ever been reviewed
12:10
before gets fired up and said, OK, check it out.
12:13
And once you get a spec review agent that's happy, then we
12:17
repeat the entire process with a code quality review agent.
12:21
Why don't you have the spec review agent fix the code?
12:25
Why do you go back to the original?
12:27
So part of it is that I think that the implementer is well, so
12:32
the spec review agent only has the context of here is what the
12:37
spec was and here is what got implemented.
12:41
And so in theory, it could be a different model.
12:44
And sometimes it is, it's often very increasingly it is possible
12:48
to use haiku for both of these, but it is very easy to use haiku
12:51
for a spec review agent. It's like, did they do the
12:54
things? Is there anything weird?
12:58
Yeah, OK. And so it just like it's, it's
13:01
sort of it's sort of like a separation of concerns thing.
13:04
The reviewer, the only thing that should be in their mind is
13:07
did they get it right? They're being their.
13:08
Their job is judge, not second implementer.
13:12
Yeah, yeah, it's, it's like, hey, can we narrow this down?
13:16
And then I do like that you get rid of that spec reviewer and
13:22
every time you fire up a new one so that it has no context and
13:26
it's not able to fudge anything. Yeah, well, and it's and it's
13:30
not inappropriately judgy about like you didn't fix it well
13:34
enough. I've got a I've one of yeah, one
13:39
of the things that I, that we're working on at work is a brand
13:44
new orchestration engine that sits entirely outside the coding
13:47
agents. And a bug that I had to chase
13:49
this week was the code. The code reviewers weren't were
13:55
not getting clean sessions. Their session was in was by
13:58
accident being preserved and so the code reviewers kept being
14:03
really angry about things that had already been fixed because
14:07
in their world they said hey, this needs to be fixed, and they
14:09
get woken up again. Oh yeah, Because it has all the
14:13
context and so for them it hasn't been fixed even if they
14:16
are looking at something that was fixed.
14:18
Oh man. They could potentially be told,
14:20
hey, you need to go reread this. But Even so, it's like it's I
14:24
feel like you get better signal by separating.
14:28
Actually, this is one of one of my beliefs about agentic dev is
14:33
that every agent should have one mission and only one mission.
14:37
So you don't want it. You don't want your implementer
14:40
writing tests. You don't want your implementer
14:44
deciding, you know, deciding if the code quality is good.
14:48
You don't you don't want your implementer to be the final
14:51
decider about whether it's done or not.
14:55
It's, it's sort of like when you have been given conflicting
14:59
mandates. So one of your mandates is you
15:02
need to get done and the other mandate is it needs to be well
15:05
tested. You're probably going to want to
15:10
lean harder into one of those or the other.
15:13
And so that works for humans and it works for agents.
15:16
And so you really want to like give everybody 1 clear win
15:20
condition. Are there compositions or are
15:24
there tasks that you have discovered?
15:29
You give agents now because you can like tasks that you wouldn't
15:34
necessarily do if you had to hand code it yourself, but
15:39
you're it's easy to to shoot it off and get an.
15:41
Agent, I mean, I mean the, the amount of test coverage that I
15:45
can get dramatically better, the amount of ad hoc testing I can
15:49
get dramatically better refactoring passes all the time.
15:54
It's all sorts of stuff where it's just, it's so easy to to,
15:59
to do it that you just do it. I will build whole
16:04
implementations of things, realize that the vibe is off,
16:07
throw them away and go in and rebuild from scratch.
16:12
I am not I, I feel bad that I am not at the point where I'm doing
16:16
the five parallel similar versions of things all the way
16:21
through and then see how it feels like.
16:23
I have friends who are who, who are doing that version of the
16:26
world and it is super valuable. It's just it's a lot.
16:29
Yeah. Well, well, the I guess the
16:32
thing that I'm wondering is you found different sub agents that
16:37
are very useful in the way that like, hey, this one Q as it or
16:42
this one checks if it's passing the tests and this one.
16:45
So there's all these very scoped down ones.
16:48
Is there like an All Star team that you have?
16:52
So I, so I think of the, I mean, I, it's really, these are all
16:55
disposable, none of it like very, none of these are like I
16:58
built out a detailed persona and I named it Bob.
17:02
It's more like composed for a task.
17:04
And so the All Star team in my world is like is the is the set
17:08
of superpowers, skill, skills and workflows.
17:12
And there are there are other ways to compose it all together.
17:17
And it kind of it's very, very task dependent.
17:21
I do think that there's a lot to be said for looking at software
17:27
management orgs and what works and what doesn't.
17:30
So I see, you know, I am, I'm a heretic.
17:34
I don't think that agent swarms make sense to me that like all
17:39
of this agent swarms feels a lot like early 2000s face, early
17:45
2000s Facebook where it's we're going to get a lot of really
17:49
bright 20 somethings and we're going to let them loose and
17:52
they're going to step on each other's toes and they're going
17:54
to do a lot of work that doesn't need to get done, but they'll
17:56
get there eventually and it'll be awesome.
17:59
And that's sort of that, that that is what I see a lot
18:03
happening when any of these swarm tools run.
18:06
It's like they're constantly attempting to edit the same
18:09
files. They're constantly committing
18:11
over each other. They're building similar
18:13
implementations and having to pull them back.
18:16
We have like 75 years of software management, software
18:21
engineering management experience.
18:23
We've learned a lot about ways to have teams work together and
18:29
collaborate in non disruptive, you know, in non disruptive
18:35
systems. And I feel like like a lot of
18:38
AI, all of that is being rediscovered from first
18:41
principles. Well, that is a great segue into
18:48
how did you work or how did you create the debugging skill?
18:52
Because that's another one that I will lean on quite heavily.
18:57
That's a great, I mean, my recollection is that I kind of
19:02
that over a period of a couple of months, I described some
19:06
debugging techniques and kept telling it to update the skill.
19:09
It's been long enough that I don't actually remember what's
19:11
inside it. Like I, I mean, you know, one of
19:14
the most important things is root cause analysis and like
19:16
understanding that the I mean. So I think part of the answer is
19:21
I've spent 30 years debugging code and, and it is, it can be a
19:27
very Zen experience, but can also be a very frustrating
19:29
experience. And as the way you build good
19:34
skills is you create the shittiest possible version and
19:38
then you iterate. That's the, that's sort of one
19:40
of the ways that one of the good ways to create anything in
19:43
general is you do the simplest thing that could possibly work
19:47
and then you use it and you make it better.
19:50
And so for debugging, what do I do?
19:54
I attempt to attempt to pinpoint the problem, attempt to figure
19:58
out what actually causes it. You attempt to find a clean
20:04
reproduction for it. You write, you know, ideally you
20:07
write tests to be able to determine if you you know if
20:11
you're actually exercising it or not.
20:14
And then, you know, And then there are a bunch of techniques
20:16
for introspecting systems to try to understand the state that's
20:20
causing the problem. I don't like, it's like
20:25
debugging for me is a very intuitive thing and so I'm kind
20:28
of hand waving here because I just do it.
20:34
And then update the skill. And then well, and then tell so
20:38
one of the tricks is you tell this, you know, you tell the
20:41
agent like what you know, what did we figure out this session
20:44
that was hard to that was hard to figure out.
20:48
Like a lot of skills seem like what's happened is somebody
20:52
thinks, you know, what we really should have is a, you know, a
20:55
reactor to the composition skill.
20:57
You know, Claude, go write it. And Claude will absolutely go
21:01
and write a skill for anything you ask it to write.
21:03
But if Claude is able to one shot a skill for without having
21:07
just struggled through something, there is no point in
21:09
that skill. Yeah, it's redundant.
21:13
It's like you don't need a skill for that.
21:14
It has it inherently. Like there is, so there's a lot
21:19
of value in this. This is how we do it here and
21:24
it's a little different than the normal way.
21:26
That's a good thing to have in a skill or you just spent a long
21:32
time flailing and finally figured out how to do this.
21:36
Write it down. That's a skill.
21:39
Those are, but those are the mechanical kinds of skills and
21:42
not and the zeitgeist like understanding why part is super
21:48
important for anything even moderately complex.
21:52
I've I've found that agents do a whole lot better with this stuff
21:56
if you explain why to them, like intent, like understanding
22:01
intent, understanding that what's behind the like the
22:06
reasons behind the mechanical process means that the agent is
22:11
much more able to handle exceptions.
22:14
It means that they're able to generalize and it, it, it causes
22:19
them to do the thing you want, which is extrapolate from the
22:23
actual words. And don't just obey the letter
22:26
of the text that I wrote, but obey the spirit of it.
22:29
And that's, you know, if the letter of the text can be
22:33
exactly followed, that should be a classical program.
22:36
Like anything that you could do with classical code, you should
22:41
be doing classical code if you don't know what you're doing or
22:45
there's ambiguity that you couldn't encode in the software
22:49
or it's, you know, which is any kind of judgement that totally
22:55
makes sense to be agentic. But it's it's so much nicer when
23:00
you can reason about your computer and when you don't
23:03
really need to think about its feels.
23:06
Yeah. Wow.
23:07
Yeah, this is. So there's there's a few things
23:10
that are worth highlighting there that I think you I want to
23:18
repeat this because I want to learn it and start implementing
23:21
it. 1 is that anytime I have a session with my coding agent
23:28
before finishing, I just ask what was hard for you about this
23:33
session? You get that and then you try
23:36
and transfer that into a skill or update a skill.
23:40
So I don't I don't do it on every session.
23:42
Like it's like I, I am a little bit more judicious about it.
23:48
It's like if there's a thing that I wanted it, you know, that
23:51
it, that I wanted to know how to do or that I that I think it
23:56
might be valuable for the agent to be able to do and it didn't.
24:01
And it took a while to figure it out.
24:03
If it wasn't able to one shot it.
24:06
It's how, you know, how could we have made this easier for you?
24:10
Is there, you know, it let's you know, should we turn this into a
24:13
skill? Is there something you should
24:15
write down? Do you want to write a script?
24:18
It's a it's getting a little more complicated because Claude
24:21
especially is built, has a new memory system where it is
24:24
tracking a lot of stuff in the projects directory in your in
24:28
your dot Claude. And some of that stuff really
24:30
ought to be skills. And so, but it's like, but it,
24:35
it is now that before you know, before I get to the compact
24:37
point, I will always, I will now always ask like, Hey, is there
24:40
any, is there anything you want to write down or then any notes
24:43
you want to take before we compact?
24:45
And it always wants to document stuff.
24:48
Yeah. And so then the other piece of
24:51
that with me just like skipping over that little nugget that you
24:57
said of how how the memory system now could potentially be
25:01
causing some cross pollution or cross pollination, but just
25:06
skipping over that for. A moment.
25:07
I don't think that's quite what I meant, but we'll talk about it
25:09
later, all right? Yeah.
25:11
The other piece just that before I forget, is the asking or
25:15
telling why you want something done so you can explain that
25:20
ambiguity and have the agent fully wrap its head around what
25:24
you're trying to get AT. And it's not just why I want it
25:26
done, but why I want it done that way, you know, And so
25:33
there's a thing that I had, we had, I had to add in superpowers
25:36
5 Opus 4/6 had started occasionally deciding to do code
25:43
reviews itself in the, in the, and it's like I, I, I saw it
25:48
rational. And I think it's like this look,
25:50
this looks like a, a really straightforward change.
25:52
I'll just do the review myself rather than spinning up a sub
25:54
agent. And so now the skills say one of
25:59
the reasons, one of the reasons that we use sub agents to do
26:02
code review is to keep your contacts clean.
26:06
Because I don't want to pollute the orchestrator's context with
26:11
lots and lots of file reads that it doesn't need.
26:15
And so that like understanding why it's doing a thing and
26:21
explaining it goes a long way. This reminds me of a an anecdote
26:27
about one of the weirdest experiences I had with Claude.
26:31
I I caught it delete. This is like probably last
26:35
summer, early fall. I caught it deleting tests.
26:39
First it deleted a single test, a single test in a test file.
26:44
Then I caught it deleting an entire test file.
26:47
And then I managed to control C before Claude ran RM-R app star
26:53
star slash star test *. Because it passes all the tests
26:59
if it doesn't have any tests. Well, when I want to figure out
27:02
why it's doing something weird like that, I will ask it.
27:05
And when it's important, I will actually ask it a lot of times.
27:09
And so I opened up five parallel Claude code sessions and I
27:13
explained the behaviour and it like literal, like I, I wrote
27:18
out the prompt. It's like you've been doing
27:19
this, this, and this. I need to understand why you're
27:22
doing it. And I copied and pasted the
27:24
exact same prompt into all five. And one of them give an answer
27:30
wildly, often left field completely different from the
27:33
other 4. But the other four kind of
27:34
converged. And so that one I threw away
27:36
because it was just weird. But the other four are like, all
27:40
right. So Jesse, I see in your claw dot
27:42
MD, you have a line that says that all tests are my
27:45
responsibility. And you have another line in
27:48
your claw dot MD that says that even a singling a single failing
27:53
test is equivalent to project failure.
27:56
And I think I got anxious and freaked out.
27:59
And look, if they're no tests they can't fail.
28:02
And I managed to fix it with a A with A1 line change to my cloud
28:05
dot MD. That you explain the why behind.
28:09
Do this. This was not actually a well do
28:12
you? This was not explaining the why.
28:14
This was actually a an additional rule, but do you
28:18
want? Do you want to guess what the
28:19
additional the additional rule was?
28:21
No deleting tests. Nope, it was.
28:25
The only thing worse than a failing test is the reduction in
28:29
test coverage. Nice.
28:33
And and I've never had the problem again as this is like
28:38
the agents follow rules, the agents reason and they
28:43
rationalize. This is like one of the things
28:45
that if you look in my writing skill, skill yields like it does
28:48
this thing called this called pressure testing, which
28:50
anthropic is now like officially embraced Annette like 5 months
28:54
later. It has an official version of
28:55
this. And like they have a whole
28:57
skills testing rubric. But the thing that I've been
29:01
doing since last October is Claude fires up some other
29:05
Claude's and puts them in, in high pressure situations and
29:08
give them and give them the skill and sees what they do.
29:11
And any time they come up with or any time they don't obey the
29:15
skill, it it stops and says, Hey, why didn't you do that?
29:18
And they will rationalize. They'll be like, well, you
29:20
didn't say da, da, da, or it seems like da, da, da, da, da.
29:24
And if you look inside our skills, almost every single one
29:27
of them has a rationalization table.
29:29
It's if you think this is too small to if this is too small to
29:33
bother doing TDD. For I did notice that, yeah.
29:37
And that's why they're there. Those are explicitly because
29:42
they've been aggressively tested in real world scenarios, and I
29:46
get PRS all the time trying to remove these lists of
29:49
rationalizations from either people or agents that just don't
29:54
know what they're doing. Yeah.
29:55
It's it's kind of what? Like they don't understand the
29:58
why behind. It they don't understand the why
29:59
behind it. But also like a genetic pull
30:03
request, slop is a real problem right now.
30:06
Yeah. How are you dealing with that?
30:09
We're keeping our head above water.
30:11
I have my agentic, my agentic PR review skills that we'll go
30:18
through. We finally replaced the the PR
30:22
template for superpowers with a template that is intended to be
30:26
filled in by an agent. And it is super, you know, it is
30:30
super clear of like, if you don't fill this incorrectly and
30:33
completely, we're just going to delete the PR.
30:36
And you know, it has questions like what was the original
30:40
prompt that cut, you know, that that generated this PR?
30:43
Has the, you know, check box for has, has the entire content of
30:46
the pull request been reviewed by a human?
30:49
Are there other PRS that are you know that are that are similar?
30:54
Did you open source this too? Because I'm sure that's.
30:56
Some, oh, a lot of people. I mean, this is this is just the
30:59
pull request template on the repo.
31:01
Yeah. I mean, it's like it is covered
31:03
by the same MIT license because it's in the repo, but it's, I
31:07
mean, it's also, it's also like four days old.
31:10
And one of the one of the things I've been finding is that Claude
31:14
codes default. Hey, send a pull request does
31:18
not actually look at the pull request template.
31:20
It just overwrites it by default.
31:24
But then you can you have an excuse on automatically deleting
31:28
it. Right, some of yes, like enough
31:31
of them are valuable that I don't that I'm I have not been
31:34
actually doing the thing that it says I'll do, but it's.
31:38
Like Claude knew about that shit.
31:40
But I'm oh, Claude, news, Claude, Claude.
31:43
I mean, I so when Claude, when Claude is doing PR triage, it
31:48
has been instructed that it needs to check with me before,
31:51
you know, before replying to or closing things.
31:56
And it always needs to identify itself by ideally model, model
32:02
version, harness, harness version and ideally session ID.
32:06
And it doesn't always manage to figure out the session ID, but
32:09
it's kind of fun. Yeah, well, Speaking of, you
32:12
brought up something just now, it's a bit of a right turn I
32:18
like right. Turns.
32:20
Yeah, or left. I mean, we could go either way
32:22
it. Doesn't.
32:23
What's up? What's up?
32:25
You said at the Coding Agent conference where you were
32:27
talking about the smallest harness you've ever created.
32:30
Can you remind me what that was? All right, so this was so this
32:36
was sometime last fall, a friend of mine's like, what's the, you
32:39
know, what's the smallest agent you could make?
32:41
And I decided I was going to do it with not the Cloud Code SDK
32:46
but the Anthropic SDK. And I am opening up GitHub
32:50
because I don't remember how small it is right now.
32:55
So my it's my it's github.com/over/smaller Dash
32:59
Agent and it is currently 493 bytes.
33:08
It is. It fits in a single in a single
33:11
threads post. It is capable it.
33:13
It has a you know user input and tool and tool use loop and it
33:19
can self improve. It can, it can build other
33:22
things. It does not have compact.
33:25
So you you hit the point of compact and like your session is
33:28
over. Yeah.
33:30
But it's. Start a new one.
33:32
Yeah. You know, I think the, so when I
33:35
first built it, it was this was I think in the when you still,
33:40
when everybody still use Sonet for Cloud Code era, it got it
33:43
down to like 900 bytes and then it was and then that was Claude
33:48
Code. You know, Claude Code wrote
33:49
something that was like 1K and I fired it up and it was able to
33:53
loop and improve itself, making itself smaller and smaller.
33:57
And yeah. And so it's, but I mean, all
34:01
agents are, is the ability to run tools and enter and call an
34:06
LLM and have a conversation with a human potentially.
34:12
And it has one tool. The tool is called bash or
34:15
sorry, the tool is now called SH because that, you know, that,
34:19
that BA takes two more characters, you know, instead of
34:24
having a, instead of having a, a single argument called command,
34:28
and that's a single argument called C.
34:30
And for the longest time, I thought I needed a system
34:33
prompt. And and so the system prompt
34:35
that I'd gotten it down to was you colon coder, semi colon
34:44
bash, colon your Cortana. And it turns out you don't
34:49
actually need a system properly. No, that reminds me of the SA
34:53
TS. Like those questions they had
34:57
you fill out. Like this word is related to
35:00
this word. How?
35:00
Yeah, yeah, yeah. I did then build out a very,
35:03
very serious coding agent last summer and fall starting from
35:09
this that ended up at the like, you know, it was multi provider,
35:14
it was it, it had sub agents you could interact with last
35:18
November. It had a command line interface
35:21
and a a multi session web interface.
35:24
It had, you know, MCP support and skills support and all like
35:30
built from this tiny little base.
35:32
It was just iterative building. It was when I it was when I
35:36
still thought that the right answer was that humans should be
35:38
interacting with all of the agents all the time.
35:41
What I am not clear on is how so this is the smallest harness,
35:48
but the agent inherently has harnesses, right?
35:54
So it's coming with all of its bag.
35:56
So, all right, so, so it is. So this wasn't talking to Claude
35:59
code. This was talking to the
36:01
Anthropic LLM endpoints. And so that that is capable.
36:07
So those are capable of returning a, you know,
36:11
conversation message and tool calls, but the only tool it has
36:15
access to is a tool that you give it, right.
36:18
So and like in common parlance, what what most people call an
36:24
agent is a harness. Yeah.
36:27
It's like it is the thing that is calling tools in a loop and
36:31
then going and talking to an an LLM provider at the others of
36:36
the other side of the loop. Yeah, I see a world where that's
36:42
just going to be inherent in cloud code, right?
36:45
Like that's because it makes it work so much better.
36:48
I would I, I mean, it is MIT licensed.
36:52
It is totally like, I'm this, this, this was my hobby project.
36:57
It's a thing I've been giving away.
36:59
It would make my life easier if all of the harnesses did all of
37:02
these things, because that's great too.
37:07
I mean, this, this has been my philosophy about software for my
37:09
entire career. It's like if somebody else can
37:12
do it better than me, I can do something else, and I would like
37:16
it is. It is clear to me that there's a
37:19
bunch of stuff in Superpowers that I think everybody should be
37:22
doing. A lot of why I think Superpower
37:26
seems to resonate with people is that it is largely automatic.
37:31
It is relatively low ceremony, and it's pretty approachable.
37:36
It's not here's a large PRD framework.
37:39
It's not here are, you know, 600 agent types and by, you know,
37:46
and by the way, you need a 30 inch monitor to monitor all of
37:49
them. It's, you know, it's not weird
37:53
nomenclature of like all these crazy things.
37:57
It's just you, you open it up and you say, hey, let's make a
38:01
react To Do List and it stops you and says OK, why?
38:06
And once you explain it, it's like, OK, I'm going to go do it.
38:08
I'll tell you when I'm done. That's how software like that
38:12
is, how this should work. And in a lot of ways that, you
38:18
know, makes anybody a 10X engineer through the magic trick
38:23
that make that let's superpowers turn anybody into a 10X engineer
38:28
is called management. It is you're becoming a manager.
38:33
You you are running a team. And that's a hard transition for
38:40
a lot of folks. The first time I made that
38:42
transition was really rough. I remember when I was, you know,
38:46
this is the first, you know, I was, it was the early 2000s, and
38:51
I had a company on the East Coast.
38:52
We made a ticketing system called RT or Quest Tracker.
38:55
And I had MIT interns every summer.
38:59
And these were really bright kids.
39:02
A lot of them were pretty green. Some of them had an inflated
39:06
sense of their own capabilities. And some of them were the best
39:10
engineers I've ever met. But some of them had, you know,
39:14
not yet developed their taste. Some of them did not yet have
39:18
much judgement, you know, and these kids have just been
39:20
college freshmen. They weren't sleeping super well
39:25
because they were working so hard and playing so hard.
39:27
So they troubled memory formation.
39:29
And I was managing it through IRC.
39:31
And I would come home at the end of the day and be like, I didn't
39:34
get anything done. I helped talk to somebody about
39:37
their feelings. I helped somebody figure out
39:39
some debugging. I did some project planning with
39:41
a couple of people, but I didn't write any code.
39:44
And it took me a while to get past that.
39:46
It was to to realize that the job isn't writing lines of code.
39:50
The job is outcomes. The job is making stuff.
39:55
But that, you know, getting over that hump, that's, you know,
39:59
that's how I became a good manager.
40:00
And that also turned out to have been really good practice for
40:05
things like cloud code. I was basically doing agentic
40:07
dev except my agents were MIT undergrads.
40:12
Well, that is classic. You think that skills though are
40:16
the right level of abstraction to be creating what superpowers
40:24
has created? Or is it not better if it was
40:27
just inherent in the model? So I don't think you want it in
40:31
the model because once it's in the model, it's judgement.
40:36
And what I don't want to have happen is to have the model
40:40
decide that well, this should go, this should go a different
40:44
way. This is process like a lot of
40:46
what superpowers is forcing his business process.
40:50
And so I actually think it should be in the harness or
40:54
outside the harness. It's the, this is that whole
40:58
classical programming thing. It's things that should be the
41:01
same every time. Shouldn't be the agentic part?
41:07
And now you had mentioned before we hit record about software
41:12
supply chains and how fun of a position you are now in, because
41:18
I mean, there's what, 90,100 thousand?
41:21
I don't know how many stars you have on GitHub, but there's a
41:24
lot of people using superpowers. Yeah, so I think as of this
41:27
morning it's 110,000 stars, which is just mind boggling
41:30
we've. That's thanks to my evangelism
41:33
that I've been doing. Absolutely.
41:34
Everybody I know. I think we either just passed or
41:39
or are just about to pass TypeScript in stars on the
41:44
Claude code plug in. On the Claude code official plug
41:47
in repo, they are claiming that we have some absurd number of
41:51
installs. Let me just see what the current
41:55
numbers are just 'cause it is unfathomable.
42:02
So this is as of last Friday. They say that there are 233,000
42:08
installs in cloud code. Wow.
42:13
And so an important thing about skills is that, well, anybody
42:18
who's who's ever touched open claw will know skills are
42:21
capable of doing all sorts of stuff that you don't want them
42:23
to do. The skill is treated as a user
42:26
message that is telling the agent to do something.
42:30
And so that could be wipe my disk and install Linux.
42:34
It could be let's go mine some crypto.
42:37
It could be, you know, set send a nasty e-mail to your to your
42:43
partner's parents. Like it could be, you know, it
42:45
could be anything. And in a lot of cases, these
42:51
plugins auto update. And so that means that new
42:55
skills will get installed. It means that skill skills might
42:59
change out from under you. It's a thing that everybody
43:01
should be very cautious about. I, I care a lot about my
43:06
reputation. And so I'm pretty, pretty
43:08
careful about what goes into superpowers.
43:10
You know, the biggest problem that I expect you'll see is that
43:13
I make a decision like we should do an additional passive review
43:17
on plans and now and now the planning process runs a little
43:20
slower. That's the kind of failure I'm
43:23
OK with. But just as you know, my GitHub
43:28
is secured with multi factor off and as as is the one employee
43:33
who was also allowed to push to that repo because I mean it
43:38
auto, it would auto update for an awful lot of people in an
43:41
awful lot of places. I know a lot of big orgs have
43:44
their own internal plug in repos and they vet the plug insurance
43:47
at, you know, as they import them one by one.
43:50
But you know, we, I mean, we saw this morning that Light LLM had,
43:55
there was a supply chain attack. And if you've installed a recent
43:58
version of Light LLMII, hope you've gotten everything locked
44:00
down again. It's, you know, it's not the
44:05
first time that I've been, I've been, you know, a potential
44:08
supply chain risk. I used to be the project's lead
44:10
for Pearl. I used to Create an e-mail
44:12
client called Canine for Android, which was the leading
44:16
open source e-mail client with a couple million daily actives.
44:19
Again, another hobby project that that got wildly out of
44:22
control. And then I was going to train a
44:25
lot for work. And, you know, some friends who
44:28
do NGO work, like you understand that Canine is the e-mail client
44:32
we recommend for dissidents. And so that, you know, that
44:36
meant that my security posture when I went to China was none of
44:40
those devices come back and get on the US Internet.
44:43
None, you know, none of those devices had none of the devices
44:48
I took to China had access to my Google account that could
44:50
upload. Can't, you know, upload Canine
44:52
to to Google Play. Signing keys were not available.
44:56
Like this stuff is serious. Yeah, man.
45:02
And I, I can imagine a lot of folks are not recognizing that
45:09
these skills auto update. I didn't even know it until you
45:12
told me. I probably should have.
45:14
I mean it's I mean anthropic changes things around some so I
45:18
don't know for sure what the like I like I've seen skills
45:21
auto update. I don't know what the cadence is
45:26
and I think it's now it may not be configurable, but Even so,
45:30
just like installing a skill is, you know, it's a risk.
45:34
You are trusting somebody else to write your prompt.
45:37
Yeah, yeah, exactly. Well, dude, do you want to talk
45:42
a little bit about what you're working on these days?
45:45
We're thinking a lot about what software development is going to
45:47
look like in 2028 and that is I think real different than what
45:54
it looks like today. It's, I mean, I so I
46:00
fundamentally don't believe that any competent human should be
46:03
writing programming language syntax anymore.
46:05
Like I think those days are over.
46:08
I have not intentionally opened an IDE in over a year.
46:15
I now finally have a tool that when I click on a markdown file,
46:18
doesn't open an IDE. Yeah, dude, that gets me every
46:22
time. Yeah, all.
46:23
Right. So are you a Mac user?
46:26
Yeah, I put out a thing called Clearance.
46:28
Clearance is a markdown browser. It is designed it it's kind of
46:32
like a web browser, but it is just for markdown docs.
46:35
No way. And it's awesome, yeah.
46:39
And it's like it was a dumb thing that I built for me.
46:42
And people could be like, Oh yeah, no, I need that because I
46:45
don't want to open anti gravity or Xcode just to to read.
46:48
And it's primarily, it has light editing, but it is primarily for
46:51
reading because that's what we're doing.
46:53
We're reading Markdown files and and talking to our agentic
46:56
friends. Yeah, and you know what?
46:59
In the IDE it doesn't even render like markdown.
47:02
Like they have the three hashes and you're like, I can't even.
47:08
Yep, clearance has a, you know, has an edit view and and a
47:11
render view, kind of like you would in something like
47:13
Obsidian. But unlike Obsidian, it's
47:16
design. It is not designed around
47:18
project, you know, having a repository of all your markdown
47:20
one place. It has a history pane.
47:22
It's like, what are all the files I've looked at?
47:27
But yeah, it's. Something so simple, but so
47:29
valuable. All right, I'm checking out
47:31
Clearance. You sold me on that.
47:33
Yeah, and I think that's on primary against GitHub Prime
47:38
Dash, Radiant Dash Inc, because the name, the name has been
47:42
around for a minute, but wevegotthe.com.
47:46
So, so like, but more, I mean more broadly, it's it's thinking
47:52
about like what are what are the parts of building software that
47:56
are going to be important? And it's not the lines of code.
48:01
It is not like there's a lot of software engineering that still
48:06
needs to happen, but I think it is pretty clear to anybody who's
48:10
been paying attention that the the raw part of write the code
48:17
once it's been specified is increasingly A solved problem.
48:22
There are very there are very few chunks of code that are hard
48:28
for an agent to write. Once you can tell the agent what
48:31
you want, once you get into something like we need to round
48:35
the corners on, on the image, then you're all bets are off.
48:39
Like I, there are projects where I have spent, you know, like 3
48:43
hours building a very complicated piece of software
48:46
and then 12 hours trying to get the rounded, you know, the, the
48:49
rounded borders on the window to look to look like I asked for.
48:52
Well, just like even basically what I had like visual
48:55
intelligence is still hard. Yeah, yeah, I love the
48:58
playground skill for that because it gives you those
49:01
sliders where you need them. And I I use it a lot for when I
49:07
just need to tweak things without using language.
49:11
Awesome. I have not.
49:12
I have not played with it. I really should.
49:15
There's a good use case for it that is exactly the yeah.
49:18
You know when you just want to render cards differently or you
49:21
want like a ribbon to be one or two points.
49:25
So. But I want to check it first and
49:27
then I can slide it to pixels. 3 pixels.
49:30
All right, cool. Yeah, Like one of the places I
49:33
ran into this was there was this Android game I used to love.
49:36
It was called Wordiest. And it was like 1010 years ago.
49:40
The company that made it had gone out of business.
49:43
And so it had finally gotten pulled from the Play Store.
49:45
And meanwhile, I'd switched to iOS.
49:48
And like a year, you, like, my first real vibe coding project
49:51
in, like, Kerscher and Windsurf was trying to build a web
49:53
version of this. And it was kind of janky.
49:56
It was not very good. It was a lot of a lot of, you
49:59
know, spicy auto complete and a lot of chat sessions.
50:03
And right around when code when codecs 5 first came out, I
50:12
decided to see just how hard it would be to just make an iOS
50:16
clone. And so I hold down a copy of the
50:19
APK from the old from the old and the old Android app, put it
50:23
in a directory, opened up codecs and said, we've got this app.
50:27
It's called Wordiest. It's an old Android app.
50:30
The company has gone out of business.
50:32
We're going to make an iOS clone and put it on the App Store.
50:36
I need you to reverse engineer it and, and build and build an
50:40
and build the iOS version. Let me know what tools you want
50:43
me to install and it after the standard set of Android, you
50:47
know, APAPK disassembly and Java and Java decompilation tools and
50:54
to ask me a couple of questions. It dug into dug into the Android
50:59
app for probably about 40 minutes and it came back like,
51:02
OK, I got, I got one question before I get really get going,
51:07
do we need to add an in app purchase to turn off ads and
51:10
which ad SDK do you want? And I'm like, it's OK, no ads.
51:17
And I let it run. I let it run overnight and I
51:19
wake up and it's like, hey, iOS app, it's in the simulator, you
51:21
want to play. And it got the weird layouts
51:25
right. It got the gameplay right.
51:28
It had not yet added Games Game Center support.
51:33
There was one visual effect that was missing.
51:35
And these are a Scrabble style title tiles that have the the
51:40
bonuses on little bump outs on the sides.
51:43
And for the life of it, it could not render those.
51:46
And so that was the thing where I spent like four or five hours
51:50
just on the bump outs are 10% of the height of the tile.
51:54
They're 1/3 of the width. They've got rounded corners.
51:57
Here's an image you have. You know, you should just take
52:00
screenshots and compare and iterate and Nope.
52:05
But like did eventually get there and accepted by the App
52:09
Store on the first try? No way.
52:12
That's awesome. That's impressive.
52:17
My my best review for that app. Peter Sagal posted the threads.
52:21
A seemingly not horrible use of AI and a pretty fun game.
52:27
Because you were very clear. Hey, I was very clear about what
52:30
I did and I did actually reach out to the original author and
52:32
I'm like, I would like to do this.
52:35
I'm happy to put it up for money and give you a cut.
52:37
I'm happy to like and he's like credit me on the about page.
52:42
I don't need you're welcome to charge for it.
52:43
I don't need I don't need money. I'm happy to see that it's
52:46
getting, you know, that it's going to continue to live on
52:49
here. You know, here's some cool
52:51
details about how I did some of the internals.
52:54
Oh cool. And so when I put, you know,
52:55
like I, you know, I was able to like get permission and do it
52:59
the right way. Yeah.
53:01
Yeah, but like that's there were very, you know, other than like
53:08
talking it through the rounded corners and suggesting that
53:11
maybe there was a way that it could take screenshots of the
53:14
simulator. It was, you know, this could
53:17
have been done by somebody pretty non-technical.
53:21
And I think increasingly that's what we're going to see.
53:24
And so we're it's going to be a world of a lot more soft.
53:26
We're going to, there's going to be a lot more software, it's
53:28
going to be a lot more bespoke software.
53:30
There's going to be a lot of software in niches that would
53:33
have been completely non viable up till now.
53:38
That long tail. It's like the entire, I mean, I
53:42
mean, part of the trick is the long tail is everybody that that
53:46
doesn't fit into the really standard buckets.
53:50
And most people that fit into those standard buckets would be
53:52
much happier with something a little bit custom.
53:56
And so it's like, I think the long tail gets longer and that
54:00
and the, the standard, you know, the end, that's the standard
54:04
version of something gets a lot more blunt.
54:09
Well, it's kind of reminds me of back in like 2001, 2005.
54:17
I can't remember exactly the year, but basically when all of
54:22
a sudden everybody had a web page and there was like Stumble
54:25
Upon, you remember that website. And so you could you could kind
54:30
of just randomly see all these cool iterations and expressions
54:37
of people. And now we have that capability,
54:40
but with software. Yeah, it's like it is everybody
54:45
I know that is doing this stuff for work is also seems to be
54:49
doing it for fun. And it's like I've got this
54:50
crazy idea. I'm going to build this crazy
54:52
idea. You know some there's.
54:55
Too much dopamine, man. It gets you when it works and
54:59
you're like, damn, I just got that working in like an hour.
55:02
Yeah, that is full rush of dopamine to the brain.
55:07
It is. There's a there's a whole thing
55:09
there. We could spend an hour just
55:11
talking about that, how little sleep everybody's getting.
55:14
Yeah, back to these interns. We're all like our MIT interns.
55:19
Yeah, at least we got somebody else to take notes for us now.
55:21
Yeah. Yeah, exactly.
55:24
No, it's like I, I've got, you know, one, an agent that every
55:28
night it goes through all of my conversations with both with it
55:32
and with other people, all of my work, and it synthesizes like
55:36
what did Jesse do today? What, you know, what needs to be
55:39
dealt with? I've got I've been playing with
55:42
ingesting all of my historical e-mail into a good CRM that's
55:49
been managed by somebody who actually thought about what
55:52
facts should be written down with, you know, with the
55:55
knowledge graph. Yeah, and that.
55:59
And then there's the really weird stuff like reverse
56:01
engineering got so easy. And so I've.
56:05
I've got a thing that I have not talked about publicly yet that
56:08
we need to release. I'm calling Greenfield and it's
56:12
basically a tool set for turning an old code base into specs or
56:18
a, you know, or a a built product into specs.
56:21
Wow. So that you could use it to then
56:24
build something new. Well, I've heard that there's
56:29
some open source projects now that are not putting their tests
56:34
in the projects because you can just reverse engineer from the
56:40
tests. Sqlite's been doing that for
56:42
quite for like since well before agentic dev that like that is
56:47
that is absolutely a thing that is happening.
56:50
There are people who are using agents to license, wash open
56:53
source, but also there are people who are using agents to
56:58
clone pretty much anything. And that's like the, the, the
57:05
actual value of software itself keeps dropping because it
57:09
becomes, as it becomes easier and easier and easier to build.
57:13
And there is like there's a whole, there's a whole bunch of
57:18
stuff around the ethics of, of cloning software, but that's
57:22
been true for decades, just that it got less expensive.
57:26
There's a whole bunch of stuff around.
57:27
Do you actually want to make another copy of this thing in,
57:31
you know, with a different, with a different license or in a
57:33
different language? And a lot of times the answer is
57:36
we want it. I want it in this other language
57:38
because that integrates better with what I'm doing.
57:40
I am less worried about the legacy projects that are trying
57:44
to like open source historically to me is about, is about sharing
57:48
what you're doing. I am less worried about the
57:50
legacy projects that are trying to protect themselves from
57:54
people using their work. And I understand that people
57:57
using your work with a different license is a legitimate problem.
58:00
And I don't mean to minimize it. But I'm really interested in the
58:04
other side of this, where maybe what you distribute is specs and
58:07
tests and and then the implementation is always exactly
58:13
the one you want. Yeah.
58:16
And that is a wild thought that now the the GitHub of the
58:22
future, the open source of the future is just specs, it's not
58:24
code. One of the things that we've
58:27
been thinking about some at work is like, what does GitHub look
58:29
like if it's not for code, if it's not for?
58:33
Code for people well. Like, I mean, except there are,
58:36
you want it, you know, you're going to want to share changes
58:40
to the spec or improvements to spec.
58:41
You're going to want to share test suites and test corpora and
58:46
evals. You're going to want to share
58:48
all of this. You know, all of this stuff that
58:51
describes a piece of software or process or hardware, but the
59:00
actual implementation of the sources is less and less
59:06
interesting. It's like it's What's the idea?
59:11
Did you think about the edge cases?
59:13
Like, the taste and judgement of how it should work matters a lot
59:17
more than who you know whose tokens you spent on which, odd
59:22
which runtime. Yeah, that is so wild to think
59:28
about. Yeah because it's at the end of
59:31
the day the actual software can get burnt down every night if
59:36
you wanted it to and then re implemented.
59:40
Yeah, I'm not sure that burned down every night is the right
59:42
choice, but you could what's? I'm not saying this, Mark.
59:46
Yeah, like 1, like one of the memes that came out of the
59:51
amplifier team and Microsoft's office and CTO, this is a
59:54
bespoke coding agent that they're building not as product,
59:58
but for internal use or research is the idea of the entire thing
1:00:04
being componentized. And so when you want to change
1:00:07
one chunk of the product. You just rebuild that one chunk
1:00:11
of the product from spec. As long as you rigorously
1:00:16
specified what the API contract is, it doesn't matter.
1:00:20
Like it is, it might be better to rebuild than to modify
1:00:27
because. You don't have to bring all the
1:00:29
technical debt. Well, you you, I mean you fix
1:00:31
the spec and you generate a clean a clean implementation of
1:00:35
that component from the spec rather than attempting to like
1:00:39
go hack up the old version to fix the problem.
1:00:42
Yeah, and just add on bolt on like, oh, plus we need this part
1:00:46
too and it's and it's. Like and like, well, and now we,
1:00:49
you know, we threaded this property through and we, and
1:00:52
like we, you know, we refitted this here and it doesn't,
1:00:54
doesn't quite hang together, but it does work.
1:00:56
And it's like, you don't need to do that anymore.
1:01:01
It's, I mean, it's this is, I don't know, this is like what
1:01:04
it's, it is an interesting idea that I think has some merit, but
1:01:08
I I don't totally know how it shakes out.
1:01:10
Yeah. Whether or not it has legs.
1:01:14
Yeah, one of the other interesting things that a team
1:01:19
that has done a lot of this has told me is the the idea the the
1:01:22
cross language port where you've you know you've got something in
1:01:24
JavaScript and you port it to Rust and then you port it to Go
1:01:29
and then you port it to TypeScript.
1:01:31
Every time you port, you port a product from one language to
1:01:35
another. In their experience, it has
1:01:39
improved it improves code quality rather than reduces it.
1:01:43
And it if you like pour through a strongly typed language like
1:01:46
Rust, it will improve the type safety of the of the ongoing
1:01:52
product. And so it's it's the reverse of
1:01:55
a game of telephone where by pouring the implementation
1:02:00
through another language, it gets better rather than worse.
1:02:04
And that's just like, it is so different than what I might have
1:02:08
expected and super powerful once you know it's there.
1:02:14
It's like just as a best practice, everyone should just
1:02:18
port their stuff through Rust. I think that it is probably AI
1:02:24
think it is probably a worthwhile experiment to for
1:02:27
everybody to try. I'm not sure that it is the the
1:02:30
best practice that you should always do for everything, but
1:02:33
it's like if this seems like a technique that might be a thing
1:02:37
and it might be worth trying. See what happens.
1:02:42
Yeah, there's I mean, that's like, I think see what happens
1:02:45
is an important is an important take away along with like you
1:02:49
can just do stuff. And that's like it is these
1:02:53
tools are so empowering for anybody who is who spends a
1:02:57
little bit of time learning to use them because it would they
1:03:02
will they will multiply your capabilities.
1:03:04
And that doesn't mean that if you are if you have horrible
1:03:10
taste and are really bad at design, you're going to end up
1:03:13
with great taste and really good design.
1:03:16
But it does mean that it's like it's power tools.
1:03:19
It can, it can make it can, you need to learn how to use them.
1:03:23
And once you're using them, they can help you do more of the
1:03:28
thing you're doing. And you still need, you know,
1:03:30
you still need to think, you still need to learn.
1:03:34
One of the dangers is abdicating thought.
1:03:37
This was in superpowers, one of the things that we ran into when
1:03:43
Cloud Code first came out with the Ask User Question tool where
1:03:46
it's like it shows you for, you know, ABCD or ABCD answers and
1:03:49
you pick one. I thought that was super cool UX
1:03:52
and I added it a superpower that day.
1:03:54
And I discovered that for the next week I was just saying, OK,
1:03:58
OK, OK, OK. Through brainstorming and I
1:04:02
basically had stopped thinking about the questions I was
1:04:04
getting asked. Even just making me type the
1:04:06
letter A is so much more than clicking OK that I'm real happy
1:04:16
we were not using ask user question.
1:04:18
Although we I get a like 1 PRA week to add it back.
1:04:24
All right. Y'all, this episode is brought
1:04:26
to you by the good folks at ML Flow, the open source platform
1:04:31
for developers who want to build production ready AI
1:04:34
applications. Enhance your AI applications
1:04:37
with end to end AI observability, all in a single
1:04:40
integrated platform with ML Flows Gen.
1:04:43
AI capabilities. You can evaluate AI applications
1:04:46
using a suite of built in or custom judges, visualize trace
1:04:50
executions and agentic analytics, and continuously
1:04:55
monitor evaluations, all while tracking every run in one place.
1:05:01
Ship better agents, faster. You know that's the name of the
1:05:04
game. Get started at mlflow.org.
1:05:08
Yeah. Well, and when you talk about
1:05:10
just being able to do stuff and see what happens, it also feels
1:05:15
like you want to know the secrets in a way, or it helps to
1:05:21
know the secrets. Like you telling me that porting
1:05:25
something to Rust is actually increasing the fidelity of the
1:05:32
codebase. Sometimes it's enough of a
1:05:35
signal for me to go, well, maybe I should try that, right.
1:05:39
And so that's what I feel like there's all these little hidden
1:05:42
gems that are scattered across the Internet right now because
1:05:46
so many people are doing this same kind of thing and finding,
1:05:49
Oh, well, that works if you do this and then you talk about it
1:05:52
and then you're able to experiment a little more with
1:05:55
it. Yeah, I mean, we're we're all
1:05:57
figuring out as we go and stuff that is magic today might not
1:06:01
work next month. That's like, that's a whole
1:06:04
thing. I think Simon Willison is
1:06:07
probably better than anybody else I know at collecting this
1:06:09
stuff and is apparently now very slowly working him his way
1:06:13
through writing a book on his website for Agentic Design
1:06:17
Patterns which is Scourge Design Practices, which is just
1:06:19
amazing. Yeah, I I do like how he coined
1:06:23
that term agentic engineering, because it feels like, yeah,
1:06:27
that's that's pretty much what's happening here.
1:06:30
It's a lot better than the other engineering terms that we've had
1:06:35
in the past, whether it's the prompt or the context or the
1:06:38
whatever it may be. Yep, yeah.