Latest / The Pragmatic Engineer / AI Skills with Matt Pocock

Transcript

AI Skills with Matt Pocock · The Pragmatic Engineer

• 0:00
  SPEAKER_01: I got the grilling of my life from building a pretty simple API endpoint using
• 0:03
  SPEAKER_01: the Grill Me skill.
• 0:04
  SPEAKER_01: It asked me 35 questions, I kid you not.
• 0:07
  SPEAKER_01: It was intense and annoying.
• 0:09
  SPEAKER_01: And it forced me to think more.
• 0:11
  SPEAKER_01: Today's guest is the creator of this popular skill, Matt Pocock.
• 0:14
  SPEAKER_01: Matt is a developer turned educator,
• 0:16
  SPEAKER_01: well known for his Total TypeScript series and now for his AI skills
• 0:20
  SPEAKER_01: and educational videos.
• 0:21
  SPEAKER_01: Today we cover Matt's unusual path into tech after years of being a voice coach
• 0:25
  SPEAKER_01: and building his own If you want to understand
• 0:28
  SPEAKER_01: which software engineering fundamental approach
• 0:41
  SPEAKER_01: And I wanted to take you back in time to remind you how we used to get work done.
• 0:44
  SPEAKER_01: Back when every lineup code was written by an engineer like you or me,
• 0:47
  SPEAKER_01: a tracker's job was to keep people in sync without slowing people down.
• 0:51
  SPEAKER_01: Linear was built to be fast and low friction and you could tell.
• 1:09
  SPEAKER_01: In last year's The Pragmatic Engineer survey,
• 1:11
  SPEAKER_01: Linear was the most loved tracker tool and Jira the most disliked one
• 1:14
  SPEAKER_01: for its sluggish performance.
• 1:16
  SPEAKER_01: And data coming from The Pragmatic Engineer audience showed how Linear started
• 1:19
  SPEAKER_01: to gain traction against existing tools,
• 1:21
  SPEAKER_01: especially as startups and mid-sized companies.
• 1:24
  SPEAKER_01: And since then, Linear grew up.
• 1:27
  SPEAKER_01: They added all the stuff that larger companies need to manage work.
• 1:29
  SPEAKER_01: Projects, initiatives, roadmaps, and customer requests.
• 1:32
  SPEAKER_01: And large companies started to switch.
• 1:34
  SPEAKER_01: For example, healthcare company Oscar helped move 600 engineers from Jira to Lanier.
• 1:39
  SPEAKER_01: OpenAI started with 100 seats and moved all 3000 staff without any mandate.
• 1:44
  SPEAKER_01: Coinbase, Cash App, Brexit, and Ramp are all on Lanier.
• 1:47
  SPEAKER_01: Many of them saw linear as a way to consolidate a single tool that brings planning
• 1:51
  SPEAKER_01: and building together.
• 1:52
  SPEAKER_01: So now, let's fast forward to today.
• 1:55
  SPEAKER_01: When you have AI agents inside a company, those agents need context to work well.
• 1:59
  SPEAKER_01: They need access to things like specs, customer requests, history.
• 2:03
  SPEAKER_01: Oh wait, these are all already in linear.
• 2:07
  SPEAKER_01: So when agents arrived, linear became the ideal context layer.
• 2:10
  SPEAKER_01: Today, 80% of enterprise workspaces in linear have adopted agents.
• 2:14
  SPEAKER_01: You can use agents like Codex, Cloud Code linear agent, or your own agent.
• 2:18
  SPEAKER_01: Coinbase and RAM both built their own internal agents and described linear as
• 2:22
  SPEAKER_01: a place that their agent goes and picks up the context before starting work.
• 2:26
  SPEAKER_01: See how it works at linear.app slash pragmatic.
• 2:30
  SPEAKER_00: Matt, it's great to have you on the podcast.
• 2:31
  SPEAKER_00: Great to finally be here.
• 2:33
  SPEAKER_00: I'm a huge fan.
• 2:34
  SPEAKER_00: I've watched so many of these.
• 2:35
  SPEAKER_00: I feel like this is like the tiny desk of being a software engineer.
• 2:38
  SPEAKER_00: You know what I mean?
• 2:39
  SPEAKER_00: This is big stuff, so I'm glad to be here.
• 2:41
  SPEAKER_01: And it's also great to reconnect because about a year ago, we had lunch.
• 2:45
  SPEAKER_01: Absolutely not.
• 3:03
  SPEAKER_00: For six years before I became a developer, I was a voice coach.
• 3:07
  SPEAKER_00: I was a singing teacher working in London and working in Exeter where I went to university.
• 3:12
  SPEAKER_00: I was teaching accents.
• 3:13
  SPEAKER_00: I was teaching singing.
• 3:15
  SPEAKER_00: I was teaching voice.
• 3:16
  SPEAKER_00: I did a master's in it.
• 3:17
  SPEAKER_00: I spent a lot of time thinking that was what my career was going to be.
• 3:21
  SPEAKER_00: You know, I didn't have any inkling of tech, didn't sort of think about it at all.
• 3:25
  SPEAKER_00: I sort of ran my own website and stuff.
• 3:27
  SPEAKER_00: But yeah, so I did that for a long time and it's been...
• 3:32
  SPEAKER_00: An extremely important influence on my life and I think my personality as well.
• 3:37
  SPEAKER_00: Can you get a bit deeper?
• 3:39
  SPEAKER_01: Where did this voice come from and what do you do as a voice coach?
• 3:43
  SPEAKER_00: Who are people who came to you for help and what kind of help?
• 3:47
  SPEAKER_00: So I started as a singing teacher.
• 3:49
  SPEAKER_00: I was in a band and stuff at university.
• 3:50
  SPEAKER_00: I sort of had a bit of experience doing singing.
• 3:53
  SPEAKER_00: And so I set up my own company kind of at university and doing that stuff.
• 3:58
  SPEAKER_00: And it was...
• 3:59
  SPEAKER_00: People who just wanted to sing better,
• 4:01
  SPEAKER_00: who wanted to use their voice for choirs, who wanted to just do it as a hobby.
• 4:05
  SPEAKER_00: It wasn't anything particularly professional.
• 4:07
  SPEAKER_00: And I went and did a master's in it and I started going to drama schools
• 4:11
  SPEAKER_00: to teach people Shakespeare and stuff and like getting people in who wanted
• 4:15
  SPEAKER_00: to do public speaking.
• 4:17
  SPEAKER_00: I did a couple of big gigs for consulting companies, you know, going in.
• 4:20
  SPEAKER_00: Teaching them how to deliver speeches and how to talk better.
• 4:24
  SPEAKER_00: It was wild,
• 4:24
  SPEAKER_00: you know,
• 4:25
  SPEAKER_00: and it was the reason I got out of it was because I realized in order to do it at
• 4:30
  SPEAKER_00: a decent level, you had to live in London.
• 4:32
  SPEAKER_00: I didn't want to live in London.
• 4:33
  SPEAKER_00: I tried it for like two years.
• 4:34
  SPEAKER_00: I just hated it.
• 4:35
  SPEAKER_00: I hated it.
• 4:36
  SPEAKER_00: I didn't grow up in London.
• 4:37
  SPEAKER_00: I wanted to get back to the countryside and where I was from.
• 4:40
  SPEAKER_00: And that's what I did.
• 4:41
  SPEAKER_00: And so I learned how to be a developer.
• 4:43
  SPEAKER_00: I was essentially self-taught in order to have something I could do remotely.
• 4:48
  SPEAKER_01: So basically you were looking at like professions that you could do from outside
• 4:52
  SPEAKER_01: of London that had a career or perspective or future.
• 4:56
  SPEAKER_00: Exactly.
• 4:57
  SPEAKER_00: And I was,
• 4:58
  SPEAKER_00: I sort of taught myself how to build stuff and just sort of build basic stuff
• 5:02
  SPEAKER_00: in JavaScript.
• 5:03
  SPEAKER_00: Because I was interested in making my lessons better for my students.
• 5:06
  SPEAKER_00: So I'd actually made sort of little flashcard apps.
• 5:09
  SPEAKER_00: I was working like the first app I ever built was the most ambitious thing I've ever attempted.
• 5:14
  SPEAKER_00: It was like a web audio analyzer.
• 5:17
  SPEAKER_00: So I could analyze the spectrogram of your voice to see
• 5:20
  SPEAKER_00: which resonant frequencies were happening,
• 5:22
  SPEAKER_00: whether your T1 and T2 were like properly balanced and things like that.
• 5:27
  SPEAKER_00: Extremely in-depth,
• 5:28
  SPEAKER_00: ran terribly, but actually, you know, made my lessons that little bit better.
• 5:32
  SPEAKER_00: And so I was doing pretty hardcore stuff terribly straight away.
• 5:35
  SPEAKER_00: And I realized, OK, I started looking at job postings.
• 5:39
  SPEAKER_00: I thought, well, I could do a bit of JavaScript.
• 5:40
  SPEAKER_00: I could do a bit of SAS.
• 5:41
  SPEAKER_00: I could do a bit of bits and bobs.
• 5:43
  SPEAKER_00: And I just jumped into it.
• 5:44
  SPEAKER_00: I quit my job, had a couple of months off and eventually got a job.
• 5:48
  SPEAKER_00: This was about 2017, where it was a little bit easier to get a job in the UK than it is now.
• 5:53
  SPEAKER_00: And I just went from there.
• 5:55
  SPEAKER_01: I guess in some ways you were also lucky because that was the peak.
• 5:58
  SPEAKER_01: That was a time where the demand was so high for engineers that people had
• 6:01
  SPEAKER_01: to boot camps with a few months of experience.
• 6:03
  SPEAKER_01: And I think people got chances from a lot of places who had the drive and
• 6:08
  SPEAKER_01: the motivation and the smarts, right?
• 6:10
  SPEAKER_01: Yeah.
• 6:11
  SPEAKER_00: And because I had this history of talking to people, that was an unbelievable advantage, right?
• 6:16
  SPEAKER_00: I could actually go into an interview and sound like a reasonable person instead
• 6:20
  SPEAKER_00: of someone who comes straight from a CS degree who maybe didn't have those skills.
• 6:23
  SPEAKER_00: So I had this bizarre ability of having zero technical knowledge or very little.
• 6:28
  SPEAKER_00: In the beginning.
• 6:29
  SPEAKER_00: But the ability to explain technical knowledge to people, right?
• 6:33
  SPEAKER_00: And so that basically all I needed to do was increase my technical knowledge
• 6:37
  SPEAKER_00: a little bit and I was very passionate about it and That increased quite quickly.
• 6:41
  SPEAKER_00: And then it was sort of seemed to be an unfair combination because I just rose
• 6:44
  SPEAKER_00: through the ranks very quickly at various different companies.
• 6:47
  SPEAKER_00: And I don't know,
• 6:48
  SPEAKER_00: it felt, I felt different from the other software developers I was working with.
• 6:52
  SPEAKER_01: And then how did you step up on the ladder?
• 6:56
  SPEAKER_01: So like you decided, I'm going to do this.
• 6:57
  SPEAKER_01: You taught yourself,
• 6:58
  SPEAKER_01: you went to some interviews,
• 6:59
  SPEAKER_01: you got to give, I'm assuming it must have been a small company, right?
• 7:03
  SPEAKER_00: Yeah, a tiny company with a couple of really inspiring software developers
• 7:08
  SPEAKER_00: who worked Basically a guy,
• 7:09
  SPEAKER_00: I won't say his name because he likes his anonymity,
• 7:11
  SPEAKER_00: but basically a guy who lived in sandals,
• 7:13
  SPEAKER_00: who lived in a canal boat for a long time, like a,
• 7:16
  SPEAKER_00: you know,
• 7:16
  SPEAKER_00: long hair,
• 7:17
  SPEAKER_00: proper hardcore, you know, it was around the time that Microsoft bought GitHub.
• 7:22
  SPEAKER_00: I remember him coming in almost in tears.
• 7:24
  SPEAKER_00: Yeah.
• 7:28
  SPEAKER_00: Yeah, absolutely.
• 7:31
  SPEAKER_00: I remember the first thing he got me to do was set up CentOS 6 on my Windows PC.
• 7:37
  SPEAKER_01: It's a pretty hardcore Linux distribution.
• 7:39
  SPEAKER_00: A really hardcore Linux distribution because that's what
• 7:42
  SPEAKER_00: So I was just bouncing around different agencies.
• 8:04
  SPEAKER_00: And then I was working in open source, which is kind of the next part of the story.
• 8:08
  SPEAKER_01: And with the agencies, what tech stack were you using at the time?
• 8:12
  SPEAKER_01: Yeah, it was TypeScript.
• 8:14
  SPEAKER_00: It was React.
• 8:14
  SPEAKER_00: Oh, it was TypeScript already back then?
• 8:16
  SPEAKER_00: Well, I was pretty hardcore on TypeScript.
• 8:19
  SPEAKER_00: Typescript already,
• 8:20
  SPEAKER_00: almost as in my second job,
• 8:22
  SPEAKER_00: I think I was doing, you know, presentations on how important Typescript was.
• 8:26
  SPEAKER_00: We were working for an automobile manufacturer building a learning management system, right?
• 8:31
  SPEAKER_00: You know, classic, boring agency stuff, right?
• 8:34
  SPEAKER_00: And the front-end team at that time was pretty small and we had a back-end team
• 8:38
  SPEAKER_00: in Portugal, right?
• 8:39
  SPEAKER_00: So classic front-end, back-end split.
• 8:41
  SPEAKER_00: The back-end team were racing ahead and At the time I joined,
• 8:44
  SPEAKER_00: the front end team was really slow.
• 8:46
  SPEAKER_00: We had a ton of bugs.
• 8:47
  SPEAKER_00: The back end team kept changing their contracts without telling us.
• 8:50
  SPEAKER_00: And we thought we need something to link us up a bit better.
• 8:53
  SPEAKER_00: TypeScript felt like the obvious thing.
• 8:54
  SPEAKER_00: And once we shipped it,
• 8:55
  SPEAKER_00: we like our velocity just went,
• 8:57
  SPEAKER_00: you know, we were faster than the back end team.
• 8:59
  SPEAKER_00: And eventually they took people off our team because we were so quick.
• 9:02
  SPEAKER_00: So, yeah, that was my history with TypeScript.
• 9:04
  SPEAKER_00: That's kind of my origin story with it.
• 9:06
  SPEAKER_00: How did you get into open source?
• 9:07
  SPEAKER_00: Was it at work?
• 9:08
  SPEAKER_00: Was it on the side?
• 9:09
  SPEAKER_00: It was.
• 9:10
  SPEAKER_00: I would been constantly playing around with open source on the side
• 9:14
  SPEAKER_00: and I was interested in different things.
• 9:15
  SPEAKER_00: By then I was into Twitter.
• 9:17
  SPEAKER_00: I was sort of looking at people online and thinking that's something,
• 9:20
  SPEAKER_00: someone I want to emulate, someone I want to look at.
• 9:23
  SPEAKER_00: And there was a guy who crossed my radar called David Korsheed,
• 9:26
  SPEAKER_00: who's the state machine and TypeScript guy on Twitter.
• 9:30
  SPEAKER_00: A lovely, lovely guy.
• 9:31
  SPEAKER_00: And I owe a lot of,
• 9:32
  SPEAKER_00: you know, my career to him, really.
• 9:35
  SPEAKER_00: And I was working on a project,
• 9:37
  SPEAKER_00: this is I think in my fourth job, where we needed a state machine.
• 9:41
  SPEAKER_00: It was a very complex application where you were on a video call with someone
• 9:45
  SPEAKER_00: and you could navigate around a house in real time together using some sort
• 9:49
  SPEAKER_00: of Matterport integration.
• 9:51
  SPEAKER_00: And there was a lot of linking up that needed to be doing across
• 9:54
  SPEAKER_00: the network boundary, a lot of complicated states.
• 9:57
  SPEAKER_00: And so I used a library called XState at the time, XState version 4, I think.
• 10:03
  SPEAKER_00: That was a resounding success.
• 10:05
  SPEAKER_00: And so I wondered.
• 10:06
  SPEAKER_00: Okay, how can I make this more type safe?
• 10:08
  SPEAKER_00: And so I started to sort of build some tooling around it,
• 10:11
  SPEAKER_00: have a fiddle, build a sort of CLI that constructed around it.
• 10:14
  SPEAKER_00: And that got me the attention of David and I became a member of the X-State core team.
• 10:19
  SPEAKER_00: So I started contributing issues, started...
• 10:22
  SPEAKER_00: Having discussions about the future of the library and it brought me into contact
• 10:26
  SPEAKER_00: with just a level of developer that I'd never seen before.
• 10:29
  SPEAKER_00: David and another guy called Mateusz Bozinski called Andrus Rake on Twitter.
• 10:34
  SPEAKER_00: These are the most talented developers I've ever seen.
• 10:36
  SPEAKER_00: Like this is another level.
• 10:39
  SPEAKER_00: Eventually, David wanted to form a company out of it.
• 10:42
  SPEAKER_00: He wanted to make a big bet on state charts and visual programming as the future of development.
• 10:48
  SPEAKER_00: He got some funding, and that was my first job where I was being paid American money, basically.
• 10:55
  SPEAKER_00: And it was a huge step up for me.
• 10:57
  SPEAKER_01: Yeah, which,
• 10:58
  SPEAKER_01: as we know,
• 10:59
  SPEAKER_01: it's quite different when a European or local,
• 11:01
  SPEAKER_01: even UK company are paying because, yeah,
• 11:05
  SPEAKER_01: I also covered some of it in the tri-model nature
• 11:08
  SPEAKER_01: of software engineering compensation where, yeah,
• 11:10
  SPEAKER_01: US companies,
• 11:11
  SPEAKER_01: especially in Europe and also in the US,
• 11:13
  SPEAKER_01: they think about compensation differently and value generated differently, right?
• 11:17
  SPEAKER_00: Totally.
• 11:18
  SPEAKER_00: It changed my life,
• 11:18
  SPEAKER_00: you know,
• 11:19
  SPEAKER_00: in terms of the way I was thinking about money and the way I was thinking
• 11:21
  SPEAKER_00: about flexibility.
• 11:23
  SPEAKER_00: And it meant I was working on something I was passionate about.
• 11:26
  SPEAKER_00: And I started while I was there doing a bit more advocacy for it because obviously
• 11:31
  SPEAKER_00: the company is very small.
• 11:32
  SPEAKER_00: I was doing a lot of development,
• 11:33
  SPEAKER_00: but also I wanted to be an advocate for it because I believed in it, you know.
• 11:37
  SPEAKER_00: And I still think state charts are incredible primitive for certain kinds of work.
• 11:41
  SPEAKER_00: I've sort of rode back a little bit on my belief of them, especially in the AI age.
• 11:47
  SPEAKER_00: I was doing a bit more of that.
• 11:48
  SPEAKER_00: And that got me the attention of a couple of guys at Vercel.
• 11:53
  SPEAKER_00: Because Vercel, at that time, Lee Robinson was the guy in charge of developer education there.
• 11:58
  SPEAKER_00: They had this incredible team,
• 11:59
  SPEAKER_00: Delbert de Oliveira, Lydia Halley, both of whom are now at Claw Code, Lee himself.
• 12:06
  SPEAKER_00: And I was working...
• 12:08
  SPEAKER_00: I got a job there under Jared Palmer as like the, yeah.
• 12:12
  SPEAKER_01: Wow, that Jared Palmer.
• 12:14
  SPEAKER_01: That Jared Palmer, yeah.
• 12:15
  SPEAKER_01: He's a really good mate, actually.
• 12:17
  SPEAKER_01: He's the one who later moved to GitHub.
• 12:21
  SPEAKER_01: He started or spearheaded it.
• 12:23
  SPEAKER_01: Stacked Diffs or Stacked PRs, and now he's at Cognition.
• 12:27
  SPEAKER_00: Yep.
• 12:27
  SPEAKER_00: He went into GitHub, shipped Stacked Diffs, left, refuses to elaborate, and is now at Cognition.
• 12:32
  SPEAKER_00: Exactly.
• 12:33
  SPEAKER_00: Yeah, but he's also an industry legend, yes.
• 12:35
  SPEAKER_00: Yes.
• 12:36
  SPEAKER_00: I mean, he's a great guy, and I worked on So I was only there about three months.
• 12:42
  SPEAKER_00: From there,
• 12:43
  SPEAKER_00: I got a funny contract at Vercel because I'd already been floating this idea of sort
• 12:50
  SPEAKER_00: of TypeScript and thinking about TypeScript and thinking
• 12:52
  SPEAKER_00: about maybe making educational material for TypeScript.
• 12:55
  SPEAKER_00: I had this urge while I was at Stately, the ex-state company, to teach stuff.
• 13:01
  SPEAKER_00: I have been teaching for six years before.
• 13:04
  SPEAKER_00: I've been not teaching for four or five years at that point, maybe six years.
• 13:07
  SPEAKER_00: And I thought, I need to get back to this.
• 13:09
  SPEAKER_00: I miss it.
• 13:10
  SPEAKER_00: And I love making stuff.
• 13:12
  SPEAKER_00: I love making content.
• 13:13
  SPEAKER_00: I love teaching people.
• 13:15
  SPEAKER_00: And so that's what I started doing.
• 13:16
  SPEAKER_00: And I started doing it for advanced types.
• 13:18
  SPEAKER_00: I'd got in contact with a lot of crazy typing tricks,
• 13:22
  SPEAKER_00: a lot of really advanced TypeScript stuff while I was trying to force X-State
• 13:27
  SPEAKER_00: to be type safe.
• 13:28
  SPEAKER_00: Very, very hard job.
• 13:30
  SPEAKER_00: I think a mostly impossible job.
• 13:33
  SPEAKER_00: So I made a couple of tips.
• 13:35
  SPEAKER_00: I made these two-minute tips,
• 13:36
  SPEAKER_00: posted them on Twitter, and they just went, you know, in a way I'd not felt before.
• 13:40
  SPEAKER_00: And I realized, okay, there's a market here.
• 13:43
  SPEAKER_00: And so one Sunday, I just made like 13, 15 of these two-minute tips.
• 13:48
  SPEAKER_00: I just queued them up over the next few weeks,
• 13:50
  SPEAKER_00: and my follower count went from, you know, 4,000 to 10,000 or something, you know.
• 13:54
  SPEAKER_01: You suddenly felt that there was huge interest in this, right?
• 13:57
  SPEAKER_01: Yeah.
• 13:58
  SPEAKER_00: Exactly.
• 13:58
  SPEAKER_00: A massive wave of something was,
• 14:00
  SPEAKER_00: you know,
• 14:01
  SPEAKER_00: some combination of the way I was speaking,
• 14:03
  SPEAKER_00: the material I was delivering that was clicking in a way that I'd not felt before.
• 14:07
  SPEAKER_00: And that's only really happened twice in my career.
• 14:10
  SPEAKER_00: So I was already floating the idea of a course and I knew I could do it well.
• 14:15
  SPEAKER_00: I knew I could do a really great course if I just had the right audience and if it clicked.
• 14:20
  SPEAKER_00: And so I went into Vercel,
• 14:22
  SPEAKER_00: I got a contract there for only three days a week for three months initially,
• 14:26
  SPEAKER_00: which is
• 14:27
  SPEAKER_01: very unusual.
• 14:28
  SPEAKER_01: Is that what you wanted or this is like how,
• 14:30
  SPEAKER_01: you know, Vercel was probably testing the water, see how it goes?
• 14:34
  SPEAKER_01: Vercel wanted to be full-time straight away.
• 14:36
  SPEAKER_01: You knew that there's this other thing,
• 14:38
  SPEAKER_01: so let me kind of hedge my bets if I'm able to do, right?
• 14:42
  SPEAKER_01: Vercel was this weird backup to what I...
• 14:47
  SPEAKER_00: Which is wild!
• 14:49
  SPEAKER_00: Which is wild to me.
• 14:50
  SPEAKER_01: For most people, this would be the dream job, right?
• 14:53
  SPEAKER_00: I know.
• 14:54
  SPEAKER_00: So it's a little embarrassing to say because obviously it's
• 14:57
  SPEAKER_00: so many people's dream job,
• 14:58
  SPEAKER_00: but I went into it going, okay,
• 15:00
  SPEAKER_00: I need a stable nine to five for three days a week
• 15:04
  SPEAKER_00: while I test this other thing out.
• 15:06
  SPEAKER_01: But I mean, just to be fair...
• 15:10
  SPEAKER_01: I think this is sensible, right?
• 15:11
  SPEAKER_01: Like at this point,
• 15:13
  SPEAKER_01: if we just go back to where you are,
• 15:14
  SPEAKER_01: like you've been a voice coach for a good part of your career, let's say six years.
• 15:21
  SPEAKER_01: And let's say now for five years,
• 15:22
  SPEAKER_01: you've been building software,
• 15:24
  SPEAKER_01: you love doing it, you think You think you're good at it.
• 15:26
  SPEAKER_01: You think you might be able to teach, but who knows, right?
• 15:30
  SPEAKER_01: And at that point saying,
• 15:32
  SPEAKER_01: all right,
• 15:32
  SPEAKER_01: let me take a gamble and like do this thing that might or might not work out.
• 15:37
  SPEAKER_01: Whereas if you can pull it off,
• 15:40
  SPEAKER_01: When you have something stable and it gets traction, now that's different, right?
• 15:44
  SPEAKER_01: You know,
• 15:44
  SPEAKER_01: a lot of engineers have aspirations, ideas,
• 15:46
  SPEAKER_01: especially because with software engineers,
• 15:47
  SPEAKER_01: you can work remotely,
• 15:48
  SPEAKER_01: you can take your idea,
• 15:50
  SPEAKER_01: build a company, and they're thinking, all right, should I just plunge?
• 15:53
  SPEAKER_01: Should I quit my job?
• 15:54
  SPEAKER_01: Should I not quit my job?
• 15:55
  SPEAKER_01: So in some ways, I guess this is one model that is kind of unique.
• 16:01
  SPEAKER_01: And if you're able to pull
• 16:02
  SPEAKER_00: it off, I mean...
• 16:03
  SPEAKER_00: It was the most bizarre thing because it became very clear very quickly
• 16:07
  SPEAKER_00: that I couldn't stay at Vercel, basically.
• 16:10
  SPEAKER_00: So we had about two months into my work at Vercel.
• 16:13
  SPEAKER_00: I was there actually over a very tumultuous time because I was there
• 16:17
  SPEAKER_00: when they released TurboPack.
• 16:19
  SPEAKER_00: I actually wrote some of the documentation,
• 16:20
  SPEAKER_00: the initial documentation for TurboPack and met some of the team.
• 16:23
  SPEAKER_01: Which was a lot faster build
• 16:25
  SPEAKER_00: system, right?
• 16:25
  SPEAKER_00: Yeah, it was a build system.
• 16:27
  SPEAKER_00: Essentially, at the time they were trying to rival Webpack, what they were working with.
• 16:32
  SPEAKER_00: And I was there initially when they were building the docks.
• 16:35
  SPEAKER_00: I flew out to San Francisco.
• 16:36
  SPEAKER_00: I was there for Next.js Conf when they announced it.
• 16:39
  SPEAKER_00: Big, really fun experience.
• 16:41
  SPEAKER_00: And I was there with everyone while they're getting everything ready for it.
• 16:46
  SPEAKER_00: And so, you know,
• 16:47
  SPEAKER_00: I do that and already in the back of my head,
• 16:49
  SPEAKER_00: I'm thinking I've seen the newsletter sort of
• 16:52
  SPEAKER_00: for my Total TypeScript stuff creep up.
• 16:54
  SPEAKER_00: I understand, OK, there's something really big here.
• 16:57
  SPEAKER_00: And when I made a pre-release sale, that just went crazy.
• 17:03
  SPEAKER_00: I was earning, let's say, X in...
• 17:08
  SPEAKER_00: And that was like 30, 40 X or something, you know, it was immediate.
• 17:12
  SPEAKER_01: And X at Vercel was already a really, really good compensation.
• 17:17
  SPEAKER_00: Absolutely.
• 17:17
  SPEAKER_00: Very, very, very happy with that.
• 17:19
  SPEAKER_00: But yeah, so I just, it was obvious.
• 17:21
  SPEAKER_00: There was no other decision I could make.
• 17:24
  SPEAKER_00: I loved working at Vercel.
• 17:25
  SPEAKER_00: I would probably go back at some point, but I just couldn't stay.
• 17:30
  SPEAKER_00: So I had to do this thing.
• 17:31
  SPEAKER_01: And then tell me about Total TypeScript.
• 17:33
  SPEAKER_01: So you started to,
• 17:34
  SPEAKER_01: you had this idea,
• 17:35
  SPEAKER_01: you started to build it two days a week on the weekends,
• 17:39
  SPEAKER_01: and then you did this pre-release sale.
• 17:41
  SPEAKER_00: Yeah.
• 17:41
  SPEAKER_00: What's the...
• 17:43
  SPEAKER_00: I almost, I try never to work on weekends, basically.
• 17:46
  SPEAKER_00: I'm extremely radical about this.
• 17:48
  SPEAKER_00: I just, I don't know.
• 17:48
  SPEAKER_00: I mean, I think it's something I mostly fail at because I'm a quite...
• 17:52
  SPEAKER_00: I'm an obsessional person.
• 17:53
  SPEAKER_00: I like trying to make something work,
• 17:56
  SPEAKER_00: but I'm not one of these guys who's doing,
• 17:58
  SPEAKER_00: what was it like, what's the SF thing where people go like 99, six days a week?
• 18:03
  SPEAKER_00: 996.
• 18:03
  SPEAKER_00: 996.
• 18:04
  SPEAKER_00: It turns my stomach,
• 18:05
  SPEAKER_01: you know, I
• 18:05
  SPEAKER_01: just hate that stuff.
• 18:07
  SPEAKER_00: Like I'm,
• 18:08
  SPEAKER_00: I am trying to,
• 18:09
  SPEAKER_00: with everything I do,
• 18:10
  SPEAKER_00: build a lifestyle and build a So Turtle TypeScript,
• 18:15
  SPEAKER_00: I was working with a guy called Joel Hooks.
• 18:28
  SPEAKER_00: Joel Hooks is extremely funny, extremely influential on me.
• 18:34
  SPEAKER_00: I've worked with him now for four years and he came up with Egghead.
• 18:39
  SPEAKER_00: He's worked with Ken C. Dodds on his courses.
• 18:42
  SPEAKER_00: Extremely successful course creator in the background.
• 18:47
  SPEAKER_00: And I basically reached out to him and I said, would you like to make this course?
• 18:50
  SPEAKER_00: And he said, hell yes.
• 18:52
  SPEAKER_00: And we went from there.
• 18:53
  SPEAKER_00: And so straight while I'm at Vercel,
• 18:56
  SPEAKER_00: I'm also working with Joel and And we do this pre-release and as I said,
• 19:00
  SPEAKER_00: just goes nuts.
• 19:01
  SPEAKER_00: And I realized, okay, I've got to fully commit to this.
• 19:05
  SPEAKER_00: And we get to, I think about January, 2023, February, 2023, and we release the full course.
• 19:13
  SPEAKER_00: And I don't know,
• 19:14
  SPEAKER_00: I think I need to look at the charts from around that time,
• 19:17
  SPEAKER_00: but it reaches seven figures extremely quickly.
• 19:20
  SPEAKER_00: And that's a revenue split between me and Joel,
• 19:21
  SPEAKER_00: of course,
• 19:22
  SPEAKER_00: there's expenses in that, but in terms of raw revenue, it was extremely exciting.
• 19:26
  SPEAKER_01: Yeah, but the seven figures,
• 19:27
  SPEAKER_01: that's $1 million, which is, I mean, an incredible milestone, right?
• 19:32
  SPEAKER_00: Which is nuts, you know, and life-changing.
• 19:35
  SPEAKER_00: And I realized, okay,
• 19:36
  SPEAKER_00: I can wake up in the morning and this money's still going to come in,
• 19:40
  SPEAKER_00: you know, this is...
• 19:42
  SPEAKER_00: This is something that I dreamed about for a long time when I was a singing teacher
• 19:45
  SPEAKER_00: as well, making material that I could sell online.
• 19:48
  SPEAKER_00: This is something I've been aiming for for a long time,
• 19:51
  SPEAKER_00: sort of high leverage work where I can do the work and then step back and go back
• 19:57
  SPEAKER_00: to my family.
• 19:58
  SPEAKER_00: And for the next couple of years,
• 20:01
  SPEAKER_00: I worked on TypeScript,
• 20:02
  SPEAKER_00: sort of expanding the course, selling a couple of supplementary courses.
• 20:06
  SPEAKER_00: And yeah, that's basically where Total TypeScript was.
• 20:09
  SPEAKER_00: And so that was the main point.
• 20:12
  SPEAKER_00: Portion of my success in the last four years has been Total Tabscope and building that out.
• 20:16
  SPEAKER_01: Yeah, and Total Tabscope has been very inspirational,
• 20:19
  SPEAKER_01: especially that you openly shared a big milestone when it hit two and
• 20:23
  SPEAKER_01: a half million dollars of total revenue,
• 20:26
  SPEAKER_01: which again,
• 20:26
  SPEAKER_01: I think for many software engineers,
• 20:28
  SPEAKER_01: you know,
• 20:28
  SPEAKER_01: that is,
• 20:29
  SPEAKER_01: of course,
• 20:30
  SPEAKER_01: we know this is Before revenue share and there's expenses involved as well.
• 20:35
  SPEAKER_01: But it's something that is pretty clearly a higher earning potential
• 20:40
  SPEAKER_01: than many great software engineering jobs.
• 20:43
  SPEAKER_01: Not necessarily all of them,
• 20:44
  SPEAKER_01: especially when we're looking at the US And some of the AI labs and whatnot,
• 20:48
  SPEAKER_01: which was probably an exception.
• 20:49
  SPEAKER_01: But the fact that there is a market and a business to be made of what I feel is
• 20:55
  SPEAKER_01: a bit kind of an honest model in a sense of like, hey, I created this thing.
• 20:59
  SPEAKER_01: People pay for it because they want to learn and they hopefully get value from
• 21:03
  SPEAKER_01: because otherwise they would ask for a refund, right?
• 21:05
  SPEAKER_00: Exactly.
• 21:05
  SPEAKER_00: We do like a very extended refund policy.
• 21:08
  SPEAKER_00: I don't tend to want to accept any other forms of money either.
• 21:12
  SPEAKER_00: I don't like necessarily doing sponsored content.
• 21:14
  SPEAKER_00: I'm not going to say never,
• 21:15
  SPEAKER_00: you know,
• 21:16
  SPEAKER_00: but I don't really,
• 21:18
  SPEAKER_00: I've got a GitHub sponsors page, but I'm really trying to take it down.
• 21:21
  SPEAKER_00: Yeah.
• 21:22
  SPEAKER_00: For a long time, I've tried to take it down.
• 21:24
  SPEAKER_00: I like the idea of just being someone who has, okay, these are the products you can buy from me.
• 21:27
  SPEAKER_00: This is how you can support me.
• 21:29
  SPEAKER_00: And hopefully this gives you 10x in terms of returns because this is
• 21:33
  SPEAKER_00: a very lucrative industry, right?
• 21:36
  SPEAKER_00: And a lot of people have education budgets that they can spend.
• 21:38
  SPEAKER_00: And if you want to spend some of your education budget on me, that's basically my model.
• 21:43
  SPEAKER_00: And a lot of that money,
• 21:45
  SPEAKER_00: a lot of the people taking the course, this is from people's education budgets.
• 21:49
  SPEAKER_00: This is companies coming and...
• 21:51
  SPEAKER_00: Spending big on education.
• 21:53
  SPEAKER_00: And that was a market that Joel was very key in pushing me towards
• 21:57
  SPEAKER_00: and realizing this,
• 21:58
  SPEAKER_00: you know,
• 21:59
  SPEAKER_00: I didn't have much of a sense for how much money was sloshing around in
• 22:03
  SPEAKER_00: the industry, especially in that age and honestly still.
• 22:07
  SPEAKER_00: But the fact that it just hit that milestone so quickly within a couple of years,
• 22:13
  SPEAKER_00: I mean, life changing.
• 22:15
  SPEAKER_01: Well, I mean,
• 22:15
  SPEAKER_01: this sounds like an amazing story and it could be a fairytale ending
• 22:18
  SPEAKER_01: where like you keep creating educational content for the rest of your life
• 22:22
  SPEAKER_01: and it's highly in demand, but then AI happened.
• 22:25
  SPEAKER_01: Yeah.
• 22:25
  SPEAKER_01: And as we know, it's changing a lot of how we work, how we finance.
• 22:30
  SPEAKER_01: And information, for example, like I don't Google that much.
• 22:34
  SPEAKER_01: I actually work with AI agents or deep research or some of those things.
• 22:38
  SPEAKER_01: I heard stories about educators,
• 22:39
  SPEAKER_01: online educators who are saying that their revenue and market share
• 22:43
  SPEAKER_01: and mind share is just falling down because people might not want to sit
• 22:47
  SPEAKER_01: through courses or sessions when you can just turn to the bot.
• 22:52
  SPEAKER_01: How did you see AI impacting the industry,
• 22:56
  SPEAKER_01: how people learn, and also your business and also you as a teacher?
• 22:59
  SPEAKER_00: It's complicated because AI has changed the game, right?
• 23:03
  SPEAKER_00: It's changed how important knowledge is and specifically the types of knowledge
• 23:08
  SPEAKER_00: that are important.
• 23:10
  SPEAKER_00: So when I'm teaching my courses, I sort of think of there as being two layers.
• 23:14
  SPEAKER_00: I'm teaching the syntax, obviously.
• 23:16
  SPEAKER_00: I'm teaching the what, but there's also the why behind it, right?
• 23:20
  SPEAKER_00: And it's very hard to teach the why without touching on the what, if that makes sense.
• 23:24
  SPEAKER_00: So the sort of medium is I'm going to teach you the syntax
• 23:29
  SPEAKER_00: and maybe you might gather the sort of wisdom around it, right?
• 23:33
  SPEAKER_00: I'm teaching you knowledge, but I'm also trying to teach you wisdom.
• 23:36
  SPEAKER_00: Knowledge is now very cheap to acquire, right?
• 23:39
  SPEAKER_00: Very, very cheap.
• 23:40
  SPEAKER_00: You can just look it up.
• 23:42
  SPEAKER_00: I have a teach skill that can just take you and just teach you the knowledge that you need.
• 23:47
  SPEAKER_00: But the wisdom has gotten no easier to learn, right?
• 23:50
  SPEAKER_00: It's still knocking about.
• 23:53
  SPEAKER_00: You are still going to run into the same issues that you ran into if you didn't have
• 23:56
  SPEAKER_00: that wisdom before, even with AI.
• 23:59
  SPEAKER_00: So in terms of my revenue from Total Timescript, that's gone down.
• 24:04
  SPEAKER_00: Obviously, because I think people are not so interested in that material anymore.
• 24:09
  SPEAKER_00: And I think people teaching that material are going to find it tricky because,
• 24:13
  SPEAKER_00: again, that knowledge is really, really hard to come by.
• 24:16
  SPEAKER_00: The only way that I've been able to,
• 24:18
  SPEAKER_00: not necessarily survive,
• 24:19
  SPEAKER_00: but it took me a long time to figure out where I wanted to be in the AI space.
• 24:24
  SPEAKER_00: Because I'm not a researcher from OpenAI,
• 24:26
  SPEAKER_00: I don't have the credentials to like talk about this stuff really,
• 24:29
  SPEAKER_00: especially in 2020, late 2023 was when I started looking at it.
• 24:34
  SPEAKER_00: And I was initially making courses about how you put AI into applications.
• 24:40
  SPEAKER_00: I thought, okay, I've been building fronted applications for a long time.
• 24:43
  SPEAKER_00: It makes sense that AI is changing things a bit there.
• 24:45
  SPEAKER_00: It started to be clear to me that that was the wrong bet to make.
• 24:49
  SPEAKER_00: I wasn't sort of seeing the returns I was expecting.
• 24:52
  SPEAKER_00: And I...
• 24:53
  SPEAKER_00: The material was good and I feel proud of it, but I didn't think I wanted to make more of it.
• 24:59
  SPEAKER_00: And around December last year, which is a date that many people cite... Oh, yes.
• 25:05
  SPEAKER_01: We know, or as we call the kind of the winter break where everyone came back.
• 25:11
  SPEAKER_00: Yeah, exactly.
• 25:11
  SPEAKER_00: The Peter break, right?
• 25:14
  SPEAKER_00: The open claw break when Opus 4.5 is out.
• 25:17
  SPEAKER_00: People have a lot of time off and they just start slamming it and they realize,
• 25:20
  SPEAKER_00: wow, okay, things are really happening.
• 25:22
  SPEAKER_00: And that happened to me too.
• 25:24
  SPEAKER_00: And I realized, okay, the AI is now good enough that you can delegate to it.
• 25:30
  SPEAKER_00: You can actually make structures around the agents.
• 25:35
  SPEAKER_00: The agents can handle the knowledge, the syntax, the sort of, I call it the tactical stuff.
• 25:42
  SPEAKER_00: And you can handle the strategic stuff, the long-term thinking.
• 25:46
  SPEAKER_00: I use that a lot.
• 25:47
  SPEAKER_00: I know you had John Asterhow on this podcast.
• 25:49
  SPEAKER_00: I've been really wanting to chat to him myself.
• 25:51
  SPEAKER_00: Like he's a huge influence on me.
• 25:53
  SPEAKER_00: And he talks about the difference between tactical programming and strategic programming.
• 25:58
  SPEAKER_00: AI has largely eaten tactical programming in my view, and it's up to us to handle the strategic.
• 26:05
  SPEAKER_00: And I realized, okay, in the strategic layer, there's a course I can create there.
• 26:10
  SPEAKER_00: I was looking at Ralph Loops at the time,
• 26:12
  SPEAKER_00: sort of Jeffrey Huntley was building this really cool stuff where you can loop
• 26:17
  SPEAKER_00: the agent and get it to follow these goals.
• 26:19
  SPEAKER_00: And I thought, okay, there's definitely material here.
• 26:23
  SPEAKER_00: I just need to find a structure with...
• 26:26
  SPEAKER_00: And I
• 26:29
  SPEAKER_01: remember the Ralph Loops,
• 26:30
  SPEAKER_01: you also made a video that became very popular on YouTube, Axa,
• 26:34
  SPEAKER_01: everywhere,
• 26:34
  SPEAKER_01: where you basically said like, all right, like here's how I created a Ralph Loop.
• 26:40
  SPEAKER_01: I have a project that has a lot of like to-dos and you did a great job
• 26:43
  SPEAKER_01: and we'll link that video in the show notes below where you said like,
• 26:47
  SPEAKER_01: all right,
• 26:47
  SPEAKER_01: here's how usually we would try to get agents to work,
• 26:50
  SPEAKER_01: like do a plan upfront and then implement each step,
• 26:53
  SPEAKER_01: like the kind of traditional top-down plan And you're saying the problem is that
• 26:56
  SPEAKER_01: as you're implementing or even the agents implementing it,
• 26:58
  SPEAKER_01: realizes, hang on, I need to do more stuff.
• 26:59
  SPEAKER_01: And then how do you modify the plan?
• 27:02
  SPEAKER_01: And then enter the Ralph loop where you gave the structures that you used at the time.
• 27:05
  SPEAKER_01: There was like an MD file.
• 27:07
  SPEAKER_01: It keeps it as there and it kind of like eats stuff.
• 27:10
  SPEAKER_01: Actually,
• 27:10
  SPEAKER_01: the couple of years that I spent trying to put agents
• 27:12
  SPEAKER_01: into applications was really beneficial there.
• 27:25
  SPEAKER_00: Because when you try to build an app that contains an agent,
• 27:29
  SPEAKER_00: you're always thinking about data flow.
• 27:31
  SPEAKER_00: You're thinking about how the data is going to get in,
• 27:34
  SPEAKER_00: what shape it's going to be,
• 27:35
  SPEAKER_00: what priority, whether you're going to put it in the system prompt, the user prompt.
• 27:40
  SPEAKER_00: And so when I got to working with the harnesses,
• 27:44
  SPEAKER_00: it felt like, oh, this just feels very familiar.
• 27:46
  SPEAKER_00: I just need to, you know, where is the state going to live?
• 27:49
  SPEAKER_00: How am I going to pass the state into the agents?
• 27:52
  SPEAKER_00: What shape is that going to look like?
• 27:54
  SPEAKER_00: How am I going to compact it or clear it?
• 27:57
  SPEAKER_00: And this was still pretty early days of Claw Co.
• 28:00
  SPEAKER_00: Claw Co had been out five, six months at that point.
• 28:03
  SPEAKER_00: And it just felt very natural.
• 28:05
  SPEAKER_00: And from there,
• 28:06
  SPEAKER_00: I just got obsessed with these,
• 28:08
  SPEAKER_00: I suppose we would call them loops now, but really they're just processes.
• 28:12
  SPEAKER_00: They're sort of different ways of stringing agents together.
• 28:14
  SPEAKER_01: It's just kind of like the diagram, like if you can draw...
• 28:17
  SPEAKER_01: I would call it a finite state machine.
• 28:19
  SPEAKER_01: It felt very similar to the stuff I've been working on in XState,
• 28:22
  SPEAKER_01: which is process-based,
• 28:22
  SPEAKER_01: which is state-based,
• 28:23
  SPEAKER_01: event-based sometimes as well,
• 28:24
  SPEAKER_01: where you have an agent at the bottom there
• 28:26
  SPEAKER_00: that's calling an event.
• 28:39
  SPEAKER_00: And I started just to see really good results from that.
• 28:43
  SPEAKER_00: And I would do these experiments where I would try sort of building out my process
• 28:47
  SPEAKER_00: and I would build out a feature of,
• 28:50
  SPEAKER_00: you know, I have a few apps that I work on kind of to extend what I do.
• 28:54
  SPEAKER_00: Like I have a custom video editor.
• 28:55
  SPEAKER_00: I have a huge thing, a huge code base.
• 28:58
  SPEAKER_00: I have a few open source projects as well.
• 29:00
  SPEAKER_00: And I was just building these little loops and little pipelines.
• 29:05
  SPEAKER_00: And I would sometimes just drop it and go back to what the default setup was.
• 29:09
  SPEAKER_00: And I just noticed a huge difference.
• 29:11
  SPEAKER_00: Like I just felt, wow, OK, the stuff that I'm doing here is really setting me up for success.
• 29:17
  SPEAKER_00: And I started thinking, what's the best way that I can distribute that?
• 29:21
  SPEAKER_00: How can I share that with other people better?
• 29:24
  SPEAKER_00: And that's where I sort of started landing on skills as the distribution mechanism
• 29:28
  SPEAKER_00: for this stuff.
• 29:30
  SPEAKER_01: These are the skills for AI.
• 29:33
  SPEAKER_01: Bots harnesses where you can typically define them and now you can,
• 29:36
  SPEAKER_01: once you install them, you can invoke them with a slash command.
• 29:39
  SPEAKER_01: Exactly.
• 29:40
  SPEAKER_00: Skills really, they're just a folder of markdown files that can sit in your computer somewhere.
• 29:46
  SPEAKER_00: And the agents can either invoke them themselves,
• 29:49
  SPEAKER_00: so model invoked skills,
• 29:51
  SPEAKER_00: or you can have skills that the agent doesn't know about,
• 29:53
  SPEAKER_00: but you can invoke yourself, so user invoked skills.
• 29:56
  SPEAKER_00: And I started seeing these skill sets pop up everywhere,
• 29:59
  SPEAKER_00: like superpowers and, you know, clawed code plugins that you can install.
• 30:03
  SPEAKER_00: I think GStack as well.
• 30:05
  SPEAKER_00: I realized, OK,
• 30:06
  SPEAKER_00: maybe I can distribute what I have,
• 30:08
  SPEAKER_00: this process, as a set of skills and see what people think of it.
• 30:12
  SPEAKER_00: And initially, I just put it up and I was doing other stuff.
• 30:15
  SPEAKER_00: I was working on a course and I checked back in and I realized,
• 30:18
  SPEAKER_00: oh, it's got more stars than anything else I've ever done.
• 30:22
  SPEAKER_00: I've not even really talked about it.
• 30:24
  SPEAKER_00: You know, it's just sat there on its own.
• 30:26
  SPEAKER_00: Word of mouth, I suppose.
• 30:27
  SPEAKER_00: I've done a little bit of documentation, but really not much.
• 30:30
  SPEAKER_00: And it's just exploded already.
• 30:32
  SPEAKER_00: So I thought, OK, maybe I should put a little bit more work into this.
• 30:35
  SPEAKER_00: Maybe I should talk about them.
• 30:37
  SPEAKER_00: And I did a talk at, I think, where we met last, which was AI Engineer London, about in April.
• 30:44
  SPEAKER_00: That talk was entitled Software Fundamentals Still Matter.
• 30:48
  SPEAKER_00: And that talk is now up to, I think, 1.2 million views or something.
• 30:52
  SPEAKER_00: And I mentioned the skill set.
• 30:53
  SPEAKER_00: The skill set is now at 230,000 stars.
• 30:57
  SPEAKER_00: It is now the second most starred skills repo in the world.
• 31:01
  SPEAKER_00: I think somewhere like 20th to 25th of the most starred repos of all time.
• 31:06
  SPEAKER_00: Wow.
• 31:07
  SPEAKER_00: You know what I mean?
• 31:08
  SPEAKER_00: Like, what's going on?
• 31:10
  SPEAKER_00: So there's obviously a hunger for this.
• 31:12
  SPEAKER_00: So this was the second time in my career,
• 31:14
  SPEAKER_00: just like when I was putting out the little TypeScript videos where I felt,
• 31:16
  SPEAKER_00: wow, there's a momentum here.
• 31:17
  SPEAKER_00: There's something happening.
• 31:19
  SPEAKER_00: And so I felt I had to double down on that.
• 31:22
  SPEAKER_01: The skills, how did you write them?
• 31:24
  SPEAKER_01: Is this trying to capture your workflow, your understanding of what works with agents?
• 31:30
  SPEAKER_01: You know,
• 31:30
  SPEAKER_01: not just right now,
• 31:31
  SPEAKER_01: but of course,
• 31:31
  SPEAKER_01: you're thinking about state,
• 31:33
  SPEAKER_01: you're thinking about how you were integrating AI into applications,
• 31:37
  SPEAKER_01: which again, didn't take off all that much, but you learned.
• 31:41
  SPEAKER_01: So is this kind of like Matt's workflow, Matt's way of what works for me?
• 31:46
  SPEAKER_00: Yes, that's what it is.
• 31:47
  SPEAKER_00: I try to think,
• 31:49
  SPEAKER_00: first of all,
• 31:50
  SPEAKER_00: People are going to use these skills and they're going to tinker with them.
• 31:53
  SPEAKER_00: So how do I make the simplest set of skills that people can audit very easily?
• 31:58
  SPEAKER_00: I'm trying to think, how do I maximize people picking these up and using them at work?
• 32:05
  SPEAKER_00: So, for instance, the grill me skill, which is the most popular one.
• 32:09
  SPEAKER_00: I don't know if you've used it.
• 32:10
  SPEAKER_00: I use it as well,
• 32:12
  SPEAKER_01: yeah.
• 32:14
  SPEAKER_01: It's annoying how damn it grilled me.
• 32:16
  SPEAKER_01: I just asked it.
• 32:17
  SPEAKER_01: I was like,
• 32:17
  SPEAKER_01: I'd like to expose an API endpoint that can tell whoever has
• 32:23
  SPEAKER_01: the authenticated token,
• 32:25
  SPEAKER_01: have some basic authentic Is this email a subscriber to my email list or not?
• 32:29
  SPEAKER_01: Because I want to connect it with one of the events that I'm doing with
• 32:34
  SPEAKER_01: to get priority to pay subscribers.
• 32:36
  SPEAKER_01: And that's very simple, right?
• 32:37
  SPEAKER_01: And then the grail me thing,
• 32:39
  SPEAKER_01: it starts to just really grail me like, okay, Okay, so what about authentication?
• 32:42
  SPEAKER_01: Do you want the bearer token or do you want it in JSON, which is not as safe, et cetera?
• 32:46
  SPEAKER_01: I'm like, okay, well, that's a decision to make.
• 32:47
  SPEAKER_01: And then we go through all of these decisions and it goes really low level,
• 32:51
  SPEAKER_01: including like, okay, like how do we enforce rate limits?
• 32:54
  SPEAKER_01: When it comes to rate limits,
• 32:55
  SPEAKER_01: you want to exactly do it when you get like a thousand per day and not allow
• 32:59
  SPEAKER_01: a single more, which is more complexity.
• 33:02
  SPEAKER_01: I just realized it's been a long time since I've had such
• 33:06
  SPEAKER_01: an involved design discussion with a team or an engineering team.
• 33:09
  SPEAKER_01: And you typically have it when someone has deep domain knowledge.
• 33:11
  SPEAKER_01: And I was both annoyed by, this is just simple, like, no, don't worry about that.
• 33:16
  SPEAKER_01: But also impressed that this thing,
• 33:19
  SPEAKER_01: this AI, this LLM through a series of prompts is able to do all of this.
• 33:23
  SPEAKER_00: I mean, everyone's got a grill me story,
• 33:25
  SPEAKER_00: so I get so many of these at conferences,
• 33:27
  SPEAKER_00: people say,
• 33:27
  SPEAKER_00: you know,
• 33:29
  SPEAKER_00: this very,
• 33:30
  SPEAKER_00: very simple skill,
• 33:31
  SPEAKER_00: it's really just telling the agent to interview you relentlessly about the topic.
• 33:36
  SPEAKER_00: It's a very small skill.
• 33:37
  SPEAKER_00: It just has this weird emergent behaviour with it,
• 33:40
  SPEAKER_00: where the models just start thinking a little bit outside the box.
• 33:44
  SPEAKER_00: And they start throwing ideas at you.
• 33:46
  SPEAKER_00: I think I got it originally from like a Tariq who works at ClawCode.
• 33:49
  SPEAKER_00: He's saying basically get the agent to interview you and then you'll see better results.
• 33:54
  SPEAKER_00: So I encode that into a little skill and I realized,
• 33:56
  SPEAKER_00: wow, okay, it's just sort of 10 times better than anything I've ever used.
• 34:00
  SPEAKER_00: And that grill me skill was the first one.
• 34:02
  SPEAKER_00: That sort of reminded me of the discussions I would have at my first job,
• 34:05
  SPEAKER_00: you know, with the guy with the sandals.
• 34:07
  SPEAKER_00: It's this very senior engineer in the room really getting me to think
• 34:11
  SPEAKER_00: about everything that I'd done.
• 34:12
  SPEAKER_00: It was the most familiar thing to me to actually working
• 34:15
  SPEAKER_00: with someone like Anderis Drake at Xstate, you know.
• 34:18
  SPEAKER_00: It just felt like a really high quality developer was asking me these good questions.
• 34:22
  SPEAKER_00: And I thought, wow, okay.
• 34:24
  SPEAKER_00: And then I started sort of taking that and going like,
• 34:28
  SPEAKER_00: how do I mine this agent for more software fundamental stuff?
• 34:36
  SPEAKER_00: How do I make it feel more like a proper developer, a real senior?
• 34:41
  SPEAKER_00: How do I...
• 34:43
  SPEAKER_00: What I liked about the Grill Me skill is...
• 34:58
  SPEAKER_01: It forced me to make decisions that I know what decision to make when I think
• 35:02
  SPEAKER_01: about it, but it is my decision.
• 35:05
  SPEAKER_01: So unlike when I tell the when you do the slash goal command, like build this and it goes off.
• 35:13
  SPEAKER_01: And it makes all the decisions or most key decisions.
• 35:16
  SPEAKER_01: What I like about grill me is I both make the decision,
• 35:19
  SPEAKER_01: but also sometimes it reminds me about things that I didn't think too much about,
• 35:24
  SPEAKER_01: or maybe it reminds me that I should do a bit of a research,
• 35:26
  SPEAKER_01: for example,
• 35:26
  SPEAKER_01: like it asked me like,
• 35:27
  SPEAKER_01: which all Authentication,
• 35:29
  SPEAKER_01: would I want to do bear token or over post or even over get?
• 35:33
  SPEAKER_01: And then I'm like,
• 35:34
  SPEAKER_01: hang on, like,
• 35:35
  SPEAKER_01: I'm going to look up like what the differences are or ask a different session
• 35:38
  SPEAKER_01: to educate.
• 35:39
  SPEAKER_01: So like it makes me a better professional.
• 35:42
  SPEAKER_01: And I do have this belief that when you're working with AI,
• 35:45
  SPEAKER_01: like as long as we're learning, I think we're fine.
• 35:48
  SPEAKER_01: As long as we stop learning and outsource the learning to this thing,
• 35:51
  SPEAKER_01: trouble will be brewing maybe, you know, months or years down the road.
• 35:55
  SPEAKER_00: 100%.
• 35:55
  SPEAKER_00: There's two things there, right?
• 35:56
  SPEAKER_00: I think what everybody underestimates about agents,
• 35:59
  SPEAKER_00: everybody, is that there is a communication gap between you and the agent, right?
• 36:04
  SPEAKER_00: There is a barrier there.
• 36:05
  SPEAKER_00: You feel like because the agent is not a human and
• 36:09
  SPEAKER_00: because you understand your hierarchy of values,
• 36:12
  SPEAKER_00: you think that the agent will just pick up on them, right?
• 36:14
  SPEAKER_00: There's this sort of feeling of, yeah,
• 36:15
  SPEAKER_00: just trust the model,
• 36:16
  SPEAKER_00: especially with the top tier models, you know, just trust the model.
• 36:19
  SPEAKER_00: But the agent,
• 36:21
  SPEAKER_00: however good it is,
• 36:22
  SPEAKER_00: however smart the model is, you know, even mythos, it can't read your mind.
• 36:27
  SPEAKER_00: It can't read your mind.
• 36:28
  SPEAKER_00: So you have to, there has to be some process of communicating your values to the agent.
• 36:33
  SPEAKER_00: Because often when you do it like a goal,
• 36:36
  SPEAKER_00: when you just go, okay, just spam me out some code, give me some slop.
• 36:39
  SPEAKER_00: The agent is going to produce something that's totally misaligned from you
• 36:42
  SPEAKER_00: because it doesn't understand what you think is important.
• 36:45
  SPEAKER_00: And so Grill Me is not only about implementation details.
• 36:48
  SPEAKER_00: It's also about establishing, okay, do this.
• 36:50
  SPEAKER_00: This is in scope.
• 36:51
  SPEAKER_00: This is not in scope.
• 36:52
  SPEAKER_00: Here's what I think is important.
• 36:54
  SPEAKER_00: And so it's the agent getting to know you.
• 36:57
  SPEAKER_01: Matt just described the grill me skill.
• 36:59
  SPEAKER_01: When I used this skill to design an API endpoint,
• 37:01
  SPEAKER_01: the first questions it asked were about what the endpoint was and wasn't allowed
• 37:05
  SPEAKER_01: to do and who it was allowed to do it for.
• 37:08
  SPEAKER_01: Now, in my case,
• 37:08
  SPEAKER_01: I had a decent idea of what I wanted,
• 37:11
  SPEAKER_01: but it's generally a terrible idea to let an agent improvise authentication
• 37:14
  SPEAKER_01: and authorization as they would often do.
• 37:17
  SPEAKER_01: This brings us to our season sponsor, WorkOS.
• 37:20
  SPEAKER_01: How do you authorize AI agents?
• 37:22
  SPEAKER_01: The problem you have is how you want to control the scope of the agent.
• 37:26
  SPEAKER_01: The tricky part is how permissions are static, but the job of what the agent does is dynamic.
• 37:31
  SPEAKER_01: So teams pick between two bad options.
• 37:34
  SPEAKER_01: Either you read the prompt,
• 37:35
  SPEAKER_01: then approve every tool invocation call by hand,
• 37:38
  SPEAKER_01: and then read the prompt again,
• 37:39
  SPEAKER_01: then approve by hand again, until you eventually just stop reading the prompt.
• 37:44
  SPEAKER_01: Or you just run in YOLO mode,
• 37:45
  SPEAKER_01: letting it rip and hoping that whatever the agent does is not irreversible.
• 37:50
  SPEAKER_01: But there's a better way.
• 37:51
  SPEAKER_01: Worko has just launched Airlock, intent-based access control for agents.
• 37:55
  SPEAKER_01: You write the rules in plain English, for example.
• 37:57
  SPEAKER_01: Read repos and comments on PRs.
• 38:00
  SPEAKER_01: Anything touching alt or billing needs sign-off.
• 38:02
  SPEAKER_01: Never push to main.
• 38:03
  SPEAKER_01: Every call that the agent makes is just I'd also like
• 38:05
  SPEAKER_01: to mention our presenting sponsor, TurboPuffer.
• 38:08
  SPEAKER_01: Matt and I are discussing a fundamental question.
• 38:11
  SPEAKER_01: How do you get agents to remember what's important?
• 38:12
  SPEAKER_01: Here's an idea.
• 38:33
  SPEAKER_01: What if instead of building a complex memory system,
• 38:36
  SPEAKER_01: you just let the agent search its entire history?
• 38:39
  SPEAKER_01: This seems like it will be very, very expensive, but with TurboPuffer, it isn't.
• 38:43
  SPEAKER_01: TurboPuffer's object storage native architecture means that the marginal cost
• 38:47
  SPEAKER_01: to source sessions transcript is almost nothing,
• 38:50
  SPEAKER_01: making it economical to index the entire chat history.
• 38:53
  SPEAKER_01: And because TurboPuffer namespaces scale virtually without limit,
• 38:57
  SPEAKER_01: you can create a dedicated search index for every agent.
• 38:59
  SPEAKER_01: Here's a good example of this.
• 39:01
  SPEAKER_01: Entire, another seasoned sponsor of the podcast,
• 39:03
  SPEAKER_01: indexes hundreds of millions of agents session transcripts for search and then lets
• 39:07
  SPEAKER_01: the coding agent retrieve what it needs to recall how and why
• 39:11
  SPEAKER_01: an engineering decision was made.
• 39:12
  SPEAKER_01: Entire showed that their agent was more accurate,
• 39:15
  SPEAKER_01: Use fewer tokens,
• 39:16
  SPEAKER_01: and took less time to find memories when it used TurboPuffer instead of Git history
• 39:20
  SPEAKER_01: and a CLI.
• 39:21
  SPEAKER_01: Agent memory is a complex and evolving use case, but perhaps there's a bitter lesson here.
• 39:25
  SPEAKER_01: Maybe the best solution is the simple one.
• 39:28
  SPEAKER_01: Just search every transcript.
• 39:30
  SPEAKER_01: With TurboPuffer, this is actually possible.
• 39:32
  SPEAKER_01: If agent memory is something you're trying to solve,
• 39:34
  SPEAKER_01: then please reach out to the TurboPuffer team at turbopuffer.com slash pragmatics.
• 39:38
  SPEAKER_00: And which other skills did you create?
• 39:40
  SPEAKER_00: So from there, I thought, OK, how do I take that conversation and turn it into code?
• 39:46
  SPEAKER_00: And I was immediately scared because I'd been working with models just
• 39:53
  SPEAKER_00: before they were good and before the December winter where things got really good.
• 39:59
  SPEAKER_00: And so I felt the constraint.
• 40:02
  SPEAKER_00: I knew that,
• 40:04
  SPEAKER_00: for instance,
• 40:05
  SPEAKER_00: the more context you put, you give to the agent, the worse it performs.
• 40:09
  SPEAKER_00: I know you had Dex Haworthy on this podcast.
• 40:11
  SPEAKER_00: And Dex is a really big influence on me,
• 40:14
  SPEAKER_00: especially his idea of the smart zone and the dumb zone.
• 40:16
  SPEAKER_00: Smart zone and dumb zone.
• 40:17
  SPEAKER_00: Yeah.
• 40:18
  SPEAKER_00: So idea of that just to kind of,
• 40:20
  SPEAKER_00: so you don't have to go and listen to that podcast in full, although you should.
• 40:24
  SPEAKER_00: You have essentially the more context you give to the agent,
• 40:28
  SPEAKER_00: every token is shouting for attention.
• 40:31
  SPEAKER_00: And the more voices you put into that room, the harder it is to hear the important ones.
• 40:36
  SPEAKER_00: And so the model starts losing the connections between things and making mistakes
• 40:40
  SPEAKER_00: because of that.
• 40:41
  SPEAKER_00: And you can think of that as a slow decline, but there is a And so I started thinking...
• 41:12
  SPEAKER_00: How do I take work that's bigger than 150k tokens,
• 41:15
  SPEAKER_00: which is not very large,
• 41:17
  SPEAKER_00: and portion it out over multiple context windows, multiple sessions?
• 41:21
  SPEAKER_00: And this took me a lot of tries, a lot of different fiddling around with different approaches.
• 41:27
  SPEAKER_00: So that's the idea.
• 41:28
  SPEAKER_00: So I started thinking, how do I take that Ralph Loop idea
• 41:55
  SPEAKER_00: But make it a little bit more stable and turn that into skills.
• 41:58
  SPEAKER_00: And so what I realized I needed was two different types of documents.
• 42:02
  SPEAKER_00: You need a document for where you're going, which is the destination document.
• 42:06
  SPEAKER_00: I used to call that a product requirements document or a spec is what I call it now.
• 42:10
  SPEAKER_00: So that's the specification that declares when you've reached the end.
• 42:14
  SPEAKER_00: And then you need to break that spec down into individual tickets, one ticket per session.
• 42:19
  SPEAKER_00: And so I'd have a very simple skill just to spec and then to tickets.
• 42:24
  SPEAKER_00: And so you take that grilling session that you've had and you turn it into a spec.
• 42:28
  SPEAKER_00: Now that spec can work over, you know, 30, 40 tickets, let's say.
• 42:32
  SPEAKER_00: You can have really massive, great big chunks of work that are all tied into that spec.
• 42:37
  SPEAKER_00: And so that's the main idea.
• 42:38
  SPEAKER_00: You just grill,
• 42:39
  SPEAKER_00: you turn that grilling into a spec,
• 42:41
  SPEAKER_00: and then you just run some kind of implementer loop over those tickets
• 42:45
  SPEAKER_00: until you've got a huge chunk of work.
• 42:46
  SPEAKER_01: After grilling, do you get user input as well or throughout this process or it depends?
• 42:51
  SPEAKER_00: I was mostly designing this to be run for the user to be away from
• 42:56
  SPEAKER_00: the keyboard totally because...
• 42:58
  SPEAKER_00: There's this idea of like the day shift and the night shift.
• 43:01
  SPEAKER_00: Have you heard of this?
• 43:02
  SPEAKER_00: No, no, no.
• 43:02
  SPEAKER_00: It's great.
• 43:03
  SPEAKER_00: Basically,
• 43:03
  SPEAKER_00: the optimal way to work with agents is to plan during the day shift and then get
• 43:09
  SPEAKER_00: the agents to work during the night shift, right?
• 43:12
  SPEAKER_00: And so hopefully you wake up in the morning and you've got beautiful, clean code to look at.
• 43:17
  SPEAKER_00: And that's what I was trying to optimize my process around because I was really sick
• 43:21
  SPEAKER_00: of what I,
• 43:22
  SPEAKER_00: and what I still do to an extent of just switching between terminals,
• 43:26
  SPEAKER_00: context switching all the time, just going boom, boom, boom, boom, boom.
• 43:29
  SPEAKER_00: What I wanted and what I'm trying to optimize for is to just get a good chunk
• 43:34
  SPEAKER_00: of planning done and then let the agent work for a couple of hours.
• 43:38
  SPEAKER_00: And then I can do other work,
• 43:39
  SPEAKER_00: decent chunks of time,
• 43:41
  SPEAKER_00: 15-minute chunks working on one thing,
• 43:43
  SPEAKER_00: planning on stuff, and then I can review the code and do that.
• 43:46
  SPEAKER_00: So that's what I was trying to optimize for all the time when I was doing Ralph loops.
• 43:49
  SPEAKER_00: And that was the big thing that I found in December is these guys are good enough
• 43:53
  SPEAKER_00: to delegate to.
• 43:54
  SPEAKER_00: And so I can run them AFK.
• 43:56
  SPEAKER_01: And then you have a different skill as well,
• 43:57
  SPEAKER_01: which is a bit more ambitious called the Wayfinder skill.
• 44:00
  SPEAKER_01: Can we talk about that?
• 44:01
  SPEAKER_01: Absolutely.
• 44:02
  SPEAKER_00: So in exactly the same way that implementation,
• 44:05
  SPEAKER_00: I noticed,
• 44:06
  SPEAKER_00: needed to be split out over multiple sessions,
• 44:09
  SPEAKER_00: sometimes you're grilling something and you're hitting the limits.
• 44:13
  SPEAKER_00: You're going to grill something, you know, build me a Stripe clone or something, right?
• 44:17
  SPEAKER_00: You are going to hit the limits there.
• 44:18
  SPEAKER_00: There's no way you can plan that in 150k tokens.
• 44:21
  SPEAKER_00: And so I thought,
• 44:21
  SPEAKER_00: how do I break that up so that I can run grilling sessions
• 44:26
  SPEAKER_00: that can be infinite length, right?
• 44:29
  SPEAKER_00: How do I split up grilling so that it can work like that?
• 44:33
  SPEAKER_00: And so I came up with this idea.
• 44:35
  SPEAKER_00: Again, I'm thinking about the flow of information.
• 44:38
  SPEAKER_00: Essentially, like, what does it need to perform well in a grilling session?
• 44:42
  SPEAKER_00: It probably needs to understand exactly what the purpose of that grilling session is.
• 44:47
  SPEAKER_00: But it also needs to understand what's been decided so far.
• 44:49
  SPEAKER_00: It needs to understand what other grilling sessions might be happening at that moment.
• 44:53
  SPEAKER_00: And I came up with this idea of a map.
• 44:57
  SPEAKER_00: And the map would be the sort of center point of everything that was needed for all
• 45:03
  SPEAKER_00: the decisions that you were coming up with.
• 45:05
  SPEAKER_00: And once you've got a map,
• 45:05
  SPEAKER_00: you realize, okay,
• 45:07
  SPEAKER_00: there are certain things I can, like,
• 45:08
  SPEAKER_00: as I'm trying to find my way to a destination,
• 45:11
  SPEAKER_00: there are certain things I know I need to decide,
• 45:14
  SPEAKER_00: certain points that are kind of like milestones on the map.
• 45:16
  SPEAKER_00: And there's a fog of war.
• 45:18
  SPEAKER_00: And that lovely metaphor just sort of carried me through designing the rest of the skill, right?
• 45:22
  SPEAKER_00: Because you've got your map, you've got your fog of war, you vaguely know where you're going.
• 45:27
  SPEAKER_00: And every time you have a grilling session, it opens out more points on the map.
• 45:30
  SPEAKER_00: And so you sort of figure out where you're going.
• 45:33
  SPEAKER_00: And so this is kind of like a directed acyclic graph where you're walking down
• 45:37
  SPEAKER_00: until you reach your final destination.
• 45:40
  SPEAKER_00: And so you've got the map and then each individual session in there are tickets on that map.
• 45:45
  SPEAKER_00: And I realized, okay, grilling is good, but what if you need to prototype?
• 45:49
  SPEAKER_00: What if you need to do research?
• 45:50
  SPEAKER_00: What if you need to do like an arbitrary task, like provisions and infrastructure or something?
• 45:54
  SPEAKER_00: Well, those are different types of tickets on the map.
• 45:57
  SPEAKER_00: And Wayfinder basically just guides you through this process.
• 46:01
  SPEAKER_00: I've had maps that have,
• 46:02
  SPEAKER_00: you know, At 50, 100 tickets or something until I finally reach my destination.
• 46:06
  SPEAKER_00: I've actually been using it for course planning as well.
• 46:09
  SPEAKER_00: So non-technical stuff, which is really great.
• 46:11
  SPEAKER_00: I've been using it to build a garden office in my garden, right?
• 46:16
  SPEAKER_00: A lot of these skills we say.
• 46:17
  SPEAKER_00: Okay, these are great for engineering.
• 46:20
  SPEAKER_00: Then you realize, okay, engineering is just a discipline that what are we doing here?
• 46:24
  SPEAKER_00: We're just discussing something.
• 46:26
  SPEAKER_00: We're doing things in real life, like clicking around websites and stuff.
• 46:30
  SPEAKER_00: You realize how easily that can map onto other domains.
• 46:34
  SPEAKER_00: So that's kind of,
• 46:35
  SPEAKER_00: maybe we can touch on that a bit later,
• 46:37
  SPEAKER_00: which is I am thinking how transposable this stuff is into different disciplines and
• 46:42
  SPEAKER_00: into different areas of life.
• 46:43
  SPEAKER_00: So Wayfinder has been great.
• 46:45
  SPEAKER_01: Yeah,
• 46:45
  SPEAKER_01: but if we think one interesting thing about engineering and software engineering,
• 46:49
  SPEAKER_01: when Hillel Wayne was on the podcast,
• 46:51
  SPEAKER_01: he interviewed engineers like who he thought are real engineers,
• 46:56
  SPEAKER_01: chemical engineers,
• 46:56
  SPEAKER_01: mechanical engineers,
• 46:57
  SPEAKER_01: civil engineers, and to try to find out is software engineering real engineering.
• 47:01
  SPEAKER_01: And in the end, he found that it probably is.
• 47:04
  SPEAKER_01: But he said that one interesting thing with software that is very different
• 47:07
  SPEAKER_01: to every other engineering profession is the The materials that we work with.
• 47:11
  SPEAKER_01: In every single place,
• 47:12
  SPEAKER_01: mechanical engineering,
• 47:13
  SPEAKER_01: civil engineering,
• 47:14
  SPEAKER_01: even chemical engineering, you have a material that has a threshold of things.
• 47:18
  SPEAKER_01: You don't know exactly what it's like.
• 47:20
  SPEAKER_01: You know that it'll be like, it can take about this much load, etc.
• 47:23
  SPEAKER_01: But in software,
• 47:24
  SPEAKER_01: the But software, a code,
• 47:31
  SPEAKER_01: you
• 47:36
  SPEAKER_01: run it a thousand times and it does the same thing a thousand times,
• 47:39
  SPEAKER_01: whereas in other fields it doesn't.
• 47:41
  SPEAKER_01: And he said that he sees a big difference.
• 47:44
  SPEAKER_01: But now I guess with LLMs,
• 47:46
  SPEAKER_01: maybe we have this thing where we have a thing where you run it a thousand times
• 47:49
  SPEAKER_01: and it will have these variants that most of engineering has.
• 47:52
  SPEAKER_01: So who knows if If both what works with LLMs will be useful at other engineering
• 47:57
  SPEAKER_01: where, again,
• 47:58
  SPEAKER_01: they already had this variance or we can take some approaches
• 48:01
  SPEAKER_01: from other engineering professions that will maybe work nicely with working
• 48:06
  SPEAKER_01: with this material called AI.
• 48:07
  SPEAKER_00: I totally agree.
• 48:09
  SPEAKER_00: What I think is interesting about software engineering and the way
• 48:13
  SPEAKER_00: the reason agents are good with it is it's all the inputs and all
• 48:17
  SPEAKER_00: the outputs are text based.
• 48:19
  SPEAKER_00: Everything.
• 48:20
  SPEAKER_00: So the inputs, code, documentation, instructions for the agent on what to do, all text-based.
• 48:26
  SPEAKER_00: And the output is more code,
• 48:28
  SPEAKER_00: is test suites, is type checking results, linting, all that stuff is text-based.
• 48:34
  SPEAKER_00: The thing that agents really struggle with is anything that's non-text-based.
• 48:38
  SPEAKER_00: You know, you see these amazing demos of people one-shotting a perfect UI first time.
• 48:43
  SPEAKER_00: Well, what about if you have an interaction problem in that UI?
• 48:46
  SPEAKER_00: What if you're like hovering over something and the animation doesn't look right?
• 48:49
  SPEAKER_00: How are you going to get that to the agent?
• 48:51
  SPEAKER_00: I mean, you can record it a video,
• 48:53
  SPEAKER_00: I suppose,
• 48:54
  SPEAKER_00: and it sort of pauses on certain frames,
• 48:56
  SPEAKER_00: let's say, but it's actually not that good in terms of vision just yet.
• 49:00
  SPEAKER_00: And so anything that's non-text based is just garbage from the agent.
• 49:05
  SPEAKER_00: It just can't handle it.
• 49:11
  SPEAKER_00: If you can turn, I assume they're doing simulations, right?
• 49:14
  SPEAKER_00: I assume they're doing some kind of,
• 49:16
  SPEAKER_00: you know, I don't know if you have like a linter that can work on architectural diagram.
• 49:19
  SPEAKER_00: I'm sure you have some variety of that, right?
• 49:22
  SPEAKER_00: Some simulation.
• 49:23
  SPEAKER_00: If you can make that text-based,
• 49:24
  SPEAKER_00: if you can take the interactions that you have in your day-to-day life and turn them
• 49:28
  SPEAKER_00: into text,
• 49:29
  SPEAKER_00: which mostly they are anyway, then the agents are going to do a pretty good job.
• 49:32
  SPEAKER_00: That's something I'm trying to do currently is take all of the services that I use
• 49:37
  SPEAKER_00: and plug them into agents, right?
• 49:38
  SPEAKER_00: Make them available to the agents.
• 49:40
  SPEAKER_00: Yeah.
• 49:41
  SPEAKER_00: The more we can make our work agent friendly, the better results we're going to get.
• 49:45
  SPEAKER_01: Circling back to AI as a whole and then what has changed.
• 49:50
  SPEAKER_01: It has changed so many things, but...
• 49:54
  SPEAKER_01: One thing that comes up with AI is often,
• 49:57
  SPEAKER_01: especially researchers and people working in AI companies, is no priors.
• 50:00
  SPEAKER_01: With AI, you should let go of everything that we know before because this thing is different.
• 50:04
  SPEAKER_01: Start from scratch.
• 50:05
  SPEAKER_01: The approaches might not work.
• 50:07
  SPEAKER_01: In fact, let's assume they don't work and come up with new approaches.
• 50:10
  SPEAKER_01: Having been a developer before AI and actually like you were really interested
• 50:15
  SPEAKER_01: in building quality, great software.
• 50:17
  SPEAKER_01: How much do you think AI has changed of everything, including the fundamentals?
• 50:23
  SPEAKER_00: This is something that I thought too.
• 50:26
  SPEAKER_00: I thought, right, AI has changed everything.
• 50:30
  SPEAKER_00: I'm going to throw the baby out with the bathwater, right?
• 50:33
  SPEAKER_00: I think we just need to look at everything in a new way.
• 50:36
  SPEAKER_00: I started doing that a lot.
• 50:37
  SPEAKER_00: I was especially looking at like spec-driven developments, you know, which is...
• 50:42
  SPEAKER_00: I have sort of mixed feelings towards, I think it's a strange term, it encompasses too much.
• 50:46
  SPEAKER_00: And I thought, okay, right, maybe English is the hot new programming language, right?
• 50:51
  SPEAKER_01: Which went viral at some point when under Capra repulsed it.
• 50:54
  SPEAKER_01: Exactly.
• 50:54
  SPEAKER_00: Like,
• 50:55
  SPEAKER_00: maybe I can just write a spec and that specification is going to be persistent,
• 51:00
  SPEAKER_00: it's going to be something I can edit and just get the agent to change it
• 51:03
  SPEAKER_00: as it goes.
• 51:03
  SPEAKER_00: And as I experimented with it,
• 51:05
  SPEAKER_00: I tried it a lot and I was just getting worse results than if I'd coded it by hand.
• 51:09
  SPEAKER_00: And it wasn't getting better as well.
• 51:11
  SPEAKER_00: And I noticed that every time I would sort of run this loop of change the spec,
• 51:16
  SPEAKER_00: see the code change, the code would get worse.
• 51:19
  SPEAKER_00: He's not supposed to look at the code, of course, but I looked at the code and it was garbage.
• 51:23
  SPEAKER_00: And I thought, how is the agent going to perform well in here?
• 51:25
  SPEAKER_00: How is it going to work?
• 51:27
  SPEAKER_00: Because the feedback loops are so important to the agent.
• 51:29
  SPEAKER_00: If you have a bad test suite,
• 51:31
  SPEAKER_00: the agent is going to get bad signal from it, just like a human would.
• 51:35
  SPEAKER_00: And I thought, how do I improve the test suite?
• 51:36
  SPEAKER_00: How do I get this setup not churning out garbage every time?
• 51:42
  SPEAKER_00: And I opened a book that I had on my shelf that I think was still wrapped in plastic
• 51:48
  SPEAKER_00: the first time I took it out, which was The Pragmatic Programmer.
• 51:52
  SPEAKER_00: Which is, everyone told me to read it.
• 51:55
  SPEAKER_00: Everyone said, yeah, this is the best book ever.
• 51:57
  SPEAKER_00: You just got it.
• 51:58
  SPEAKER_00: And I bought it and I didn't read it for some reason.
• 52:00
  SPEAKER_00: And I opened it and it had a whole chapter, a whole section on software entropy.
• 52:04
  SPEAKER_00: And software entropy is the concept that,
• 52:06
  SPEAKER_00: you know, entropy is the idea that things I started looking more into that book.
• 52:22
  SPEAKER_00: So many smart ideas.
• 52:37
  SPEAKER_00: Yeah.
• 52:53
  SPEAKER_01: I guess the idea of the tracer bullet is like a tracer bullet that leaves a mark,
• 52:58
  SPEAKER_01: you implement a path that works like an important piece of a software instead
• 53:03
  SPEAKER_01: of like building a database layer and the application layer
• 53:06
  SPEAKER_01: and I don't know whatever layer like building That was the problem
• 53:13
  SPEAKER_00: I was seeing
• 53:14
  SPEAKER_00: with agents.
• 53:15
  SPEAKER_00: You would get it to build a piece of software,
• 53:17
  SPEAKER_00: even with RALF loops,
• 53:18
  SPEAKER_00: and it would build the entire database,
• 53:21
  SPEAKER_00: and then it would build the entire application layer on top of it.
• 53:23
  SPEAKER_00: And so another concept is vertical slices, right?
• 53:24
  SPEAKER_00: Instead of these horizontal slices across these different deployable units,
• 53:27
  SPEAKER_00: you have a vertical slice where it gets feedback on what it's doing straight away
• 53:31
  SPEAKER_00: and builds out from there.
• 53:53
  SPEAKER_00: And so I just started using these phrases in my prompts when I was talking to the agent.
• 53:58
  SPEAKER_00: And I started noticing that it was saying those phrases back to me.
• 54:02
  SPEAKER_00: It was repeating them back to me.
• 54:04
  SPEAKER_00: It was saying, OK, I'll turn this into a tracer bullet.
• 54:07
  SPEAKER_00: Because this is a tracer bullet, I'll do this.
• 54:10
  SPEAKER_00: It was using the words that I was using in its own reasoning traces.
• 54:13
  SPEAKER_00: And so this is what I call a leading word,
• 54:16
  SPEAKER_00: a light word, let's say, which is a sort of fancy literary term.
• 54:19
  SPEAKER_00: Where you lead the agent just with a simple phrase that you repeat a couple of times
• 54:24
  SPEAKER_00: in the skill or the prompt to change its behavior.
• 54:28
  SPEAKER_00: And so Tracer Bullets was a fantastic one.
• 54:30
  SPEAKER_00: And I just started diving into different books,
• 54:32
  SPEAKER_00: all the books I could find to try to mine them for leading words.
• 54:36
  SPEAKER_00: And another one was John Osterhout's book,
• 54:38
  SPEAKER_00: Philosophy of Software Design,
• 54:39
  SPEAKER_00: where I picked up tons of great stuff like deep modules,
• 54:42
  SPEAKER_00: which is a massive one for me.
• 54:44
  SPEAKER_01: It's interesting to consider if these agents have obviously been trained
• 54:48
  SPEAKER_01: on those books, just still available for print.
• 54:50
  SPEAKER_01: And of course, there's arguments of like what they're doing with those books and whatnot.
• 54:54
  SPEAKER_01: But if it's in their training data and the agents,
• 54:58
  SPEAKER_01: as they're trained, they Connect all these different concepts.
• 55:01
  SPEAKER_01: And yeah, these, I guess, leading words could invoke those concepts.
• 55:06
  SPEAKER_01: And I wonder if this is much different to when on a topic you talk with
• 55:11
  SPEAKER_01: a professional and you're trying to describe as an amateur what you Definitely.
• 55:14
  SPEAKER_01: That idea led
• 55:34
  SPEAKER_00: Because obviously you've got these leading words that are in the agent's priors, right?
• 55:37
  SPEAKER_00: Like tracer bullets, all that stuff.
• 55:39
  SPEAKER_00: What about describing my application?
• 55:41
  SPEAKER_00: What about describing my code?
• 55:43
  SPEAKER_00: How do I get the agent to... Because the agents are just awfully verbose, right?
• 55:49
  SPEAKER_00: Especially Opus 5, for some reason, people really go after that model for being verbose.
• 55:54
  SPEAKER_00: And it really is.
• 55:55
  SPEAKER_00: And I thought, how do I get it to be less verbose?
• 55:57
  SPEAKER_00: How do we start talking a common language between me and the...
• 56:01
  SPEAKER_00: Agents, this communication barrier again, and it led me to DDD, domain-driven design.
• 56:07
  SPEAKER_00: Eric Evans' incredible book where he talks about ubiquitous language,
• 56:10
  SPEAKER_00: a language, again, really deep in the agent's prize.
• 56:13
  SPEAKER_00: It understands it really well.
• 56:14
  SPEAKER_00: I sort of started...
• 56:16
  SPEAKER_00: Toying with the idea of maybe changing grill me a little bit,
• 56:19
  SPEAKER_00: because grill me is very simple skill.
• 56:21
  SPEAKER_00: But what if we,
• 56:23
  SPEAKER_00: while we were ideating,
• 56:24
  SPEAKER_00: while we were thinking about the application we were going to build,
• 56:27
  SPEAKER_00: what if we were also building a domain language?
• 56:30
  SPEAKER_00: What if we were also deciding on the right terms to use?
• 56:33
  SPEAKER_00: And this turned into a skill called Grill with Docs,
• 56:36
  SPEAKER_00: which is a terribly named skill,
• 56:37
  SPEAKER_00: but it essentially creates this domain language as you go.
• 56:41
  SPEAKER_00: And if you get the agent to use the domain language,
• 56:45
  SPEAKER_00: the difference is night and day,
• 56:46
  SPEAKER_00: because suddenly you're speaking the same language,
• 56:48
  SPEAKER_00: you're able to describe the things you want to change in so many fewer words.
• 56:53
  SPEAKER_00: Like I had this app that I sort of work on,
• 56:55
  SPEAKER_00: there's this complicated interaction where There are ghost lessons and real lessons.
• 56:59
  SPEAKER_00: And what happens when you turn a ghost lesson that's inside a ghost section,
• 57:04
  SPEAKER_00: inside a ghost course, into a real lesson?
• 57:06
  SPEAKER_00: That means the ghost section needs to become real and the ghost course needs to become real.
• 57:11
  SPEAKER_00: How do you explain that?
• 57:12
  SPEAKER_00: Well, that's the materialization cascade, right?
• 57:15
  SPEAKER_01: And you came up with these...
• 57:16
  SPEAKER_00: Yeah, with the agent, right?
• 57:18
  SPEAKER_00: The agent is actually really good at coming up with these terms.
• 57:20
  SPEAKER_00: And so I have a domain modeling skill.
• 57:23
  SPEAKER_00: And we talk about jargon, but really it's domain language.
• 57:26
  SPEAKER_00: And if you can integrate that and integrate that not only with the way you talk
• 57:30
  SPEAKER_00: about the app, but the app code itself.
• 57:31
  SPEAKER_00: It's just gorgeous.
• 57:33
  SPEAKER_00: So that's something I've been really integrating with every part of my setup is DDD.
• 57:50
  SPEAKER_01: But this is so interesting because in an effort
• 57:53
  SPEAKER_01: to make these agents work more efficiently or do workflows that just mean
• 57:58
  SPEAKER_01: that you can produce better software with fewer mistakes and these things,
• 58:02
  SPEAKER_01: you start to go back in time,
• 58:04
  SPEAKER_01: found this book that is now, I think, what is it, like 20, 30, 40 years old?
• 58:10
  SPEAKER_01: And you're even still going back and finding gems from,
• 58:13
  SPEAKER_01: I'm sure at some point you'll get to the Mythical Man month.
• 58:15
  SPEAKER_00: Yeah, I've got it already.
• 58:16
  SPEAKER_00: Absolutely.
• 58:17
  SPEAKER_01: Which is now more than 50 years.
• 58:19
  SPEAKER_01: And you're trying to find the right words to describe things,
• 58:22
  SPEAKER_01: which is very curious,
• 58:23
  SPEAKER_01: because when I talk with Kent Beck on how they used to program with Ward Cunningham
• 58:29
  SPEAKER_01: as they were coming up with the content,
• 58:30
  SPEAKER_01: Right now, this feels we're going back to the fundamentals.
• 58:45
  SPEAKER_01: How that people have been asking themselves and every now and then people write it
• 58:49
  SPEAKER_01: in books and it kind of spreads as wisdom.
• 58:52
  SPEAKER_01: And now we're back to where we started, which is what you're trying to teach is the wisdom part.
• 58:56
  SPEAKER_00: It's wild, right?
• 58:57
  SPEAKER_00: Like because AI is so different to humans, you need to optimize it.
• 59:04
  SPEAKER_00: Imagine you essentially had a human who...
• 59:07
  SPEAKER_00: Wakes up every morning and cannot remember who they are, right?
• 59:10
  SPEAKER_00: The guy from Memento, you know?
• 59:12
  SPEAKER_00: This is Memento-driven developments, right?
• 59:14
  SPEAKER_00: We are trying to optimize our codebases for new starters.
• 59:17
  SPEAKER_00: So we're trying to have the most healthy codebase that we've ever had.
• 59:21
  SPEAKER_00: Because if you... A human can work around a bad codebase.
• 59:25
  SPEAKER_00: They just develop memory.
• 59:26
  SPEAKER_00: They just slam their head against the wall again and again and again until they've got there.
• 59:30
  SPEAKER_00: But...
• 59:32
  SPEAKER_00: An agent can't do that.
• 59:33
  SPEAKER_00: It starts fresh every single session.
• 59:35
  SPEAKER_00: And so you need to optimize your code base for that person.
• 59:38
  SPEAKER_00: That leads you down into really interesting paths.
• 59:41
  SPEAKER_00: And it turns out that software fundamentals have been saying we've been trying to do
• 59:45
  SPEAKER_00: that for the entire time, right?
• 59:47
  SPEAKER_00: I am fully like, I don't know, Eric Evans pilled.
• 59:50
  SPEAKER_00: I'm fully like software fundamentals pilled.
• 59:53
  SPEAKER_00: We are sort of changing the rules a little bit,
• 59:55
  SPEAKER_00: but maybe we're just emphasizing rules that we knew we were supposed to do,
• 59:59
  SPEAKER_00: but maybe we didn't.
• 1:00:01
  SPEAKER_00: And I find that really fascinating and it's definitely a lot of fun.
• 1:00:04
  SPEAKER_01: Now, okay,
• 1:00:05
  SPEAKER_01: like I think it's easy enough to follow with this train
• 1:00:08
  SPEAKER_01: of thought why fundamentals matter, but which fundamentals?
• 1:00:12
  SPEAKER_01: And if I'm an engineer,
• 1:00:12
  SPEAKER_01: especially maybe someone who has been just kind of like heads down coding,
• 1:00:16
  SPEAKER_01: more tactical coding,
• 1:00:18
  SPEAKER_01: how do I go about and find those fundamentals that matter and go back
• 1:00:22
  SPEAKER_01: to what have you found work?
• 1:00:24
  SPEAKER_00: This is a really tough question, right?
• 1:00:25
  SPEAKER_00: It's really tough because...
• 1:00:27
  SPEAKER_00: Strategic programming has always been really hard to learn.
• 1:00:31
  SPEAKER_00: The reason for that is that the feedback loop on it is really long.
• 1:00:35
  SPEAKER_00: You would often find people who quit their jobs after six months,
• 1:00:40
  SPEAKER_00: their strategic mistakes never catch up with them, right?
• 1:00:43
  SPEAKER_00: Maybe that strategic mistake takes nine months to come back at you.
• 1:00:46
  SPEAKER_00: I think of learning strategic programming as kind of like you've got
• 1:00:49
  SPEAKER_00: a huge mixing desk in front of you with loads of these different sliders.
• 1:00:53
  SPEAKER_00: Maybe one of those sliders is like the amount of deployable units that you have.
• 1:00:56
  SPEAKER_00: You turn it up, you've got more microservices, right?
• 1:00:59
  SPEAKER_00: You turn it down, you've got a monolith.
• 1:01:01
  SPEAKER_00: How do you make that decision?
• 1:01:02
  SPEAKER_00: Where do you put that slider?
• 1:01:03
  SPEAKER_00: Because it's kind of like you're mastering something,
• 1:01:05
  SPEAKER_00: you're mixing some music,
• 1:01:07
  SPEAKER_00: but you can't hear what's wrong until nine months later, right?
• 1:01:10
  SPEAKER_00: Until the mistakes come and get you.
• 1:01:12
  SPEAKER_00: So I think the only thing...
• 1:01:14
  SPEAKER_00: And so what you need to be thinking about is that your code is the environment
• 1:01:17
  SPEAKER_00: the agent operates in.
• 1:01:18
  SPEAKER_00: And you should always be thinking about improving that environment,
• 1:01:20
  SPEAKER_00: thinking about how to do it better.
• 1:01:37
  SPEAKER_00: And obviously that requires a bit of tactical knowledge, right?
• 1:01:40
  SPEAKER_00: You need to understand what code is and how it fits together and what
• 1:01:43
  SPEAKER_00: the memory constraints are and all that stuff.
• 1:01:45
  SPEAKER_00: But in order to get better at strategic programming,
• 1:01:48
  SPEAKER_00: you just need to be thinking on that level all the time.
• 1:01:50
  SPEAKER_00: And I would say reading these books as well,
• 1:01:52
  SPEAKER_00: because just having the language to explain that and understanding the difference
• 1:01:56
  SPEAKER_00: between applying strategic techniques and not is the whole game.
• 1:02:02
  SPEAKER_01: I mean, up to pre-AI for senior developers,
• 1:02:07
  SPEAKER_01: for senior engineers,
• 1:02:08
  SPEAKER_01: staff engineers,
• 1:02:08
  SPEAKER_01: they were the people who often you didn't see a senior engineer under five years
• 1:02:13
  SPEAKER_01: of experience because you typically needed,
• 1:02:14
  SPEAKER_01: even in a fast-paced environment, you needed that much time.
• 1:02:17
  SPEAKER_01: It's time to get the feedback loops, to make the mistakes, make your own mistakes.
• 1:02:21
  SPEAKER_01: And by the time people got to Staff Engineer,
• 1:02:23
  SPEAKER_01: oftentimes around 10 plus years of experience,
• 1:02:25
  SPEAKER_01: some people did it earlier, but they often just had paddle skulls all over them.
• 1:02:29
  SPEAKER_01: And they would,
• 1:02:30
  SPEAKER_01: you know,
• 1:02:31
  SPEAKER_01: someone would start a new project and they would go in and they would just make
• 1:02:33
  SPEAKER_01: a tweak and it wouldn't be clear why.
• 1:02:35
  SPEAKER_01: And they were like,
• 1:02:36
  SPEAKER_01: trust me on this, we're avoiding disaster and production or on-call or whatnot.
• 1:02:42
  SPEAKER_01: But all of this came through lived experience.
• 1:02:45
  SPEAKER_01: Now, AI speeds things up.
• 1:02:47
  SPEAKER_01: It also makes it easier to fix mistakes.
• 1:02:50
  SPEAKER_01: So I'm wondering how this might change.
• 1:02:52
  SPEAKER_01: Like on one end, I can see how it could just speed up experience.
• 1:02:56
  SPEAKER_01: Like you can in a year.
• 1:02:57
  SPEAKER_01: Some people,
• 1:02:58
  SPEAKER_01: some teams will ship more projects than they have in four years or about the same as
• 1:03:02
  SPEAKER_01: in, let's say, three or four years before.
• 1:03:03
  SPEAKER_01: So you get a lot more experience.
• 1:03:05
  SPEAKER_01: But I wonder if sometimes the mistakes that you make are just not as serious
• 1:03:09
  SPEAKER_01: because you can fix them quickly.
• 1:03:11
  SPEAKER_01: And now I wonder if the learning If you're a company right now,
• 1:03:13
  SPEAKER_01: and you want to train the next junior developer, like...
• 1:03:34
  SPEAKER_00: Because this strategic programming knowledge is so valuable now,
• 1:03:40
  SPEAKER_00: because you can use it at such higher leverage,
• 1:03:43
  SPEAKER_00: are you really going to employ someone without it?
• 1:03:46
  SPEAKER_00: Why would you?
• 1:03:47
  SPEAKER_00: I was asking,
• 1:03:48
  SPEAKER_00: I did an interview with Uncle Bob the other day,
• 1:03:50
  SPEAKER_00: and his recommendation was,
• 1:03:52
  SPEAKER_00: okay, you just hire someone and you treat them as an agent for a while.
• 1:03:56
  SPEAKER_00: You just delegate to them,
• 1:03:56
  SPEAKER_00: you keep them in that tactical mindset for a while
• 1:03:58
  SPEAKER_00: until their mistakes start coming up at you.
• 1:04:01
  SPEAKER_00: But that's such an enormous waste of money for people, right?
• 1:04:04
  SPEAKER_00: Like when software engineering,
• 1:04:06
  SPEAKER_00: when the tactical stuff has gone below minimum wage in a lot of countries.
• 1:04:10
  SPEAKER_00: So I don't know is the answer.
• 1:04:13
  SPEAKER_00: I only know that the strategic stuff,
• 1:04:15
  SPEAKER_00: the understanding of the code,
• 1:04:17
  SPEAKER_00: the understanding of the long view has gotten more valuable than it's ever been,
• 1:04:22
  SPEAKER_00: right?
• 1:04:23
  SPEAKER_00: Because you can just get so much leverage out of it.
• 1:04:26
  SPEAKER_01: I asked about interesting things I'd like to know from you, and this is very related to this.
• 1:04:31
  SPEAKER_01: This person asked, like,
• 1:04:32
  SPEAKER_01: how do you convince non-engineering stakeholders that investing
• 1:04:35
  SPEAKER_01: in software fundamentals are important,
• 1:04:36
  SPEAKER_01: even if they might reduce the speed and productivity on paper?
• 1:04:39
  SPEAKER_01: I think the question here is,
• 1:04:41
  SPEAKER_01: if some people advocate, like, look,
• 1:04:43
  SPEAKER_01: we do want to get the fundamentals right,
• 1:04:45
  SPEAKER_01: which means we want to take it a bit slower,
• 1:04:47
  SPEAKER_01: think about their decisions,
• 1:04:48
  SPEAKER_01: maybe educate ourselves as well, as opposed to just like churning it out.
• 1:04:53
  SPEAKER_00: I mean...
• 1:04:54
  SPEAKER_00: You could have asked the same question 10 years ago, right?
• 1:04:55
  SPEAKER_00: And like, it would have still been relevant.
• 1:04:57
  SPEAKER_01: You know what I mean?
• 1:04:58
  SPEAKER_01: Except we would have asked about paying off tech debt.
• 1:05:02
  SPEAKER_00: Exactly.
• 1:05:03
  SPEAKER_00: And it's the same thing, right?
• 1:05:04
  SPEAKER_00: Like, we have been having the same conversation,
• 1:05:06
  SPEAKER_00: which is quite satisfying to me because,
• 1:05:08
  SPEAKER_00: I mean, you need some sort of metric for, like, figuring this out.
• 1:05:11
  SPEAKER_00: And it's a little easier to figure this out because...
• 1:05:15
  SPEAKER_00: Agents allow you to move faster.
• 1:05:17
  SPEAKER_00: And the first step to this is getting observability in your organization
• 1:05:21
  SPEAKER_00: over every single agent on what it's doing and what its success and failure rate is.
• 1:05:25
  SPEAKER_00: We've never been able to have that with developers before.
• 1:05:29
  SPEAKER_00: It's kind of invasive for developers.
• 1:05:31
  SPEAKER_00: But for agents, it's okay.
• 1:05:34
  SPEAKER_00: It's okay, right?
• 1:05:34
  SPEAKER_00: We are paying for this service, right?
• 1:05:36
  SPEAKER_00: We need to understand how well we're optimizing for it.
• 1:05:38
  SPEAKER_00: The first step there is actually getting a harness
• 1:05:41
  SPEAKER_00: or observability around your agents,
• 1:05:43
  SPEAKER_00: the entire organization to work out what's working and not.
• 1:05:46
  SPEAKER_00: And you probably need someone whose job it is or part of their job is to look at
• 1:05:50
  SPEAKER_00: that data and figure out what we're doing.
• 1:05:52
  SPEAKER_00: Maybe some repos in your organization have better success rates than others.
• 1:05:56
  SPEAKER_00: And so you take the lessons that are in there and you...
• 1:05:59
  SPEAKER_00: Pass them out.
• 1:06:00
  SPEAKER_00: I also think that most organizations need to gather around a common set of skills.
• 1:06:04
  SPEAKER_00: You need a common software workflow process so that everyone can contribute back
• 1:06:08
  SPEAKER_00: to it, so that you can experiment with things, you can A-B test things.
• 1:06:12
  SPEAKER_00: You know,
• 1:06:12
  SPEAKER_00: you can have one team doing one set of stuff and one team doing another set of stuff
• 1:06:16
  SPEAKER_00: and then you ask them afterwards.
• 1:06:18
  SPEAKER_00: And so...
• 1:06:19
  SPEAKER_00: Everyone working with agents in any kind of organization needs this experimental mindset.
• 1:06:24
  SPEAKER_00: You need to be thinking, how do we get more juice out of these tokens that we're spending?
• 1:06:29
  SPEAKER_00: And observability is the first step there.
• 1:06:32
  SPEAKER_01: Yeah, and I also wonder if there's a human feedback loop in the sense that,
• 1:06:35
  SPEAKER_01: I mean, just talk to your colleagues.
• 1:06:39
  SPEAKER_01: Like, you know, we do have rituals, team meetings, company-wide meetings for a reason.
• 1:06:44
  SPEAKER_01: Like, there, share, here's what's working for me.
• 1:06:47
  SPEAKER_01: Here's where it didn't work.
• 1:06:48
  SPEAKER_01: Here's what I'm learning.
• 1:06:49
  SPEAKER_01: In the end,
• 1:06:50
  SPEAKER_01: we are in charge of setting up the rules,
• 1:06:53
  SPEAKER_01: deciding how we use them,
• 1:06:54
  SPEAKER_01: where we use them,
• 1:06:55
  SPEAKER_01: where we don't use them, and where we say, no, humans need to take 100%.
• 1:06:59
  SPEAKER_01: We're not even getting AI involved, which again will be different everywhere.
• 1:07:03
  SPEAKER_00: And it's not only that.
• 1:07:05
  SPEAKER_00: A lot of this stuff now you don't need to be human in the loop for, right?
• 1:07:09
  SPEAKER_00: You don't actually need to delegate that much time in order to build up a better codebase.
• 1:07:14
  SPEAKER_00: I have loops
• 1:07:15
  SPEAKER_00: that essentially every morning it will run my improve codebase architecture skill
• 1:07:20
  SPEAKER_00: and give me a proposal for something that I could improve in the codebase.
• 1:07:23
  SPEAKER_00: And then I can just press a button.
• 1:07:25
  SPEAKER_00: I can say, okay, turn that into tickets and then let's ship that.
• 1:07:28
  SPEAKER_00: That is pretty easy to do and it's pretty easy to stream that in with other work.
• 1:07:33
  SPEAKER_00: And so...
• 1:07:34
  SPEAKER_00: I think that,
• 1:07:35
  SPEAKER_00: I don't know whether you need like 20% of your time focusing on the factory
• 1:07:39
  SPEAKER_00: that builds your software as well as the software,
• 1:07:41
  SPEAKER_00: because I feel like that's a massive,
• 1:07:43
  SPEAKER_00: incredible investment into your future leverage and not only your leverage
• 1:07:47
  SPEAKER_00: with your work,
• 1:07:48
  SPEAKER_00: but also your team's leverage and understanding and getting better at those skills.
• 1:07:52
  SPEAKER_00: But of course,
• 1:07:52
  SPEAKER_00: you need results and you might need to hide that work for a bit
• 1:07:55
  SPEAKER_00: before you actually reveal it to this is what we've been doing.
• 1:07:58
  SPEAKER_01: Well, and this is down to your environment, but yeah.
• 1:08:01
  SPEAKER_01: And no one's going to be mad at you if you come back saying,
• 1:08:03
  SPEAKER_01: oh, by the way, guys, I also did this.
• 1:08:05
  SPEAKER_01: Yeah, exactly.
• 1:08:06
  SPEAKER_01: I wanted to ask you about your specific kind of how you use tools.
• 1:08:11
  SPEAKER_01: First one is coding agents, local or in the cloud.
• 1:08:14
  SPEAKER_01: And you recently posted a pretty provocative tweet, which I'll quote you.
• 1:08:19
  SPEAKER_01: I'm moving away from my local dev setup.
• 1:08:20
  SPEAKER_01: Makes zero sense to me now.
• 1:08:22
  SPEAKER_00: A lot of people ask me, how do you make your skills collaborative?
• 1:08:26
  SPEAKER_00: How do you have a collaborative grilling session?
• 1:08:28
  SPEAKER_00: And the answer to that is that you need more than just your terminal and you.
• 1:08:33
  SPEAKER_00: Right.
• 1:08:34
  SPEAKER_00: We're in a phase now where every dev has like 100 terminals available to them.
• 1:08:40
  SPEAKER_00: And that seems crazy.
• 1:08:41
  SPEAKER_00: It feels like you need those 100 terminals available to your entire organization.
• 1:08:46
  SPEAKER_00: You need to be able to collaborate in a shared space.
• 1:08:48
  SPEAKER_00: You need to be able to ask someone,
• 1:08:49
  SPEAKER_00: tag someone in to your grilling session and say, OK, do this.
• 1:08:54
  SPEAKER_00: It makes a lot of sense for me to have a lot of those interactions in the place
• 1:08:57
  SPEAKER_00: where you already work in Slack or in Discord or in Teams, whatever, or linear.
• 1:09:02
  SPEAKER_00: And that is really the thing that's driving me to explore this.
• 1:09:06
  SPEAKER_00: I don't work with a team particularly, but I understand the value of that.
• 1:09:09
  SPEAKER_00: And I've been trying to build that into my flows.
• 1:09:12
  SPEAKER_00: So on the train over here,
• 1:09:13
  SPEAKER_00: I'm in Discord chatting to my Hetzner box,
• 1:09:16
  SPEAKER_00: you know,
• 1:09:16
  SPEAKER_00: building stuff for my course or fixing bugs that students are coming across.
• 1:09:20
  SPEAKER_00: So I can see less value now in just doing things locally when I have this setup
• 1:09:25
  SPEAKER_00: that I can port forward into,
• 1:09:27
  SPEAKER_00: let's say, and see the dev server as it's making changes and I don't know.
• 1:09:33
  SPEAKER_00: It just feels like it makes way more sense to me than having a very,
• 1:09:37
  SPEAKER_00: very expensive laptop that can do this stuff.
• 1:09:39
  SPEAKER_00: It feels like wasted compute.
• 1:09:40
  SPEAKER_00: And especially because on that remote box, I can set up schedules.
• 1:09:44
  SPEAKER_00: I know the box is always going to be on.
• 1:09:46
  SPEAKER_00: I have like a morning stand up with my agent where it schedules my day for me
• 1:09:50
  SPEAKER_00: and it understands all of my Discord chats and all that.
• 1:09:54
  SPEAKER_00: Yeah, having that remote feels like it makes just so much more sense for me.
• 1:09:57
  SPEAKER_00: And the only thing I do locally now is debugging issues with the remote bot.
• 1:10:01
  SPEAKER_01: Yeah, I think I wonder if there's a question of how easy it is
• 1:10:05
  SPEAKER_01: to replicate some pretty complicated local setups in the cloud.
• 1:10:08
  SPEAKER_01: But once that becomes possible, it's probably a matter of when, not an if.
• 1:10:12
  SPEAKER_00: Yeah.
• 1:10:13
  SPEAKER_00: And if anything, people are having this similar issue with local setups, right?
• 1:10:17
  SPEAKER_00: With just a thousand Git work trees just spamming their hard drive.
• 1:10:21
  SPEAKER_00: And with how do I have a work tree that I've got to run like five Docker containers
• 1:10:26
  SPEAKER_00: in order to get my local dev setup?
• 1:10:28
  SPEAKER_00: And by the way,
• 1:10:31
  SPEAKER_01: we're seeing that companies like Ramp, Stripe, Uber,
• 1:10:37
  SPEAKER_01: that have platform teams that manage to take a local devs full setup
• 1:10:42
  SPEAKER_01: and put it You can now invoke with an ad, Slack, or a website.
• 1:10:47
  SPEAKER_01: They're seeing people use these agents far more,
• 1:10:50
  SPEAKER_01: except for front-end work, which you still want to have that feedback loop.
• 1:10:55
  SPEAKER_01: There are a few exceptions where you really want to have that local dev setup
• 1:11:01
  SPEAKER_01: for latency or whatnot,
• 1:11:03
  SPEAKER_01: but they're also seeing 70-80% of devs are just voluntarily going for the cloud.
• 1:11:08
  SPEAKER_00: Yeah.
• 1:11:08
  SPEAKER_00: I mean, I think you can just tunnel through and just get the,
• 1:11:12
  SPEAKER_00: if it's running a dev server and you just have that appearing on your local machine,
• 1:11:16
  SPEAKER_00: how is that different from having it locally, right?
• 1:11:19
  SPEAKER_00: I don't know.
• 1:11:20
  SPEAKER_00: I think I've not experimented with that,
• 1:11:21
  SPEAKER_00: but that's when I talked about that and said,
• 1:11:24
  SPEAKER_00: oh, maybe front end is a good exception.
• 1:11:25
  SPEAKER_00: That was the immediate response that I got.
• 1:11:27
  SPEAKER_00: And it makes sense to me.
• 1:11:29
  SPEAKER_01: I want to ask you about planning and requirements.
• 1:11:31
  SPEAKER_01: You're a big believer in grill me and planning upfront or getting the plan and
• 1:11:35
  SPEAKER_01: then having the agent work.
• 1:11:36
  SPEAKER_01: But there's a devil's advocate here.
• 1:11:38
  SPEAKER_01: Agents are so fast at implementing.
• 1:11:40
  SPEAKER_01: You can actually even have like several, like a few agents implement different architectures.
• 1:11:45
  SPEAKER_01: What about the approach of like, well, they're fast at implementing.
• 1:11:47
  SPEAKER_01: So I might not need to do as much upfront planning.
• 1:11:51
  SPEAKER_01: I can just course correct as I go.
• 1:11:53
  SPEAKER_00: It depends what type of work you're doing.
• 1:11:55
  SPEAKER_00: Right.
• 1:11:55
  SPEAKER_00: Because I believe that you shouldn't be using Grill Me for everything.
• 1:11:59
  SPEAKER_00: Essentially,
• 1:11:59
  SPEAKER_00: you need Grill Me for pieces of work where the actual thing being done is going
• 1:12:05
  SPEAKER_00: to be quite large and hard to row back from.
• 1:12:08
  SPEAKER_00: If you feel like, OK,
• 1:12:09
  SPEAKER_00: this feature,
• 1:12:10
  SPEAKER_00: maybe it's a whole new page,
• 1:12:11
  SPEAKER_00: maybe it's a big feature,
• 1:12:12
  SPEAKER_00: this code is You think if the agent gets it wrong,
• 1:12:17
  SPEAKER_00: then the wrong code is going to be in its context window influencing everything
• 1:12:20
  SPEAKER_00: that comes afterwards.
• 1:12:21
  SPEAKER_00: And actually going back and editing the stuff afterwards and doing the alignment
• 1:12:25
  SPEAKER_00: after the fact is going
• 1:12:26
  SPEAKER_01: to be expensive.
• 1:12:27
  SPEAKER_00: Whereas for those cases,
• 1:12:29
  SPEAKER_00: it makes sense to align first to answer all of
• 1:12:31
  SPEAKER_00: the tricky questions like your JSON cookie or whatever,
• 1:12:35
  SPEAKER_00: your authentication token first, and then do it.
• 1:12:38
  SPEAKER_00: But for some cases,
• 1:12:39
  SPEAKER_00: like simple bug fixes or just like move this button three pixels to the left,
• 1:12:43
  SPEAKER_00: It's obvious that you don't need to align before that.
• 1:12:46
  SPEAKER_00: You can see the thing if it's just like a five line change or something.
• 1:12:50
  SPEAKER_00: You can align afterwards.
• 1:12:52
  SPEAKER_00: And so that's how I think of it is that where you can,
• 1:12:56
  SPEAKER_00: you should shift right as much as possible.
• 1:12:59
  SPEAKER_00: And actually,
• 1:12:59
  SPEAKER_00: there is actually certain features that I have a little in my video editor,
• 1:13:03
  SPEAKER_00: I have a button that I can send feedback to it.
• 1:13:07
  SPEAKER_00: And I often use this for very simple tasks where I send the feedback,
• 1:13:10
  SPEAKER_00: it goes into a GitHub issue.
• 1:13:12
  SPEAKER_00: This immediately gets picked up by An implementer agent gets just worked on immediately.
• 1:13:16
  SPEAKER_00: Then a code review agent comes in and reviews the code.
• 1:13:19
  SPEAKER_00: And then at the end, I get to see this actual thing being fixed and I can do my alignment then.
• 1:13:25
  SPEAKER_00: And that's worked really well for things that are very easy to specify,
• 1:13:28
  SPEAKER_00: things that I don't need to grill on.
• 1:13:30
  SPEAKER_00: So those are the choices you've got.
• 1:13:32
  SPEAKER_00: Is it a small enough thing that I can align afterwards?
• 1:13:35
  SPEAKER_00: Then don't use grill me.
• 1:13:36
  SPEAKER_00: Does it fit into a single session?
• 1:13:38
  SPEAKER_00: Then use grill me.
• 1:13:39
  SPEAKER_00: Does it span multiple sessions?
• 1:13:41
  SPEAKER_00: I need to align over the entire thing, then use Wayfinder.
• 1:13:44
  SPEAKER_01: Interesting because this is not all that different to
• 1:13:47
  SPEAKER_01: where some tech companies landed years before,
• 1:13:49
  SPEAKER_01: which is on the PRD, the product reference document.
• 1:13:52
  SPEAKER_01: If it's something trivial, just build it.
• 1:13:54
  SPEAKER_01: If it requires the team,
• 1:13:57
  SPEAKER_01: like it's a team level scope, I mean,
• 1:13:58
  SPEAKER_01: write a PRD,
• 1:13:59
  SPEAKER_01: send it out to the team, maybe CC some other teams, but it's not a blocker.
• 1:14:04
  SPEAKER_01: And if it's something bigger, then it's a blocker.
• 1:14:07
  SPEAKER_01: Like we need to wait for feedback.
• 1:14:09
  SPEAKER_01: Basically the way we would say it is like, look,
• 1:14:11
  SPEAKER_01: If it's like a one-month project,
• 1:14:13
  SPEAKER_01: like spend two days,
• 1:14:14
  SPEAKER_01: like it's not a bad thing to spend like one or two days planning it
• 1:14:16
  SPEAKER_01: because we're going to save time on it.
• 1:14:18
  SPEAKER_01: But if it's a one-day project, like forget about it.
• 1:14:21
  SPEAKER_01: If it's a one-year project, I mean, what are we doing?
• 1:14:23
  SPEAKER_01: Like it should be a smaller one.
• 1:14:24
  SPEAKER_00: Totally.
• 1:14:25
  SPEAKER_00: And I want to like,
• 1:14:26
  SPEAKER_00: there's a bit of sort of criticism I hear just from outside the room when you say
• 1:14:29
  SPEAKER_00: that, which is that doesn't this sound like waterfall, what we're doing?
• 1:14:33
  SPEAKER_00: When I'm talking about Wayfinder and when I'm doing any kind
• 1:14:36
  SPEAKER_00: of like Building up any kind of spec,
• 1:14:39
  SPEAKER_00: I do a lot of upfront aggressive prototyping before we get there.
• 1:14:44
  SPEAKER_00: That's something that comes up again and again and again.
• 1:14:46
  SPEAKER_00: It's like, this is just waterfall.
• 1:14:47
  SPEAKER_00: What are we doing going back to the 70s?
• 1:14:50
  SPEAKER_00: But agents give you this ability of just churning out slop, right?
• 1:14:54
  SPEAKER_00: And sometimes you can use that to your advantage because a prototype, right,
• 1:14:58
  SPEAKER_00: just getting a sense for what it should look like,
• 1:14:59
  SPEAKER_00: you can build out three or four different versions and just choose your favorite
• 1:15:03
  SPEAKER_00: and iterate on it and just keep churning, churning, churning.
• 1:15:06
  SPEAKER_00: That can be a really powerful setup that we've not really had before, right?
• 1:15:10
  SPEAKER_00: It was always expensive to produce prototypes.
• 1:15:13
  SPEAKER_00: Now it's the cheapest that it's ever been.
• 1:15:15
  SPEAKER_00: And that's an essential part of writing specs to me.
• 1:15:17
  SPEAKER_00: It's actually producing these prototypes.
• 1:15:18
  SPEAKER_01: Yeah, but also like with the waterfall criticism,
• 1:15:20
  SPEAKER_01: I think Grady Booch might have You told me this as well.
• 1:15:23
  SPEAKER_01: It's like,
• 1:15:24
  SPEAKER_01: don't forget, like,
• 1:15:25
  SPEAKER_01: we should not criticize Waterfall because,
• 1:15:27
  SPEAKER_01: for example,
• 1:15:27
  SPEAKER_01: a lot of big tech,
• 1:15:29
  SPEAKER_01: the largest tech companies from like Amazon,
• 1:15:32
  SPEAKER_01: Microsoft,
• 1:15:33
  SPEAKER_01: Google, Meta, you name it, they are kind of doing MIDI Waterfall, like pre-AMS.
• 1:15:37
  SPEAKER_01: The problem was never this with Waterfall.
• 1:15:39
  SPEAKER_01: The problem with Waterfall was the planning was literally
• 1:15:52
  SPEAKER_01: Taking like a year, like one year, and then the implementation taking three years.
• 1:15:57
  SPEAKER_01: And by the time it was ready, four years later, it's not what we wanted.
• 1:16:01
  SPEAKER_01: And that was the problem.
• 1:16:02
  SPEAKER_01: He was like,
• 1:16:02
  SPEAKER_01: the problem is not like having like a one or two month project or a one week project
• 1:16:06
  SPEAKER_01: with a Waterfall.
• 1:16:08
  SPEAKER_01: The problem was always this.
• 1:16:09
  SPEAKER_01: We're talking years.
• 1:16:10
  SPEAKER_01: And he said that the industry has not seen waterfalls for decades now.
• 1:16:15
  SPEAKER_01: And so here we're using this term,
• 1:16:16
  SPEAKER_01: which is a bit like we're criticizing or many waterfalls were criticized in
• 1:16:20
  SPEAKER_01: that one.
• 1:16:21
  SPEAKER_01: It's actually that's not It's not a bad thing, necessarily.
• 1:16:23
  SPEAKER_00: You see what I mean?
• 1:16:24
  SPEAKER_00: It's kind of a scarecrow that we're punching or something.
• 1:16:27
  SPEAKER_01: Yeah, it's a piñata which stopped existing.
• 1:16:30
  SPEAKER_01: It might exist in some crazy enterprise project that none of us know about
• 1:16:35
  SPEAKER_01: in regulated industries, but I feel even there it's probably caught out of style.
• 1:16:38
  SPEAKER_00: Yeah, I think it's like,
• 1:16:39
  SPEAKER_00: if we're hitting the piñata, I think it's actually a useful thing to have up there.
• 1:16:44
  SPEAKER_00: It's like a useful ghost or useful cautionary tale, right?
• 1:16:50
  SPEAKER_00: Which one fits the agentic setup more closely?
• 1:16:54
  SPEAKER_00: It's going to be agile, right?
• 1:16:55
  SPEAKER_00: Because the cost of labor has gone down so much, we can just make changes very, very quickly.
• 1:17:00
  SPEAKER_00: I don't know.
• 1:17:01
  SPEAKER_00: That feels like the right metaphor to me.
• 1:17:03
  SPEAKER_00: So I don't mind hitting on waterfall, even though no one really does it anymore.
• 1:17:06
  SPEAKER_01: Well, one other thing that just went out of style,
• 1:17:10
  SPEAKER_01: we didn't hate it, but test-driven development, TDD.
• 1:17:13
  SPEAKER_01: What is your take on using them for AgendaX?
• 1:17:15
  SPEAKER_01: When I talked with Ken Beck,
• 1:17:16
  SPEAKER_01: we talked about how this could be a great fit for many reasons,
• 1:17:20
  SPEAKER_01: but I still don't see people really using it.
• 1:17:22
  SPEAKER_01: I see people writing tests,
• 1:17:23
  SPEAKER_01: the agents also like write tests after the fact, which is how most people work.
• 1:17:28
  SPEAKER_01: But I think you've been an advocate for TDD, right?
• 1:17:30
  SPEAKER_00: Yeah, so I have a TDD skill, which I...
• 1:17:33
  SPEAKER_00: Recommend using,
• 1:17:34
  SPEAKER_00: and this is quite timely because I have been thinking about it,
• 1:17:36
  SPEAKER_00: but I haven't really posted about it yet.
• 1:17:38
  SPEAKER_00: TDD optimizes for having a very small working memory.
• 1:17:42
  SPEAKER_00: You write one test and that test is supposed to fail.
• 1:17:46
  SPEAKER_00: And it means that even if you get distracted,
• 1:17:48
  SPEAKER_00: you go for a coffee or something, you go for a long walk.
• 1:17:51
  SPEAKER_00: When you come back,
• 1:17:51
  SPEAKER_00: the test is still failing,
• 1:17:52
  SPEAKER_00: reminding you of where you are in the implementation and guiding you to
• 1:17:55
  SPEAKER_00: the next thing.
• 1:17:56
  SPEAKER_00: Agents don't need that.
• 1:17:58
  SPEAKER_00: The thing that's great about agents is that they have a much larger working memory than humans.
• 1:18:05
  SPEAKER_00: They can actually hold a lot more in their heads than humans can currently,
• 1:18:09
  SPEAKER_00: which is very useful.
• 1:18:10
  SPEAKER_00: But they don't have an infinite working memory.
• 1:18:13
  SPEAKER_00: And TDD, it's sort of aiming at the wrong problem, I think.
• 1:18:18
  SPEAKER_00: But the thing that agents really do need is that they need to have feedback loops.
• 1:18:24
  SPEAKER_00: So they need to see what they're doing and how it's interacting with the environment
• 1:18:29
  SPEAKER_00: of the code.
• 1:18:30
  SPEAKER_00: They need to probe it all of the time.
• 1:18:32
  SPEAKER_00: And having an agent that builds it,
• 1:18:34
  SPEAKER_00: builds the failure first, it's also very hard for an agent to cheat that.
• 1:18:39
  SPEAKER_00: So not only are you forcing the agent to build its own feedback loops,
• 1:18:43
  SPEAKER_00: the agent is providing proof to you that the thing is actually working as it goes.
• 1:18:48
  SPEAKER_00: And even if I'm not using TDD directly,
• 1:18:52
  SPEAKER_00: where it writes the failing test first,
• 1:18:54
  SPEAKER_00: then fixes it,
• 1:18:55
  SPEAKER_00: then refactors,
• 1:18:56
  SPEAKER_00: I will often say,
• 1:18:57
  SPEAKER_00: provide proof that your change does the thing it's purported to do.
• 1:19:01
  SPEAKER_00: Give me TDD evidence, right, that it would fail without this change.
• 1:19:06
  SPEAKER_00: And that's been really good for just improving the feedback loops,
• 1:19:09
  SPEAKER_00: essentially,
• 1:19:10
  SPEAKER_00: because another thing with TDD
• 1:19:12
  SPEAKER_00: that agents get wrong is they will often just write crap tests.
• 1:19:15
  SPEAKER_00: They'll often just write Especially tautological tests where
• 1:19:19
  SPEAKER_00: the test is just asserting the implementation itself.
• 1:19:23
  SPEAKER_00: It's just like a duplicate of it.
• 1:19:25
  SPEAKER_00: It writes a constant and then it says, expect this constant to be this value.
• 1:19:30
  SPEAKER_00: I mean, what's the point in that test?
• 1:19:32
  SPEAKER_00: It's just asserting the implementation.
• 1:19:34
  SPEAKER_00: So yeah, I have a mixed relationship with TDD.
• 1:19:38
  SPEAKER_00: I do still recommend it just because it gives you so much more confidence
• 1:19:42
  SPEAKER_00: in what you're building from a human perspective.
• 1:19:44
  SPEAKER_00: But yeah, I'm starting to see the counter-arguments.
• 1:19:48
  SPEAKER_01: Let's talk about tech depth.
• 1:19:50
  SPEAKER_01: Jared Friedman at Y Combinator wrote a tweet that I'll quote from him.
• 1:19:56
  SPEAKER_01: Technical depth used to be something you just had to live with with
• 1:19:59
  SPEAKER_01: a sufficiently large code base no longer.
• 1:20:01
  SPEAKER_01: And to which you replied, yes, now you can live with it even in a tiny code base.
• 1:20:07
  SPEAKER_00: That's good to read out loud, actually.
• 1:20:08
  SPEAKER_00: You really gave the sense to that one.
• 1:20:13
  SPEAKER_00: Yeah, it's just so easy for agents to produce rubbish, right?
• 1:20:18
  SPEAKER_00: Even really smart,
• 1:20:20
  SPEAKER_00: powerful agents,
• 1:20:21
  SPEAKER_00: because they're unable to think strategically,
• 1:20:24
  SPEAKER_00: they're just focused on what they're doing right now.
• 1:20:26
  SPEAKER_00: It's very easy for them to produce tech debt.
• 1:20:29
  SPEAKER_00: What is tech debt, right?
• 1:20:31
  SPEAKER_00: Tech debt is anything that makes the codebase harder to make modifications to over time.
• 1:20:36
  SPEAKER_00: A good codebase is one that's easy to change,
• 1:20:39
  SPEAKER_00: easy to make a change in that doesn't result in cascading failures, right?
• 1:20:44
  SPEAKER_00: So a codebase with a solid test coverage and a good test suite is
• 1:20:49
  SPEAKER_00: a codebase that's easy to change.
• 1:20:52
  SPEAKER_00: But it's so easy for agents to just make a codebase worse over time.
• 1:20:56
  SPEAKER_00: And it's a really hard problem.
• 1:20:58
  SPEAKER_00: And it's one that you need a strategic mindset to think about.
• 1:21:01
  SPEAKER_00: Because one thing that I found works really well is automated review.
• 1:21:05
  SPEAKER_00: So you have one implementer agent to do the thing,
• 1:21:07
  SPEAKER_00: and then you have another automated review agent that sort
• 1:21:11
  SPEAKER_00: of imposes your coding standards,
• 1:21:13
  SPEAKER_00: that looks for these tautological tests,
• 1:21:15
  SPEAKER_00: improves the quality of the test suite over time.
• 1:21:17
  SPEAKER_00: But then how do you know if the automated review agent is doing a good job?
• 1:21:22
  SPEAKER_00: And so even in tiny code bases, even in one-line changes, the agent can produce crap.
• 1:21:28
  SPEAKER_00: And so I think it's just something we need to live with and something we need to be
• 1:21:32
  SPEAKER_00: in a constant battle against.
• 1:21:34
  SPEAKER_01: No, it's also not a bad thing.
• 1:21:36
  SPEAKER_01: We bring a bunch of value when you understand what good code looks like,
• 1:21:41
  SPEAKER_01: when you can recognize what tech debt is.
• 1:21:43
  SPEAKER_00: And it's also a problem that we've always had.
• 1:21:46
  SPEAKER_00: You know what I mean?
• 1:21:47
  SPEAKER_01: It hasn't gone away.
• 1:21:48
  SPEAKER_00: Hasn't gone away.
• 1:21:48
  SPEAKER_00: You know, this is just, this is what I feel like.
• 1:21:51
  SPEAKER_00: We're just having the same conversations we've had for 20 years.
• 1:21:54
  SPEAKER_00: It's just there's this new elephant in the room.
• 1:21:55
  SPEAKER_01: I want to ask you about...
• 1:21:58
  SPEAKER_01: Living in the UK and AI, this is a question that also came from one of the readers.
• 1:22:04
  SPEAKER_01: Now that you're based in the UK and outside of London,
• 1:22:07
  SPEAKER_01: but you're now educating about AI,
• 1:22:11
  SPEAKER_01: is being further away from Silicon Valley and the HQ of
• 1:22:15
  SPEAKER_01: the labs making things easier or harder for you?
• 1:22:17
  SPEAKER_00: I'm really just trying to plow my own furrow, really.
• 1:22:20
  SPEAKER_00: What I realised quite early on is that I have no power to predict the future.
• 1:22:24
  SPEAKER_00: Because I'm so far away from things, I'm just a person in the field working with this stuff.
• 1:22:30
  SPEAKER_00: I have no way of knowing what's coming.
• 1:22:33
  SPEAKER_00: I don't know whether the model is going to improve.
• 1:22:35
  SPEAKER_00: I don't have privileged access to stuff.
• 1:22:37
  SPEAKER_00: And so I'm just trying to focus on what's working right now.
• 1:22:41
  SPEAKER_00: And because of that, I think that's narrowed my scope a little bit.
• 1:22:45
  SPEAKER_00: That means I can just try to get my stuff working.
• 1:22:49
  SPEAKER_00: And it's sort of quite surprising to me that it's working as well as it is,
• 1:22:53
  SPEAKER_00: you know, because I don't have this privileged access.
• 1:22:56
  SPEAKER_00: I'm just trying to make this one approach work.
• 1:22:59
  SPEAKER_00: So I think, yeah, you're probably right.
• 1:23:00
  SPEAKER_00: I probably would be able to do this stuff if I lived in San Francisco,
• 1:23:03
  SPEAKER_00: but then I'd have to live in San Francisco.
• 1:23:05
  SPEAKER_00: I don't want to do that.
• 1:23:06
  SPEAKER_00: That's miserable.
• 1:23:07
  SPEAKER_00: I've got a great setup here.
• 1:23:08
  SPEAKER_00: My parents are just down the road.
• 1:23:10
  SPEAKER_00: I've got my son growing up in the countryside.
• 1:23:12
  SPEAKER_00: So it is what it is.
• 1:23:15
  SPEAKER_01: Now, you're an educator at heart.
• 1:23:18
  SPEAKER_01: How have you seen the business of teaching or educating software engineers change
• 1:23:24
  SPEAKER_01: and also how people want to learn if you've observed any trends from before?
• 1:23:30
  SPEAKER_01: Already when you started,
• 1:23:31
  SPEAKER_01: I feel you were on at the time where online courses and learning over video became
• 1:23:36
  SPEAKER_01: a lot more popular as opposed to,
• 1:23:38
  SPEAKER_01: let's say,
• 1:23:38
  SPEAKER_01: a decade ago where it was maybe tutorials and then before that it was books.
• 1:23:42
  SPEAKER_01: Obviously, they still exist, but they're just different preferences.
• 1:23:45
  SPEAKER_00: Yeah, it was around COVID time that sort of video tutorials really took off.
• 1:23:49
  SPEAKER_00: I think you wanted a much richer learning experience and I was kind of just after
• 1:23:55
  SPEAKER_00: that wave, I suppose.
• 1:23:56
  SPEAKER_00: I think that people's People's way they've learned hasn't changed that much, right?
• 1:24:03
  SPEAKER_00: And their desire for...
• 1:24:05
  SPEAKER_00: Certain types of materials hasn't changed.
• 1:24:07
  SPEAKER_00: I think it's very sexy, the idea that an agent can just come in and teach you everything.
• 1:24:13
  SPEAKER_00: And that sort of works in some contexts, but really what you want is curation, right?
• 1:24:19
  SPEAKER_00: You want a human to have come in, understand the flow of the information.
• 1:24:24
  SPEAKER_00: I always think of information as kind of like a graph, right?
• 1:24:27
  SPEAKER_00: You have a piece of information that's dependent on another piece of information,
• 1:24:31
  SPEAKER_00: dependent on another piece of information,
• 1:24:33
  SPEAKER_00: and that Turning that graph into a linear path is how I think of my job, right?
• 1:24:40
  SPEAKER_00: I'm just trying to teach you Dijkstra's algorithm through the graph so
• 1:24:44
  SPEAKER_00: that you can learn it in the most sensible way.
• 1:24:47
  SPEAKER_00: And that level of curation is just not something that, again, that's strategic, right?
• 1:24:51
  SPEAKER_00: That's not something that AI is particularly good at.
• 1:24:53
  SPEAKER_00: So, I mean,
• 1:24:54
  SPEAKER_00: I've obviously made this huge pivot from TypeScript,
• 1:24:58
  SPEAKER_00: from tactical stuff, really, to this strategic layer.
• 1:25:02
  SPEAKER_00: And it's working okay for me.
• 1:25:04
  SPEAKER_00: I really can't speak for other folks doing this work.
• 1:25:07
  SPEAKER_00: And I know that lots of people are not having this level of success, I suppose.
• 1:25:14
  SPEAKER_00: So I think what it shows is that agents have just changed the game in terms
• 1:25:19
  SPEAKER_00: of what people value and what people prioritize.
• 1:25:20
  SPEAKER_00: And the industry has shifted in seven months faster than it's
• 1:25:24
  SPEAKER_00: I think ever done, you know, this is a huge shift.
• 1:25:26
  SPEAKER_00: It doesn't mean we need to throw away our working practices,
• 1:25:29
  SPEAKER_00: but it does mean that what we need to focus on is difference.
• 1:25:32
  SPEAKER_00: And I feel like I've been able to move with that quite well,
• 1:25:35
  SPEAKER_00: whereas I think others just haven't because they're focused on different things.
• 1:25:39
  SPEAKER_01: And I wonder if in your case,
• 1:25:41
  SPEAKER_01: it's also with Total TypeScript and even before with TypeScript,
• 1:25:45
  SPEAKER_01: some of the things you shared,
• 1:25:47
  SPEAKER_01: you were helping people use the very popular tool at the time.
• 1:25:52
  SPEAKER_01: TypeScript was gaining market share.
• 1:25:54
  SPEAKER_01: There were migrations happening from JavaScript to TypeScript,
• 1:25:57
  SPEAKER_01: from Python to TypeScript and so on.
• 1:26:00
  SPEAKER_01: Developers wanted to get really good,
• 1:26:02
  SPEAKER_01: a lot of them,
• 1:26:03
  SPEAKER_01: or the top 10% or top 20%,
• 1:26:05
  SPEAKER_01: you name it,
• 1:26:06
  SPEAKER_01: wanted to get really,
• 1:26:07
  SPEAKER_01: really good with TypeScript and they were looking for efficient ways of doing it.
• 1:26:11
  SPEAKER_01: Now, AI is here is changing how we work as software engineers.
• 1:26:15
  SPEAKER_01: And I think it's pretty clear that building software is valuable.
• 1:26:18
  SPEAKER_01: But there's a question of how do I use these tools more efficiently,
• 1:26:21
  SPEAKER_01: which is more pressing right now than how do I write TypeScript efficiently,
• 1:26:24
  SPEAKER_01: especially with agents.
• 1:26:25
  SPEAKER_01: So I wonder if you've kind of just a little bit how you pivoted from voice acting
• 1:26:30
  SPEAKER_01: to what you couldn't do from outside of London to a thing that you could do outside
• 1:26:35
  SPEAKER_01: of London, which was still teaching.
• 1:26:37
  SPEAKER_01: You've just pivoted to teaching a different area,
• 1:26:39
  SPEAKER_01: which right now is, again, it's on so many people's minds.
• 1:26:43
  SPEAKER_00: I think I've just been lucky, basically, of choosing the right thing at the right time.
• 1:26:47
  SPEAKER_00: It would have been very easy for me to,
• 1:26:48
  SPEAKER_00: and I actually took quite a fair bit of convincing to move into AI.
• 1:26:51
  SPEAKER_00: Like back a couple of years ago,
• 1:26:53
  SPEAKER_00: it was Joel,
• 1:26:55
  SPEAKER_00: my business partner,
• 1:26:55
  SPEAKER_00: who was pushing me to actually go, you've really got to try this.
• 1:26:58
  SPEAKER_00: It's actually pretty good and you can use it for all sorts of stuff.
• 1:27:01
  SPEAKER_00: And it took about three months of Me actually trying it and failing and trying
• 1:27:06
  SPEAKER_00: and failing before I realised, okay, this is great.
• 1:27:08
  SPEAKER_00: I just feel quite fortunate that I've landed in the right place at the right time.
• 1:27:13
  SPEAKER_00: And I try not to narrativise it.
• 1:27:15
  SPEAKER_00: I try not to think, well,
• 1:27:16
  SPEAKER_00: well done,
• 1:27:17
  SPEAKER_00: Matt, you've been so smart, you know, making the right play at the right time.
• 1:27:20
  SPEAKER_00: I've made several mistakes as well and I could have easily found myself in a different zone.
• 1:27:25
  SPEAKER_00: And I mean, that's no bad thing.
• 1:27:27
  SPEAKER_00: I would just go back to being an engineer.
• 1:27:28
  SPEAKER_00: That's what I love too.
• 1:27:30
  SPEAKER_01: Putting yourself back into the shoes of when you were someone just starting out in
• 1:27:33
  SPEAKER_01: the industry.
• 1:27:34
  SPEAKER_01: Today, for people starting out in the industry,
• 1:27:36
  SPEAKER_01: early career, junior folks, what would you recommend them for tactical things to do?
• 1:27:41
  SPEAKER_01: Like, they will know, like, look, I want to get that experience.
• 1:27:44
  SPEAKER_01: I want to get that judgment, that taste, those fundamentals.
• 1:27:47
  SPEAKER_01: You'll need to get repetitions.
• 1:27:49
  SPEAKER_01: And if you found yourself in those shoes, how would you approach?
• 1:27:53
  SPEAKER_01: Like, I want to be a builder, a software engineer with...
• 1:27:56
  SPEAKER_01: All these AI tools,
• 1:27:57
  SPEAKER_01: whatnot,
• 1:27:57
  SPEAKER_01: which is now confusing because now there's a mix of do I use these AI tools just
• 1:28:01
  SPEAKER_01: to do stuff for me?
• 1:28:03
  SPEAKER_01: Do I get in the fundamentals, which is slow and so on.
• 1:28:05
  SPEAKER_00: Yeah, I mean, I would love to be a junior right now.
• 1:28:08
  SPEAKER_00: I would love to be in the exact position I was in like 2014,
• 1:28:11
  SPEAKER_00: where I was building these tools for my students, right?
• 1:28:14
  SPEAKER_00: I actually got really nostalgic for it on the other day.
• 1:28:18
  SPEAKER_00: I thought,
• 1:28:18
  SPEAKER_00: I'd love to get back and do some singing teaching because just the ability to, like,
• 1:28:22
  SPEAKER_00: I could finish a lesson and then just prompt the agent,
• 1:28:24
  SPEAKER_00: okay, this tool didn't quite work in that way.
• 1:28:26
  SPEAKER_00: I could maybe modify it a little bit and, you know, see it working.
• 1:28:30
  SPEAKER_00: I just think the right thing to do is to use these agents as much as possible,
• 1:28:34
  SPEAKER_00: because that's how people are going to be working now.
• 1:28:37
  SPEAKER_00: And I think the thing that I find valuable about my skill set is you're constantly
• 1:28:42
  SPEAKER_00: in touch with the changes that are happening.
• 1:28:45
  SPEAKER_00: GrillMe not only... You're having a discussion with a senior developer, right?
• 1:28:50
  SPEAKER_00: That's beneficial for the developer, but it's also beneficial for you.
• 1:28:54
  SPEAKER_00: Keeps you thinking about these deeper ideas.
• 1:28:56
  SPEAKER_00: And the absolute...
• 1:28:58
  SPEAKER_00: So, yeah.
• 1:29:20
  SPEAKER_00: I think that there's never been a more empowering time to work on this stuff as long
• 1:29:25
  SPEAKER_00: as you're interested in not only the code you're producing,
• 1:29:28
  SPEAKER_00: but also the process of creating the code.
• 1:29:31
  SPEAKER_00: There's never been a better time to be a kind of navel-gazing programmer,
• 1:29:35
  SPEAKER_00: just constantly thinking about your own processes and being introspective.
• 1:29:39
  SPEAKER_01: So it sounds like if you're motivated,
• 1:29:40
  SPEAKER_01: you should be able to learn really fast compared to even before.
• 1:29:43
  SPEAKER_00: Absolutely.
• 1:29:44
  SPEAKER_00: It's just about being curious about being adaptable.
• 1:29:46
  SPEAKER_00: And that's the people that I see who are thriving in this new environment are
• 1:29:49
  SPEAKER_00: the same people who were thriving 10 years ago because they're just interested
• 1:29:54
  SPEAKER_00: in this work,
• 1:29:55
  SPEAKER_00: interested in making better software and interest in their own process.
• 1:29:59
  SPEAKER_01: And all interesting in making better software is I want to ask you about gardening.
• 1:30:03
  SPEAKER_01: A software engineer on XLauren posted, I'll quote her, every team needs a gardener.
• 1:30:08
  SPEAKER_01: Someone quietly watching the stream of PRs flowing into your codebase,
• 1:30:11
  SPEAKER_01: noticing the smells,
• 1:30:12
  SPEAKER_01: the lint suppressions creeping like ivy across your careful garden,
• 1:30:16
  SPEAKER_01: a steady hand intending the weeds that would otherwise engulf the garden.
• 1:30:20
  SPEAKER_01: And to which you replied, I'd argue the only thing your team needs are gardeners.
• 1:30:25
  SPEAKER_00: You probably do need a couple of other people as well.
• 1:30:27
  SPEAKER_01: Yeah, but more specifically, I want to ask about this concept of gardening.
• 1:30:31
  SPEAKER_01: I actually really love how Lauren described the weeds taking over the garden
• 1:30:36
  SPEAKER_01: and getting them out.
• 1:30:37
  SPEAKER_00: I think I made a tweet a while ago that we are,
• 1:30:40
  SPEAKER_00: this was when I was sort of thinking about Ralph and sort of the agents sort
• 1:30:43
  SPEAKER_00: of looping over stuff.
• 1:30:44
  SPEAKER_00: We are essentially just Ralph's platform team, right?
• 1:30:47
  SPEAKER_00: That's what we are now.
• 1:30:48
  SPEAKER_00: And we are our agent's platform team.
• 1:30:50
  SPEAKER_00: We are trying to build the environment for them to succeed.
• 1:30:54
  SPEAKER_00: That's exactly how you should be thinking about it.
• 1:30:56
  SPEAKER_00: Again, it's strategic.
• 1:30:57
  SPEAKER_00: And that gardener metaphor is nice because,
• 1:31:00
  SPEAKER_00: you know, it's very easy for the garden to itself just to suffer entropy, right?
• 1:31:05
  SPEAKER_00: To gather weeds and to do all that stuff.
• 1:31:07
  SPEAKER_00: So understanding...
• 1:31:09
  SPEAKER_00: And diagnosing that stuff before it becomes a problem in your own codebase is
• 1:31:13
  SPEAKER_00: an essential skill and might be the essential skill, right?
• 1:31:17
  SPEAKER_00: As long as you can queue up work for agents,
• 1:31:19
  SPEAKER_00: as long as you can build these loops now that we're starting to see,
• 1:31:22
  SPEAKER_00: these processes where agents improve the codebase based on bug reports
• 1:31:26
  SPEAKER_00: and feedbacks,
• 1:31:27
  SPEAKER_00: that feels to me like really cool work and noble, interesting work as well.
• 1:31:33
  SPEAKER_01: We talked about some great standout software engineers that you learned from,
• 1:31:37
  SPEAKER_01: you got inspiration from.
• 1:31:39
  SPEAKER_01: Today, what skill sets, experience, approach do you think makes a great software engineer?
• 1:31:47
  SPEAKER_00: I'll use an example,
• 1:31:48
  SPEAKER_00: which is Lars Grammel,
• 1:31:50
  SPEAKER_00: who works at Vercel on the AISDK, who I had a chat with the other day.
• 1:31:55
  SPEAKER_00: And he is building an entire software factory
• 1:31:58
  SPEAKER_00: for his extremely popular open source library that gets a ton of issues.
• 1:32:04
  SPEAKER_00: We're talking about plumbing again.
• 1:32:05
  SPEAKER_00: We're talking about gardening.
• 1:32:06
  SPEAKER_00: We're like thinking about the processes of software development.
• 1:32:10
  SPEAKER_00: And I suppose...
• 1:32:11
  SPEAKER_00: If I had to put it in a word, it would be introspection.
• 1:32:13
  SPEAKER_00: It would be looking at yourself and the ability to take what you do and put that
• 1:32:19
  SPEAKER_00: into something the AI can work with.
• 1:32:22
  SPEAKER_00: You're essentially trying to put your process into words,
• 1:32:25
  SPEAKER_00: and that's what I've been doing with the skills.
• 1:32:27
  SPEAKER_00: That's what I've been trying to do with the automations I've been creating as well.
• 1:32:31
  SPEAKER_00: It's I just look at what I'm doing and think, how could I do this better?
• 1:32:35
  SPEAKER_00: And also, how could I encode this into this strange animal that I have in front of me?
• 1:32:41
  SPEAKER_00: How can I make it work like I want to?
• 1:32:44
  SPEAKER_00: And that attitude has been really, really helpful for me.
• 1:32:48
  SPEAKER_00: And it's something that I value in Lars and I value in all the people that I work
• 1:32:52
  SPEAKER_00: with when they approach agents.
• 1:32:54
  SPEAKER_01: And then as closing, what is a book that you would recommend or multiple books?
• 1:32:59
  SPEAKER_00: I'll go with Pragmatic Programmer, Philosophy of Software Design by John Osterhout.
• 1:33:04
  SPEAKER_00: And I'd say the first like three chapters of DDD,
• 1:33:08
  SPEAKER_00: the Eric Evans book, the ubiquitous language one.
• 1:33:10
  SPEAKER_00: That one in particular,
• 1:33:11
  SPEAKER_00: it's really great for the ubiquitous language concepts,
• 1:33:14
  SPEAKER_00: the domain modeling, the actual sort of encoding it into code.
• 1:33:17
  SPEAKER_00: I'm not such a huge fan of, but those three are the big three.
• 1:33:21
  SPEAKER_01: It was so
• 1:33:26
  SPEAKER_00: nice
• 1:33:30
  SPEAKER_01: to sit down with Matt and I have to say,
• 1:33:34
  SPEAKER_01: knowing that he was a voice coach and actor makes me understand how he talks
• 1:33:37
  SPEAKER_01: so smooth and how he's so pleasant to listen to.
• 1:33:40
  SPEAKER_01: Probably the most amusing part from this conversation was how,
• 1:33:43
  SPEAKER_01: as Matt was searching for how to work better with AI,
• 1:33:46
  SPEAKER_01: it wasn't much There's some irony as to how
• 1:33:48
  SPEAKER_01: the best practices documented 20 plus years ago, like tactical versus
• 1:34:02
  SPEAKER_01: The strategic programming in this book,
• 1:34:04
  SPEAKER_01: not only do they still work,
• 1:34:05
  SPEAKER_01: but they become more important when writing code with AI agents.
• 1:34:09
  SPEAKER_01: A related point I want to emphasize is the importance of leading words with AI.
• 1:34:13
  SPEAKER_01: When Matt started to use terms like tracer bullet or vertical slices,
• 1:34:17
  SPEAKER_01: the model started to follow his ideas better in planning.
• 1:34:20
  SPEAKER_01: And if you think about it,
• 1:34:22
  SPEAKER_01: this makes sense,
• 1:34:22
  SPEAKER_01: because software engineering literature is part of LLM training,
• 1:34:26
  SPEAKER_01: so these terms are also part of the model's priors.
• 1:34:29
  SPEAKER_01: Just as interestingly, using the right words for describing your problem is not a new concept.
• 1:34:34
  SPEAKER_01: For example, when I had Ken Beck on This was just another full circle moment on how words
• 1:34:51
  SPEAKER_01: Finally,
• 1:34:51
  SPEAKER_01: I appreciated Matt's push on how you should want a clean codebase,
• 1:34:55
  SPEAKER_01: not just because it's easier for a human to navigate,
• 1:34:57
  SPEAKER_01: although I think you really want to do it for that as well,
• 1:35:00
  SPEAKER_01: but also conveniently,
• 1:35:02
  SPEAKER_01: agents do not have a long-term memory,
• 1:35:04
  SPEAKER_01: and they will look at Thanks, and see you in the next one.