Latest / The Pragmatic Engineer / Developer productivity with Dr. Nicole Forsgren (creator of DORA, co-creator of SPACE)

Transcript

Developer productivity with Dr. Nicole Forsgren (creator of DORA, co-creator of SPACE) · The Pragmatic Engineer

0:00
SPEAKER_01: S is the satisfaction metric.
0:01
SPEAKER_01: Just ask people.
0:02
SPEAKER_01: Are you satisfied with the tools you use?
0:04
SPEAKER_01: Are you satisfied with something?
0:05
SPEAKER_01: Because people can usually tell you pretty quickly.
0:07
SPEAKER_01: I've seen data across an entire tool chain that was pretty fast.
0:10
SPEAKER_01: It was good for the company.
0:11
SPEAKER_01: But you ask the developers, and they're like, this is nearly impossible.
0:14
SPEAKER_01: Performance, outcome metrics, we definitely want.
0:17
SPEAKER_01: A is activity, all accounts, anything that's account.
0:19
SPEAKER_01: We usually have a lot of activity metrics.
0:21
SPEAKER_01: C is communication and collaboration.
0:23
SPEAKER_01: This could be looking at meeting schedules.
0:24
It can also be looking at how well APIs are working or if
0:27
they're breaking.
0:28
SPEAKER_01: And then E is efficiency and flow.
0:29
Those types of things,
0:30
those types of data points and metrics give you additional insight.
0:33
Nicole Forsgren is probably the best known expert on developer productivity.
0:37
SPEAKER_00: She is a co-creator at the widely used frameworks Dora and Space,
0:40
SPEAKER_00: lead author of the book Accelerate.
0:43
SPEAKER_00: Nicole started her career as a software engineer at IBM,
0:46
where she experienced firsthand the challenges of being given
0:48
unrealistic deadlines
0:49
SPEAKER_00: by management who understood nothing about developer productivity.
0:53
SPEAKER_00: Seeing how developers burned out around her,
0:55
SPEAKER_00: she entered academia,
0:56
SPEAKER_00: researching DevOps and developer productivity.
0:58
She then joined Chef Software and then co-founded a bootstrap
1:01
startup called Dora.
1:03
Dora was acquired by Google,
1:04
and Nicole worked on developer productivity here.
1:07
SPEAKER_00: She later joined GitHub as VP of Research and Strategy,
1:10
SPEAKER_00: and she is currently leading research initiatives at Microsoft.
1:13
SPEAKER_00: She's also working on a book related to developer productivity,
1:16
SPEAKER_00: one that we'll touch on in the podcast.
1:18
SPEAKER_00: In our conversation today,
1:19
SPEAKER_00: we cover Dora and Space,
1:22
examples of using these frameworks and advice on how to make
1:25
the most of them.
1:26
SPEAKER_00: Measuring developer productivity,
1:28
whether measuring PRs makes sense and why this area remains
1:31
challenging to quantify.
1:33
Traits of high performing engineering teams and why time to
1:35
onboarding can be very
1:36
SPEAKER_00: telling about the efficiency of a team and many more interesting topics.
1:40
If you're looking for advice on making your engineering team
1:42
more efficient or are
1:43
SPEAKER_00: interested in ways to measure developer productivity,
1:46
SPEAKER_00: this episode is for you.
1:47
If you enjoy the show,
1:48
please subscribe to the podcast on any podcast platform and on YouTube.
1:51
SPEAKER_00: Thank you.
1:52
SPEAKER_00: This greatly helps the show to spread to even more listeners and viewers.
1:55
SPEAKER_00: With this, I'm very excited to have Nicole on the podcast.
1:59
SPEAKER_00: I wanted to put a pointed question at you.
2:01
SPEAKER_00: You've...
2:04
SPEAKER_00: You've been studying developer productivity both at the theory level,
2:08
SPEAKER_00: but also very practical level.
2:11
SPEAKER_00: What is your take on the importance of PRs or diffs?
2:14
And the background is here that there's now a renewed
2:17
conversation about this.
2:19
There's been this viral threat going around on the concept of
2:24
ghost engineers who
2:25
SPEAKER_00: might not be producing as much production.
2:29
kind of like code on a monthly basis and you know that's
2:32
something that that can be
2:33
measured and also companies like uber and meta they do have
2:36
tools for managers and
2:37
directors to track the number of diffs per team they don't say
2:40
that they're going
2:41
to break it down for engineer it is possible and also there's a
2:44
new dx core 4
2:45
SPEAKER_00: framework which is something that uh is is going around and the
2:51
They also suggest that the number of diffs at the team level
2:57
should be measured.
2:58
SPEAKER_00: I'm curious,
2:59
SPEAKER_00: how important is the diff frequency and how good of a predictor is it for
3:04
SPEAKER_00: productivity or unproductivity at a team at an individual level?
3:07
SPEAKER_01: I love this question.
3:09
SPEAKER_01: Thank you so much.
3:11
SPEAKER_01: I will say PRs and diffs.
3:15
SPEAKER_01: both are good signals and are terrible signals.
3:18
Which is probably not helpful,
3:19
but let me kind of dig into this a little bit more.
3:22
SPEAKER_01: When we think about using the word productivity in particular,
3:25
SPEAKER_01: many folks,
3:26
SPEAKER_01: especially in leadership,
3:28
SPEAKER_01: are thinking about the traditional economic definition of productivity,
3:31
SPEAKER_01: which is the rate of output,
3:35
SPEAKER_01: right?
3:35
SPEAKER_01: What do you produce?
3:37
SPEAKER_01: And then sometimes that'll be a ratio, right?
3:38
SPEAKER_01: The rate of output per inputs.
3:43
SPEAKER_01: And so
3:44
as we know,
3:45
that doesn't apply very well to software or it's really hard to disentangle,
3:49
right?
3:49
SPEAKER_01: Is it a feature?
3:50
SPEAKER_01: Is it a shipment?
3:51
SPEAKER_01: Is it a full product?
3:52
SPEAKER_01: Is it what, right?
3:54
SPEAKER_01: It's different than in like farming.
3:56
SPEAKER_01: I grew up, my grandfather was a farmer, right?
3:58
SPEAKER_01: Like how much do you put in and then how many potatoes do you get out?
4:02
SPEAKER_01: Like that's fairly straightforward.
4:03
SPEAKER_01: And when we think about GDP, that's often what it's looking at.
4:06
SPEAKER_01: But in technology,
4:08
SPEAKER_01: Things are just different because many times you'll get many,
4:13
SPEAKER_01: many,
4:13
SPEAKER_01: many more things when we think about cameras and phones or photos,
4:17
SPEAKER_01: right?
4:17
SPEAKER_01: We could take a handful of photos.
4:19
SPEAKER_01: Now we can take millions of photos and it's basically free.
4:23
SPEAKER_01: And so many things in technology are just flipped.
4:25
SPEAKER_01: Yeah.
4:26
SPEAKER_01: So with that context, I will say PRs are important in a few ways.
4:35
One way is that it does give you a relatively reasonable view
4:40
into what work is
4:41
getting done and what work can get done through the current
4:44
system and processes
4:45
SPEAKER_01: that you have.
4:46
SPEAKER_01: right?
4:48
SPEAKER_01: With a couple important caveats and disclaimers, right?
4:50
SPEAKER_01: Many senior engineers don't have very high PR output or code commits.
4:57
Now,
4:58
for those of us who have been in software for a while,
5:00
this isn't surprising,
5:01
right?
5:01
Because as you become more and more senior,
5:03
you're doing things like unblocking folks.
5:05
SPEAKER_01: You're doing things like architecture and design work.
5:08
You're doing things like supporting reviews and mentoring and
5:10
maybe like pair
5:11
SPEAKER_01: programming with juniors.
5:13
SPEAKER_01: And so your commits
5:15
SPEAKER_01: Your PRs, yours, finger quote, right?
5:18
SPEAKER_01: Aren't showing up.
5:19
SPEAKER_00: This episode is brought to you by DX,
5:21
SPEAKER_00: the engineering intelligence platform designed by leading researchers.
5:25
Measuring developer productivity is a decades old challenge and
5:28
many technology
5:29
SPEAKER_00: organizations still struggle to get it right.
5:32
DX takes a scientific approach that offers complete clarity
5:35
into productivity and
5:36
SPEAKER_00: its underlying factors.
5:38
It combines data from development tools with self-reported data
5:41
collected from
5:42
SPEAKER_00: developers so you can see both the what and the why.
5:46
The insights from their platform cater to leaders at every
5:48
level of the organization,
5:50
SPEAKER_00: from CTOs to platform teams to frontline managers.
5:53
SPEAKER_00: Learn why some of the world's most iconic companies like Dropbox,
5:57
SPEAKER_00: Plaid,
5:57
SPEAKER_00: Twilio,
5:58
SPEAKER_00: Vercel,
5:59
SPEAKER_00: and Gusto rely on DX.
6:01
SPEAKER_00: Just visit getdx.com slash pragmatic engineer.
6:05
SPEAKER_00: That is G-E-T-D-X dot com slash pragmatic engineer.
6:11
And then you're also pulled into recruitment,
6:14
your leadership meeting,
6:15
performance calibration.
6:17
SPEAKER_00: Oh, there's this team on fire.
6:18
SPEAKER_00: We need you to help out.
6:19
SPEAKER_00: The higher you are staff and principal level, it happens all the time.
6:23
SPEAKER_01: Exactly, exactly.
6:25
SPEAKER_01: And so I think it's really important to think about not just recruiting,
6:29
SPEAKER_01: what we're measuring,
6:30
SPEAKER_01: but in which context and who we're measuring,
6:33
SPEAKER_01: because that's not,
6:34
SPEAKER_01: I wouldn't call that output.
6:36
SPEAKER_01: I would call that one aspect of throughput, right?
6:39
Because it can be really interesting and important to see if
6:44
there are blockers,
6:45
why is it?
6:46
SPEAKER_01: If there's friction, why is it?
6:47
SPEAKER_01: Is it an overly complicated code base?
6:49
SPEAKER_01: Is it a really, really difficult to use, uh,
6:53
SPEAKER_01: tech stack and developer ecosystem?
6:55
SPEAKER_01: Is it that our PR assignment process is just like bad, right?
7:01
Either it only goes to one person or it doesn't like get shared
7:05
out very well.
7:05
SPEAKER_01: And so one person or just the assignment is the blocker.
7:09
SPEAKER_01: So there are a handful of things to look at there.
7:14
It's bad as a just straight solo metric,
7:18
but I think everything as bad as a solo metric.
7:22
What we always wanna do is take kind of a constellation of
7:26
metrics that help us
7:28
think more holistically about the context in which developers
7:32
are working.
7:33
SPEAKER_01: What are the trade-offs that they are making?
7:35
SPEAKER_01: So I've been working with some folks to kind of kick up,
7:40
SPEAKER_01: we started the Eng Thrive effort at Microsoft.
7:43
And it's a way to understand and start conversations at several
7:46
levels in the
7:47
SPEAKER_01: business about how can we help our engineers thrive?
7:50
SPEAKER_01: And we use a handful of metrics across many dimensions.
7:53
SPEAKER_01: And it was inspired in part by space, the space framework.
7:58
SPEAKER_01: Because then we can look at qualitative feedback, right?
8:01
Microsoft famously has NSAT,
8:05
which is kind of like a satisfaction qualitative survey.
8:08
SPEAKER_00: That's like every six months, right?
8:10
SPEAKER_00: I think I remember when I was there.
8:11
SPEAKER_00: It was still going on 10, 15 years ago.
8:15
SPEAKER_01: Yeah,
8:15
some of them are quarterly now so that we can make sure we stay
8:17
on top of any trends,
8:19
SPEAKER_01: any changes.
8:21
But then we also want to capture some objective data from our
8:24
systems because that
8:24
SPEAKER_01: can provide additional insight
8:26
SPEAKER_01: And you're not asking people for it all the time.
8:28
SPEAKER_01: You're not interrupting their work.
8:30
SPEAKER_01: And so there you're looking at examples of quality and outcome metrics.
8:35
SPEAKER_01: You're looking at examples of any delays that may come up.
8:38
SPEAKER_01: Now, I'm just giving broad examples.
8:40
SPEAKER_01: This isn't necessarily what N5 uses,
8:42
but build time or deployment pieces or security outcomes or
8:48
quality and stability outcomes.
8:50
SPEAKER_01: PRs
8:52
SPEAKER_01: And the data that are around PRs, right?
8:55
So it can be number of PRs,
8:56
it can be time for PR,
8:57
it can be the time that awaits from review.
9:01
SPEAKER_01: Those can be helpful in this constellation, right?
9:06
Because then you're also seeing and you can think about a story
9:11
of why people are
9:12
SPEAKER_01: making decisions to do the work that they're doing.
9:15
SPEAKER_01: And what are the impacts and implications?
9:17
SPEAKER_01: And I love that because it can help inspire
9:20
SPEAKER_01: Larger questions around when we're making investments.
9:24
Now,
9:24
say investments,
9:25
whether it's money or capital or time,
9:27
where should those go,
9:29
right?
9:29
How do we want to balance feature delivery and customer value
9:34
delivery with
9:35
SPEAKER_01: internal investments?
9:36
SPEAKER_01: Because those are so tightly related, right?
9:39
SPEAKER_01: I like to joke.
9:40
SPEAKER_01: This is our hot AI moment, right?
9:41
SPEAKER_01: And everyone's like, well, build all the AI things.
9:44
SPEAKER_01: Everything's AI.
9:44
SPEAKER_01: You have to go faster.
9:45
SPEAKER_01: It's real hard to build the AI things when your pipeline's trash.
9:51
SPEAKER_01: It's super hard if your developer experience just isn't there.
9:54
SPEAKER_00: I agree.
9:55
I also feel that the reason,
9:56
I've been thinking,
9:57
why is this trending right now?
9:58
People seem to be really touchy-feely about the fact that you
10:02
can measure diffs.
10:03
This survey was specifically about remote engineers,
10:07
you know,
10:07
like potentially doing less.
10:09
SPEAKER_00: And I feel that what is happening,
10:11
there's a lot of things that what you're talking about and what
10:13
people talk about
10:14
SPEAKER_00: in developer productivity,
10:15
SPEAKER_00: it's like you have a team that's
10:17
SPEAKER_00: doing well,
10:18
SPEAKER_00: everyone is,
10:18
SPEAKER_00: you know,
10:18
SPEAKER_00: doing their work and we're trying to figure out how to do better.
10:21
SPEAKER_00: What's happening?
10:22
SPEAKER_00: Where are people stuck?
10:23
SPEAKER_00: Where are the bottlenecks?
10:24
SPEAKER_00: But there's another question,
10:25
SPEAKER_00: which is performance management,
10:27
SPEAKER_00: which is,
10:28
SPEAKER_00: uh,
10:28
SPEAKER_00: I have a team and I'm a non-technical manager,
10:31
SPEAKER_00: or I might be,
10:32
SPEAKER_00: you know,
10:32
SPEAKER_00: removed from the layers.
10:33
I'm the director or CEO,
10:35
and I just want some stats that tell me who's performing well
10:38
and not.
10:38
SPEAKER_00: And there's an assumption for some reason for people who are not, uh,
10:42
SPEAKER_00: working here that I can just look at some numbers,
10:44
SPEAKER_00: may this be output or some scores,
10:46
SPEAKER_00: and I'll see people scored.
10:48
SPEAKER_00: And my question to you,
10:50
how important or is it possible to run a good engineering team
10:57
without a manager or
10:58
a lead who is hands-on and kind of understands this context and
11:02
knows that what's
11:04
SPEAKER_00: going on is kind of involved?
11:06
SPEAKER_00: Because I'm kind of seeing that there's a real desire to
11:09
SPEAKER_00: from people,
11:10
SPEAKER_00: investors,
11:10
founders who are removed from technical details to get some of
11:14
these stats,
11:15
some of these developer frameworks so that they can tell
11:18
without understanding anything,
11:19
SPEAKER_00: oh,
11:20
SPEAKER_00: this engineer is good and productive.
11:22
SPEAKER_00: That engineer is not.
11:24
SPEAKER_00: A, is this realistic?
11:25
SPEAKER_00: And B, is this something new?
11:27
SPEAKER_00: Or have the business always been asking for something like this?
11:30
SPEAKER_01: I'll start with that last one.
11:31
SPEAKER_01: Is this new?
11:32
SPEAKER_01: I would say it's not new, right?
11:33
SPEAKER_01: Like we've been, we, the royal we.
11:35
SPEAKER_00: That's good.
11:36
Business and researchers have been watching people work and
11:39
trying to measure how
11:42
people work and trying to assess how people work with summaries
11:44
and numbers for a
11:46
SPEAKER_01: really long time,
11:47
SPEAKER_01: right?
11:47
SPEAKER_01: There's something in research called the Hawthorne effect.
11:49
SPEAKER_01: which is when- The what effect?
11:53
SPEAKER_01: I love it.
11:54
SPEAKER_01: So this gentleman was working with,
11:57
SPEAKER_01: I want to say it was like some type of production for process.
12:01
SPEAKER_01: And he would come into the office,
12:02
SPEAKER_01: he would flip the lights on,
12:04
SPEAKER_01: watch people work,
12:05
SPEAKER_01: count things,
12:06
SPEAKER_01: and he would leave.
12:07
SPEAKER_01: And they would count things and the numbers would be different.
12:10
SPEAKER_01: Well, so he was like, it must be because the lights are on.
12:12
Because I guess there's,
12:14
if I'm remembering correctly,
12:15
right,
12:15
like he turned the lights on.
12:16
SPEAKER_01: So they start turning the lights on, but it's not like consistent.
12:19
It turns out like people are doing more like when you're there,
12:23
when someone is there.
12:23
SPEAKER_00: When they know they're being watched, right?
12:25
SPEAKER_01: Athletes tend to perform better when there's an audience, right?
12:28
SPEAKER_00: As developers, we'll definitely do it.
12:30
As soon as we know what's measured,
12:31
we're like,
12:31
all right,
12:31
we're optimized for that.
12:32
SPEAKER_01: Absolutely.
12:33
SPEAKER_01: And I think this is where we have some real limitations around things.
12:37
SPEAKER_01: You touched on it a little bit when you said like, we can measure PRs.
12:40
So many times people will default to the measures that are easy
12:44
and quick to operationalize.
12:45
Number of commits,
12:46
number of PRs,
12:47
number of lines of code,
12:48
number of what can we grab quickly?
12:51
SPEAKER_01: And I think this is one thing that prompted space.
12:54
SPEAKER_01: The space framework is counts can be useful.
12:58
SPEAKER_01: Yes.
12:58
SPEAKER_01: If you have access to data that you can quickly operationalize and grab.
13:02
SPEAKER_01: Sure.
13:02
SPEAKER_01: Right.
13:03
inserting caveats around like,
13:05
what's the accuracy,
13:06
the reliability of the data?
13:07
SPEAKER_01: How are you gathering it?
13:09
SPEAKER_01: Are you doing cohort analysis?
13:11
SPEAKER_01: But also that will only show you a few things.
13:15
And the few things it will show you are the parts of the
13:17
business that are easy to
13:18
SPEAKER_01: operationalize and it will ignore everything else.
13:21
And so we often want to think about,
13:22
you know,
13:23
S is a satisfaction metric.
13:25
SPEAKER_01: Just ask people, right?
13:26
SPEAKER_01: Are you satisfied with the tools you use?
13:28
SPEAKER_01: Are you satisfied with something?
13:29
SPEAKER_01: Because
13:30
SPEAKER_01: People can usually tell you pretty quickly.
13:33
SPEAKER_01: I've seen data across an entire tool chain that was pretty fast.
13:36
SPEAKER_01: It was good for the company.
13:37
SPEAKER_01: But you ask the developers and they're like, this is nearly impossible.
13:41
SPEAKER_01: We are going to break soon because so much of it was in systems.
13:45
SPEAKER_01: So you'd think it was automated,
13:46
but every single handoff point had to be manual because they
13:49
were not integrated.
13:50
And that's one of the biggest weaknesses for operationalized
13:53
data is missing that handoff.
13:56
SPEAKER_01: the boundaries between systems and the boundaries between processes.
13:59
SPEAKER_01: So people can tell you pretty quickly.
14:03
SPEAKER_01: So performance, you know, outcome metrics, we definitely want.
14:06
SPEAKER_01: A is activity, all accounts, anything that's account, right?
14:08
SPEAKER_01: We usually have a lot of activity metrics.
14:12
SPEAKER_01: C is communication and collaboration.
14:14
SPEAKER_01: This can be looking at meeting schedules.
14:15
SPEAKER_01: It can also be looking at how well APIs are working.
14:19
SPEAKER_01: or if they're breaking and then ease efficiency and flow.
14:21
SPEAKER_01: And so that's where we're looking at time handoffs, um,
14:27
SPEAKER_01: You know,
14:27
SPEAKER_01: those types of things,
14:29
SPEAKER_01: those types of data points and metrics give you additional insight.
14:32
SPEAKER_01: And if you can grab at least three different categories,
14:35
SPEAKER_01: you're going to be in a much better position.
14:37
SPEAKER_01: Now,
14:38
SPEAKER_01: you also asked,
14:39
SPEAKER_01: and I'm kind of tying into the earlier question,
14:41
SPEAKER_01: which was,
14:42
SPEAKER_01: can you evaluate and understand how well a team's doing?
14:45
SPEAKER_01: Can an eng leader, an eng manager do it if they're not like there?
14:49
I think at that level,
14:51
right,
14:51
when you've got a tech lead,
14:52
when you've got an eng leader,
14:53
an
14:56
SPEAKER_01: absent and unaware of context does no one any favors, right?
15:00
SPEAKER_02: Yeah.
15:02
SPEAKER_01: At some level though,
15:03
I think there can be benefit to having a holistic set of
15:07
metrics to help leaders
15:10
SPEAKER_01: guide decisions around which type is an example,
15:13
what type of blockers or barriers or friction points are
15:16
affecting most of the company?
15:18
SPEAKER_01: Would that be a good use of a central resource?
15:20
SPEAKER_01: What types of barriers, um,
15:24
show up that are probably out of the realm or scope of control
15:29
for a development team to fix.
15:31
SPEAKER_01: Sometimes their documentation can be,
15:33
SPEAKER_01: but if it's something like a central build system,
15:36
SPEAKER_01: no individual team is going to be able to fix that.
15:40
Or it will be heroic effort,
15:42
but the larger the company,
15:43
the more impossible it is.
15:44
SPEAKER_00: Exactly.
15:46
SPEAKER_01: This is where I think
15:48
SPEAKER_01: there can be some value, right?
15:50
Especially when we're doing things like cohort analysis and
15:54
kind of holistic
15:55
measures and having it be a discussion where people can get
15:58
curious and can talk
16:00
SPEAKER_01: through what they think they're seeing and then go confirm.
16:04
Because it's just not realistic for an executive to meet and be
16:08
fully embedded with
16:09
every single engineering team that exists and every single
16:11
product team that exists
16:12
SPEAKER_01: and every single marketing team that exists to try to figure out,
16:16
SPEAKER_01: get a good feel for the business.
16:18
SPEAKER_00: This makes a lot of sense.
16:19
SPEAKER_00: And I really like about space how to me,
16:22
that was the first framework that kind of made it really clear
16:24
that this thing is
16:25
complex and we can measure a lot of things and we should
16:28
measure multiple things.
16:29
And sometimes they will tell us different stories,
16:32
but we need to look at the whole picture.
16:35
SPEAKER_00: But before we go too much into space and what happened before...
16:39
SPEAKER_00: Before space, there was Dora.
16:41
SPEAKER_00: You mentioned your first startup.
16:43
SPEAKER_00: It had a massive impact on the industry.
16:45
SPEAKER_00: The book Accelerate, I think, had a huge impact as well.
16:48
SPEAKER_00: I see it on the shelves of almost every engineering leader.
16:51
And if you've not read it,
16:53
I think it's a great read to start with,
16:56
at least.
16:57
SPEAKER_00: And in the Dora framework, there are four...
16:59
SPEAKER_00: It's key metrics.
17:00
SPEAKER_00: The deployment frequency for D,
17:02
SPEAKER_00: lead time for changes,
17:04
SPEAKER_00: change failure rate,
17:05
SPEAKER_00: time to restore service.
17:07
And still today,
17:08
a lot of engineering leaders who I meet,
17:10
they read the book.
17:11
SPEAKER_00: They kind of hold it up and say, like, I've read the book.
17:13
SPEAKER_00: I've cracked it.
17:15
SPEAKER_00: What we need to do to be a highly performing team.
17:18
SPEAKER_00: Here are these four things.
17:19
SPEAKER_00: We're going to measure it.
17:20
SPEAKER_00: And if we have high scores on it, we'll be there.
17:23
SPEAKER_00: Now, clearly, things have evolved since.
17:25
SPEAKER_00: But I'm interested.
17:28
SPEAKER_00: Where do you see the usefulness of Dora?
17:29
So teams who are optimizing for this and getting to this thing,
17:32
you know,
17:33
how far can they go?
17:34
SPEAKER_00: And what is the point where it's kind of time to look at other things?
17:37
And,
17:37
you know,
17:38
like,
17:38
for example,
17:39
how space and other things evolve from here?
17:43
SPEAKER_01: Dora is a lot of things.
17:45
SPEAKER_01: Dora is a...
17:47
SPEAKER_01: how I kind of refer to the entire research program,
17:49
SPEAKER_01: but a lot of folks just when they hear Dora,
17:51
SPEAKER_01: they think about the four metrics.
17:52
SPEAKER_01: And so I think understanding that there are both of those are important.
17:56
SPEAKER_01: And here's why the four metrics end up being,
17:59
SPEAKER_01: you know,
17:59
SPEAKER_01: through a lot of statistical tests over 10 years now,
18:03
SPEAKER_01: like even when I'm not doing the analysis,
18:05
SPEAKER_01: it ends up being a very good indicator,
18:09
SPEAKER_01: a really good signal for how well your development pipeline is working.
18:14
SPEAKER_01: Is it relatively fast?
18:17
SPEAKER_01: How many things can you deploy?
18:20
SPEAKER_01: What are the quality outcomes at the end of that?
18:24
And so for folks who are just starting out,
18:27
that can be a pretty good place to start.
18:32
Now,
18:32
I will say sometimes folks just hate me because they're like,
18:34
well,
18:34
this doesn't work here.
18:35
SPEAKER_01: I'm in air gap systems.
18:36
SPEAKER_01: I'm in wherever.
18:38
SPEAKER_01: find ways to accommodate for your environment.
18:43
SPEAKER_01: I do work with folks that work in air gap systems that use Dora.
18:45
SPEAKER_01: And so instead of the final...
18:46
SPEAKER_00: And by air gap systems,
18:48
can you just tell us for the ones of us who are not familiar
18:50
with that term?
18:51
SPEAKER_01: Yeah.
18:52
So air gap systems are where you have some gap in your
18:56
deployment process,
18:57
SPEAKER_01: usually right before final deployment,
18:59
SPEAKER_01: where there is no
19:02
SPEAKER_01: communication between the two sides, right?
19:03
SPEAKER_01: There's no input.
19:05
SPEAKER_01: You can't just push and it deploys all the way through.
19:08
So some folks that I work with that have air gap systems is the
19:10
US Navy,
19:11
right?
19:11
SPEAKER_01: Yeah.
19:12
SPEAKER_01: They do it.
19:12
SPEAKER_00: It kind of makes sense why they would have that.
19:15
SPEAKER_01: Right.
19:15
SPEAKER_01: Many times it's for security purposes,
19:17
but sometimes it's because you literally have to take something
19:20
out to a ship or a
19:21
SPEAKER_01: submarine and install it there,
19:22
SPEAKER_01: right?
19:23
SPEAKER_00: That's a good way to ship on a ship.
19:25
SPEAKER_01: Yeah, right?
19:27
And so there,
19:28
we say,
19:29
okay,
19:29
well,
19:29
then what do we want to consider our deployment pipeline?
19:32
When we want to think about optimizing and improving this,
19:35
what should that be?
19:36
SPEAKER_01: Now,
19:37
SPEAKER_01: at some point,
19:37
you do want to consider flying out to a ship or a submarine and
19:41
installing it.
19:42
SPEAKER_01: But for the teams on the ground,
19:44
you can go until that pre-prod environment because that's
19:47
within the scope of control.
19:48
SPEAKER_01: And that,
19:49
SPEAKER_01: for all intents and purposes,
19:50
SPEAKER_01: the way they're thinking about it,
19:52
SPEAKER_01: that is what they're thinking about.
19:54
SPEAKER_01: Now, the same is true for...
19:57
like iPhone and Android apps,
19:59
you can't push to the app store every single day.
20:03
SPEAKER_01: That's not real, right?
20:05
SPEAKER_01: But you can go to pre-prod and that can give you great insights.
20:09
So don't get super caught up in like precise definitions
20:15
because we can always accommodate,
20:17
SPEAKER_01: right?
20:18
SPEAKER_01: Now, your other question is how far should you go and what else is there?
20:23
SPEAKER_01: Now, sometimes folks also tell me that they
20:26
SPEAKER_01: hate DORA, the DORA 4, because it just tells them how bad they're doing.
20:29
SPEAKER_01: But that's where the rest of the research program comes in, right?
20:32
SPEAKER_01: Because there's a BFD,
20:34
a big friendly diagram that kind of points to the many things
20:38
that can improve
20:40
SPEAKER_01: these four metrics.
20:41
SPEAKER_01: Things like improving automated testing,
20:43
SPEAKER_01: things like improving continuous integration,
20:46
SPEAKER_01: things like improving your cloud deployments and your cloud usage.
20:52
SPEAKER_01: So it can help
20:53
SPEAKER_01: It's a signal, but it's not the action, right?
20:56
SPEAKER_01: How well are we doing?
20:58
And it can be good because as you generally start to improve,
21:01
you start to improve.
21:02
If there's a huge drop-off,
21:04
you should probably take a look at your system,
21:06
right?
21:07
SPEAKER_01: So from a signal perspective, it's pretty good.
21:11
SPEAKER_01: It's not everything though.
21:12
And one thing I always try to point out is that DORA only goes
21:15
from commit to production.
21:17
SPEAKER_01: Now, back when we were starting this, this was like 2013.
21:22
That end-to-end software systems had not been studied very much
21:26
and measured
21:26
SPEAKER_01: because it's so,
21:28
SPEAKER_01: so difficult to measure them.
21:29
And so we focused on that part of the engineering system
21:32
because we thought we
21:32
SPEAKER_01: could have some really good impact,
21:33
SPEAKER_01: deliver some good insights.
21:36
Also because that's the part of the process that can be highly engineered,
21:39
right?
21:42
SPEAKER_01: We want things to work better and be faster,
21:44
but also sometimes we just need to take 30 minutes to go walk
21:47
around outside to
21:48
SPEAKER_01: solve a problem.
21:49
SPEAKER_01: Sometimes we're sketching on a notebook to solve a problem.
21:51
SPEAKER_01: Sometimes we're having discussions around prioritization.
21:55
SPEAKER_01: Once you commit, though...
21:58
SPEAKER_01: you should have opportunities to optimize all of that system.
22:03
SPEAKER_01: It should be high predictability and low variability.
22:06
And so that is also a nice way to think about that because if
22:08
your tool chain is
22:10
SPEAKER_01: really out of whack,
22:11
SPEAKER_01: if it requires a ton of manual intervention,
22:13
SPEAKER_01: if all of your tests and your builds are failing,
22:15
SPEAKER_01: if it's highly,
22:16
SPEAKER_01: highly variable,
22:17
sometimes it's super fast and sometimes it absolutely breaks
22:19
and you can't figure
22:20
SPEAKER_01: out why.
22:21
SPEAKER_01: It's a great place to start because you can engineer the heck out of it.
22:24
SPEAKER_01: Okay.
22:24
SPEAKER_01: But it's not everything.
22:26
SPEAKER_01: Developer experience ends up being incredibly important.
22:29
Now it's related because if you can't get feedback on something
22:31
for two weeks and
22:33
SPEAKER_01: then you find out that,
22:34
SPEAKER_01: you know,
22:34
SPEAKER_01: the thing you committed,
22:36
SPEAKER_01: you thought you were done with,
22:37
SPEAKER_01: you were no longer done with,
22:38
SPEAKER_01: I have to drop all of my work.
22:39
SPEAKER_01: I have to reset my environment.
22:40
SPEAKER_01: I have to get my head back in the right head space.
22:43
SPEAKER_01: I have to figure out everything that's happening.
22:45
SPEAKER_01: So fast feedback is important,
22:46
SPEAKER_01: but the rest of the developer experience,
22:48
SPEAKER_01: the holistic developer experience is also important.
22:51
SPEAKER_01: Can I set up an environment quickly?
22:52
SPEAKER_01: Can I provision the resources that I need quickly?
22:54
SPEAKER_01: Can I write code and do local build and test quickly with good insights?
22:59
And so that's where identifying the opportunities to really
23:04
understand and get to,
23:06
SPEAKER_01: I like to look at this as a constraints problem,
23:08
SPEAKER_01: theory of constraints.
23:09
SPEAKER_01: Where are my biggest blockers?
23:11
Because I can't fix everything,
23:12
and I sure can't fix everything in the next month.
23:15
SPEAKER_00: Do I understand correctly that Dora started with developer productivity,
23:22
SPEAKER_00: if you will,
23:23
SPEAKER_00: as you said,
23:24
SPEAKER_00: until we ship,
23:25
SPEAKER_00: just to get some data from there?
23:27
And a lot of the research that has been happening since
23:30
thinking about space,
23:31
SPEAKER_00: thinking of the DevX framework,
23:33
thinking about some new things coming are starting to look at
23:37
not just this,
23:38
SPEAKER_00: but also the other part or get a piece of the puzzle.
23:41
SPEAKER_00: I'm just putting it here.
23:42
SPEAKER_00: As you mentioned, developer productivity and how all of this works.
23:47
SPEAKER_00: is affecting one or the other.
23:49
SPEAKER_00: Is this a fair way to say where the space is progressing?
23:53
SPEAKER_00: Or how do you see things evolving since Dora,
23:56
together with space and some of the newer research that you're
23:59
embarking on?
24:00
SPEAKER_01: Yeah.
24:01
So a lot of people have studied various ways to improve the
24:06
active writing software,
24:07
right?
24:07
SPEAKER_01: I don't want to say development, because development is a lot of things.
24:09
SPEAKER_01: But when you're coding in an IDE, what are ways that we can improve that?
24:15
But I think there has been kind of this continued evolution
24:18
both with the tools
24:19
SPEAKER_01: that we have,
24:20
SPEAKER_01: with the processes we have,
24:22
with the things that we've learned prior to think about this
24:26
from an end-to-end
24:28
SPEAKER_01: type of a view,
24:30
SPEAKER_01: right?
24:30
SPEAKER_01: So not just commit and not just writing code.
24:33
Although many people focus on that and they're the experts and
24:35
they can tell you
24:36
everything down to the millisecond of how long things take and
24:39
like at what point
24:41
SPEAKER_01: you lose focus and you break concentration.
24:43
So I think all of that,
24:45
all of these individual pieces are very,
24:46
very important.
24:48
SPEAKER_01: And for some folks,
24:50
SPEAKER_01: myself included,
24:51
SPEAKER_01: my team,
24:52
we like to look at the full experience to kind of test what
24:56
that looks like.
24:56
And I think it's fair to say that that is part of the evolution
24:59
of space is we had
25:02
SPEAKER_01: learned so much about the commit to prod.
25:08
SPEAKER_01: And how else do we want to think about that?
25:09
And I had been chatting with so many companies for so long that
25:12
kept saying,
25:12
SPEAKER_01: well,
25:12
SPEAKER_01: how do I define a metric then?
25:14
SPEAKER_01: How do I pick the right thing?
25:15
SPEAKER_01: Or once you're in DORA and let's say DORA,
25:18
SPEAKER_01: you know,
25:18
SPEAKER_01: the analysis,
25:19
some kind of constraint analysis tells you that build is the
25:22
biggest problem or PR
25:23
SPEAKER_01: is the biggest problem.
25:24
SPEAKER_01: Then the question is, well, how do I measure it?
25:26
And so space was useful here because it can measure the entire
25:29
developer end-to-end tool chain.
25:31
SPEAKER_01: Or you can just come up with space metrics specific to PRs.
25:36
SPEAKER_01: How satisfied are people with the PR assignment process?
25:39
SPEAKER_01: What's the outcome of PRs, right?
25:40
Like what percentage of PRs get through within a day,
25:45
a certain timeframe or number of viewers?
25:47
SPEAKER_01: Activity metrics, how many PRs do people commit?
25:50
SPEAKER_01: How many PRs are they assigned?
25:52
SPEAKER_01: Communication, right?
25:53
SPEAKER_01: Like what does the communication piece look like?
25:54
SPEAKER_01: And then efficiency and flow, how long does it take?
25:56
SPEAKER_01: Where are the delays?
25:58
SPEAKER_01: So you can apply space to a specific component, right?
26:02
SPEAKER_01: as well.
26:03
SPEAKER_01: And it turns out, when I went back and looked, Dora is an instance.
26:08
SPEAKER_01: It's an instantiation of space.
26:11
SPEAKER_00: Oh, wow.
26:11
SPEAKER_00: That's kind of reassuring.
26:13
SPEAKER_01: Yeah.
26:13
SPEAKER_01: So deploy frequency is a count, right?
26:15
SPEAKER_01: That's an activity metric.
26:16
SPEAKER_01: Lead time to deploy is efficiency and flow.
26:19
SPEAKER_01: And then change, fail, and time to restore are both performance metrics.
26:23
SPEAKER_00: So getting into developer experience,
26:28
SPEAKER_00: As you mentioned,
26:30
SPEAKER_00: this is becoming increasingly important,
26:33
SPEAKER_00: especially at mid-sized companies and larger companies.
26:35
SPEAKER_00: Maybe startups don't care as much just yet.
26:37
SPEAKER_00: But what does this term mean to you?
26:41
SPEAKER_00: What is a place that has good developer experience?
26:45
SPEAKER_01: I think developer experience is what...
26:51
SPEAKER_01: A developer goes through, like, what's their lived experience?
26:55
SPEAKER_01: I'm very, very reciprocal in using the same word.
26:59
But in research,
26:59
we like to talk about,
27:00
like,
27:00
what's a lived experience,
27:02
right?
27:02
SPEAKER_01: What does it feel like and what is it like to do the job day to day?
27:06
SPEAKER_01: Um,
27:07
now many times people kind of conflate like a good developer
27:09
experience with just
27:10
SPEAKER_01: using the word developer experience,
27:12
but I do think it's important to think about them at least a
27:14
little bit separately
27:15
SPEAKER_01: because the experience is just what the work is,
27:18
SPEAKER_01: right?
27:19
SPEAKER_01: Could be good.
27:19
SPEAKER_01: It could be bad.
27:20
SPEAKER_01: That's what it is.
27:21
SPEAKER_01: And then if we want to improve it,
27:23
SPEAKER_01: we can look for ways where we can identify ways,
27:25
SPEAKER_01: uh,
27:26
SPEAKER_01: that take away from,
27:28
SPEAKER_01: or in inject negative,
27:30
SPEAKER_01: uh,
27:32
SPEAKER_01: like not wanted things in that developer experience,
27:35
SPEAKER_01: which often looks like friction,
27:37
SPEAKER_01: which often looks like blockers and barriers,
27:39
SPEAKER_01: which often looks like confusion,
27:42
SPEAKER_01: right?
27:42
If we can't find information,
27:44
if we can't find the docs,
27:45
if we can't find any of the things.
27:47
SPEAKER_01: then that gives us an improving developer experience.
27:51
Or sometimes there's a degradation in developer experience when
27:53
we roll out.
27:54
SPEAKER_01: I'm sure people will think about this, right?
27:55
When a company rolls out a new HR system or a new expense
27:59
system or a new budgeting system,
28:01
SPEAKER_01: it's going to slow me down.
28:02
And that might not be what people typically think of as my role
28:05
as a developer.
28:06
But if I need to ever interact with these systems with a type
28:09
of regularity,
28:10
SPEAKER_01: you've got an initial learning curve,
28:11
SPEAKER_01: which
28:13
SPEAKER_01: is understandable,
28:14
but if that delay or if that friction remains for an extended
28:17
period of time,
28:20
SPEAKER_01: that is going to affect the way I do my work.
28:22
SPEAKER_00: It will affect the time that I have available.
28:24
SPEAKER_00: Just as you say, an example that came to my mind.
28:26
SPEAKER_00: When I worked at Uber, when I started, things were pretty kind of free.
28:30
You could access a lot of systems and there was some logging of
28:34
accessing sensitive data.
28:35
When you went to profile,
28:37
you had to just write quickly while you're doing it and then boom.
28:40
SPEAKER_00: And then as Uber grew, obviously they needed more compliance.
28:43
So what happened is when you wanted to access that systems or
28:46
even data as I was
28:47
SPEAKER_00: debugging customer issue,
28:49
SPEAKER_00: I needed to fill it out.
28:50
SPEAKER_00: And my manager or my director had to approve it.
28:53
SPEAKER_00: Now, suddenly stuff started happening.
28:55
SPEAKER_00: We're like, okay, I need to just fix this bug.
28:58
And I couldn't because my director or my manager was traveling
29:02
and it was like two
29:03
SPEAKER_00: days because then the skip level and suddenly I lost context.
29:05
SPEAKER_00: I didn't get back.
29:06
SPEAKER_00: And I mean, this policy change that had nothing to do with developer.
29:12
SPEAKER_00: It was just really like hammering down.
29:14
SPEAKER_00: I never thought of it as the developer experience was there.
29:17
SPEAKER_00: I was like, oh, there's just, you know, things.
29:19
SPEAKER_00: But the result was like things were tasks were getting done slower,
29:22
SPEAKER_00: especially these debugging tasks.
29:24
SPEAKER_00: That's an interesting way to think of it.
29:26
SPEAKER_00: I don't think we thought of it like that.
29:27
SPEAKER_01: And I love that you bring up not just the time delay,
29:30
SPEAKER_01: but also the cognitive implications,
29:32
SPEAKER_01: right?
29:32
How does this affect your cognitive load in terms of having to
29:35
come back to it
29:35
SPEAKER_01: later and regain context?
29:37
SPEAKER_01: But also,
29:38
SPEAKER_01: maybe this is just me,
29:39
but I'll joke that I just end up having to use computers in
29:42
anger too often,
29:43
SPEAKER_01: right?
29:43
SPEAKER_01: Because when it keeps breaking,
29:44
I can handle so much frustration and still be at my best in
29:48
terms of creativity.
29:49
SPEAKER_01: And sometimes it inspires me.
29:50
SPEAKER_01: But at some point, I'm like,
29:52
SPEAKER_01: Not today.
29:53
SPEAKER_01: We're done.
29:55
SPEAKER_01: I've got to jump over to some administrivia that needs done anyway.
29:58
And it just doesn't require the type of problem solving and
30:02
creativity and
30:03
SPEAKER_01: innovation that great work does.
30:06
And I can't do great work when I'm just surrounded by friction
30:10
and interruptions
30:11
SPEAKER_01: and broken systems and unnecessary process.
30:14
SPEAKER_01: Some of it's good.
30:15
SPEAKER_01: But also, how can we find ways to minimize that process?
30:20
SPEAKER_01: Security and compliance is super important.
30:22
Are there ways where any environment that gets spun up
30:25
automatically has all of the
30:28
SPEAKER_01: or most of the preconditions in the settings done versus a developer,
30:32
every single developer having to do it manually every single
30:34
day and every single time,
30:36
SPEAKER_01: multiple times a day?
30:37
SPEAKER_00: Now,
30:38
SPEAKER_00: in your experience,
30:38
SPEAKER_00: because you talk with a lot of different companies,
30:41
SPEAKER_00: large and small,
30:42
SPEAKER_00: you also work at some very well-known ones.
30:46
SPEAKER_00: Again, pretty pointed question.
30:47
Do you think developer experience in general is better at some
30:51
of the largest companies,
30:53
SPEAKER_00: you know,
30:53
SPEAKER_00: the Facebooks,
30:54
SPEAKER_00: the Microsofts,
30:55
SPEAKER_00: the Metas,
30:56
SPEAKER_00: et cetera,
30:57
SPEAKER_00: or at small startups?
30:59
SPEAKER_00: Which,
30:59
SPEAKER_00: you know,
30:59
SPEAKER_00: like these big companies that I mentioned,
31:01
they do invest a lot in developer productivity and sometimes
31:04
smaller companies look
31:04
SPEAKER_00: at them as envy.
31:05
SPEAKER_00: They have platform teams,
31:06
SPEAKER_00: they build custom tools,
31:07
SPEAKER_00: they have teams that measure all these things and try to improve it.
31:11
Startups,
31:11
you know,
31:12
small like two,
31:12
three,
31:13
five person startup has none of that.
31:14
SPEAKER_00: They just kind of do whatever they can.
31:17
SPEAKER_00: So do you see a difference between them?
31:20
SPEAKER_00: Because sometimes startups move pretty darn fast.
31:22
SPEAKER_01: Yeah, I will say yes and no.
31:24
And a joke I used to make all the time when people would come
31:28
to me with Dora and
31:28
SPEAKER_01: they would say,
31:28
SPEAKER_01: well,
31:29
this can't apply to me because I'm at a startup or there's no
31:31
way this could apply
31:32
SPEAKER_01: to me because I'm in a large company.
31:33
SPEAKER_01: There has to be something different.
31:34
SPEAKER_01: I'm like, well, startups...
31:36
SPEAKER_01: tend to be faster.
31:37
SPEAKER_01: They have less bureaucracy.
31:38
SPEAKER_01: They have less levels.
31:40
SPEAKER_01: Large companies have more funding.
31:41
SPEAKER_01: They have more resources.
31:42
They have more ways to help improve what that developer
31:45
experience looks like.
31:46
SPEAKER_01: So TLDR, no excuses.
31:49
SPEAKER_01: But what do I see?
31:50
SPEAKER_01: I see a mix, right?
31:53
SPEAKER_01: So I've been at a large company that had Google, right?
31:58
SPEAKER_01: Probably the best developer experience I've ever seen.
32:00
SPEAKER_01: It was incredible.
32:01
SPEAKER_01: I was committed to coding the first day.
32:05
SPEAKER_01: It was great.
32:07
They have really taken time to invest in that and prioritize
32:10
that and research that,
32:11
right?
32:12
I worked with some incredible researchers there that did great
32:15
works here.
32:16
SPEAKER_01: Jasper.
32:19
SPEAKER_01: Emerson Murphy Hill, he's at Microsoft now.
32:22
SPEAKER_01: They care deeply about that.
32:24
SPEAKER_01: And it's evident through almost every system that they have.
32:27
SPEAKER_01: They do not tolerate friction.
32:29
SPEAKER_01: I've been at other large companies and advised,
32:31
SPEAKER_01: especially I've seen,
32:32
SPEAKER_01: you know,
32:32
SPEAKER_01: as I kind of talk to large companies,
32:35
SPEAKER_01: some of them are very good and some of them are...
32:39
SPEAKER_01: Very bad.
32:40
SPEAKER_01: Like it's, I don't want to say bad.
32:41
It's not a value judgment,
32:42
but it's incredibly difficult to get work done there.
32:45
SPEAKER_01: Why?
32:46
Because many times they're working in large legacy systems,
32:49
legacy code bases.
32:51
They're working with layers of bureaucracy that slowly built up
32:54
over time.
32:55
SPEAKER_01: And now it just makes it incredibly difficult to do work.
32:58
SPEAKER_02: Yeah.
32:59
SPEAKER_01: Startups have often a slightly different set of constraints.
33:04
SPEAKER_01: They can move fast.
33:05
SPEAKER_01: They can pick whatever tool they want.
33:06
SPEAKER_01: They can just go.
33:08
SPEAKER_01: That also means they have less infrastructure in place.
33:11
SPEAKER_01: They have to build that as they go.
33:12
SPEAKER_01: They have to acquire that.
33:15
SPEAKER_01: If they have to go through security and compliance,
33:16
SPEAKER_01: there is not a team dedicated to security and compliance at DORA.
33:20
We were a tiny little like three person startup that pretended
33:23
to be like a 10 person startup.
33:25
We did sock two compliance because some of our customers
33:27
required that getting a
33:28
SPEAKER_01: sock two and basically two,
33:31
SPEAKER_01: two and a half,
33:31
SPEAKER_01: three person company.
33:32
SPEAKER_01: That's a lot, right?
33:34
SPEAKER_01: Like that.
33:35
SPEAKER_00: Congrats.
33:36
SPEAKER_00: I did that.
33:37
SPEAKER_00: We did that at Uber, but we had a massive team dedicated people.
33:40
SPEAKER_01: It stopped work.
33:42
SPEAKER_01: Now shout out to Jez because he did the lion's share of that work.
33:47
Um,
33:48
but there was no development happening through those,
33:52
those that week or two.
33:53
SPEAKER_01: Right.
33:53
It just doesn't happen unless it was like emergency bug fixes
33:56
or something.
33:57
SPEAKER_02: Yeah.
33:59
SPEAKER_01: So there's, there, there are always trade-offs wherever you go.
34:04
SPEAKER_00: One thing that keeps coming,
34:06
SPEAKER_00: I just keep asking,
34:07
because I've also been thinking and covering and being involved
34:11
in developer
34:11
SPEAKER_00: productivity here and there for many,
34:13
SPEAKER_00: many years now.
34:15
SPEAKER_00: Why is it so darn hard?
34:18
SPEAKER_00: And you've been doing this for a lot longer than I have.
34:21
why it just feels you know like we're software engineers we
34:24
write code we have an
34:25
output okay we're people as well but we're kind of pretty
34:27
organized people it feels
34:29
eventually we should we should converge on something that's
34:31
kind of you know like
34:33
easy enough to understand and and the space should go down a
34:36
little bit you know we
34:37
should understand more and more and there's less ambiguity it
34:39
doesn't seem to be
34:40
happening how are you seeing it why is it hard i think it's
34:45
hard for a lot of
34:45
SPEAKER_01: reasons right one is that
34:48
SPEAKER_01: Most of the work that we do is, let's say, invisible.
34:51
SPEAKER_01: You can't walk down to a production line and see where something breaks.
34:56
SPEAKER_01: That's part of it.
34:57
And the systems that we use and we work with are increasing in complexity.
35:02
SPEAKER_01: When we think about cloud systems,
35:05
that changed the way we had to think about distributed systems
35:09
and orchestrating
35:10
SPEAKER_01: them and architecting them.
35:12
SPEAKER_01: Now, we have AI.
35:14
AI also changes the way we need to think about the
35:16
infrastructure and the pipelines
35:18
SPEAKER_01: and the tooling and the model management,
35:21
SPEAKER_01: right?
35:23
SPEAKER_01: How we think about that and how we account for that,
35:26
just its existence and its emergence with continued growth
35:30
changes the complexity
35:31
SPEAKER_01: of the work that we need to do and the supporting systems that we need.
35:35
SPEAKER_01: Another thing that can be challenging is many times leaders and execs.
35:39
SPEAKER_01: aren't feeling the pain, right?
35:40
SPEAKER_01: How many CEOs have been developers and are currently,
35:45
SPEAKER_01: so some of them are developers,
35:46
but also how many of them are currently using our systems to do
35:50
work and using them
35:52
SPEAKER_01: for at least a week or two,
35:55
SPEAKER_01: right?
35:55
SPEAKER_01: They're not managing their calendars.
35:57
SPEAKER_01: They're not, which they shouldn't be, but if they had to feel that pain,
36:03
SPEAKER_01: I think the motivation would change.
36:05
SPEAKER_00: This is so interesting because when I talk with David Singleton,
36:09
SPEAKER_00: who used to be the CET of Stripe for seven years,
36:12
SPEAKER_00: he told me that every few months,
36:15
SPEAKER_00: he takes a few days where he does
36:18
development work,
36:19
which I found very interesting because not many CTOs at his
36:22
level do it.
36:22
SPEAKER_00: He had thousands of people underneath him.
36:24
SPEAKER_00: At the same time,
36:25
SPEAKER_00: Stripe is one that invests very heavily into platform teams,
36:29
into developer tools that they had one of their AI systems
36:33
early on to help developers.
36:35
SPEAKER_00: And
36:37
SPEAKER_00: I was thinking,
36:37
SPEAKER_00: and he told me,
36:38
SPEAKER_00: look,
36:38
SPEAKER_00: if you're not doing this,
36:40
SPEAKER_00: he said what you're saying,
36:42
SPEAKER_00: but very few people do it.
36:43
SPEAKER_00: So I think that's just a really interesting point.
36:46
SPEAKER_00: I think anyone listening who's in a leadership position,
36:49
SPEAKER_00: especially in a C-level position,
36:51
SPEAKER_00: if you're not doing it,
36:52
SPEAKER_00: there's companies that are doing it a lot better.
36:56
SPEAKER_00: And you might be missing some things.
36:58
SPEAKER_01: Right.
36:58
SPEAKER_01: And in that case, because maybe...
37:01
SPEAKER_01: For whatever reason,
37:02
an executive team can't be carving out several days to do
37:06
development work.
37:07
SPEAKER_01: Courtney Kistler is at Starbucks now.
37:09
SPEAKER_01: I met her back when she was at Nordstrom.
37:12
She had a phrase that has always really kind of stuck with me
37:15
and resonated,
37:15
SPEAKER_01: and it's,
37:16
SPEAKER_01: honor their reality.
37:19
If you ask a handful of developers and engineering managers
37:22
what it's like to work
37:23
SPEAKER_01: in systems,
37:24
SPEAKER_01: you don't even need a survey,
37:24
SPEAKER_01: right?
37:25
SPEAKER_01: Like if you want to send someone,
37:26
SPEAKER_01: if you want to chat with a handful of them,
37:28
SPEAKER_01: you need to honor that reality.
37:30
SPEAKER_01: You might not agree with it,
37:31
SPEAKER_01: but that doesn't mean that isn't their lived experience of doing work.
37:36
SPEAKER_01: And sometimes that's going to be the best proxy you can get,
37:38
but getting it is super important and listening to it and being
37:41
curious about it is
37:43
SPEAKER_01: super important.
37:44
SPEAKER_00: This episode was brought to you by Sentry.
37:47
SPEAKER_00: Buggy lines of code and long API calls are impossible to debug,
37:50
SPEAKER_00: and random app crashes are things no software engineer is a fan of.
37:54
This is why over 4 million developers use Sentry to fix errors
37:58
and crashes and
37:59
SPEAKER_00: solve hidden or tricky performance issues.
38:02
SPEAKER_00: Sentry costs debugging time in half,
38:04
SPEAKER_00: no more soul-crushing lock sifting,
38:06
SPEAKER_00: or vague user reports like,
38:08
SPEAKER_00: it broke,
38:08
SPEAKER_00: fix it.
38:10
SPEAKER_00: Get the context you need to know,
38:11
SPEAKER_00: what happened,
38:12
SPEAKER_00: when it happened and the impact,
38:14
SPEAKER_00: down to the device,
38:15
SPEAKER_00: browser and even a replay of what the user did before the error.
38:18
Sentry will alert the right dev on your team with the exact
38:21
broken line of code so
38:22
they can push a fix fast or let autofix handle the repetitive
38:26
fixes so your team
38:27
SPEAKER_00: can focus on the real problems.
38:30
Sentry helped monday.com reduce their errors by 60% and sped up
38:33
time to resolution
38:34
SPEAKER_00: for next door by 45 minutes per dev per issue.
38:38
Get your whole team on Sentry in seconds by heading to
38:40
sentry.io slash pragmatic.
38:43
SPEAKER_00: That is S-E-N-T-R-Y dot I-O slash pragmatic.
38:48
Or use the code pragmatic on sign up for three months on the
38:50
team plan and 50,000
38:51
SPEAKER_00: errors per month for free.
38:54
SPEAKER_00: Now,
38:55
most people listening who are software engineers or engineering
38:58
managers who are
38:58
SPEAKER_00: pretty hands-on or stay close to the teams,
39:01
SPEAKER_00: they will agree with both of us that this is important.
39:05
SPEAKER_00: Developer productivity is important.
39:06
SPEAKER_00: It's hard.
39:07
And if you're above a certain size,
39:10
mid-size or above,
39:11
you kind of need some investment.
39:13
SPEAKER_00: It's nice to have dedicated folks who help out.
39:16
May that be provisioning infrastructure,
39:18
standardizing things,
39:19
or building systems.
39:21
SPEAKER_00: The number one question,
39:22
SPEAKER_00: or not the number one,
39:23
SPEAKER_00: but a very frequent question I get from people,
39:26
SPEAKER_00: especially engineering managers,
39:27
SPEAKER_00: senior managers,
39:28
SPEAKER_00: directors,
39:28
SPEAKER_00: is I see this as a problem.
39:30
I think we should invest more into developer productivity or
39:34
internal tools that
39:35
SPEAKER_00: help us with developer productivity platform teams.
39:38
SPEAKER_00: I need to get funding for this, and I like to make a case that
39:41
SPEAKER_00: What is your suggestion of doing so?
39:44
And I'm asking you as someone who's advised a lot of companies
39:48
and seen them,
39:50
what are ways you've seen engineering making the case that
39:54
there should be an
39:55
investment in platform teams that directly or indirectly
39:58
address some of these topics?
40:00
Because,
40:00
you know,
40:00
the business,
40:01
the usual pushback will be engineering expensive enough.
40:04
SPEAKER_00: We gave you enough resources.
40:06
SPEAKER_00: You should just start it out as you are.
40:09
SPEAKER_01: Yep.
40:10
In general,
40:11
I would say some of the best ways to communicate include both
40:16
data and stories.
40:18
SPEAKER_01: If we can have some numbers, that can be incredibly helpful.
40:21
SPEAKER_01: It can drive it home.
40:21
SPEAKER_01: If you only have numbers...
40:25
SPEAKER_01: it becomes abstract.
40:26
SPEAKER_01: It's just another spreadsheet or a number on a document.
40:29
SPEAKER_01: Now, if we only have stories, it can be easily dismissed as a one-off.
40:33
SPEAKER_01: That's just that one thing.
40:34
SPEAKER_01: Someone is just like a disgruntled engineer, right?
40:37
SPEAKER_01: Someone, some team just doesn't like, it was just that week.
40:42
And many times resourcing will come down to like,
40:46
what are the trade-offs?
40:47
SPEAKER_01: And you want to take a realistic look.
40:51
SPEAKER_01: Are your engineers having to
40:54
SPEAKER_01: you know, rewrite, like re-roll scripts every single time they deploy?
40:59
SPEAKER_01: How much time does that take?
41:00
SPEAKER_01: So go around to do just a rough estimate.
41:03
SPEAKER_01: Like at Chef, we did this early on and we just had a spreadsheet.
41:07
SPEAKER_01: This does not have to be, you know, super intense or rigorous.
41:10
SPEAKER_01: We had a spreadsheet where at the end of the week,
41:12
we asked engineering managers to like kind of chime in on like
41:15
about how long
41:16
SPEAKER_01: certain things took.
41:17
Because then you can do some kind of like rough back of the
41:20
napkin math to say,
41:21
SPEAKER_01: this is how much time we're losing in aggregate.
41:24
SPEAKER_01: Now,
41:24
SPEAKER_01: if you want us to repurpose engineers,
41:26
SPEAKER_01: we can,
41:27
but this is the impact on the product because we can come with
41:31
a recommendation,
41:32
SPEAKER_01: but it's going to be someone else's decision.
41:34
And the more information we can give them about the trade-offs
41:36
that exist in the system,
41:38
SPEAKER_01: the more valuable it will be and then pair it with a story.
41:41
SPEAKER_01: Now it can be an actual story.
41:43
SPEAKER_01: It can be a quote of a comment or it can be a very true amalgamation,
41:48
like almost a persona of what a day in the life of development
41:51
looks like now and
41:53
SPEAKER_01: then what it could look like later.
41:55
And then it comes back to being a decision around where are we
41:58
going to invest the resources?
41:59
SPEAKER_01: We're always in a constrained environment.
42:01
SPEAKER_01: No one has unlimited resourcing.
42:02
SPEAKER_01: Maybe a couple of AI companies, right?
42:05
SPEAKER_01: We always have constraints.
42:06
SPEAKER_01: So what's the best way to do this?
42:08
I will say one additional thing to keep in mind for folks kind
42:12
of kicking this off
42:13
SPEAKER_01: or making decisions about whether they should be investing in this is
42:17
is to acknowledge both to yourself and to leadership that there
42:22
is a tipping point
42:23
SPEAKER_01: in this trade-off where further investments,
42:26
SPEAKER_01: explicit,
42:28
SPEAKER_01: you know,
42:28
SPEAKER_01: dedicated investments...
42:30
SPEAKER_01: won't deliver that much more progress.
42:34
SPEAKER_02: Yeah.
42:35
SPEAKER_01: Which I think is important because there's,
42:37
SPEAKER_01: you know,
42:37
SPEAKER_01: I've talked to execs before who were like,
42:39
SPEAKER_01: I'm not going to take this meeting.
42:41
SPEAKER_01: It's just a black box.
42:42
SPEAKER_01: Everything's broken all the time.
42:44
SPEAKER_01: So being upfront about the fact and saying,
42:46
I know that there are so many problems and we could like spend
42:49
all of the company's
42:50
SPEAKER_01: resources fixing all of it.
42:51
SPEAKER_01: I know that's not what we want to do.
42:53
I am trying to optimize our workflow so that we can deliver
42:57
value to our customers.
42:59
SPEAKER_01: And I think this is going to be a very important leverage point.
43:02
And once we get to a point we'll continually reassess where
43:04
it's good enough,
43:06
SPEAKER_01: we will retain these investments,
43:07
SPEAKER_01: right?
43:07
SPEAKER_01: We'll keep a platform team,
43:09
but we won't need additional investment until it starts
43:12
negatively impacting the business.
43:14
SPEAKER_01: And that kind of opens up a door to have a real conversation around like,
43:20
SPEAKER_01: what are the constraints that we have,
43:21
SPEAKER_01: right?
43:21
SPEAKER_01: Don't just ask for headcount.
43:22
SPEAKER_00: Yeah, I love the both that you're saying, making the case.
43:26
Well,
43:26
first of all,
43:26
making about trade-offs,
43:27
because I think that's a lot easier to work with.
43:30
SPEAKER_00: The fact that mentioning that you can later repurpose teams,
43:34
SPEAKER_00: and I think just the reality right now in the market,
43:37
SPEAKER_00: it's all about efficiency.
43:39
There's now a push with a lot of companies where they're
43:42
hesitant to hire more
43:43
SPEAKER_00: software engineers.
43:44
So I do think folks will need to get a lot more creative and
43:47
also just accept the reality that
43:49
SPEAKER_00: We might have,
43:50
SPEAKER_00: you know,
43:51
SPEAKER_00: we've had a really good run of the last 10 years where we,
43:53
SPEAKER_00: like so many places,
43:54
SPEAKER_00: got almost unlimited investment in these areas as well,
43:58
SPEAKER_00: which meant that engineers didn't have to feel some of the pain.
44:01
We might need to,
44:02
you know,
44:02
just decide that some of this pain,
44:03
it is here.
44:04
SPEAKER_00: Let's solve it.
44:05
SPEAKER_00: Let's try to, you know, work around it.
44:07
SPEAKER_00: But let's solve the most important pain points.
44:09
SPEAKER_00: So I feel there might be a bit of a turning point.
44:11
SPEAKER_00: A lot of companies,
44:12
SPEAKER_00: not all of them,
44:13
SPEAKER_00: were like what you've been used to,
44:15
SPEAKER_00: you know,
44:15
SPEAKER_00: getting resources to just building teams that helped it.
44:19
Either help yourself or or sort it or look at tools that can
44:24
make it change up your workflow.
44:26
SPEAKER_00: So is this also a little bit what you might be seeing?
44:29
SPEAKER_01: Yeah.
44:29
SPEAKER_00: Yeah, absolutely.
44:31
SPEAKER_00: So talking about highly productive companies in now in 2025.
44:39
SPEAKER_00: Some of the most productive companies that you would say are teams,
44:42
SPEAKER_00: engineering teams,
44:43
SPEAKER_00: I would say,
44:43
SPEAKER_00: that just get stuff done.
44:45
SPEAKER_00: What are some of the common traits that you see across them?
44:48
SPEAKER_00: And are there some common practices that they use?
44:51
SPEAKER_00: May they be a startup or a team within a larger company?
44:56
Many times what I get instead from a company is how do I make
44:59
my company high performing,
45:01
SPEAKER_01: right?
45:01
SPEAKER_01: But it really is very contextual to teams, right?
45:04
SPEAKER_01: Because there are team technologies,
45:06
SPEAKER_01: there are team onboarding,
45:07
SPEAKER_01: there are team cultures,
45:08
SPEAKER_01: there are team norms.
45:09
SPEAKER_01: So it can be a group of teams, right?
45:12
But I often see very different,
45:16
let's say like performance profiles,
45:17
right?
45:18
SPEAKER_01: Across a company.
45:20
SPEAKER_01: Now, what makes them great?
45:21
SPEAKER_01: And you can be great anywhere.
45:23
SPEAKER_01: I've seen a great team working on mainframes.
45:25
SPEAKER_01: I've seen great teams working on SaaS software.
45:28
SPEAKER_01: I've seen bad teams working on SaaS software.
45:30
SPEAKER_01: I've seen bad performing teams working on- We've all seen it, no?
45:33
SPEAKER_00: We've all seen this, right?
45:34
SPEAKER_00: We might have even been that team.
45:37
SPEAKER_01: Maybe, yes.
45:39
SPEAKER_01: That's a yes.
45:42
The thing that I see in these teams is a few key
45:46
characteristics and values.
45:49
SPEAKER_01: psychological safety is always in there, right?
45:51
SPEAKER_01: I'm sure several of us have heard about that.
45:52
SPEAKER_01: You feel safe taking risks.
45:54
SPEAKER_01: You feel safe asking stupid questions.
45:56
SPEAKER_01: You feel safe doing so many things.
46:00
Another part of psychological safety is depending on your team
46:03
and also knowing
46:04
SPEAKER_01: that your work has purpose.
46:07
SPEAKER_01: And that can come back to asking stupid questions, right?
46:09
SPEAKER_01: Sometimes stupid.
46:10
They're very important questions as like,
46:13
how does the project that we're working on right now
46:16
SPEAKER_01: relate to the group or the business goals and does it.
46:22
I was at a company once where we rolled some of these things
46:24
out and a few
46:25
SPEAKER_01: engineers quietly back channel came to me and they're like,
46:28
SPEAKER_01: I don't think this aligns to any of the key,
46:31
SPEAKER_01: you know,
46:31
SPEAKER_01: the exec OKRs.
46:32
SPEAKER_01: What am I doing?
46:33
SPEAKER_01: And so we were able to have a discussion around it does.
46:37
SPEAKER_01: It's just worded differently.
46:38
And you're not only impacting your line of business,
46:41
you're impacting three others.
46:43
SPEAKER_01: Or I don't see how it aligns at all.
46:46
You should chat with your manager because this might be a good
46:48
opportunity for prioritization.
46:51
SPEAKER_01: So psychological safety is one.
46:53
A very related one is getting curious,
46:57
being very open to asking questions and learning more.
47:00
SPEAKER_01: And I especially see this in large companies,
47:04
SPEAKER_01: being open to the possibility and frankly,
47:07
the high likelihood that the way you do things now is not the
47:11
best way to do things.
47:12
Just because something was difficult or impossible to solve
47:15
five years ago doesn't
47:17
SPEAKER_01: mean it hasn't been solved.
47:18
And so when you open yourself up to the possibility that there
47:21
could be better ways
47:22
SPEAKER_01: to do things,
47:23
SPEAKER_01: there may be other teams doing better than you.
47:27
SPEAKER_01: then you can learn what that is.
47:29
SPEAKER_01: It offers an opportunity for improvement.
47:31
SPEAKER_01: I've worked with teams in larger,
47:34
SPEAKER_01: older orgs where I came in,
47:36
SPEAKER_01: this was a problem,
47:37
SPEAKER_01: and they're like,
47:37
SPEAKER_01: oh,
47:37
SPEAKER_01: well,
47:38
but that's not one we're going to look at because that just
47:40
can't be solved.
47:42
Because the last time they really put their heads down to try
47:44
to solve it,
47:44
SPEAKER_01: it was seven or eight years ago.
47:45
SPEAKER_01: And it's actually been solved now.
47:47
SPEAKER_01: I can point you to conference talks.
47:50
SPEAKER_01: And so being willing to consider and ask several questions ends up being
47:57
SPEAKER_01: super important.
47:58
SPEAKER_01: And then tactically, onboarding ends up being a great indicator, right?
48:02
SPEAKER_01: How long does it take a new hire to commit code?
48:06
SPEAKER_01: And how can we improve onboarding practices and environments?
48:09
Because it often puts a spotlight on things that are really,
48:14
really difficult.
48:15
I was working with one really large org once where they're
48:18
onboarding time for a
48:19
SPEAKER_01: brand new college hire engineer.
48:21
Was the same as a senior developer who just switched divisions
48:26
in the company.
48:28
Because the tooling changes,
48:30
the documentation changes,
48:32
the context they needed to get.
48:34
SPEAKER_00: Wow.
48:34
SPEAKER_01: Was basically non-existent.
48:37
SPEAKER_00: Those two things shouldn't be the same.
48:39
SPEAKER_00: Just to be clear, it was a long time, right?
48:41
SPEAKER_00: We weren't talking a day.
48:42
SPEAKER_00: It was probably like.
48:43
SPEAKER_00: We were talking several months ago.
48:45
SPEAKER_00: Wow.
48:47
SPEAKER_00: Okay.
48:48
SPEAKER_00: That's a real unexpected.
48:50
SPEAKER_00: I mean, that's just terrible.
48:52
SPEAKER_00: Let's say it how it is.
48:52
SPEAKER_01: Some of the best teams were at 60 days.
48:55
SPEAKER_00: Wow.
48:55
SPEAKER_01: Which is not great, right?
48:57
SPEAKER_00: No, it's bad.
48:59
SPEAKER_01: It's bad.
49:00
SPEAKER_01: It's bad.
49:01
SPEAKER_01: And when we worked with them on some...
49:06
SPEAKER_01: general interventions, right?
49:07
SPEAKER_01: What are some of the concrete things they could do to improve it?
49:10
SPEAKER_01: There were basic things like improving documentation,
49:12
SPEAKER_01: improving how easy it is to find documentation,
49:16
SPEAKER_01: creating a dummy exercise for a dummy pull request very,
49:20
SPEAKER_01: very early that has to be completed within the first week or two.
49:25
SPEAKER_01: Now,
49:25
SPEAKER_01: what we found is that even if you know,
49:27
SPEAKER_01: so some research on this has been done at Microsoft,
49:30
SPEAKER_01: my colleague Brian Hough did a bunch of this,
49:31
SPEAKER_01: which is really incredible.
49:33
SPEAKER_01: And shout out to the teams that partnered with us on this.
49:36
SPEAKER_01: Even something like a dummy pull request, right?
49:39
SPEAKER_01: Not dummy, but like fix a title, right?
49:43
SPEAKER_01: ends up being hugely impactful.
49:44
SPEAKER_01: It increases productivity.
49:46
Some measures of easily automated measures of productivity
49:51
through the whole rest
49:51
SPEAKER_01: of the year,
49:52
SPEAKER_01: right?
49:52
There was like a 30 to 50% increase because you've gone through
49:57
the system.
49:58
SPEAKER_01: Now, people push back.
49:59
SPEAKER_01: They're like, well, what if you're gaming the system?
50:01
SPEAKER_01: Great.
50:02
This is the best way to game the system because then someone
50:04
has gone through every
50:05
SPEAKER_01: single step of the process.
50:07
SPEAKER_01: They are now more familiar with it.
50:08
SPEAKER_01: They've done it early on something that...
50:11
was fine but like kind of didn't matter like they didn't have
50:14
to have full context
50:16
of the entire code base to get familiar with the tool chain and
50:20
this is this is
50:22
just so interesting it was great and they could surface obvious
50:24
problems that other
50:25
people uh i when i joined microsoft i joked because some people
50:29
just knew where
50:29
everybody was buried they knew where every document was they
50:32
knew where every
50:32
everything was but it's because they'd been there for 14 years
50:35
and they had a list
50:35
SPEAKER_01: of bookmarks
50:38
SPEAKER_01: bookmarks are not transferable, right?
50:41
SPEAKER_01: Now, I mean, the same was true at IBM.
50:42
The same ends up being true at many large companies and even
50:45
small companies.
50:45
SPEAKER_01: Because if there's just a handful of you, you can ping somebody and ask.
50:50
SPEAKER_00: Yeah.
50:51
SPEAKER_00: When I used to work at where I had a dock of like the things to do.
50:55
And I think most people had after a while because there's so
50:57
many systems.
50:59
SPEAKER_00: But what you said is just so fascinating,
51:01
SPEAKER_00: especially about the onboarding,
51:02
SPEAKER_00: because I didn't think,
51:02
SPEAKER_00: you know,
51:03
SPEAKER_00: as we're talking developer productivity,
51:04
SPEAKER_00: I thought about a lot of things.
51:05
SPEAKER_00: And I think a lot of people are like this.
51:07
SPEAKER_00: They don't think about how onboarding can predict.
51:10
SPEAKER_00: But as you said, it's not just the onboarding, but the internal transfer.
51:13
And one super interesting thing that might tie back to this is
51:17
a famous thing,
51:18
a famous law that a lot of engineering managers will quote as
51:21
Brook's Law,
51:21
SPEAKER_00: which says,
51:22
SPEAKER_00: for a late project,
51:23
SPEAKER_00: if you add more people,
51:25
SPEAKER_00: it's going to be even more late.
51:27
SPEAKER_00: But this is true at most companies where onboarding takes a long time.
51:32
SPEAKER_00: The project is estimated two months.
51:34
If you add three more people,
51:36
they'll take a month to onboard and suck time away,
51:38
et cetera.
51:39
SPEAKER_00: But if you're working at a company,
51:40
SPEAKER_00: and a company that's kind of known for super fast onboarding,
51:45
SPEAKER_00: externally at least,
51:46
SPEAKER_00: is meta.
51:47
SPEAKER_00: New hires will often push something code on day one, day two.
51:50
SPEAKER_00: They have monorepo.
51:52
SPEAKER_00: If you move teams, you will probably work on the same code base.
51:55
SPEAKER_00: If you have an organization where changing teams means that the new...
52:00
the person who changed teams in a day,
52:02
they will push code to a different part of the stack.
52:05
SPEAKER_00: Brooks Law might not entirely apply,
52:08
SPEAKER_00: assuming you can,
52:08
SPEAKER_00: you know,
52:08
SPEAKER_00: pick up parallel tasks,
52:10
SPEAKER_00: which a lot of people don't realize.
52:11
SPEAKER_00: Because I once had this experience at a company where,
52:14
SPEAKER_00: this was actually at Uber,
52:15
SPEAKER_00: where we had people said like,
52:17
SPEAKER_00: you know,
52:18
SPEAKER_00: If you get more people on your team, can you get it done faster?
52:20
SPEAKER_00: And at first I was at Brooks Law, but my director said like, no, no, no.
52:23
SPEAKER_00: Like those people, they're onboarded.
52:24
SPEAKER_00: Like they know it's the monorepo, the mobile monorepo.
52:27
SPEAKER_00: They can do productive on day one.
52:29
SPEAKER_00: And we added more people and we got it done faster.
52:32
SPEAKER_00: And later I was thinking, how did we, because this is 50 years ago.
52:36
SPEAKER_00: It was quite 50 years ago.
52:37
SPEAKER_00: But as you said,
52:38
I think this is the part where you need to be open to things
52:43
potentially changing
52:44
SPEAKER_00: around you for the better.
52:46
SPEAKER_00: Yeah.
52:46
And that's where the question kind of becomes when someone's
52:48
onboarding to a team,
52:49
are they onboarding to the entire system and the entire
52:54
environment in which they work?
52:56
Or are they onboarding to a slightly new team with like maybe a
53:00
slightly different
53:01
SPEAKER_01: culture and learning a different part of the code base?
53:05
SPEAKER_01: but everything else remains unchanged.
53:07
SPEAKER_01: If that's the case,
53:08
SPEAKER_01: maybe not similar productivity on day one,
53:11
SPEAKER_01: but a handful of days to really dig into the new part of the code base,
53:15
SPEAKER_01: new functionality,
53:16
SPEAKER_01: but everything else is just done.
53:19
SPEAKER_01: Whereas sometimes,
53:20
SPEAKER_01: even if you're within the same company,
53:22
SPEAKER_01: you're onboarding to an entirely new system.
53:24
You have maybe the same tool chain,
53:26
but it's configured entirely differently.
53:28
SPEAKER_01: So it's basically a different tool chain.
53:29
SPEAKER_01: You have different request processes.
53:31
SPEAKER_01: You have different documentation.
53:33
SPEAKER_01: If everything's changed, that's onboarding.
53:37
SPEAKER_01: That's new hire onboarding.
53:39
SPEAKER_01: They just already have their login.
53:41
SPEAKER_00: Yeah.
53:43
SPEAKER_00: Related to this, cultural turnarounds.
53:46
A lot of companies that you will turn to experts like yourself
53:49
and seek out
53:50
SPEAKER_00: different methodologies know that we've got a problem.
53:52
SPEAKER_00: We're not very productive.
53:54
SPEAKER_00: We look at our competitors.
53:55
SPEAKER_00: Other companies, they seem to be getting stuff a lot faster.
53:58
SPEAKER_00: We're going to turn around the culture.
53:59
SPEAKER_00: We're going to go from a, you know, like do whatever it needs to be done.
54:03
SPEAKER_00: Have you seen successful turnarounds?
54:06
SPEAKER_00: And if so, how did they happen and over what time frame?
54:10
SPEAKER_00: And,
54:10
SPEAKER_00: you know,
54:11
SPEAKER_00: let's talk about like companies that are like,
54:13
SPEAKER_00: you know,
54:13
SPEAKER_00: like not a tiny startup,
54:16
SPEAKER_00: but like company that like 100 plus people or even bigger.
54:19
And I'm more interested in the kind of the generic learnings
54:22
because what I see is,
54:24
SPEAKER_00: you know,
54:25
SPEAKER_00: people at the top,
54:26
SPEAKER_00: they think like,
54:27
SPEAKER_00: oh,
54:27
SPEAKER_00: yeah,
54:27
SPEAKER_00: we'll just hire some consultants,
54:28
SPEAKER_00: you know,
54:29
SPEAKER_00: we'll become a...
54:30
SPEAKER_00: tech first company in no time.
54:32
SPEAKER_00: I've actually worked at a company like this a long time ago,
54:34
SPEAKER_00: a financial company that kind of advertised itself.
54:36
SPEAKER_00: And on the ground,
54:37
SPEAKER_00: developers are just rolling their eyes like,
54:39
SPEAKER_00: okay,
54:39
SPEAKER_00: yeah,
54:39
SPEAKER_00: we're getting,
54:40
SPEAKER_00: you know,
54:40
SPEAKER_00: another agile training or whatever that is.
54:43
And you feel not much is changing and there's usually a
54:45
disconnect between the two.
54:47
SPEAKER_00: And there are some companies that over time,
54:49
SPEAKER_00: by the way,
54:50
SPEAKER_00: like they do manage to become a lot more tech first companies,
54:53
SPEAKER_00: like old school companies who you'd be surprised,
54:55
SPEAKER_00: you know,
54:55
SPEAKER_00: like.
54:56
SPEAKER_00: A lot of banks.
54:58
SPEAKER_00: are here.
54:59
I've worked with a lot of banks that have undergone that
55:01
transformation successfully.
55:03
SPEAKER_00: For the ones that succeed,
55:07
SPEAKER_00: what do you usually see in terms of success,
55:10
SPEAKER_00: the time it takes,
55:11
SPEAKER_00: and what do engineers feel on the ground?
55:14
SPEAKER_01: You're right.
55:14
SPEAKER_01: I ended up talking to a lot of folks about it.
55:16
I think it's important to realize that if you're trying to
55:20
change the tech culture,
55:23
SPEAKER_01: that's not going to be possible.
55:25
SPEAKER_01: And by that,
55:25
SPEAKER_01: I mean,
55:26
you need to think about changing the culture of the company or
55:30
the culture of the organization.
55:32
You can do things to have it and help it influence the
55:37
technology culture,
55:38
but it's really difficult to try to just carve out the
55:40
technology culture and
55:41
change the way people think about software without changing the
55:46
broader culture.
55:47
SPEAKER_01: And I will say one, you know,
55:49
SPEAKER_01: I think notable cultural transformation is at Microsoft, right?
55:52
Satya Nadella has really changed what Microsoft is like
55:56
compared to the Balmer years.
55:59
SPEAKER_00: I used to work at the Balmer time.
56:00
SPEAKER_00: It was very different.
56:01
SPEAKER_00: Oh my gosh.
56:02
SPEAKER_01: It's very, very different.
56:03
Now,
56:04
there are a handful of things you could do in general and then
56:07
specific to tech.
56:08
I'll probably focus on the specific to tech ones because I'm
56:11
sure other people are
56:11
SPEAKER_01: familiar with communication and commitment and those types of things.
56:16
SPEAKER_01: When we think about technology,
56:18
I think we have additional opportunities because there we can
56:22
look at some previous
56:25
SPEAKER_01: research that's been done.
56:26
SPEAKER_01: John Shook did some great work.
56:27
SPEAKER_01: We used to think that to change a culture, you would change...
56:32
SPEAKER_01: or to change the way people do work.
56:33
SPEAKER_01: You would change the culture.
56:34
SPEAKER_01: You would change the way you talk about it.
56:36
And then that would just trickle down into the way people build
56:41
tools and kind of
56:43
SPEAKER_01: execute their work.
56:45
SPEAKER_01: What he found is that when you change the way people do their work,
56:49
SPEAKER_01: that bubbles up to cultural impacts.
56:52
And I think that's where having an environment where these
56:55
changes are allowed and
56:57
SPEAKER_01: supported is super important,
56:58
SPEAKER_01: right?
56:58
SPEAKER_01: From that broader environment.
57:00
SPEAKER_01: Um,
57:00
SPEAKER_01: but you know,
57:01
SPEAKER_01: as an example,
57:02
SPEAKER_01: uh,
57:02
SPEAKER_01: you know,
57:02
SPEAKER_01: people talk about,
57:03
SPEAKER_01: uh,
57:04
SPEAKER_01: knew me all the time,
57:04
SPEAKER_01: but I,
57:05
SPEAKER_01: the cart factory,
57:07
SPEAKER_01: but I love talking about it,
57:08
SPEAKER_01: kind of situating it in companies.
57:10
If you're in a company or in an organization or a team or a
57:14
situation where your
57:15
SPEAKER_01: tool chain is just incredibly slow,
57:18
SPEAKER_01: it's very high friction.
57:19
SPEAKER_01: There's a lot of process to go through.
57:22
SPEAKER_01: You can talk about doing agile, um,
57:26
SPEAKER_01: And it's going to help to a point, right?
57:28
SPEAKER_01: People can wrap their minds around it.
57:30
SPEAKER_01: They can think about what it means.
57:32
SPEAKER_01: But actually changing...
57:34
the core culture isn't going to happen if you're working in
57:37
systems that are incredibly slow.
57:40
SPEAKER_01: Now,
57:40
if you introduce new tools and new ways of working that are
57:45
actually much faster,
57:46
SPEAKER_01: that actually make it easier to communicate with others,
57:48
SPEAKER_01: that make it easier to surface and fix problems early,
57:51
that make any prioritization decisions that you're talking
57:54
about fairly easy to get done,
57:57
SPEAKER_01: now you've changed people's lived experience.
57:59
SPEAKER_01: You've changed what it feels like to do the work
58:03
SPEAKER_01: And that's paired with larger communications,
58:07
SPEAKER_01: larger education,
58:08
SPEAKER_01: larger commitments to what is happening.
58:11
SPEAKER_01: Now,
58:12
commitments is also important because are you committing to
58:14
improving the difficult
58:16
pieces of the technical stack and the workflow to get to the
58:18
desired outcome?
58:19
SPEAKER_01: Are you investing in the communication that's required?
58:23
SPEAKER_01: You have to say things at least three times for it to kind of land.
58:26
And it needs to come from several areas and several levels of
58:29
the business.
58:30
And so I think those two together can be incredibly powerful,
58:33
but also needs to be explicit.
58:36
SPEAKER_00: And just being a little bit more specific,
58:39
say I'm either a staff senior plus engineer senior staff
58:43
engineer principal
58:44
SPEAKER_00: engineer who joined a tech company or an engineering manager,
58:47
SPEAKER_00: a senior engineering manager,
58:48
SPEAKER_00: not a C-level,
58:49
SPEAKER_00: right?
58:49
SPEAKER_00: So I can't set the things.
58:52
SPEAKER_00: But
58:52
SPEAKER_00: In a sense, what is it that I could do?
58:54
SPEAKER_00: I've kind of onboarded.
58:56
SPEAKER_00: I took it in.
58:57
SPEAKER_00: I didn't rush to copy, paste, whatever I saw at the previous place.
59:01
This is a frequent criticism of big tech employees joining startup,
59:04
bringing process.
59:05
SPEAKER_00: But we didn't do that mistake.
59:07
I took it all in and I realized there are things that are just
59:10
really inefficient.
59:12
I can see if I was a CTO,
59:14
I could do stuff,
59:15
but obviously I cannot do that.
59:17
SPEAKER_00: What are ways that I can...
59:19
SPEAKER_00: improve the engineering culture, the efficiency?
59:23
SPEAKER_00: And is it going to be what you just said,
59:26
which is around the lines of starting to change my team's lived experience,
59:30
SPEAKER_00: for example,
59:31
SPEAKER_00: and hope it bubbles up?
59:33
SPEAKER_01: Yeah, I think it's going to be a mix, right?
59:34
SPEAKER_01: Just on a smaller scale, right?
59:36
SPEAKER_01: So what is within the scope of your control?
59:38
SPEAKER_01: And it'll start by,
59:39
SPEAKER_01: I love that you said,
59:40
SPEAKER_01: you know,
59:41
SPEAKER_01: it's when you've joined,
59:42
SPEAKER_01: you know,
59:42
SPEAKER_01: you've done your first 90 days,
59:44
SPEAKER_01: you've done a listening tour,
59:46
SPEAKER_01: you've been able to kind of identify what some of the challenges are.
59:49
SPEAKER_01: It can be super important to then,
59:51
SPEAKER_01: you know,
59:51
SPEAKER_01: write up a summary,
59:52
SPEAKER_01: run it past one or two people to make sure like it kind of makes sense.
59:57
SPEAKER_01: Yeah, sanity check.
59:58
SPEAKER_01: Meet with the teams.
59:59
SPEAKER_01: Say, this is what I'm observing.
1:00:01
SPEAKER_01: This is what I'm hearing from you.
1:00:03
SPEAKER_01: Does this resonate?
1:00:04
SPEAKER_01: Does this sound right?
1:00:05
SPEAKER_01: Is there anything I'm missing?
1:00:08
SPEAKER_01: And then end that without a big call to action,
1:00:11
SPEAKER_01: but say,
1:00:12
SPEAKER_01: I would like for us to look for opportunities to improve this.
1:00:17
And I would love for you to be involved if that's something
1:00:20
that you're excited about.
1:00:21
Give people a little bit of time to kind of like marinate in
1:00:24
that kind of digest.
1:00:25
SPEAKER_01: Yeah.
1:00:26
SPEAKER_01: And then go back and say, what are some ways that we can improve this?
1:00:32
SPEAKER_01: These one or two things really stood out to me.
1:00:36
SPEAKER_01: What does that mean for you?
1:00:38
SPEAKER_01: And then depending on – this is where some resourcing comes in, right?
1:00:41
SPEAKER_01: Because we want to change the way that people work.
1:00:44
And so that could be creating new education if it's just about
1:00:48
changing a process.
1:00:48
SPEAKER_01: It can be changing.
1:00:49
SPEAKER_01: It can be improving the tech, right?
1:00:52
SPEAKER_01: And so we can have discussions about either,
1:00:54
SPEAKER_01: you know,
1:00:54
SPEAKER_01: kind of carving people off or maybe setting up a hack day.
1:00:57
SPEAKER_01: Sometimes a hack day can be great to just deal with paper cuts.
1:01:00
And that already,
1:01:02
the thing that's great about that is you get this quick win.
1:01:04
SPEAKER_01: It's visible.
1:01:04
SPEAKER_01: It's a quick win.
1:01:06
SPEAKER_01: It impacts people.
1:01:07
SPEAKER_01: So they see it.
1:01:08
SPEAKER_01: And if it doesn't impact me personally, maybe it impacts my coworkers.
1:01:10
SPEAKER_01: So I'm at least seeing progress.
1:01:13
SPEAKER_01: And then we have continued communication.
1:01:15
SPEAKER_00: Again, you've got to repeat yourself.
1:01:17
I also love what you said,
1:01:17
which is talk with people and basically make allies.
1:01:21
SPEAKER_00: Because when you started a new company,
1:01:22
you have no real social capital beyond maybe a fancy title or
1:01:26
if you have some
1:01:27
SPEAKER_00: impressive credentials,
1:01:28
SPEAKER_00: but not there.
1:01:29
SPEAKER_00: And I think what people forget a lot of times,
1:01:31
SPEAKER_00: because I've seen so many times where someone on paper,
1:01:35
SPEAKER_00: you know,
1:01:36
SPEAKER_00: comes in at a high level,
1:01:37
SPEAKER_00: tries to make changes.
1:01:38
There's a pushback because,
1:01:40
you know,
1:01:40
like we just want to do our work,
1:01:42
right?
1:01:42
SPEAKER_00: Like, what is this person trying to do?
1:01:44
SPEAKER_00: And then eventually they leave,
1:01:47
SPEAKER_00: eventually disappointed that this company doesn't want to change.
1:01:51
But what they miss doing is,
1:01:52
first of all,
1:01:53
not realizing it takes a longer time.
1:01:55
SPEAKER_00: You do want to build a team who...
1:01:58
who supports you you understand them they understand you and
1:02:01
now a team and then
1:02:02
later another team joins in and do this so i i i feel like
1:02:09
you've kind of conveyed
1:02:10
this really important thing and the fact that it's also not a
1:02:13
sprint in a sprint
1:02:14
you i mean you can do something but it'll probably be washed
1:02:17
away with like a
1:02:17
SPEAKER_01: sandcastle on a beach right yep um and i will say i have found uh
1:02:25
SPEAKER_01: Having aligned incentives ends up being important as well, right?
1:02:28
SPEAKER_01: If you say something's important,
1:02:30
SPEAKER_01: acknowledge it and follow up with that actually being important.
1:02:33
SPEAKER_01: So if you're just on the team, sometimes that can be a little tough.
1:02:36
If you're just an engineer,
1:02:37
like an IC you just started out of school,
1:02:40
right?
1:02:42
SPEAKER_01: But you can thank people.
1:02:44
SPEAKER_01: You can highlight what they've done.
1:02:46
SPEAKER_01: You can send a thank you email and cc their manager.
1:02:48
SPEAKER_01: And as a manager,
1:02:49
SPEAKER_01: you can say,
1:02:49
this is something that I find very important and we decided
1:02:52
together is important.
1:02:54
Any work that's done here,
1:02:55
even if it's not feature work,
1:02:58
is going to be valued.
1:03:00
SPEAKER_01: And then reflect that in any performance reports or one-on-ones,
1:03:05
because it can be really discouraging to go along with like
1:03:09
some transformation
1:03:10
SPEAKER_01: effort or fix effort or whatever.
1:03:12
SPEAKER_01: And then it just kind of, to your point, right?
1:03:14
SPEAKER_01: It just kind of washes away.
1:03:15
SPEAKER_01: It just goes away.
1:03:17
SPEAKER_00: So far,
1:03:18
SPEAKER_00: everything we talked about might as well have been two years before,
1:03:22
SPEAKER_00: because before ChatDGPT came out,
1:03:24
SPEAKER_00: before any of the AI tools went mainstream.
1:03:27
But these days,
1:03:28
software engineers are using AI tools,
1:03:31
coding,
1:03:31
ID assistance,
1:03:32
and many others.
1:03:33
SPEAKER_00: It is changing software engineering as a whole.
1:03:36
How has it changed your thinking about developer productivity
1:03:39
and how we should
1:03:40
SPEAKER_00: think of developer productivity and developer experience?
1:03:43
SPEAKER_00: Yeah.
1:03:44
With AI,
1:03:44
I'm getting so many questions now about how can we improve the
1:03:48
way developers work?
1:03:49
SPEAKER_01: How can we make sure that we don't detract from their work?
1:03:51
SPEAKER_01: Does this impact space or DoraMetrics?
1:03:55
SPEAKER_01: So I'll quickly say DoraMetrics
1:03:58
SPEAKER_01: in general, won't change.
1:04:00
SPEAKER_01: Their utility,
1:04:02
I expect to remain about the same because we're looking at how
1:04:05
well that outer loop
1:04:08
SPEAKER_01: is running.
1:04:09
SPEAKER_01: In terms of space,
1:04:10
I think it's still incredibly applicable because we want to
1:04:13
know how satisfied
1:04:14
SPEAKER_01: people are with the development experience.
1:04:15
SPEAKER_01: We want to know about performance outcomes like quality and security.
1:04:18
SPEAKER_01: We want to know about how many things they're able to get done.
1:04:22
SPEAKER_01: the way they communicate,
1:04:24
I think efficiency and flow is also super important because
1:04:26
it's changing the way
1:04:27
SPEAKER_01: we do work,
1:04:28
SPEAKER_01: right?
1:04:28
SPEAKER_01: When we're writing code,
1:04:30
SPEAKER_01: we're writing code,
1:04:30
SPEAKER_01: but we're also now reviewing a lot more code than,
1:04:33
SPEAKER_01: you know,
1:04:33
SPEAKER_01: it's almost a review exercise instead of just purely writing.
1:04:37
SPEAKER_01: There may be opportunities, I think, to expand space, right?
1:04:40
SPEAKER_01: Because trust ends up being a much larger factor.
1:04:45
uh to everyone right so i i studied uh trust with sys admins
1:04:49
and people doing you
1:04:50
know high consequence highly complex work and and trusting a
1:04:54
system and trusting
1:04:55
the outcome of a command and a new tool was incredibly
1:04:58
important developers didn't
1:05:00
see that as much because you know we go through school we use
1:05:02
an ide we use a
1:05:03
compiler right does the compiler work like do our people people
1:05:06
are not asking does
1:05:07
the compiler work and i understand like the underlying math is
1:05:09
very different but
1:05:10
SPEAKER_01: suddenly you've introduced this question
1:05:13
SPEAKER_01: to an everyday, all day workflow, is this the right output?
1:05:17
SPEAKER_01: Can I trust what I'm seeing?
1:05:21
SPEAKER_01: So I think it's good to kind of capture that and watch.
1:05:24
SPEAKER_01: Now, what does that mean kind of more broadly?
1:05:28
SPEAKER_01: I think there,
1:05:28
and we're seeing a lot of opportunities to kind of reinvent
1:05:32
development in software engineering,
1:05:33
SPEAKER_01: right?
1:05:34
SPEAKER_01: We're seeing it in the IDE right now.
1:05:35
We're seeing it in tests where,
1:05:37
you know,
1:05:37
we're asking questions around code quality.
1:05:39
SPEAKER_01: There was a great study done with GitHub and,
1:05:42
Office of the Chief Economist here that found that four
1:05:46
developers that had been
1:05:48
SPEAKER_01: seasoned developers,
1:05:48
SPEAKER_01: right?
1:05:48
SPEAKER_01: They have at least five years of experience.
1:05:51
They were finding that,
1:05:53
you know,
1:05:53
they're saying that the code is easier to read.
1:05:55
SPEAKER_01: They have good approval ratings.
1:05:57
SPEAKER_01: They're seeing over a 50% increase in unit tests getting passed.
1:06:03
SPEAKER_01: So it can have some really good.
1:06:05
And this is just from devs using,
1:06:06
you know,
1:06:07
tools like Copilot or other coding assistants.
1:06:10
SPEAKER_01: Yep.
1:06:10
SPEAKER_01: Yep.
1:06:11
SPEAKER_01: It's just from that.
1:06:12
SPEAKER_00: Interesting.
1:06:12
SPEAKER_00: And these were the reviewers saying that,
1:06:14
SPEAKER_00: you know,
1:06:14
SPEAKER_00: the PRs that are coming in just seem like nicer to work with.
1:06:18
SPEAKER_01: Mm-hmm.
1:06:20
SPEAKER_01: And, you know, you can have a PR review cycle that is done.
1:06:24
SPEAKER_01: You know, we're used to linters.
1:06:26
SPEAKER_01: Why not have an AI take a look and offer some suggestions?
1:06:31
SPEAKER_01: You know, that doesn't mean it's approved.
1:06:33
I think there's already like startups are building it,
1:06:36
but I cannot see this not being the case.
1:06:38
SPEAKER_00: Oh, absolutely.
1:06:39
SPEAKER_00: Right.
1:06:40
SPEAKER_01: Some folks, it's not quite on their radar, though, because we are so...
1:06:43
SPEAKER_01: Many times when we think about a developer,
1:06:45
SPEAKER_01: we think about someone like sitting there typing,
1:06:47
SPEAKER_01: right?
1:06:47
SPEAKER_01: Or like it's hackers or something, right?
1:06:49
It's very IDE focused,
1:06:50
but there's so much more to the development tool chain,
1:06:53
right?
1:06:54
One thing that my team and I are focused on that I'm like real
1:06:57
kind of bullish
1:06:58
SPEAKER_01: about is how do we think about the rest of the software process?
1:07:01
SPEAKER_01: How do we think about roles like release engineering and deployment?
1:07:05
SPEAKER_01: Because historically,
1:07:07
that's been very difficult and something that a person makes a
1:07:11
decision on.
1:07:12
And it's basically a risk-based decision and they have to take
1:07:14
data from the builds,
1:07:15
SPEAKER_01: from the documentation,
1:07:16
SPEAKER_01: from the downstream system,
1:07:18
SPEAKER_01: from so many different places and make a decision.
1:07:23
SPEAKER_01: this is an opportunity for LLMs to really jump in, right?
1:07:27
SPEAKER_01: Because the person is still making the decision,
1:07:29
but instead of spending days collating all of this data and
1:07:33
trying to summarize it,
1:07:34
you can instead get a quick summary with links out to what you
1:07:38
want to look at so
1:07:39
you can confirm or see if there's any like nudge or edge case
1:07:41
or something.
1:07:42
SPEAKER_01: And then you can make a more informed decision.
1:07:46
SPEAKER_01: So I'm personally excited about several ways that AI can kind of help
1:07:53
bolster and support and you know reinvent and disrupt right the
1:07:58
way we do the full
1:07:59
SPEAKER_01: engineering life cycle
1:08:02
SPEAKER_00: Earlier,
1:08:02
SPEAKER_00: you told me that some of the most productive engineers or teams,
1:08:09
SPEAKER_00: but also engineers you've seen are curious,
1:08:12
SPEAKER_00: but also they don't hold on strongly to beliefs.
1:08:15
SPEAKER_00: They're open to things changing.
1:08:17
And you gave the example of like a team tried an approach eight
1:08:21
years ago and it failed,
1:08:22
SPEAKER_00: but now it's actually,
1:08:23
SPEAKER_00: it was possible.
1:08:24
SPEAKER_00: Yeah.
1:08:25
Do I sense correctly that it's probably a smart approach for a
1:08:27
lot of us engineers
1:08:28
SPEAKER_00: to just kind of put away a lot of our reservations because LLMs are just
1:08:33
SPEAKER_00: fundamentally changing a lot of things.
1:08:34
SPEAKER_00: They'll turn out to do some things wonderfully.
1:08:36
SPEAKER_00: Some things will just crash and burn.
1:08:38
SPEAKER_00: But unless we do this,
1:08:40
will probably be that team that kind of closed itself down to
1:08:44
how things are.
1:08:46
SPEAKER_00: Is this the kind of advice you give to engineers?
1:08:50
SPEAKER_00: Or what would you tell people?
1:08:52
SPEAKER_00: Because there are some experienced engineers,
1:08:53
SPEAKER_00: they've been around the block,
1:08:54
SPEAKER_00: and there's some people who are skeptical.
1:08:55
SPEAKER_00: Like, okay, they're just a fancy autocomplete.
1:08:58
SPEAKER_00: We're giving them too much credit here.
1:09:00
SPEAKER_00: It's not going to change what we do at the fundamental level.
1:09:03
Or there are concerns that it will introduce errors or security
1:09:06
bugs or other types
1:09:08
SPEAKER_01: of things that I would never do or my team would never do.
1:09:11
SPEAKER_01: Right?
1:09:13
SPEAKER_01: Yeah,
1:09:14
I would say if you're in an organization that is deploying or
1:09:19
allowing tools like
1:09:20
SPEAKER_01: Copilot and Tab9 and Cursor,
1:09:24
SPEAKER_01: Really take it for a spin.
1:09:26
SPEAKER_01: See what works, see what resonates, see what lands.
1:09:29
SPEAKER_01: If you don't like it, don't use it, right?
1:09:32
If you only like it for specific types of use cases and
1:09:35
specific types of tasks,
1:09:37
SPEAKER_01: leverage it for the things that it is good at.
1:09:41
SPEAKER_01: And don't use it for the things that don't fit into your workflow,
1:09:43
SPEAKER_01: that don't fit into your mental model.
1:09:45
But give yourself a little bit of time to open up what that
1:09:48
mental model is.
1:09:50
Because it can shift and it can shift in ways that are very,
1:09:52
very valuable.
1:09:53
SPEAKER_01: It can open up time.
1:09:56
SPEAKER_01: Now you don't have to do all of the setup in your code.
1:09:59
SPEAKER_01: You can just focus on really solving the problem,
1:10:04
SPEAKER_01: really looking externally,
1:10:05
SPEAKER_01: really trying to figure out where the blocker is or what the longer term
1:10:08
SPEAKER_01: architectural components should be.
1:10:12
Now,
1:10:13
when I talk with developers and also when I'm building with AI,
1:10:18
I see two types of usages.
1:10:19
SPEAKER_00: One is you have just specifically talking about the coding.
1:10:22
So I know what the problem is,
1:10:24
I know what I want to do,
1:10:25
and I now need to do the work.
1:10:27
One of them is you have the inline AI systems,
1:10:30
which show automatic code completion.
1:10:33
SPEAKER_00: May this be co-pilot, a cursor, or some other ones.
1:10:36
SPEAKER_00: And it kind of helps you move faster as you go.
1:10:40
SPEAKER_00: It continuously gives you that additional context,
1:10:42
SPEAKER_00: but it also kind of takes away your attention a little bit.
1:10:44
SPEAKER_00: As soon as you type it, it shows up things.
1:10:47
An interesting thing that I talk with a developer who's
1:10:50
actually building a game
1:10:51
SPEAKER_00: He uses AI very differently.
1:10:53
He says he actually goes to one of these tools,
1:10:56
ChatGPT or Claude or one of the others.
1:10:58
And he goes there with a skeleton code,
1:11:01
pastes it in with a to-do,
1:11:03
and it generates.
1:11:03
SPEAKER_00: And I asked, hey, why aren't you using it in your ID?
1:11:06
SPEAKER_00: And he said,
1:11:06
I actually find that I can focus way more when I don't have
1:11:12
this thing interrupting
1:11:13
SPEAKER_00: me all the time.
1:11:14
SPEAKER_00: But I'm like, no, I need help.
1:11:16
SPEAKER_00: And I want to get your feedback.
1:11:17
SPEAKER_00: And I'm iterating with you versus going, I'm in my flow state.
1:11:22
Are you aware of either some research or even just personal
1:11:24
observations on how
1:11:27
SPEAKER_00: developers can preserve flow?
1:11:28
SPEAKER_00: Is it important or can you actually have a flow state either way?
1:11:33
SPEAKER_00: Is it something that everyone needs to figure out for themselves?
1:11:35
SPEAKER_00: Yeah.
1:11:36
SPEAKER_01: Um,
1:11:37
SPEAKER_01: I just want to say that story you started with,
1:11:39
I think is incredible because someone played around with AI and
1:11:42
they figured out
1:11:43
SPEAKER_01: the workflow that makes sense for them,
1:11:46
SPEAKER_01: that works for them.
1:11:47
SPEAKER_01: Right.
1:11:47
SPEAKER_01: It doesn't have to be, you know, line completions inside the IDE.
1:11:51
SPEAKER_01: It could be writing up a chunk of code and then asking it for review.
1:11:54
SPEAKER_01: Right.
1:11:55
SPEAKER_01: Um,
1:11:57
SPEAKER_01: I will say anecdotally,
1:12:00
SPEAKER_01: I've seen folks sometimes adopt that type of workflow or they'll turn off
1:12:06
SPEAKER_01: autocomplete or auto suggestions as they're typing.
1:12:10
SPEAKER_01: I haven't seen any hard data on that,
1:12:11
SPEAKER_01: although I wouldn't be surprised if there's some coming out.
1:12:14
In terms of flow,
1:12:17
there are a few things that we've discovered and noticed.
1:12:19
So there's a paper that some colleagues of mine did here in MSR around,
1:12:25
they use the CUPS model.
1:12:26
SPEAKER_01: So it's how does coding change, right?
1:12:30
SPEAKER_01: You write something,
1:12:31
SPEAKER_01: you wait for feedback,
1:12:31
SPEAKER_01: you have to reassess it,
1:12:32
SPEAKER_01: you have to decide what's coming next.
1:12:35
And some of that work and early work we did with the GitHub
1:12:37
team even influenced
1:12:39
SPEAKER_01: how the timing of suggestions happens,
1:12:43
SPEAKER_01: right?
1:12:43
SPEAKER_01: We ended up slowing some down because as you pointed out,
1:12:46
SPEAKER_01: if you're getting like suggestion after suggestion after suggestion,
1:12:50
SPEAKER_01: it ends up being an interruption.
1:12:54
SPEAKER_01: When I think about,
1:12:55
SPEAKER_01: you know,
1:12:55
SPEAKER_01: your question around how do we think about flow,
1:12:59
I think everyone will kind of need to play around with this and
1:13:02
see what works for
1:13:03
SPEAKER_01: them and what that means,
1:13:04
SPEAKER_01: right?
1:13:04
SPEAKER_01: Because when you're coding in an IDE,
1:13:06
SPEAKER_01: I know sometimes I'll turn it off because I just want to code,
1:13:08
SPEAKER_01: right?
1:13:09
SPEAKER_01: I did that this weekend because I wanted to play around with some code.
1:13:12
SPEAKER_01: And I wanted to be kind of driving and doing all of the writing.
1:13:16
SPEAKER_01: Yeah.
1:13:17
SPEAKER_01: But that also is,
1:13:18
SPEAKER_01: I think,
1:13:18
one helpful way to think about it is imagine a world where
1:13:24
there's just a different
1:13:25
SPEAKER_01: way to do the work.
1:13:26
SPEAKER_01: And maybe that means you call it something else in your head.
1:13:29
SPEAKER_01: Maybe instead of coding, maybe I'm driving and reviewing or I'm guiding.
1:13:34
SPEAKER_01: Because if we can think about it differently,
1:13:36
SPEAKER_01: then we're not comparing it to something that's not the same thing.
1:13:39
SPEAKER_01: And we can find what that new workflow is.
1:13:43
SPEAKER_00: I like it.
1:13:44
SPEAKER_00: I also like your initial suggestion of driving versus guiding.
1:13:47
SPEAKER_00: It is just very different.
1:13:48
But I do think that a lot of us engineers just need to accept
1:13:52
that it's different.
1:13:54
SPEAKER_00: It will change.
1:13:55
SPEAKER_00: We'll look back at this 10 years later.
1:13:56
SPEAKER_00: People will look back.
1:13:57
SPEAKER_00: People are starting their career now.
1:13:58
SPEAKER_00: They'll be like, oh, yeah, a little bit loud.
1:14:01
SPEAKER_00: Anyone who's a coder, it's like, oh, Stack Overflow, it always existed.
1:14:04
SPEAKER_00: I mean, interesting enough, now we've kind of stopped using it.
1:14:07
But there was a time where it was just a given and there were
1:14:10
people before and people after.
1:14:11
SPEAKER_00: So I think we're going to have that.
1:14:16
SPEAKER_00: It's a good and timely reminder.
1:14:19
SPEAKER_00: And also, we're just in the beginning of this, right?
1:14:21
Like this whole thing,
1:14:21
it's just wild how much it's gone in such a short time.
1:14:26
I don't think there's been any technology in the history of
1:14:29
software engineering
1:14:30
that has spread across a whole industry so quickly and is
1:14:33
changing how people work,
1:14:36
SPEAKER_00: changing their productivity,
1:14:37
SPEAKER_00: what they're doing,
1:14:40
SPEAKER_00: and still work in progress.
1:14:43
SPEAKER_01: Yeah, absolutely.
1:14:44
SPEAKER_01: I totally agree, right?
1:14:45
When I think about,
1:14:46
you know,
1:14:46
big impactful technology shifts in tech,
1:14:50
cloud is one for sure.
1:14:51
SPEAKER_01: But it didn't hit almost the entire industry within a year, right?
1:14:58
SPEAKER_01: It was many years.
1:15:00
SPEAKER_01: And specifically like LLMs, right?
1:15:02
SPEAKER_01: To the AI folks out there, AI has been around for years.
1:15:05
SPEAKER_01: Anytime you're doing any type of autocomplete in an email or,
1:15:08
SPEAKER_01: you know,
1:15:09
SPEAKER_01: IntelliCode,
1:15:09
SPEAKER_01: this is all AI.
1:15:11
SPEAKER_01: I think LLMs really changed,
1:15:13
SPEAKER_01: you know,
1:15:14
these foundation models really changed the way we can do work
1:15:18
and the types of
1:15:18
SPEAKER_01: development we can do.
1:15:20
SPEAKER_00: So closing off with some rapid questions,
1:15:22
SPEAKER_00: I'll just ask and you fire off whatever comes to mind.
1:15:25
SPEAKER_00: The first one is about coding full time versus being a researcher.
1:15:31
SPEAKER_00: You were a software engineer.
1:15:33
SPEAKER_00: Now you're a researcher.
1:15:34
SPEAKER_00: What is the one thing that you miss from coding full time?
1:15:39
SPEAKER_01: I miss just like packing something up and being in the code.
1:15:42
There's the thing I love about research is that it's about
1:15:45
identifying and solving
1:15:47
SPEAKER_01: problems that are hard to solve where we don't know the answer.
1:15:50
SPEAKER_01: Coding is very similar.
1:15:52
SPEAKER_01: And I love that.
1:15:53
I kind of miss the thought process of figuring out the best way
1:15:56
to write the code.
1:15:58
SPEAKER_01: What's the best way to structure the code?
1:16:00
SPEAKER_01: How do I want to do some of the work?
1:16:02
SPEAKER_01: I'm a joke, do the typey.
1:16:05
SPEAKER_01: I miss those parts of coding.
1:16:07
SPEAKER_01: And luckily, I kind of shifted into something where
1:16:11
SPEAKER_01: it still kind of scratches that itch of where are the big problems?
1:16:14
SPEAKER_01: What's the strategy I want to adopt, right?
1:16:15
SPEAKER_01: There's strategy in writing code,
1:16:17
there's strategy in communications and product direction and
1:16:20
research program.
1:16:24
SPEAKER_00: Plus, even if you're not coding, you're working with people who do code.
1:16:28
SPEAKER_01: Yes.
1:16:28
SPEAKER_01: And I,
1:16:29
SPEAKER_01: like I said,
1:16:29
SPEAKER_01: I'm,
1:16:29
SPEAKER_01: I still play around with it,
1:16:30
SPEAKER_01: but,
1:16:31
SPEAKER_01: but missing,
1:16:32
I think a little bit also is like the rush of writing something
1:16:35
for customers that
1:16:36
SPEAKER_01: I know is going to be a production system.
1:16:38
SPEAKER_01: I had folks at IBM a couple of years ago,
1:16:40
contact me and say that one of the systems I wrote there as an
1:16:42
intern was still
1:16:43
SPEAKER_01: being used.
1:16:44
SPEAKER_01: And I was like,
1:16:45
awesome first of all sorry second of all it's there's just a
1:16:50
different kind of
1:16:51
excitement around writing code and seeing it deployed and
1:16:54
pushed and shipped to
1:16:55
customers right which luckily i get a little bit with some of
1:16:57
my research i'm
1:16:58
realizing now as i tell you this like i've i've found another
1:17:01
proxy i've found a
1:17:02
way to kind of duplicate some of my coding and just another
1:17:05
discipline well and you
1:17:06
SPEAKER_00: still code a little bit so yeah
1:17:09
SPEAKER_00: What is the next book you're working on, if you can tell us?
1:17:12
SPEAKER_01: Yeah.
1:17:13
SPEAKER_01: Right now, I'm working on a developer experience book.
1:17:17
It's kind of geared toward folks who want to think about
1:17:21
improving developer experience.
1:17:22
SPEAKER_01: They want to learn more about developer experience.
1:17:24
SPEAKER_01: It could be an IC or an engineering manager.
1:17:28
SPEAKER_01: It could be leadership.
1:17:30
SPEAKER_01: So we cover things like, I'm writing this with Abinoda.
1:17:34
So we're covering things like what type of data should we be
1:17:37
thinking about?
1:17:37
SPEAKER_01: How should we communicate data?
1:17:39
SPEAKER_01: How should we communicate like the ROI, right?
1:17:43
SPEAKER_01: A lot of this is communication.
1:17:45
SPEAKER_01: How do we want to think about a long-term analysis and data collection?
1:17:49
SPEAKER_01: kind of program.
1:17:51
SPEAKER_01: This is not a stats book.
1:17:53
SPEAKER_01: But getting people pointed in the right direction, right?
1:17:56
Examples of survey questions,
1:17:58
examples of types of data,
1:17:59
examples of ways to operationalize.
1:18:01
Because my kind of heuristic is once I've had at least 100
1:18:06
people ask me a question
1:18:07
SPEAKER_01: that ends up being in the same flavor of answer,
1:18:10
SPEAKER_01: right?
1:18:10
SPEAKER_01: It's very, very similar.
1:18:13
SPEAKER_01: Something written,
1:18:13
something like a book can be really helpful because then
1:18:16
everyone can kind of cover
1:18:17
basics and then think of the next level questions that they're
1:18:20
going to get to eventually.
1:18:21
But it's not a good use of time if like I'm a blocker if we're
1:18:25
trying to do
1:18:26
SPEAKER_01: something over an hour.
1:18:27
What's a practice or a tool that helps you personally be more
1:18:31
productive on the
1:18:31
SPEAKER_00: day-to-day basis?
1:18:33
SPEAKER_01: Day-to-day.
1:18:33
SPEAKER_01: So day-to-day at work, I don't want to say a tool.
1:18:37
I've got an incredible PM who has absolutely up-leveled all of
1:18:41
the work that we're doing.
1:18:42
SPEAKER_01: Jason Entenman, shout out.
1:18:44
SPEAKER_01: Having a very good PM, TPM could just be a game changer.
1:18:49
SPEAKER_01: I had an incredible one at Google too.
1:18:51
SPEAKER_01: It was just so, so good.
1:18:54
SPEAKER_01: But when I think about technology tools,
1:18:58
SPEAKER_01: not in my work life,
1:19:00
SPEAKER_01: because I kind of maintain this,
1:19:02
SPEAKER_01: I for sure maintain like a security boundary.
1:19:04
SPEAKER_01: Claude, Claude's incredible.
1:19:07
SPEAKER_01: I'm able to use Claude to do many, many things.
1:19:11
SPEAKER_01: And that, I would say, has been the most notable impact.
1:19:16
SPEAKER_00: It is very interesting how the competition between LLMs,
1:19:21
SPEAKER_00: it's just so great,
1:19:22
SPEAKER_00: because OpenAI obviously started all of this,
1:19:25
SPEAKER_00: and now there's Entropic,
1:19:27
SPEAKER_00: Claude,
1:19:28
SPEAKER_00: Sonnet,
1:19:28
SPEAKER_00: in many ways,
1:19:30
SPEAKER_00: jumping ahead.
1:19:31
SPEAKER_00: But I feel it's just so great for the ecosystem.
1:19:33
SPEAKER_00: There's so much competition everywhere, and they're all...
1:19:36
inspiring one another to become just so much better so i i feel
1:19:40
like as a someone
1:19:41
using all these things we're just in the middle of some of the
1:19:44
best time in the
1:19:45
sense of like getting all of this these things for free because
1:19:48
you know these
1:19:49
teams want to like outclass one another and we're just
1:19:52
benefiting from it and you
1:19:53
know like you said it's so early that everyone is still
1:19:55
learning and i'll jump back
1:19:57
SPEAKER_01: and forth because some models are you know better for something else but
1:20:01
SPEAKER_00: It can change every month, right?
1:20:03
SPEAKER_01: Yeah, it can change all the time.
1:20:05
SPEAKER_01: But having a rubber duck that responds to some of your questions,
1:20:10
SPEAKER_01: sometimes I just want the rubber duck,
1:20:12
SPEAKER_01: but sometimes I really want to think through something like,
1:20:14
SPEAKER_01: what are my limitations?
1:20:15
SPEAKER_01: What am I missing?
1:20:16
SPEAKER_01: What are the holes?
1:20:17
SPEAKER_01: Okay, no, that's not actually a hole.
1:20:18
SPEAKER_01: What else would it be?
1:20:19
SPEAKER_01: It ends up being really useful, and it's super, super fast.
1:20:24
SPEAKER_00: What are one or two books that you read and would recommend?
1:20:27
SPEAKER_01: It's tech related.
1:20:28
SPEAKER_01: Inspire by Marty Kagan is great.
1:20:31
I'm still making my way through parts of it because it is not a
1:20:34
small book.
1:20:35
SPEAKER_01: Yeah, it's right over there.
1:20:38
It's really kind of solidified and articulated things that I've
1:20:43
kind of known,
1:20:44
SPEAKER_01: but like in a very fuzzy way.
1:20:45
And it's also like introducing some new things for me to think about,
1:20:49
which I really appreciate.
1:20:51
SPEAKER_00: Awesome.
1:20:52
SPEAKER_01: Another one that I loved that was just kind of practical.
1:20:56
SPEAKER_01: I just like to dive into random things.
1:20:59
SPEAKER_01: Outlive by Peter Atiyah.
1:21:03
SPEAKER_01: I want to nerd out and learn something, which my partner jokes.
1:21:06
I've heard a lot of good things about Peter Atiyah from people
1:21:12
working in tech.
1:21:13
SPEAKER_00: Yeah.
1:21:15
SPEAKER_01: And then for a fun book, I always love Ender's Game.
1:21:18
SPEAKER_01: It's like an oldie but goodie.
1:21:19
SPEAKER_01: So if I just want to escape,
1:21:22
I'll just grab a trashy beachy read or I'll grab Ender's Game
1:21:26
or Ready Player One
1:21:27
SPEAKER_01: or something because it's just a fun story that I can just race through.
1:21:30
SPEAKER_00: They're such fun stories.
1:21:32
SPEAKER_00: I feel the movies were fine as well, but nothing beats the book.
1:21:36
SPEAKER_01: Yeah.
1:21:37
And then for news and articles,
1:21:39
Tressie McMillan-Cotton and Anne Helen Peterson.
1:21:42
SPEAKER_01: are probably the most high value rates that I get outside of tech.
1:21:47
SPEAKER_00: Awesome.
1:21:48
SPEAKER_00: Well, Nicole, thank you so much for being on this podcast.
1:21:51
SPEAKER_00: This has been so fascinating,
1:21:53
SPEAKER_00: interesting,
1:21:53
SPEAKER_00: and I think just like motivating,
1:21:56
but both on how much we've come with developer productivity and
1:21:59
also just how,
1:22:00
SPEAKER_00: just when we're getting closer to figuring things out,
1:22:02
SPEAKER_00: things change again with AI and teams behaving differently.
1:22:06
SPEAKER_00: It's an exciting time.
1:22:07
SPEAKER_01: Yeah, it's really fun.
1:22:09
SPEAKER_01: This is a great time to be working, especially in tech.
1:22:13
Thank you to Nicole for this deep dive into the ever-evolving
1:22:15
field of developer productivity.
1:22:17
You can read more from Nicole on her website or follow her on
1:22:19
social media,
1:22:20
SPEAKER_00: both linked in the show notes below.
1:22:23
In the Pragmatic Engineer,
1:22:24
we've done several deep dives on developer productivity.
1:22:26
SPEAKER_00: Check these out linked in the show notes below or by searching developer
1:22:29
SPEAKER_00: productivity in the Pragmatic Engineer newsletter.
1:22:32
SPEAKER_00: If you enjoyed this podcast,
1:22:33
SPEAKER_00: please do subscribe on your favorite podcast platform and on YouTube.
1:22:36
SPEAKER_00: Thanks and see you in the next one.