Latest / DevQuestions with Tim Corey / 314. Measuring Developer Performance and ROI is Hard But Possible
Transcript
- 0:00SPEAKER_01: Using lines of code produced to measure a developer's performance is a terrible metric.
- 0:06SPEAKER_01: So is features added.
- 0:08SPEAKER_01: So is bugs fixed or tickets closed.
- 0:11SPEAKER_01: Let's talk about why these are terrible metrics as well as what we can use to evaluate developer performance in today's episode of DevQuestions.
- 0:30SPEAKER_00: Welcome to the DevQuesti podcast with Tim Corey.
- 0:34SPEAKER_01: Software development is more than just writing code.
- 0:37SPEAKER_01: So let's talk about the rest of it.
- 0:38SPEAKER_01: Specifically, let's talk about measuring developer performance.
- 0:42SPEAKER_01: Now, there are a lot of bad ways to measure developer performance.
- 0:46SPEAKER_01: Let's go through a few and see why each is a bad metric.
- 0:51SPEAKER_01: Number one, lines of code.
- 0:53SPEAKER_01: This is a common one that has been around forever and also been kind of talked down about for a long time, but let's talk about why.
- 1:01SPEAKER_01: And first of all, why do we even measure lines of code and say, well, that would be a theme we measure against, or that's what we want to see?
- 1:11SPEAKER_01: Well, because there's gotta be some measure, right?
- 1:14SPEAKER_01: As when you're evaluating a person, it's hard to say this nebulous, you're doing good, or you're doing bad.
- 1:22SPEAKER_01: Um, because what is that based upon?
- 1:25SPEAKER_01: If if it's a sliding scale based upon vibes, that's not great.
- 1:29SPEAKER_01: And so companies love to have metrics.
- 1:32SPEAKER_01: They love to have something that's a number where they can say, number go up, number good, number go down, number bad.
- 1:40SPEAKER_01: But it's it's hard to quantify, especially the thought work that goes into software development with these numbers.
- 1:50SPEAKER_01: But you know, specifically lines of code.
- 1:52SPEAKER_01: Well, what does that do?
- 1:55SPEAKER_01: It rewards verbosity.
- 1:58SPEAKER_01: So think through whatever metric that you're using to evaluate a person, that's what you are going to reward.
- 2:07SPEAKER_01: So you're saying, I want to see lines of code.
- 2:11SPEAKER_01: So what are you going to get from your employees?
- 2:14SPEAKER_01: Lines of code.
- 2:17SPEAKER_01: Does that mean that lines of code is the a good thing?
- 2:22SPEAKER_01: No, it doesn't.
- 2:23SPEAKER_01: In fact, when have you ever evaluated an application and said, you know what?
- 2:28SPEAKER_01: This is this feels like a great application.
- 2:31SPEAKER_01: I love how it works, but I just don't feel like it has enough lines of code in it.
- 2:36SPEAKER_01: That's a silly thing.
- 2:37SPEAKER_01: No one ever opens up an app and goes, I bet there's not enough lines of code in there.
- 2:43SPEAKER_01: That's not how that works.
- 2:44SPEAKER_01: Or you might they might even say, you know what, that's got a lot of lines of code, I bet.
- 2:48SPEAKER_01: That's probably a great application.
- 2:49SPEAKER_01: No one thinks that.
- 2:51SPEAKER_01: Because lines of code don't matter.
- 2:55SPEAKER_01: What matters is did you get the job done?
- 2:57SPEAKER_01: Does it do what it's supposed to do effectively?
- 3:00SPEAKER_01: So measuring lines of code rewards verbacity or verbosity, not maintainability.
- 3:06SPEAKER_01: So if I look at a method and it it's you know, uh a good one is to see us as AI, right?
- 3:13SPEAKER_01: So if I want to write a little method that calls Dapper to call SQL, I just want a little method.
- 3:21SPEAKER_01: I can do that in about four lines of code, maybe five, for everything, right?
- 3:26SPEAKER_01: So I've connected to the database, open the connection, um, and then actually getting the data, mapping it back to a list of T, and then returning it.
- 3:34SPEAKER_01: That's about four or five lines of code.
- 3:37SPEAKER_01: If I ask AI for that, it might give me 30 or 50.
- 3:42SPEAKER_01: And you know what?
- 3:43SPEAKER_01: It's not adding value.
- 3:45SPEAKER_01: What it's adding is a line of code, lots of lines of code.
- 3:48SPEAKER_01: Now, if we were to compare those two and say, well, the AI wrote 30 lines of code, that must mean it's better.
- 3:57SPEAKER_01: Well, that's not true.
- 3:59SPEAKER_01: What it is is is more verbose and it's hard to maintain because there's more to maintain, and some of that stuff is just not necessary.
- 4:07SPEAKER_01: So if you look at, you know, if you look at people who are writing code, if you see a person who writes lots and lots of lines of code and some of it's not necessary, well, it's actually more likely to cause bugs down the road, and it's even harder to maintain.
- 4:22SPEAKER_01: So that's not a good thing, which is typically why I say that junior developers solve simple problems with simple code bits.
- 4:31SPEAKER_01: And then mid-level developers tend to solve complex problems with a significant number of lines of code, and then senior developers solve complex problems with a few lines of code, right?
- 4:47SPEAKER_01: So that's where you see the progression is as you start getting better in software development, you write more code, but then at some point you tip over into writing better code that has less lines of code.
- 5:01SPEAKER_01: Typically, when you see a software developer come in who's a senior developer to work on a project, they'll start doing commits where maybe the commit is removed 200 lines of code and added 15, right?
- 5:15SPEAKER_01: You'll see commits like that where if you are measuring based upon lines of code, they're going the opposite direction.
- 5:20SPEAKER_01: And yet what they're doing is making the code better by being more efficient.
- 5:25SPEAKER_01: So lines of code is not a good metric.
- 5:29SPEAKER_01: You're rewarding verbosity, not maintainability.
- 5:32SPEAKER_01: Number two, features added.
- 5:35SPEAKER_01: So what this does is it rewards you adding new features into the product.
- 5:42SPEAKER_01: Well, what are the downsides of that?
- 5:44SPEAKER_01: Because that sounds good, right?
- 5:45SPEAKER_01: More features is good, right?
- 5:47SPEAKER_01: Well, first of all, not necessarily.
- 5:49SPEAKER_01: Don't just chase more features because more features equals harder to maintain because you have more things going on.
- 5:54SPEAKER_01: And it's also more confusing of a product, which means the customers kind of get overwhelmed a bit.
- 6:00SPEAKER_01: If you've never worked with Photoshop and then you open it up for the first time, you go, ooh, there's a lot of stuff here.
- 6:07SPEAKER_01: Same with Visual Studio, right?
- 6:08SPEAKER_01: You open up Visual Studio and it's got all these menus and things, and you're like, wow, there's a lot going on here.
- 6:14SPEAKER_01: Um so the first time you open up, very uh intimidating.
- 6:18SPEAKER_01: Well, that's what at you know, feature bloat over time, that's what happens.
- 6:24SPEAKER_01: And you know what, there's a place for that, but not in every app.
- 6:27SPEAKER_01: And we can get more into that in another episode.
- 6:31SPEAKER_01: But when you reward features added, what you're rewarding is whenever your developers add something new.
- 6:39SPEAKER_01: So what are they going to do?
- 6:40SPEAKER_01: Add something new all the time.
- 6:44SPEAKER_01: That means you'll get shoddy work because they're not focused on crafting the perfect feature, it's gonna be, you know, spending two weeks working on one feature that is gonna be just right and making sure it's you know performant, making sure that it's going to be the right feature and all the rest.
- 7:01SPEAKER_01: They don't care about that because they then get dinged for not putting out enough features.
- 7:07SPEAKER_01: But if they can crank through that feature in two hours, they can move on to two more, three more in one day.
- 7:14SPEAKER_01: So you're rewarding shoddy work, not long-term sustainability.
- 7:18SPEAKER_01: Plus, you're rewarding them doing new things, which means they're going to ignore bug fixes because that's not a feature ad.
- 7:28SPEAKER_01: So then number three, we can kind of flip that around, rewarding bug fixes and saying, hey, I want you to fix bugs.
- 7:34SPEAKER_01: And the more bugs you fix, the more I'm gonna reward you.
- 7:38SPEAKER_01: Well, you're rewarding shoddy work because what you're rewarding is them fixing problems.
- 7:44SPEAKER_01: And if they do the things the right way the first time, well, then they don't get a reward.
- 7:49SPEAKER_01: Even if combine those two and say features and bugs fixed.
- 7:53SPEAKER_01: Well, if they spend a long time creating a feature, then they're only doing one thing.
- 7:58SPEAKER_01: And if they spend a short time doing that feature and then fix 10 bugs later on, you actually give them more rewards because they did 11 things.
- 8:08SPEAKER_01: Where the reality is that they just didn't write the first time, they would have gotten only one thing done.
- 8:14SPEAKER_01: So you reward them for basically messing up.
- 8:19SPEAKER_01: And so they're going to mess up.
- 8:21SPEAKER_01: So you're actually rewarding them for creating bugs.
- 8:25SPEAKER_01: Now, number four is kind of that same concept of combining all of these, and that's tickets closed.
- 8:32SPEAKER_01: So maybe a ticket is a feature, maybe a ticket is a bug, maybe a ticket is a tweak to a system.
- 8:39SPEAKER_01: The problem is that you're rewarding speed over quality.
- 8:45SPEAKER_01: So what's gonna happen?
- 8:47SPEAKER_01: Well, you're gonna have a lot of work done in areas that aren't priorities, because if I can see that, well, this ticket is gonna be, I don't know, 10 minutes to do, but it's on something that no one's ever gonna have a you know, see or have an issue with, but is submitted as a bug.
- 9:07SPEAKER_01: You know, um, one person, if they you know on a leap year, they have this bug on this one computer because it's you know based upon the name and and you go, I can fix that in 10 minutes.
- 9:18SPEAKER_01: Well, but does it need to be fixed?
- 9:20SPEAKER_01: That's a that's a super low priority.
- 9:22SPEAKER_01: But yet if I fix that, then I get credit for that and it's a quick fix.
- 9:27SPEAKER_01: So I'll do that rather than working on the important thing where hundreds of customers are being affected, but it's a more complicated fix.
- 9:35SPEAKER_01: So if you worked in that complicated fix, it might take you a week or two to get done, and you might have to involve multiple people, and that's one ticket for multiple weeks, and that's gonna not be good either.
- 9:50SPEAKER_01: So you're rewarding speed over quality, you're rewarding speed over priority of tickets.
- 9:56SPEAKER_01: And so what happens is you get what you look for.
- 10:01SPEAKER_01: So you're looking for tickets closed, that's what you're gonna get.
- 10:04SPEAKER_01: But is that what the end user wants?
- 10:08SPEAKER_01: Again, going back to that example of always think about what's the end user looking for.
- 10:14SPEAKER_01: The end user, for the most part, doesn't care about lines of code.
- 10:19SPEAKER_01: Well, they definitely don't care about lines of code.
- 10:21SPEAKER_01: They don't care about the number of tickets you close.
- 10:24SPEAKER_01: They don't care about the features added as much as you think they do, because they want good features added.
- 10:32SPEAKER_01: And if you put out half-baked features, that's frustrating.
- 10:36SPEAKER_01: If you put out too many features, that's frustrating.
- 10:39SPEAKER_01: If you put out confusing features, that's frustrating.
- 10:42SPEAKER_01: And bug fixes, yeah, they want their bugs fixed immediately.
- 10:46SPEAKER_01: But if you're rewarding shoddy work, well then their bug is getting fixed, supposedly, but then it happens again a different way.
- 10:54SPEAKER_01: So there's simply another issue.
- 10:56SPEAKER_01: And what you have is this ever-changing list of where the bug is.
- 11:01SPEAKER_01: And that's frustrating too.
- 11:03SPEAKER_01: So what the end user really wants is an application that works, where when there is a problem, it gets addressed and addressed right.
- 11:11SPEAKER_01: That's what they want.
- 11:12SPEAKER_01: And if they want new features, they're gonna want to see those fully executed on in a way where they're not buggy, where they make sense, where it fits in well the current product, et cetera.
- 11:24SPEAKER_01: So you want to make sure that you're rewarding what you actually want to see.
- 11:29SPEAKER_01: And what you actually want to see isn't lines of code, features added, bugs fixed, or tickets closed.
- 11:34SPEAKER_01: That's not the right thing.
- 11:36SPEAKER_01: So in all these metrics, you get what you measure.
- 11:38SPEAKER_01: If you want more lines of code, that's what you get.
- 11:41SPEAKER_01: If you want more features, that's what you get.
- 11:44SPEAKER_01: So if you actually want to evaluate the results, what you get is a subpar solution and short-term gain at the expense of long-term problems.
- 11:56SPEAKER_01: So if you're looking at just at just the results of these things, you know, you're not gonna get the right, the right things.
- 12:03SPEAKER_01: So, what is the right metric?
- 12:05SPEAKER_01: To answer that, I think you need to look at what your actual goal is.
- 12:10SPEAKER_01: Like I said, your application you're building isn't really the end result you're looking for.
- 12:16SPEAKER_01: Um, the action your applications take are you wanna meet the need of the customer.
- 12:22SPEAKER_01: So even the idea that your application has a cool form or a cool look, that's not really the goal you're looking for.
- 12:29SPEAKER_01: What your actual goal is can your user use the application well?
- 12:34SPEAKER_01: Yes, do they enjoy using it?
- 12:36SPEAKER_01: Does it look nice?
- 12:37SPEAKER_01: But you know, sometimes we spend too much time tweaking font size when we need to think about the fact that the login flow is just horrible.
- 12:46SPEAKER_01: So let's look at a couple examples.
- 12:49SPEAKER_01: So these are all fictional.
- 12:51SPEAKER_01: Um, no, you know, no one issue is from one situation my previous life, but a mobile app for a pizza shop.
- 13:00SPEAKER_01: So we're gonna build a mobile app for a pizza shop.
- 13:02SPEAKER_01: The goal is to facilitate the customers buying pizzas.
- 13:08SPEAKER_01: That's your goal.
- 13:09SPEAKER_01: Okay.
- 13:10SPEAKER_01: So your goal isn't to have tons of features.
- 13:14SPEAKER_01: Your goal isn't to even have a flashiest app.
- 13:18SPEAKER_01: Your goal is to facilitate the customer buying pizzas.
- 13:22SPEAKER_01: Now, what about a lineup business application for a manufacturing plant?
- 13:26SPEAKER_01: The goal is to capture and process the data of the business supporting the customer's or the company's bottom line.
- 13:34SPEAKER_01: So it may be an internal app in a manufacturing plant.
- 13:37SPEAKER_01: Been there, done that a bunch of times.
- 13:39SPEAKER_01: And the goal there is to process, capture and process that data and support the business's bottom line.
- 13:46SPEAKER_01: Now, if you're building a software as a service that handles bulky emailing, the goal is to allow your customer to send out the emails they want to send.
- 13:56SPEAKER_01: In all of these examples, the software facilitates the customer, even if the customer is internal.
- 14:03SPEAKER_01: So that should be your goal to create software that supports the customer in an ever-increasing way.
- 14:09SPEAKER_01: With that goal in mind, here are some ways to track performance.
- 14:14SPEAKER_01: Number one, customer satisfaction.
- 14:18SPEAKER_01: Is the customer satisfied using your product?
- 14:21SPEAKER_01: Do they like using your product?
- 14:23SPEAKER_01: Is it helpful to them or is it frustrating?
- 14:27SPEAKER_01: Number two, the speed of customer completion.
- 14:31SPEAKER_01: One of the things that people often overlook is how fast can a customer use your application and be done?
- 14:39SPEAKER_01: Because typically your application is not the center of the universe.
- 14:44SPEAKER_01: It's the means to an end.
- 14:46SPEAKER_01: It's the tool that your end user uses to do something else.
- 14:52SPEAKER_01: I worked for a company once where we were doing, I'm gonna generatize this a little bit, we were doing data entry, um, and it was high volume data entry.
- 15:02SPEAKER_01: And when I first started working on the application, I was young.
- 15:04SPEAKER_01: When I first started working on the application, I created really cool forms and got roasted for them by the people who were earning at the time like$10 an hour.
- 15:15SPEAKER_01: But they were the ones using my application.
- 15:18SPEAKER_01: And they roasted me for it.
- 15:20SPEAKER_01: And I'm like, well, this is a great form.
- 15:22SPEAKER_01: And they're like, it's not usable.
- 15:25SPEAKER_01: And the reason why is because it relied on using a mouse.
- 15:30SPEAKER_01: And I expected a form where you'd hit next.
- 15:33SPEAKER_01: But that's not what they wanted to do because speed of completion was really important.
- 15:39SPEAKER_01: When you have a stack that is literally three foot tall on your desk of papers, you have to process.
- 15:47SPEAKER_01: You don't want to spend time looking at the screen going, click next.
- 15:52SPEAKER_01: That's not you.
- 15:53SPEAKER_01: What you wanted to keep your head down at your keyboard, looking at your documents and going type, type, type, type, type, tab, type, type, type, type, type, enter, type, type, type, and just move through them fast.
- 16:04SPEAKER_01: So I rewrote the entire thing and it wasn't as pretty, but you know what?
- 16:10SPEAKER_01: Everything was focused on speed of completion.
- 16:14SPEAKER_01: I got out of the way.
- 16:16SPEAKER_01: My application got out of the way.
- 16:18SPEAKER_01: That's how I knew my application was being successful, is I was allowing my end users to move through the application faster than before.
- 16:28SPEAKER_01: And all of a sudden, those same people loved me because their day was already misery.
- 16:36SPEAKER_01: They were in a low-paying job that was miserable and putting data into the computer over and over and over again all day long.
- 16:43SPEAKER_01: By allowing them to move faster, they were able to get their job done quicker, which meant their bosses were happier with them, they got better raises, they had a better situation because my application got out of the way.
- 16:56SPEAKER_01: So, speed of customer completion is a really big metric.
- 17:01SPEAKER_01: Number three, feature utilization.
- 17:04SPEAKER_01: There are some applications that have so many features.
- 17:09SPEAKER_01: Microsoft is one of the biggest, they have the most problems with this.
- 17:15SPEAKER_01: They're the biggest offender.
- 17:17SPEAKER_01: So when you open up Microsoft Word, they have tried streamlined, they have tried changing it.
- 17:23SPEAKER_01: Look at Microsoft Word.
- 17:25SPEAKER_01: We all, you know, we probably all know what it looks like, but have you ever actually gone through and looked at all the features?
- 17:32SPEAKER_01: Have you ever actually used all of them?
- 17:35SPEAKER_01: There are so many that I never even see.
- 17:38SPEAKER_01: There's there's ribbons, whole ribbons that I don't even go to.
- 17:42SPEAKER_01: So there's so many features there that aren't being used by the vast majority of people.
- 17:49SPEAKER_01: And yet they're being supported, they're being updated.
- 17:53SPEAKER_01: Now, having lots of features is not necessarily a bad thing.
- 17:57SPEAKER_01: Microsoft Word is being used by probably millions of people worldwide.
- 18:01SPEAKER_01: And so, yes, there are probably still a lot of people using every feature.
- 18:06SPEAKER_01: But when you build your application, you should look at how your features are being utilized.
- 18:12SPEAKER_01: Because what you'll find is some just aren't being used.
- 18:16SPEAKER_01: And that's one of a couple different reasons.
- 18:19SPEAKER_01: One is people don't know it's there.
- 18:22SPEAKER_01: Or two, they know it's there, but they don't want to use it because they have a better way of doing things.
- 18:28SPEAKER_01: So look at those things, figure out why.
- 18:31SPEAKER_01: Or number three, it's just a dumb feature.
- 18:32SPEAKER_01: Okay.
- 18:33SPEAKER_01: So, but look at why, talk to people, figure it out, because that's one of your goals you should be tracking is feeder, feature utilization.
- 18:42SPEAKER_01: Why are we doing work on something that's not being used?
- 18:46SPEAKER_01: Number four, of course, is application downtime.
- 18:49SPEAKER_01: If your application is down, and it's your application's fault, right?
- 18:53SPEAKER_01: If the server goes down, unless you caused it, your application caused it, then it's not your fault.
- 18:58SPEAKER_01: But if your application is down because it bugs, if users can't use your application because it crashes, that's something you need to track.
- 19:06SPEAKER_01: That's a goal you should have is to have less and less and less downtime because of your application.
- 19:14SPEAKER_01: Now, this is where things get tricky.
- 19:16SPEAKER_01: Notice that all of these are team metrics.
- 19:19SPEAKER_01: They're not really individual metrics.
- 19:22SPEAKER_01: That's because it's the best way to actually evaluate a team.
- 19:27SPEAKER_01: If a developer isn't doing well, the team should be incentivized to help them get better.
- 19:32SPEAKER_01: Senior developers should be coaching junior developers.
- 19:34SPEAKER_01: Now, I know there's a lot in here that you're like, ooh, wait a minute.
- 19:39SPEAKER_01: I've done team projects before, there's always one Slacker that's not pulling their weight.
- 19:43SPEAKER_01: And yes, there needs to be some individual things as well.
- 19:47SPEAKER_01: I'm not disagreeing there.
- 19:49SPEAKER_01: But really, if you're actually a team, if you're actually working on the same project, you should be measured as a team.
- 19:56SPEAKER_01: That does not mean that you need to be, you know, assigning these to raises or to you know to punishments based upon how the team is doing, but your team should be tracking these things and talking about it and figuring out how to make these things better.
- 20:12SPEAKER_01: They should be in front of your team at all times.
- 20:16SPEAKER_01: Now, there are a few things you can do at an individual level.
- 20:20SPEAKER_01: For instance, if a person is consistently the cause of bugs, downtime, poor customer satisfaction, or other issues, that can be noted.
- 20:28SPEAKER_01: Also, if a person performs well in comparison to the rest of the team, that too can be noted.
- 20:34SPEAKER_01: However, there's not a clear metric that will give you a clear insight into a person's performance.
- 20:39SPEAKER_01: Otherwise, people will focus on that specific metric to the detriment of both the team and the overall goal of the product.
- 20:48SPEAKER_01: This is why return on investment is so hard to measure for an individual developer.
- 20:54SPEAKER_01: It's also why it's almost impossible to identify if AI is having a positive impact on ROI unless you're evaluating the team like out on the buff.
- 21:06SPEAKER_01: Then you can see how AI impacts the team's results.
- 21:09SPEAKER_01: So if your team is slowly moving in this direction and you bring AI in, is it moving faster towards your goals, or is it hurting your goals?
- 21:19SPEAKER_01: Again, remember, not lines of code, not features added, but the ones where we actually have find value in that, the customer satisfaction, the speed of customer completion, the feature utilization, and the application downtime.
- 21:35SPEAKER_01: So when you see people say they're writing code faster or getting more features out the door, remember that these are terrible metrics and don't actually indicate a positive impact towards the actual goal.
- 21:49SPEAKER_01: If you want to see if they're succeeding, look at the end results.
- 21:53SPEAKER_01: Is the product easier to use?
- 21:56SPEAKER_01: Are the new features being used heavily?
- 21:58SPEAKER_01: Is the product more stable?
- 22:01SPEAKER_01: Are the customers finding that their application is more valuable now than it was before?
- 22:08SPEAKER_01: Is the application running more?
- 22:10SPEAKER_01: Is it have less downtime than before?
- 22:14SPEAKER_01: This is how you measure success.
- 22:16SPEAKER_01: Okay?
- 22:17SPEAKER_01: So there's not gonna be one, here is the metric to track.
- 22:22SPEAKER_01: It's a lot more difficult than that.
- 22:23SPEAKER_01: But tracking first the team metrics and then looking at the individual performance inside those team metrics can be much more helpful and can still give you some things to measure while at the same time you've got to acknowledge the reality of there's no one single number, you can say that's the one.
- 22:41SPEAKER_01: All right.
- 22:42SPEAKER_01: Thanks for listening.
- 22:43SPEAKER_01: And as always, I am Tim Corey.
- 22:45SPEAKER_00: Thank you for joining us for this episode of Dev Questions.
- 22:48SPEAKER_00: When you're ready to learn to think and code like a professional developer, head over to I amtimcorey.com and enroll in a course.