Latest / DevQuestions with Tim Corey / 296. Does Code Quality Matter When AI Writes the Code?

Transcript

296. Does Code Quality Matter When AI Writes the Code? · DevQuestions with Tim Corey

• 0:00
  SPEAKER_00: Does code quality matter anymore?
• 0:02
  SPEAKER_00: If we let AI write our code, should it matter if it's readable code?
• 0:06
  SPEAKER_00: Aren't we just going to let the AI fix any code that's broken?
• 0:10
  SPEAKER_00: Does well-written, performant code matter when we can just generate more of it on demand.
• 0:15
  SPEAKER_00: Now, I don't believe the AI can or should be writing all of our code, but let's pretend for a minute that it will, at some nebulous date in the future, be able to write all of our code.
• 0:27
  SPEAKER_00: In that case, does code quality matter?
• 0:30
  SPEAKER_00: That's what we're going to discuss in today's episode of Dev Questions.
• 0:42
  SPEAKER_01: Welcome to the Dev Questions Podcast with Tim Corey.
• 0:46
  SPEAKER_00: Software development is more than just writing code.
• 0:49
  SPEAKER_00: So let's talk about the rest of it.
• 0:51
  SPEAKER_00: Specifically, let's talk about code quality in an AI-powered development world.
• 0:56
  SPEAKER_00: Now, before we jump in, I think it's important for you to understand just a bit about why I don't believe AI can or should write all of our code, because one of the reasons will directly impact this conversation.
• 1:10
  SPEAKER_00: With an AI, the context window is important.
• 1:13
  SPEAKER_00: How many tokens it can ingest?
• 1:15
  SPEAKER_00: You see, every response it gives is independent of all previous responses, unless it takes those previous responses in as new inputs.
• 1:25
  SPEAKER_00: This is called context.
• 1:27
  SPEAKER_00: Context is also when it reads your code to better understand what you're doing and how you're doing it.
• 1:34
  SPEAKER_00: For example, it wouldn't know how to call an existing method you wrote unless it read it from a list of existing methods that are in your code.
• 1:44
  SPEAKER_00: It also needs to know what those methods do.
• 1:46
  SPEAKER_00: Plus, it probably needs to pull in information from MCP servers, from your markdown files, and from other locations to round out its knowledge.
• 1:56
  SPEAKER_00: All of that gives the AI the context it needs to hopefully make an informed response.
• 2:02
  SPEAKER_00: But that context isn't free.
• 2:05
  SPEAKER_00: Every bit of it takes up tokens.
• 2:07
  SPEAKER_00: A token can represent a single word or even just a part of a word.
• 2:12
  SPEAKER_00: Today's mega models usually operate on up to a million tokens of context.
• 2:17
  SPEAKER_00: The reason that's important to this discussion on why I don't believe AI can or should write all of your code is because the context window for AIs is too small for production apps.
• 2:29
  SPEAKER_00: Most production apps that you'd work on at your job will be a couple hundred thousand lines of code to over a million.
• 2:36
  SPEAKER_00: If a token is at most a small word, then we have at most one million tokens, then there's no way for an AI to ingest the entire context of just that application if it's a million lines of code.
• 2:52
  SPEAKER_00: That doesn't count the markdown files, the response history, the information about the technology has to look up via MCP, et cetera.
• 2:59
  SPEAKER_00: That's why AIs are much better at greenfield applications where they're building a new app from scratch, rather than brownfield applications where they're modifying an existing application.
• 3:10
  SPEAKER_00: I can go into a lot of further detail into this, but I think it gives you a good picture of one of my reasons why I don't think an AI can or should do all development.
• 3:21
  SPEAKER_00: So let's talk about AI code quality.
• 3:23
  SPEAKER_00: Even if you think that the AI could write all of your code, here are some reasons why code quality still matters in a code base exclusively written by AI.
• 3:33
  SPEAKER_00: Number one, a Greenfield app is only a Greenfield app until you deploy it.
• 3:39
  SPEAKER_00: Then it becomes a brownfield app.
• 3:41
  SPEAKER_00: Meaning, yes, the AI can build up an application, but remember, AI doesn't remember.
• 3:47
  SPEAKER_00: It's not like a human where it kind of keeps in its brain, oh yeah, this is what I did.
• 3:52
  SPEAKER_00: The only way it does that is by writing it in markdown files or looking it back up for every single call.
• 4:00
  SPEAKER_00: So that greenfield application at some point becomes an existing application, a brownfield application, meaning that now it's an existing application that it has to then take in as context in order to figure out what to do next.
• 4:18
  SPEAKER_00: So that application that was built by an AI at some point will be too big and unmanageable for an AI to actually fully understand and do work on.
• 4:30
  SPEAKER_00: That's point number one.
• 4:31
  SPEAKER_00: Number two is the AI cannot be trusted to write safe code.
• 4:37
  SPEAKER_00: I did a video not too long ago on Monday where I said, hey, let's ask the AI to write one method, one little bit of code.
• 4:46
  SPEAKER_00: I said, just do this one bit of code.
• 4:50
  SPEAKER_00: And almost every line of that code was wrong.
• 4:55
  SPEAKER_00: Now, the worst part in my mind was not that it was wrong on every line or almost every line, but the problem was that it worked right out of the gate.
• 5:06
  SPEAKER_00: So the code worked, but it didn't do so in a way that was good.
• 5:11
  SPEAKER_00: For example, it deleted files without ever asking.
• 5:16
  SPEAKER_00: I don't love that.
• 5:17
  SPEAKER_00: I don't love the idea of we go, wait, where was that file?
• 5:21
  SPEAKER_00: Didn't we have this file?
• 5:22
  SPEAKER_00: Oops, our code deleted it and didn't tell us.
• 5:26
  SPEAKER_00: That was for less than 30 lines of code.
• 5:30
  SPEAKER_00: It wrote bad code, it wrote non-performant code, and it wrote code that made choices that were less than safe.
• 5:39
  SPEAKER_00: Okay, that's less than 30 lines of code and it did all of that.
• 5:43
  SPEAKER_00: Imagine what it would do in 30,000 lines of code.
• 5:47
  SPEAKER_00: So the AI cannot be trusted to write safe code.
• 5:52
  SPEAKER_00: Number three, performant code is important regardless of who wrote it.
• 5:58
  SPEAKER_00: So when your user is using your application, it doesn't, the user doesn't care what language is written in.
• 6:06
  SPEAKER_00: The user does not care how great the code looks.
• 6:10
  SPEAKER_00: The user doesn't care if it was written by an AI or by a user.
• 6:14
  SPEAKER_00: What they care about is does it work and does it work fast?
• 6:19
  SPEAKER_00: And the answer if it's non-performant code is no.
• 6:24
  SPEAKER_00: So performance matters.
• 6:26
  SPEAKER_00: Your code performance still matters.
• 6:29
  SPEAKER_00: It doesn't matter that an AI wrote it because the users still need to use it.
• 6:34
  SPEAKER_00: Number four, the more code there is, the more repetition there will be.
• 6:40
  SPEAKER_00: Now, the initial pass by some people, especially those who haven't spent as much time writing code, maybe as others have, is that repetition doesn't matter because AI is maintaining it.
• 6:53
  SPEAKER_00: But why does dry exist?
• 6:56
  SPEAKER_00: Don't repeat yourself.
• 6:57
  SPEAKER_00: And this is a confusion among developers, because often developers say, well, if you wrote this model and you wrote this model again, that's a violation of dry.
• 7:05
  SPEAKER_00: No, it's not.
• 7:07
  SPEAKER_00: That's not what dry is about.
• 7:08
  SPEAKER_00: Dry is about repetition of logic.
• 7:12
  SPEAKER_00: So when I say, you know, this addition method over here, one plus one, you know, and it does the addition, and then I write that method over here, and I write it over here as well, this logic in two different places, but then I find I have a bug.
• 7:27
  SPEAKER_00: Maybe it's it's a division method instead of addition, and divide by zero, I'm not handling properly.
• 7:33
  SPEAKER_00: If I change it in one spot but miss the other spot, well then it half works, and it might hide the fact that there's still a bug in the code sometimes.
• 7:45
  SPEAKER_00: That's why repetition is dangerous, because that repetition is going to cause us to fix some things, but not all the things.
• 7:54
  SPEAKER_00: And then the bugs will hide inside of our code.
• 7:58
  SPEAKER_00: That doesn't change if AI is writing our code.
• 8:01
  SPEAKER_00: If AI writes a flaw in the logic or writes a logic a little differently, that's even worse.
• 8:06
  SPEAKER_00: It works, but it works a little differently in different spots.
• 8:10
  SPEAKER_00: That's a major problem.
• 8:12
  SPEAKER_00: And the worst part about AI code is most often it will work.
• 8:18
  SPEAKER_00: It just won't always be the right thing.
• 8:22
  SPEAKER_00: So if code works, it's more likely that people will say, sounds good.
• 8:27
  SPEAKER_00: I tested it and it seems to work.
• 8:29
  SPEAKER_00: And it goes to production, and it's only there that you find that edge case or that time when it didn't account for this, or it's doing it a little bit different, one versus the other, and it's a weird bug, it's hard to track down, and you find out, oh, that's why.
• 8:44
  SPEAKER_00: So that's number four.
• 8:45
  SPEAKER_00: Number five, when, not if, human developers need to help the AI, they're gonna need to understand the code.
• 8:54
  SPEAKER_00: Coming back to the point one where your greenfield application becomes a brownfield, at some point the AI is going to need people to step in and help.
• 9:04
  SPEAKER_00: It's gonna need some help on looking at the code and figuring out what the problem is and diagnosing why things are happening, because too often AIs will spin their wheels, they'll make bad changes, they'll make things worse and worse and worse.
• 9:18
  SPEAKER_00: And you have to step in and say, okay, let's address this.
• 9:23
  SPEAKER_00: But if humans are helping the AI, they need to understand the code.
• 9:29
  SPEAKER_00: And the more lines of code there are, the harder it's going to be to understand because there's more to comprehend.
• 9:35
  SPEAKER_00: Humans have a context window too.
• 9:38
  SPEAKER_00: Number six, the AI is known to lie with unit tests.
• 9:44
  SPEAKER_00: So how will you know that your code works in all situations?
• 9:48
  SPEAKER_00: I have people tell me, well, I just let the AI write the code, but I have great testing to make sure that it's doing the right thing.
• 9:54
  SPEAKER_00: Who wrote the tests?
• 9:56
  SPEAKER_00: Did you write the tests, or did you let the AI write the tests?
• 10:00
  SPEAKER_00: Because quite frankly, I'm not sure who likes to write tests.
• 10:03
  SPEAKER_00: Tests are not usually fun to write.
• 10:06
  SPEAKER_00: So what we do is we outsource it to the AI to write the tests.
• 10:10
  SPEAKER_00: But it has been proven over and over and over and over again that an AI without very close supervision will change the tests.
• 10:20
  SPEAKER_00: It will, you know, in its mind, it's not thinking, but what it's going to see is, hey, that test keeps failing.
• 10:27
  SPEAKER_00: And so what it does is it changes the test to make sure it passes.
• 10:31
  SPEAKER_00: But that's not how unit testing is supposed to work.
• 10:34
  SPEAKER_00: But there are times when you have to change the test.
• 10:37
  SPEAKER_00: This is where human knowledge comes in.
• 10:39
  SPEAKER_00: Because if you change the logic so that the outputs will be different and you intend them to be, well, then you have to change the test to reflect that.
• 10:49
  SPEAKER_00: But if you change the test and you do it in a way that is going to pass, that's a problem.
• 10:56
  SPEAKER_00: That's why you have red-green refactor.
• 10:58
  SPEAKER_00: You want to see it fail first before you see it pass to make sure that the changes you are making are actually the ones that are making this test pass.
• 11:07
  SPEAKER_00: Otherwise, you've probably written a bad test.
• 11:10
  SPEAKER_00: AIs are known to cheat.
• 11:12
  SPEAKER_00: They're known for changing the testing in order to get past some of the problems in their code.
• 11:21
  SPEAKER_00: That's a problem.
• 11:22
  SPEAKER_00: So these are all things that we need to think about when we all think about AI writing all of our code.
• 11:31
  SPEAKER_00: Now, even if you believe the faulty premise that AI can should write all of your code, code quality still matters.
• 11:40
  SPEAKER_00: You still need to think about code quality.
• 11:41
  SPEAKER_00: So I encourage you first, don't just let the AI write all the code.
• 11:45
  SPEAKER_00: But then, if you let the AI write any code at all, you need to make sure that it's writing code of good quality.
• 11:53
  SPEAKER_00: It's secure, it's performant, and it's readable.
• 11:57
  SPEAKER_00: All right.
• 11:58
  SPEAKER_00: Thanks for listening.
• 11:59
  SPEAKER_00: As always, I am Tim Corey.
• 12:01
  SPEAKER_01: Thank you for joining us for this episode of Dev Questions.
• 12:04
  SPEAKER_01: When you're ready to learn to think and code like a professional developer, head over to IamtimCorey.com and enroll in a course.