Latest / DevQuestions with Tim Corey / 315. Using AI Will Not Make You A Better Developer

Transcript

315. Using AI Will Not Make You A Better Developer · DevQuestions with Tim Corey

• 0:00
  SPEAKER_01: Using AI will not make you a better developer.
• 0:04
  SPEAKER_01: In fact, it will almost certainly make you a worse developer.
• 0:08
  SPEAKER_01: Whenever I bring this up, I usually get people in the comments that say something like, well, that may be true for others, but I do it the right way.
• 0:15
  SPEAKER_01: And that may be, but statistically, you almost certainly are not.
• 0:20
  SPEAKER_01: Let's talk about why and what you can do to prevent it in today's episode of Dev Questions.
• 0:34
  SPEAKER_00: Welcome to the Dev Questions Podcast with Tim Corey.
• 0:38
  SPEAKER_01: Software development is more than just writing code.
• 0:41
  SPEAKER_01: So let's talk about the rest of it.
• 0:43
  SPEAKER_01: Specifically, let's talk about why AI will not make you a better developer.
• 0:48
  SPEAKER_01: And let's start off with some of the ways that I hear that AI can make you a better developer and why I disagree with each.
• 0:55
  SPEAKER_01: Number one, AI teaches me things I didn't know.
• 0:59
  SPEAKER_01: That absolutely may be true.
• 1:01
  SPEAKER_01: It may teach you things you didn't know.
• 1:04
  SPEAKER_01: There's a couple problems though.
• 1:06
  SPEAKER_01: First off, and we need to address this probably at every point here, but AIs hallucinate.
• 1:12
  SPEAKER_01: And in fact, OpenAI came out not that long ago and said, we know they hallucinate and we cannot fix it.
• 1:19
  SPEAKER_01: So it's not like, well, the next version is better.
• 1:22
  SPEAKER_01: The next version won't hallucinate as much.
• 1:25
  SPEAKER_01: The company that's making the LLM says it's it's not going to go away.
• 1:30
  SPEAKER_01: It's part of how AIs work.
• 1:34
  SPEAKER_01: Even if you say, well, you know, hallucinations aren't that bad, which they are.
• 1:39
  SPEAKER_01: But if you ask an AI a question and then come back the next day and ask the exact same question, it will not give you the same answer.
• 1:49
  SPEAKER_01: It may give you an answer as close, but it won't be the same one.
• 1:52
  SPEAKER_01: And why?
• 1:53
  SPEAKER_01: Because AIs are not deterministic.
• 1:56
  SPEAKER_01: They don't give a set answer to a set question, even if you ask the same question the same way every time.
• 2:03
  SPEAKER_01: So when it comes to learning new things from the AI, how do you know it's the right thing?
• 2:10
  SPEAKER_01: How do you know it's the correct thing?
• 2:12
  SPEAKER_01: AIs are not trained on the latest version of everything, because that's just it comes out after the LLM was trained, which means that when you're talking about the latest information, like, hey, how should I do this in.NET 10?
• 2:29
  SPEAKER_01: Well, it has to go out and use retrieval augmented generation or RAG to bring in additional sources, learn from that, and figure it out.
• 2:38
  SPEAKER_01: It might not bring the right sources in, or it might not even know to bring certain sources in.
• 2:44
  SPEAKER_01: For a very long time, until just about a month ago, the AIs were saying that.NET 10 was still in preview.
• 2:53
  SPEAKER_01: And that was in people's documentation, that was in their, you know, in the code that was written, in the answers that were given, not just the AI itself, but also the things the AI generated, which then reinforced that the AI was quote unquote correct when it was not.
• 3:10
  SPEAKER_01: Because once it starts getting out there, then it kind of self-reinforces if you're not careful.
• 3:17
  SPEAKER_01: So hallucinations are a problem.
• 3:19
  SPEAKER_01: So it might teach you things you didn't know, but it might not be the correct things that you didn't know that it's teaching you.
• 3:27
  SPEAKER_01: Also, and this is one that people sometimes miss, is if you know the question to ask, you can usually get a decent answer.
• 3:38
  SPEAKER_01: But too often you don't know what you don't know.
• 3:44
  SPEAKER_01: This is the um, you know, when people come into a field, they start to think they know a lot about the field because they've learned a lot.
• 3:52
  SPEAKER_01: When I was teaching C sharp in classrooms, I was, you know, teaching basically almost all the syntax for C sharp in the first five weeks of the of a semester.
• 4:02
  SPEAKER_01: The problem is the semester is 16 weeks long.
• 4:05
  SPEAKER_01: And so students would start to say, well, you know, I know all of this.
• 4:09
  SPEAKER_01: And yeah, you know all the syntax, but you don't know the questions to ask.
• 4:14
  SPEAKER_01: You don't know the depth of how to use those things well.
• 4:18
  SPEAKER_01: You don't know what you don't know.
• 4:20
  SPEAKER_01: And by the time they got through weeks 10, 11, 12, they started going, wow, there's a lot I don't know.
• 4:28
  SPEAKER_01: Because they learned enough to know what they didn't know, or at least know there's so much more that is out there.
• 4:34
  SPEAKER_01: When you don't know what you don't know, you don't know to ask for it.
• 4:38
  SPEAKER_01: And so you can't prompt your way into figuring out what you don't know.
• 4:41
  SPEAKER_01: Sometimes the AI will expose you to things you didn't think about or didn't know about.
• 4:46
  SPEAKER_01: But that doesn't mean that you've answered all those questions.
• 4:51
  SPEAKER_01: So that's a problem because you don't, you're leaving gaps in your knowledge, you're putting the wrong things in your knowledge, and so what you're building on is a really shaky foundation.
• 5:03
  SPEAKER_01: Number two, this uh you know, I just use an AI to answer questions.
• 5:09
  SPEAKER_01: So I don't use AI for anything else.
• 5:10
  SPEAKER_01: I just use AI to answer questions.
• 5:13
  SPEAKER_01: So, okay, and the AIs can do a pretty good job of answering questions.
• 5:18
  SPEAKER_01: I use an AI, just to be clear, this is not anti-AI, but you have to be careful how you use the AI.
• 5:24
  SPEAKER_01: So I use an AI to answer questions a lot.
• 5:29
  SPEAKER_01: But here's the thing: you need to verify because I also, when I go to the links where it says, hey, this is the, you know, this backs up my claim.
• 5:38
  SPEAKER_01: I go to those links and often find out that those links are either incorrect or they say the opposite of what the AI just told me.
• 5:47
  SPEAKER_01: So you need to verify those claims by actually going to the source, not just stopping at what the AI told you, because the AI will sound intelligent.
• 5:58
  SPEAKER_01: It will sound coherent, it will be reasonable, it will make sense.
• 6:03
  SPEAKER_01: It just won't always be right.
• 6:06
  SPEAKER_01: The other thing is that how you ask a question will determine what type of answer you get.
• 6:14
  SPEAKER_01: So if you ask a question in a positive manner, you're more likely to get a positive result.
• 6:22
  SPEAKER_01: If you ask a question in a skeptical manner, you are more likely to get a skeptical result.
• 6:29
  SPEAKER_01: And this is not always true.
• 6:31
  SPEAKER_01: And yes, you can ask a really negative question and get back a positive answer sometimes.
• 6:36
  SPEAKER_01: It this is not an always thing.
• 6:38
  SPEAKER_01: That's the problem, is that AI will give good answers a lot.
• 6:43
  SPEAKER_01: But too often what happens is an AI will fall into the yes man category.
• 6:49
  SPEAKER_01: The idea that you're looking for validation and it will give you validation.
• 6:54
  SPEAKER_01: But that's not a good way to get questions answered.
• 6:58
  SPEAKER_01: Because then what happens is it will start to enable you to do things that are incorrect or that are just not wise.
• 7:06
  SPEAKER_01: Uh, I did an example of this recently where I asked the AI, how do I use SQLite with Blazor WebAssembly?
• 7:14
  SPEAKER_01: Now, the answer should be don't do that.
• 7:18
  SPEAKER_01: Because it's almost certainly the wrong solution for the job.
• 7:24
  SPEAKER_01: Because the fact that Blazor WebAssembly puts all the code in the client, which means the client can read all the user secrets, they can read the database directly, which means that nothing in that database should be or can be sensitive.
• 7:39
  SPEAKER_01: Otherwise, you're exposing secrets or exposing data that you didn't expect to because the user has access to more than just what the Blazor WebAssembly allows them to get into.
• 7:50
  SPEAKER_01: So there are use cases where you might want to think about maybe kind of SQLite, but there's much better solutions than SQLite for Blazor Web Assembly if you're going to go the client-side data route.
• 8:09
  SPEAKER_01: You know, for example, if you're caching data, well, you could put it into Index DB or more likely put it into something like session storage or local storage.
• 8:19
  SPEAKER_01: So kind of a side trail, but I want you to understand this is not what the AI told me.
• 8:24
  SPEAKER_01: The AI did say, hey, you should probably think about using an API and put the SQLite database behind that.
• 8:33
  SPEAKER_01: That is correct.
• 8:34
  SPEAKER_01: However, in its conclusion, it said that that solution was good for testing and for um if you're teaching it in a classroom, which it knows I'm a teacher.
• 8:46
  SPEAKER_01: Um so that may be why it said that, but that's not true.
• 8:50
  SPEAKER_01: That's the preferred method if you need to have data that is securable.
• 8:57
  SPEAKER_01: Now, if you're doing caching data or offline, what it said was, hey, yes, you can do SQLite.
• 9:03
  SPEAKER_01: And it showed me how to do that.
• 9:05
  SPEAKER_01: But then in one of the little notes, it said, one of the times, it said, just so you know, this is only in memory, it will go away on page refresh, and it's not gonna be persistent, et cetera.
• 9:19
  SPEAKER_01: That's probably not what you were asking for.
• 9:22
  SPEAKER_01: And so then it will, in passing, sometimes mention, hey, index db is a way to go.
• 9:28
  SPEAKER_01: And that could be the case, but there's again better solutions probably for Blaze WebAssembly.
• 9:34
  SPEAKER_01: What it didn't do was ask questions that revealed how I was going to use that data.
• 9:40
  SPEAKER_01: It didn't go further and say, well, what's your exact situation?
• 9:44
  SPEAKER_01: And let's talk about going back and forth on figuring out what the right solution is for you, for your use case, for what you're wanting to do.
• 9:52
  SPEAKER_01: It didn't do that, and it gave some bad recommendations that are correct for a limited set of circumstances, but probably are not right for me.
• 10:03
  SPEAKER_01: So this is the problem when you ask a question, is it will give you the answer, but it will not necessarily give you the right answer for your situation.
• 10:13
  SPEAKER_01: It will give you a general answer.
• 10:15
  SPEAKER_01: And that's that's why we say it depends so much.
• 10:18
  SPEAKER_01: The AI does not say it depends almost ever.
• 10:22
  SPEAKER_01: Instead, what it does is it gives you, and even in my case, it gave me a kind of an it depends answer, but gave me two or three solutions, but never try to figure out why one might be better than the other for me.
• 10:35
  SPEAKER_01: So just asking it questions, you're gonna get some bad answers.
• 10:39
  SPEAKER_01: And again, build up uh a level of bad knowledge in your head and add kind of more potholes to the road of your of your learning journey.
• 10:48
  SPEAKER_01: Number three, I hear people say, I let AI write the code, but then I review it.
• 10:55
  SPEAKER_01: Yeah, I I really doubt this is true.
• 10:58
  SPEAKER_01: Um, and the reason why is because even before AI came out, there had been studies that found that the larger the pull request, the less it got reviewed.
• 11:08
  SPEAKER_01: So if you have a pull request of 50 lines, 20 lines of code, it got thorough review more often.
• 11:16
  SPEAKER_01: But if you had a pull request of 2000 lines, more often you got the looks good, or here's a couple tweaks, but for the most part, yeah, go for it.
• 11:26
  SPEAKER_01: Um that was even before AI.
• 11:28
  SPEAKER_01: And now we're doing it every day.
• 11:31
  SPEAKER_01: It's very easy to get um laxadaisical.
• 11:35
  SPEAKER_01: It's very easy to um just kind of skim over things and see that it looks right.
• 11:40
  SPEAKER_01: Does it have the right naming conventions?
• 11:42
  SPEAKER_01: Does it have the right file conventions?
• 11:44
  SPEAKER_01: Is it does it have the right physical structure?
• 11:48
  SPEAKER_01: Well, those are all great, but that's really not what a code review should be thinking about primarily.
• 11:54
  SPEAKER_01: A code review primarily should be thinking about is this the right code?
• 11:57
  SPEAKER_01: Is this the right way of solving it?
• 11:59
  SPEAKER_01: Um, is this the way we should go about it compared to the rest of our code, et cetera.
• 12:06
  SPEAKER_01: So yeah, it's it's hard.
• 12:09
  SPEAKER_01: And the other thing is it's hard to catch on review.
• 12:12
  SPEAKER_01: So when you're looking at your code and you're skimming through code, and that's what you're doing is skimming, you're not actually reading, um, it's hard to really understand what it's doing and then understand is this the right thing to do?
• 12:24
  SPEAKER_01: Um, it's hard to catch on review.
• 12:26
  SPEAKER_01: Versus if you wrote it.
• 12:27
  SPEAKER_01: When you're writing it, you're thinking about it more.
• 12:29
  SPEAKER_01: You're thinking about it more deeply.
• 12:32
  SPEAKER_01: So one of the things I ask people is how many times do you push back on the AI for any any block of code?
• 12:39
  SPEAKER_01: Because the reality is you should probably be pushing back on every pull request for sure, probably every method that the AI writes, you should be tweaking.
• 12:49
  SPEAKER_01: And if you're not, maybe you're just not writing good enough code or having the AI write good enough code.
• 12:56
  SPEAKER_01: So this is, you know, when you're thinking about reviewing code, you're not actually learning much except for the fact that you're, you know, maybe learning how to review better because you're practicing it more.
• 13:10
  SPEAKER_01: And that's, you know, that is valuable.
• 13:12
  SPEAKER_01: But it takes a lot of work to be good at review, and it's even harder because of the fact that you really should be writing a lot of code to help you in your review of code.
• 13:24
  SPEAKER_01: But you're not writing, you're just letting the AI write that.
• 13:27
  SPEAKER_01: Uh, number four, I use AI to write the code I don't understand.
• 13:34
  SPEAKER_01: This is for me as a developer, it terrifies me.
• 13:38
  SPEAKER_01: Uh, I see this a lot.
• 13:39
  SPEAKER_01: You know, I don't know its language, but I I had AI write code for it and it worked.
• 13:44
  SPEAKER_01: Um, how do you know it's the right code?
• 13:48
  SPEAKER_01: How do you review that code?
• 13:50
  SPEAKER_01: How do you know what to do to support that code?
• 13:54
  SPEAKER_01: When that code breaks, how do you fix it?
• 13:57
  SPEAKER_01: And the answer can't just be, well, how do the AI fix it?
• 13:59
  SPEAKER_01: Because as we know, AI is worse at maintaining code than it is at creating new code.
• 14:05
  SPEAKER_01: And so often what happens is AI will throw code out and start over.
• 14:10
  SPEAKER_01: That happens a lot.
• 14:11
  SPEAKER_01: Um and that's not the right solution.
• 14:14
  SPEAKER_01: It can't just keep throwing code out and starting over.
• 14:17
  SPEAKER_01: So how do you maintain that code if you don't understand how it works?
• 14:22
  SPEAKER_01: And if you do write, you know, have the AI write code you don't understand, because you know, I don't know how to do authentication, so I'll have AI do it.
• 14:30
  SPEAKER_01: Terrifying.
• 14:31
  SPEAKER_01: But um, if that is the case, how do you guard against hallucination?
• 14:36
  SPEAKER_01: How do you guard against it doing the wrong thing?
• 14:38
  SPEAKER_01: Because now if you're gonna try and learn from that, you're learning the wrong thing, you're learning the hallucination.
• 14:44
  SPEAKER_01: The worst thing that AI does, in my opinion, is it writes code that compiles.
• 14:50
  SPEAKER_01: And if you notice, if you're letting the code, the agent write the code, um, and again, I do some of this, but when you let the agent write the code, and it writes the code and it compiles and it fails.
• 15:02
  SPEAKER_01: And then it goes, oh, I'll I'll check this and it tries something else and it fails, and I'll check something else.
• 15:06
  SPEAKER_01: Oh, and then it fixes it.
• 15:08
  SPEAKER_01: And you're like, wow, that's great.
• 15:09
  SPEAKER_01: It fixed all the problems.
• 15:11
  SPEAKER_01: No, what it did was it got the code to compile.
• 15:14
  SPEAKER_01: That's different than fixing all the problems.
• 15:17
  SPEAKER_01: If you think about it, it wrote bad code, code that did not compile.
• 15:24
  SPEAKER_01: And so it's not understanding the code, it's just writing code.
• 15:29
  SPEAKER_01: And it's you know, throwing darts against the wall sometimes until it gets it right.
• 15:34
  SPEAKER_01: That's actually worse than code that doesn't compile.
• 15:37
  SPEAKER_01: Because if it works, if it passes your tests, well, then it must be right, correct?
• 15:42
  SPEAKER_01: No.
• 15:43
  SPEAKER_01: What it means is maybe the happy path works, but there's probably a lot of hidden things we don't know.
• 15:51
  SPEAKER_01: So when you let the AI write code you don't understand, then again, you're putting potholes in your path to learning because what you're gonna learn is code that often we write, but sometimes won't.
• 16:05
  SPEAKER_01: And you won't know when it's not.
• 16:07
  SPEAKER_01: Okay.
• 16:07
  SPEAKER_01: Now I'm sure there's more ways that people use AI they think is actually helping them be a better developer.
• 16:14
  SPEAKER_01: But here's the truth about software development.
• 16:17
  SPEAKER_01: The way to get better as a developer is to do the work.
• 16:21
  SPEAKER_01: You don't get stronger by watching a video about lifting weights or by watching a robot lift the weights.
• 16:26
  SPEAKER_01: What you might learn a better technique through a video, and the robot might be helpful for setting things up or tearing things down.
• 16:35
  SPEAKER_01: But at the end of the day, you need to lift the weights.
• 16:39
  SPEAKER_01: So this is not about can I use AI as a tool?
• 16:43
  SPEAKER_01: Absolutely.
• 16:44
  SPEAKER_01: Can AI make your development life faster?
• 16:47
  SPEAKER_01: Absolutely.
• 16:48
  SPEAKER_01: But don't think that it's making you a better developer because that's different.
• 16:53
  SPEAKER_01: Yes, it can make you faster.
• 16:55
  SPEAKER_01: We we have so many tools to make us faster as developers.
• 16:59
  SPEAKER_01: My primary preferred language is C sharp.
• 17:02
  SPEAKER_01: That itself is a tool to make me faster.
• 17:06
  SPEAKER_01: It's a high-level language, not a low-level language.
• 17:10
  SPEAKER_01: It does a lot of the work for me.
• 17:12
  SPEAKER_01: That makes me faster.
• 17:14
  SPEAKER_01: Visual Studio with code completions and other things makes me faster as a developer.
• 17:19
  SPEAKER_01: But that doesn't make me a better developer.
• 17:22
  SPEAKER_01: Because I have Visual Studio doesn't make me a better developer.
• 17:25
  SPEAKER_01: I can have a piece of notepad or notepaper and be a better developer, right?
• 17:31
  SPEAKER_01: So it's not the tool that makes me a better developer.
• 17:34
  SPEAKER_01: It makes me be faster sometimes.
• 17:36
  SPEAKER_01: But to be a better developer, you have to put in the work.
• 17:40
  SPEAKER_01: You have to be the one that actually exercises those skills of development.
• 17:46
  SPEAKER_01: Not watching somebody else develop, not just having someone tell you what to do step by step in development.
• 17:54
  SPEAKER_01: You need to put the work in.
• 17:56
  SPEAKER_01: Prompting AI gives you very little experience.
• 17:59
  SPEAKER_01: Outsourcing your thinking gives you very little experience.
• 18:03
  SPEAKER_01: The only thing that will make you a better developer in the long term is putting in the effort of actually building things.
• 18:11
  SPEAKER_01: And again, it doesn't mean that AI can't help you.
• 18:14
  SPEAKER_01: It is a tool.
• 18:16
  SPEAKER_01: But you the more you let the tool do the work for you, the less value you get out of it.
• 18:22
  SPEAKER_01: So make sure that you're putting in the work, you're doing the heavy lifting, at least in some places, to make sure you keep up your strength and improve your strength as a developer.
• 18:33
  SPEAKER_01: Thanks for listening.
• 18:34
  SPEAKER_01: As always, I am Tim Corey.
• 18:37
  SPEAKER_00: Thank you for joining us for this episode of Dev Questions.
• 18:40
  SPEAKER_00: When you're ready to learn to think and code like a professional developer, head over to IamtimCorey.com and enroll in a course.