Latest / DevQuestions with Tim Corey / 328. Do We Still Need QA In Light Of AI

Transcript

328. Do We Still Need QA In Light Of AI · DevQuestions with Tim Corey

• 0:05
  SPEAKER_00: With AI writing so much code, do we even need a quality assurance department?
• 0:10
  SPEAKER_00: If I can ask my AI to evaluate code, what's the purpose of having a QA person around?
• 0:16
  SPEAKER_00: Now right up front, I'm gonna tell you that yes, QA is absolutely valuable and not replaced by AI.
• 0:23
  SPEAKER_00: And as an aspiring developer, going into a QA role might be a good choice for you.
• 0:29
  SPEAKER_00: But let's talk about why QA is valuable still and why AI can't just replace them in today's episode of DevQuestions.
• 0:45
  SPEAKER_00: Software development is more than just writing code.
• 0:48
  SPEAKER_00: So let's talk about the rest of it.
• 0:50
  SPEAKER_00: Specifically, let's talk about quality assurance team members and why they are still valuable.
• 0:56
  SPEAKER_00: And let me start off by saying that not every company can afford a dedicated QA person or team.
• 1:02
  SPEAKER_00: I've been on a number of those teams.
• 1:04
  SPEAKER_00: But in that case, you absolutely need someone who can fill that role.
• 1:09
  SPEAKER_00: So the question becomes: what does a good QA person do?
• 1:13
  SPEAKER_00: And I did say good because not every person in QA does a good job, just like not every developer does a good job.
• 1:21
  SPEAKER_00: But let's talk about what a good QA person does.
• 1:23
  SPEAKER_00: And I put it into four categories.
• 1:26
  SPEAKER_00: The first category is ensure software quality, not necessarily make software quality.
• 1:33
  SPEAKER_00: Again, QA people are not necessarily developers.
• 1:36
  SPEAKER_00: They can be, and I find that's often valuable if they have some development skills.
• 1:42
  SPEAKER_00: But they're ensuring software quality by doing things like testing.
• 1:47
  SPEAKER_00: They're looking at regression and making sure that they prevent regression in their code.
• 1:52
  SPEAKER_00: By the way, if you don't know what regression is, it means maybe you fix one problem, but you solve or you cause other problems while you do so.
• 2:00
  SPEAKER_00: That's regression.
• 2:01
  SPEAKER_00: Um, something that was working isn't working anymore.
• 2:04
  SPEAKER_00: So they they work on testing, they work on preventing regression, they work on identifying defects.
• 2:09
  SPEAKER_00: They try and pick apart your code, how it works, and find all the issues.
• 2:15
  SPEAKER_00: So they ensure software quality.
• 2:17
  SPEAKER_00: That's number one category.
• 2:19
  SPEAKER_00: Number two is they understand the requirements, which sounds weird, right?
• 2:25
  SPEAKER_00: That's that's one of their jobs.
• 2:26
  SPEAKER_00: Yes, absolutely.
• 2:28
  SPEAKER_00: They need to be the ones that really understand the requirements.
• 2:31
  SPEAKER_00: Now, developers, we should know the requirements backwards and forwards.
• 2:35
  SPEAKER_00: But the QA person needs to know from the user perspective what the requirements are.
• 2:42
  SPEAKER_00: And a lot of times the requirements don't get put into a document.
• 2:46
  SPEAKER_00: And yes, I know some people love to have these huge long documents that explain everything about the system.
• 2:53
  SPEAKER_00: It's never enough.
• 2:54
  SPEAKER_00: It's never enough, it's never right, uh, it's never complete.
• 2:58
  SPEAKER_00: Understanding the requirements from the user's perspective is really important for QA.
• 3:03
  SPEAKER_00: Now, this may be where there's a hybrid role with the business analyst.
• 3:08
  SPEAKER_00: Business analysts and QA people are different sometimes and sometimes the same person.
• 3:14
  SPEAKER_00: Um, I've I've seen pros and cons for both.
• 3:16
  SPEAKER_00: But if they're in a hybrid role, this is where they also have gathered the requirements in the first place.
• 3:22
  SPEAKER_00: They're the person that sits between the user and the developer, and they know the role really well.
• 3:29
  SPEAKER_00: So either way, they understand the requirements.
• 3:32
  SPEAKER_00: This is really important because they're the ones that are gonna test the requirements.
• 3:36
  SPEAKER_00: They're the ones that are gonna validate if the requirements are being met.
• 3:40
  SPEAKER_00: They're a person outside of the developer who knows what should happen and verifies it does happen.
• 3:49
  SPEAKER_00: Because too often as developers, we're really close to the issue.
• 3:52
  SPEAKER_00: And you know, we've got our nose right next to the code.
• 3:54
  SPEAKER_00: We're we're working on making the code, we kind of lose track a bit of the overall goal sometimes.
• 4:00
  SPEAKER_00: Or we get you know working on our feature and somebody else working on their feature, and we kind of mash all together and say, hey, there you go.
• 4:06
  SPEAKER_00: And there's no one looking at it and says, wait, wait, wait, here's the overall requirements.
• 4:11
  SPEAKER_00: We're missing some.
• 4:13
  SPEAKER_00: Or these things work, but they don't work well together.
• 4:16
  SPEAKER_00: Like understanding the requirements is really important.
• 4:19
  SPEAKER_00: And then understanding how to evaluate if you're meeting those requirements is really important.
• 4:25
  SPEAKER_00: Number three, kind of goes with number two, but number three category is they represent the user.
• 4:31
  SPEAKER_00: And that means they're looking for things like is the is the application intuitive?
• 4:38
  SPEAKER_00: Your application can work, it can meet all the requirements, and it might not be intuitive, which means it really should fail QA because the user has to be able to use it.
• 4:50
  SPEAKER_00: I was talking to a person on YouTube today about this where they said, Hey, I think I've built the right user interface, but the user says they want something different.
• 4:59
  SPEAKER_00: What do I do?
• 5:01
  SPEAKER_00: Well, at the end of the day, the user is the one using your application.
• 5:06
  SPEAKER_00: I built systems that that worked and that were logical and that had all the right pieces and they weren't intuitive or they didn't behave as expected.
• 5:17
  SPEAKER_00: That's another part of this, is they check to make sure it behaves as expected.
• 5:20
  SPEAKER_00: For example, tab order.
• 5:22
  SPEAKER_00: This is one that I used to get wrong every once in a while when I was a young developer.
• 5:26
  SPEAKER_00: Uh I was using uh VB6 when I was first building applications, and you would drag things out of the form.
• 5:32
  SPEAKER_00: But if you drag things in the form in the wrong order, well, it would assign tab order based upon what the last tab item was that was put in the form.
• 5:40
  SPEAKER_00: So I'd I'd put things in the form and then drag and rearrange them, but the tab order wouldn't rearrange with it.
• 5:47
  SPEAKER_00: So maybe you know, a simple example, you might be in a uh a field that asks you for your address.
• 5:51
  SPEAKER_00: So it's street address, city, state, zip.
• 5:54
  SPEAKER_00: And you go from street address and you hit tab and it puts in the zip code field.
• 5:58
  SPEAKER_00: And then you hit tab and it goes back to state.
• 6:01
  SPEAKER_00: And you hit tab and it goes to street address too, or like it, it's all over the place.
• 6:04
  SPEAKER_00: And you're like, what's going on here?
• 6:06
  SPEAKER_00: That's not behaving as expected.
• 6:09
  SPEAKER_00: Checking tab order is not necessarily something that developers often do.
• 6:13
  SPEAKER_00: We say, yeah, they're all there.
• 6:15
  SPEAKER_00: But the QA person goes, yes, but it's not behaving as expected.
• 6:20
  SPEAKER_00: And then with this, they're representing the user when it comes to performance, where they go, yeah, it all works, but it takes five seconds to load a new page and they need to move.
• 6:30
  SPEAKER_00: I've worked with people who have worked on a heads-down program, I call them, where they're not using their mouse.
• 6:37
  SPEAKER_00: They're just putting as much information in as possible.
• 6:41
  SPEAKER_00: They're just taking the data from paper and putting it in the computer, or they're filling out information as fast as possible, and they want to make sure their system doesn't hold them back.
• 6:52
  SPEAKER_00: And I've built systems where they held them back, and they're like, you're killing me.
• 6:56
  SPEAKER_00: Because my job is dependent on how fast I do things, and your software is making me go slower.
• 7:03
  SPEAKER_00: That's a problem.
• 7:04
  SPEAKER_00: QAs have to step in and say, hey, you know what?
• 7:06
  SPEAKER_00: No, this is not performant.
• 7:08
  SPEAKER_00: This is not doing the right thing for the user.
• 7:11
  SPEAKER_00: This is not the representing the user and saying, hey, they need better.
• 7:15
  SPEAKER_00: So that's category number three.
• 7:17
  SPEAKER_00: Category number four is they improve the process.
• 7:20
  SPEAKER_00: So the process of getting the requirements to the developers, getting the requirements turned into code, getting the code tested, and getting the code back out to the user, that whole process can be improved by QA.
• 7:34
  SPEAKER_00: That means they might create automated tests.
• 7:36
  SPEAKER_00: It might mean they improve uh procedures and how you do things.
• 7:40
  SPEAKER_00: That might mean they train the developers on where the bugs live or what the users actually want from their application so they can just do it the next time instead of having to have it come back from QA as a fail.
• 7:52
  SPEAKER_00: And they build better test cases where they say, hey, you know what?
• 7:56
  SPEAKER_00: There's some things that I've seen in when the users are using these things that we hadn't accounted for.
• 8:01
  SPEAKER_00: And let's let's account for those things.
• 8:03
  SPEAKER_00: So QA does a lot of great work.
• 8:06
  SPEAKER_00: And there's a there's a difference between working on software and working on good software, right?
• 8:15
  SPEAKER_00: There's a difference between saying, hey, it works and it works well.
• 8:20
  SPEAKER_00: And QA is often the group that brings out that difference.
• 8:24
  SPEAKER_00: So let's ask the question here: why can't AI do this?
• 8:28
  SPEAKER_00: Because it seems like that's a role for AI, right?
• 8:32
  SPEAKER_00: No, because AI doesn't actually fully test its own code.
• 8:36
  SPEAKER_00: You might see, you know, AI writes code and goes, oh, I've also written 150 tests.
• 8:41
  SPEAKER_00: Yeah, but those tests are written by the same person who created the code in the first place.
• 8:47
  SPEAKER_00: They I'll often find the tests are really bad because they have the same assumptions as the code does.
• 8:54
  SPEAKER_00: And so they're not actually testing that the functionality works the way it should.
• 8:59
  SPEAKER_00: They're testing the same assumptions they had that were incorrect.
• 9:03
  SPEAKER_00: So that's a problem, but it doesn't even understand all the code.
• 9:10
  SPEAKER_00: And so it doesn't understand the greater context because it doesn't even have enough context window to put the entire code base in.
• 9:17
  SPEAKER_00: And so what it does instead is it makes assumptions.
• 9:21
  SPEAKER_00: And we all know that assumptions aren't good when it comes to code.
• 9:26
  SPEAKER_00: Number two, AI only understands what you tell it.
• 9:29
  SPEAKER_00: And even then, not necessarily all what you tell it.
• 9:33
  SPEAKER_00: So when it comes to requirements, somebody has to gather those.
• 9:37
  SPEAKER_00: And the user saying, here's my requirements, isn't gonna be good enough.
• 9:41
  SPEAKER_00: And the developer saying, Well, I'm gonna give it better requirements.
• 9:44
  SPEAKER_00: Unless you're good at QA, you're probably not gonna give it great instructions either.
• 9:49
  SPEAKER_00: You'll get better instructions, much better instructions.
• 9:52
  SPEAKER_00: But there is something to understanding fully what the user actually wants, how it actually needs to work, not just what they say, and then translate that into something that can be easier to understand as a developer.
• 10:07
  SPEAKER_00: That's just something that AI can't do.
• 10:09
  SPEAKER_00: Also, AI does not represent the user.
• 10:12
  SPEAKER_00: It doesn't understand the user, it doesn't understand where they're coming from, it doesn't empathize with how they've done the work.
• 10:19
  SPEAKER_00: It hasn't actually done the job.
• 10:20
  SPEAKER_00: It hasn't learned because AIs don't learn.
• 10:24
  SPEAKER_00: And so, yeah, even the memory files, not good enough.
• 10:27
  SPEAKER_00: So it doesn't represent the user to go, hey, this needs to be more performant because this user does it things this way.
• 10:33
  SPEAKER_00: It doesn't know about that unless you told it, which again comes back to it doesn't understand anything you don't tell it.
• 10:40
  SPEAKER_00: And number four, AI isn't good at performing or improving procedures.
• 10:45
  SPEAKER_00: Yes, if you push it and ask it and work with it, it can give you better procedures sometimes if it actually understands the problem.
• 10:53
  SPEAKER_00: But more often than not, it's basically you like frog marching it until it gets to the point where it like gives you back what you were asking for in the first place.
• 11:04
  SPEAKER_00: Right.
• 11:04
  SPEAKER_00: So it's not good at improving procedures.
• 11:07
  SPEAKER_00: So you need QA.
• 11:10
  SPEAKER_00: And as a developer, if you haven't worked with a good QA department or you don't have access to one, I would encourage you to work on building up your own skills in this area because this is going to make you a better developer.
• 11:23
  SPEAKER_00: The more you understand the QA role, the better you get at it, the better you will be as a developer.
• 11:28
  SPEAKER_00: And which is also why I encourage new developers who are struggling to find development roles to look at the QA roles.
• 11:34
  SPEAKER_00: They aren't the same and they often don't include development.
• 11:38
  SPEAKER_00: So it's not like you're just, you know, developer and waiting necessarily.
• 11:42
  SPEAKER_00: They can be.
• 11:43
  SPEAKER_00: But what it does do is it gives you the skills and experience you're gonna need to be a better developer when your time comes.
• 11:50
  SPEAKER_00: The role of a QA developer is something that is badly needed in a lot of organizations.
• 11:56
  SPEAKER_00: I see a lot of companies that are kind of skipping that because who needs quality anymore, right?
• 12:01
  SPEAKER_00: We have quantity.
• 12:02
  SPEAKER_00: Quantity is not the solution.
• 12:04
  SPEAKER_00: Quality is a solution.
• 12:05
  SPEAKER_00: That's where you get the sticking power of an application.
• 12:07
  SPEAKER_00: That's where things go from they work to they work well.
• 12:12
  SPEAKER_00: So if you have your QA people in your team, give them a hug.
• 12:16
  SPEAKER_00: They're awesome.
• 12:17
  SPEAKER_00: Um learn from them.
• 12:19
  SPEAKER_00: And if you don't, learn how to be them.
• 12:21
  SPEAKER_00: And even if you do, learn from them.
• 12:23
  SPEAKER_00: Learn how to be a better developer because you learn from your QA people and get better at understanding what they do, why they do it, and how you can incorporate some of those same techniques into your development experience.
• 12:36
  SPEAKER_00: Okay.
• 12:37
  SPEAKER_00: QA is important.
• 12:39
  SPEAKER_00: AI can't replace it.
• 12:40
  SPEAKER_00: It is a necessary role to put out good software, not just working software.
• 12:46
  SPEAKER_00: All right.
• 12:46
  SPEAKER_00: Thanks for listening.
• 12:47
  SPEAKER_00: As always, I am Tim Corey.