Latest / DevQuestions with Tim Corey / 247. Do Developers Need To Know About Databases?

Transcript

247. Do Developers Need To Know About Databases? · DevQuestions with Tim Corey

• 0:00
  SPEAKER_01: Do developers need to know about databases?
• 0:03
  SPEAKER_01: Do you need to know SQL to be a good developer?
• 0:07
  SPEAKER_01: Let's talk about it in today's episode of Dev Questions.
• 0:18
  SPEAKER_00: Welcome to the Dev Questions Podcast with Tim Corey.
• 0:21
  SPEAKER_01: Software development is more than just writing code.
• 0:24
  SPEAKER_01: So let's talk about the rest of it.
• 0:26
  SPEAKER_01: Specifically, let's talk about if developers need to know about the database.
• 0:32
  SPEAKER_01: Being a developer means you need to know at least a little bit about a lot of things, including source control, CICD, multiple languages, the cloud, and more.
• 0:42
  SPEAKER_01: For most of those areas, you can get by with knowing just enough.
• 0:46
  SPEAKER_01: But what about your database?
• 0:48
  SPEAKER_01: Should you know more?
• 0:50
  SPEAKER_01: Now, normally my answer is it depends.
• 0:54
  SPEAKER_01: And since I said the word need, I guess that's true here.
• 0:58
  SPEAKER_01: However, the real answer is yes.
• 1:01
  SPEAKER_01: Yes, you need to know about the database.
• 1:03
  SPEAKER_01: But let's talk about why.
• 1:05
  SPEAKER_01: Okay?
• 1:06
  SPEAKER_01: It's important to understand, and let's just use SQL today, but it can be any database that you're using.
• 1:13
  SPEAKER_01: So it's important to understand SQL well when you're using it, not just say, well, entity framework will handle that for me.
• 1:21
  SPEAKER_01: All right.
• 1:22
  SPEAKER_01: So here's why.
• 1:23
  SPEAKER_01: Number one, applications exist to store and retrieve data.
• 1:28
  SPEAKER_01: That's really the purpose behind applications.
• 1:32
  SPEAKER_01: And you might say, well, no, no, no, not my application.
• 1:35
  SPEAKER_01: Think about it.
• 1:36
  SPEAKER_01: What's it doing?
• 1:37
  SPEAKER_01: It's showing information, it's asking for information, it's storing information, it's manipulating information, it's displaying information again.
• 1:45
  SPEAKER_01: Okay?
• 1:46
  SPEAKER_01: That's what your application does.
• 1:49
  SPEAKER_01: Now, there's a lot of complexities around that, but that's the whole purpose of your application.
• 1:55
  SPEAKER_01: So your application really exists to process, handle, etc., that data from your database.
• 2:06
  SPEAKER_01: So the two of those need to be able to talk well together.
• 2:11
  SPEAKER_01: Because if your application is awesome and your database isn't, well, guess what?
• 2:16
  SPEAKER_01: They're connected, they're linked.
• 2:19
  SPEAKER_01: And so you're going to have a bad application.
• 2:22
  SPEAKER_01: So it's important that the two work well together.
• 2:26
  SPEAKER_01: Because the application exists to store and retrieve data.
• 2:30
  SPEAKER_01: All right.
• 2:31
  SPEAKER_01: Now, number two, performance is a key factor of your application's health, which makes sense, right?
• 2:39
  SPEAKER_01: The faster your application runs, the more people can use it concurrently, the more you can scale up your application to more users.
• 2:49
  SPEAKER_01: Application performance is important.
• 2:53
  SPEAKER_01: Even in today's society where we have so much RAM and so much hard drive space and so much processor power that it feels like we don't need to think about these things.
• 3:03
  SPEAKER_01: But the reality is you do, at least to an extent, especially if you're using your application at any scale.
• 3:11
  SPEAKER_01: So with that performance being a key factor, well, again, your application is tied directly to your database.
• 3:20
  SPEAKER_01: So if your database has poor performance, doesn't matter how good your application is, it's just not getting the data fast enough.
• 3:30
  SPEAKER_01: And I've seen this a lot.
• 3:31
  SPEAKER_01: I've seen a lot of applications where they're working on, oh, we should make sure we don't concatenate strings together, because if we do that more than five million times in a loop, then we'll have a two-second lag on our application.
• 3:45
  SPEAKER_01: Well, sure, yeah, that's true.
• 3:47
  SPEAKER_01: Um, but normally you're doing that for like five times or three times or one time.
• 3:52
  SPEAKER_01: And so that performance difference is not even measurable compared to we wrote this select star from five tables with weird joins query that is going to bring back 100,000 rows, so we can display a number on the screen that indicates the overall total of our customer base.
• 4:15
  SPEAKER_01: Well, you could probably do that better, right?
• 4:17
  SPEAKER_01: And you're transmitting megabytes worth of information across from your database to your application server, and maybe even down to your client, and you're crunching all this data when you could have done it much more efficiently on the database server.
• 4:33
  SPEAKER_01: So thinking through application performance, which is a key part of application health, means knowing how to make your database performant, not just your application.
• 4:45
  SPEAKER_01: And since you're probably talking to your application, your database a lot in your application, well, you need to know how to make those calls efficient.
• 4:54
  SPEAKER_01: You need to know how to measure the performance of those calls.
• 4:59
  SPEAKER_01: Let me ask you a question.
• 5:01
  SPEAKER_01: When is the last time you went to your database and said, what are the unhealthy calls?
• 5:07
  SPEAKER_01: What are the calls that are really expensive?
• 5:09
  SPEAKER_01: What are the calls we're doing a lot that may not be quite as expensive, but because we're doing a million times a day, that costs a majority of our server resources?
• 5:18
  SPEAKER_01: How can we optimize those?
• 5:19
  SPEAKER_01: When's the last time you did that?
• 5:21
  SPEAKER_01: That's really important.
• 5:23
  SPEAKER_01: And yet it's in the database side, not the application side.
• 5:28
  SPEAKER_01: Now, number three, your company would probably fail if they lost all their data.
• 5:34
  SPEAKER_01: Think of that through.
• 5:35
  SPEAKER_01: Your application that displays data, that captures data, manipulates data, your application is using the data that's probably the lifeblood of your application, of your company.
• 5:48
  SPEAKER_01: So I worked at an insurance company.
• 5:51
  SPEAKER_01: Well, guess what?
• 5:52
  SPEAKER_01: That database was all of our customers, all their insurance plans and policies, all the payment information, all that information.
• 6:00
  SPEAKER_01: I worked at another company where it was a college, it was all the students, it was all their transcripts, all their courses they took, all the assignments they turned in, all this stuff.
• 6:11
  SPEAKER_01: I worked at a number of different organizations where their data, the database, stores essentially the lifeblood of the company.
• 6:21
  SPEAKER_01: And while the company may feel like, well, that's not what we do.
• 6:25
  SPEAKER_01: Maybe we make widgets.
• 6:27
  SPEAKER_01: And so data is not really important, right?
• 6:29
  SPEAKER_01: Well, it's your customers, right?
• 6:31
  SPEAKER_01: And your customers have orders, and those orders have to be fulfilled, and you need to know who is paid and who hasn't.
• 6:38
  SPEAKER_01: And that's kind of critical.
• 6:40
  SPEAKER_01: And if you don't have that one day, what happens?
• 6:44
  SPEAKER_01: If your database goes away, what happens?
• 6:48
  SPEAKER_01: Well, if you are writing application code and you're talking to the database, you are dealing with a very sensitive portion of your company's information.
• 7:02
  SPEAKER_01: You are dealing with something that if you do it wrong, you could destroy your applic your company's application.
• 7:08
  SPEAKER_01: You could destroy your company's entire livelihood.
• 7:12
  SPEAKER_01: A simple example of this SQL injection.
• 7:15
  SPEAKER_01: This is still a thing today, which is stunning and shocking.
• 7:19
  SPEAKER_01: But taking data from the front end and packaging it up to do a query, and you don't do the right steps necessary to make sure that that data that you're given is filtered and cleaned up so that you don't accidentally allow malicious code from your form through to your database.
• 7:42
  SPEAKER_01: This happens a lot where companies can have all their data lost, or even worse yet, corrupted.
• 7:48
  SPEAKER_01: And this is the important thing data integrity.
• 7:52
  SPEAKER_01: Because your form may capture data and it may not allow anything malicious, but maybe it doesn't do it quite right.
• 8:00
  SPEAKER_01: Well, imagine if you're capturing information and you're updating and overriding information in the database, but it's not quite right.
• 8:09
  SPEAKER_01: Well, that'll probably be something that gets past all your backups because you've been making changes, but it doesn't really, it's not noticeable.
• 8:16
  SPEAKER_01: It's a smaller thing, but then you find out we can't depend on any of this data.
• 8:23
  SPEAKER_01: At that point, it's too late, right?
• 8:25
  SPEAKER_01: So data security, data integrity, these are important things to know.
• 8:30
  SPEAKER_01: And it's not just an application thing.
• 8:33
  SPEAKER_01: You have to understand your database as well.
• 8:36
  SPEAKER_01: Otherwise, you could endanger your entire company.
• 8:41
  SPEAKER_01: Number four, the choices you make up front can have a long-lasting impact on your application.
• 8:50
  SPEAKER_01: I've been a consultant for a number of years, I'm not anymore, but I've spent a lot of time working with companies when things went wrong, right?
• 9:00
  SPEAKER_01: When they call me in, typically it's Tim, can you help us?
• 9:04
  SPEAKER_01: We've made some mistakes, we've got some problems.
• 9:07
  SPEAKER_01: And often performance is a big issue, or data integrity is a big issue, or they find out they fail an audit because they have some serious issues going on and they need help.
• 9:20
  SPEAKER_01: But one of the things I notice is that a lot of people don't necessarily live with the consequences of their own actions.
• 9:27
  SPEAKER_01: They they build something, they think this is a great architecture, and they build something up at a company, and then they leave.
• 9:33
  SPEAKER_01: And it works.
• 9:35
  SPEAKER_01: Okay, it works through the first five years of the company.
• 9:38
  SPEAKER_01: And so they may stay for five, 10 years, even, and then they leave the company, but their application stays there.
• 9:45
  SPEAKER_01: And it continues to work until some point they have some problems.
• 9:49
  SPEAKER_01: And then it's kind of too late.
• 9:52
  SPEAKER_01: I've worked with a company recently where they're working through the implications of a poorly implemented entity framework application, where it's.NET framework, they can't upgrade it yet, they're working on it.
• 10:06
  SPEAKER_01: But even so, the issue isn't.NET framework.
• 10:10
  SPEAKER_01: The issue isn't they use entity framework.
• 10:13
  SPEAKER_01: The issue is they poorly implemented it.
• 10:16
  SPEAKER_01: They didn't understand their database well enough.
• 10:19
  SPEAKER_01: So they said, well, but we know entity framework.
• 10:22
  SPEAKER_01: Kind of.
• 10:23
  SPEAKER_01: They actually didn't.
• 10:24
  SPEAKER_01: They thought they did because it worked.
• 10:26
  SPEAKER_01: And that's the scary bit, is it works.
• 10:28
  SPEAKER_01: So what happened was they started down a path.
• 10:32
  SPEAKER_01: And that path went this direction.
• 10:34
  SPEAKER_01: It went, it went off to the left.
• 10:37
  SPEAKER_01: But then they could have made a decision back here at the beginning to go slightly different.
• 10:42
  SPEAKER_01: And if they had done that, they would be in a much better position than they are today because those paths diverge the farther down the road you go.
• 10:50
  SPEAKER_01: And at some point, the distance from where they are to where they want to be is just too far.
• 10:58
  SPEAKER_01: They've integrated things too tightly into the application to change how they do things now.
• 11:03
  SPEAKER_01: And so what happens is the solution often, the cost-effective solution often is to throw more resources at it.
• 11:13
  SPEAKER_01: And when you're working in the cloud, that gets expensive fast because the cloud charges you the more resources you use.
• 11:22
  SPEAKER_01: Well, when your database is millions of people large, even though you're a small organization, and your database is massive, it's hard to back up, and it bogs your entire application down when you have 100, 200, 500 concurrent people using it, that's a problem.
• 11:44
  SPEAKER_01: But the solution can't always be let's put more RAM in it, let's get a bigger CPU.
• 11:50
  SPEAKER_01: Because at some point you run out of those.
• 11:52
  SPEAKER_01: And in fact, they had run out of those.
• 11:55
  SPEAKER_01: And so they actually took it to the cloud because the cloud has more resources than they can have in one physical server.
• 12:01
  SPEAKER_01: But that's not the solution you want to get to.
• 12:04
  SPEAKER_01: Okay, so you want to think about the long-term impact of your decisions up front.
• 12:09
  SPEAKER_01: The database you choose is important.
• 12:12
  SPEAKER_01: The way you architect your database is important.
• 12:16
  SPEAKER_01: The decisions you make about how you access your data and how you put data back into your database is important.
• 12:24
  SPEAKER_01: And if you only know half of the equation, the application side, then you're not going to make good enough decisions without luck.
• 12:32
  SPEAKER_01: And that's not a great place to be.
• 12:34
  SPEAKER_01: So just knowing your database a little bit or even having a passable knowledge is probably not good enough.
• 12:41
  SPEAKER_01: You need to spend time to know your database well so you can make informed decisions about what your application is going to do and how it's going to work.
• 12:52
  SPEAKER_01: Your application's health and even the level of success your company sees depends on how well you build and maintain that database.
• 13:01
  SPEAKER_01: Spend the time necessary to get it right.
• 13:04
  SPEAKER_01: All right.
• 13:05
  SPEAKER_01: Thanks for listening.
• 13:06
  SPEAKER_01: And as always, I am Tim Corey.
• 13:08
  SPEAKER_00: Thank you for joining us for this episode of Dev Questions.
• 13:11
  SPEAKER_00: When you're ready to learn to think and code like a professional developer, head over to IamtimCorey.com and enroll in a course.