Latest / DevQuestions with Tim Corey / 295. Developing For Analog - Why Software Development is Messy

Transcript

295. Developing For Analog - Why Software Development is Messy · DevQuestions with Tim Corey

• 0:00
  SPEAKER_01: Software development is messy.
• 0:02
  SPEAKER_01: If you've ever worked on a production app, you know what I'm talking about.
• 0:06
  SPEAKER_01: There are shortcuts, there are poorly written code, there's workarounds, repeated code, and a general lack of best practices.
• 0:13
  SPEAKER_01: These are all common in long-running software applications.
• 0:17
  SPEAKER_01: But why is this?
• 0:18
  SPEAKER_01: In this episode of Dev Questions, we're going to talk about developing digital software in analog world.
• 0:33
  SPEAKER_00: Welcome to the DevQuesti podcast with Tim Corey.
• 0:37
  SPEAKER_01: Software development is more than just writing code.
• 0:40
  SPEAKER_01: So let's talk about the rest of it.
• 0:41
  SPEAKER_01: Specifically, let's talk about why code gets so messy.
• 0:46
  SPEAKER_01: Now, before we jump in, I want to give you a quick primer on my illustration.
• 0:50
  SPEAKER_01: So digital refers to specific values.
• 0:53
  SPEAKER_01: For instance, computers deal in ones and zeros.
• 0:56
  SPEAKER_01: That's digital.
• 0:57
  SPEAKER_01: It's very much precision, one or zero, on or off.
• 1:01
  SPEAKER_01: Analog refers to a range of values.
• 1:04
  SPEAKER_01: There are an infinite number of values between that zero and one.
• 1:08
  SPEAKER_01: So think of this like looking at your monitor versus looking out the window.
• 1:14
  SPEAKER_01: I have a desktop background that has my company's logo on it, and it's got a gradient background.
• 1:20
  SPEAKER_01: So it goes from light gray to a dark gray almost black.
• 1:24
  SPEAKER_01: But on certain monitors, it actually is striped instead of actually having a smooth transition.
• 1:31
  SPEAKER_01: Why is that?
• 1:32
  SPEAKER_01: Well, part of that is how you encode the image and so on.
• 1:37
  SPEAKER_01: But the foundation of it is that a monitor is a series of pixels.
• 1:43
  SPEAKER_01: And each pixel can be a color.
• 1:45
  SPEAKER_01: Now there's a lot of things they can do with subpixels and try and get smaller and smaller, but the reality is that there is a color in a spot.
• 1:56
  SPEAKER_01: And so that blending is okay, there's a gray here, there's a little bit lighter gray here.
• 2:02
  SPEAKER_01: And so our eyes tend to see the blend of that, but it's not real blend.
• 2:07
  SPEAKER_01: It's actually stepped values.
• 2:09
  SPEAKER_01: When you look out the window, you don't see that.
• 2:11
  SPEAKER_01: When you look at the sky, you don't see, we call it a blue sky, but it's never just blue.
• 2:18
  SPEAKER_01: If you were to take a picture of that and try and get the value for the color value for any one pixel in that picture, they'd all be different.
• 2:27
  SPEAKER_01: Why?
• 2:27
  SPEAKER_01: Because even in what appears to be a uniform sky, there's variations, there's a little bit of subtle differences in that coloration.
• 2:35
  SPEAKER_01: And that's analog, it's the idea that there are infinite number of values between any two steps.
• 2:43
  SPEAKER_01: It's not a step like a one versus a zero.
• 2:47
  SPEAKER_01: So what does this have to do with software development?
• 2:50
  SPEAKER_01: Well, as a developer, we're creating digital solutions to an analog world.
• 2:55
  SPEAKER_01: We get used to perfection.
• 2:57
  SPEAKER_01: The same exact inputs give you the same exact outputs for a method.
• 3:02
  SPEAKER_01: We actually create tests to make sure that's the case.
• 3:05
  SPEAKER_01: When I give you these two values, I expect this value back.
• 3:09
  SPEAKER_01: That's a unit test.
• 3:10
  SPEAKER_01: The problem is that it only works until it comes in contact with the real world.
• 3:16
  SPEAKER_01: So here are some examples, just kind of get your brain thinking about this.
• 3:21
  SPEAKER_01: The big one, user input.
• 3:23
  SPEAKER_01: You probably heard me say before, never trust users.
• 3:27
  SPEAKER_01: Never trust users.
• 3:28
  SPEAKER_01: Why?
• 3:29
  SPEAKER_01: Because users will do things that you just don't expect.
• 3:33
  SPEAKER_01: The reason we have so many bugs in software is because of users.
• 3:37
  SPEAKER_01: When you ask what is your age, and you expect them to type in something like a 2.1 for 21, instead they type out T-W-E-N-T-Y-O-N-E.
• 3:50
  SPEAKER_01: Well, that's 21 as well, but it's not what you expected.
• 3:53
  SPEAKER_01: You expected a number, you expected an integer value.
• 3:57
  SPEAKER_01: Maybe once in a while they put 21.5 because they're halfway through 21, but that's not how we do age.
• 4:03
  SPEAKER_01: Like you don't always get what you expect.
• 4:07
  SPEAKER_01: If you've ever maintained a database and that database consists of data received from the user in any way, you will know that even with your best efforts to make sure you validate the data first, make sure you limit the data, there will be junk data throughout the database.
• 4:27
  SPEAKER_01: And part of our job is to figure out how to survive that junk data.
• 4:34
  SPEAKER_01: Now, let's look at it from a more practical, hardware-based look at the real world.
• 4:41
  SPEAKER_01: Um, full self-drive.
• 4:43
  SPEAKER_01: This is one that's been in the news for a long time.
• 4:45
  SPEAKER_01: This is something that Elon Musk has been trying really hard to make work.
• 4:49
  SPEAKER_01: And quite frankly, I am looking forward to the day when that happens.
• 4:53
  SPEAKER_01: Not because I want to give up driving, but because there's so many times when it'd be great, for example, my parents or uh my in-laws to be able to say, hey, get in the car and just let the car drive you.
• 5:05
  SPEAKER_01: And I can feel safe knowing the car is going to take care of them.
• 5:07
  SPEAKER_01: We're not there yet.
• 5:10
  SPEAKER_01: But in 2013, Elon Musk said that 90% of the miles driven in 2016 will be driven autonomously.
• 5:21
  SPEAKER_01: Okay, 2016.
• 5:23
  SPEAKER_01: That was 10 years ago, over a decade ago.
• 5:26
  SPEAKER_01: Did it happen?
• 5:27
  SPEAKER_01: No.
• 5:28
  SPEAKER_01: But why?
• 5:29
  SPEAKER_01: Why didn't it happen?
• 5:30
  SPEAKER_01: Was it just because he's bad at estimating?
• 5:32
  SPEAKER_01: And yeah, he estimates poorly a lot of time, but that's not really the issue, I think.
• 5:36
  SPEAKER_01: I think the issue is that in the digital realm, yes, because driving is a very simple set of rules.
• 5:45
  SPEAKER_01: We give 15-year-olds and 16-year-olds in the US a book, a small book, and say, hey, these are the rules for driving.
• 5:53
  SPEAKER_01: If you know these rules and you come back and we test you and make sure you know most of those rules, then we'll give you the ability to drive on the road with a permit.
• 6:03
  SPEAKER_01: Well, if we can do it for a 15-year-old, why couldn't we do that with a computer?
• 6:07
  SPEAKER_01: Why couldn't the computer say, okay, I can turn right on red at a light if no one's coming?
• 6:15
  SPEAKER_01: Like that's one of the rules in the US and a lot of places.
• 6:17
  SPEAKER_01: Or I have to stop at stoplights and stop signs.
• 6:20
  SPEAKER_01: Or when it says yellow, I need to slow down, be cautious.
• 6:23
  SPEAKER_01: Like all these different things that are rules are very clear, cut, and dry.
• 6:28
  SPEAKER_01: Except for the fact that when you get into the real world, that's not what actually happens.
• 6:34
  SPEAKER_01: Those digital rules, those very specific rules, are great until you actually get into construction zones and an accident in front of you.
• 6:44
  SPEAKER_01: And what happens if there's any kind of weather, whether it's uh wind or rain or blinding sunlight or so many other things?
• 6:52
  SPEAKER_01: What happens when you know you have an issue with a driver in front of you doing something that is unexpected?
• 6:59
  SPEAKER_01: There's so many things that happen that make the idea of that software so much harder, so much more difficult.
• 7:07
  SPEAKER_01: Because it's not just that simple set of rules.
• 7:10
  SPEAKER_01: The real world is analog.
• 7:13
  SPEAKER_01: The possibilities between zero and one are infinite.
• 7:18
  SPEAKER_01: And so we have to figure out how to calculate for that.
• 7:21
  SPEAKER_01: And it's a very difficult problem to solve.
• 7:24
  SPEAKER_01: Another one that just came across my Twitter feed recently, it kind of came back up.
• 7:29
  SPEAKER_01: It's always a problem for a developer, and that is time.
• 7:32
  SPEAKER_01: So the tweet that I saw was that a customer had purchased something on January 31st at 6 p.m.
• 7:41
  SPEAKER_01: When they got the invoice, it said they had purchased it on February 1st.
• 7:47
  SPEAKER_01: Well, why?
• 7:48
  SPEAKER_01: They were upset because they purchased it in January and the store said they purchased it in February.
• 7:55
  SPEAKER_01: Well, yes, because difference in time zones.
• 7:57
  SPEAKER_01: Is it the customer's time zone or the business's time zone or the server that it's running on, their time zone?
• 8:05
  SPEAKER_01: And a lot of times we we you know, quote unquote solve this with UTC, universal time coordinates.
• 8:12
  SPEAKER_01: Uh so we say, hey, this is always a time and we can just add or subtract based upon your time zone.
• 8:19
  SPEAKER_01: But that's not really the full solution because there's so much more that goes into it.
• 8:24
  SPEAKER_01: Let's back this up a month.
• 8:26
  SPEAKER_01: Did you purchase it in the 2025 or did you purchase it in 2026?
• 8:32
  SPEAKER_01: The same exact time is 2026 for the business and 2025 for the customer.
• 8:37
  SPEAKER_01: So does the business claim the income for 2026, but the customer claimed the expense in 2025?
• 8:44
  SPEAKER_01: Like, time is hard.
• 8:47
  SPEAKER_01: Even just a clock makes so many differences in how we do things.
• 8:52
  SPEAKER_01: The real world is messy.
• 8:54
  SPEAKER_01: Even as far as in one city, there's one time zone.
• 8:58
  SPEAKER_01: You go just a little bit outside of it, it's a different time zone.
• 9:01
  SPEAKER_01: Or they do allow for daylight savings time, or they don't.
• 9:05
  SPEAKER_01: And the time can change.
• 9:07
  SPEAKER_01: It's complicated.
• 9:09
  SPEAKER_01: Um, here's another one that we do in our health is there is this concept of body mass index or BMI.
• 9:18
  SPEAKER_01: And that's what a lot of health insurance companies and even doctors will use as an easy metric to identify: is a person at a good weight, are they overweight, are they underweight.
• 9:29
  SPEAKER_01: The problem is it doesn't take into account properly things like height or age or bone density or ethnicity or anything else.
• 9:39
  SPEAKER_01: And so we try and give it a very clear digital, this is, you know, ones and zeros, yes or no.
• 9:46
  SPEAKER_01: And the reality is at such a sliding scale about really comes down to every person's a bit different.
• 9:52
  SPEAKER_01: Yes, we can generalize, but that's not what we're dealing with.
• 9:55
  SPEAKER_01: We're dealing with specifics.
• 9:57
  SPEAKER_01: So how does this apply to normal developers?
• 10:00
  SPEAKER_01: Well, number one, every application you develop will interact with analog systems.
• 10:07
  SPEAKER_01: Every application.
• 10:08
  SPEAKER_01: At the very least, you're dealing with how the CPU runs, how the RAM interacts, if there's any internet glitches or service outages or power fluctuations, or then it comes to the input that you get, your application gets, how the user runs your application, what other applications are running, what Windows updates are running.
• 10:30
  SPEAKER_01: That caused crashes recently a lot.
• 10:33
  SPEAKER_01: So every application you develop is going to interact with analog systems.
• 10:39
  SPEAKER_01: That's number one.
• 10:40
  SPEAKER_01: Number two is you can't account for every possibility.
• 10:44
  SPEAKER_01: It's just not possible.
• 10:46
  SPEAKER_01: There are so many things that can happen in the real world because again, the real world is analog.
• 10:51
  SPEAKER_01: The real world has an infinite number of possibilities.
• 10:55
  SPEAKER_01: And we can't account for all of that.
• 10:57
  SPEAKER_01: So what we need to do instead is figure out how to account for the most common things and then patch the rest.
• 11:06
  SPEAKER_01: Figure out, okay, well, that one also is going to affect me.
• 11:08
  SPEAKER_01: I didn't realize that.
• 11:09
  SPEAKER_01: Let's go ahead and address that.
• 11:12
  SPEAKER_01: So which edge cases you cover will change based upon your specific situation.
• 11:18
  SPEAKER_01: So the time issue, does that matter to you?
• 11:21
  SPEAKER_01: Maybe it doesn't matter to this application, but maybe it matters a great deal to another.
• 11:26
  SPEAKER_01: So a simple one is when you're asking users for input, does it matter if we have their address or not?
• 11:33
  SPEAKER_01: For example, my business, I don't care.
• 11:36
  SPEAKER_01: Uh in fact, I actually moved off of, I use Teachable for my hosting of my courses, but I moved off of Teachable to a company called Fast Spring for the payments.
• 11:46
  SPEAKER_01: And part of the reason why is because Teachable said, you know what, we're gonna ask for the customer's full address, we're gonna ask for their I think phone number and email address and all this.
• 11:57
  SPEAKER_01: Why?
• 11:58
  SPEAKER_01: It doesn't matter to me.
• 12:00
  SPEAKER_01: It doesn't matter.
• 12:00
  SPEAKER_01: I'm I'm not shipping you physical goods.
• 12:03
  SPEAKER_01: So I don't really care.
• 12:04
  SPEAKER_01: Now, there is some extent we need to know, like, hey, what's your location in order to know how to tax you correctly and that kind of thing.
• 12:13
  SPEAKER_01: But I don't want to capture that information where other businesses absolutely have to.
• 12:18
  SPEAKER_01: So which edge cases you want to allow about?
• 12:21
  SPEAKER_01: Do you just require state and zip, or do you require city state zip, or do you require a full address and make sure it's verified, et cetera?
• 12:31
  SPEAKER_01: The edge cases you deal with are going to change based upon your specific situation.
• 12:35
  SPEAKER_01: And that may change over time, too.
• 12:38
  SPEAKER_01: Now, you're going to need to fix the flaws as they appear.
• 12:42
  SPEAKER_01: This is something that is the reason why all of our applications seem to be a bit messy.
• 12:49
  SPEAKER_01: Is that we didn't know right up front that this little thing was going to cause a problem, this edge case or this occurrence of something leap year was going to cause a problem for our application, whatever the case may be.
• 13:04
  SPEAKER_01: And so you have to fix the flaws as they appear.
• 13:08
  SPEAKER_01: And that means sometimes patching a system and working around it.
• 13:12
  SPEAKER_01: And then going along with that, number five, time is always a factor in development.
• 13:17
  SPEAKER_01: It's always a factor.
• 13:18
  SPEAKER_01: And that's something we don't think about necessarily when we think about the previous developer.
• 13:23
  SPEAKER_01: We look at the code and say, you know what, this previous developer was an idiot.
• 13:26
  SPEAKER_01: Um you find out later it's you, but you know, um, they were an idiot because they didn't do this.
• 13:30
  SPEAKER_01: They should have done this.
• 13:31
  SPEAKER_01: Well, yeah, but you don't realize that when you were in that situation, they were working under a tight deadline where they had to get other things done as well, and they had to get this out the door within minutes or hours because of the fact that customers were down and were depending on this and we're waiting for that patch to be done right away.
• 13:50
  SPEAKER_01: There's a lot of pressure that comes into development, and you can't always spend the time to just take your time and do it the right way and make sure you dig into all the code and make sure it's right.
• 14:03
  SPEAKER_01: So time is a factor.
• 14:04
  SPEAKER_01: So we're gonna fix those flaws, number four, but time is gonna be a factor in that development as well.
• 14:10
  SPEAKER_01: And then number six, rebuilding the application means learning these lessons all over again.
• 14:15
  SPEAKER_01: If you take nothing else away from this, I want you to hear that.
• 14:19
  SPEAKER_01: So let me repeat that.
• 14:21
  SPEAKER_01: Building the application or rebuilding the application means learning these lessons all over again.
• 14:28
  SPEAKER_01: Too often I see people look at code, they come into a company and they say, Oh, this code's a mess.
• 14:34
  SPEAKER_01: We just need to rewrite it.
• 14:36
  SPEAKER_01: I've seen this done.
• 14:38
  SPEAKER_01: I have been there, I have been the one rewriting.
• 14:42
  SPEAKER_01: I have seen others do it.
• 14:43
  SPEAKER_01: I have consulted for a number of companies that were in various stages of this process.
• 14:50
  SPEAKER_01: What almost inevitably happens, almost every time, is that you start off with this concept of we know what the application does now, let's make it perfect.
• 15:02
  SPEAKER_01: And you start with this perfect idea, this perfect design, and you you have this perfect idea of what it's gonna be, and then a weird issue pops up, and you're like, oh, I didn't know that was gonna happen.
• 15:14
  SPEAKER_01: I have to fix that.
• 15:15
  SPEAKER_01: And you fix that problem, you fix another problem, then you start to realize these problems were already fixed in the original code.
• 15:22
  SPEAKER_01: Well, how did they fix it?
• 15:23
  SPEAKER_01: And you realize they did the workaround too.
• 15:25
  SPEAKER_01: Why?
• 15:26
  SPEAKER_01: Because they were in the same shoes you were just three years ago or six years ago.
• 15:31
  SPEAKER_01: And what you're doing now is relearning the lessons from the past.
• 15:36
  SPEAKER_01: So be very careful when you think about just rewriting.
• 15:40
  SPEAKER_01: Because what you're gonna do is lose a lot of institutional knowledge and a lot of institutional protections.
• 15:47
  SPEAKER_01: Things that were built up over time to protect the application, you might not realize are in place.
• 15:54
  SPEAKER_01: That weird code that seems to be poorly written might actually be the fix that was necessary to solve a weird edge case.
• 16:03
  SPEAKER_01: When you take that out, all of a sudden the edge case comes back until you figure out how to redo that.
• 16:08
  SPEAKER_01: I have seen a lot of applications that start out with great intentions and end up right back where the original application was, just having spent months or even years and a lot of money on the project.
• 16:22
  SPEAKER_01: That's not a great end result to be back where you started.
• 16:26
  SPEAKER_01: Maybe instead of that, learn from a system, make it better over time, rather than trying to rewrite the thing and forgetting all those fixes that really are keeping your application up and running.
• 16:38
  SPEAKER_01: So I'm not saying never rewrite an application, but there's a reason why applications are a bit messy.
• 16:46
  SPEAKER_01: So at the end of the day, yes, we're trying to build digital applications, but we have to realize the real world is messy.
• 16:54
  SPEAKER_01: The real world is not perfect.
• 16:56
  SPEAKER_01: I used to work with wood as my hobby.
• 16:59
  SPEAKER_01: I'm allergic to sawdust, so I can't do that anymore.
• 17:02
  SPEAKER_01: But one of the things that always used to frustrate me when I first started off is the fact that wood is never perfect.
• 17:08
  SPEAKER_01: Never perfect.
• 17:09
  SPEAKER_01: You watch these YouTube videos where it's perfect and things just slide together and they're perfect.
• 17:13
  SPEAKER_01: That's not really true.
• 17:14
  SPEAKER_01: That only happens in the video.
• 17:16
  SPEAKER_01: Because, yes, if you spend a lot of time, there are some master craftsmen that can do some amazing things.
• 17:22
  SPEAKER_01: But wood swells when it's when it's uh hot and and moist in the air, or it shrinks, or it it changes over time and it's not perfect because the real world isn't perfect.
• 17:34
  SPEAKER_01: The real world isn't this precise Lego factory where we just snap things together, it just works.
• 17:41
  SPEAKER_01: So we have to realize when we're building applications to be flexible, to be able to handle changes, to be able to handle the idea that things aren't gonna be perfect.
• 17:51
  SPEAKER_01: And how do we keep things loose and flexible rather than tight and rigid and in theory perfect?
• 17:58
  SPEAKER_01: Because that perfection is gonna get broken really quickly when it hits the real world.
• 18:02
  SPEAKER_01: All right.
• 18:02
  SPEAKER_01: So thanks for listening.
• 18:04
  SPEAKER_01: As always, I am Tim Corey.
• 18:06
  SPEAKER_00: Thank you for joining us for this episode of Dev Questions.
• 18:09
  SPEAKER_00: When you're ready to learn to think and code like a professional developer, head over to imtimcore.com and enroll in a course.