Latest / DevQuestions with Tim Corey / 132 A Shared Development vs Local Services and Containers

Transcript

132 A Shared Development vs Local Services and Containers · DevQuestions with Tim Corey

• 0:00
  SPEAKER_00: Welcome to the Dev Questions Podcast with Tim Corey.
• 0:03
  SPEAKER_00: Join us each episode as we tackle the questions you are asking about a career in software development, understanding the industry, and new technology.
• 0:12
  SPEAKER_00: If you're just starting out or you want to grow stronger as a developer, this is the place to get your questions answered.
• 0:19
  SPEAKER_00: Now, here's your host, expert developer and online educator, Tim Corey.
• 0:25
  SPEAKER_02: Should I use a shared development environment or should I try to replicate everything from production into every developer's local machine?
• 0:35
  SPEAKER_02: This is a question that's asked on the suggestion site.
• 0:38
  SPEAKER_02: And if you have a suggestion, go to suggestions.imtimcorey.com, and I'll try to answer in a future episode of DevQuestions.
• 0:46
  SPEAKER_02: So should we use a shared environment for development, or should we try to replicate everything locally, or should you have a hybrid solution of some kind?
• 0:57
  SPEAKER_02: That's the question we're going to answer today.
• 0:59
  SPEAKER_02: So my opinion, and this is an opinion, and we'll talk about exceptions because there are some.
• 1:06
  SPEAKER_02: But my opinion is that you should have a local environment, meaning local to every developer, that replicates as much of production as possible, if not everything, in production.
• 1:19
  SPEAKER_02: That does take some work.
• 1:21
  SPEAKER_02: But let's talk about some different scenarios and why I think it's important that you have everything local, not just some stuff.
• 1:30
  SPEAKER_02: Okay?
• 1:30
  SPEAKER_02: Let's start with a simple example.
• 1:33
  SPEAKER_02: So let's think about if we have one C solution, maybe a couple of projects in there, but you know, one solution and then one database.
• 1:42
  SPEAKER_02: That's your uh your entire environment.
• 1:45
  SPEAKER_02: So your production application might be one web page or one website or one web application or one desktop application, something like that, whatever it is, with one database.
• 1:56
  SPEAKER_02: Let's say a SQL database, doesn't really matter.
• 1:58
  SPEAKER_02: So that's our simple solution.
• 2:00
  SPEAKER_02: Well, why should we have everything fully on the developer's machine, all the way locally?
• 2:06
  SPEAKER_02: And the answer is what's called the developer loop or development loop.
• 2:12
  SPEAKER_02: And what that is, is the feedback loop.
• 2:14
  SPEAKER_02: It's when you make a change and you quickly see what it does and quickly see how it affects everything else, and can quickly change that and iterate on that process.
• 2:26
  SPEAKER_02: So why is this important to do locally as opposed to with a shared development database, for example?
• 2:33
  SPEAKER_02: Well, let's just say that you are going to test specific data.
• 2:38
  SPEAKER_02: Maybe you were given a bug to fix where every 1000th time that you enter something in the system, everything crashes.
• 2:48
  SPEAKER_02: Well, that's a pretty specific bug you have to track down and fix.
• 2:52
  SPEAKER_02: And maybe you find a solution for it, but how do you test that?
• 2:55
  SPEAKER_02: Well, you put a thousand in, just to be sure.
• 2:58
  SPEAKER_02: And when you do, it either crashes the system, which means you didn't fix it, or it doesn't.
• 3:04
  SPEAKER_02: But imagine if you were on a shared development environment.
• 3:09
  SPEAKER_02: If you add a thousand new people into the database, well, that might mess somebody else's testing up.
• 3:14
  SPEAKER_02: Maybe they were testing, you know, how many records in the database and all of a sudden it keeps changing and so their tests are off.
• 3:21
  SPEAKER_02: Or maybe they're trying to do something where they delete certain users, but but you're adding users and that causes problems.
• 3:30
  SPEAKER_02: So you're gonna have a conflict of who's testing and who's trying things out is more important.
• 3:38
  SPEAKER_02: And so one developer has to slow down in order to let the other developer do their work.
• 3:44
  SPEAKER_02: It's kind of a work in series instead of a work in parallel.
• 3:48
  SPEAKER_02: Now it's not all the time.
• 3:49
  SPEAKER_02: A lot of times you can work in parallel, but there are times when you both need the same resource or you both want to affect a certain table in a certain way, or something like that, where you would cause a conflict.
• 4:01
  SPEAKER_02: The other thing is a breaking change.
• 4:04
  SPEAKER_02: Imagine if you are a newer developer.
• 4:07
  SPEAKER_02: Let's say a junior level developer, you just started and you're working on this, and you say, hey, you know, I'm gonna write this new SQL query to do something on just a dev server because we're trying this out.
• 4:19
  SPEAKER_02: And you're trying something out, and you you know, you want to delete one record.
• 4:24
  SPEAKER_02: And like almost every SQL database developer ever, you forget once to put the where clause on your delete.
• 4:33
  SPEAKER_02: And you delete every user out of your development database.
• 4:38
  SPEAKER_02: It's not a problem, it's only development, right?
• 4:41
  SPEAKER_02: Yes and no, because yeah, it didn't hurt anybody in production, it didn't delete any real data.
• 4:47
  SPEAKER_02: You're safe, and that's why you should have a development environment and not work on production.
• 4:52
  SPEAKER_02: But you still cause a problem because it's not just you that's down.
• 4:58
  SPEAKER_02: Imagine you have a team of 10 developers.
• 5:01
  SPEAKER_02: Well, all those developers are gonna be using that same shared development environment, that same shared development database.
• 5:10
  SPEAKER_02: And all of a sudden, all 10 are down because they can't test against the user table because there are no more users, because you deleted them all.
• 5:19
  SPEAKER_02: Well, that's a problem because now it's not just that you took your database down, you took the entire team's database down.
• 5:28
  SPEAKER_02: And so you're not just costing the company your time, you're costing the company the time of the entire team, 10 people.
• 5:38
  SPEAKER_02: So that gets much more expensive and it slows everything down.
• 5:43
  SPEAKER_02: You see a theme here?
• 5:45
  SPEAKER_02: Whenever you have a shared environment, things cause it to slow down.
• 5:51
  SPEAKER_02: So if the other thing is if you're doing testing and maybe you're doing a performance test to see how things have changed, well, you have to tell everybody else, pause what you're doing on a development server, I want to see how this performance compares to the last time I ran this.
• 6:10
  SPEAKER_02: Well, you can't have somebody else doing work too, because that would slow down the SQL Server and that would throw off your performance numbers.
• 6:17
  SPEAKER_02: So you kind of need to tell everybody to get off.
• 6:20
  SPEAKER_02: So just like deleting the database, you're gonna be waiting, having everybody wait on you to fix things or wait on you to get your test done.
• 6:28
  SPEAKER_02: And so it again creates this backlog of people waiting.
• 6:34
  SPEAKER_02: So that's a simple environment.
• 6:36
  SPEAKER_02: And that's probably the simplest application you can have, especially with a team of more than one person.
• 6:42
  SPEAKER_02: Um, that's probably the simplest thing you're gonna see.
• 6:44
  SPEAKER_02: Let's talk about if you get into more complex examples.
• 6:47
  SPEAKER_02: Because this is where the the um the really the advice of well, keep everything in one development environment seems more practical.
• 6:55
  SPEAKER_02: And that's if you have a microservice environment or you have some other large, complex set of servers and interactivity that's happening that you think, well, I can't replicate that locally.
• 7:08
  SPEAKER_02: I shouldn't have to, right?
• 7:09
  SPEAKER_02: I'm just gonna do it on a shared development environment.
• 7:12
  SPEAKER_02: And maybe we'll pull down just the thing we're working on and test that locally, but against the shared environment.
• 7:18
  SPEAKER_02: And that's the more common uh example of when people think, well, I should have just that shared environment.
• 7:25
  SPEAKER_02: And so let's talk about that.
• 7:27
  SPEAKER_02: So a complex example.
• 7:29
  SPEAKER_02: Let's let's keep it simple in our complex example.
• 7:33
  SPEAKER_02: So we have two applications.
• 7:36
  SPEAKER_02: We have a Redis cache, we have a MongoDB database, we have a SQL database, and we have some sort of message queuing system, whether it's RabbitMQ or Azure Service Bus or whatever it's gonna be, but we have this is our environment.
• 7:50
  SPEAKER_02: Okay, so it's a little more complex.
• 7:52
  SPEAKER_02: This is a really small microservice environment, but that's what a microservice environment might look like.
• 7:58
  SPEAKER_02: Okay.
• 7:59
  SPEAKER_02: So what issues do we have in this type of scenario when it comes to a shared uh development environment, even if we pull some things down locally?
• 8:10
  SPEAKER_02: Well, first of all, the same issues all apply.
• 8:13
  SPEAKER_02: If you're testing specific data, or if you break things, or if your changes take time to test or complete, you're still gonna have that backlog.
• 8:24
  SPEAKER_02: Well, what if let's see some additional things that might be a problem?
• 8:29
  SPEAKER_02: What if we decide we're going to um test adding something to the queue?
• 8:34
  SPEAKER_02: Okay, so your application is, let's say, the web application, one of them, and it talks to MongoDB and and what it does is it processes queue messages.
• 8:44
  SPEAKER_02: Okay, so you decide, well, I'm gonna add something to the queue.
• 8:49
  SPEAKER_02: Well, remember that the whole purpose of a queue is it doesn't know who it's talking to.
• 8:54
  SPEAKER_02: It just says, hey, I've got messages here.
• 8:56
  SPEAKER_02: What if those messages affect other services?
• 8:59
  SPEAKER_02: Well, now you're affecting other services in your development environment that aren't local that could cause a problem with somebody else who is testing.
• 9:07
  SPEAKER_02: Maybe they're testing what happens when the queue is empty, but you're filling the queue up.
• 9:12
  SPEAKER_02: Or they're testing what happens when a certain queue message comes through and now it's buried in the noise.
• 9:19
  SPEAKER_02: Okay, so messing that queue could be a big problem.
• 9:24
  SPEAKER_02: What if you decide to change the structure of the database?
• 9:29
  SPEAKER_02: Well, if anything else is depending on that database structure, you've got a problem because now applications are breaking because they depended on that database structure that you've now changed and you haven't changed those two.
• 9:44
  SPEAKER_02: And so you want to have everything locally.
• 9:47
  SPEAKER_02: Now, quick aside right here, if you're talking about true microservice environments, they shouldn't be sharing a database.
• 9:54
  SPEAKER_02: Therefore, a database structure change shouldn't affect anybody.
• 9:58
  SPEAKER_02: However, that's not always the case.
• 10:00
  SPEAKER_02: There are times when the people forget or people decide to cut a corner and then rely on that database.
• 10:08
  SPEAKER_02: It does cause a bad problem.
• 10:10
  SPEAKER_02: And it's one of the reasons why you should really think hard about if you need to do microservices and if you do, making sure that you enforce some really strict rules around it.
• 10:21
  SPEAKER_02: That's a kind of an aside.
• 10:23
  SPEAKER_02: But let's just say that more than one application depends on a particular database and you change the structure.
• 10:29
  SPEAKER_02: That could be a problem.
• 10:31
  SPEAKER_02: All right.
• 10:32
  SPEAKER_02: Simulating failure.
• 10:33
  SPEAKER_02: What if you decide, hey, I want to simulate what happens to my application if the queue goes away?
• 10:39
  SPEAKER_02: Maybe you're gonna change the IP address or you're gonna turn the queue service off entirely.
• 10:45
  SPEAKER_02: Well, all of a sudden, now nobody has access to that queue.
• 10:50
  SPEAKER_02: And yes, other services dependent on having that queue available, and it's not anymore.
• 10:55
  SPEAKER_02: And maybe you that's a good test to make sure you know what happens to those other services, but what if other people are depending on that queue to be available in order to do their testing?
• 11:05
  SPEAKER_02: That's a problem.
• 11:06
  SPEAKER_02: All right.
• 11:07
  SPEAKER_02: So, and there's lots more to go into.
• 11:09
  SPEAKER_02: What if you have the cash go down or you take the cash down intentionally, and the list goes on of things where maybe you're working on a specific web application, but you need to affect other services, or you need to test what happens when things happen to those other services.
• 11:26
  SPEAKER_02: Other people depend on those same services.
• 11:29
  SPEAKER_02: So my recommendation is to bring everything down to the local machine.
• 11:36
  SPEAKER_02: The benefits there are pretty enormous because not only can you have that faster cycle of test, check, and verify, but you don't affect the rest of the team if you have a big problem.
• 11:51
  SPEAKER_02: There is no chance of a central failure of the development server.
• 11:56
  SPEAKER_02: What if the development server gets a bug or a virus or just crashes entirely?
• 12:01
  SPEAKER_02: Well, no problem.
• 12:02
  SPEAKER_02: You still have your local development environment, you can all still work.
• 12:06
  SPEAKER_02: What happens if your laptop crashes?
• 12:08
  SPEAKER_02: It's a bummer and it's gonna cause you problems, but it won't cause the rest of the team problems.
• 12:13
  SPEAKER_02: So it's a big difference as far as the resiliency of your development team.
• 12:18
  SPEAKER_02: And development speed is important, right?
• 12:22
  SPEAKER_02: So I recommend pulling everything down.
• 12:25
  SPEAKER_02: Um yes, I do recommend having a shared development environment, but it doesn't serve the same purpose as your local development environment.
• 12:36
  SPEAKER_02: Your shared development environment should really be about integrating all the changes, making sure they all integrate properly when you're ready to commit them, making sure that you're ready to go on to the next stage of testing and QA and production.
• 12:50
  SPEAKER_02: So it should really be the first step in your deployment, in my opinion, rather than just be your place where everyone does work.
• 13:00
  SPEAKER_02: All right.
• 13:02
  SPEAKER_02: So instead of having a failure of one person affect everybody, it's going to be a failure of one person that affects one person.
• 13:10
  SPEAKER_02: So yes, there's a really big reason why you should work locally.
• 13:16
  SPEAKER_02: Now, I said there's going to be exceptions.
• 13:18
  SPEAKER_02: And yes, there will be exceptions.
• 13:20
  SPEAKER_02: Not many, but there will be exceptions.
• 13:23
  SPEAKER_02: I think that exceptions are really going to be pretty rare because you really should be able to replicate as much as possible locally.
• 13:32
  SPEAKER_02: And if you can't replicate everything, you need to think through is that really true, or is it just that we haven't found the right solution for that?
• 13:42
  SPEAKER_02: Now, there are tools to help you with this.
• 13:45
  SPEAKER_02: Uh the biggest one probably is automation, whether you create scripts, whether you use like GitHub Actions or Azure DevOps and the pipelines there, whatever you use to automate even your local process.
• 13:59
  SPEAKER_02: For instance, I've got a video on YouTube where I talk about using Docker to automate the uh the taking your backup from your SQL database and restoring it to a clean Docker instance that's been scrubbed and doesn't have any sense of a personal information in it, but that you can have running automated in the background that developers can just pull that image down and say, there's my database.
• 14:25
  SPEAKER_02: So stuff like that you can do a lot.
• 14:28
  SPEAKER_02: Docker, like I mentioned, is a big tool because you can have things like Redis and a cache or a message queuing system and your log server and a whole bunch of other things all running in containers so that when you don't need them, you turn them off.
• 14:45
  SPEAKER_02: When you need them, you turn them on, and it's very efficient on your system.
• 14:50
  SPEAKER_02: This does mean that developers' machines should probably be pretty of pretty performant.
• 14:55
  SPEAKER_02: You probably shouldn't be looking at the cheapest possible laptop to be your development machine.
• 15:03
  SPEAKER_02: Now, I get it.
• 15:04
  SPEAKER_02: If you're just getting started, if you're self-funding, there's a reason why you might do that.
• 15:11
  SPEAKER_02: And that's okay as you get started.
• 15:14
  SPEAKER_02: But as you are working for a company, if you're working for in a larger environment with a team, you should have the resources to invest in the things that will make you back money.
• 15:25
  SPEAKER_02: And this is one of those areas where it's going to make you back money because additional performance in your laptop, additional performance in your desktop, I prefer a desktop for working with development, the more performance you have, the more efficient you will be.
• 15:39
  SPEAKER_02: Therefore, the more you'll get done and the cheaper per hour you will be, because you're getting more done.
• 15:47
  SPEAKER_02: And therefore, you you the what you accomplish is greater for the number of hours you put in.
• 15:54
  SPEAKER_02: And that's important.
• 15:55
  SPEAKER_02: So Docker is a big deal, but another one that's kind of newer on the radar, it's still developing, it's still being developed and kind of brought up to.
• 16:04
  SPEAKER_02: I'm not even sure it's version one yet, but uh Project Tie is another solution that can be really helpful for this kind of thing, because not only can you use it to automate, kind of like what you do with maybe a Docker Compose or something like that, but you can also use it to run your systems locally where they're all running outside of containers.
• 16:25
  SPEAKER_02: That way you can debug multiple applications if you have a microservice type structure, multiple applications at the same time using Project TI.
• 16:36
  SPEAKER_02: So that's another solution.
• 16:38
  SPEAKER_02: There's a lot of solutions out there.
• 16:39
  SPEAKER_02: You don't have to have a full Kubernetes installed on your machine in order to test out what or simulate what would be in a production environment, even if it is a Kubernetes environment.
• 16:51
  SPEAKER_02: So there's a lot of different options out there that you can do, but I'd highly encourage you when you're developing to have a full environment on your machine.
• 17:03
  SPEAKER_02: Get it there so you can make mistakes, you can break things, cause problems, and reset and only affect you, not your entire team.
• 17:13
  SPEAKER_02: The cost difference for your company will be huge.
• 17:17
  SPEAKER_02: Because again, taking the time of one developer is a lot cheaper than taking the time of a dozen developers.
• 17:25
  SPEAKER_02: So those are my thoughts on why you should have a local dev environment.
• 17:30
  SPEAKER_02: Now, if you're a team of one, put the environment wherever because it's just you.
• 17:35
  SPEAKER_02: But eventually you're gonna want to have that locally.
• 17:37
  SPEAKER_02: Another benefit there would be you can work on a plane.
• 17:40
  SPEAKER_02: You can work without internet.
• 17:42
  SPEAKER_02: There's reasons why having a local environment is still very, very helpful.
• 17:46
  SPEAKER_02: It's just not every reason will apply because you only have one person.
• 17:52
  SPEAKER_02: Okay, thanks for listening.
• 17:54
  SPEAKER_02: And as always, I am Tim Corey.
• 17:56
  SPEAKER_01: Thank you for joining us for this episode of Dev Questions.
• 17:59
  SPEAKER_01: Tim is committed to making it easier for you to become a developer.
• 18:03
  SPEAKER_01: If you would like to help make more content like this possible, please like, subscribe, rate, and share Dev Questions.
• 18:10
  SPEAKER_01: You can also send your questions to questions at IamtimCore.com.
• 18:15
  SPEAKER_01: Until next time, remember, you are too smart and your time too valuable to waste it making all the mistakes Tim did.
• 18:22
  SPEAKER_01: When you're ready to learn to think and code like a professional developer, head over to IamtimCore.com and enroll in a course.