Latest / Talking Security: Insights from Microsoft Security Experts / #33- DevSecOps series - Secure your build process and build environment
Transcript
- 0:00Welcome to the Talking Security Podcast. We will talk about items related to Microsoft
- 0:11Security.
- 0:14Hello, welcome again to the new recording of the Talking Security Podcast. This time
- 0:25again in the Defender for DevSecOps series and we're back with Pouyan and Sander. What
- 0:34are we going to do today?
- 0:35Today we're going to talk about the build section of DevSecOps. So last episode we talked
- 0:40about code and this episode we're going to talk all about the build environment and build
- 0:46process.
- 0:47How to build such piece of software or some infrastructure for example to enroll that.
- 0:58So let's dive in. Do we have, first of all, do we have anything to announce? It's almost
- 1:06MVP Summit, so we're both, all three of us, we are moving to America for a week. So we
- 1:14will do some recordings there, I guess.
- 1:17Hopefully.
- 1:18We'll take our gear, not this gear, but we'll take the gear with us and then let's see if
- 1:24we can talk to some people over there. So really looking forward. It will be my first
- 1:29time so hopefully it will be awesome. First time in Seattle.
- 1:33Your first time at the Summit as well?
- 1:35Yes.
- 1:36Oh really? I thought you were gone there for like a million times already.
- 1:39I think I'm the youngest MVP in this room.
- 1:41You're certainly not the youngest person in the room.
- 1:44No, no, no, no, definitely not. But let's dive into the DevSecOps build process. What
- 1:53is it? What things do we need to cover?
- 1:57Well I kind of see this in two different sections. So you have like the build process and you
- 2:02have the build environment. And I think it's really, really important to talk about security
- 2:07aspect here because in the normal DevOps flow, when someone talks about DevOps they usually
- 2:11talk about CI, CD, like building your software when something changes and then releasing
- 2:16it automatically instead of doing it by hand with FTP or just doing stuff by hand. But
- 2:21with DevSecOps it's all about the security aspect. So how do we make sure that this building
- 2:26process is very secure? Because it's very easy to introduce security issues with this.
- 2:31Yeah, first, first dive in the process and the environment, but first the process. Let's
- 2:37see what kind of things do we need to cover? Like the secure build, you talk already about
- 2:46security. What kind of aspects around security do we need to look into?
- 2:54Well I think it's good before we dive in into the aspects is to define what we mean when
- 3:01we are talking about build, right? Because when we are doing build on an infrastructure
- 3:06project, it's completely different than when we are talking about build, when we are about
- 3:11a.NET application where you need to generate stuff. So I think that's initially because
- 3:18after that we can start thinking, okay, there are multiple parallels that we can discuss
- 3:24and see, okay, what are the security challenges on that aspect? Is that a good intro?
- 3:30Yeah, that sounds good. I think it's a good thing that you're saying, Clique. We're not
- 3:34just releasing source code onto a machine. Also if we're talking about stuff like infrastructure
- 3:40as code or like bicep or stuff like this, you're not sending the bicep files to Azure.
- 3:46You're actually sending the result. So I think the build process could really be summed up
- 3:50as taking the source of a project and turning it into an artifact, turning it into something
- 3:55that will eventually run on a machine somewhere. And that can take many forms. Like in.NET
- 3:59you're compiling an application with a normal website. You're maybe just minifying it, but
- 4:04the HTML, the CSS and JavaScript stays the same. And with bicep, well, bicep is just
- 4:08turning it into ARM templates in the end.
- 4:11Yeah. I think what we also were discussing on before the start is how the build is also
- 4:18changing, right? Because with the introduction of the infrastructure as code, for example,
- 4:24what I told you guys earlier, in the early days when we did the infrastructure as code
- 4:28in Azure DevOps, we needed to do build and then create an artifact which we could use
- 4:32in the release deployed, which is now not the case anymore as well, right?
- 4:36Kinda, kinda. So now you use, yeah, you used to like click ops kind of the pipeline together
- 4:44with your by hand and it worked and it was really nice to begin with, but you didn't
- 4:48really have any backups. It wasn't in your Git repository or something like that. Nowadays,
- 4:52exactly what you say, you had like, well, you had, you used to have like a build section
- 4:56and a release section. And nowadays the modern approach is to use like a YAML file with your
- 5:00pipeline in it. And in there you define all of the build steps and then you can also define
- 5:05your release steps in there. Usually you do that in a separate stage or in a separate
- 5:08file. So it's nicely separated, but now it's a whole more, it's a, it's a different approach
- 5:12to building and releasing your application.
- 5:15Yeah. So if you focus on the, let's start with the infrastructure as code, for example,
- 5:20if you take a look when we are doing infrastructure as code or we are doing conditional access
- 5:27deployment through DevOps processes, right? Things, configuration as code. How do you
- 5:33see, how do we see the build stage and do we see any security issues in there as well?
- 5:40When only having infrastructure as code and deploying workplace related items, identity
- 5:48related items like conditional access, for example, can we deploy that in a secure way
- 5:55and how should we do that?
- 5:59So the authentication part you mean, or is it for example, for example,
- 6:03you need to connect to a certain Azure actor and try the environment. You need to deploy
- 6:09within the entire ID. You need to deploy the conditional access policies. And in this case
- 6:16we need an app rack that app rack needs some permissions, right? How do we secure that
- 6:24optimally? So nowadays we're using Key Vault. Is that the right way or do we store that
- 6:30in another way?
- 6:31So I think what you're going to is like, for this is of course kind of Azure DevOps specific,
- 6:36but I think you're leaning into like the service connections bit. Like how do we connect our
- 6:41Azure DevOps environment to our Azure environment to make the changes and how do we create a
- 6:46secure connection?
- 6:48It could be that that could be the way, but you can also run a script on a PowerShell
- 6:56script for example, or whatever. So there needs to be a connection and that connection
- 7:02can be established in different ways in my opinion.
- 7:05Yeah, yeah. You can definitely do multiple things. Yeah.
- 7:07So what you often see is I think at the moment that in Azure DevOps or people, you can create
- 7:13service connection, which you can use in your pipelines for example. But we had also seen
- 7:18scenarios where people hijacked the connection or the deployment process and then got the
- 7:25token out of the connectivity. And then through that they hopped through. So I think that's
- 7:32what you're meaning with the authentication part. But GitHub it's different. I think you
- 7:36need to store, you cannot define connections. You need to define credentials, which you
- 7:42can use to connect.
- 7:45Not really sure. I think under the hood, it kind of works the same in Azure DevOps where
- 7:48you create like a service connection to Azure. You can like use like an app registration,
- 7:51for example, like a service principle under the hood. But more modern approaches are like
- 7:55using a managed identity or a workload federation, something. Does someone here know the correct
- 8:02term? Some workload federation.
- 8:03Federated credential. That's something new. And that's really changing it because it's
- 8:09like managed identity, but it's so you don't have a secret anymore.
- 8:13Yeah, something like that. Yeah. And I think using, I mean, you can definitely use a PowerShell
- 8:18script, right? To like in your pipeline, say I want to connect to Azure. And instead of
- 8:21using a service connection, I'm just going to do it myself. You could, but I think this
- 8:26is going to come up often more in this episode is why would you do it yourself if you can
- 8:30use like a cloud hosted, Microsoft hosted or provider hosted option, like a service
- 8:35connection.
- 8:36And if we want, if we want to use that service connection, for example, what do we need to
- 8:40do to secure that optimally? Because last days I talked to a few customers regarding
- 8:47phishing resistant MFA that is more user based. From a user perspective, there are multiple
- 8:54challenges regarding MFA and phishing regard with the MFA token and so on. Is that, is
- 9:03that the same in the area with the service connection and using a managed, a managed
- 9:07identity of service principle, whatever, do we face the same challenges as well in, in
- 9:14creating connections between a DevOps environment, the Git em environment and a backend system,
- 9:20for example, Azure Active Directory or the, the, the build environment where you are publishing
- 9:26your, your software.
- 9:28I think, I think definitely, I think then it's more like, I don't, if you're using
- 9:33service connections, what I'm trying to say is that then it really depends on how much
- 9:38access you give that Azure resource, right? Like in the end you're using somewhere in
- 9:43Azure is defined a service principle, a user, let's just say it's like a user and that user
- 9:49is the thing that you're using in Azure DevOps to connect to your Azure environment. And
- 9:52I've seen a lot of cases where then they're like, Oh, it's really difficult to manage
- 9:55that. So we're just going to give that, give that service principle global admin and there
- 9:59we go. And then like what you said, like if you get your bill process hijacked, for example,
- 10:04you're using like a task that you don't know is malicious, then that of course could misuse
- 10:11that Azure credential and like hijack your systems. I think it's really important that
- 10:15you then know if I'm going to use service connections, if I'm going to use a PowerShell
- 10:18script, how much can I do? Like I want a minimal access.
- 10:22And I thought that we, in the last few months, we saw a few of these, these kinds of cases
- 10:27that a certain user, a developer is hacked and that the password and the multi-factor
- 10:33token was stolen and then they are into a DevSecOps pipeline or whatever. And then
- 10:40they running the software and then the software with malicious code in it is deployed to customers.
- 10:45So that is, if a user, a developer has access to that managed instance that apprack with
- 10:53a lot of rights in a certain environment, that could be potential big damage could be
- 11:02there.
- 11:03Yeah. Well then definitely in cases like this is of course, how do you say that? Like if,
- 11:11for example, I was hijacked, right? And I have a lot, I can have administrator rights
- 11:15somewhere, then that is bad, but hopefully at least my credentials are not used in the
- 11:20automated build pipeline. That is hopefully exactly. You see something you shouldn't do.
- 11:24You should definitely have like a separate entity or separate identity that connects
- 11:28to your Azure environment automatically. Of course, if someone has my token, they could
- 11:31probably go to the Azure portal. And if I have a lot of administrator, I can still delete
- 11:35stuff or delete the code, but at least it could not really be automated that much anymore.
- 11:40And then we're definitely going to have to talk about the, like exactly what you say,
- 11:43phishing resistant MFA, pass keys, passwordless authentication and that kind of stuff. But
- 11:49don't know how well that fits in the build process here.
- 11:52No, no, no. Now we are looking a little bit broader than only the build pipeline, but
- 11:59let's go back to the build process. What do we need to, we talked to some points. What
- 12:08do we need to cover next? Because we have, we have a built and then I've tried to deploy
- 12:16that built. What do we need to do in that, in that kind of steps? What do we need to
- 12:21do to secure? What do we store the credentials that are using? Are we only using a service
- 12:27connection or what is more within the build? So I think also pointing a little bit to the
- 12:35last question, I think it's a great topic. And I think if you zoom out on that topic,
- 12:43it's the whole reason why we are having this podcast, right? Because it's not in one process
- 12:47where we can solve that kind of issue. So the whole DevSecOps, it's the idea is to build
- 12:53that checks and controls in every stage and every phase. So for example, let's say we
- 12:59have that connection and if we have the right permission, we can use that connection. And
- 13:04we have, as you mentioned, our workflow instead of just somebody clicking it through the portal,
- 13:11but code defined stored in our repository. And like we discussed last time, we have our
- 13:16branch policies and repo protection in place. It minimizes the impact, the chance that somebody
- 13:23can hijack our workflow to build steps into it without checks being done ahead. So I think
- 13:34yes, there are possibilities, and I think there are a couple of big challenges when
- 13:38it comes to authentication side of the build or even release, because you're always working
- 13:43with service principles or user accounts or API keys, which have permanent permission
- 13:50basis.
- 13:51Exactly. Are you talking really a lot like minimizing the chances that your build process
- 13:55gets corrupted, gets attacked?
- 13:59Regarding on first time we talked about artifacts that we are not using anymore and we're still
- 14:08using it, but not for infrastructure scope. If we look at nowadays, should we use DevOps,
- 14:18GitHub, whatever, and why should we do that in a certain way? What are tools that we can
- 14:24use to optimally use it in the build process and to see how we can move forward on that?
- 14:32I got a couple of notes for this. One of the things, the tools, we're talking about toolings
- 14:36and how to optimize the security for your build and your build pipeline. There's definitely
- 14:40things that we also talked about in the last episode. We talked about vulnerability scanning.
- 14:44We talked about container scanning. There's definitely still a mention here for code QL,
- 14:48which is checking your code quality and stuff like that, maybe also vulnerabilities. You
- 14:52want to use those tools and those tools are best used in a build environment. Before you
- 14:57do a deployment to your production environment, there's going to be some vulnerability scanning
- 15:02happening. They're like, hey, in this version of the SQL that you're using in the package,
- 15:07we found a vulnerability and the deployment will be stopped.
- 15:11Stuff like this is really how you can secure your code deployment process. That is definitely
- 15:20something that with a DevSecOps approach, you want to automate. You don't want someone
- 15:24to say, oh no, let's just take a look at the vulnerabilities. We'll look at the vulnerabilities
- 15:28each sprint. No one actually does that. That needs to be automated.
- 15:32We had a great session on this topic with David, of course, on the DefSecOps, on the
- 15:37Defender for DevOps, sorry. How Microsoft is also helping to bring visibility into that
- 15:46process of the build and release. For example, what you exactly said is you want to see which
- 15:52vulnerabilities are coming in the code maybe, but also in the configuration-wise. Maybe
- 16:00secret scanning, we know GitHub has it already or the advanced security has it in house.
- 16:07With tools as Defender for DevOps, you can integrate, for example, with GitHub or with
- 16:12Azure DevOps. Then you step in into the build process and into the PR process. Then we can
- 16:23say, okay, if you create a PR to make certain changes, then this pipeline needs to run and
- 16:30then do some checks before we allow it to be merged. If there are any findings like
- 16:37there is a static password in your code or you are creating for... If we go back to
- 16:43the infrastructure scope, you are doing storage account with weak encryption or with a VM
- 16:51with a public IP address. You see that kind of configuration or at least which can end
- 16:58up into being vulnerable already into that build process. That will be integrated to
- 17:04Defender for DevOps as well, so in Defender for Cloud. What I really love about the Defender
- 17:10for DevOps is it has a feature. They call it PR integration where the feedback goes
- 17:19back to your PR. As a developer, you see, okay, I have a PR. Oh, on line 13, there was
- 17:26a static password found. On line 15, I had an adjacent template for a storage account,
- 17:33but I had a weak encryption model configured. As a developer, you don't need to escape your
- 17:40environment for that build phase, but they can bring it back to you.
- 17:44You're talking now about Defender to secure your code and the tools that you're using.
- 17:55Defender is integrating in GitHub and Azure DevOps. Are they the only tools that we are
- 18:01still using at the moment or are there any other tools that we are using to create a
- 18:07build environment?
- 18:12To create a build environment?
- 18:14To create your build, to create your code.
- 18:17I think there's definitely, I mentioned CodeCube a little bit. Things like SonarCube and SonarCloud
- 18:22that will also do a lot of code scanning for you. That will also make sure that the thing
- 18:28that you are going to deploy is secure or is at least secure enough according to the
- 18:33policies that you've set up.
- 18:36In the other end, the question is more or less, we are still using GitHub at the moment.
- 18:43I don't know exactly, are you still using GitHub as well?
- 18:48I use Azure DevOps and Azure DevOps Pipeline for work.
- 18:52Both are within the Microsoft ecosystem, more or less general products that we are using.
- 18:59Are there other tools like GitHub, like Azure DevOps that people in the wild are using to
- 19:06create code?
- 19:07Sure. To deploy their applications, maybe you mean.
- 19:10For example?
- 19:11Yeah, there's things like Jenkins, right? Which used to be GitLab, CircleCI, I believe
- 19:17GitLab definitely. CircleCI is a really famous one.
- 19:20GitLab is an on-prem environment.
- 19:22I don't know, it's also served. It's also hosted for you.
- 19:26It's hosted, but you can also build it by yourself.
- 19:30GitHub as well, I believe. I think a part of it.
- 19:32Not GitHub.
- 19:33I know Azure DevOps in the past, you could have it on-prem.
- 19:36Yeah, yeah, yeah.
- 19:37DevOps Foundation.
- 19:38DevOps Foundation.
- 19:39Yeah, yeah, yeah, exactly.
- 19:41But Jenkins is in the cloud service?
- 19:44Not sure, actually.
- 19:45Why is that?
- 19:46I'm happy to not really use it that often.
- 19:48For me, it's either. But if we are using tools like Defender or SonoCube or whatever to secure
- 19:55your code, is it possible to have that on the other tools as well?
- 19:59Because in my opinion, Defender is only available at the moment for GitHub and Azure DevOps.
- 20:06So within the Microsoft ecosystem, it's okay. But if we extend that, and we want to extend
- 20:10that to other developers, probably we can make a switch there as well.
- 20:15Yeah, I think the good thing about GitHub and Azure DevOps and that kind of stuff is
- 20:20that all of these things like vulnerability scanning, secret scanning, and hopefully in
- 20:24the future things like automatic package updates, it's just all integrated into GitHub, into
- 20:30Azure DevOps.
- 20:32For other platforms like GitLab and what we talked about also, you can definitely still
- 20:37install SonoCube, but it's more of a manual job. But I think, regardless of the platform
- 20:42you're using, you should be using these kind of tools to ensure that what you're going
- 20:46to deploy is secure and meets your expectations.
- 20:49Yeah, and in the modern world, we're using GitHub, for example, or Azure DevOps. And
- 20:54we try to, within the build process, we want to do everything within GitHub, like credentials,
- 21:01like a service connection, like code scanning, everything.
- 21:03Yeah, definitely use the tools and the options that your provider gives you. Don't start
- 21:09building stuff yourself. There's nothing really wrong with building stuff yourself. Who's
- 21:13going to maintain it? Who's going to maintain that PowerShell script that might be using
- 21:17an old version of the Azure API? Just use Azure DevOps service connections and Microsoft
- 21:21will host it for you.
- 21:22Another thing I might want to bring up into this process thing to kind of move away from
- 21:25the tooling. So as a developer, if I think about securing the build process, for me it's
- 21:31really important to also make sure that what you are trying to build and trying to publish
- 21:37is secure. There's a lot of articles online about people that accidentally also commit
- 21:44and build their Git folder. So they actually deploy their.git folder to a website. And
- 21:49then if you visit the website, you can just go to slash.git and steal all of the source
- 21:52code.
- 21:53So that's also definitely a thing that you, when you're going to build something, really
- 21:57inspect what is the result of that build. And am I deploying, for example, stuff like
- 22:02the Git ignore file that's really not necessary? Am I deploying some files that should actually
- 22:07be in the Git ignore? Where is my Git folder in there? Because it really shouldn't be.
- 22:12You want to make sure that your build is optimized.
- 22:14So let's search the internet.
- 22:16Yeah, there's tools for that. There's tools for that. You can just, you know, it will
- 22:21just go through tons of websites on the internet and just go to slash.git and bam, you got
- 22:25all the source code.
- 22:26You have the whole Git history then from there.
- 22:27Yeah, you have the entire, you have all the source code. And that's also the thing that
- 22:29I always tell people, like sometimes they commit a secret into their repo and they're
- 22:33like, oh no, I'm just going to put another commit on top of it to revert that. But I'm
- 22:37like, no, if someone has access to your Git history, they can just go back to last month
- 22:41and they'll find the original thing again.
- 22:43The same is for PRs. So some people say, okay, I have multiple commits in my PR and that contains
- 22:51a secret and then, but I merge it and I smash it to one. It's still in the history of your
- 22:56PR, you can find it.
- 22:58Just a bit of self-promotion. I have a blog post on my web page. No, but exactly. You
- 23:01want to, but if you're using...
- 23:03But how do you control that then?
- 23:06So what you do is you, if you're using stuff like GitHub or if you're using stuff like
- 23:09Azure DevOps, there's this feature called GitHub advanced security or GitHub advanced
- 23:14security for Azure DevOps. And this contains tools that you can just click with one click.
- 23:18You can enable it and then it will actually like every time you, when you will try to
- 23:22push code to Azure DevOps or to GitHub, it will scan all of your code. And if there's
- 23:28a secret in there, which it just learns machine learning, it's just smart. And then it will
- 23:32just reject your push. And there's it was...
- 23:35That's what GitHub has, right?
- 23:36Yeah. But also Azure DevOps.
- 23:37Yeah. That's pre-commit checking if there's a password.
- 23:40On the server as well. On the server it checks it. So the server receives a request with
- 23:45the new code and then before it accepts it, it scans it for secrets. And there's also
- 23:49really, really interesting where GitHub does is if it, for example, there's services like
- 23:54SendGrid, right? For email sending. And those API keys, like you said, they are just like
- 23:59permanently valid, which is a big problem. If that leaks, you have a really big issue.
- 24:05So a lot of companies are now doing that. The API key starts with like a prefix, like
- 24:09for SendGrid, I think it's SG and then a colon. And then what GitHub does is it when it detects
- 24:16a secret, starting with SG and colon, it will automatically send us a request to SendGrid
- 24:21to invalidate that API key. Which is like, even if you leak the secret, it will be immediately
- 24:27invalidated.
- 24:28You can do that as well for your own projects. You can define your own keys, which it will
- 24:32search in other repositories, I believe.
- 24:36That is useful, but it is scary because like if the secret just doesn't meet that regular
- 24:40expression or something, or doesn't meet the filters, then it will leak through. But yeah,
- 24:44you definitely want to make sure that your code. And also what I do in my pull requests,
- 24:49I have a pull request template set up. So every time one of my team members has a pull
- 24:52request, I have like a little checklist that you can just create in GitHub and then should
- 24:56develop. So like, did you add documentation? Does the project requirements need to be updated?
- 25:02For example, if you add a connection to a new database, you need to update so people
- 25:05can use that.
- 25:06Also in there, it's a bit of a manual thing, but like, did you leak any secrets? And I
- 25:10hope to make people think twice. Like first of all, I always tell people, don't put them
- 25:13in your repo and all that kind of stuff. I try to teach them about it, but also make
- 25:16it part of the PR flow that you actually have to visually check that checkbox.
- 25:21It's a check mark that you need to check. And it's visible. So you see it, it's visible
- 25:27and you think twice. That's what you realize with that one.
- 25:31You definitely want to automate it, but it's also just good to teach people. Just don't,
- 25:35you know, don't rely on tooling too much. Like then people are just not going to think
- 25:39about it anymore. They're going to work in an environment where these tools are not in
- 25:42place and then things are going to go unstuck.
- 25:45In the normal user environment, we have security awareness to educate people with phishing,
- 25:50with clicking on URLs and so on and so on. Within the Defender for DevOps, often in the
- 25:57DevOps, DevSecOps world, we need to create awareness regarding these kinds of stuff.
- 26:03So do you have a secret stored in your code or do you use other stuff that is malicious,
- 26:10that you can do stuff that you don't want to know that it does?
- 26:14I think awareness is one of the key aspects within a DevSecOps world.
- 26:20Yeah, definitely. That's also the thing that we're trying to integrate, right? Like the
- 26:24DevOps thing, it kind of exists. Lots of companies are doing it. It's so much better than doing
- 26:27stuff manually or Lisa's computer is the production release machine. You know, that's just that
- 26:32you don't want to work like that. You want an automated system and making it secure is
- 26:35very important.
- 26:36What I really love that I think I mentioned a couple of times on the PR integration of
- 26:40the Defender for DevOps. I mean, it doesn't force the developer to come to a security
- 26:45environment and the phish and phersa. But they talk the same on the same platform that
- 26:51they need to intend. But if we take a look on the build, right? So on the build side,
- 26:56there are credentials involved. And I think from our perspective, it's often about some
- 27:01ARM templates, PowerShell coding and that kind of, but I think when you are building
- 27:05an application, you have more requirement to that agent where you do the build, right?
- 27:13So bring us to that. What, if you are doing a build of an application, what are the requirements
- 27:20often that you are meet? Because I can imagine that organizations say, well, you can do this,
- 27:25but it's an, it needs a certain version of Java or net or it needs a certain network
- 27:34connectivity. Do you use like the GitHub runners and Azure DevOps runners or do you use your
- 27:41own stuff?
- 27:42Oh, we can talk about that. That's really interesting. One little thing that I wanted,
- 27:46we talked kind of about this yesterday and you triggered me about something. So one thing
- 27:49that I want to, I want to talk about really, really, it's really interesting is that there's
- 27:53also now, instead of like building your own CI CD pipeline and having to think about what
- 27:57version of Java you're using, are we using Ubuntu 20 or are we using Ubuntu 18? A lot
- 28:02of people are still using Ubuntu 18. They shouldn't do that anymore. There's also these
- 28:05approaches in Azure where you can just say, Hey, this is my GitHub link and this is a
- 28:10react project. Go release it. And again, Azure will just figure it out for you. So there's
- 28:14not even, you don't even need to think about it anymore. Just when there's a new commit
- 28:17on main, when some code is finished, Azure will automatically detect it kind of in a
- 28:21GitOps approach, just push it to the server. And then it's even more secure because you
- 28:26don't even need to think about the builds anymore.
- 28:28Yeah, we have the same with Sentinel for example, where it's a little bit different what they
- 28:32are doing. Sentinel is a Microsoft SIEM solution. So it's where they bring all the alerts and
- 28:37incidents together to create an overview, which we use often for our SOC and that kind
- 28:41of services. But what they do is you have also repository integration. And if you go
- 28:47and you say click and say, yeah, is it on DevOps or on GitHub? You say it's on GitHub,
- 28:52just as authenticated GitHub. And what they do is they install an extension, which creates
- 28:58a deployment workflow for you on the show. I think even there, there are multiple version
- 29:05because it looks like that Sentinel is pulling that. But because you created that connection,
- 29:11you installed an extension, which is doing...
- 29:14Yeah, it's pushing it. Yeah, yeah, something like that. Yeah, yeah. But that just, it's
- 29:18just another thing that could be automated and could be secured by Microsoft. But let's
- 29:23go back to what you're like, let's go back a step to what you're saying about like as
- 29:26a developer, what do I need? Because if we're thinking about deploying a bicep or deploying
- 29:31ARM or deploying Terraform, we usually don't need that much, right? I think you, like I'm
- 29:35just saying you guys probably need stuff like, you still need, your only thing you need is
- 29:39probably like some Linux machine with like the...
- 29:41PowerShell.
- 29:42With PowerShell.
- 29:43Azure PowerShell.
- 29:44I feel like that is all you need. But for like a developer for me, so I'm like deploying
- 29:47maybe an application with a backend that's written in.NET, a front-end written in React.
- 29:51So I'm probably, I'm gonna use modern.NET, so I'm running on Linux. But I definitely
- 29:56need a specific.NET version installed. I need a version of NPM installed, maybe yarn,
- 30:02maybe a lot of project dependencies that it all needs to pull in. And yeah, there's a
- 30:06lot more dependencies to build. Luckily with stuff like Docker, which we also kind of mentioned
- 30:11this, some of this pain goes away because you can just let Docker do all of this. But
- 30:14yeah, you definitely need more dependencies, which you also hope are secure.
- 30:17So do you use, even when you need to have certain specific version of software tooling,
- 30:26the public hosted, the managed versions of GitHub or Azure DevOps, or do you use your
- 30:32own? Because there has been, I think definitely in the past, there has been a lot of discussions
- 30:37like sharing and build agent with multiple organization. And there were some scenarios
- 30:43that people could see things that they shouldn't be able to see.
- 30:47Oh yeah, yeah, yeah. I remember it. So yeah, yeah, you're right. I always use that's a
- 30:51true risk, right? If you are talking about build, seeing, being able. So if you're running
- 30:58your program and I'm running my program on the same build agent and I find a way to escape
- 31:04and then pull your code out there as well, we talked about the connection, right? To
- 31:09take over the connection. But maybe instead of connection, we can step in and change the
- 31:14code or pull some of the code away.
- 31:18I think it's like, you know, there's this security thing where if you're securing your
- 31:22house, the only thing you need to care about is that your house is more secure than your
- 31:26neighbor because then the robbers will rob your neighbor and not you. And if I'm thinking
- 31:30about like setting up my, either my own like a hosted agent where my code is built on,
- 31:37like I have a VM in Azure somewhere that runs and builds all my code, I have the agent installed
- 31:42on it or I'm just using the Microsoft agent. I prefer to use the cloud version. We at the
- 31:48company I work in the beginning, we did use the, we just had our own VM. And then, I don't
- 31:54know, there was one guy that always took care of it, right? He was a great guy. He was really
- 31:57good at what he did. And at some point he was working with some other stuff. Four years
- 32:01later we discovered that there were no updates. That's like, and that's the thing. Like if
- 32:05you're working depends really on the company, on the company culture as well. If there's
- 32:08actually security specialists in there, if there's ops people on there and they, and
- 32:13you're building really important code that really needs to stay in your environment and
- 32:17your people are specialized in keeping that stuff up to date.
- 32:20Yeah. But if you are self hosting staff, then you need to take care of life cycle management.
- 32:26Exactly. And update, not only updating, but also, also like from a security perspective
- 32:35on day one, we have, I have zero days or you need to update on the second day. There's
- 32:41an open port that is discovered because you need to connect to an agent on a specific
- 32:47port and that's in that specific scenario needed. But two months later, that specific
- 32:53scenario has changed. Yeah. You need to change that port, close it, open another one or whatever,
- 32:59that you need to, you need to take care of all the stuff on prem. And if you're using
- 33:05a cloud agent, for example, then yeah, Microsoft or someone else is taking care of all that
- 33:11kind of stuff. And you have it on about your own house, secure it more than your neighbor
- 33:19because otherwise you're robbed. But what if you are living in an apartment where you
- 33:24have one house with 10 apartments and the main entrance, the main entrance should be
- 33:29secured. And that's the people from the apartment building that are securing that entrance. So
- 33:37in that case, it's the same for the cloud agent where I use the same agent as you do
- 33:43with your own company and Microsoft or another vendor needs to take care of the security
- 33:50on top of that agent that is running. But also if it's multi-threat, so I have multiple
- 33:58organizations that are using the same agent and so on. So Microsoft is taking care of
- 34:02that. Yeah. But the thing is for me that, you know, I'm not a security specialist. I
- 34:06do like to think that I know a bit about security, otherwise I might not be sitting here. But
- 34:12my job is to develop applications. It's to kind of, you know, come up with architecture
- 34:16for applications. My job is not to secure stuff. So I really, I can't, I can't, I cannot
- 34:23in good conscience use a self-hosted agent because I know that Microsoft will do a much
- 34:27better job securing it. And it's, you know, this is, it's, if you're working at a company,
- 34:30like what is your core thing that you're working on? Like a web shops core thing is building
- 34:35a web shop. So should they also be building their own identity provider? No, they should
- 34:38use a built one. Are you, are you in a very, are you working in like, I don't know, like
- 34:42an airline or like at a bank, you might be using your own agents because it's really
- 34:46important to you that you need to manage it yourself for normal companies. This is just
- 34:50not their thing. So I really advocate for using the cloud version and Microsoft has
- 34:56people working there 24 seven to keep that stuff secure and any zero days should be fixed
- 35:01immediately and you don't even need to think about it.
- 35:03From a life cycle perspective, I totally agree. But if you, let's, let's look in that apartment
- 35:09house where Microsoft needs to close that, that, that the first door, if you are running
- 35:15multi-threat agents and you are running on the same agent as I do, how do I, how do I
- 35:23know that is secured, that I can access your code in case of, I think, I think that's more
- 35:30of an implementation detail, if I'm honest. Like I know that there's probably like, let's
- 35:34say there's 10 VMs, right? And you're thinking that you're working on the same VM. It's probably
- 35:38some Docker host. It's probably some hyper V stuff. Maybe it's running on the same machine,
- 35:43but the only thing you may be sharing is a kernel underwater. I'm not sure even how it
- 35:47works. I don't need to think about it.
- 35:48Well, I think it's, it's, they are changing. Definitely with GitHub, um, it hasn't been
- 35:55a big shift on Azure DevOps. It was mainly on, uh, I hear myself very weird. Is it me
- 36:01or no, no, no. Okay. Um, on Azure DevOps, it was always a hosted windows and the agent
- 36:08just run that. Um, but when we went to the migration to GitHub, GitHub's all, uh, hosted,
- 36:15uh, uh, containers. Um, and I believe Azure DevOps is also now using mainly because they
- 36:22share that backend. I mean, I've been using hosted Linux agents on Azure DevOps for at
- 36:28least six years, I think. But are there now really VMs or are there already changed to
- 36:34a container? I believe.
- 36:35I'm pretty sure you can run a Docker container with the Azure DevOps agent in it. You can
- 36:41definitely do that. So that would already make it easier, right? You can just kind of
- 36:45get the latest version of that agent and it will update all of the vulnerabilities. Again,
- 36:49something you have to manage yourself.
- 36:50Yeah. But if I want to deploy that to an Azure subscription or I deploy that, that specific
- 36:58application on a certain environment, do I need to integrate that agent within my own
- 37:04environment or is what are you?
- 37:07That's a good question. I mean, one of the, where do you actually, where do you put it
- 37:11in your infrastructure somewhere? Where do you put it in your landscape? One of the most,
- 37:15um, like most often attacked environments, I think, or one of the most popular things
- 37:19to attack in an, in a company is the build server. Cause sometimes people don't really
- 37:23care about it. They don't keep it up to date. So if you can just hijack that, you can just
- 37:27find all the source code there. You can find all the credentials there. So maybe you want
- 37:30to sandbox it, I guess. I feel like you really do not want it to have access to all of your
- 37:36own premise or all the other servers. At the other hand, if you're deploying to virtual
- 37:43machines from the build agent, it has to have access to the VMs, right? Cause it needs to
- 37:49be able to come to put the artifacts on there.
- 37:51Yeah. So that's, I think, um, in my opinion, one of the biggest re, uh, wall, one of the
- 37:56reasons why we see, uh, that people are, or customers ask for having hosted self hosted
- 38:01agents. It's often, it has to do with networking because they have like networks, databases
- 38:09or something in the on-prem environment or in a cloud environment, which is not the same
- 38:15is you will get the same issue when, for example, if you go into cloud, right? If you go to,
- 38:20let's say you have Azure web app, uh, where you have everything, Fina integrated and you
- 38:25have Azure SQL as a backend. Um, so your Azure SQL is not publicly accessible anymore. And
- 38:33then you are going to do a build, uh, you are going to push the changes to your, uh,
- 38:38SQL database. That's not accessible anymore.
- 38:41True. I had a little note about that because some time ago I read that GitHub now does
- 38:46support this. So GitHub supports using a cloud hosted agent. So you don't need to think about
- 38:51it anymore, but you can then talk to V net integrated stuff on Azure. I'm like 99% sure
- 38:57I'm sure I'm right that I read about this. And then of course the first million comments
- 39:00were when will this be released for Azure devils? So I just looked that up because I
- 39:04believe, I believed it was completely impossible to use like a Microsoft cloud hosted agent
- 39:08and deployed to V net in Azure. And it kind of is the thing that Microsoft says on their
- 39:13documentation is that you just need to whitelist some IPs from that. And they change every
- 39:18week. They say they could change every week. So every week they upload a new list and then
- 39:22you need to upload those and whitelist those every week. But you can automate that. Yeah.
- 39:25But that's not new. That's the same as whitelisting the whole Western Europe data center IP address
- 39:31space. I mean, I had once a customer that said, yeah, give me the, I'm not going to
- 39:36whitelist that. Give me, I'm not going to open a firewall. Give me the IP address that
- 39:40I need to whitelist. You can download it. It's thousands and thousands of IP addresses
- 39:44and the changes that they remove. They remove IP addresses and so on. So changes are. So
- 39:51one of the, one of the other reasons we use ourselves, we use a self hosted containers
- 39:57on Kubernetes. And one of the reasons why we did that is because through that model,
- 40:03we have now the option to use the manage identity. You can do the same as well with the self
- 40:12hosted. So in Azure DevOps, you have like two versions of VM agent, but the one that's
- 40:17with the manage identity, I believe, I don't know. You can create like an Azure service
- 40:21connection with a managed identity and that will be used by the cloud hosted agent. Yeah.
- 40:25So you, what you do is you use the manage identity of the agent to authenticate to your
- 40:32Azure subscription, for example, to do some activities. Yes, exactly. So that's also a
- 40:39really awesome. Well, that's kind of the same thing you're doing, right? You just have a
- 40:43container. Yeah. You have the kind of the same thing. I want to ask you a question about
- 40:46this. So let's say that we need to, let's think in a modern way. So GitHub, if I'm correct,
- 40:51has the option to use a cloud hosted agent, which you don't need to update, which you
- 40:55don't need to maintain Microsoft or GitHub cares about all the security aspects of it,
- 40:59which is great. I can sleep at night. And let's say that it's correct that they can
- 41:03just talk to a V net integrated resources. So things that the internet cannot access,
- 41:08let's say Azure DevOps gets this as well, which you then still want to use self hosted
- 41:12agents. So for us it's double. Um, we do it partially for, so we use the build agents
- 41:21partially to use the manage identity, for example, to configure and maintain our own solutions,
- 41:27right? So our own platform and that kind of stuff, which are not internet facing. Um,
- 41:34but we also use that manage identity to do other stuff, uh, other resources, for example,
- 41:40without going too deep in it. So, um, how do I say it without saying too much about
- 41:45this? That's the fun part. But what you can do is for example, if you're in a scenario
- 41:49that you are, you're an MSP, right? You're not doing nothing with security. You're an
- 41:54MSP and you are using Azure Lighthouse to manage your customers. Normally you need to
- 42:00have a service principle which is going there. So you have always an epi registration with
- 42:05a secret, which is vulnerable for that. So in that scenarios, you can also decide to
- 42:11go to manage identities of your containerized solution. So when you are doing something
- 42:16as infrastructure scope to your customer or you are doing as your lending zone configuration,
- 42:21you can use those connections, right? Um, to do the proficienting of that. So in our
- 42:28opinion, being able to using that manager identity, um, but it solves a lot of the problem
- 42:35that you initially mentioned.
- 42:38I don't really understand the difference though between using a, like I configure like a managed
- 42:43identity in Azure for my Azure DevOps or GitHub action build, right? Build agents. The one
- 42:50that's made by Microsoft in the cloud somewhere. Um, and I only use that one for pro deployment
- 42:55permissions. It's not going to be able, it's not going to be allowed to like, I don't know,
- 42:58change Azure settings, you know, that as little access as possible. You're doing kind of the
- 43:02same thing. You're hosting your, uh, build agent yourself, but you're also talking about
- 43:08the managed identity access. But how does that differ from what I'm saying with the
- 43:12service connection and managed identity?
- 43:16Um, service connection with the managed identity.
- 43:21Yeah. You're saying like on your Kubernetes environment, you can like configure, uh, it
- 43:27can be a VM as well. It can be a Docker, uh, container. Yeah. Your pod in Kubernetes has
- 43:32a managed identity associated with it. And in my Azure DevOps cloud hosted agent also
- 43:37has a managed identity because of the service connection. Yeah. But it's on the VM, right?
- 43:43On the cloud hosted. Yeah. So, okay. So is your, your, your, you don't have on the cloud
- 43:48hosted one. You don't have managed identity, right?
- 43:50No, no, no. But the cloud hosted one uses a service connection to talk to Azure. Yeah.
- 43:54And that service connection uses a managed identity.
- 44:00That is not possible. All right. Pretty sure it is a federated connection. I mean, I'm
- 44:06pretty sure it is possible. Okay. Let's just say service principle because the idea behind
- 44:11managed identity is that the identity is not managed by you. Exactly. So you don't know
- 44:19the secret of it. Exactly. And I, I, um, maybe we need to take this offline podcast, have
- 44:24a look around. Maybe we can put it in the show notes or something like that. Something
- 44:26like that. That would be awesome. In any case, what the thing that, um, let's say that the
- 44:32security aspect would also be managed by Microsoft. Yeah. So the, the, the update parts would
- 44:36be managed by Microsoft. I could integrate with V net. I could integrate with my private
- 44:40resources. Would there still be benefits to using self hosted agents? If the security
- 44:45aspect was handled by Microsoft in the cloud and if the V net integration was handled by
- 44:49Microsoft. Um, well, um, depends on your level of security, I think. And the, the application
- 45:02and how much risk you accept. For example, if you are doing some infrastructure code
- 45:07and deploying some resources, it's way different than if you are building databases and application
- 45:14and which are doing financial stuff or true. Yeah. That's a good point. So I think it's
- 45:19an, there's always a risk with the public hosted by someone else. Um, from a modern
- 45:27way I do accept it. Like if you're a small organization, you don't have the resources,
- 45:31don't start with it. Right. I do completely because you need to manage it. You have lifecycle
- 45:36management, you need to patch it, you need to monitor it. You need to have the fender
- 45:41on it. You need to alert, react on it. So yeah, on that sense, I do agree with you guys
- 45:47that that if you don't have the resources and the capacity to do it well, um, then don't
- 45:55do it. But if you have like, but I think that's in always the case. If you have the resources,
- 46:01the budgets and to make, if it makes fun, if it makes sense financially, if there's
- 46:06like, if you did like your risk assessment and you're saying, well, this is a pretty
- 46:09big risk for organization. It depends on the user. It depends. That's a great consultant
- 46:13answer here. Definitely. I have one last thing. Um, we talked about to build process, the
- 46:20built environment, but if we want to access from that development, a developer perspective,
- 46:27if I want to access that, uh, that the built environment, the, to, to create code and so
- 46:36on, you're doing that on your own laptop. I'm creating code on my own laptop is what
- 46:41you're saying, right? Yeah. I'm using my local digital studio or rider, stuff like that.
- 46:44How can, how can, and maybe we, we, we need to take another episode on that, but how does
- 46:51dev box within Azure Azure dev box? How does that come into place regarding the things
- 46:58that we're talking about? Because securing your build, uh, stuff, right. It's also securing
- 47:05the entrance, how you access your build. Yeah. I think that's, I think, I think we talked
- 47:10a bit about this in the first episode about workspace and management a bit. Yeah. I talked
- 47:15to a colleague yesterday cause they were working for a client and I was like, so what kind
- 47:19of environment are you using? He said, like, I'm using, I'm using the, uh, the Microsoft,
- 47:22what was it called? Microsoft box in Microsoft or Azure dev box or something. And he said
- 47:25that works really, really well, which we talked in the first episode about it, that I've heard
- 47:30of people were using it or like stuff like GitHub code spaces, which is even more like
- 47:34as a sauce platform. I think it is definitely very interesting. And we could maybe dedicate
- 47:38another episode to this. Maybe I miss it. And maybe what that also our listeners, can
- 47:44you maybe, uh, what is that? That's a great point. Yeah. Let's talk about that. The, the
- 47:50dev box is, um, more or less a workplace within Azure that you can spin up a sort of an ACD
- 47:58environment. Um, it's not AVD. Don't don't go to virtual desktop. Yeah. It's, it's not
- 48:04an AVD environment, but it's a virtual workplace that you run in Azure. So you can connect
- 48:10it remotely, but it's dedicated to a subscription or a, a, a, an environment that is used to
- 48:18develop stuff. All the, all the software needed for program software development is already
- 48:23installed on it. So instead of like getting a new laptop from your work and spending two
- 48:26days configuring visual studio, official leader code, GitHub, get crack. It's already all
- 48:32done for you. So you can just, you know, click on start or there's a new developer. It's
- 48:36a VM still it's under the hood. It's a VM AVD probably as well. It's just AVD on steroids,
- 48:41I guess. Yeah. It's not AVD, but it's, it's a VM where you can, yeah. But otherwise, I
- 48:47hear people are complaining dedicated decision about definitely, but for interesting about
- 48:54the dev box and that's, that's one of the nicest experiences that it's dedicated to
- 49:00a subscription. And with that, from a cost perspective, that subscription belongs to
- 49:06one project. So you have a project, you're running that project and what are the costs
- 49:11of that for the prep, the running costs for that project. So you have a subscription,
- 49:15all the costs are on that subscription. So you have a one project with that, with that
- 49:19costs. So it helps also when you hire externally, external people, they can get access to the
- 49:25dev box and exactly. And only if you are running a bill pipeline or whatever, it's only based
- 49:35on that specific Azure subscription or multiple Azure subscription that you are attached to
- 49:40the dev box. You can access other resources in your environment. So it's really specific.
- 49:46But I think it's definitely, I definitely have some more questions or things to say
- 49:50about this. I think it's definitely good to talk more about this later, but one little
- 49:53question about this. So how the build process, how is this related to dev box?
- 49:59Now if we, the thought that was in my head was if we are building a, an application and
- 50:08so you're running a pipeline or whatever, we can secure that on an optimal way, but
- 50:15accessing that bill pipeline, if you don't secure the access to your bill pipeline. And
- 50:21we talked a little bit about in the first episode regarding a workplace, right? But
- 50:25on the other hand, you can access a GitHub environment, you do authentication and multi-factor
- 50:33and so on. But if you have your source code, source code within a official study, your
- 50:40local machine. Yeah. So yeah, exactly.
- 50:43That was the reason why it pops up in my head.
- 50:46Exactly. You could, you could definitely secure and like put all the security measures in
- 50:49place to make sure that your code is deployed securely onto your, well, the thing you're
- 50:55your, your landing zone, wherever you're putting it. But yeah, then we kind of go back to the
- 51:00question of how do you secure the access to the thing that is being built. And that could
- 51:05be exactly something like dev box, like where at the other hand, I feel like you could,
- 51:09you know, pry this laptop from my cold dead hands. Like don't, I love having a laptop,
- 51:13but that you're completely right. If I left the company somewhere, I could still have
- 51:16that code on my laptop. I could take that with me with dev box. Yeah. You could just,
- 51:20you know, delete, disable my Azure account and you don't have, you don't need to format
- 51:25my computer anymore. It doesn't need to collect dust for five years in a closet somewhere.
- 51:29I think I know someone from, from the Microsoft team who is on, on that box. So we probably
- 51:36can invite a special guest to have a specific scenario or a specific episode on that box.
- 51:43Let's do that. Sure. It's really good. And regarding the build process environment and
- 51:49stuff that we do, do we need to take another thing or did we miss something? Maybe, maybe
- 51:56a simple thing that I sometimes see people do. I think we, we definitely talked a lot
- 52:00about making sure that your build process is secure, making sure that you have like
- 52:05a well fitting build environment, like either self hosted or like cloud hosted, making sure
- 52:12that it fits. One little thing that I sometimes see people do is that they're kind of new
- 52:16to this build process and they're like, Oh, I need a specific task. Right. And then they're
- 52:20looking on the marketplace for like GitHub actions or Azure DevOps tasks. You really
- 52:26want to make sure that the tasks you're using are okay tasks. Like there's even settings
- 52:32in Azure DevOps, I believe to not allow marketplace tasks. If you could have everything secured,
- 52:38but then you download a task that just copies all of the source code and sends it to Russia.
- 52:43You could, so you definitely want to make sure that the tasks you're using are secure.
- 52:50They are, they're trusted. And yeah, cause you don't know, you don't know the source
- 52:55code of the task. You don't know what they're doing. So the things that are happening in
- 52:58your build, you want to make sure that those are okay. Those are secure. That's a really
- 53:02good one because I think that's happening more often than people. Yeah, exactly. I need
- 53:06to replace some tokens. Oh, and I know that this one has a 600 downloads and the last
- 53:11update was four years ago, but it does what it needs to do. So I'm just going to use that
- 53:14one. Like, ah, so we spoke about the technology and I think the last one was great, but is
- 53:19there also a human interaction that we can discuss if it comes to our bark, more role
- 53:26based access requirements? Other, what do we expect from, um, our team when it comes
- 53:34to, I think if you're like running a build pipeline, it's also definitely, you want to
- 53:39make sure that it's only specific people can run specific pipelines where like deploying
- 53:43your cloud resources in a really big corporate environment. Maybe only the, you know, the,
- 53:48the ops people need to be able to deploy actual Azure production resources. There needs to
- 53:52be the four I principle where if I want to change the connectivity, uh, stuff in Azure,
- 53:58you know, all of the V net stuff, all the connections of Azure firewall, you want to
- 54:02make sure that your colleague Steve, that is also really good and you know, works with
- 54:06the, is specialized in this. He also needs to have a look at it. That's kind of, I guess
- 54:10the human aspect. You want to automate your, automate your build process. That's the entire
- 54:13thing about DevOps and kind of, but you want to also make sure that when specific and important
- 54:20things are happening, that's, I think where you need a human to double check.
- 54:23Yeah. But also a role based, you have a pull request. It need to be approved and so on
- 54:30just before the belt is actually running for ice principle, maybe six eyes that you want
- 54:36to cover. I think that's, that's important. And also what you mentioned a few minutes
- 54:41earlier regarding the, the, the, the list that you need to approve the check mark that
- 54:48you need to do before moving into PR. I think that are things that awareness we talked a
- 54:56little bit about that, the awareness also within the DevSecOps, the DevOps people. Yeah.
- 55:01That needs to be, that, that needs to be raised in my opinion.
- 55:05Yeah. We want to make sure we want to know what we're deploying. Like the, I think that's,
- 55:09I think that's a good way to say we want to know what we're deploying and we, the deployment
- 55:12should have an automatically, that is what we're doing all of this for. We just want
- 55:15to have code and we want it to run on production and we don't want to use Lisa's computer to
- 55:20do that. We want to use a cloud hosted agent, a self hosted agent, just let it happen. But
- 55:24what is being deployed? So you don't know that thing. It's like a whole, it's a whole
- 55:28old joke about, you know, I don't know. I don't know. Introduce her to me. No, but I
- 55:34don't, it's on the flight and the eight. Sure thing. So like one specific, one team
- 55:40members laptop is always used to like, deploy to production and that's an old man, whatever
- 55:45we all know that scenario with customers that there's all always one server that has enough
- 55:50permission to do the deployment. Exactly. I have faced that personally with a, with a
- 55:54really big enterprise customer when it came, every need, everything needed to be dev ops
- 55:58and everything needed to be infrastructure scope. But then you needed to email the code
- 56:03to somebody and then that person went to an SSH to an Ubuntu server because that server
- 56:07had the right login already to the Azure site. So they run it manually then and they, oh,
- 56:14nice. I can add one more thing to add one more thing on top of that. I used to do an
- 56:19internship and I also like finished my studies at the Dutch tax services to the blessing
- 56:23needs. And it was also, I worked at like a very modern department there where they built
- 56:27a mobile app. So it was really fun. And they also started doing like deploy and build agency
- 56:31ICD in there. So, you know, the code was built and quality was tested, like all the stuff
- 56:36we talked about today. And then there was a step to go to a specific Bali, you know,
- 56:41to go to a specific place in the company where you had to fill in an A4 paper. And then afterwards
- 56:45the process would continue, which I just found really, really funny. It's all super automated
- 56:49until you get to the deeper and deeper parts where you actually need to write a piece of
- 56:52paper and go somewhere. And yeah, it's not, it's not, it's not particularly wrong. It's,
- 56:59it's good that you have to sign off, I guess in a government environment. Yeah. Yeah. On
- 57:02the other hand, we want to try to digitalize everything. And then if you need to sign a
- 57:09paper, it could be good. It could be weird for some people, but on the other hand, if
- 57:15you have, you need to take that step. So you edit for awareness perspective, I think is
- 57:20really good. Yeah. Okay. I guess. Yeah. I mean, everything could of course be optimized
- 57:23better, but you do definitely have a point, especially that's more to cover the change
- 57:27management than I think. Right. I guess a little bit like who is, who is authorizing
- 57:31the deployment, you know, that kind of stuff. Yeah. Because that's fun to me. Big important
- 57:37thing, I think in our next step, because I believe the next one is going to be about
- 57:40release then. I'm not really sure. We'll have to, we'll have to take a look. Yeah. Okay.
- 57:45But because the change management is going to be a big one, I think, because when it
- 57:51comes to build and deployment, we deploy more often than we did previously. Right. So maybe
- 58:01in the past we did collect all the changes and then we did a deployment once a month
- 58:05or something. But now we are doing multiple ones. So, um, and this is a really, uh, challenge
- 58:13for this with a couple of customers of mine that say, okay, we need to be in control.
- 58:18We need to see how much deployments we need to the production environment. Um, but I think
- 58:24we'll take that for the next episode. Let's close this one. Yeah. We are almost an hour
- 58:31or so. We make that. Um, let, let's look a little bit back. Um, we covered the build
- 58:40pro the build process, the built environments. Um, we talked about a little bit dev box,
- 58:48make a little bit, a little step outside. Um, we talked about securing code, uh, also
- 58:54within this episode. So, uh, we, I think there are lots of aspects within the build that
- 59:00comes into place. Um, and that's make the mix. This one of the interesting topics, the
- 59:07steps within creating, uh, an application or building, um, an infrastructure, uh, you
- 59:14need to take care of your build because a lot can go wrong within this step. Yeah, definitely.
- 59:22So we talked about it, about the defender for DevOps, um, in a few episodes earlier
- 59:28on the, uh, what was it? The defender for service, uh, of the, uh, and for series. So
- 59:39if you want to have, you are interested in that one, uh, take a, have a look at that
- 59:43episode. Um, on the other end, that was it for today built and that let's look into the
- 59:50next one and probably release or something in that, uh, maybe at summit we'll see. Yeah,
- 59:56let's see if we can connect with, uh, with, with some people over there and see if we
- 1:00:00can create, uh, some other content, uh, on the other side of the ocean. Definitely. Let's
- 1:00:06do that. So thanks for today. If you liked this, uh, this podcast series, um, like and
- 1:00:11subscribe, um, and also, um, let us know if you want to know more about a specific scenario
- 1:00:19or what else, uh, we will take care of that and see if we can create, um, an, an next
- 1:00:24episode on that. So reach out to us on the socials and, uh, I see you next time. Thank
- 1:00:28you guys. Thank you. Thank you.