Latest / Microsoft Cloud IT Pro Podcast / Episode 349 – The War of the Policies
Transcript
- 0:03Welcome to episode 349 of the Microsoft Cloud IT Pro Podcast
- 0:08recorded live on August 18th, 2023.
- 0:11This is a show about Microsoft 365 and Azure from the perspective of it pros and
- 0:16end users where we discuss a topic or recent news and how it relates to you
- 0:21today.
- 0:21Ben and Scott talked through some considerations when using Azure policy to
- 0:26enable diagnostic settings at scale. They discussed various tools, oddities,
- 0:30permissions and more to help you get your environment configured the way you
- 0:34want even quicker. Ben, the button.
- 0:39Clicker stitching .
- 0:42There you go. That's it.
- 0:44We just kinda started talking and didn't bother hitting record buttons today.
- 0:48Now we're all recording. That's what we were waiting for Pirate.
- 0:51We were waiting for us to click the record buttons .
- 0:54All right, now that we got that out of the.
- 0:56Way, now that we got that out of the way,
- 0:57are we gonna talk about clicking more buttons?
- 1:00We are going to talk about how to get away from clicking buttons
- 1:05and having the system do the work for you. Once you,
- 1:09now you might have to click a few buttons. You still.
- 1:11Have to click buttons.
- 1:12Does the energy count as a button? Click.
- 1:14Unless you don't have the permissions to not click the buttons and you still
- 1:18have to click a big bunch of buttons to avoid clicking buttons.
- 1:22A button, click here a button, click there a button, click everywhere.
- 1:25This is spiraling downhill. Quickly, Scott,
- 1:27maybe we should just start talking about diagnostic settings.
- 1:31Should we talk about diagnostic settings and Azure policies and
- 1:35monitoring things in Asher ?
- 1:40Joey said if I were a professional wrestler,
- 1:42the button clicker would be my nickname.
- 1:45I wonder how that would go in professional wrestling.
- 1:47Like I feel like there could be some, I don't know,
- 1:50some unique inferences drawn from a wrestler called the button Clicker.
- 1:55Yeah, I, I don't know. , I mean Stone Cold had the stunner. You know,
- 1:59we'd have to think about what your move is gonna be.
- 2:01I can't think of anything that's uh, safe for work .
- 2:05I was just thinking, I was like, yeah,
- 2:06we're just not even gonna go where that secret special button clicker
- 2:11move. What? That would be Okay, enough with button clicking.
- 2:16Do you wanna talk about diagnostic settings and Azure policies and initiatives
- 2:20today? Scott?
- 2:20So you've been doing some fun stuff here and there were a couple things
- 2:25we were chatting earlier this week to kind of prepare. I, there's,
- 2:29there's like some new stuff in just initiatives and policies when
- 2:34it comes to discovery that like I wasn't aware of.
- 2:37So I think there's some cool new things kind of going on out there.
- 2:41And I run into this one a lot.
- 2:43I see a lot of customers who want
- 2:48to take action in their environments.
- 2:50Like they have some type of proactive action,
- 2:53they want to take on a bunch of resources at once and
- 2:58they don't always know that policy exists or they don't know that these types of
- 3:03settings are available to them and they potentially run into issues in
- 3:08other places. Like practical example, I,
- 3:11I was having a conversation with somebody and they have hundreds of storage
- 3:15accounts and these storage accounts are managed in
- 3:20Terraform. Okay great. Like we've got this whole ecosystem for doing that.
- 3:25But one of the things that they run into is the way the Terraform
- 3:30provider for Azure is currently written.
- 3:33It's not too great when it encounters throttling in the management plane.
- 3:38So one of the considerations for customers is like, hey,
- 3:41I have N resources that I need to manage.
- 3:45And one of the things that you need to think about is the types of operations
- 3:49that you fire off against ARM and the specific resource providers.
- 3:53'cause there's like limits for Arm and then there could be other specific limits
- 3:57within a resource provider like compute, storage, networking,
- 4:01all those kinds of things. And sometimes this external tooling,
- 4:05like it just doesn't deal too well with say like an HT P 4 29 and
- 4:10you should just kind of hang out and retry this request a little bit later.
- 4:13Like if you're trying to manage say a thousand resources in your subscription
- 4:18and you know, whatever the limit is,
- 4:20it's like oh you can make 500 read requests or 800 read requests every five
- 4:25minutes. You can't even do the read requests for the thousand resources at once,
- 4:30right?
- 4:30Like you have to have these long running processes that kind of come back and
- 4:34figure these things out.
- 4:35So I've been kind of trying to have conversations with folks and and steer them
- 4:40in different directions like, hey,
- 4:42like Terraform might be really good for what you want it to do here,
- 4:45but it's only really good when you have 10 of these things.
- 4:47It might not be good when you have hundreds or thousands because of these
- 4:52constraints. And let's be clear, like there are constraints, right?
- 4:56There are limits that are that are there in arm and you know,
- 5:00maybe there's a different way for you to do it and you know,
- 5:03maybe you do think about things like policy instead of having all these things
- 5:08managed in like Terraform Plan and policy can just run remediation tasks
- 5:13in the background. And it's not that it's performing different activities,
- 5:17it's performing different activities at the same scale.
- 5:20Like hey I need to manage the same number of resources but in a slightly
- 5:25different way. Like listing can be different.
- 5:28Even like application of policy, like practical effect. Like okay,
- 5:33I have a policy that's in an initiative that you know,
- 5:36potentially alters a bunch of resources with like D N E employee if not exists
- 5:40kinds of configurations on them.
- 5:42So I thought it was a good one to talk about 'cause I didn't even realize that
- 5:47maybe Microsoft had started to publish some guidance out here.
- 5:51Like you pointed me to this nifty article about creating diagnostic settings at
- 5:55scale using Azure policy and initiatives.
- 5:59It was interesting,
- 6:01like I found that and then when we were talking about this earlier,
- 6:04you also pointed me to the GitHub repo.
- 6:06But kinda where this started coming up for me was
- 6:11a client we're working on, and it ties into our conversation last week,
- 6:16two weeks ago, I'm trying to remember where we started talking about Sentinel.
- 6:19The defender stuff. Yeah.
- 6:21Yes. Defender Sentinel. And for me it was,
- 6:26hey we have all these resources out there already.
- 6:30Now we wanna start onboarding Sentinel and we wanna start pushing
- 6:35all these logs into Sentinel.
- 6:37And really Sentinel relies very heavily for a lot of these
- 6:42resources on these policies and on the well not on policies,
- 6:46on diagnostic settings coming from these resources.
- 6:49And when you actually go into Sentinel and you go into what's now the content
- 6:53hub, new feature in Sentinel, instead of going into connectors,
- 6:57you deploy these solutions. They include all the different connectors,
- 7:02workbooks, analytics rules, all of that. But a lot of these different resources,
- 7:07storage accounts, SQL databases, app services,
- 7:12when you go in and click on the connector,
- 7:15really all the connector is for all of these is just go set this Azure
- 7:19policy to go enable this diagnostic logging and send it to the log
- 7:24analytics workspace that is associated with
- 7:29your Sentinel instance or the di the diagnostic,
- 7:33the log analytics workspace Sentinel is installed into.
- 7:37'cause really Sentinel isn't its own resource.
- 7:39It doesn't even show up in a resource group,
- 7:41it just gets added into log analytics.
- 7:44So that's where I kinda started with all these policies was, okay,
- 7:48I have all these storage accounts, I have app services,
- 7:52I have SQL servers and the button click reference.
- 7:55I don't want to go click through every single one of these and
- 8:00go set up policies for all of the,
- 8:03or go manually enable diagnostic logging.
- 8:07This ended up leading me down a completely different rabbit hole.
- 8:10But these policy definitions in this PowerShell script you I pointed you to
- 8:15that we came up with was really out of this.
- 8:20I went and installed all these connectors that were out of the box but
- 8:25I ran across a lot of resources in Azure that don't
- 8:30necessarily have a built-in policy for diagnostic logging.
- 8:34And I wanted policies for some of these other resources.
- 8:39And this article kind of points you to,
- 8:44there's this PowerShell script out there and if you
- 8:49have the AZ module, the Azure module for PowerShell installed,
- 8:54you can go install this script and you really run a very basic
- 8:59command.
- 8:59And this is where I should actually have the PowerShell script in front of me
- 9:03and I have lost it. You install the script,
- 9:07create AZ diag policy and once you install this script you
- 9:12can go run create AZ policy PSS one and add a parameter onto
- 9:17it. That's export la.
- 9:19If you want a policy that exports these diagnostic policies into log analytics,
- 9:23you can do a export eh for event hub export directory to
- 9:28export them to or no export directories where you want these policies
- 9:33to get exported to.
- 9:35And you can add additional parameters in here if you wanna point it to a
- 9:38specific subscription id, a specific resource type, all of that.
- 9:43Or the other option is that's really nifty is you go just do create az diag
- 9:48policy.ps one dash export la which is what I did.
- 9:52And then it actually enumerates all of your subscriptions that you have access
- 9:57to.
- 9:57You obviously have to log into Azure and it says go type in number one through
- 10:0223 for each one of your subscriptions.
- 10:04So it gives you a whole numbered list of subscriptions.
- 10:08So instead of having to find the subscription,
- 10:10ID copy and paste that you just type in like 16, that's a subscription I want.
- 10:14After you type in the subscription,
- 10:16it goes in and enumerates all of your resources within that
- 10:21subscription and you get the same type of thing,
- 10:23a very numerical list of one through 50,
- 10:28one through 75.
- 10:29You obviously have to make sure your buffer's big enough on your command line
- 10:32that you can see everything,
- 10:33but it's all of the different resource types that are in that
- 10:38particular subscription and same type of thing. You hit, I don't know,
- 10:42resource type number five for Azure data factory or something.
- 10:45And it goes through and look figures out,
- 10:49looks at the arm template, I don't know what it actually looks at,
- 10:53but says here's all the diagnostic settings that are on this
- 10:58particular resource type and spits out the J s O files to define the
- 11:02Azure policy for that specific resource type truth be told,
- 11:07you can go grab all that Jason and just copy and paste it right into creating
- 11:12new Azure policy. And it works. I actually found it was a little,
- 11:17I did not like what it generated,
- 11:19it generated some extra parameters like every single one.
- 11:23One of the parameters was the location that you wanna limit creating resources
- 11:27in. And I was like, this is policy, I don't, I want diagnostic logs,
- 11:31I don't care about the location. I.
- 11:33Gotta stop you for a second. Okay. So I didn't realize I, oh my gosh,
- 11:36like this was one of the first things I worked on when I joined Azure Storage.
- 11:42So I went and looked at the GI repo for this project. Yeah, I'm looking at who,
- 11:46who wrote it. So the gentleman's name is Jim Britt, he's awesome.
- 11:50But basically what this thing does is it reaches out to the
- 11:56arm APIs and then it performs dynamic
- 12:00discovery of what diagnostic settings are available on that
- 12:05resource.
- 12:06So one of the things that I worked with him on on this particular project was
- 12:11there are resource types in Azure that present
- 12:16via arm that aren't all the way arm resources . So,
- 12:21uh, one of the biggest areas like uh, that I'm aware of is storage.
- 12:26So in storage we have this concept of proxy services.
- 12:29So when you create a storage account, a storage account has blob,
- 12:33it has d f s files, tables queues, and uh, a web endpoint, right?
- 12:38Like every single storage account has those endpoints.
- 12:41If you've ever actually like dug into arm like and these diagnostic
- 12:46settings, when you deploy a diagnostic setting for a storage account,
- 12:50in many cases you're not actually deploying it at the parent storage account
- 12:54resource level, you're deploying it at the proxy service.
- 12:57So what happens is you end up with, you know,
- 13:00an arm management u R I that goes like, you know,
- 13:03forward slash resource groups resource name blah blah blah,
- 13:07Microsoft storage and your account name.
- 13:09And then if you wanna do blob diagnostics you have to append forward slash
- 13:14default services forward slash blob if you wanna work with files forward slash
- 13:17default services forward slash files. And it's,
- 13:20it's weird because you can't perform this dynamic discovery because like ARM
- 13:25isn't really aware of the proxy resource which is is the challenge,
- 13:29which is why you don't see like your blob endpoint in resource graph.
- 13:34You only see your parent storage account basically like you see like the parent
- 13:39resource but not any child or proxy resources that are after it. Like oh man,
- 13:43this is bringing back nightmares for me.
- 13:45I remember this is a gnarly problem .
- 13:48So yeah with that and same thing like the policy it spit out, it's great,
- 13:51it gives you a lot of details and for me the biggest thing that it gave me
- 13:57was all the different diagnostic settings that you could do.
- 14:01So what I actually did is I used this to kind of enumerate what are those
- 14:06properties behind the scenes for the different check boxes that you check in
- 14:11diagnostic logs.
- 14:12I want to monitor antivirus or H T T P calls or whatever that is.
- 14:16And I kind of took what this spit out along with going and looking at
- 14:21the definition of the default ones that Microsoft provided,
- 14:24like going and looking at an app services one or something and kind of merging
- 14:28the two of 'em together.
- 14:28Just doing some quick copy and paste to make some modifications.
- 14:33I tweaked a few of the rules in terms of when that particular policy would
- 14:38apply.
- 14:39Specifically when you get into things like app services where technically a
- 14:44function is an app service but you don't want the same policy applied to a
- 14:48function that you maybe wanna apply to an app service. So you have to,
- 14:50to put some of that logic in there to help determine when
- 14:55you have a function and the policy should apply versus when you have a web
- 15:00app that the policy should apply to. Yeah,
- 15:02but this at least for me was really valuable and being able to
- 15:07pull out different properties around diagnostic logging that I wanted to
- 15:12include in these custom policies I started building. It's.
- 15:15A weird problem to solve. I don't Did you have a chance when you,
- 15:19so when you did this, like there's kind of two paths you can take.
- 15:21So one is take the policies like the definitions,
- 15:26parameter files, everything Yep.
- 15:28That create AZ diag policy spits out and then you can
- 15:33go off and apply that however you'd like, like you said,
- 15:35like tweak it and do all that. There's actually, uh,
- 15:38the other part that goes along with this repo is there's scripts in there to
- 15:43trigger a policy evaluation and also trigger initiative remediations.
- 15:48Like did you have a chance to play around with those at all or did you just go
- 15:51down the other path?
- 15:52Did not.
- 15:52I just went down the other path and I don't know if you wanna go down the path
- 15:56of an issue I ran into with applying policies yet and remediation of these.
- 16:01It's.
- 16:01Your show, we can go wherever we want.
- 16:03No, I already told you that earlier that it was your show .
- 16:06So I did it just for creating policies and this was a little bit of a weird
- 16:11scenario. So I'm curious Scott, have you ever, when you've done policies,
- 16:16not had anything but read access to the
- 16:21subscription and just had contribute access on the
- 16:26resource group and tried to do all of the policy stuff
- 16:32at the resource group level?
- 16:34No, I'm a spoiled Brad. Every time I play around with policy,
- 16:37it's in some place where I'm effectively a deity at subscription.
- 16:41You know what? Let me tell you that makes it a whole lot easier .
- 16:46Turns out.
- 16:47Turns out. So there's a couple different things I ran into with this.
- 16:52I don't know which path I want to go on first. Let's go,
- 16:54let's keep going down this path since this is the one I started on.
- 16:58Turns out if you're a contributor you can't assign
- 17:03policies at the resource group level.
- 17:06You need to have an additional permission and it'll tell you,
- 17:11I should have pulled up the article, I'll have to dig through it,
- 17:14we can throw it in the show notes.
- 17:15There's an additional permission level you need to be
- 17:20able to go in and actually create policies.
- 17:23And it's actually at the resource group,
- 17:26I think it was at the resource group level.
- 17:28I had to have them go in and grant me these additional permissions and then I
- 17:32was actually able to create policies that solved. My first problem was,
- 17:36okay go grant me this permission level. Great, that's done.
- 17:39I can go create policies. So then I went in and created a remediation task.
- 17:44Guess what? Remediation tasks require managed identities.
- 17:49Uh, yes. And guess what you can't create in a resource group .
- 17:54It's a tough one. Like with, with all of these, right?
- 17:56Because the kind of boxed roles I think make some assumptions and
- 18:01sometimes to make like overly broad assumptions, right? Like hey,
- 18:04we can potentially give you a little bit more than we when you need because we
- 18:08really do want you to be successful.
- 18:10So there's this weird line between lock it down too much and make it so hard
- 18:13nobody can do it and making it so easy that everybody can do it.
- 18:17Like maybe there's gotta be a little bit of friction in there.
- 18:20But policies an interesting one.
- 18:21Like if you ever go and dig around in like Microsoft policy insights,
- 18:27there's everything from management actions to data actions.
- 18:31'cause of course you have data plan operations as well as management plane kind.
- 18:35I like storage.
- 18:36It's another little bit of a confusing one to wrap your head around the first
- 18:40time if you're customizing it or trying to get into that. That's.
- 18:43What I was doing as I was trying to create these policies.
- 18:45I wanted remediation tasks so I could actually leverage policies to go in and
- 18:49enable diagnostic logging because I had permission to do diagnostic logging and
- 18:54to go enable it.
- 18:55I had permission to create policies but because those policies required
- 19:00the management or the managed identity,
- 19:04I didn't have permissions to create managed identity.
- 19:06So I couldn't do a remediation task to turn on diagnostic logging.
- 19:11The other thing kind of along these lines that I've found,
- 19:16and this is another interesting thing with policies,
- 19:19is if you've gone in and used these out of the box policies to go in and enable
- 19:23diagnostic logging,
- 19:24these policies have a parameter that defines the name
- 19:29of the diagnostic log that it wants to create.
- 19:33It's by default it's set.
- 19:34So it's a hidden parameter when you go in and create policies.
- 19:38And if you create, and let's say this scenario we were talking about,
- 19:42maybe we have two different policies.
- 19:44One to go to this log analytics account for app insights
- 19:49and for kind of the developers to use.
- 19:51But then another log analytics for all the sentinel stuff to go into where
- 19:55sentinel's kind of isolated. But if you go in and apply the same policy,
- 20:00they start fighting over each other because now you're trying to
- 20:05deploy the same policy with the same name on the same resource group to go to
- 20:10two different log analytics workspaces.
- 20:12So what I started doing with my custom policies is actually unh.
- 20:17And you could do this one of two ways.
- 20:18You could go in and customize the out of the box policy so that that's not set
- 20:23to a default invisible and it makes you go in and define a value for the name
- 20:29or you can actually go in and click the checkbox to show
- 20:34all parameters, not just the required parameters.
- 20:37And then you go in and change the name and then if you deploy a policy
- 20:43one of these outta the box policies twice, but give it two different names,
- 20:46it'll actually set up that logging and diagnostic storage to log analytics
- 20:51workspace.
- 20:51It'll set up or you can set it up to set up both of them so they don't start
- 20:56fighting over each other really.
- 20:58They can even end up overriding each other based on the order these policies get
- 21:02evaluated. So one will be compliant, one will be not compliant,
- 21:06it'll go in and fix it,
- 21:07the other one goes non-compliant and they just keep fighting back and forth
- 21:10because these diagnostic setting policies or these diagnostic
- 21:15logging names set by policy have the same name. I.
- 21:19Wonder how much of that is like just by design, like it's not,
- 21:23it's meant to be purposely obfuscated,
- 21:25like the scenario that you're in.
- 21:29Like sure there's the functionality and like day-to-day aspect of it. Yep.
- 21:33And then there's the whole like just rationalizing cost piece of it.
- 21:38Like you're targeting multiple log analytics workspaces with the same
- 21:43dataset.
- 21:44Like there's a penalty there for paying
- 21:49to store those two times.
- 21:51Yeah,
- 21:51and it's an interesting discussion and we started talking about this the other
- 21:55day too, is how do you,
- 22:00how do you manage this? So
- 22:03I think there's an article out there that's in preview around
- 22:08table level permissions for log analytics,
- 22:13but you also start thinking like your developers want some of this diagnostic
- 22:17logging,
- 22:18but then from a security perspective you don't really want your
- 22:22developers maybe seeing all of the Azure activity because Azure activity
- 22:28is a subscription.
- 22:30You need subscription owner level access to deploy the Azure
- 22:35activity policy to ingest all of that data into Sentinel.
- 22:40Maybe you have things like Microsoft Defender for cloud coming into Sentinel
- 22:45and you're absolutely right in that there is definitely data getting ingested
- 22:50into log analytics that you end up paying for them twice.
- 22:52But how do you set up log analytics and Sentinel in a singular
- 22:57workspace while limiting maybe what
- 23:01developers can get to or what people that are doing some app
- 23:06insight stuff, what they can get to,
- 23:09to architect it in a way that people only have access to what they need without
- 23:14paying for duplicate data within those log analytics environments.
- 23:18It's not an easy problem to solve .
- 23:22That's where some of this started coming up,
- 23:24at least for me was in this case in particular,
- 23:29and this is the other thing to think about, in this case in particular,
- 23:31they had a dev resource group,
- 23:33a QA resource group and a prod resource group.
- 23:37They already had log analytics set up, they had all their app insights,
- 23:40all the logging, all the devs were used to it.
- 23:42And then they came in and said we wanna deploy Sentinel and
- 23:47Defender. How should we do this? And it's like, well at this point in time now,
- 23:51do I deploy Sentinel to all those log analytics workspaces in each of the
- 23:56environments?
- 23:57Do I apply Sentinel at one outside of that?
- 24:03But then do like cross log analytics queries and do a bunch of joins and modify
- 24:07a bunch of the rules and the alert queries to kinda look at
- 24:12all these different log analytics workspaces.
- 24:15I know there's a preview coming with Sentinel too,
- 24:17where you can kind of have a management sentinel instance and then have all
- 24:21these sub sentinel instances, but it doesn't work in that.
- 24:25It kind of pulls everything in.
- 24:26It's more let's make sure we're deploying the same alerts and workbooks and
- 24:31all of that from this parent Sentinel instance to all these child sentinel
- 24:35instances managed. It just,
- 24:37it leads to a very interesting discussion about how do you architect this
- 24:42and make sure you are seeing all the relevant data,
- 24:46especially with something like Sentinel where you're maybe relying on some of
- 24:50those incident alerts and that type of stuff.
- 24:53That really does depend on a broad swath of data,
- 24:58uh, being within those scheduled queries that are running.
- 25:01It's tough. I I put a link in the chat and I'll have it in the show notes.
- 25:06So it's one thing that I think the Sentinel team did a
- 25:10good job on.
- 25:11Like I'm sure they encountered this early on with customers that had this split
- 25:16between single-tenant, multi-tenant access control.
- 25:20You mentioned like table level security,
- 25:23there's maybe considerations like as log analytics improves its capabilities
- 25:27over time for table level retention and, and what that gets in there.
- 25:32There's a whole bunch of moving pieces and I don't know that
- 25:37there's a whole bunch of good answers. ,
- 25:39like you're probably bound by reg, you know,
- 25:41just pure regulation and like regulatory compliance.
- 25:45Like thinking about like having to split data because you're doing business in
- 25:50the US and eu, right?
- 25:51Like so you're company that's doing the global international thing, like the,
- 25:56you're gonna have those kinds of drivers in there that are gonna
- 26:01push you one way or the other, but you're gonna pay for all of it. .
- 26:05Yes.
- 26:05Cost is, cost is gonna be a consideration for you. Which is,
- 26:09it's probably the roughest one of all these like I think you can given the
- 26:13capabilities in like custo today and all these data sources like resource graph
- 26:18monitor, so like Microsoft Insights, resource provider,
- 26:23Sentinel, like all these thing log analytics,
- 26:25like all of them being built on top of Custo as like an underlying data engine.
- 26:30Like it,
- 26:30it's been a lot easier with like cross cluster queries and all that kind of
- 26:34stuff.
- 26:34Like I do things today where a lot of the time I don't even go to the Azure
- 26:39portal anymore to query things like there's end points that you can hit. I,
- 26:43I forget what they're off the top of my head.
- 26:44It's like ad dot log analytics do us or something like that for,
- 26:49for like GovCloud for log analytics.
- 26:51But if you spin up like basically if you spin up a log analytics workspace,
- 26:55you can connect to that log analytics workspace just through default tooling
- 26:59like Custo explore and all that stuff from your desktop.
- 27:02So it makes it like a lot easier to frame your head around like the cross
- 27:06cluster query join and all those kinds of things. But it's,
- 27:10it's muddy.
- 27:11Yeah it really is.
- 27:15Do you feel overwhelmed by trying to manage your Office 365 environment?
- 27:19Are you facing unexpected issues that disrupt your company's productivity?
- 27:23Intelligent is here to help much like you take your car to the mechanic that has
- 27:27specialized knowledge on how to best keep your car running Intelligent helps you
- 27:31with your Microsoft cloud environment because that's their expertise.
- 27:35Intelligent keeps up with the latest updates in the Microsoft cloud to help keep
- 27:39your business running smoothly and ahead of the curve.
- 27:41Whether you are a small organization with just a few users up to an organization
- 27:46of several thousand employees they want to partner with you to implement and
- 27:50administer your Microsoft Cloud technology,
- 27:53visit them at intelligent.com/podcast.
- 27:57That's I N T E L L I G I N
- 28:01k.com/podcast for more information or to schedule a 30 minute
- 28:06call to get started with them today.
- 28:09Remember intelligent focuses on the Microsoft cloud so you can focus on your
- 28:13business. Ken,
- 28:16to your point we started talking about this is could you set up Sentinel in a
- 28:20way where if you went in and started modifying some of those built-in
- 28:25queries and set all that up is could you set it up in a way that you could take
- 28:29advantage of all those other log analytics instances and start doing those
- 28:35cross log analytics queries within your sentinel rules and your sentinel alerts
- 28:40to set it up in a way where you aren't paying for that data twice and you're
- 28:44able to figure out how to ingest it from all these different places.
- 28:48I think then the trick also comes in is
- 28:52going back to the policy thing.
- 28:54It's easy to set up policies that ingest everything into one resource group.
- 28:59Like you go put it at the tenant root group, right?
- 29:01And maybe you say all of my storage accounts or if you're depending on how
- 29:05you're doing your management groups, you have it set up.
- 29:08But now if you start doing different log analytics for various subscriptions or
- 29:12resource groups,
- 29:14how do you know when a new log analytics instance gets spun up that you
- 29:18need to go now add to all these cross log analytics
- 29:24queries.
- 29:24This feels like it could be a whole project in and of itself that I don't have
- 29:27time for .
- 29:30Yeah, it takes you back down, it's back down the rabbit hole. Yeah.
- 29:34The other thing that I found out, this is a little, it's still policy related.
- 29:38The other thing I found out in going through all of this was,
- 29:42okay I can't set up policies without subscription level
- 29:47permissions is do I set up these policies in a way
- 29:53that now I can just report on, get a compliance report,
- 29:58do I have this particular policy added to all
- 30:03of these different resources?
- 30:05Are they sending to the right log analytics workspace for Sentinel?
- 30:09Do they have the right name?
- 30:11All of this stuff because now I am doing some click, click, click.
- 30:15It wasn't enough resources that I went in and scripted it all out but I wanted
- 30:19to make sure everything was set up the right way.
- 30:21Come to find out while doing the deploy,
- 30:24if not exist works great if you change the name.
- 30:28If you just look for the right setting in the policy to report on compliance
- 30:32or not and you have multiple diagnostic policies in there,
- 30:37it doesn't look at all of them and it doesn't say okay this is one of the three
- 30:42in there so it's compliant.
- 30:44It like looks at the first one and says, uh,
- 30:48no that diagnostic setting isn't configured to go to the right resource group.
- 30:53So this resource is now not in non-compliant with your policy but when in fact
- 30:57it was in this case it was the third one in the list of where these
- 31:02diagnostic settings were going.
- 31:04So it was compliant but these policies weren't able to evaluate
- 31:09all of those settings on those particular resources. Yeah.
- 31:13I wonder if you can back to the resource graph thing if you can make your life a
- 31:16little easier. So policy,
- 31:19so things in Microsoft dot policy insights, like that resource type,
- 31:24they bubble up into resource graph and you can actually see compliant and
- 31:28noncompliance state in there.
- 31:30So I wonder if you could craft 'cause 'cause you can.
- 31:33So the other thing you can do is resource graph queries can be fired off via
- 31:37like C L I or PowerShell and then you can bring 'em back and parse 'em however
- 31:40you want.
- 31:41So I wonder if you could have this kind of like hodgepodge of like say like
- 31:44Azure PowerShell fire off a call to resource graph for this query and then based
- 31:49on that query output,
- 31:50figure out like where the deficiencies are and then actually go query those
- 31:54resources natively using like Azure,
- 31:57PowerShell diagnostics commandlets and then figure out like what the true
- 32:02and and meaningful state of them is.
- 32:04Yeah, I may have to go explore some of that. And again,
- 32:08this whole working through all of this in this particular case was just
- 32:14fascinating because like you,
- 32:15I've always been at that level before where it's like we want you to come in and
- 32:19help us with all this stuff here. We'll just make you a subscription owner,
- 32:22a subscription,
- 32:23even a subscription contributor so you can actually see some of this stuff.
- 32:27And I,
- 32:28I felt like I was very much working with one hand tied behind my back in this
- 32:32case in terms of what I was able to get to, what I was able to see,
- 32:36what I was able to access.
- 32:38Like in this case I even saw places these diagnostic logs were getting sent to.
- 32:43I didn't know where they were going or what they were for.
- 32:45Like there were these settings and it was going off somewhere.
- 32:49So I was like okay, I have to be careful not to mess with these because ,
- 32:53if I delete one,
- 32:54I can't even go in and fix it because I don't have access to the log analytics
- 32:58workspace it's going to or wherever it was getting shipped off to to go set it
- 33:02back up again.
- 33:03It's the joy of being a consultant sometimes I can't say I miss those days where
- 33:08like you ,
- 33:08I I that that struggle of walking into a customer and saying like, hey uh,
- 33:13we've got the s,
- 33:14you've signed it off like and I'm ready to do the work by the way I
- 33:19need you to do this,
- 33:20this and this and this for me because you didn't give me the ability to do it
- 33:24myself. And then they kind of come back and look at you and they go, wait,
- 33:27I'm paying you for what? ,
- 33:30oh you're paying me 'cause you made it harder to .
- 33:33Yeah.
- 33:33To get out along the way here. So.
- 33:35It's been interesting.
- 33:36There's a whole nother rabbit hole we could go down to around the deploy if not
- 33:40exists.
- 33:41But we should probably wrap it up unless you want me to bring up this last one.
- 33:45Is.
- 33:45It a quick one? Because we can,
- 33:46we can actually like close out the whole thing at.
- 33:48Once. Okay. So let's close out the whole thing with this.
- 33:50So here's the last interesting thing I found. We've talked all about policies,
- 33:55right? And deploying these diagnostic settings.
- 33:58What if you wanna make these an initiative and you don't want to go in and apply
- 34:0310 policies and
- 34:07go set deploy if not exist and you wanna wrap it up as one initiative that
- 34:12has all your policies for all your diagnostic settings but you also need a
- 34:16deploy if not exists on those. It gets really you.
- 34:19Struggle a little bit uh.
- 34:20You do.
- 34:22'cause you can only set one deploy if not exist when you set up the initiative.
- 34:27So you kind of guess which one it is because the other thing that ended up,
- 34:31and I haven't gone back in to figure out if there's a way around this,
- 34:34all the deploy if not exist for policies have the exact same name in it.
- 34:38So you like put 10 of these policies in there,
- 34:41go to deploy if not exist and click the dropdown for what you wanna do and you
- 34:45get a list of 10 things all with the exact same name ,
- 34:48you pick one and you deploy it after the fact.
- 34:51You can go in and start going through this list and you can actually add all the
- 34:56rest of the deploy if not exists.
- 34:58So there is a way to do it through that policy or through that initiative.
- 35:03But it's the same thing where now as you're going through and like adding
- 35:07another remediation, add this deploy if not exist, add this one, add this one,
- 35:11add this one. Yeah. Trying to keep track of which ones you've deployed.
- 35:16I don't know like if you could go into the policy definitions for all of these
- 35:20create custom ones. I think you can,
- 35:22you could go in and actually rename those deploy if not exist to
- 35:27something meaningful within these policies. You.
- 35:30Should be able to, like the D N A task is like that's fixed.
- 35:34That's part of the schema and the, and the j ss o n for the definition,
- 35:37but the name should be fungible.
- 35:39Yeah for that particular like d n a task inside that given like
- 35:43definition, but Right. It's not the easiest thing to like figure out .
- 35:48Yeah.
- 35:49'cause now you're going in and instead of just using all these policies that are
- 35:52built in, you're having to go in and create custom versions of all of these.
- 35:55Again, big enough environment,
- 35:57it might be worth it to go in and figure all of those out.
- 36:01But there are definitely some challenges that I have run
- 36:06into in this case when it comes to policies and initiatives and diagnostic
- 36:11settings and permissions and duplicate policies.
- 36:14But it's been interesting to dive into some of these policies specifically
- 36:19around diagnostic settings this much.
- 36:21Have you ever seen or tried,
- 36:25there's a project out there, Julian Hayward maintains.
- 36:29So he did some of the uh,
- 36:31arm visualization stuff that I made initially eventually made its way into Lake
- 36:35vs code and all that. But there's a project out there that's called the,
- 36:40it's, it's basically the Azure governance visualizer.
- 36:44So it's back to PowerShell again.
- 36:46It's a PowerShell script that you can run and it
- 36:52spits out some HT m l it can also do like j ss, o N and C S V,
- 36:57all that kind of stuff. But depending on how like you wanna visualize it,
- 37:00it actually goes and enumerates a given subscription or set
- 37:05of subscriptions.
- 37:05Like it can start at a management group scope and then walk that hierarchy all
- 37:10the way down to the bottom.
- 37:11So policy from the MG policy at the subscription group,
- 37:15scope pol at the subscription id, uh,
- 37:18subscription scope rather like at the subscription ID resource group.
- 37:21Like all that kind of stuff. So you just kind of walk it top to bottom.
- 37:25And one of the cool things that it does, like, and,
- 37:28and this is where it might be interesting to you, is that it can, uh,
- 37:33it has a component called Definition Insights.
- 37:36So it actually walks like all the way down to that definition level and it tells
- 37:40you how things are applicable, how they're being configured.
- 37:43Like you might wanna check that one out.
- 37:44I'm gonna have to go check this out, Scott. It's on my list.
- 37:48I have not seen this before.
- 37:49All sorts of fun stuff out there if you know where to look.
- 37:52Yes, absolutely. And the other thing,
- 37:55so to wrap it up also with resources, you also pointed me to another resource.
- 37:59We talked earlier about that PowerShell script and me going in and creating a
- 38:02bunch of my own policies.
- 38:03And then you pointed me to this GitHub repo with a whole bunch of policy
- 38:07definitions for monitoring that has a bunch of the ones in there that I already
- 38:10created again. Yeah, I just did not know to look in the right spot.
- 38:15Two different ways to get the same results.
- 38:18The PowerShell was interesting for me just to gimme insight into it.
- 38:20But this is also a very handy GitHub repo to be
- 38:25aware of if you're working on policy definitions for.
- 38:28Anyone who's made it this far. Pro tip,
- 38:29this thing is called the Azure Community Policy
- 38:34repo. It's a GitHub repo. It's, it's out under the azure org. I'll,
- 38:38I'll put a link in the show notes.
- 38:40But if you've ever looked at an Azure resource,
- 38:44like you're back at that resource type level and you're looking at the built-in
- 38:48definitions for policy and you're going like, huh,
- 38:52it really feels like it should be able to do
- 38:57this. Like this seems like a common thing that lots of people would ask for.
- 39:01Like why doesn't a built-in definition do that, do this yet?
- 39:05It could be that we actually wrote the definition and it just
- 39:09hasn't made its way all the way over to like be an actual default definition in
- 39:14the system yet. So I know for like me personally,
- 39:17like back to that stuff that I did with Jim Britt,
- 39:20like in the way back when I was pushing a lot of my policies that I was writing
- 39:24for him around the proxy services over to the community policy repo because
- 39:29it's a super quick, it's not even like a quick and dirty,
- 39:32it's just a super quick way for me to get policies out there that I know are
- 39:37gonna be helpful to a bunch of people while then we go work with service teams
- 39:41to get 'em integrated in and,
- 39:43and where they need to be and all that good kind of stuff.
- 39:46So there's a whole bunch of services that are covered in that
- 39:51repo, like it's not just Azure Monitor, so things like diagnostics, storage,
- 39:55cosmos, db data lake, virtual machines, security centers in there,
- 40:00synapse is in there, tags, key vault, Kubernetes, all that could kind of stuff.
- 40:04So I'll,
- 40:04I'll put a link to that in the show notes as well 'cause it is
- 40:09a great and wonderful resource. Like usually my step is,
- 40:13is there a built-in definition like or a built-in policy? Oh no,
- 40:17that doesn't exist off to community policy.
- 40:19And then if it's not in community policy, go and create it myself.
- 40:23Yes. Very cool. Well thanks Scott. Lots of good resources,
- 40:27lots of interesting stuff about policies and
- 40:32things to think about, but with that indeed.
- 40:36Always things to think about.
- 40:38I should go back to my midlife crisis,
- 40:40just debates of what I do with the rest of my life.
- 40:43What do I wanna do when I grow up? Scott? Nothing.
- 40:45When you figure it out,
- 40:46come back and tell me what I wanna do when I grow up though.
- 40:49Alright,
- 40:49we just need to get like a thousand more members to the podcast and then
- 40:55we can, no, not a thousand.
- 40:56We need like 10,000 more members to the podcast and then we can retire.
- 41:00We're we're.
- 41:01We're not gonna get there. Yeah, no, those think,
- 41:04think unless those that are out there think we make money off membership.
- 41:07We don't make many money off membership.
- 41:09We lose money on this sucker website hosting and editing and things every week
- 41:13is, uh, not an inconsequential amount, but yes, it's for the community.
- 41:19'cause Ben's an M V P. I'm not an M V P, so I just do fun.
- 41:22But you just love community. Scott has a big heart for the community. ,
- 41:26Sean says,
- 41:26I better consult Pirate says to monetize YouTube and Joey says to become a pro
- 41:30wrestler that is quite a wide swath of
- 41:36future endeavors for Ben. But.
- 41:38I feel like if you did that,
- 41:39like you would be living that content creator lifestyle, right?
- 41:42Like every YouTuber that you like,
- 41:44not all of them are making you know enough to live off every year. Like some,
- 41:47some could be your barista at Starbucks,
- 41:49but in your case you can go be a wrestler. It all works out.
- 41:53There you go. I could monetize my YouTube or monetize on YouTube.
- 41:57My wrestling bouts as been the button pusher . Oh man,
- 42:02we gotta go, Scott, before.
- 42:03This on that note.
- 42:04Yeah, on that note, let's call it a day. .
- 42:09Uh, thanks bud. All right.
- 42:11Thanks Scott. If you enjoyed the podcast,
- 42:15go leave us a five star rating in iTunes.
- 42:17It helps to get the word out so more it pros can learn about Office 365 and
- 42:22Azure.
- 42:23If you have any questions you want us to address on the show or feedback about
- 42:27the show, feel free to reach out via our website, Twitter, or Facebook.
- 42:32Thanks again for listening and have a great day.