Latest / Microsoft Cloud IT Pro Podcast / Episode 377 – Microsoft Copilot for Security
Transcript
- 0:03- Welcome to episode 377
- 0:05of the Microsoft Cloud IT Pro podcast
- 0:08recorded live on May 21st, 2024.
- 0:12This is a show about Microsoft 365
- 0:14and Azure from the perspective of it pros
- 0:16and end users where we discuss a topic for recent news
- 0:19and how it relates to you.
- 0:21This week, Ben and Scott discussed a recent Google Cloud
- 0:24event where a customer account
- 0:26and all of their data was completely wiped out without
- 0:29notice and we share some thoughts we have
- 0:32around customers protecting their cloud deployments.
- 0:35We also have some updates
- 0:36and thoughts around co-pilot for security
- 0:39after Ben has been able
- 0:40to get some hands-on experience with it.
- 0:42We talk about pricing, approaching the experience
- 0:45and how to think about leveraging it in your environment.
- 0:50We've had a bit of a chaotic week, so we're gonna get
- 0:52through today, but I should go bring up this article.
- 0:56This was an interesting one
- 0:57and we don't like throwing other cloud providers under the
- 1:00bus because, oh what am I doing?
- 1:02Because stuff happens.
- 1:03Microsoft has stuff happens, Google has stuff happen
- 1:06and Amazon has stuff happen.
- 1:08But this was just, there's some thoughts that I had
- 1:11that came out of this news article that I ran across.
- 1:16I actually had a friend that sent this to me the other day
- 1:18and I'm curious to see what you think Scott.
- 1:20So this was on ours, Technica,
- 1:22and you can actually go read this article as well on
- 1:26the company's website that this happened to.
- 1:28But the news headline is unprecedented.
- 1:31Google Cloud event wipes out customer account
- 1:35and it's backup and then it's UniSuper,
- 1:38I think it's UniSuper as how you pronounce it, UniSuper,
- 1:41UniSuper, UniSuper,
- 1:43$135 billion pension account details,
- 1:47its Cloud compute nightmare.
- 1:49- This is a rough one. Reading
- 1:50- Through this essentially sounds like
- 1:54there was some configuration change, something
- 1:59that happened as Google was standing up the private cloud
- 2:03for this client and somehow
- 2:07like they wouldn't have been standing up a brand new one
- 2:09because they already had a bunch of data up there,
- 2:11but it essentially wiped out UniSuper
- 2:15GCP account including all of its information in all
- 2:19of its backups that were stored in GCP.
- 2:22It's like they went in
- 2:24and just said Delete UniSuper from GCP .
- 2:28- Yeah, that's one way to think about it.
- 2:30So you sent this one over
- 2:31and I was kind of reading through it
- 2:32and trying to think about it in context.
- 2:35It really, I think it'll be interesting
- 2:37to see if an RCA ever comes out
- 2:39for this one like root cause on what actually happened here.
- 2:43Yeah. But from the outside looking in, what it looks like
- 2:49is this customer had an existing set of workloads
- 2:53and an existing account with A GCP and with Google Cloud
- 2:59and for some reason that existing account
- 3:03was deleted.
- 3:05I would be willing to bet like weird series
- 3:09of circumstances like that.
- 3:12It was something like really crazy like the
- 3:17customer's account got flagged for fraud
- 3:19because of something that was happening maybe due
- 3:22to like an automated fraud detection
- 3:23system or something like that.
- 3:25And then due to a series
- 3:26of errors in the fraud detection system,
- 3:29it just went completely off the rails
- 3:32and potentially took out their data.
- 3:35And the other things that sidecars alongside with, hey,
- 3:39this is a customer's account and weird things happen.
- 3:43So you hear about this all the time.
- 3:44I think like think about it in context maybe of like,
- 3:46let's take a step back from cloud provider
- 3:49and think about like social media providers.
- 3:51Like I don't know if you've ever been on a platform
- 3:53where you've had your account banned just unilaterally.
- 3:56Yep. Hey, this thing goes away and you can't do anything.
- 3:59You know, for me, I had my Google Ads account banned
- 4:03and I'm banned for life for Google ads
- 4:06and there's no remediation forum,
- 4:09there's no one for me to go talk to.
- 4:12This is a permanent decision
- 4:13and it's been made kind of thing.
- 4:15I was reading about one last week
- 4:17with a venture capitalist MG Siegler
- 4:20where his Instagram account got banned by meta.
- 4:23Same thing like hey, you've been banned unilaterally.
- 4:26There's nothing that you can do about it.
- 4:29In that case, they ha, you know, MG happened
- 4:32to have contacts at meta and can turn it back on.
- 4:34And in this case, this provider at UniSuper,
- 4:38so it's a superannuation fund in Australia,
- 4:41effectively think like 401k provider here in the
- 4:45United States, but government mandated
- 4:48retirement fund kind of thing.
- 4:50Like not a good look to you have it all go away
- 4:53and get blown up that way.
- 4:55So you quite often
- 4:57do hit these very weird code paths
- 5:00and things that you don't think you're, that you don't know
- 5:03that you were going to encounter until you encounter them.
- 5:06And unfortunately in this case it took down an entire
- 5:11customer along the way and not just like
- 5:14the entire customer.
- 5:15It took down their entire estate
- 5:18because they were so all in on Google
- 5:21and GCP for their workloads.
- 5:24Like so all in to the point
- 5:26where they've been quoted in joint press releases in the
- 5:30past of saying, Hey, we're all in on Google for their
- 5:35VMware engine, you know, their migration capabilities
- 5:39for getting us on-prem
- 5:40to the cloud, all these kinds of things.
- 5:42So what a not great week
- 5:45that company had , right?
- 5:47You know, be it from their CEO all the way down
- 5:50to their folks who were the ones who made the decision
- 5:53to go all in on Google
- 5:55and then for Google itself, like these are the kinds of
- 6:00things that follow you around as a company for a while.
- 6:04And I wouldn't be surprised if this one doesn't get more
- 6:07mainstream press or hasn't had more mainstream
- 6:09press beyond ours.
- 6:11I think ours Technico was just the first place
- 6:13that you had seen it and sent it over my way.
- 6:15- The part that blows me away about this
- 6:18and Google said this should never have happened,
- 6:21which is kind of apparent,
- 6:23but one, they said someone should never just
- 6:25be able to delete their account.
- 6:26But it was deleted to the point that in this joint statement
- 6:31from it was a joint statement from UniSuper
- 6:34and Google, both their CEOs
- 6:37or the Google cloud CEO was that
- 6:41UniSuper had a bunch of redundancy in place.
- 6:42They had two geographies to protect against outages
- 6:45and losses, but none of that protected them.
- 6:47What protected them was they had their backup
- 6:50in another cloud provider.
- 6:52It wasn't even like Google could go back to some backup
- 6:56that even they had internally in restore it.
- 6:59To your point about like this unilateral decision,
- 7:02I would've expected that Google would've had some
- 7:06customer account backup
- 7:07or protection in place, even behind the scenes to
- 7:11where maybe UniSuper couldn't recover their own account
- 7:13because it was all deleted and everything.
- 7:15But it sounds like Google couldn't even recover their
- 7:17account that they had
- 7:18to go stand up a brand new GCP instance
- 7:21and restore all of their servers, all
- 7:25of their data from a backup they had
- 7:27with another cloud provider.
- 7:29- It depends on how that data is stored.
- 7:33So I can absolutely see how such a thing would happen.
- 7:37You know, if you think about all the things
- 7:39that we've talked about in the past
- 7:40with your data in the cloud, when it comes
- 7:42to things like data ownership, you know,
- 7:44here in the United States, if you think about things like
- 7:47NIST standards for shredding hard drives, right?
- 7:50If I have, if I have my data on a hard drive at a provider,
- 7:52how does that provider shred my hard drive
- 7:54and ensure that I'm the only one who has access to my data?
- 7:57Those kinds of things. Yep.
- 7:58There are these very real kind of kill switches in place
- 8:02that effectively once the key, the primary key is gone
- 8:07and it's severed from the data, there's really no way
- 8:11to bring that relationship back.
- 8:13And quite often the hyperscalers are doing things like doing
- 8:17garbage collection on data pretty aggressively.
- 8:20You know, it's not a bank error in anybody's favor to retain
- 8:25terabytes, petabytes, potentially tens
- 8:28or hundreds of petabytes of backups for customers.
- 8:31And you know, just have those sitting around for weeks
- 8:34and weeks and weeks waiting for a customer to go, oops,
- 8:37I didn't really mean to delete that kind of thing.
- 8:39And especially if it's a a back
- 8:42to the whole like you know, how could this happen?
- 8:44You know, think about it, especially if it's a fraudulent
- 8:47workload, like you don't want
- 8:48that stuff on your system to begin with.
- 8:50True. So you potentially just nuke it from above
- 8:54and call it a day, especially if you're very, very sure that
- 8:59it is in fact data that should be nuked
- 9:02or a workload that should be nuked an account, uh,
- 9:05a billing account, things like that.
- 9:07Whatever it happens to be. So pretty unfortunate.
- 9:10More, more than pretty unfortunate. Very, very unfortunate.
- 9:14Good lesson though, in kind of thinking about
- 9:18the multi-cloud thing
- 9:20and DR in a multi-cloud world,
- 9:24how you think about positioning yourself
- 9:27with other providers as a customer, right?
- 9:31Like you might be all in on Google,
- 9:33but then you might leverage say like enter ID
- 9:36for your identity and as your security token service
- 9:40you could be all in on AWS,
- 9:43but you might leverage a component of Microsoft
- 9:46or Google for something in your workload.
- 9:50You know, there's a whole bunch of customers
- 9:52that do those kinds of things too.
- 9:54It's rough. Make sure you got backups right.
- 9:57That whole rule of three thing becomes,
- 10:00becomes pretty critical here.
- 10:01And not just do you have backups?
- 10:03This was another good lesson in even though they had
- 10:06backups, recovery was still forever.
- 10:10Like it wasn't about just having RPO
- 10:13RTOs were extremely elongated in this case.
- 10:17And if you think about it for a financial firm, that's kind
- 10:20of a super critical thing when you have money flowing in out
- 10:23in the case of something like this,
- 10:24which is a effectively a pension fund, pensioners who are in
- 10:28that fund, like you still need
- 10:30to get your payment right, right.
- 10:32To be able to to buy food and survive and,
- 10:35and all those kinds of things too.
- 10:37So just a bad situation all around. Yeah.
- 10:41And hopefully they find a way through.
- 10:44- It was out two weeks here.
- 10:46It was, it was May two was when it started
- 10:48and they full restoration of services on May 15.
- 10:52So it sounds like everything's back up now as
- 10:55of about a week ago.
- 10:56But yeah, being down for almost two weeks and I agree.
- 10:59I think the biggest part that one of the biggest
- 11:01takeaways from me even thinking about like my Office 365
- 11:05environment was that whole thing
- 11:07of having some of those backups.
- 11:08'cause you are, you hear so much about this
- 11:10and I've talked about this a little bit more recently too,
- 11:13of, oh well Microsoft has multiple data centers
- 11:15or multiple regions.
- 11:16I mean AWS Google, everybody has multiple data centers,
- 11:19multiple regions redundancies in place
- 11:22and some people in I would say six,
- 11:25seven years ago I prescribed
- 11:27to this a little bit more than I probably should have
- 11:29of those redundancies are gonna protect me.
- 11:32Like I don't need to have my own backups
- 11:34because Microsoft is building in so many different backups
- 11:37that why do I need to go pay for another one?
- 11:40This one went in and highlighted of, it's not common.
- 11:43I mean this is a one-off in Google's case.
- 11:46I can't say I've heard
- 11:47of any accounts in the Microsoft cloud
- 11:50where somebody's gone in and
- 11:51just everything's gotten deleted.
- 11:52But of having those backups to your point somewhere else so
- 11:56that if you are the one that finds yourself in one
- 11:59of those one-off scenarios, you can get your backups.
- 12:01Like I can't imagine a company this size if they didn't have
- 12:05those backups in another cloud,
- 12:07how much worse this could have been for them.
- 12:09- Yeah, definitely detrimental.
- 12:12You know the other thing to think about here is so, so
- 12:15you know you're kind of calling out,
- 12:18Hey do I have the backups?
- 12:19Do I have the backups? Like let me think about that.
- 12:21Like absolutely like think about backing up your data.
- 12:24But I think it's also critical
- 12:26and if you go read through, I'll,
- 12:28I'll put the link in the show notes to UniSuper kind
- 12:32of timeline of what happened
- 12:34and if you read through that timeline
- 12:35and how it goes, one of the things
- 12:37that potentially delayed them was also
- 12:41not just having the backups
- 12:42but having the configuration right
- 12:44and the ability to stand it all back up on that side.
- 12:48So for you, let's say for you as an M 365 subscriber Yep.
- 12:52Are you using things like the community tools
- 12:56for like M 365 DSE to back up the configuration
- 12:59of your tenants on a regular basis?
- 13:01Are you testing that you can stand up a new tenant
- 13:05with a similar configuration beyond just kind
- 13:10of the data pieces and the backup and restore bits there?
- 13:13- You're gonna catch me. I'm not doing that with mine.
- 13:15I have clients that I'm doing that
- 13:17for but I don't do it with mine.
- 13:18- So I think that's the other click stop
- 13:19that folks need to consider.
- 13:20Like we talk a lot about kind of uh, application recovery
- 13:24and having backups
- 13:25and user data is absolutely a critical piece of that.
- 13:30The other thing that comes into play here very much is
- 13:34recovery and configuration and,
- 13:35and all those kinds of things.
- 13:37So to the degree you can with the providers that you have
- 13:39and the systems that you stand up, really do think about
- 13:42that stuff holistically if it's within your wheelhouse
- 13:46- Configuration - And it, for some people it is
- 13:48for some people it isn't.
- 13:50You know, if you're out there and you're listening to this
- 13:52and you go like, oh I pay an MSP
- 13:54or something to do all this for me
- 13:56so I don't have to worry about it.
- 13:57Yeah, maybe go ask 'em some questions, right?
- 13:59Just make sure that you've got the warm fuzzies about
- 14:02what they're doing and how they're
- 14:06actually providing you value in cases like this.
- 14:09- Yes, I would highly encourage you
- 14:12to ask your MSPs questions about this type of stuff
- 14:15and it is, it's your value.
- 14:16And I will say like the clients I'm backing up
- 14:19configurations for it's, it makes sense.
- 14:21I think in my case, like for my tenant personally,
- 14:24if I lost my conditional access policies,
- 14:27I wouldn't really care App registrations.
- 14:31I mean some of those you think about too,
- 14:33like all my app registrations that are tied into Azure ad,
- 14:35if I lose Microsoft NID, it is always going
- 14:38to be Azure AD Scott.
- 14:39Either way if I would lose that
- 14:41and lose all my app registrations
- 14:43and then not be able to authenticate to some
- 14:45of my third party apps, like do I have backup credentials
- 14:48saved for the native logins
- 14:50for those versus just my SSO logins and
- 14:53and it was a a good callout.
- 14:55There's a lot of stuff besides just do I have my emails
- 14:59and my files to think about when you're in these cloud DR
- 15:03scenario situations, do
- 15:05- You even know what the stuff is?
- 15:06is an interesting one.
- 15:08So yeah, the other thing I often think about is if you are
- 15:13building and deploying software, how do you think about
- 15:18recovery within and standing up assets again if you have to
- 15:22around things like build pipelines and deployment pipelines.
- 15:25Uh, you know, so if you're using like GitHub actions,
- 15:29do you have that YAML save someplace?
- 15:32Like what happens if somebody comes in
- 15:34and nukes that repo right?
- 15:36And it just goes away one day, like
- 15:38how do you get over that and how do you do it?
- 15:40So you know, I think taking a step back, having that good
- 15:44holistic view of your entire estate, not only
- 15:48what resides in your estate
- 15:50but how that stuff was built, how it's configured
- 15:53becomes extremely important.
- 15:55And in some cases, you know, you can't automate your way out
- 15:59of a job when it comes to
- 16:00doing recovery with some of these things.
- 16:02But I think it's important to just understand kind of
- 16:05where those rough edges are
- 16:07and that you've accounted for 'em in your
- 16:08runbooks and all the other things.
- 16:10- Yes, a hundred percent.
- 16:15Do you feel overwhelmed by trying
- 16:17to manage your Office 365 environment?
- 16:19Are you facing unexpected issues
- 16:21that disrupt your company's productivity?
- 16:23Intelligent is here to help much like you take your car
- 16:26to the mechanic that has specialized knowledge on how
- 16:28to best keep your car running intelligent helps you
- 16:31with your Microsoft cloud environment
- 16:33because that's their expertise.
- 16:35Intelligent keeps up
- 16:36with the latest updates in the Microsoft Cloud
- 16:38to help keep your business running
- 16:40smoothly and ahead of the curve.
- 16:41Whether you are a small organization
- 16:43with just a few users up to an organization
- 16:45of several thousand employees, they want to partner with you
- 16:49to implement and administer your Microsoft Cloud technology,
- 16:53visit them at intelligent.com/podcast.
- 16:56That's I-N-T-E-L-L-I-G-I-N
- 17:01k.com/podcast.
- 17:04For more information or to schedule a 30 minute call
- 17:06to get started with them today,
- 17:09remember intelligent focuses on the Microsoft cloud
- 17:11so you can focus on your business.
- 17:15All right Scott, should we move on to our next topic?
- 17:19- We spent a quick, we've spent
- 17:21- a quick 20 minutes, quick 15 minutes there.
- 17:2215, 20 minutes on this one. Yeah, but it's an important one.
- 17:26Again calling out, I think using this story
- 17:28to highlight the importance of some
- 17:30of these backups and things to think about.
- 17:32This next one is a little bit more
- 17:33of an update on a topic we talked about earlier.
- 17:36Diving a little bit more focused into the Microsoft space
- 17:40is security copilot.
- 17:42I don't remember which episode it was.
- 17:43It's been a few episodes ago when it first came out of GA
- 17:49and it was available to stand up and I had said that I went
- 17:51and turned it on in my environment.
- 17:53I went and created a security compute unit instance
- 17:57or an instance of copilot for security,
- 18:00which you can scale based on security compute units.
- 18:02It's $4 per hour per security compute unit.
- 18:07And I spun it up for like two or three days
- 18:09and didn't see anything on my Azure bill.
- 18:12I was like huh, I've only used it a couple times.
- 18:15Maybe it only bills when you ask it questions.
- 18:18Something like that. And people were like,
- 18:20well let me know when you find out what happens.
- 18:22So I turned it off for a while
- 18:24'cause I started getting scared that it was just racking up
- 18:27a bill in the background
- 18:28and even though I had my quota on there
- 18:33and I just got nervous.
- 18:34So the other day I turned it back on again just to find out
- 18:37what would happen and it started charging me
- 18:39very quickly. Do
- 18:40- Me a favor.
- 18:41Yeah, just so everybody has context,
- 18:43flip your web browser back over to
- 18:45- You for those that are seeing this . Yeah,
- 18:47- Flip your screen back over to cost management here.
- 18:49- . Yes. So here is it.
- 18:52I went from, and if you can't see it, I had like a $59
- 18:57that I had accumulated on May three, may four.
- 18:59I was up to $162
- 19:02and then it kept going very linearly.
- 19:05So the first two days it looked like a massive hockey stick
- 19:08'cause I went from increasing my bill like 12 bucks a day
- 19:11to increasing it by roughly
- 19:14$113 a day, a hundred dollars a day
- 19:19and then it continued and I do still have like my
- 19:23$150 limit on the subscription.
- 19:26So even though it shows my accumulated cost up
- 19:28around $1,400 now because I may
- 19:31or may not have forgotten
- 19:32to go turn it off once I saw it hit
- 19:35that it is very much one security compute unit at $4 per
- 19:40security compute unit per hour has absolutely nothing to do
- 19:43with how frequently you use it.
- 19:44Once you light this thing up
- 19:46and turn it on, you are going to get billed $4 per hour
- 19:51as long as it is created in in existence.
- 19:55So realistically security compo copilot is absolutely going
- 19:59to cost you, I think it's like $2,920 a month if
- 20:04you go do the math and multiply out how many hours
- 20:07and average days in a month over the course of a year.
- 20:11All of that math. And that's just
- 20:13for one security compute unit.
- 20:14Know when you go spin these up, Microsoft
- 20:17recommends a minimum of three security compute units.
- 20:19Again, not saying you need it, they let you still do one,
- 20:23but three would absolutely cost you $12 per hour
- 20:26per yeah $12 per hour over the course of a month,
- 20:29which you can do the math 3000 multiply by three
- 20:32or at like $9,000 a month
- 20:34for Microsoft's recommended minimum configuration
- 20:36for security copilot.
- 20:38So pricing it is absolutely if you wanna use this, going
- 20:41to cost you a minimum of three grand per month
- 20:45unless you do something different.
- 20:48And I had some conversations with
- 20:52some people the other day that are like, I'm starting to use
- 20:56like logic apps or PowerShell
- 20:58or things like that to ramp up their security compute units
- 21:01or actually to like,
- 21:03I don't know if they were actually going to the point
- 21:05of destroying security copilot
- 21:07and recreating it when they'd wanna use it.
- 21:09So again, if you're not gonna use it at all
- 21:11during the night, do you really need to have three or four
- 21:14or five security compute units provisioned from 6:00 PM
- 21:18until 8:00 AM the next morning
- 21:21or do you actually blow it away and recreate it?
- 21:23It led to an interesting discussion I had about different
- 21:26ways people are trying to manage costs
- 21:29of security copilot based on number
- 21:31of security compute units provisioned within that instance
- 21:35of security copilot
- 21:36or even just like, can we just blow it away?
- 21:39It's expensive Scott,
- 21:40which led us into other discussions about ROI
- 21:43or even some of the pros
- 21:45and cons of blowing it away versus
- 21:47ramping it down, things like that.
- 21:49- And other news AI be expensive , right, I guess is
- 21:55is the takeaway there.
- 21:56So while it's a loss leader in some places
- 22:00maybe think like co-pilot consumer versus
- 22:04co-pilot within versus M 365 co-pilot, co-pilot
- 22:09for security is definitely not
- 22:14a loss leader kind of thing.
- 22:16So you need these SCUs, these security compute units
- 22:22to actually be able to
- 22:25have the associated compute to run
- 22:28through an action on your queries within the underlying LLM.
- 22:33So you know, responding to a query in an LLM takes a bunch
- 22:37of CPU and and memory
- 22:40and other things on the host to go
- 22:42and actually like retrieve the data read out
- 22:45of the vector databases, do all that kinda stuff.
- 22:48If you're doing RAG or something like that along the way,
- 22:51retrieve augmented generation,
- 22:52well then you've also gotta go out
- 22:54and have the compute to be able to retrieve that data.
- 22:58Say it was like a Word document
- 22:59or a PowerPoint, something like that.
- 23:01Be able to parse that in an LLM
- 23:04and then be able to construct these meta prompts and
- 23:07and all the other things so it's not free to get there.
- 23:10It's also very kind of fuzzy
- 23:16as to what that looks like and how it manifests.
- 23:20So you know you do have some usage monitoring within
- 23:24copilot for security.
- 23:25So directly within like the security copilot portal,
- 23:29security copilot.microsoft.com
- 23:32and being able to go thing, go and see things that way
- 23:35or you have kind of this just raw view
- 23:38of costing within cost management
- 23:40and how that carries through.
- 23:42But it's an interesting thing.
- 23:43You're sitting at $4 per hour, at least in the hero
- 23:49regions out here in the US you know that equates to,
- 23:52like you said, basically three grand a month, three K 29 90
- 23:57I I think it's safe to round up a little bit and
- 24:00and just call it three grand in that case.
- 24:01Yep. And then you have the kind of recommendation
- 24:05for compute units.
- 24:07So if you don't know what you're doing with these things
- 24:09and you just kind of look at 'em and you go out
- 24:11and read like hey where should I start?
- 24:13Well you can start with one,
- 24:14I believe the recommendation is three.
- 24:16So you're kind of sitting at a min cost of
- 24:20nine grand per month before you've really done much with it.
- 24:25So like all things that needs
- 24:27to be measured and weighed, right?
- 24:29Like what's the ROI there
- 24:31and what's the value for me as a customer?
- 24:34Like once you're starting to hit like nine grand, you know,
- 24:37is it worth having that for an entire month
- 24:40or should you just pay for an MSP pay
- 24:43for a consultant, something like that.
- 24:45Once you're doing a couple of these
- 24:48and you're getting up to maybe not like the nine grand
- 24:50marker but let's say you hit the point where you're at six
- 24:54SCUs and you're running those for an entire year, now all
- 24:57of a sudden you've gone from 90 plus KA year
- 25:01to 180 k plus pretty quickly
- 25:03and that's theoretically somebody's salary , right?
- 25:08Including benefits and
- 25:09and other things on top of it, at least here in the us.
- 25:12So now is having one person better than having a bunch
- 25:16of commute units, I don't know, you know, needs
- 25:19to be weighed out organization by organization. I think
- 25:23- This is where it does start getting interesting
- 25:25for me in some of our discussion even
- 25:27before we started recording was
- 25:30how do you start showing the ROI of co-pilot
- 25:34for security in that particular case?
- 25:36Because again me, I have myself a couple
- 25:40of contractors that are doing work for me.
- 25:43I am not gonna go out
- 25:44and spend 30 KA year for security copilot.
- 25:49I can go into my audit logs, I can go write power shell,
- 25:53I can go write KQL
- 25:55and it is probably not going to save me 30 grand
- 25:58of time a year to have co-pilot for security in place.
- 26:02I'm not gonna spend that much time asking security questions
- 26:06of my environment with the size of company I am.
- 26:09But to your point, this is where the scale gets interesting
- 26:12and can you show
- 26:13that ROI is now you start getting up into a hundred,
- 26:16150 employees, 10,000, 20,000 employees,
- 26:20there's a lot more data.
- 26:21If I go write a KQL query, I may pull a lot more data
- 26:25that I have to sort through or if I'm running a PowerShell
- 26:27script, there's just a lot more data
- 26:28as you get a bigger organization.
- 26:30So does having copilot for security where I can go in
- 26:34and ask natural language type of questions,
- 26:38ask questions about my audit logs
- 26:41if I have Intune ask questions about devices
- 26:45and about events and Intune does having a copilot to go over
- 26:49that much data, give me an ROI
- 26:52or to your point, do I start ramping up now
- 26:55because I have that many more employees, I'm having
- 26:58to buy six 10 security compute units
- 27:03and now my cost is getting up into the multiple hundreds
- 27:07of thousands of dollars like you said now it's a salary
- 27:10and I can pay somebody
- 27:13or multiple, somebody's a full-time salary to go in
- 27:17and write KQL queries
- 27:18and build out other processes to detect the data.
- 27:23It's an interesting ROI discussion.
- 27:26I don't know that I have the answer on
- 27:27how you would calculate that
- 27:30but I think it's something that a lot
- 27:32of companies are going in looking through
- 27:34and to your point, it's expensive.
- 27:36I get it. I would love
- 27:38to see this be a little bit more per use
- 27:40and maybe if there's Microsoft maybe would add some auto
- 27:44scaling in the future where you can scale up
- 27:46and down your security compute units
- 27:47because there may be some validity
- 27:49to you just have one security compute unit during the night
- 27:52and then ramp it up to six or nine or 10 during the day.
- 27:55'cause one of those things we talked about,
- 27:57if you actually completely blow it away,
- 27:59you are gonna lose the history of your queries.
- 28:02There's that conversational history
- 28:05and while not retraining of the models, some benefits
- 28:09that come from keeping that co-pilot instance up
- 28:12and having those queries
- 28:13that historical que in your copilot.
- 28:16But I could see where there could be some rationalization
- 28:19to having, especially a large company
- 28:21where they're using copilot all the time,
- 28:23having 10 security compute units
- 28:25during your working hours when you're actually diving in
- 28:28or if there's some type of incident
- 28:30that is raised in your environment, you need
- 28:32to ramp up those security compute units So you can get
- 28:35responses to these questions a lot quicker
- 28:38but then when you're not using it ramp it down
- 28:40to one security compute units, you retain your history,
- 28:42you retain that environment
- 28:45but you're not paying
- 28:47for all those security compute units 24 7. I
- 28:50- Think it depends a lot on the functionality volume like
- 28:55so one of the, I don't know, maybe you can help me out here.
- 28:58So one of the confusing things to me with the copilot
- 29:03for security thing is it's kinda like back to
- 29:06that like suite of suite stuffs, right?
- 29:08Like that we've been talking about with Intune
- 29:11and and other things.
- 29:12So there's the concept
- 29:14of the embedded experiences within copilot for security.
- 29:18So that could be like co-pilot for security
- 29:20as it's embedded inside enterra.
- 29:22It could be co-pilot for security
- 29:24as it's embedded in Intune, it could be co-pilot
- 29:26for security as it's embedded within defender.
- 29:29And then each one of those defended
- 29:31embedded experiences has its own set
- 29:34of like nuance in the way things like logs are stored,
- 29:38how you query those logs, how you write effective prompts
- 29:42around those and and how all that stuff goes.
- 29:45So there's that piece of it
- 29:46and then there's the things that actually happen kind
- 29:48of like automagically in the background, right?
- 29:51If I think about like hey I have an active incident
- 29:55and I'm trying to query for risky users in entra
- 30:00and I don't know how to do that today.
- 30:03You know there there could be value in having
- 30:04that stuff spun up right there
- 30:06and you probably don't need a whole bunch of SCUs
- 30:09and a whole bunch of compute sitting behind it just
- 30:12to effectively prompt
- 30:14and get a KQL query that then you can go and run and
- 30:17and bring that data back.
- 30:19The thing that you might need it for is something like,
- 30:22say you're an organization who experiences a lot
- 30:26of live sites and you're doing a ton
- 30:28around incident management.
- 30:30So like one of the capabilities in the embedded experience
- 30:32for my co-pilot for security with Microsoft Defender
- 30:36is doing automated incident summaries.
- 30:39So if you can start to automate incident summaries
- 30:43and distill those down
- 30:44and potentially automate summarization of RCAs,
- 30:49can you eventually get to the point
- 30:50where this thing can write salient RCAs for you?
- 30:54Maybe is it tomorrow? Probably not. Is it a year from now?
- 30:58Is it two years from now?
- 31:00I don't know but that's like an inflection point
- 31:02that's likely to be on the horizon.
- 31:04And then if you think about that like being able
- 31:06to do like really good crisp incident summary response
- 31:10and RCAs RCA summaries and then eventually write RCAs
- 31:15or potentially even automate incident response,
- 31:19super valuable.
- 31:20Like if you think about being like an on-call, uh, A DRI
- 31:23or something like that, like the burnout could be real if
- 31:26you're the human that's on call 24 7 versus having the
- 31:31AI bot LLM whatever thing that can do it for you.
- 31:34Like I don't see how most folks wouldn't kind of pine for
- 31:38that and grasp onto it
- 31:41but you know, you kind of need to get it all to the point
- 31:43where it's like ooh those incident summaries are really good
- 31:47and ooh those RCA summaries are really good
- 31:49and can I take this to the next click stop
- 31:52and get it to where you know, it's turning more into
- 31:55automate all the things and if you can like hey great,
- 31:59there's likely to be more than enough value that's kind
- 32:02of inflicted by the tool there that makes it worth,
- 32:05you know, whatever the cost is that's associated with it.
- 32:08So I don't know, we'll see where a lot of this stuff goes.
- 32:12It does feel a lot like automate yourself out
- 32:15of a job when it comes to things like copilot for security
- 32:18and I think there's a ton of angst there in general,
- 32:21you know you mentioned like writing queries when you're
- 32:24doing threat hunting like at some point you're probably just
- 32:27building a whole body of individuals
- 32:29who are either getting really good at prompting
- 32:32or they can be really good at the actual hunting experience.
- 32:35I think for now you still want them
- 32:37to be good at the hunting experience.
- 32:38You don't want them to just be good at prompting
- 32:41and being able to draw on that as kind
- 32:43of their, their superpower.
- 32:45So yeah, we'll see where all this stuff goes.
- 32:47It's gonna be interesting like
- 32:49and I don't know what the timeline is like it's very hard
- 32:51to understand right now like
- 32:53'cause this stuff is just moving so fast.
- 32:55Is that tomorrow? I don't know. Probably not.
- 32:59But is it like six months from now? Is it a year from now?
- 33:01Is it two years? Is it five years?
- 33:04That's very hard to discern.
- 33:06- The other thing you still run into with copilots,
- 33:08and I saw this some even with copilot for security
- 33:12and to be fair I haven't
- 33:13had a ton of chance to play with it.
- 33:14'cause frankly I can't keep it running long enough
- 33:17to play with it for very long.
- 33:18I need to like set out dedicated points of time
- 33:20where it's like I'm gonna spend this day really diving into
- 33:24it and go spin it up
- 33:25and spend 50 bucks to have it running for a day.
- 33:28But there's still the thing of hallucinations too, right?
- 33:30Like if you're doing threat hunting
- 33:33and you are saying show me all the attacks on my
- 33:39exchange online environment
- 33:40or on my Azure front door instance coming from
- 33:45this set of IP addresses
- 33:48or you are creating these prompts to bring back your
- 33:53investigation, you want that to be 100% accurate.
- 33:56You don't want to miss something
- 33:58because co-pilot misinterpreted something
- 34:02or hallucinated on something or anything like that.
- 34:05So I think that's still very much an aspect
- 34:08of especially co-pilot for security
- 34:11into your point about you still are gonna want good people
- 34:14that are experienced in hunting
- 34:15and not just prompt engineering
- 34:17because I think while it can still pull a bunch
- 34:20of data quickly, there's still a validation
- 34:22that you'd wanna take place, especially initially
- 34:26to make sure that whatever is happening in the background
- 34:30when you're asking copilot for security these questions
- 34:33that it's returning these the right information.
- 34:35Because I did see where like when I did play with it
- 34:38and I was asking it different questions about Intune
- 34:42or about different data where it,
- 34:44I wouldn't get the same responses all the time
- 34:46and my data hadn't changed.
- 34:48I would expect the same responses every time around some
- 34:52of those and even some of the co-pilot for Microsoft 365.
- 34:55When I ask it about different tasks
- 34:58or tasks, the coming due dates
- 35:00or different tables to summarize different things,
- 35:03it's not always the same.
- 35:05And I think that's still one risk with all
- 35:08of these co-pilots is people,
- 35:10and we've talked about it, people always treating it
- 35:13as this is 100% accurate all the time when it's maybe not.
- 35:17And I, it's going to continue to improve as they continue
- 35:20to improve models, continue to look at data
- 35:23and figure out how do we make these in a way
- 35:26that they're more accurate.
- 35:27I am not worried about it running me out of a job right now.
- 35:31If anything, there are days when I'm pouring through rows
- 35:33and rows and rows of data, I'm like man,
- 35:35if copilot could help me dig through all of this quicker
- 35:39so I can move on to the next task that I have
- 35:42because I have more to do than I have time to do it,
- 35:44I'm actually looking forward to the day
- 35:46where co-pilot can help me optimize my time a little bit
- 35:49better because I don't, I, I don't see IT security co
- 35:52or co-pilot for security running
- 35:53me out of a job anytime soon. I
- 35:55- Mean I think that's the right way to think about it is
- 35:58as an accelerator, like so walk into it with
- 36:03kind of some intentionality like hey,
- 36:07this is all fairly new stuff.
- 36:09It's early days. Can I use it as an accelerator? Yes or no?
- 36:14Can our business use it as an accelerator? Yes or no?
- 36:17Are we thinking about it the right way?
- 36:19Like do we need to think about it as something that's on
- 36:22for a year or do we use it for three months
- 36:25to upskill ourselves and get to where we need to be?
- 36:28Right? Build that kind of prompt book and you know, wikis
- 36:32and all those kinds of things that you potentially want
- 36:36to have in place.
- 36:38Like use this to augment
- 36:40and improve your process is a good way to think about it.
- 36:45And you know, from that lens, I've worked places where like
- 36:48dropping 200 grand on a consultant to have them come in
- 36:52and write a 20 page document
- 36:53for you is something companies do a lot , right?
- 36:56Like, hey come help me improve this process kind of thing.
- 36:59Sure, whatever we, we've got just the consultant for you
- 37:02that that can help you do that.
- 37:04And there's value in those kinds of
- 37:09scenarios and lifting that stuff along.
- 37:12But you do have to be kind of walking into it with
- 37:14that level of intentionality.
- 37:16You can't be just sitting here
- 37:18and saying like, what am I gonna use this
- 37:20for and how's it gonna go?
- 37:21- Should we call it a day about twenty, fifteen, twenty
- 37:23minutes on backup and 20 minutes
- 37:26or so on co-pilot for security?
- 37:29And I'm guessing we both have meetings coming up. We
- 37:31- Can do it.
- 37:32I'm running low on coffee.
- 37:33- I am out of coffee. My coffee's gone.
- 37:35I just have a few, ended up with a few grounds in my coffee.
- 37:38I have a few grounds in the bottom
- 37:39of my coffee cup, but that's about it.
- 37:40- Fire your barista. Yeah,
- 37:42- Another story, another day.
- 37:43I don't know , but well that Scott, enjoy.
- 37:48What day is it? Tuesday. Enjoy the rest of your week.
- 37:50We normally do this on a Friday.
- 37:51We've had some sickness, some crazy busy schedules
- 37:55and recorded a bit at an off time and an off day.
- 37:58So enjoy the rest of your week.
- 37:59- We'll get back on track here. Hope
- 38:01- Everybody is back healthy
- 38:03and we should be back to our normal schedule here.
- 38:06Well maybe soon.
- 38:07We've got, now we have summers
- 38:08and vacations coming up, Scott.
- 38:10- Yeah, we do. My kids end school on
- 38:12Friday last day for them. So woo-hoo.
- 38:14- Are you excited for all the noise to return to the house?
- 38:17- ? My kids are teenagers.
- 38:19There's no such thing as noise.
- 38:20They're gonna be sitting in their bedrooms
- 38:21and playing video games, let's be
- 38:23- Honest.
- 38:24Sounds good. Well congrats on the end
- 38:26of school summer coming up and enjoy your week
- 38:28and we'll talk to you again soon.
- 38:30- All right, great. Thanks - Ben. Thanks Scott.
- 38:33If you enjoyed the podcast,
- 38:35go leave us a five star rating in iTunes.
- 38:37It helps to get the word out
- 38:39so more it pros can learn about Office 365 and Azure.
- 38:43If you have any questions you want us to address on the show
- 38:46or feedback about the show, feel free
- 38:48to reach out via our website, Twitter, or Facebook.
- 38:51Thanks again for listening and have a great day.