Latest / Microsoft Cloud IT Pro Podcast / Episode 381 – Microsoft Graph Data Connect
Transcript
- 0:03Welcome to episode
- 0:05381 of the Microsoft Cloud IT Pro Podcast,
- 0:08recorded live on July 26,
- 0:112024.
- 0:12This is a show about Microsoft 365
- 0:14and Azure from the perspective of IT pros
- 0:17and end users, where we discuss a topic
- 0:19or recent news and how it relates to
- 0:21you. Today, we have some updates to an
- 0:23older, but not often shared feature of Microsoft
- 0:27365.
- 0:28So without further ado, we bring you an
- 0:31introduction to Microsoft Graph Data Connect as well
- 0:35as some recent announcements about data that is
- 0:37accessible via the service.
- 0:41We are live with another episode of the
- 0:43Microsoft Cloud IT Pro podcast recorded live on
- 0:46Friday, July 26th in the middle of a
- 0:49thunderstorm
- 0:49in Florida because
- 0:51it's Florida, and it's afternoon. It is
- 0:53starting
- 0:55to sound eerily ominous out there,
- 0:58and we are into
- 1:00thunderstorms
- 1:00in the afternoon season.
- 1:03So if I drop out, it's probably
- 1:06not you. The thunderstorm? It's me.
- 1:09And my awesome
- 1:11my awesome access to electricity over here on
- 1:14the beach side of town. I just want
- 1:15the thunderstorms to pass so I can go
- 1:17play pickleball tonight. Oh, you're really dating yourself
- 1:19if you're if you're if you've already started
- 1:21playing pickleball.
- 1:22My my parents play pickleball,
- 1:24Ben. Pickleball pickleball is not for the youth
- 1:27like us.
- 1:29You know what? You say that, but it's
- 1:31better than not doing anything, I figure, and
- 1:33I actually do kind of enjoy it. Alright.
- 1:35It's kinda fun. I can't play tennis. It's
- 1:37so for me, I can't play tennis. I
- 1:39love ping pong, but, look, I mean, I
- 1:41don't get to work out playing ping pong,
- 1:43and ping pong takes room indoors. So pickleball
- 1:45is, like, a happy medium between for me,
- 1:47where I can still carry over some of
- 1:50my ping pong abilities
- 1:52and get some exercise at the same time.
- 1:54If you find that it's that much fun
- 1:55call me old. You might have to invite
- 1:57me. I mean, I have I have all
- 1:58the gray hair, so, you know, if it
- 2:00does end up being, like, an old person
- 2:01contingent thing, you just pull me along, and
- 2:04I'll I'll come play pickleball with you. I
- 2:06think we have all the stuff in the
- 2:07garage. My my my kids have a pickleball
- 2:09set that they bought to play pickleball with
- 2:10their grandparents. See, and you were just telling
- 2:12me I was dating myself by playing pickleball,
- 2:14but, yeah, your kids play pickleball. No. I
- 2:15said, we have a pickleball set in the
- 2:18garage
- 2:19that was bought so my kids could play
- 2:21pickleball with their grandparents. I didn't say that
- 2:23my kids actually were playing pickleball with their
- 2:25grandparents
- 2:26or that that set was being used at
- 2:28all.
- 2:29Oh, okay. I stand corrected then. So speaking
- 2:33of buying other things, I spent more money
- 2:34today. Did you see did you get a
- 2:36chance to watch the YouTube video I sent
- 2:38you? Yeah. Elgato's got some new stuff out.
- 2:40So they have
- 2:42a new XLR interface
- 2:45and a new USB hub, so I'm interested.
- 2:47What did you pick up? We have all
- 2:49our podcasts set up, so I was like,
- 2:50I do not need an XLR
- 2:52audio interface. However,
- 2:54if you do podcasts or have an XLR
- 2:57mic and you just need like a single
- 2:59XLR port,
- 3:01that is super
- 3:02intriguing and interesting
- 3:04to be able to incorporate an XLR audio
- 3:07interface
- 3:08straight into the Stream Deck. I picked up
- 3:10the USB 1. So for me, it was,
- 3:13what, I think it's like $59
- 3:15for the USB
- 3:16add on, and you essentially, like, unscrew the
- 3:19base of the Stream Deck Plus
- 3:21and then pop this attachment on the back
- 3:23and screw it back in, and it has
- 3:252 USB C ports, 2 USB 3 ports,
- 3:28and it actually has, like, an SD card
- 3:30reader and all of that.
- 3:32And here's my use case. I have 2
- 3:34Stream Decks. I have, like, a USB
- 3:37speakerphone on my desk. I charge my keyboard
- 3:40sometimes off of USB, and I have all
- 3:43these cables, like, running back and around
- 3:45up
- 3:46to docks hidden under my desk, behind my
- 3:49monitor, all of that.
- 3:50But there are times I, like, just want
- 3:52to plug a USB device in that's right
- 3:55here on my desk, so I can minimize
- 3:57my USB cables running and snaking all around
- 4:01the desk and get myself a couple USB
- 4:03ports easily accessible,
- 4:06like right in front of me on the
- 4:08desk
- 4:09without adding something extra to the desk because
- 4:12they just kinda hide right in the back
- 4:13of the Stream Deck there. You'll have to
- 4:15tell me how this one works out for
- 4:16you. Like, one of my frustrations with the
- 4:18Stream Decks
- 4:20is,
- 4:21you know, they're they're bound to the software
- 4:23on your desktop.
- 4:24So you end up,
- 4:26you know, in these kinda mismatched state areas
- 4:30where sometimes, like, the button on your Stream
- 4:32Deck thinks it's on when the device really
- 4:34isn't on over here, like, the software is
- 4:35not reporting things correctly. So one of the
- 4:37things that I do with my Stream Deck
- 4:39at least once a day is I unplug
- 4:41it, and I plug it back in. And
- 4:43I just do that by taking the USB
- 4:44cable. So
- 4:46out the back. So one of the things
- 4:48so these new devices, I think they were
- 4:50only applicable to the Stream Deck Plus, which
- 4:52is fine. Like, that's what I have.
- 4:54But the way they also plugged in was
- 4:56basically
- 4:57having, like, a permanent USB,
- 4:59not cable, but just a plug sticking out
- 5:02on the mount device, And then you're basically
- 5:04mounting straight on top of that. So I
- 5:05don't know if I wanna go from just
- 5:07pulling a cable out once a day to
- 5:09pulling the whole thing off the mount or
- 5:10potentially figuring out how to, like, pull the
- 5:12cable from the
- 5:13cable kind of thing depending on where it
- 5:15is. So I'd be interested in feedback once
- 5:17you've had a chance to play with it
- 5:19a little bit. And even on, like, the
- 5:21XLR interface, like, I look around a lot,
- 5:23and I'm like, you know, I drive 2
- 5:24mics most of the day. So for my
- 5:26meetings, when I'm at work, I typically drive
- 5:28a shotgun mic.
- 5:31So to give folks a sense, so if
- 5:33I come over here, this is normally the
- 5:35microphone that I use on Teams calls and
- 5:38things like that just so I can be
- 5:39hands free and not have this big honking
- 5:41mic sitting in front of my face.
- 5:44But then when you and I do these
- 5:45things, you know, I I I bring the
- 5:47the the nice ESOS out and and pull
- 5:49that over for some of that, like, rich
- 5:51deep,
- 5:52you know, broadcasty goodness.
- 5:54And
- 5:55so I I really like, I I only
- 5:57need 2 XLR interfaces, and I could probably
- 5:59get by with 1. Like, there's really no
- 6:00reason that I couldn't just use 1 and
- 6:02and be done with it. And it would
- 6:03buy back a bunch of space on my
- 6:05desk. So the cool thing about these, by
- 6:07kind of combining the functionality of a USB
- 6:09hub or the XLR thing, it's so stupid.
- 6:12It's completely low hanging fruit, but it makes
- 6:14a 100% of sense. Like, in your use
- 6:16case and in mine, it'd be buying back
- 6:18desk space. Right? Like, if I could get
- 6:20rid of this honking
- 6:21device next to me that's driving, like, 4
- 6:23XLR interfaces when I only need
- 6:26to, that actually wouldn't be too bad.
- 6:29So so that's the nice part. The bad
- 6:31part is you're doing everything in software, and
- 6:32then you have to contend with Elgato software,
- 6:35which isn't the
- 6:36most robust stuff from what I've seen. So
- 6:38trade offs continue to abound in technology land.
- 6:41Yeah. I'll have to let you know. So
- 6:43I'm curious, like, for your use case of
- 6:46unplugging USB and plugging it back in, I
- 6:48think you would still do that. You would
- 6:49just unplug it from
- 6:52the USB hub instead of
- 6:55the back of the device because it looks
- 6:57like the cable just plugs into the USB
- 6:59hub, and then there's, like, a pass through
- 7:01into the stream deck for Yep. Connecting that.
- 7:04So I think it's still but then you
- 7:06disconnect
- 7:07every USB device. That's the rub. Right? Like,
- 7:09so today,
- 7:11I'm only unplugging the Stream Deck, but if
- 7:13I have a bunch of other stuff plugged
- 7:14in there, like, let me see. What else
- 7:15do I have USB? Oh, my my mouse,
- 7:17my keyboard, like the little, you know,
- 7:21the 2.4 gigahertz dongles for those hang off
- 7:24a USB hub.
- 7:26Yeah. So I I wouldn't wanna lose, like,
- 7:28my mouse and my keyboard and everything while
- 7:29I'm waiting for the whole hub to go
- 7:31through its machinations
- 7:32and
- 7:34rebooting and all that stuff. So we'll see.
- 7:36I'd be I'd be interested in the feedback
- 7:37on stability specifically. I will let you know.
- 7:40I might not be a good test because,
- 7:43ironically,
- 7:44I do not have the same issue you
- 7:47do around
- 7:49stuff getting out of sync. So this is
- 7:51would be another curious thing as to why
- 7:53that is. It's either sync or it just
- 7:56straight up freezes.
- 7:57Like, my stream deck my stream deck freezes
- 8:00all the time.
- 8:01I cannot remember
- 8:03the last time I have unplugged and replugged
- 8:06in my stream deck. It just keeps working
- 8:09for me.
- 8:10You're living a magical world, my friend, every
- 8:12single day. Apparently.
- 8:14But that being
- 8:16said,
- 8:17I will and I'm curious if this works
- 8:20for you if if you've tried it. What
- 8:22I do have to do every once in
- 8:24a while because this to your point, it
- 8:26kinda streams through the software,
- 8:28is my software
- 8:30loses
- 8:31its connection
- 8:32back to, like, my Philips Hue or it
- 8:35loses my connection to my
- 8:37companion app
- 8:39or my Govee app, and I don't unplug
- 8:42and replay in the device. I just kill
- 8:43the Stream Deck software and reopen it, and
- 8:46it reconnects everything back up and starts working.
- 8:48I've become a ninja killing the back end
- 8:50daemons for Stream Deck, particularly on, like, Mac
- 8:52OS.
- 8:54So, like, I drive a lot of my
- 8:55automations either through HomeKit or through Home Assistant,
- 8:59like, we've talked about both of those in
- 9:00the past.
- 9:01So there's a specific daemon that runs for
- 9:03the Home Assistant plug ins, so you don't
- 9:05actually have to kill your whole Stream Deck.
- 9:07You just have to kill the Home Assistant
- 9:081, and then it magically
- 9:10kinda comes back. But, yeah, I I have
- 9:13a whole, like, I have a whole script
- 9:15I run on the side. Like, just go
- 9:17run this bash file,
- 9:19and that that kills everything and then brings
- 9:21it back to where it needs to be.
- 9:22Nice. Unless it was CrowdStrike, in which case
- 9:24there is no coming back. Another topic, another
- 9:27day. There's coming back from CrowdStrike. It just
- 9:29involves 20 reboots and a USB key and,
- 9:34a whole bunch of extra steps Manually touching
- 9:36every device and if you are a cloud
- 9:39hosted
- 9:40virtual machine. Yes. We could talk about that,
- 9:43but it truth be told, it I mean,
- 9:45it a pet impacted Azure. It didn't really
- 9:47impact Microsoft 365. It was more Windows. We
- 9:50have other topics, Scott. Maybe we should Mabel
- 9:52would come back and talk about Crowd Strike.
- 9:54Did the Crowd Strike impact Azure?
- 9:57So there was an Azure outage, which doesn't
- 9:59tie to the CrowdStrike outage. They were
- 10:02unfortunately, like, within, like, minutes of each other,
- 10:04like, 20 minutes or something like that. Yes.
- 10:06But completely unrelated. And I did like, I
- 10:09legitimately felt bad for Microsoft because I was
- 10:12reading some of these news articles, and you
- 10:14were getting these companies like,
- 10:17this is my pet peeve with the media.
- 10:19I get why people do it, but they're
- 10:21always want to be first, and they always
- 10:23wanna have the breaking news, they always wanna
- 10:24have the most information.
- 10:26But that inherently, I feel like leads to
- 10:28them getting stuff really wrong sometimes.
- 10:31And in this case, I read so many
- 10:33articles where it was like, CrowdStrike outage takes
- 10:36down Azure and Microsoft 365, and I'm like,
- 10:40no. Microsoft 360 5 and Azure went down,
- 10:42like, the day before due to another issue.
- 10:44And from what I read, those 2 were
- 10:47somewhat related. But then the whole CrowdStrike thing,
- 10:50like you said, very unfortunate timing
- 10:52happened right after Azure and Microsoft 365 got
- 10:56came back up. And so many news companies
- 10:59munged the 2 together
- 11:02when the CrowdStrike thing really was not Microsoft's
- 11:05fault at all. So the PIR
- 11:09post incident review
- 11:11for
- 11:12the
- 11:13US central outage
- 11:16is up and out there. I'll I'll I'll
- 11:18see if I can pull it up for
- 11:19folks and at least include it in the
- 11:22show notes or I will post it over
- 11:24to Discord where if folks want to become
- 11:26members of the show because Ben told me
- 11:28I should talk about membership or,
- 11:30Hey. Become a member. You'd ask us access
- 11:32to Discord. Come and join us while we
- 11:33record these shows live, and follow along and
- 11:36chat with us and generally have fun.
- 11:38How was that? Was was my spiel good
- 11:40there? You did good. Good job, Scott. So
- 11:42that being said, should we talk about our
- 11:44topic today? This is like an old one
- 11:46that has some new updates. I mean, that's
- 11:49pretty much what we're good at, right, is
- 11:50talking about old ones. That's Right.
- 11:56Do you feel overwhelmed by trying to manage
- 11:58your Office 365 environment? Are you facing unexpected
- 12:01issues that disrupt your company's productivity?
- 12:04Intelligink is here to help. Much like you
- 12:06take your car to the mechanic that has
- 12:08specialized knowledge on how to best keep your
- 12:10car running, Intelligent helps you with your Microsoft
- 12:13cloud environment because that's their expertise.
- 12:15Intelligent keeps up with the latest updates in
- 12:18the Microsoft cloud to help keep your business
- 12:20running smoothly and ahead of the curve. Whether
- 12:22you are a small organization with just a
- 12:24few users, up to an organization of several
- 12:261000 employees,
- 12:28They want to partner with you to implement
- 12:30and administer your Microsoft Cloud technology.
- 12:33Visit them atintellijinc.com
- 12:36/podcast.
- 12:37That's
- 12:38intelligink.com/podcast
- 12:44for more information or to schedule a 30
- 12:46minute call to get started with them today.
- 12:49Remember, IntelliJint focuses on the Microsoft cloud, so
- 12:52you can focus on your business.
- 12:56Talking about old stuff that resurfaced,
- 12:58so this is an article
- 13:01that I saw
- 13:02published on July 11, and it brought back
- 13:06memories and we started digging into it, was
- 13:08there was a post from
- 13:10Jose
- 13:11Barreto,
- 13:11that's probably how you pronounce his name,
- 13:14in
- 13:15tech community about updates on the SharePoint files
- 13:18dataset. And I was like, the SharePoint files
- 13:21dataset from the headline, I couldn't remember what
- 13:23it was. And then I started reading it,
- 13:25and it's under the subcategory
- 13:27of Microsoft Graph Data Connect
- 13:30for SharePoint. And if you go start looking
- 13:32into this,
- 13:33this ties back to
- 13:36a feature that came out. I think it
- 13:38was originally announced. I had the YouTube video
- 13:40here, and then I lost it.
- 13:412021,
- 13:44I believe, like, 3 years ago,
- 13:46around this Microsoft
- 13:48Graph Data Connect
- 13:50and
- 13:51allowing you to
- 13:53essentially
- 13:55go in and grab a bunch of analytics
- 13:58data out of your tenant
- 14:00around
- 14:01various metrics.
- 14:03And some of the metrics from back in
- 14:062021 when this was originally announced
- 14:09was being able to go in and pull
- 14:10datasets around
- 14:12users,
- 14:13direct reports, and managers from like an Office
- 14:15365
- 14:16people perspective, and then they had it for
- 14:20Exchange Online being able to pull in datasets
- 14:23around
- 14:24calendars, events, messages,
- 14:26contacts, mail folders, mailbox settings, sent items,
- 14:30and then they announced that Teams chats would
- 14:31be coming.
- 14:33And I feel like when they originally announced
- 14:35this back in 2021,
- 14:38they,
- 14:39like, it just I I don't re remember
- 14:41hearing much about it over the last 3
- 14:43years. And then this article popped up with
- 14:473 updates to it
- 14:50with some public availability for these updates coming
- 14:54here in the coming months up here in
- 14:57August.
- 14:58So this is kinda it's an interesting feature,
- 15:01and I don't I'll be honest, I don't
- 15:03know where to start. Do we wanna start
- 15:04with the updates, or do we wanna kinda
- 15:06start with, like, what is the Microsoft Graph
- 15:09Data Connect? Like, what you would use this
- 15:11for? We should go back, and we should
- 15:13definitely start this one with,
- 15:16what is this?
- 15:18And it took me
- 15:20more than a hot minute to wrap my
- 15:22head around
- 15:24what this is
- 15:26and the value of this and and kind
- 15:29of
- 15:30I I think where it can inflict value
- 15:33on customer workloads, and I I I I
- 15:35kind of love features that inflict value.
- 15:39And this was definitely one of those that
- 15:41does this, but it's kinda so confusing with
- 15:43the way it's marketed and it's put together.
- 15:45So
- 15:46give me your pitch for it and your
- 15:48understanding. And then if that matches with mine,
- 15:50we're good. But if not, I'll I'll give
- 15:52you a kind of some of my take
- 15:54and where I think this actually sits and
- 15:56why I think it's
- 15:57kinda cool and maybe underappreciated. Like, I'm surprised
- 16:00I don't hear about it more based on
- 16:02what I learned about it after you pointed
- 16:04it out to me, and then spent a
- 16:05week researching it and clicking buttons and playing
- 16:08around. Okay. So here's my take on it.
- 16:10Microsoft Graph has I mean, Microsoft Graph is
- 16:14the underlying
- 16:16it's not really the underlying data. It's the
- 16:18underlying
- 16:19API, essentially,
- 16:20for all things
- 16:22Microsoft 365.
- 16:23Like, when you go
- 16:25write different connectors, when you're going and writing
- 16:28custom code, when you're doing stuff in Power
- 16:31Apps, when you're doing stuff in PowerShell, ultimately
- 16:33you're connecting to Microsoft Graph for performing certain
- 16:36actions. The way I understand Microsoft
- 16:39Graph Data Connect is all these APIs also
- 16:43surface different things, and they have a whole
- 16:47list of different
- 16:51activities that are in micros in the Microsoft
- 16:55Graph data. And my understanding is is that
- 16:58this is probably where a lot of the
- 16:59reports
- 17:00come from when you are looking at reports
- 17:03on Microsoft 365 or storage trends,
- 17:07around mailbox reports,
- 17:09emails sent, emails received, all of that. It's
- 17:12coming from Microsoft Graph. What Microsoft Graph Data
- 17:15Connect
- 17:16does is,
- 17:18by nature, all of that data has a
- 17:20time frame. Microsoft does not let you keep
- 17:22that data forever. They have to store it
- 17:24somewhere. Understandably,
- 17:26it's going to have a lifespan. Usually, it's
- 17:28somewhere between 30 90 days.
- 17:30What Microsoft Graph Data Connect does is allow
- 17:33you to go
- 17:34set up a connector
- 17:36to these datasets in the Microsoft Graph and
- 17:40essentially extract that data to whatever
- 17:44storage location
- 17:46you want to, and it's coming through something
- 17:49like Azure Synapse or Azure Data Factory where
- 17:52you go connect to Graph, pull this data
- 17:54through one of those, shove it off into
- 17:57some location
- 18:00where now you control the location so you
- 18:02can keep this data as long as you
- 18:04want to, but you also have all these
- 18:06different data sets and activities
- 18:08that you can now
- 18:09build your own reporting, your own dashboards,
- 18:13view trends, all of that off of all
- 18:16these different activities and data that is surfaced
- 18:19through Microsoft Graph via this Microsoft Data Connect
- 18:24functionality
- 18:25feature, whatever you wanna call it. So that
- 18:28is my not necessarily a 32nd synopsis, but
- 18:31my
- 18:32understanding
- 18:33and rough how I would kind of describe
- 18:35what it is. I think you
- 18:37mostly
- 18:38nailed it as far as my understanding. I
- 18:40I I would extend on a couple things.
- 18:43So this is called
- 18:45Microsoft Graph Data Connect, and then it has
- 18:48all these datasets that come from
- 18:50services that support the Graph, Exchange, SharePoint, all
- 18:53this stuff.
- 18:55Well, it has Graph in the name.
- 18:58The nice thing about it, and I'm gonna
- 19:00say this is a nice thing,
- 19:01is it allows you to not have to
- 19:04work with the graph APIs directly. Right? There
- 19:07is no
- 19:08invoke web request. There is no going out
- 19:10and figuring out, am I using the beta
- 19:12endpoint versus the current GA endpoint?
- 19:16Which version of which PowerShell module do I
- 19:19use, and how much jankiness is there, like,
- 19:21any of that stuff. Like, the these are
- 19:23effectively
- 19:25snapshots
- 19:26of telemetry from the system
- 19:28that are pre aggregated and then pushed out
- 19:31on a schedule.
- 19:32And
- 19:34the the the cool thing here is they're
- 19:37not only aggregated and pushed out on a
- 19:39schedule, they're pushed out
- 19:41in a format that's consumable
- 19:44and with a defined schema
- 19:46that has a hard contract against it. So
- 19:49each one of these datasets,
- 19:51like the Outlook dataset versus the SharePoint dataset
- 19:55versus,
- 19:56you know, some of the identity ones, they
- 19:58have hard schemas that are associated with them
- 20:01where you can choose as a customer to
- 20:03consume the entire schema
- 20:05or to only consume
- 20:07portions of that schema. Right. So let's say
- 20:09you had a table with,
- 20:11you know, 50 columns in it
- 20:13based on the schema. And it turns out
- 20:14you only really need 10 of those.
- 20:16Great. You only take 10 of those, and
- 20:18that's what you get in your snapshot. And
- 20:20that's the data that's kinda dumped out continuously
- 20:23to get things going.
- 20:26So I I actually, like, I I looked
- 20:28at this, and I was like, why don't
- 20:30more services
- 20:31do this? It kinda makes sense if you
- 20:33think about it because Oh, 100%.
- 20:35You don't always need near real time
- 20:39extracts or data flows. And in a world
- 20:42where you don't require
- 20:44near real time things,
- 20:46near real time
- 20:50aggregation, transformation, all those things, like, it's expensive.
- 20:54It's it's a ton of telemetry to store.
- 20:56It's a ton of telemetry to aggregate together.
- 20:58Quite often, it's on, like, you as the
- 21:00customer to put it together. Alright? So if
- 21:02I was gonna take, like, Azure Resources as
- 21:04an example let's say we took virtual machines,
- 21:07and and we imagine something like that. Right?
- 21:09Like, so I have VM 1 and VM
- 21:112. VM 1 and VM 2 can both
- 21:12emit their telemetry, but I still need to
- 21:15rationalize which telemetry is from which one. How
- 21:17do I put it together? What are the
- 21:18right aggregates that I need to get to?
- 21:20Versus having something like this, say virtual machines
- 21:22was part of this data connect mechanism. You
- 21:24would just get one big dump daily
- 21:27or on your schedule of, hey, here's all
- 21:29your VM information as you wanted it. And
- 21:31that's basically what this is doing
- 21:34for these various datasets that sit out there.
- 21:36So if you're looking at more, like,
- 21:39long term reporting scenarios,
- 21:41and long term is really, like, honestly, anything
- 21:44past, like, a couple minutes, because, you know,
- 21:47at that point, like, once you're outside of
- 21:49a a couple minutes to a handful of
- 21:51hours, like, you're not doing real time data
- 21:53anyway. Like, figure out a way to snap
- 21:55to, like, a daily process or a weekly
- 21:57process or something like that, and then just
- 21:59track
- 22:00general trends and how those things go over
- 22:02time. And then maybe fill the gap with,
- 22:03like, some NRT stuff where you actually need
- 22:05it, maybe with, like, searching audit logs or
- 22:08real time metrics
- 22:09or or things like that along the way.
- 22:12So so it it's really cool because, like
- 22:15you said, for me, 1, you don't have
- 22:17to deal with the jank that is the
- 22:19Graph API. And let's be honest, the Graph
- 22:21API is pretty darn janky and continues to
- 22:23be.
- 22:24And it gives you this fixed, durable contract
- 22:27that then you can go ahead and proceed
- 22:29forward with.
- 22:31You don't need to be a
- 22:33developer,
- 22:34you know, like, hands on, slinging code against
- 22:36things. Like, it all comes out in common
- 22:38formats,
- 22:39and you can just wire that up immediately
- 22:41to
- 22:43a Synapse solution if you wanted to do
- 22:44some further aggregates on top of it with,
- 22:46like, Spark or something like that, stand up
- 22:48a Databricks cluster. Great.
- 22:50Or just pound point Power BI at that
- 22:53folder in a storage account and let it
- 22:55go. It's fascinating, and I don't know if
- 22:57you watched,
- 22:59I'm assuming in your studying too, you watched
- 23:01this video, and even back then they talked
- 23:03about,
- 23:041, not needing all the tables because
- 23:07columns
- 23:08in the table, all of the fields in
- 23:10the datasets,
- 23:11but also from a privacy perspective. They were
- 23:13like, if you want to run analytics on
- 23:15certain things,
- 23:16email, for example, it contained the email body
- 23:20in the dataset. But if you wanna filter
- 23:22out the email body, you can filter out
- 23:23the email body or filter out email subjects
- 23:27and just see, like, who's sending an email
- 23:30to who or whom is sending it to
- 23:32whom. When are when are we supposed to
- 23:34use who and whom? Neither here nor there.
- 23:37And filtering out users and groups. So maybe
- 23:40you're only pulling in telemetry
- 23:42from certain groups of users or you're excluding
- 23:45certain users from your telemetry, but
- 23:48a lot of flexibility there. And this was
- 23:50one. I'll pop this on the screen. We
- 23:52can throw an image in the show notes
- 23:54or something. But, like, this Power BI report,
- 23:57I've never seen this before, and I'm like,
- 23:58this is fascinating,
- 24:00where it's a super zoomed out,
- 24:03it's a
- 24:04it's showing like a graph of all the
- 24:06different users in a company
- 24:08with different bubbles in terms of emails that
- 24:11they've sent, but then connections of, like, who
- 24:14is emailing who within your organization.
- 24:16And they use this example as they could
- 24:18see like the HR department was emailing everybody
- 24:21else but wasn't ever emailing anybody else in
- 24:24the company and developing
- 24:27graphs of how different people were connecting, when
- 24:29they were emailing people, who they were emailing,
- 24:32how different departments were collaborating or not collaborating
- 24:35together,
- 24:36just fascinating
- 24:38analytics that you can pull out of this
- 24:41without
- 24:42necessarily,
- 24:44I mean, again, you wouldn't have to know
- 24:46what they're emailing. It's just the fact that
- 24:48emails are going from one person to other
- 24:50people.
- 24:51So, yeah, it's
- 24:53again, I remember this coming out. I started
- 24:55playing with it. This makes me wanna go
- 24:56play with it a whole lot more.
- 25:00Yeah. I it's
- 25:02it's it's an interesting one.
- 25:05I I also like that, like, the so
- 25:08so beyond, like, just, like, the generic, like,
- 25:09hey, it's a fixed contract and allows you
- 25:12to do reporting kind of your way with
- 25:14your tooling, things like that.
- 25:16It's
- 25:17also pretty agnostic.
- 25:19Like, the datasets, they just publish out to
- 25:21storage accounts, like you said, so super easy
- 25:24to kinda pick them up and play with
- 25:27them
- 25:29once they're in there.
- 25:31Open format, queryable, you can get out of
- 25:33them with
- 25:35ADF, you can get them out at them
- 25:36with Synapse, you can get out of them
- 25:37with Databricks, like,
- 25:39tons and tons of options just to
- 25:43make your life,
- 25:44a little bit a little little bit easier.
- 25:47So, yeah, it's the
- 25:49the the pricing actually is isn't too bad
- 25:52either.
- 25:53Like, I I guess we should call out,
- 25:54like, hey, like many things, this is an
- 25:56additive service.
- 25:57There is pricing. Yeah. Yeah. There there there
- 26:00are some considerations
- 26:03there for you
- 26:05just as,
- 26:06you know, how things marry up and ultimately
- 26:09come together. So,
- 26:11you know, you could be charged, and this
- 26:13could be a little hard to rationalize, Like,
- 26:14you'd be charged for storage consumption.
- 26:17You're gonna be charged
- 26:19for your usage based on the
- 26:21the the Graph Data Connect stuff.
- 26:24And then some of these, like, they have
- 26:26one off pricing.
- 26:28So, like, you opened with the update on,
- 26:30like, the SharePoint stuff. So the SharePoint files
- 26:32dataset has a different pricing construct
- 26:36than
- 26:38other parts
- 26:40of Graph Data Connect, which can be
- 26:43a little bit
- 26:45confusing
- 26:46as well. So, you know, you're you're looking
- 26:49at things like everything is based on number
- 26:52of objects copied and number of objects copy
- 26:55number of objects enumerated on one side and
- 26:58sent into, ultimately, like, the dataset before it's
- 27:01pumped out, blah blah blah. So you're talking
- 27:03about things like 0.35¢
- 27:05to,
- 27:070.75¢,
- 27:08like, per n object, like, n 1,000 objects
- 27:12loaded
- 27:14into things along the way.
- 27:16So I I I think, like,
- 27:19you know, I was, like, seeing things like
- 27:20this too. Like, you're also seeing a little
- 27:21bit of, like, the bifurcation and potentially, like,
- 27:24the associated cost on the service side to
- 27:26pull this stuff out. Like, you mentioned logging
- 27:28is expensive. It's not just expensive for customers.
- 27:30It's it's expensive for service providers as well,
- 27:33right, to generate and store and and put
- 27:35it all together.
- 27:36So so you do see some of that
- 27:38reflected kind of
- 27:40in the
- 27:41in in the pricing components as well. There
- 27:44is some chat going on in Discord
- 27:46asking about, like, is some of this pulling
- 27:48into Veeva? Yeah. This is probably also where
- 27:51they're getting some of the Veeva Insights stuff,
- 27:52but they were also asking about in-depth analysis
- 27:56of
- 27:56SharePoint online environments, and I think that gets
- 28:00into a little bit of what this
- 28:03announcement was was kind of three announcements that
- 28:06came out here a couple weeks ago
- 28:09is
- 28:10datasets
- 28:11for SharePoint, and the SharePoint file datasets are
- 28:13gonna be publicly available
- 28:15on August 20,
- 28:172024,
- 28:18so another month or so from now.
- 28:22They released the SharePoint files dataset
- 28:24pricing,
- 28:26And like you said, Scott, it's slightly different.
- 28:29And then the SharePoint files private preview extended
- 28:32to August 19, 2024, which makes sense. I
- 28:35essentially said private preview is gonna go right
- 28:38up until the GA date. So I don't
- 28:40know if private preview was before that. But
- 28:42kind of what you're getting now in this
- 28:44dataset, and this is where you get some
- 28:46of this updated
- 28:49information,
- 28:50and
- 28:51I think this is some of the stuff
- 28:52you're also seeing if you've looked at SharePoint
- 28:55Premium and some of the oversharing analytics and
- 28:57some of that is
- 28:59these updated datasets now for SharePoint
- 29:01files
- 29:03allow you to pull in, like, archive state
- 29:06of the site, you can pull in things
- 29:09like Recyco bin item counts, you can pull
- 29:11in
- 29:12communication sites, you can pull in if the
- 29:15site is a OneDrive site,
- 29:17if external sharing is enabled or not on
- 29:19the site,
- 29:20if the site is connected to a private
- 29:24Microsoft 365 group,
- 29:26privacy of the site, owners,
- 29:29last access data,
- 29:32and then you can pull in SharePoint permission
- 29:34datasets where it's total users,
- 29:37who something's created by,
- 29:39who an object is shared with based on
- 29:41the Azure AD object ID,
- 29:44how many users that's shared with, so a
- 29:46user count of how many
- 29:49files a or how many users a SharePoint
- 29:51file is shared with.
- 29:53So a bunch of additional
- 29:55data was added to the SharePoint datasets,
- 29:59that you can now start going in and
- 30:01pulling in once it becomes generally available here
- 30:04in another month.
- 30:06And, again, some of these are
- 30:08especially around the shared with type of analytics
- 30:11can help you
- 30:13with some of that oversharing that has become
- 30:15ever so popular to talk about with the,
- 30:18increasing demand for Copilot
- 30:21or even just general security. It's more like
- 30:24back to kind of what I was thinking
- 30:26earlier with just call it out again. So
- 30:28you have the kind of the the long
- 30:30term trending things that you wanna do. Like,
- 30:33this this all lends itself to trend analysis
- 30:35at scale within your environment. Like, it's definitely
- 30:38for the more, like, reactive stuff or maybe,
- 30:41like, more long term proactive.
- 30:43Like, I wanna track consumption
- 30:46of
- 30:48I I wanna track consumed
- 30:51size in my SharePoint sites versus, like, provision
- 30:54size and then, you know, track that. I
- 30:56wanna track number of users, number of share
- 30:58requests, blah blah blah,
- 31:00different user types. Like, maybe you're trying to
- 31:03make a transition from,
- 31:05like, old school
- 31:07SPNs over to, like, MSIs. Right? And and
- 31:10and do that kind of burn down and
- 31:11track that. So it's all, like, very well
- 31:13suited for that. It's also well suited for
- 31:16working around any of the limitations in the
- 31:19Graph API,
- 31:20where it just can't keep up with the
- 31:21polling rate, or you're just in such, like,
- 31:24a high churn environment
- 31:25that,
- 31:26you're gonna be subject to, like, API throttles
- 31:28or a bunch of other things that you
- 31:30don't want. So
- 31:31I I don't know that it's, like, the
- 31:33end all be all, but it's definitely
- 31:35complementary.
- 31:36Like, I could totally see having like, like,
- 31:38when I was looking at this, I could
- 31:39totally see having a dashboard that does something
- 31:41like tracking my identity consumption longer term and
- 31:44some of the dimensions around that and then
- 31:46having, like, a view in there that also
- 31:49ties into,
- 31:51you know, some of my more, like, proactive,
- 31:54real timey reports, like risky sign ins and
- 31:57things like that.
- 31:58And then you can merge all that together
- 32:00in a single view, a single report, a
- 32:01single world. Like,
- 32:03really cool stuff for being able to report
- 32:05just kinda, like, line of business or rhythm
- 32:08of business metrics
- 32:10around your consumption
- 32:11of
- 32:13these services that ultimately, like, have some hook
- 32:16or some data's
- 32:17data point into graph land. Yeah. And that
- 32:20was my thought with this because I've had
- 32:23clients come to me and say, okay. We
- 32:25have all these SharePoint sites. We have a
- 32:271000 SharePoint sites, 2,000 SharePoint sites, 3,000, whatever
- 32:31it may be,
- 32:32especially around sharing. Who is this shared with?
- 32:34How many sites do I have that have
- 32:36external sharing links?
- 32:38How many are shared with guest users? Like,
- 32:41to write a PowerShell script to do that,
- 32:431, like you said, you're gonna hit throttles,
- 32:45but, 2, it just it's not fast
- 32:48to loop through 3,000 SharePoint sites and pull
- 32:51that data. So to be able to dump
- 32:53this out and even provide regular reports
- 32:56every week, every month or so of how
- 32:59has that changed, what do we have out
- 33:00there,
- 33:02I see a lot of potential for this,
- 33:04especially around
- 33:06some of the SharePoint security.
- 33:08And I really do wanna go jump into
- 33:10some of the other datasets
- 33:12because
- 33:13there's a lot more datasets out there than
- 33:15I remember the last time I looked at
- 33:16this. Again, given that this has been around
- 33:18for a few years now. There's a whole
- 33:20bunch. There's a whole git repo that's full
- 33:22of thumbs.
- 33:23So I I think the other thing, like,
- 33:24really, like, you should be screaming in the
- 33:26back of your head when you're sitting down
- 33:28and and you have to, like, rationalize, like,
- 33:30hey, do I use something like this or
- 33:31do I just use, like, in your case,
- 33:33like, PowerShell and things like that?
- 33:36It's also just durability.
- 33:39Like,
- 33:40you
- 33:41know, I know, like, we like to talk
- 33:42a lot about, like, API surfaces being contracts
- 33:45and things like that. But the reality is,
- 33:47like, if you wrote a PowerShell script today
- 33:49that worked with the graph
- 33:51in its current instantiation,
- 33:53that that stuff could break, like, 5 minutes
- 33:55from now. Right? And then you're just left
- 33:57with the churn of having to
- 34:01figure that out and build it back to
- 34:02where it needs to be versus something like
- 34:04this where you can truly take a
- 34:07durable contract, like, hey. You're you're you're not
- 34:10only paying for the data,
- 34:11but, you know, it it's a fixed schema.
- 34:14It's a known thing. So you can take
- 34:16a little bit of a firmer dependency on
- 34:18it, like, on the order of, like, months
- 34:20to years
- 34:21versus what sometimes, like, really like, I I
- 34:24know it comes across as me being flippant,
- 34:25but it does feel like minutes sometimes
- 34:28in graph land, which is the way stuff
- 34:30changes or moves from underneath you. Absolutely.
- 34:32And I guess kinda to wrap it up,
- 34:35unless you need anything else, if you are
- 34:36looking to get started with this, we will
- 34:38throw this link in the share notes. And,
- 34:40again, I don't know how I missed this.
- 34:42This is a step by step, gather a
- 34:44detailed dataset on SharePoint sites using the Microsoft
- 34:47Graph Data Connector back from February of this
- 34:49year,
- 34:50where it does walk you step by step
- 34:53through setting up Microsoft Graph Data Connect,
- 34:57enabling the right
- 34:59features, services in Microsoft 365,
- 35:03setting up Azure. I know some Microsoft 365
- 35:05people, me included,
- 35:07maybe aren't super familiar with, I think, setting
- 35:11up like
- 35:13Synapse workspaces
- 35:14and how you would connect the data to
- 35:16Synapse and pull the data
- 35:18through, going through and building reports. Like, this
- 35:22is an end to end step by step
- 35:24walk through from
- 35:25turning the feature on to connecting it to
- 35:27Synapse
- 35:29to walking you through building a Power BI
- 35:31report.
- 35:32It is a fairly lengthy web page,
- 35:35but I would say a very good place
- 35:37to get started if you're not familiar with
- 35:39some of this stuff,
- 35:41just to see a sample of how you
- 35:43can start building this stuff out.
- 35:46Yeah.
- 35:47I saw this one. So so it exists
- 35:49in docs as well, in like the public
- 35:51docs, just not with the verbosity. Uh-huh. And
- 35:53I really wondered like, why doesn't this sit
- 35:56in docs? Because
- 35:57and it was the piece that I was
- 35:59missing when I first started going through it
- 36:00until you pointed out the article. It was
- 36:02like, hey, you you get to the end
- 36:04and, like, there's no good, like,
- 36:06how do you visualize all this stuff? Because
- 36:08ultimately, that's the point. Right? Like, we get
- 36:09to really decide, like, what's what's the
- 36:11consumption and distribution model look like, and is
- 36:14that Power BI? Is that Tableau? Is that
- 36:16Grafana? Like, who you know, whatever it happens
- 36:19to be.
- 36:20And then I was going through this article.
- 36:22I was like, oh, it's just because they
- 36:24went down the happy path of, like, everything
- 36:26the most expensive way they could. Right? Like,
- 36:29synapse here, ADF here, this this this, all
- 36:33these things, and all these things, like
- 36:35so so it's a happy path. Like, I
- 36:38will give them that 100%. Like, it it's
- 36:40it's a happy path. It works. Like,
- 36:42I can tell you, like, if you go
- 36:43through that article, like, it'll lead you into
- 36:46some bad practices,
- 36:48you know, around the way, like app registrations
- 36:50and things are configured. But whatever. Like, it
- 36:52works on the other side. Right? Like, it
- 36:53it does give you kinda, like, a point
- 36:55in time
- 36:56and get you where you need to be.
- 36:57But be mindful, like, as you're standing this
- 36:59stuff up, like, there might be other ways
- 37:01to do
- 37:02it. There could be less costly ways to
- 37:04do it. Right? Like, you might wanna consider,
- 37:06like,
- 37:07you know, do you really need Synapse?
- 37:10If you do need Synapse, like, what's the
- 37:12size of your Synapse compute that you're gonna
- 37:15pull in versus what's the size that's maybe
- 37:17in, like, the random article on TechCommunity or
- 37:20things like that. So just just keep that
- 37:22stuff in mind too.
- 37:25But I'll I'll tell you, like, I I
- 37:26was actually
- 37:27quite impressed. Like, this is, like,
- 37:30kudos because it it was very turnkey and
- 37:33and very easy to get going with. Absolutely.
- 37:36So I already see Pirate in the chat.
- 37:38He's like, you guys just filled up my
- 37:40weekend for me now.
- 37:41He's gonna be busy this this weekend playing
- 37:43with it as well. So very
- 37:46cool
- 37:47tool features
- 37:49that, like you said, I'm surprised
- 37:51it doesn't have it hasn't gotten more attention,
- 37:53visibility.
- 37:55So if you haven't seen it, played with
- 37:56it. It's probably worth a check checking it
- 37:58out and seeing what you can do, especially
- 38:01if you like numbers and reporting and analytics
- 38:04Absolutely. Or on Microsoft 365. It is for
- 38:06the data geeks. Get it get it going.
- 38:08Alright. Well, thanks, Scott. Appreciate it. Glad we
- 38:11were able to nail it and be on
- 38:13the same page with what this actually is,
- 38:15and hopefully bring some clarity and some visibility
- 38:18to it for those of you that have
- 38:20not played with it. Yeah. But with that,
- 38:22go play with data this weekend.
- 38:24Pull in a bunch of data, create a
- 38:25bunch of reports, and show me what you
- 38:27created next week. Homework. There we go. I'm
- 38:29yeah. I wanna see screenshots in Discord next
- 38:31week.
- 38:33Blur them all out. They'll just be blurry
- 38:35blobs of default Power BI colors. What we
- 38:37talked about, just abstract the data. Just mask
- 38:40usernames and all that and create some fun
- 38:43charts. Mhmm. We'll see what see what we
- 38:45can see in Discord beginning of next week.
- 38:48So hope everyone enjoys their weekend, and we
- 38:51will talk to you again soon, Scott. Great.
- 38:54Thanks, Ben.
- 38:56If you enjoyed the podcast, go leave us
- 38:59a 5 star rating in iTunes. It helps
- 39:01to get the word out so more IT
- 39:03pros can learn about Office 365 and Azure.
- 39:06If you have any questions you want us
- 39:07to address on the show, or feedback about
- 39:10the show, feel free to reach out via
- 39:12our website, Twitter, or Facebook.
- 39:15Thanks again for listening, and have a great
- 39:16day.