Latest / The Best of the Money Show / The Money Show Explainer: Local companies grapple with Microsoft service disruption
Transcript
- 0:00Speaker 1: Let's try and get a little bit more then on
- 0:01Speaker 1: what caused the Microsoft outages. We heard your frustration earlier.
- 0:05Speaker 1: You heard the impact it's had on people and anyone
- 0:07Speaker 1: trying to be productive around the country, but around the
- 0:10Speaker 1: world today. Jan Vermeulen, the editor at mybroadband.co.za. Jan, good evening.
- 0:15Speaker 1: People pay Microsoft a fortune because it's supposed to always work.
- 0:20Speaker 1: What do we know about what went wrong? It seems
- 0:22Speaker 1: to have been update after communication after problem after server
- 0:25Speaker 1: patch not taking.
- 0:28Speaker 2: Good evening, yes. This is going to be something that's
- 0:31Speaker 2: going to haunt Microsoft, I think, because it's exactly what
- 0:34Speaker 2: you described there. People have given up hosting their own
- 0:38Speaker 2: Microsoft Exchange infrastructure because these are the kinds of problems
- 0:42Speaker 2: we used to have in the 1990s when we all
- 0:45Speaker 2: used to run our own mail servers. And we now
- 0:48Speaker 2: pay these big companies so that we don't have to
- 0:50Speaker 2: worry about it. So yes, we don't yet know exactly
- 0:55Speaker 2: what caused the problem. Microsoft said that an automated server
- 1:01Speaker 2: health alert triggered essentially a cascading failure, not their words, mine,
- 1:09Speaker 2: where servers that this automated health check had obviously flagged
- 1:14Speaker 2: as in a critical state rerouted traffic from those servers
- 1:19Speaker 2: to other servers, overwhelming the backup servers with traffic, and
- 1:25Speaker 2: causing the whole system to come crashing down around their ears. Now,
- 1:29Speaker 2: what triggered that initial health check problem, we don't know.
- 1:34Speaker 2: But it seems Microsoft kind of alluded to it earlier
- 1:37Speaker 2: in the investigation, in the updates they put out, that
- 1:39Speaker 2: it might have been a network or infrastructure configuration issue.
- 1:44Speaker 2: But everything they mentioned after that about how they patched
- 1:48Speaker 2: it didn't seem to actually fix it. And it took
- 1:51Speaker 2: them hours from those initial messages saying, we think this
- 1:56Speaker 2: is a configuration issue, we're reverting it, to when the
- 2:01Speaker 2: systems were actually restored and stable. We're talking about a
- 2:04Speaker 2: six-hour gap. So, yeah, it might not have been that
- 2:08Speaker 2: at all, but we do know that something caused a
- 2:13Speaker 2: cascading infrastructure failure at Microsoft, and it's linked to this
- 2:17Speaker 2: automated server health check thing of theirs.
- 2:21Speaker 1: Isn't the issue that so many people are dependent on
- 2:25Speaker 1: one thing, but there are also so many different services? So,
- 2:27Speaker 1: I mean, people, I know some people who cannot get
- 2:30Speaker 1: through their day, can't even start their day without Copilot.
- 2:33Speaker 1: That's one thing. But Teams seems to have been affected
- 2:37Speaker 1: as well. It seems to have been a whole host
- 2:40Speaker 1: of services, which I presume are all using the same infrastructure.
- 2:43Speaker 1: And isn't that the risk?
- 2:45Speaker 2: Yes, so mercifully, Teams and Copilot came back pretty soon,
- 2:52Speaker 2: interestingly enough. So that infrastructure was not nearly as badly
- 2:55Speaker 2: affected as the email infrastructure. And so that's hosted Exchange, Outlook, Webmail,
- 3:01Speaker 2: that sort of stuff was offline for most of the day,
- 3:05Speaker 2: while Microsoft did manage to get their other stuff back
- 3:09Speaker 2: online a whole lot faster. So, yes, that is a
- 3:13Speaker 2: good point, though, is perhaps it's something for people to
- 3:16Speaker 2: worth checking to not put all their eggs in one basket. So,
- 3:20Speaker 2: you know, asking people to have multiple email providers is
- 3:23Speaker 2: perhaps a bridge too far. And asking people to self-host
- 3:27Speaker 2: again is just a disaster in the making. But perhaps
- 3:32Speaker 2: to go, OK, if I'm using Google for my email,
- 3:35Speaker 2: then let me use Microsoft for this or vice versa.
- 3:39Speaker 1: And then, I mean, you would think there are plenty
- 3:41Speaker 1: of other meetings apps. I mean, you've got Zoom and
- 3:44Speaker 1: Google Meet. You would be able to move to one
- 3:46Speaker 1: of those quite quickly. I suppose the problem is, is
- 3:49Speaker 1: everyone's configured one way. And for a lot of corporates,
- 3:51Speaker 1: they use Microsoft Teams and they use it for all
- 3:54Speaker 1: sorts of important reasons that are quite important to them.
- 3:58Speaker 2: Exactly. It's got to do with a Microsoft ecosystem integration.
- 4:02Speaker 2: It might be that certain corporates even at certain levels
- 4:06Speaker 2: of the organization. So basically your IT and more techie
- 4:11Speaker 2: employees might have more freedom on their laptops, but everybody
- 4:14Speaker 2: else is locked down tight. They can't install anything other
- 4:17Speaker 2: than what IT approves. And for good reason, because we
- 4:20Speaker 2: live in a security hellscape at the moment. And so
- 4:24Speaker 2: there's a lot of good reasons why it might not
- 4:29Speaker 2: be easy for a corporate to just go, oh, well,
- 4:32Speaker 2: quickly install Zoom and let's keep going. because they have
- 4:37Speaker 2: to make a security determination. And for smaller companies, that'll
- 4:41Speaker 2: be quite easy to do. But for a large organization,
- 4:44Speaker 2: that's not something you just pivot to in the matter
- 4:47Speaker 2: of minutes.
- 4:48Speaker 1: And I mean, this day, when you allow everyone once
- 4:51Speaker 1: off to download Zoom is the day that two years
- 4:53Speaker 1: from now, you realize when whatever it was you were
- 4:56Speaker 1: trying to keep out did get in and caused the hack, etc.
- 4:58Speaker 2: etc., etc.
- 5:01Speaker 1: So what does this mean for Microsoft over the longer term?
- 5:05Speaker 1: It is a reminder, and what a week for the
- 5:08Speaker 1: tech industry it's been of this, of concentration risk.
- 5:13Speaker 2: Exactly. And that's probably going to be the thing that
- 5:17Speaker 2: Microsoft has to manage here, is corporates going, perhaps my
- 5:22Speaker 2: entire budget for this should not go to Microsoft, and
- 5:26Speaker 2: I should diversify it to someone else so that when...
- 5:30Speaker 2: The chips are down when something breaks. I'm not completely
- 5:37Speaker 2: all in on one provider. And so Microsoft's going to
- 5:41Speaker 2: have its work cut out for it and its salespeople
- 5:43Speaker 2: and its account managers and so on are going to
- 5:45Speaker 2: have their work cut out for them to convince their
- 5:48Speaker 2: customers to not go looking for alternatives. I don't want
- 5:55Speaker 2: to say this is something that could happen to anybody,
- 5:57Speaker 2: but if you go to an alternative email provider now,
- 6:04Speaker 2: they could be down two weeks from now. When you
- 6:10Speaker 2: make these decisions, you are evaluating it based on the
- 6:13Speaker 2: company's reputation, and Microsoft's reputation took a big knock today.
- 6:18Speaker 1: If you look at the history of how companies got
- 6:20Speaker 1: here with Microsoft, It goes back a long time. You're
- 6:23Speaker 1: looking at the adoption of Microsoft Windows. You're looking actually
- 6:27Speaker 1: at the introduction of the PC era and MS-DOS. Just
- 6:31Speaker 1: old enough to remember MS-DOS 2.11. And the reason I
- 6:35Speaker 1: say that is without that long history, would so many
- 6:39Speaker 1: corporates be using Microsoft products now?
- 6:42Speaker 2: No. South Africa is planet Microsoft now. for a variety
- 6:47Speaker 2: of reasons, and one of them is the sort of
- 6:52Speaker 2: open architecture and the fact that during the sanction years
- 6:55Speaker 2: we were able to access PCs that could run DOS
- 7:03Speaker 2: and eventually Windows, which was probably smuggled in, but getting
- 7:07Speaker 2: an Apple in the country was all but impossible, and
- 7:10Speaker 2: similarly for other technology providers. And so South Africa became
- 7:15Speaker 2: quite dependent or at least quite invested, I should say,
- 7:19Speaker 2: in Microsoft. And so, yeah, when it comes to South Africa,
- 7:24Speaker 2: Microsoft is deeply, deeply entrenched in our corporate culture.
- 7:27Speaker 1: I'd never thought of it like that. Thank you for that.
- 7:29Speaker 1: Jan Vermeulen, editor at mybroadband.co.za.