Latest / Good Morning BT with Bo Thompson & Beth Troutman / Theresa Payton: Worldwide Microsoft Outage latest
Transcript
- 0:00Speaker 1: Hey, guys.
- 0:00Speaker 2: How are you?
- 0:01Speaker 3: Good.
- 0:01Speaker 1: What's going on?
- 0:02Speaker 4: Y'all are my unassociated best friends. You drive with me
- 0:05Speaker 4: to work every day.
- 0:06Speaker 2: Oh, thanks.
- 0:06Speaker 4: This is Good Morning BT. 107.9 WBT.
- 0:15Speaker 1: I may have been the last local user here at
- 0:19Speaker 1: One Julian Price Place to send an email yesterday that worked.
- 0:22Speaker 2: I 100% think you were.
- 0:24Speaker 1: Because the show ended and Beth and I went in
- 0:26Speaker 1: the studio and we had to record a commercial that
- 0:29Speaker 1: she voices And I had to send it, you know,
- 0:32Speaker 1: to our sales staff to process it once we were done.
- 0:35Speaker 4: And a lot of times I'll go home and do that.
- 0:38Speaker 1: And I came, for whatever reason, I walked in here,
- 0:40Speaker 1: was talking to Bernie, and I sat down and said,
- 0:42Speaker 1: you know what, I'll do it here. And I sent it,
- 0:45Speaker 1: and then I got in the car. And when I
- 0:46Speaker 1: got in the car, shortly after that, I tried to
- 0:48Speaker 1: check my email, and it wasn't working.
- 0:50Speaker 4: And I kind of said, oh, I'll check it again later.
- 0:52Speaker 4: And then it turned into what it turned into. But
- 0:54Speaker 4: that email that I sent went through. Yes.
- 0:57Speaker 1: You know, who knows when I'll be able to send
- 0:59Speaker 1: the next one. Let's go to the WBT hotline right
- 1:02Speaker 1: now and welcome a special cameo Friday. But we knew
- 1:06Speaker 1: this was going to happen as soon as this story
- 1:07Speaker 1: started lingering. Our cybersecurity expert, founder of Fortalus Solutions, and
- 1:11Speaker 1: really the person that we go to in times like
- 1:14Speaker 1: this to help us make sense of what's going on.
- 1:16Speaker 4: Teresa, good Friday morning.
- 1:18Speaker 1: I don't think there's ever been one of these outages
- 1:21Speaker 1: that we have been talking about with you that's gone
- 1:24Speaker 1: on this long, or at least I can't remember one
- 1:26Speaker 1: in recent memory.
- 1:28Speaker 4: No, as a matter of.
- 1:29Speaker 3: Fact, I went back and did some research. And so
- 1:32Speaker 3: 2024 is when CrowdStrike rolled out a production update that
- 1:38Speaker 3: impacted Microsoft Windows machines. It disrupted approximately 8.5 million Windows
- 1:44Speaker 3: devices worldwide, including Delta Airlines. Recoveries took days for some. But,
- 1:51Speaker 3: you know, it only impacted sort of this CrowdStrike Microsoft setup.
- 1:56Speaker 3: So it wasn't a complete and total outage. And I
- 1:58Speaker 3: had to go back in time. Amazon Web Services had
- 2:01Speaker 3: some big outages, but that was like 2021, 2017. Google
- 2:06Speaker 3: Cloud had one in 2020 that was a big outage.
- 2:09Speaker 3: I can't remember anything this big.
- 2:13Speaker 2: And you were saying, because we have been in communication
- 2:16Speaker 2: with you, and media sources are saying this as well,
- 2:20Speaker 2: that this is not a quote-unquote cyber attack. So in
- 2:25Speaker 2: your view, how does something this enormous happen without there
- 2:30Speaker 2: being some kind of nefarious outside interference?
- 2:34Speaker 3: Yeah, I'm very concerned. So I've run through a couple
- 2:38Speaker 3: possibilities in my mind. And one thought that I have
- 2:41Speaker 3: had is they must have seen a calamity of sorts
- 2:46Speaker 3: of some type of what's called a zero day, a
- 2:49Speaker 3: huge vulnerability. Because typically when you push out a production
- 2:53Speaker 3: fix to something, you typically do it in a small group.
- 2:57Speaker 2: You see if it works.
- 2:59Speaker 3: And if it doesn't have a catastrophe, you know, doing
- 3:02Speaker 3: a patch, a security patch, you then roll it out
- 3:05Speaker 3: to everybody else. It sounds like something was pushed out
- 3:07Speaker 3: to everything in a hurry. And Microsoft, they're kind of
- 3:12Speaker 3: saying it in their sort of techie, geeky updates that
- 3:15Speaker 3: they're putting out every once in a while. It sounds
- 3:18Speaker 3: like something got pushed to production globally and it is
- 3:22Speaker 3: having a major impact and they're struggling to figure out
- 3:24Speaker 3: how to roll it back. And then also people are trying,
- 3:28Speaker 3: they're hitting enter and they're trying again. And so every
- 3:31Speaker 3: time they bring a server online, it's basically struggling and
- 3:34Speaker 3: going back offline. So Is it a cyber attack? I
- 3:38Speaker 3: don't know. Obviously, the investigation will have to continue. But
- 3:42Speaker 3: for right now, it is still down. And this is
- 3:45Speaker 3: over 12 hours running. And it is a global disruption.
- 3:50Speaker 3: Seventy-four countries and territories have complained that they don't have access.
- 3:55Speaker 3: And obviously, in Europe right now, they're trying to work.
- 3:57Speaker 3: And if you're 100 percent a Microsoft shop, you're not
- 4:01Speaker 3: getting any work done today.
- 4:03Speaker 1: Well, we talk about things like email, but, you know,
- 4:06Speaker 1: so many businesses affected by this. The National Pharmacy Association
- 4:09Speaker 1: warning of delays to prescriptions. Police cannot receive crime reports
- 4:14Speaker 1: online and telling people to call. You know, for some people,
- 4:19Speaker 1: it was a minor inconvenience yesterday and it extends into today.
- 4:23Speaker 1: But then you know, businesses. I mean, we're, our entire
- 4:27Speaker 1: communication platform here at OneJulian, we had to, our boss
- 4:32Speaker 1: sent out a Gmail address yesterday as a backdoor option, essentially,
- 4:36Speaker 1: to try to do some business, but you can't have,
- 4:38Speaker 1: you know, Microsoft Teams that are so prolific in businesses now.
- 4:43Speaker 4: All of this has come to a screeching halt.
- 4:46Speaker 3: Yeah, you know, it's interesting. So I was asking, I
- 4:48Speaker 3: was talking to my cloud assistant this morning, And I
- 4:51Speaker 3: asked it to do a back of the napkin outage,
- 4:55Speaker 3: like what is the financial impact of this? And it
- 4:57Speaker 3: said back of the napkin about $ 28 billion in lost
- 5:00Speaker 3: productivity for yesterday. So, you know, and I asked it
- 5:05Speaker 3: to show me its sources. So it went through a
- 5:07Speaker 3: lot of the kind of the Microsoft numbers and some
- 5:10Speaker 3: sources at Tech Republic and a couple other places. And
- 5:14Speaker 3: it got into a lot of, like, different techie things.
- 5:16Speaker 3: But that's sort of the bottom line up front is
- 5:18Speaker 3: about $ 28 billion in lost productivity, according to my Claude assistant.
- 5:23Speaker 2: Well, this is just another reminder. I think we all
- 5:27Speaker 2: take for granted how much we rely on technology for business,
- 5:31Speaker 2: for financial transactions, for, you know, purchases. We have become
- 5:36Speaker 2: so dependent upon it that we don't necessarily, and this
- 5:40Speaker 2: is really highlighting it, we don't have backup plans for
- 5:45Speaker 2: basic business but basic life.
- 5:48Speaker 4: Yes, exactly.
- 5:49Speaker 3: I mean, here we are. We're going into the holiday
- 5:52Speaker 3: shopping season, too. So you probably have a lot of
- 5:55Speaker 3: small businesses that are trying to fulfill orders. And how
- 5:58Speaker 3: do they get those orders? If they're not on Google,
- 6:00Speaker 3: they're probably on Microsoft. If you think about getting your
- 6:04Speaker 3: delivery status alerts, coupons and things like that. So it's
- 6:07Speaker 3: not just the business side of things for a lot
- 6:09Speaker 3: of people. It's also the personal side of things. So,
- 6:13Speaker 3: you know, more to come. I mean, honestly, my thoughts
- 6:16Speaker 3: and prayers go out to all my friends at Microsoft
- 6:18Speaker 3: and all the engineers who have been working tirelessly to
- 6:21Speaker 3: try and figure this out. But what a difficult set
- 6:27Speaker 3: of problems they are dealing with right now.
- 6:30Speaker 1: Well, and we're talking about the fact that they have
- 6:33Speaker 1: obviously tried to fix things, but it keeps crashing. Many
- 6:36Speaker 1: of these servers, when they rebuild them or get them reconnected, crash.
- 6:40Speaker 1: And the idea that so many people are trying to
- 6:43Speaker 1: log back in at once, you know, I would assume
- 6:46Speaker 1: that becomes an issue too, just the amount of people.
- 6:48Speaker 1: Because yesterday I heard, you know, a down detector that
- 6:52Speaker 1: the website you've always told us about, and a lot
- 6:54Speaker 1: of these ways they measure if there's an outage. It's
- 6:57Speaker 1: usually way more than the numbers indicate because those are
- 7:01Speaker 1: just the people that report. There's so many other people
- 7:04Speaker 1: like you and me that didn't officially make a report
- 7:07Speaker 1: yesterday but are dealing with the problems here. So I
- 7:11Speaker 1: I worry about how long this might last, and I
- 7:13Speaker 1: guess maybe that's the last question for you. What's your
- 7:15Speaker 1: gut tell you as to how long this could last?
- 7:18Speaker 1: I mean, could this go several more days, or do
- 7:20Speaker 1: you think they get this somehow fixed today?
- 7:23Speaker 3: They seemed fairly optimistic in their last post that they
- 7:30Speaker 3: thought they had a fix, and they were testing it.
- 7:34Speaker 3: My gut tells me it's not going to feel like
- 7:38Speaker 3: things are back to normal until sometime on Monday. That's
- 7:42Speaker 3: what my gut tells me, having lived through major outages
- 7:46Speaker 3: and trying to deal with major outages and having resiliency
- 7:51Speaker 3: and getting things back up and running. Microsoft still hasn't
- 7:54Speaker 3: confirmed the root cause. They have a hunch, according to
- 7:57Speaker 3: what they've told all of us. And so that means
- 8:01Speaker 3: they're troubleshooting right now to try and get things back
- 8:04Speaker 3: up and running. There's one other concern that I have.
- 8:08Speaker 3: So a lot of people don't realize like there's, it'll
- 8:11Speaker 3: keep trying and the time set for trying a message
- 8:15Speaker 3: that let's say, Bo, you send something to somebody, but
- 8:18Speaker 3: if they haven't received it and opened it and their
- 8:20Speaker 3: server was down when you sent that, that commercial, if
- 8:25Speaker 3: they didn't actually receive it because the outage was going on,
- 8:29Speaker 3: It will keep trying to deliver it and then it'll
- 8:31Speaker 3: time out and it'll send you an undeliverable message. But
- 8:35Speaker 3: we've never had a situation that I can remember where
- 8:39Speaker 3: both sides of the equation were hard down for a
- 8:42Speaker 3: long time. And so in my head, I'm doing the calculus.
- 8:46Speaker 3: I'm thinking there could be emails that you thought were
- 8:49Speaker 3: sent that you'll never get an undeliverable message and it'll
- 8:53Speaker 3: never make it and you'll never be able to find it.
- 8:57Speaker 3: So what I would say to everybody is whatever it
- 8:59Speaker 3: is you did on Thursday before things went hard down,
- 9:02Speaker 3: you need to go figure out what you did, and
- 9:04Speaker 3: you're probably going to have to do it again. It
- 9:06Speaker 3: might be lost forever.
- 9:07Speaker 2: You know, I know we're up against the clock here,
- 9:09Speaker 2: but I have one really quick question for you, and
- 9:12Speaker 2: you may or may not be able to answer it,
- 9:14Speaker 2: but in your gut with all of your experience with cybersecurity,
- 9:19Speaker 2: do you believe that there may be some undetectable problems
- 9:23Speaker 2: cyber attack that has happened on the back end of this,
- 9:25Speaker 2: that that's why the servers keep crashing. Is there something
- 9:28Speaker 2: about this that doesn't feel right to you?
- 9:30Speaker 3: Yeah, I think all things are on the table, Beth.
- 9:33Speaker 3: So it is possible. I think they were up against
- 9:36Speaker 3: the clock to fight against a vulnerability. So the patch
- 9:40Speaker 3: might have gone in and might have been flawed. But
- 9:42Speaker 3: I think all things are on the table right now.
- 9:44Speaker 3: There is an open DHS alert about a zero-day vulnerability
- 9:49Speaker 3: with Microsoft. I have no idea if that is related
- 9:51Speaker 3: to what's going on here. But I think until the investigation, well,
- 9:55Speaker 3: restoration will be first. Investigation will be finalized post-restoration. And
- 9:59Speaker 3: until we know more, all options are on the table,
- 10:02Speaker 3: including that.
- 10:03Speaker 1: Bottom line, this weekend, if you want to communicate with someone,
- 10:08Speaker 1: you may have to call them.
- 10:09Speaker 4: You may even have to go see them. All right.
- 10:18Speaker 1: Teresa, we will stay in touch as this hopefully doesn't
- 10:21Speaker 1: continue too much longer. But either way, thank you so
- 10:23Speaker 1: much for coming on Friday morning.
- 10:25Speaker 4: Absolutely.
- 10:25Speaker 3: I've been encouraging people to do things analog, but this
- 10:28Speaker 3: is not the way I would go about doing it.
- 10:30Speaker 3: So everybody be safe out there. Great to be with you,
- 10:33Speaker 3: Beth and Bo.