Latest / Cybersecurity Business with Fexingo: Security Companies, Breaches, and Enterprise Defense / Why CISOs Are Using Breach and Attack Simulation Daily
Transcript
- Lucas: If these conversations are useful for what you're building or running, you're in the right place. I'm Lucas. Luna: And I'm Luna. Episode 54. We're talking about Breach and Attack Simulation — or BAS — and why it's becoming a daily habit for security teams. Lucas: Yeah. And I want to anchor this with a real story from last month. A big financial services firm — I'm not naming them, but they're a top-ten bank by assets — they ran a BAS simulation and discovered that one of their endpoint detection rules had drifted during a routine CrowdStrike sensor update. Luna: Drifted how? Lucas: The rule was supposed to flag a specific process behavior associated with a known ransomware family — Ryuk variant. But a configuration file got overwritten in the update, and the rule stopped firing. For 72 hours, they were blind to that exact technique. A human wouldn't have caught it until the next quarterly pen test. The BAS tool hit it in 12 minutes. Luna: So BAS is basically automated red teaming that runs constantly. Lucas: Exactly. Traditional pen testing is point-in-time. You bring in a firm, they spend two weeks, you get a report, you fix the findings, and then you wait six months. In between, your environment changes. New software, new configs, new cloud instances. BAS fills that gap by running small, safe simulations — thousands of them — against your live environment every day. Luna: And it's not just about finding holes. It validates that your existing controls actually fire the way they're supposed to. Lucas: Right. That's the core insight. It's not a vulnerability scan. It's a control validation. You're asking: Does my EDR detect this? Does my firewall block this? Does my SIEM alert on this? And you get a score — a prevention rate, a mean time to detect — that you can track over time. Luna: I've heard CISOs say that BAS evidence is now a standard slide in their board deck. They show the trend line: 'Our detection rate went from 89% to 97% over the quarter.' That's concrete. Lucas: And it's becoming table stakes. A lot of cyber insurance carriers are starting to ask for BAS results during underwriting. If you can show continuous validation, you get better terms. Luna: So who are the main vendors in this space? Lucas: There are three that come up most often. Pentera — they were one of the first, strong on automated penetration testing and attack path simulation. Cymulate — they have a really broad attack library, and they integrate well with SOAR platforms. And AttackIQ — they're deeply tied to MITRE ATT&CK, and they have a free tier called AttackIQ Academy that a lot of teams use for training. Luna: And they all map to MITRE ATT&CK, right? That's the common language. Lucas: Yeah. Every simulation is mapped to a specific technique — T1055 for process injection, T1486 for data encryption, etc. So when a control fails, you know exactly which ATT&CK technique you're exposed to. That makes remediation much more targeted. Luna: But I imagine there's a risk of alert fatigue. If your BAS tool is firing hundreds of simulations a day, and some of them are expected to fail because you intentionally don't block everything... Lucas: That's a real concern. A good BAS platform lets you set baselines. You say, 'I know this technique isn't covered by my current stack — don't alert me unless the detection rate drops below 80%.' And you can tune out noise. But if you just turn it on without configuration, you'll drown. Luna: So it requires some maturity to adopt well. Lucas: Absolutely. You need a clear understanding of your control framework and your risk appetite. And you need someone who can interpret the results — not just fire the simulations. Luna: Quick honest thing — we're able to have these conversations without ads because a handful of listeners chip in monthly through Buy Me a Coffee. It's at buy me a coffee dot com slash fexingo. That's literally what funds making this many episodes. Lucas: And we keep it ad-free on purpose. So if you find value in the show, that's how you can keep it going. Back to BAS — one thing I find interesting is how BAS is starting to blur the line between red and blue teams. Luna: How so? Lucas: Traditionally, red team attacks, blue team defends, and they don't share much. But BAS gives both sides a common dataset. The red team can see which techniques the blue team is weak on. The blue team can see which simulations are triggering and tune their responses. It becomes a collaborative feedback loop. Luna: Is there any pushback from pentesters who feel like BAS might replace them? Lucas: I've heard some concern, but most pentesters I talk to see BAS as a complement. BAS is great for breadth — running thousands of known techniques. But human pentesters are still better at depth — chaining multiple techniques in novel ways, exploiting business logic flaws, social engineering. BAS doesn't replace that. Luna: So you'd still do an annual human-led pen test, but you run BAS in between. Lucas: Exactly. And some organizations are moving to more frequent human testing — quarterly or even monthly — but BAS fills the gaps daily. Luna: What about cloud environments? Does BAS work for AWS or Azure? Lucas: Most modern BAS platforms have cloud-specific modules. They can simulate attacks on S3 buckets, IAM role chaining, Kubernetes cluster escapes. And they run inside your cloud tenant, so they don't generate internet-facing traffic. Luna: That's important for avoiding false positives with your cloud security monitoring. Lucas: Exactly. You don't want your own BAS tool tripping your cloud detection and causing an incident response drill. Luna: Let's talk about a specific metric. Mean time to detect — MTD. How does BAS improve it? Lucas: Great question. BAS can simulate a known attack and measure exactly how long it takes for your SIEM or EDR to generate an alert. Then you track that over time. One healthcare organization I read about reduced their MTD from 4 hours to 22 minutes over six months by running BAS weekly and tuning their detection rules after each simulation. Luna: That's a huge improvement. And it's measurable, which makes it easy to report to leadership. Lucas: Right. And that's the whole value proposition. BAS turns security from a cost center into a data-driven function. You can show: 'We blocked 94% of simulated attacks this month. Last month it was 91%. Here's the trend.' That's powerful. Luna: One concern I've heard is that BAS only tests known techniques. What about zero-days? Lucas: It's a valid point. BAS is based on known TTPs — tactics, techniques, and procedures. It won't catch a brand new exploit. But the reality is that most breaches still use known techniques. The Verizon DBIR consistently shows that a huge percentage of attacks leverage techniques that are years old. So covering the known stuff well is still a huge win. Luna: And if you want to test for zero-day-like behavior, you'd need a human red team or a more advanced simulation. Lucas: Exactly. Or you can use BAS to test your detection of the post-exploitation phase — even if the initial vector is unknown, how quickly do you detect lateral movement or data exfiltration? Luna: That's smart. So BAS gives you a baseline for your detection capabilities, and then you layer on other tools for the unknowns. Lucas: Precisely. And I think that's where the industry is heading — a layered validation approach. BAS for continuous validation, pen tests for depth, and purple team exercises for collaboration. Luna: I want to touch on the operational side. How much time does it take to run a BAS platform daily? Lucas: Once it's configured, it's mostly automated. You schedule simulations during off-peak hours, review the reports in the morning, and address any failing controls. A dedicated analyst might spend 30 minutes a day on it. The initial setup takes more — maybe a week to map your environment and tune the baselines. Luna: So it's not a huge resource drain for a mid-sized security team. Lucas: No, it's pretty manageable. And the ROI is clear when you catch something like that drifted rule — a potential ransomware outbreak that could cost millions. Luna: What about false positives from BAS itself? If a simulation looks real enough to trigger your SOC... Lucas: That's a real risk. You need to make sure your BAS tool is whitelisted in your detection systems. Otherwise, you'll get analysts investigating a fake attack. Most platforms have integration with SOAR to automatically suppress alerts from known BAS sources. Luna: So there's some operational overhead to set that up correctly. Lucas: Yes, but it's a one-time effort. And most vendors provide documentation on how to configure the whitelisting. Luna: Last question: for a CISO considering BAS today, what's the first step? Lucas: I'd start with a pilot on a non-critical segment — like a dev environment. Run simulations against a few key controls, like endpoint detection and email filtering. See what the data looks like, get comfortable with the reports. Then expand to production slowly. Luna: And measure the MTD improvement from the start. Lucas: Exactly. That way you have a baseline to show improvement. BAS is one of those tools that, once you have it, you wonder how you operated without it. Luna: I think that's a good note to end on. Thanks for the deep dive. Lucas: Thanks, Luna. And thanks to everyone listening. We'll be back with another episode soon.