Latest / Cybersecurity Business with Fexingo: Security Companies, Breaches, and Enterprise Defense / How Security Teams Use Adversary Emulation to Test Defenses
Transcript
- Lucas: So it's episode 100 of this show, and I wanted to talk about something that I think gets less attention than it should — adversary emulation. Luna: As opposed to regular penetration testing or red teaming? Lucas: Exactly. Pen testing is usually about finding vulnerabilities — you scan for CVEs, you try to exploit a few, you write a report. Adversary emulation is different. It's about simulating a specific threat actor's behavior based on real intelligence. You're not just asking 'what's broken?' You're asking 'if a specific ransomware group targeted us, would our defenses actually stop them?' Luna: So it's more scenario-driven. Almost like a fire drill but for a particular attacker. Lucas: Right. And the gold standard here is the MITRE ATT&CK framework. It maps out every technique a real adversary might use — initial access, persistence, lateral movement, exfiltration. Adversary emulation uses those techniques as a checklist. You pick a group — say, the guys behind LockBit or BlackCat — and you build a test plan based on what they actually did in past breaches. Luna: I remember MITRE published a few emulation plans. Like for APT3, APT29… Lucas: Yes. They released what they call 'adversary emulation libraries.' They're free — step-by-step plans that walk you through how a specific threat group operates. So a security team can download the one for APT29 — which is widely believed to be a Russian state-sponsored group — and run through their techniques: spear-phishing, using PowerShell for execution, abusing Windows Remote Management for lateral movement. It's not theoretical. It's based on actual intrusions. Luna: And this is different from a traditional red team engagement where the red team just uses whatever tools they like? Lucas: Exactly. Red teams often have a goal — get to a specific server, exfiltrate a file — but how they do it might not resemble any real attacker. They might use a zero-day or a really creative chain that no actual adversary would ever use. That's fun for the red team, but it doesn't tell the CISO whether they're ready for the threats that actually keep them up at night. Adversary emulation is grounded in reality. Luna: I've heard CISOs say that pen tests are sometimes just a checkbox for compliance. 'We passed our pen test, we're good.' Adversary emulation seems harder to wave away. Lucas: Much harder. Because if you run a LockBit emulation and you find that your endpoint detection system doesn't alert on certain lateral movement techniques, that's a real gap. You can't just say 'well, the pen test didn't find it.' And that's the value — it's about readiness, not just vulnerability scanning. Luna: But there's a cost. Adversary emulation takes time and expertise. Not every team has a dedicated red team. Lucas: That's true. But the landscape is changing. A few years ago, you needed a specialized team to build these emulations from scratch. Now there are platforms — like AttackIQ, Verodin, Picus — that automate a lot of the process. You can select a threat group, and the platform will run the relevant techniques against your environment and tell you where your defenses fired and where they didn't. Luna: So it's becoming more accessible. Even for mid-sized companies. Lucas: Exactly. And that's key because the threat landscape isn't just for Fortune 500s. Ransomware groups target regional hospitals, school districts, manufacturing firms. Those teams often have fewer resources, so they need tools that give them high-fidelity testing without a full red team. Luna: Let's talk about a specific example. Do you have one where adversary emulation made a real difference? Lucas: Sure. There's a case from a couple years back — a large financial services company that had been through multiple pen tests and always passed. But they decided to do a LockBit emulation based on the threat intel reports from the FBI and CISA. The emulation revealed that their endpoint detection missed a specific technique where LockBit uses a legitimate Windows tool called BITSAdmin to download payloads. That's a living-off-the-land technique — no malware, no suspicious binaries. The EDR didn't flag it because BITSAdmin is trusted. Luna: So they thought they were protected, but they had a blind spot. Lucas: A huge one. They patched it — either by tuning the EDR to alert on BITSAdmin usage outside certain contexts, or by blocking it entirely in certain user groups. But if they hadn't run that emulation, they would have continued thinking they were solid. And maybe a real LockBit group would have gotten in that way. Luna: So it's not just about finding vulnerabilities. It's about testing your detection and response logic. Lucas: Exactly. That's the shift. Vulnerability management is important, but most breaches don't use zero-days. They use known techniques that could have been detected. Adversary emulation validates that your detection stack actually works against those techniques. Luna: And I imagine it also helps with tabletop exercises. You can take the findings to the board and say 'here's what would have happened if we were targeted by a specific group.' Lucas: Absolutely. CISOs love that. Instead of abstract risk numbers, you have concrete scenarios. 'If we were hit by BlackCat, our average detection time would be 48 hours based on our current controls. After we implement X, we can reduce that to 4 hours.' That's a board-friendly metric. Luna: So is adversary emulation replacing traditional red teaming entirely? Lucas: I don't think so. They serve different purposes. Red teaming is great for testing your overall security posture end to end, including human factors like social engineering. Adversary emulation is more focused on technical detection and prevention. Many mature organizations do both. But I think the trend is that adversary emulation is becoming a baseline expectation — especially since regulators and insurers are starting to ask about it. Luna: I've heard some cyber insurance carriers now require evidence of continuous testing, not just annual pen tests. Lucas: Yes. That's one of the drivers. If you can show that you're running adversary emulation on a regular basis, your premium might be lower. It's a concrete signal that you're serious about security. Luna: Let's talk about the challenges. What stops a company from doing this well? Lucas: The biggest one is skill. Even with automated tools, you need someone who understands the threat landscape and can interpret results. You can't just push a button and get a perfect report. You need to know which threat groups are relevant to your industry, which techniques they use, and how to prioritize the findings. That takes experience. Luna: And there's the false positive problem too. If you run a hundred techniques, and fifty trigger alerts, which ones are real gaps and which are just noise? Lucas: Right. That's where a good platform helps — it correlates alerts with specific detections and tells you whether the alert actually fired properly. But it's still a human judgment call. And that's why some organizations start with a consultant-led emulation, then build internal capability over time. Luna: Speaking of building capability — a lot of your listeners and viewers are in security operations. What's one practical step they can take this week to get started with adversary emulation? Lucas: I'd say download the MITRE ATT&CK emulation plan for a group that's relevant to your sector. If you're in healthcare, look at the one for FIN8 or the ransomware groups that hit hospitals. Map it against your environment. Even just a paper exercise — compare the techniques to your detection coverage — will reveal gaps. Then you can decide which gaps to address first. Luna: And it's free. That's the beauty of it. Lucas: Totally. MITRE puts this out as a public good. There's no excuse not to start. And if you want to go deeper, there are open-source tools like Caldera that automate the emulation. You can run it in a test environment without buying expensive software. Luna: Caldera is actually from MITRE too, right? Lucas: Yes. It's an adversary emulation platform that's free and open source. You install it, pick an adversary profile, and it runs the techniques automatically. It's not as polished as the commercial tools, but for a team that wants to understand the concept, it's perfect. Luna: Alright, so let me try to summarize: adversary emulation is about testing your ability to detect and respond to specific threat actors, using real intelligence. It's more targeted than pen testing, and it gives you actionable data about your defenses. Lucas: That's the essence. And I think it's one of the most underutilized practices in security right now. A lot of teams are still doing the same old pen test twice a year and calling it done. The threat actors are evolving faster than that. Luna: One thing I appreciate about this show is that we can have conversations like this without any ads interrupting. It's entirely listener-supported. If you value that kind of deep-dive content, consider tossing a few bucks at buy me a coffee dot com slash fexingo. It's what keeps this independent. Lucas: Yeah, we don't run ads here, and that's intentional. The support from listeners makes it possible to focus on the substance. So thanks to anyone who chips in. Luna: Alright, back to the topic. What do you think is the next evolution for adversary emulation? Where's it heading? Lucas: I think we'll see more integration with security operations centers — continuous emulation that runs in the background, constantly validating that new detections work and that changes to the environment haven't introduced blind spots. Some vendors are already calling it 'breach and attack simulation' and embedding it into their platforms. Over time, it'll become as routine as vulnerability scanning. Luna: So the goal is to get to a state where you know, at any given moment, how you would fare against a specific attacker. Lucas: Exactly. And that's the direction the industry is moving. It's not about passing a test once. It's about continuous readiness.