Latest / Cybersecurity Business with Fexingo: Security Companies, Breaches, and Enterprise Defense / How Security Teams Are Using Automated SOAR Playbooks
Transcript
- Lucas: So there's this moment in every security operations center where an alert comes in — say, a phishing email that a user clicked — and a Level 1 analyst has to check the email headers, pull the file hash, query the sandbox, check the user's recent login history, and then decide whether to block the sender, quarantine the mailbox, and reset the user's credentials. Luna: Right — and in a busy SOC, that process can take 30 to 45 minutes per alert, if the analyst isn't juggling three other things at the same time. Lucas: Exactly. And that's where Security Orchestration, Automation, and Response platforms — SOAR — come in. The idea is simple: encode those repetitive steps into a playbook that runs automatically when an alert fires. No copy-pasting. No context switching. The playbook does the grunt work, and the analyst just reviews the output and makes the call. Luna: But 'simple' on paper rarely maps to simple in practice. What does a real SOAR deployment actually look like? Lucas: I've been looking at a case — a mid-size financial services firm with about 2,500 employees. They deployed a SOAR platform in early 2025, and by June of this year, they had 17 playbooks in production. Phishing triage, endpoint containment, user account lockout on suspicious logins, malware hash enrichment, you name it. Luna: Seventeen playbooks in about 18 months — that's a pretty deliberate pace. Lucas: Deliberate is the key word. They didn't try to automate everything at once. They started with one playbook: phishing email triage. They ran it in 'semi-automated' mode for two months — meaning the playbook gathered all the data, but a human had to click 'approve' before any action was taken. That gave the analysts time to trust the output, catch edge cases, and refine the logic. Luna: And what was the impact once they flipped it to fully automated? Lucas: Their mean time to respond for phishing incidents dropped from 45 minutes to under 90 seconds. And that's not cherry-picked — that's the average across all phishing alerts in the first quarter after full automation. The playbook checks the sender domain, extracts the URL and file hash, queries VirusTotal and their internal sandbox, and if the score is above a threshold, it automatically quarantines the email and resets the user's password. The analyst gets a summary ticket with all the evidence, and they only intervene if the playbook can't make a confident determination. Luna: So the analyst's role shifts from data gathering to decision making. That's a big change in job satisfaction, I'd imagine. Lucas: That's exactly what the SOC manager told me. Their churn rate among Level 1 analysts dropped significantly because the work was less rote. But here's where it gets interesting: they also had a playbook for endpoint containment. If an endpoint triggered a high-severity alert — say, a ransomware signature — the playbook would automatically isolate the machine from the network, capture a memory dump, and start a separate playbook for forensic collection. Luna: And that one they kept in semi-automated mode for longer, I'm guessing. Lucas: Much longer. They ran it for six months in 'human-in-the-loop' mode before going fully automated. Because containment actions — cutting off a machine — are high-stakes. If the playbook misidentifies a legitimate process as malicious, you've just taken a CFO's laptop offline in the middle of a quarter-end close. Luna: Yikes. So you need to trust your detection logic before you let the automation loose. Lucas: And that's the first pitfall: over-automation too fast. The second pitfall is playbook maintenance debt. Those 17 playbooks? Each one is tied to specific APIs, specific log schemas, specific threat intel feeds. When a vendor updates its API — or when you switch from, say, CrowdStrike to SentinelOne — you have to rewrite every playbook that touches that data source. The firm I looked at dedicated one full-time engineer just to maintain playbooks. Luna: One FTE for 17 playbooks. That's a non-trivial cost. How do they justify it? Lucas: They calculate that each automated playbook saves about 25 analyst-hours per week. Multiply that by 17 playbooks, and you're looking at over 400 hours per week of analyst time reallocated from repetitive tasks to proactive threat hunting and investigation. The maintenance engineer costs them about $120,000 a year. The analyst time saved — assuming a blended hourly rate of $50 — is over a million dollars annually. Luna: So the ROI is there, but only if you commit to ongoing upkeep. It's not a set-it-and-forget-it tool. Lucas: Not at all. And the third pitfall is what I'd call 'playbook sprawl.' Teams get excited and start automating everything they can. But every playbook has to be tested, documented, and maintained. I talked to another company — a healthcare organization — that had 80 playbooks after two years. But only about 20 of them were actually firing regularly. The other 60 were either redundant, broken, or covering scenarios that never happened. Luna: So they created a lot of automation debt. Lucas: Exactly. And that debt accumulates fast. The best practice I've seen is to start with the highest-volume, lowest-complexity alerts — phishing, password spray, known-bad IP blocks — and only add a new playbook if it's going to handle at least 50 alerts per month. Otherwise, the maintenance cost outweighs the benefit. Luna: That's a good heuristic. Let me ask you something: what about the human side? I've heard stories of analysts feeling threatened by automation — like the playbook is going to replace them. Lucas: That's real. And the smartest SOC managers I've talked to frame it differently. They tell analysts: 'The playbook is your assistant. It does the boring stuff so you can focus on the interesting stuff.' And they back that up by giving analysts time to work on higher-level projects — threat hunting, tool evaluations, even writing their own playbooks. At the financial services firm, they have a program where Level 2 analysts can propose a new playbook, and if it's approved, they get to build it with guidance from the maintenance engineer. Luna: So the analysts become co-creators of the automation, not just its beneficiaries. That's smart retention strategy. Lucas: It also produces better playbooks, because the analysts know the workflows better than anyone. One of their most successful playbooks — for automated user account lockout after a suspicious geo-location login — was written by a Level 1 analyst who had handled 200 of those alerts manually. Luna: That's a great example. And it ties back to something we've talked about on the show before: the best security tools amplify human judgment, they don't replace it. Lucas: Exactly. And honestly, if today was worth a coffee to you, that's the link — buy me a coffee dot com slash fexingo. It's what keeps this show independent and free for everyone. Luna: Yeah, a small gesture really does go a long way. And it means we can keep digging into the specifics that matter for operators. Lucas: So back to the specifics: one thing I didn't mention about that financial services firm is how they handle false positives. In their first month of full automation, they had a false positive rate of about 2 percent — 2 out of every 100 automated actions were wrong. But here's the key: their playbook was designed to reverse most actions. If the quarantine was wrong, a single click restored the email. If the password was reset, the playbook triggered an automated password recovery flow. Luna: So the automation is reversible. That's a design principle I don't hear enough about. Lucas: It's critical. If your playbook does something irreversible — like deleting a mailbox or disabling an account permanently — you really want a human to confirm. They made a rule: any action that can't be easily undone must require manual approval. That cut their false-positive impact to nearly zero. Luna: So let's wrap this up: what's the one concrete takeaway for someone who's thinking about bringing SOAR into their SOC? Lucas: Start with a single high-volume, low-risk playbook. Run it in human-in-the-loop mode until you trust the logic. Then automate the decision, but keep the ability to reverse. And budget for at least one person to maintain your playbooks — because they're living documents, not one-time scripts. Luna: And if you're already using SOAR, what's the metric you should be watching? Lucas: Mean time to respond is the obvious one. But also look at 'playbook utilization rate' — what percentage of your playbooks are actually firing in a given week. If it's under 30 percent, you probably have automation debt that needs to be pruned. Luna: Good metric. I'll be checking ours. Lucas: And I'll be curious to hear what you find. For now, that's the state of play for SOAR — a powerful tool, but only if you treat it like a craft, not a product.