Latest / Tech Leadership with Fexingo: Engineering Managers, CTOs, and Technical Leadership Conversations / How One CTO Uses Blameless Postmortems to Build a Learning Culture
Transcript
- Lucas: Luna, I want to talk about something that sounds obvious but is surprisingly hard for most engineering teams to actually pull off: learning from failure without blaming people. Luna: You mean blameless postmortems. I've seen them done well maybe twice. Most of the time it's a meeting where everyone's trying to figure out who to fire. Lucas: Exactly. And that instinct — to find the person — completely defeats the purpose. So I want to use a specific case. There's a fintech startup — let's call it PayBridge — that had a catastrophic outage about eighteen months ago. Their CTO, a woman named Sarah Chen, decided to use it as a catalyst to change how the whole company viewed incidents. Luna: What was the outage? Lucas: A payment processing pipeline dropped about twelve thousand transactions during a peak hour. The immediate impact was about three hundred thousand dollars in failed payments and a lot of angry merchants. The knee-jerk reaction from senior leadership was to find out which engineer pushed the bad code. That's where Sarah stepped in. Luna: Right, because if you fire the person, you lose all the context about why that code seemed fine at the time. Lucas: Exactly. She insisted on a blameless postmortem. And the key rule she established was this: the postmortem document must never name a person unless it's to thank them for catching something. The entire focus is on systems, processes, and conditions. Luna: That's hard. How do you investigate what happened without saying 'Dan pushed this config change'? Lucas: You reframe it. Instead of 'Dan pushed a bad config,' you write 'The deployment pipeline didn't have a canary stage for configuration changes, so a default value overwrote a production setting.' The language shifts from who to what and why. Luna: And what did Sarah's team actually do in the postmortem meeting? Lucas: They followed a strict four-step format. First, they wrote a factual timeline — just events, no interpretation. Second, they listed the contributing factors: the config change bypassed code review because it was flagged as low-risk, the monitoring dashboard didn't flag the transaction drop for seven minutes because the alert threshold was set too wide, and so on. Third, they identified the things that went well — like the engineer who noticed the anomaly, even though the alert didn't fire. Fourth, they generated action items with owners and deadlines. Luna: So the result wasn't a person getting blamed; it was a list of improvements. Lucas: Yes. And that's the whole point. The CTO told me that within six months, the team had reduced repeat incidents by sixty percent. Because every postmortem led to a tangible system change. Luna: But I've seen teams do blameless postmortems that still feel like a trial. What's the secret? Is it just the format? Lucas: No, the format is necessary but not sufficient. The real secret is what happens before the postmortem. Sarah did two things. First, she explicitly told the team that no one would be fired or penalized for any incident that they disclosed honestly. That's psychological safety, and you can't fake it. Second, she started doing 'pre-mortems' — before a major release, the team would spend thirty minutes imagining everything that could go wrong. Luna: A pre-mortem — I love that. It's like a postmortem in reverse. You assume the release failed and work backwards to find the causes. Lucas: Exactly. That practice normalizes talking about failure before it happens. So when an actual incident occurs, the team is already trained to think systematically, not defensively. Luna: What about teams where the culture is already punitive? Can a CTO just declare 'we're blameless now' and make it stick? Lucas: No, and this is where I see most leaders trip up. You can't just announce a policy. You have to model the behavior. Sarah told me about one incident where a junior engineer accidentally ran a script on the wrong database. The natural reaction would have been to discipline him. Sarah instead stood up in front of the whole engineering org and said, 'This is my fault. I haven't given this team enough guardrails for database access.' Luna: Wow. She took the blame publicly. Lucas: She did. And that single act did more to build a blameless culture than any policy document could. After that, the team trusted the process. Luna: So what about measuring whether blameless postmortems are actually working? Lucas: There are two key metrics. The first is incident recurrence rate — are the same types of incidents happening again? If yes, your postmortems aren't leading to effective action items. The second is the time between incident and completed action item. If action items sit in a backlog for months, the system isn't learning. Luna: And what about the human side? How do you know people feel safe enough to be honest? Lucas: That's harder to measure. Some teams use anonymous pulse surveys after each postmortem. One question: 'Do you feel you could have raised concerns without fear of retaliation?' If the score isn't high, you have a culture problem, not a process problem. Luna: I want to go back to something you mentioned earlier — the idea that the postmortem document never names a person. But what if someone truly made a reckless decision, like deliberately bypassing a safety check? Lucas: That's a edge case, and it should be handled separately. Blameless doesn't mean consequence-free. If someone acted with malice or gross negligence, that's a performance or conduct issue, not a postmortem issue. The postmortem still focuses on the system: why was it possible for one person to bypass a safety check without detection? Luna: Fair. So the system should have had a guardrail. Lucas: Exactly. And that's the mindset shift. Every incident is a window into a systemic vulnerability. Sarah's team now treats postmortems as the most important meeting in their sprint cycle. They even publish anonymized postmortems internally for the whole company to read. Luna: Publishing them — that's bold. Doesn't that risk scaring the business side? Lucas: It does, but Sarah framed it as a transparency tool. She said, 'If the business doesn't know what broke, they can't have realistic expectations about reliability.' And actually, the business team started appreciating the honesty. It built trust. Luna: That makes sense. I think the biggest takeaway for me is that blameless postmortems are not about being nice. They're about being effective. If you blame someone, you stop learning. Lucas: Exactly. And speaking of learning — if this conversation gave you a useful framework, and you'd like to see more episodes like this, there's a simple way to support the show. It's called buy me a coffee dot com slash fexingo. That's buy me a coffee dot com slash f-e-x-i-n-g-o. No pressure, just if you find value in what we do. Luna: And it keeps us ad-free, which means we get to dig into exactly the topics that matter to engineering leaders without any corporate filter. Lucas: Back to the practical side — if you're a CTO or engineering manager listening and you want to start blameless postmortems tomorrow, here's one concrete step: pick your last minor incident, write a timeline, and hold a thirty-minute postmortem with your immediate team. No blame, just learning. See how it feels. Luna: And if someone on the team starts pointing fingers, gently redirect them to the system. It'll feel awkward at first, but it gets easier. Lucas: Right. The goal isn't perfection on day one. It's building the muscle. Over time, that muscle becomes your organization's ability to fail fast and learn faster. Luna: I think that's a great note to end on. Thanks, Lucas. Lucas: Thanks, Luna.