Latest / Software Testing with Fexingo: QA, Automation, and Reliable Software Engineering / How Cost of Delay Prioritizes Your QA Backlog
Transcript
- Lucas: Every QA team I've ever talked to has the same problem: a backlog of test cases that's miles long, stakeholders who want everything automated yesterday, and maybe two engineers to actually do the work. Luna: Right. And the default move is to automate whatever's easiest first, or whatever the loudest product manager is yelling about that week. Lucas: Exactly. So today I want to talk about a framework that replaces that chaos with a simple question: what's the cost of NOT automating this test? In other words, Cost of Delay. Luna: Cost of Delay — I've heard that term in product management circles, usually tied to feature prioritization. How does it translate to testing? Lucas: The core idea is the same. Cost of Delay is the economic value you lose per unit of time that something is delayed. If a feature launch is delayed by a month, you lose a month of revenue. For testing, if a critical bug isn't caught because the test isn't automated yet, the delay costs you in production incidents, customer churn, rework hours — it's real money. Luna: So instead of asking 'which test is easiest to automate,' you ask 'which test costs us the most if we delay automating it?' Lucas: Exactly. Let me give you a concrete example. Say you're testing a fintech app. You have three candidate test suites: the login flow, the payment flow, and the reporting dashboard. Manual regression for each takes about two hours per cycle. You have one QA engineer who can automate one suite per sprint. Which one do you pick? Luna: Instinct says the payment flow — that's where money moves, highest risk. Lucas: That's probably right, but let's put numbers on it. Suppose the login flow breaks once every ten releases, and when it does, it causes a 30-minute outage that costs about $5,000 in lost productivity and support calls. So the Cost of Delay for login is roughly $5,000 per ten releases, or $500 per release. Luna: That's not nothing, but it's manageable. Lucas: Now the payment flow. It breaks once every five releases. When it breaks, transactions fail for four hours before the manual test catches it. Each hour of payment downtime costs $12,500 in lost transaction fees and customer compensation. That's $50,000 per incident, and two incidents per ten releases means $10,000 per release. Twenty times the cost of the login delay. Luna: Wow. So the payment automation is saving you $10,000 per release versus $500 for login. That's a no-brainer. Lucas: Right. And the reporting dashboard? It breaks once every twenty releases, and when it does, nobody notices for three days because the numbers are still accurate — they just take longer to render. The cost is maybe $2,000 in analyst overtime. So Cost of Delay is $100 per release. Clear priority: payment first, then login, then reporting. Luna: But this assumes you can actually estimate those costs. In a lot of teams, especially if you're not in fintech, the numbers aren't that clean. How do you handle that? Lucas: You're right — precise dollar figures are a luxury. But the framework still works with relative ordering. You can use a simple scale: high, medium, low. Or you can use the four classic delay types from Don Reinertsen's book — expedite, standard, fixed date, and intangible. Luna: Walk me through those. Lucas: Expedite is something where every day of delay is catastrophic — like a security vulnerability that's being actively exploited. Standard delay is linear value loss over time, like our payment flow. Fixed date delay means the value drops to zero after a certain date — think a test for a Black Friday promotion that's worthless if it's not ready before the sale. And intangible delay is hard to quantify but still real, like test coverage that improves developer confidence and reduces burnout. Luna: So you'd prioritize expedite first, then fixed date, then standard, then intangible — roughly. Lucas: Broadly yes. But within standard delay, you still want to rank by the magnitude of value loss per unit time. A payment test at $10,000 per release beats a login test at $500, even though both are standard. Luna: Okay, I buy it. But one thing I've seen is that teams get paralyzed trying to make Cost of Delay calculations perfect. They spend three sprints building a spreadsheet instead of just picking something. Lucas: That's a real trap. I'd say start with one test suite. Pick the one you already suspect is most critical. Estimate its Cost of Delay with rough numbers — even order of magnitude is fine. Then pick the second most critical. The gap is usually big enough that precision doesn't matter. You don't need to know if it's $9,500 or $10,500 — you just need to know it's way more than $500. Luna: Do you ever run into pushback from product managers who say 'we don't budget for test automation that way'? Lucas: All the time. But here's the thing — product managers already use Cost of Delay for features. If you present it in their language, they get it immediately. You can say 'we have three test candidates with these delay costs — which one should we automate first?' and they'll point to the payment flow every time. Luna: That's smart. It turns test prioritization from a qa internal decision into a business decision that everyone can agree on. Lucas: Exactly. And once you've done it once, you have a template. You can revisit it every quarter as the product evolves. Maybe the login flow becomes more critical if you add two-factor authentication and the downtime cost goes up. The framework adapts. Luna: So the key takeaway: don't automate the easiest test. Automate the test whose delay costs the most. Lucas: That's the headline. And you don't need a finance degree to do it — just a willingness to guess, iterate, and talk to your stakeholders about what downtime actually costs. Luna: Yeah, and if today's conversation gave you something usable, a couple of dollars a month is genuinely what keeps these episodes ad-free — buy me a coffee dot com slash fexingo, if you've gotten something out of them. Lucas: It really does make a difference. So back to Cost of Delay — one nuance I want to add is that the framework also works for deciding which existing tests to maintain. If you have a test that never catches bugs and has a very low Cost of Delay, maybe you drop it or run it only on a slower cadence. Luna: Right, because maintaining tests has its own cost. If the delay cost is lower than the maintenance cost, it's a net negative. Lucas: Exactly. So Cost of Delay isn't just for automation backlog — it's for the whole test portfolio. You can use it to prune tests that no longer earn their keep. Luna: That's a great point. I think a lot of teams accumulate cruft because they never ask that question. Lucas: And that's the thing — Cost of Delay forces you to ask it. It's not a silver bullet, but it's a lens that makes the trade-offs visible. And visible trade-offs lead to better decisions. Luna: Alright, I'm going to try this on my next sprint planning. Any final tips? Lucas: Start small. Pick one feature area. Estimate the delay cost with your team — it's a great exercise for getting everyone on the same page about what really matters. And don't let perfect be the enemy of good enough. A rough Cost of Delay is infinitely better than a perfect list of excuses. Luna: That's the kind of practical takeaway I love. Thanks, Lucas. Lucas: Thanks, Luna. See you next time.