Latest / Software Testing with Fexingo: QA, Automation, and Reliable Software Engineering / Why Most Software Tests Are Just Expensive Noise
Transcript
- Lucas: I was looking at some pipeline telemetry last week from a client who calls themselves a fast-moving fintech, but their average build time is sitting at forty-two minutes. That is not fast moving. That is a hostage situation. Luna: Forty-two minutes? That feels like an eternity when you are trying to iterate on a feature flag or debug a regression. Lucas: Exactly. And when I dug into the logs, I realized about sixty percent of that time is spent running redundant integration tests that hit a mock database instead of a real one, just to verify that a service can talk to another service in a way we already proved works. It is expensive noise. Luna: So they are paying for compute cycles to re-prove things that haven't changed in six months? Lucas: Precisely. And the worst part is the psychological safety net. Engineers love these long runs because if it passes, they feel good. But it creates a false sense of security while simultaneously making actual deployments so painful that people stop doing them frequently. Luna: It sounds like the problem isn't the tests themselves, but the lack of granularity in what we are actually measuring. Lucas: Right. We conflate volume with value. If you have ten thousand tests and only five catch a real bug in production, you have a massive efficiency problem. The industry standard answer lately has been to kill the flaky end to end tests and move toward contract testing. Luna: Contract testing feels like a bit of a buzzword now. Is it actually different from what we used to call integration testing? Lucas: It is fundamentally different because it shifts the burden of proof. Instead of spinning up the whole stack to see if Service A talks to Service B, you just verify that the data structure matches the agreed-upon schema. You decouple the verification from the infrastructure. Luna: So you are testing the interface, not the behavior. That seems much cheaper to run. Lucas: It is orders of magnitude cheaper. And faster. We saw a case where a team reduced their feedback loop from forty minutes to four minutes just by replacing heavy ui level checks with lightweight JSON schema validations. Luna: Four minutes is enough to make coffee. Forty minutes is enough to lose context. Lucas: That is the core trade-off. Speed versus coverage. Most teams think they need full coverage, but they really just need relevant coverage. The question is how you decide what is relevant without breaking things. Luna: Do you think most engineering leaders are aware that their test suite is actively slowing down their revenue cycle? Lucas: I don't think they realize it in those terms. They see it as an engineering tax. But when your competitors can deploy ten times a day and you can deploy twice a week, that is a competitive disadvantage. Luna: It feels like we are still stuck in the waterfall mindset of needing a giant gate before anything goes live. Lucas: We are. And the technology has moved past that. We have tools like Pact or Schemathese that handle these contracts automatically. The issue is usually cultural resistance to trusting the system. Luna: Trust is hard to engineer, but it is essential for velocity. Lucas: Absolutely. Let's pause here for a moment. If this conversation about cutting through the noise and focusing on what actually matters in your workflow resonated with you, remember that listener support keeps Fexingo ad-free and allows us to keep digging into these technical realities. You can support the show directly at buy me a coffee dot com slash fexingo. Thanks for keeping the lights on. Luna: Appreciate that. It helps us get the mic right for these deeper dives. Lucas: Back to the pipeline. Once you cut the fat, you are left with a leaner suite. But then you hit the next problem: how do you maintain it? Luna: Maintenance is always the silent killer of good systems. Lucas: It is. Because every new feature requires new tests, and nobody wants to write the boring validation logic. So you end up with a growing list of ignored failures. Luna: So the tests become legacy code before the product code even ships. Lucas: Exactly. And that leads to a strange phenomenon where engineers start deleting tests they know are broken just to get the build green again. It is a race to the bottom. Luna: That is terrifying. You are essentially lying to your deployment pipeline. Lucas: It is institutional gaslighting. And it only gets worse when you bring in new hires who don't know the history of why certain tests exist or why they were disabled. Luna: Is there a metric you would recommend tracking to catch this early? Lucas: Track the failure rate of your tests relative to the number of commits. If your failure rate is high but your production bug rate is low, you are over-testing. If your failure rate is low but your production bug rate is high, you are under-testing. Luna: So you want the sweet spot where the two rates align. Lucas: Yes. And ideally, you want that alignment to happen very quickly. You want to know about a broken test within minutes of a commit, not hours later. Luna: It seems like the goal is to make testing invisible, almost. Lucas: Not invisible, but frictionless. When testing becomes a chore, people avoid it. When it becomes a helpful guardrail, they lean on it. Luna: How do you shift that perception for a team that is burnt out on red bars? Lucas: You start by celebrating the green builds that come from fast, reliable tests. You highlight the developers who shipped features quickly because their suite was efficient. You change the narrative from quality control to enablement. Luna: That is a subtle but powerful reframing. Lucas: It is. Quality should be seen as a superpower, not a constraint. When teams realize that good testing lets them sleep better at night, they start caring about the craft. Luna: I wonder if we will see more specialized roles for test maintenance in the future. Lucas: Maybe. Or maybe AI agents will handle the routine validation. We are already seeing tools that can auto-generate contract tests from API specs. Luna: That could solve the maintenance problem entirely. Lucas: It might. But it also raises the bar for human oversight. If the machine writes the tests, humans have to design the contracts. That is a higher-level skill. Luna: So we are moving from writing tests to defining truth. Lucas: Exactly. Defining what correct looks like, rather than just checking if it happened. It is a significant shift in mindset. Luna: I think most teams are still stuck in the checking phase. Lucas: They are. But the ones who make the jump tend to pull ahead in market share. Speed is the ultimate moat in software. Luna: So the lesson is to prune aggressively and trust the contracts. Lucas: Prune aggressively. And never let the test suite become a monument to past fears. Luna: A clean pipeline is a happy team. Lucas: And a happy team ships better software. What do you think is the first step for a team wanting to fix their pipeline today? Luna: Audit the last month of failed builds. Find the ones that were irrelevant and delete them. Lucas: Delete them. Don't archive them. Out of sight, out of mind. Luna: If it doesn't help us ship, it doesn't belong in the repo. Lucas: Agreed. Though I would suggest keeping a backup branch just in case. Paranoia is part of the job. Luna: Smart. Safety nets are good, as long as they aren't tripwires. Lucas: Perfectly put. We will leave it there. Next time, we might look at how chaos engineering is changing reliability expectations. Luna: Breaking things on purpose to see what holds. Sounds like fun. Lucas: It is controlled fun. Until then, keep your builds green and your pipelines fast.