Latest / Software Testing with Fexingo: QA, Automation, and Reliable Software Engineering / Why Your Software Testing Strategy Is Broken
Transcript
- Lucas: Look, if you are running a large-scale application today, your test suite is probably lying to you. Not because it is missing bugs, but because it is too expensive to maintain. We see this constantly in enterprise engineering: teams spend forty percent of their sprint capacity just fixing broken tests rather than shipping new features. The problem isn't the quality of the code; it is the brittleness of the tests themselves. Luna: So you are saying that having a green build doesn't actually mean the product is stable? It just means the tests haven't changed yet. Lucas: Exactly. And that illusion of stability is dangerous. I want to talk about a specific pattern we call brittle automation, where engineers tie their critical regression tests to the exact pixel coordinates or DOM structure of a user interface. When the design team changes a button margin by two pixels, the entire end to end test fails. You have to fix the test before you can even look at the bug report. It is backwards. Luna: That sounds exhausting. How do you convince a lead engineer that breaking the test is actually the feature working correctly? Lucas: You don't convince them with philosophy. You convince them with math. Let me give you a concrete example from a major fintech platform that launched last quarter. They had over twelve thousand automated end to end tests. Every time they updated their mobile app's landing page, which happened twice a month, those twelve thousand tests would break. The engineering team would sit there for three days, manually debugging Selenium scripts, only to realize the test was looking for a class name that no longer existed. They were testing the implementation details, not the business value. Luna: Right, so they were testing the CSS instead of the function. That is like checking the paint color of a car before starting the engine. Lucas: It is worse than that. Because every time they fixed those tests, they were delaying deployment. The cycle time went from four hours to three days. That is a massive opportunity cost. The solution isn't to write fewer tests. It is to change what we consider a valid assertion. We need to move away from visual or structural assertions toward semantic ones. Does the checkout button still trigger the payment flow? That is the truth. What color the button is, or whether it uses a div or a span tag, is irrelevant noise. Luna: So how do you separate the signal from the noise? Do you just stop testing the UI entirely? Lucas: No, you never stop testing the UI. But you change the depth of the test. Most teams treat all tests as equal. They put heavy, slow, brittle end to end tests on every single interaction. That is a mistake. You should be using a layered approach where the vast majority of your logic is tested at the unit or component level, and only the critical paths are tested end to end. And even then, those end to end tests should be resilient to cosmetic changes. Luna: But if you ignore the UI structure, aren't you worried that users will actually click around and find something broken visually? Lucas: That is a fair concern. But visual bugs are rarely catastrophic. A misaligned button is annoying. A failed transaction is fatal. You prioritize resilience on the things that hurt revenue. If you focus your energy on ensuring the API contract between your frontend and backend is solid, you catch ninety-five percent of the serious bugs. The remaining five percent are usually styling issues that manual QA or simple visual regression tools can catch without breaking the whole pipeline. Luna: I see. So the key is decoupling the test from the implementation. Can you explain how that works in practice? Lucas: Sure. Let's talk about contract testing. Instead of launching a browser and clicking through the app, your test verifies that the data shape coming out of the service matches what the consumer expects. If the frontend team decides to refactor the entire component library, as long as the JSON payload remains consistent, the test passes. It ignores the HTML wrapper. It ignores the CSS classes. It only cares about the data. This makes your tests immune to refactoring. Luna: That sounds much faster too. Running a browser takes forever compared to just checking a JSON object. Lucas: It is orders of magnitude faster. And more reliable. Browser-based tests fail because of network latency, timing issues, or random element loading delays. Those are what we call flaky tests. Contract tests run in milliseconds inside a memory sandbox. There is no network jitter. There is no rendering engine. You get a deterministic result every single time. That reliability is what allows you to merge code confidently. Luna: But what about the edge cases? Like when the backend sends an unexpected field? Won't the contract test miss that if it is too loose? Lucas: Good point. Contract testing isn't a silver bullet. You still need some end to end coverage. But you treat it differently. You use it for smoke testing critical flows, not for exhaustive validation. Think of it like this: unit tests verify the bricks are good. Contract tests verify the wall is standing up. end to end tests verify the house has electricity. You don't need to check the electrical wiring in every single room to know the power is on. You check one outlet per floor. That is enough to trust the system. Luna: I like that analogy. It feels less overwhelming than trying to test everything perfectly. Lucas: It is less overwhelming because it is realistic. Perfection is the enemy of shipping. In the current market, speed matters. Companies that can deploy daily without fear are winning. Companies that are bogged down by fragile test suites are losing market share. The goal is psychological safety for the developer. When you push code, you should feel confident, not anxious. That confidence comes from knowing your tests are fast, reliable, and relevant. Luna: So, if I am a manager listening to this, what is the first step I take tomorrow morning? Lucas: Audit your test suite. Identify the top twenty tests that take the longest to run and break the most often. Those are your prime candidates for replacement. Are they testing UI elements? If so, replace them with api level checks. Are they testing database schemas? Move those to unit tests. Start small. Replace one brittle test a week. Over six months, you will transform your CI pipeline from a bottleneck into a gatekeeper. Luna: And you won't lose quality? Just reduce the scope of what you are verifying? Lucas: You aren't reducing scope. You are increasing precision. You are focusing on what actually breaks the user experience. Visual glitches are easy to spot. Data corruption is hard to spot. Shift your resources to catching the hard stuff. That is how you build robust software. Luna: This is really helpful. It changes the narrative from testing being a chore to testing being a strategic asset. Lucas: Precisely. Now, speaking of strategic assets, let's take a brief moment to talk about how we keep this show running. Luna: Yeah, if today's conversation helped you rethink your own engineering workflows, you might appreciate supporting the production behind it. Lucas: We believe in keeping our content independent and ad-free, which relies entirely on listener support. If this episode gave you a fresh perspective on software reliability, consider buying us a coffee to help us bring you more deep dives like this. Luna: You can find us at buy me a coffee dot com slash fexingo. No pressure, just appreciation if you feel it adds value. Lucas: Thanks for considering it. Now, back to the technical side. There is another layer to this discussion that many teams overlook. Luna: Which is? Lucas: Monitoring in production. Once you have shifted your testing upstream, you still need visibility downstream. You cannot rely solely on pre-deployment tests because you will always miss something. You need observability. When a contract test passes but the user sees a blank screen, what did you miss? Usually, it is a third-party dependency failure or a rate limit hit. These are runtime conditions that static tests cannot predict. Luna: So testing prevents the obvious crashes, but monitoring catches the subtle failures? Lucas: Exactly. And they work together. Good testing reduces the noise. Good monitoring amplifies the signal. If your tests are brittle, your monitoring alerts will be noisy too. Everyone gets alert fatigue. But if your tests are clean, your alerts become highly actionable. You know exactly when a real issue occurs because your baseline is stable. Luna: It sounds like testing and monitoring are two sides of the same coin. You can't have one without the other. Lucas: They are part of the same feedback loop. Testing gives you confidence to ship. Monitoring gives you confidence to iterate. Without testing, you ship blindly. Without monitoring, you fly blind after you ship. Both are essential for modern engineering. Luna: I think that clarifies why so many companies struggle. They try to solve the problem with just one tool. Lucas: Right. They throw money at monitoring tools hoping it will fix bad code. Or they hire more QA testers hoping it will cover gaps in architecture. Neither works if the foundation is shaky. You need resilient tests and intelligent monitoring. Luna: Do you think smaller startups can afford this level of sophistication? It sounds like an enterprise luxury. Lucas: Actually, it is cheaper for startups. Startups fail because they move too fast and break things they didn't mean to. By implementing lightweight contract testing early, they save weeks of rework later. The cost of fixing a bug in production is ten times higher than fixing it in development. Smart founders invest in testing infrastructure from day one. Luna: That makes sense. Prevention is always cheaper than cure. Lucas: Always. And as your team grows, that investment pays dividends. You can onboard new engineers faster because the tests serve as documentation. If the contract test passes, you know exactly what the API does. It becomes living documentation. That is a huge benefit for scaling teams. Luna: I hadn't thought of tests as documentation. That is a compelling reason to care about quality. Lucas: It is one of the best reasons. Code changes. Requirements change. Tests often stay the same until they break. But well-written contract tests evolve with the product. They capture the intent, not just the syntax. That longevity is what makes them valuable. Luna: So, it is about building a system that teaches you as you go. Lucas: Yes. And that mindset shift is what separates mature engineering organizations from chaotic ones. It is not about being perfect. It is about being adaptable. Adaptable tests, adaptable code, adaptable processes. Luna: Adaptability seems to be the real keyword here. Resilience in testing leads to adaptability in business. Lucas: Spot on. If you can change quickly without breaking, you win. If you are stuck maintaining old tests, you lose. The choice is yours. Luna: I hope our listeners choose the path of resilience. It is a harder road at first, but the view from the top is much clearer. Lucas: It certainly is. Thanks for listening, Luna. And thanks to everyone who helps keep these conversations going.