Latest / Software Testing with Fexingo: QA, Automation, and Reliable Software Engineering / How Pytest Fixtures Kill Test Debt
Transcript
- Lucas: You know that sinking feeling when you open a test file and the first two hundred lines are just setup? Dictionaries with mock objects, database stubs, environment variables being patched — and you have no idea which of those lines actually matter for the test you're trying to debug? Luna: Oh, I've been there. It's like the test is buried under scaffolding. You end up spending more time understanding the setup than the assertion. Lucas: Right, and this is what I'd call 'test debt'. It's the accumulated complexity in your test code that makes changing anything feel risky. And just like technical debt in production code, it compounds. A team I spoke with at a mid-sized fintech — they process payments for about 200 merchants — had this exact problem. Luna: What was their setup like? Lucas: They had about 2,000 test cases across their payment-processing microservice. And those tests were using a shared base class — a TestCase that inherited from unittest.TestCase. Every test subclass would call super.setUp and then add twenty or thirty lines of specific mocks. Over time, the base class grew to over 800 lines, and the subclasses each had their own mutations of that setup. Luna: So every test was essentially copying a template and tweaking it. That's fragile. Lucas: Exactly. And because the setup was mixed into the test class, changing one mock in the base could break dozens of tests downstream. They were spending roughly 20 percent of each Sprint just fixing broken tests — not because the code changed, but because the test infrastructure was brittle. Luna: Twenty percent is huge. How did they get out of it? Lucas: They rewrote the entire test suite using Pytest fixtures. Now, I know that sounds like a massive up-front investment, but they did it incrementally over about three Sprints. The key was they moved from class-based setup to function-scoped fixtures, and then used conftest.py files at the package level to share fixtures across test files. Luna: So the fixtures replaced the setUp methods? Lucas: Yes, but with a crucial difference. Fixtures are modular and composable. You can define a fixture that returns a mock database connection, and then another fixture that uses that connection to seed test data. They're not inherited — they're injected. So a test that needs a database and a specific user can just ask for those two fixtures as parameters. Luna: And because each fixture is isolated, changing the database-connection fixture doesn't break tests that don't use it. Lucas: Exactly. That's the whole point. The team I mentioned ended up reducing their total test setup code from about 12,000 lines down to 600 lines of fixture definitions. And the tests themselves became much shorter. Their average test went from 40 lines with 30 lines of setup to about 15 lines total. Luna: And the runtime? I'm guessing that improved too. Lucas: Dramatically. Because fixtures can be scoped at the module or session level, they weren't re-creating the database connection for every single test. They changed the scope of their database fixture from function to module. That alone cut the test suite run time from 90 seconds to about 12 seconds. Luna: Ninety to twelve — that's a seven-point-five-x improvement. And flakiness? Lucas: That was the best part. Before the refactor, they had about 15 to 20 flaky tests per Sprint — tests that would fail on one run and pass on retry. After switching to fixtures, flakiness dropped to near zero. Because the setup was deterministic and shared correctly, there were no more race conditions from mutable class attributes or leftover state from previous tests. Luna: I want to talk about fixture scoping for a second. Because I think a lot of teams misuse it. They default to function scope even when they should be using module or session scope. Lucas: Right, that's a common pitfall. The default scope in Pytest is function, which means the fixture is created and torn down for every test. That's safe, but it's expensive. If you have a fixture that sets up a read-only database connection, you can safely scope it to module or even session. But if the fixture mutates state — like inserts a row into a database — then function scope is usually correct. Luna: And there's also the risk of accidentally sharing mutable objects across tests. If a fixture returns a list, and one test appends to that list, the next test might see the modified list. That's a classic source of flakiness. Lucas: Absolutely. The team I mentioned had a case where a fixture was returning a dictionary that contained some cached data. They had it at session scope, but several tests were modifying that dictionary. They ended up with a hard to reproduce failure that only happened when tests ran in a specific order. Luna: So rule of thumb: if a fixture returns an immutable value, or something you can safely share, consider module or session scope. If it returns anything mutable, either use function scope or freeze it — return a copy or an immutable wrapper. Lucas: Another technique that helped that team was using fixture factories. Sometimes you need a fixture that takes a parameter — like 'create a user with this email address'. Instead of writing a separate fixture for every email, they defined a fixture that returns a function. Luna: Oh, the factory pattern. So the fixture looks like a function call inside the test. Lucas: Exactly. They'd write a fixture called 'user_factory' that returned a lambda or a closure that created a user with the given attributes. Then in a test, they'd write 'user = user_factory'. That eliminated another 20 percent of their setup code because they could create exactly the entity they needed without a dozen different fixtures. Luna: We've talked about test debt in prior episodes — flakiness, timing, environment issues. But I think fixture debt is often overlooked. It's not about a specific test failing; it's about the whole infrastructure being slow to change. Lucas: That's a good way to put it. Fixture debt is when your test infrastructure fights you instead of helping you. When you dread adding a new test because you have to untangle the setup class. When changing one mock signature breaks 30 tests. That's the signal that you need to invest in your fixture architecture. Luna: And if today's conversation gave you something useful — maybe a way to think about your own test suite — you know, the way this show stays ad-free is listener support. If you got value, you can head to buy me a coffee dot com slash fexingo and help us keep these conversations going. Lucas: Yeah, it genuinely helps. No pressure, but if you found that fintech example useful, consider it. Okay, back to fixture architecture — one more thing that team did that I think is really smart: they used conftest.py files to organize fixtures by layer. Luna: Layer? Lucas: Yeah, they had a conftest.py at the root of their tests directory that defined session-scoped fixtures like the database engine. Then in subdirectories — like 'tests/unit', 'tests/integration' — they had their own conftest.py files with fixtures specific to that layer. So integration tests had fixtures that actually called the payment API, while unit tests had fixtures that mocked everything. Luna: That makes discovery easier too. As a new developer, you can open the conftest.py in the directory you're working in and see all the shared fixtures available. Lucas: Exactly. And it prevents the 'global namespace' problem where you have hundreds of fixtures defined in one giant conftest.py at the root. By scoping fixtures to the directory, you reduce the mental overhead of 'where does this fixture come from?'. Luna: So the lesson here isn't just 'use Pytest fixtures instead of unittest'. It's about treating your test code with the same design principles as your production code: modularity, single responsibility, and clean dependencies. Lucas: Right. Fixtures are just a tool. The real win is in the architecture. And I think the fintech team's experience shows that even a few Sprints of focused refactoring can pay off for years.