Latest / Software Testing with Fexingo: QA, Automation, and Reliable Software Engineering / Why Your Test Suite Needs Contract Testing
Transcript
- Lucas: You know that sinking feeling when your CI pipeline is green, you deploy, and then the checkout flow just breaks? Not because your code is wrong, but because the API you depend on changed its response shape. Luna: Oh, that's happened to me more times than I'd like to admit. Last team I was on, the payment service decided to rename a field from 'total_amount' to 'amount_total' — no warning. Lucas: Classic. And your integration tests didn't catch it because they were mocking the old response. That's exactly the problem contract testing is designed to solve. Luna: So contract testing — we've talked about unit tests, integration tests, end to end tests. Where does this fit? Lucas: It sits between unit and integration. The core idea is simple: each consumer of an API defines their expectations — the exact request they'll send and the exact response they expect — in a contract. The provider then runs those contracts against their actual implementation before deploying. If the contract breaks, the provider knows they'll break a consumer. Luna: So instead of the consumer finding out in production, the provider catches it during their own CI. That's a big shift in responsibility. Lucas: Exactly. It's often called consumer-driven contracts. The consumer publishes a contract, and the provider verifies it. Netflix has been doing this for years with their internal tools. Uber too — they had so many microservices that end to end testing became impossible. Contract testing gave them confidence without the overhead. Luna: Let's get concrete. What does a contract actually look like? Lucas: A contract is a machine-readable file — usually JSON or YAML — that specifies the HTTP method, the endpoint, the request headers, the expected status code, and the response body structure. Tools like Pact or Spring Cloud Contract generate these files automatically from your test code. Luna: So you write a test in your consumer service that says 'when I call /orders with this payload, I expect a 200 with a JSON body containing an order_id and a status field.' Pact captures that interaction and writes it to a contract file. Lucas: Exactly. Then that contract gets published to a shared location — a Pact Broker, for example. On the provider side, the CI pipeline pulls down all the contracts for that service, runs them against the actual provider code, and fails the build if any contract doesn't match. Luna: What happens when the provider legitimately needs to change the API? Say they're deprecating a field. Lucas: Great question. That's where the negotiation comes in. The provider can see which consumers are using that field, work with them to update their contracts first, and only deploy the breaking change once all consumers have been updated. It's a coordination mechanism, not a straightjacket. Luna: So it forces teams to talk to each other before breaking things. That's actually a feature, not a bug. Lucas: Right. And it dramatically reduces end to end test flakiness. A lot of teams run end to end tests to catch integration issues, but those tests are slow and brittle. Contract tests run in milliseconds per interaction and can be executed in parallel. You can catch the same class of bugs in seconds instead of minutes. Luna: Is there a downside? I've heard contract tests can become a maintenance burden if you have too many consumers. Lucas: There's a real cost. Every consumer publishes a contract, and every provider has to verify all of them. If you have a hundred microservices all hitting the same auth service, that's a hundred contracts to verify. Teams can hit scaling limits in the Pact Broker, and contract negotiation becomes a bottleneck. Luna: So what do you do? Ignore the problem? Lucas: No, you tier your contracts. Not every interaction needs a formal contract. For stable, internal APIs with few consumers, maybe a simple schema check is enough. Save contract testing for the critical, frequently changing, high-impact integrations. The ones where a break costs real money. Luna: Like payment, authentication, order processing. Lucas: Exactly. Identify your golden paths — the core business flows — and apply contract testing there. For everything else, a lightweight approach like OpenAPI schema validation might be sufficient. Luna: Let's talk tools. Pact is the big one, but what about Spring Cloud Contract? Lucas: Spring Cloud Contract is a solid choice if you're already in a Spring ecosystem. It lets you define contracts in Groovy or YAML and generates both provider-side verification tests and consumer-side stub tests. Pact is more language-agnostic and has a richer ecosystem for publishing and versioning contracts. I'd say Pact is the default for most polyglot shops. Luna: There's also the newer kid on the block — Specmatic. It uses OpenAPI specs as the source of truth and can generate contract tests automatically. Lucas: Right. Specmatic takes a different approach: instead of consumer-driven contracts, it uses the OpenAPI spec as a living contract. It can verify both request and response conformance on both sides. It's still maturing, but it's promising for teams that already have comprehensive OpenAPI specs. Luna: So if I'm a QA lead evaluating contract testing, what's the first step? Do I need to rewrite all my integration tests? Lucas: Absolutely not. Pick one pair of services — ideally two teams that already communicate well — and start with a single endpoint. Write a consumer test, publish the contract, and have the provider verify it. Get that flow working end to end. Then expand from there. Luna: Pilot before you scale. That's the right approach. And what about versioning? If the consumer is on version 1 and the provider deploys version 2, do contracts break? Lucas: They can, but you can version the contracts themselves. Most tools allow you to tag contracts with the consumer's version, so the provider can verify against multiple versions simultaneously. The key is to make breaking changes backwards-compatible — add new fields without removing old ones — until all consumers have migrated. Luna: That's a lot of discipline. But it's the same discipline that makes microservices work in the first place. Lucas: Exactly. And honestly, contract testing forces that discipline. It's a forcing function for API design best practices. Companies like Netflix and Uber didn't adopt it because they had too much time — they adopted it because they had too many outages. Luna: One more thing — how does contract testing interact with your CI/CD pipeline? Do you run contract verification before or after unit tests? Lucas: Run it early. After unit tests, before integration tests. The faster feedback cycle is one of the biggest wins. Ideally, the provider pipeline pulls the latest contracts at the beginning of the build, runs verification, and fails within a minute if something's off. Luna: So you catch the break before you even start deploying anything. Lucas: Right. And you avoid the dreaded 'works on my machine' problem because the contract is shared and executed in a clean CI environment. Luna: If today's conversation gave you something usable, maybe a new angle on that integration test debt you've been ignoring — a handful of listeners chip in monthly through buy me a coffee dot com slash fexingo, and that's literally what funds making many of these shows. No ads, no sponsors, just people who find the content useful. Lucas: Yeah, it's a small group, but it makes a big difference. Keeps the podcast independent and lets us go deep on topics that wouldn't fit a sponsor segment anyway. Luna: Exactly. And speaking of deep — I want to come back to something you said earlier about measurement. How do you actually measure the impact of introducing contract testing? Lucas: Track two things: the number of integration-related production incidents, and the time spent debugging integration test failures. Teams I've worked with have seen a 60 to 80 percent reduction in both within two quarters. The ROI is very real. Luna: That's a strong signal. I think a lot of teams are sitting on a pile of flaky end to end tests and don't realize contract testing could be the scalpel they need. Lucas: Right. end to end tests still have their place — for critical user journeys. But contract testing handles the integration layer more efficiently. It's not either-or. It's about using the right tool for each level of the test pyramid. Luna: So if someone's listening and thinking 'I need to sell this to my team,' what's the one number they should bring to the next meeting? Lucas: Bring this: a typical contract test runs in under 100 milliseconds. An end to end test that covers the same interaction can take 30 seconds or more. Multiply that by hundreds of endpoints, and the time savings become impossible to ignore. Luna: That's a concrete number to build a business case around. Contract testing isn't just about quality — it's about velocity. Lucas: Exactly. And the earlier you catch a contract break, the cheaper it is to fix. That's the same logic as shift-left testing, but applied specifically to the integration layer. It's one of those rare investments that pays for itself in a quarter or two.