Latest / Software Testing with Fexingo: QA, Automation, and Reliable Software Engineering / How Visual Regression Testing Prevents UI Nightmares
Transcript
- Lucas: So last week I was ordering a pair of running shoes from a site I won't name, not Fexingo's sponsors, and I hit 'Add to Cart' and literally nothing happened. The button just sat there. No spinner, no error, no confirmation. Luna: Oh, I've been there. Did you refresh? Lucas: I refreshed. Same thing. So I opened the developer tools, and what do I see in the console? A CSS class mismatch. The button's click handler was attached to a class name that didn't exist anymore. Someone had renamed a utility class in a global stylesheet, and nobody caught it. Luna: Classic CSS regression. That's exactly the kind of bug that traditional functional tests won't catch because the DOM structure might still be correct, but the visual state breaks. Lucas: Right. And that's what got me thinking about visual regression testing. Most teams I talk to either don't do it at all, or they rely on manual eyeballing before release. But there's a much more systematic approach: automated visual snapshot diffs. Luna: So how does that work in practice? You take a screenshot of every page or component, then compare it pixel by pixel against a baseline? Lucas: Exactly. Tools like Percy, Applitools, or even Playwright's built-in screenshot comparison will capture a full-page or element screenshot during your test run. Then on the next run, it compares the new screenshot to the approved baseline. Any pixel difference — a shifted button, a misaligned grid, a missing icon — gets flagged as a diff. Luna: And the key is that it's automated, so it runs in CI alongside your unit and integration tests, right? Lucas: Precisely. You push a PR that changes a CSS variable, and a few minutes later the build fails because the checkout button's background color changed from the brand blue to a weird gray. The developer sees the diff image immediately and knows they need to fix it or update the baseline if it's intentional. Luna: And that's actually a subtle point — managing baselines. Because if every deployment intentionally changes the UI, you'd get flooded with false positives. Lucas: Right. So good tools let you approve diffs as new baselines. You can set up a review workflow — the developer or QA engineer looks at the diff, says 'yes, that's expected,' and the baseline updates. That way, you only get alerted on unexpected changes. Luna: That makes sense. So what kind of changes does this catch that a regular functional test wouldn't? Lucas: Think about responsive design. You might have a media query breakpoint that accidentally shifts the navigation from a horizontal bar to a hamburger menu at the wrong width. A unit test that just checks DOM structure won't see that the layout broke. But a visual snapshot at that viewport will show the menu overlapping the logo. Luna: Or font loading issues. If a web font fails and the fallback stack kicks in with a different size, the whole page can reflow. Lucas: Absolutely. I've seen a case where a CDN outage for a font file caused the heading on a landing page to drop down and cover the hero image CTA. The functional tests passed because the button was still there in the DOM, but visually it was hidden behind the header. That's a production incident waiting to happen. Luna: So visual regression testing is essentially your safety net for the frontend. But I imagine it's not free — the snapshots take time to capture and store, and the comparison can be heavy. Lucas: That's true. A full-page screenshot for a complex app like a SaaS dashboard can take several seconds per page. If you snapshot every view, your CI pipeline balloons. So the smart approach is to be intentional about what you capture. Luna: What do you recommend teams focus on? Lucas: I'd say start with the highest-traffic user journeys — login, checkout, search results, profile page. Those are the pages where a visual bug hits the most customers. Then add critical UI components like buttons, modals, and navigation. You don't need to snapshot every admin panel page that only internal users see. Luna: So prioritize by business impact. That aligns with what we've talked about in other testing approaches. Lucas: Exactly. And once you have your baseline, you also want to think about cross-browser visual testing. A button that looks perfect in Chrome might overlap text in Firefox, or have a gap in Safari. Some visual testing tools let you run the same snapshot against multiple browsers and compare them. Luna: That's powerful. Because even if you test functionally across browsers, the visual nuance can be different. A one-pixel padding difference might not break the layout, but it could make the button feel slightly off. Lucas: Yeah, and those one-pixel differences aggregate. Over several releases, the UI can drift from the original design intent without anyone noticing until a designer does a side-by-side comparison. Visual regression gives you a hard stop on that drift. Luna: I've also heard of teams using visual diffs for accessibility checks — like ensuring focus indicators are visible, or that color contrast meets WCAG standards after a style change. Lucas: That's a great extension. Some tools let you overlay accessibility color filters, so you can see if a button loses contrast when viewed with a color blindness simulation. It's not a replacement for dedicated a11y audits, but it's a cheap additional check. Luna: So let's talk tools. You mentioned Percy and Applitools. What about open-source options? Lucas: Playwright has a built-in screenshot comparison that's free and works well for smaller projects. There's also backstopjs, which is open-source and uses Puppeteer. The trade-off is that open-source tools usually lack the visual review dashboard and baseline management that a paid service like Percy offers. For a team of five, the open-source route is fine. For an enterprise with dozens of developers approving diffs, the paid tools save a lot of time. Luna: So it's a classic scale decision. Small team, low volume — start with Playwright. Once you hit the point where manual approval becomes a bottleneck, invest in a specialized tool. Lucas: Exactly. And whichever tool you use, you need to handle the non-deterministic elements. Things like animated elements, carousels, or live clocks will cause false diffs in every run. You can freeze animations in the test setup, or use a tool that ignores regions with a mask. Luna: That's a good point. If you don't account for that, your pipeline will be red all the time and people will start ignoring it. Lucas: Right. The goal is to make the signal trustworthy. I've seen teams abandon visual regression because they got too many false positives. So you have to invest in the setup: freeze animations, set smart viewports, use blur or threshold comparison to allow tiny pixel differences due to anti-aliasing in different browsers. Luna: And that's something you can tune. Like, a threshold of 0.1 percent means you're only flagging changes that affect more than a tenth of a percent of the pixels in the region. That ignores sub-pixel rendering differences. Lucas: Exactly. Most tools let you set a global or per-element threshold. I recommend starting with a very low threshold — like 0.01 percent — and then relax it only if you see consistent false positives from a particular component. Luna: So we've covered what it is, why it matters, how to pick tools, and how to avoid false positives. Anything else you think teams commonly miss? Lucas: One thing: they often forget to test the empty state, the error state, the loading state. You might have a great visual test for a list of products, but if the API returns an empty array, the page might render a broken layout instead of a friendly empty message. Snapshot those states too. Luna: That's a great tip. Edge cases are where visual bugs hide. Alright, before we wrap up, I wanted to circle back to something you said earlier about the running shoe site. Lucas: Yeah, that frustrating experience actually ties into a broader point about testing culture. The team probably had unit tests and maybe even integration tests, but they missed the visual layer. And it's not just small teams — I've talked to engineers at large e-commerce companies who admit they only added visual regression after a high-profile outage. Luna: It's one of those things that feels like a nice to have until it saves you from a costly production bug. And speaking of things that feel nice — this show is completely ad-free, and we want to keep it that way. If you got something useful out of today's conversation, consider supporting us at buy me a coffee dot com slash fexingo. Lucas: Yeah, it's a small way to help us keep producing episodes like this without any sponsor interruptions. Even a one-time contribution makes a difference. Luna: Totally. Back to visual testing — Lucas, if someone's listening and wants to start tomorrow, what's the single first step? Lucas: Pick one critical user flow — say, the checkout flow — and write a Playwright test that takes a screenshot of each step. Compare it manually to a baseline you create. That's it. Once you see the first diff that catches a real bug, you'll be hooked. Luna: And then you can expand from there. Great advice. That's all for this episode of Software Testing with Fexingo. Thanks for listening.