Latest / Software Testing with Fexingo: QA, Automation, and Reliable Software Engineering / How Regression Testing with Snapshot Diffs Prevents UI Drift
Transcript
- Lucas: Luna, I want to talk about something that bugs me — UI drift. It's not a bug crash, not a functional failure, but over time, small visual changes accumulate and the product starts to feel… off. Luna: Right — like buttons shift a pixel, fonts get slightly different padding, and suddenly the layout looks foreign. How do you even catch that systematically? Lucas: Snapshot regression testing. You take a picture — literally a screenshot — of each component or page state at a known good point, and then every time you deploy, you compare new screenshots pixel by pixel. Any difference shows up as a diff image. Luna: And that's different from just running your usual functional tests, right? Because functional tests check if the button works, not if it moved three pixels to the left. Lucas: Exactly. And UI drift is insidious because it's not one big change — it's a thousand tiny ones. A developer updates a CSS variable, a designer tweaks a margin in the design system, a third-party widget gets a minor version bump. Each change is harmless alone. But after six months, the whole interface feels broken. Luna: So snapshot tests are like a visual audit trail. What tools do teams typically use for this? Lucas: Percy and Chromatic are the two big names in the space. Percy, owned by BrowserStack, does visual testing for web apps. Chromatic is built specifically for Storybook — if you're using a component library, it's almost a no-brainer. Both integrate into CI and flag diffs before they hit production. Luna: I've used Percy on a project. What surprised me was how many diffs were intentional — we'd changed a colour or adjusted spacing on purpose, but it still flagged them. We had to review each one. Lucas: That's the double-edged sword. Snapshot tests are extremely sensitive. They catch real regressions, but they also generate a lot of noise. If your team isn't disciplined about reviewing and approving new baselines, you end up ignoring the diffs — and then the test is useless. Luna: So the process matters as much as the tool. You need a clear workflow: developer sees diff, marks it as intentional, updates the baseline. Otherwise the pipeline starts accepting changes without scrutiny. Lucas: Right. And there's a subtle point about thresholds. Most tools let you set a pixel-diff threshold — say, 0.1 percent of the image area. That helps filter out anti-aliasing variations and sub-pixel rendering differences between operating systems. But if you set it too high, you miss real drift. Luna: What's a good starting threshold? We used 0.5 percent on that project, and it felt like a decent balance. We caught actual layout shifts but didn't get bogged down in font-rendering noise. Lucas: 0.5 percent is typical. Some teams start lower — 0.1 percent — and then adjust up if they get too many false positives. The key is to understand what kind of changes actually matter for your users. A one-pixel shift in a button's border radius is probably invisible. A three-pixel shift in a product card's image height might cut off the price. Luna: And that's where the ROI shows up. I remember a case study from a major e-commerce site — I think it was Wayfair or maybe Etsy — where a 3-pixel layout shift was causing the 'Add to Cart' button to overlap a product description on mobile. They caught it via snapshot testing, fixed it, and saw a 2 percent conversion improvement. Lucas: Exactly the kind of story I love. Two percent conversion on millions of visitors is huge revenue. And it was a regression that no functional test would ever catch — the button worked, it just looked wrong. Luna: So who owns the snapshot reviews? QA, dev, or both? Lucas: Both, ideally. Developers should review diffs in their own pull requests because they know what changed. But QA should also have a periodic audit — say, once per sprint — to review the whole baseline set and make sure no drift has been accidentally approved. It's like cleaning your glasses. Luna: I like that analogy. You don't notice the smudges until you clean them and suddenly everything is crisp. Lucas: One more thing: snapshot testing works best when your UI is deterministic. If you have animations, random data, or time-dependent elements, you'll get false diffs every time. You need to freeze those — use mocked data, disable animations in test mode, or freeze the system clock. Luna: So it's not a set-it-and-forget-it solution. It requires careful setup upfront and ongoing maintenance. But the payoff is real. Lucas: Absolutely. And if today's conversation gave you something practical to try — like setting up a snapshot test for one component this week — that's exactly why we do this show. Luna: Yeah, and keeping the show ad-free and focused on useful content is something we hear people appreciate. If you found this episode valuable, you can support us at buy me a coffee dot com slash fexingo. Lucas: Small gestures like that keep us going, and it means we never have to run a single ad. So thanks to anyone who does. Luna: Alright, back to the tech. One question: how do you handle snapshot testing for responsive designs? You can't just test one viewport size. Lucas: Great question. Most tools let you define multiple viewports per test. You typically test at mobile, tablet, and desktop breakpoints. Some teams also test at a specific 'weird' size — like 375 by 812 for iPhone X — because that's where layout bugs hide. Luna: And do you snapshot every component in isolation, or full pages? Lucas: Both, but for different reasons. Component-level snapshots are great for design system consistency — you catch drift in your button or card component before it spreads. Full-page snapshots catch integration bugs, like a footer overlapping content because of a flex property change. Luna: So a layered approach: component tests for early detection, page tests for end to end visual sanity. Lucas: Exactly. And you can prioritise based on user impact. Test the checkout flow, the login page, the main product grid — the pages where visual weirdness directly affects trust and conversion. Luna: That's a good rule of thumb. Test what matters to the user, not what's easy to test. Lucas: Right. And with snapshot testing, the main cost is not the tooling — it's the human time to review diffs. So you want every diff to be meaningful. That means writing good tests with deterministic data and stable environments. Luna: I think the biggest takeaway for me is that UI drift is real, it's costly, and it's invisible to most testing. Snapshot regression is the antidote — but only if you treat it as a process, not just a tool. Lucas: Well said. Next time you're in a code review and someone says 'I just moved that button two pixels, it's fine,' remind them that two pixels times a hundred changes is a broken interface. Snapshot tests keep the team honest. Luna: And they give you a record of exactly when and how the UI changed. That's powerful for debugging and accountability. Lucas: Absolutely. Thanks for the conversation, Luna. Let's do a deep dive on visual testing for animations another time — that's a whole other beast.