Latest / Customer Journey with Fexingo: Touchpoints, Mapping, and Multi-Channel Marketing / Why Attribution Models Lie About Your Best Channels
Transcript
- Lucas: There's a number that haunts me. According to a 2024 meta-analysis by Skai, covering 178 A/B tests across multiple brands, the average correlation between last-click attribution and true incremental lift was just 0.12. For context, a perfect correlation is 1.0. So basically, your attribution dashboard is telling you almost nothing about what's actually driving sales. Luna: 0.12? That's essentially random noise. So when I look at my Google Analytics and see that direct and branded search are my top conversion channels — that could be totally wrong? Lucas: Exactly. And it's not just last-click. First-click, linear, time-decay — they all have the same fundamental problem: they're correlational, not causal. They observe which touchpoints precede a conversion, but they can't tell you whether that touchpoint actually caused the conversion or just happened to be there. The classic example is a customer who sees a podcast ad, forgets about it, then two weeks later searches for your brand and buys. Last-click gives 100% credit to search. But without that podcast ad, the search might never have happened. Luna: So how do you actually measure what's working? I've heard of incrementality testing, but it always sounds expensive and complicated. Lucas: It's simpler than you think. The gold standard is a holdout test: you take a random subset of your target audience and simply don't show them ads from a particular channel. Then you compare their conversion rate to the exposed group. The difference is your incremental lift. A DTC skincare brand called Pai Skincare did exactly this in late 2024. They wanted to know whether their Instagram ads were actually driving sales or just cannibalizing organic demand. Luna: I love Pai. They're a clean beauty brand based in the UK, right? I think they're mostly organic-driven. Lucas: Yes. So they ran a two-week holdout where 10% of their Instagram audience was randomly excluded from seeing any paid Instagram content. Their attribution dashboard showed Instagram ads had a 4.2% conversion rate — highest of any channel. But the holdout test revealed something shocking: the incremental conversion rate was just 0.3%. Meaning 93% of the conversions attributed to Instagram would have happened anyway through organic posts, direct visits, or other channels. Luna: So they were basically paying for what they were already getting for free. That's terrifying. What did they do next? Lucas: They reallocated 40% of their Instagram ad budget into podcast sponsorships, which had a much lower attributed conversion rate — only 1.8% on last-click. But when they tested that channel incrementally, the incremental lift was 1.4% — meaning 78% of those conversions were truly new. Over the next quarter, total revenue increased 22%, and cost per incremental acquisition dropped by almost half. Their attribution dashboard still shows Instagram as the 'winner,' but their bottom line tells a different story. Luna: That's the thing, right? The dashboard is designed to show last-click because that's easy to measure. But it rewards channels that are 'closing' rather than 'opening.' Podcasts, TV, billboards — they open the door, but they rarely get the last click. Lucas: Precisely. And the Skai study quantified that bias. Across 178 tests, the channels most overcredited by last-click were branded search and direct traffic — typically overestimated by 40 to 60 percent. The most undercredited were display, social, and audio — often underestimated by 30 to 50 percent. So if you're making budget decisions based on last-click, you're systematically starving the channels that actually build demand. Luna: But holdout tests sound like they need a lot of traffic to be statistically significant. What about a smaller brand? Can they do incrementality on a budget? Lucas: You don't need a massive sample. A rule of thumb is at least 1,000 people per variant, but you can also use geo-based holdouts — run ads in some regions and not others, and compare the difference in sales. Or time-based: pause a channel for two weeks and see what happens to overall revenue. If revenue drops more than you'd expect from the attributed conversions alone, you've got incrementality. One e-commerce client I worked with paused all paid search for a month — they were terrified. Their attributed revenue from search was $200,000 a month. But actual revenue only dropped by $30,000. They were overcrediting search by a factor of six. Luna: Wow. So all that money they thought was essential was actually just capturing demand that would have gone somewhere else or organically. Did they reallocate? Lucas: They shifted 60% of their search budget into retargeting and content marketing. Their overall revenue actually increased slightly because they stopped wasting money on clicks that would have happened anyway. The key insight is that most 'direct' traffic isn't really direct. It's driven by other channels — TV, podcasts, word of mouth — that don't get any credit in your attribution model. Luna: So if I'm a marketer listening, what's the one thing I should do this week? I don't have the budget for a full geo holdout or a massive A/B test. Lucas: Pick your highest-attributed non-brand channel — say, Instagram ads — and run a two-week pause. But don't just look at that channel's conversions. Look at total revenue and total conversions across all channels. If total revenue drops by more than the attributed revenue from that channel, you've got incrementality. If it stays flat, you've just proven you were paying for nothing. Then do the same for your lowest-attributed channel — maybe a podcast sponsorship. If revenue drops when you pause it, even though its attributed conversions were small, you've found a hidden gem. Luna: That's practical. And it doesn't require a data scientist. Just a willingness to be wrong about what you think works. Lucas: Exactly. The hardest part is admitting that your attribution dashboard is lying to you. But once you accept that, you can start making decisions based on actual profit, not distorted metrics. Pai Skincare didn't just save money — they grew by 22% because they found channels that truly brought in new customers. And that's the real point of the customer journey: not to assign credit to the last touchpoint, but to understand which touchpoints actually change behavior. Luna: So attribution models are a useful starting point, but never the final answer. Run small experiments, trust the data from holdouts, and be ready to surprise yourself. Lucas: Right. The question isn't 'which channel has the highest conversion rate?' It's 'which channel would I miss the most if it disappeared?' That's the channel worth investing in.