Latest / MBA Training Marketing / Incrementality testing: the playbook for replacing last-click attribution
Transcript
- 0:00Welcome to the Leaders Insights podcast.
- 0:08Today's episode, Incrementality Testing, the playbook for replacing last-click attribution.
- 0:13Let's start with a provocation.
- 0:15You've said last-click attribution has been lying to marketers for years.
- 0:19That's a strong accusation against a tool half the industry still swears by.
- 0:23It's not even a controversial one anymore.
- 0:26Crediting the perfect attribution, the model that gives 100% of the credit to whatever
- 0:30ad someone touched right before they bought, is like crediting the person who opened the
- 0:35door for winning the marathon.
- 0:41They did.
- 0:42But the question isn't did they click.
- 0:44It's would they have bought anyway?
- 0:47A huge share of last-click conversions are people who were already coming to you.
- 0:51They searched your brand name, clicked your own ad, and bought.
- 0:55You just paid to intercept a sale you already had.
- 0:58So you're paying a toll on your own bridge?
- 1:00Exactly that.
- 1:02And it gets worse.
- 1:04Since 2021, privacy changes.
- 1:06Apple killing the identifiers apps use to track you.
- 1:09Browsers dropping third-party cookies mean you can't even see the full click path
- 1:13anymore.
- 1:15So last-click isn't just biased.
- 1:17It's now guessing with half the data missing.
- 1:20Fine.
- 1:21So what replaces it?
- 1:22You're pushing incrementality testing.
- 1:24Sign it before you use it.
- 1:26Incrementality is the extra sales your ad actually caused.
- 1:30The ones that wouldn't have happened without it.
- 1:32An incrementality test measures that directly by holding some people back from seeing the
- 1:37ad and comparing.
- 1:39That sounds like a science experiment, not a marketing plan.
- 1:43It is a science experiment, and that's the point.
- 1:46You take your audience, you split it.
- 1:48One group, the test group, sees the ads.
- 1:51The other, the control group, doesn't.
- 1:54Then you look at who bought in each.
- 1:56The difference is your true lift.
- 1:58Give me a concrete version.
- 2:00What does that look like in a real channel?
- 2:02Take META, Facebook, and Instagram.
- 2:05They offer what they call a conversion lift test, where they randomly withhold ads from
- 2:09a slice of your audience.
- 2:11Say the test group converts at 4% and the control at 3%.
- 2:16That one point is your incremental lift.
- 2:19Everything above the 3% baseline is what your spend actually bought.
- 2:23And that vendor number, META running the test on META's own platform, you trust it?
- 2:28I trust it the way I trust a restaurant reviewing its own food.
- 2:31It's a starting point, not gospel.
- 2:34The cleaner method is a geotest.
- 2:36You turn ads on in some cities, and often comparable ones.
- 2:40Portland gets the campaign, Sacramento doesn't.
- 2:43You compare sales.
- 2:44Nobody's grading their own homework there.
- 2:47What kind of gap are we talking about between what last click claims and what's real?
- 2:51When I ran this at a retail brand, our branded search, ads on our own name, showed a stellar
- 2:56last click return.
- 2:58Incrementality testing said roughly 80% of those buyers would have found us anyway.
- 3:04We cut that spend by a third, and revenue didn't move.
- 3:08That's a third of a budget line doing nothing.
- 3:10That's the part that stings.
- 3:12You're admitting you were lighting money on fire.
- 3:15Everyone is.
- 3:16They just haven't run the test to prove it.
- 3:18The comfortable dashboard number feels great.
- 3:21Incrementality is the friend who tells you the truth at the wrong moment.
- 3:25So how does an exec actually start without hiring a team of statisticians?
- 3:29Start with your single biggest channel and your most suspicious line item.
- 3:34Almost always branded search or retargeting.
- 3:36Ads chasing people who already visited your site.
- 3:40Those are where the lying is worst.
- 3:42Why those two specifically?
- 3:43Because both target people already leaning toward you.
- 3:47Retargeting shows ads to someone who was about to buy anyway.
- 3:51The test tells you how much you're paying to nudge a decision already made.
- 3:55And when the number comes back ugly?
- 3:57You reallocate.
- 3:59Move the money to a channel where the lift is real and let it run.
- 4:02That's the whole loop.
- 4:04Test, cut the fake wins, feed the real ones.
- 4:07One thing they can do Monday morning?
- 4:09Pick your branded search budget, run a two-week geo holdout, on and half your markets, off
- 4:15and the other half, and compare total sales.
- 4:18If nothing drops, you just found your first real savings.
- 4:21Done.
- 4:22Want to know where you actually stand?
- 4:24Take the CMO self-assessment at MBA-training.com.