Latest / Sleeping Barber - A Marketing Podcast / SBP 231: The Sharp Cut - Marketing has more data than ever. So why does nobody believe it?

Transcript

SBP 231: The Sharp Cut - Marketing has more data than ever. So why does nobody believe it? · Sleeping Barber - A Marketing Podcast

• 0:00
  SPEAKER_01: Welcome to the Sleeping Baba Podcast.
• 0:02
  SPEAKER_00: Mark, are you ready for this one?
• 0:04
  SPEAKER_00: This is a good one.
• 0:06
  SPEAKER_00: Sharp.
• 0:07
  SPEAKER_00: I think it's gonna be a fun.
• 0:09
  SPEAKER_00: It's gonna be a hopefully a very fun experience for you and I and you the listener.
• 0:16
  SPEAKER_00: Let's start with this though.
• 0:18
  SPEAKER_00: Marketing arguably has never been more measurable than it is right now.
• 0:22
  SPEAKER_00: And marketing has probably never been more short-term than it is right now.
• 0:26
  SPEAKER_00: Both of those are true, and they probably shouldn't be.
• 0:30
  SPEAKER_01: Now honestly, and this is what this episode is about.
• 0:32
  SPEAKER_01: If measurement was the fix, the last 10 years should have been the golden age of marketing effectiveness.
• 0:39
  SPEAKER_01: Instead, we got better instruments and worse outcomes.
• 0:43
  SPEAKER_00: So let's start with whether that's actually true.
• 0:46
  SPEAKER_00: Uh, because I don't want to just argue from vibes, V.
• 0:50
  SPEAKER_01: You do vibe a lot, Mark.
• 0:52
  SPEAKER_01: So Boathouse runs a study on this every year.
• 0:56
  SPEAKER_01: Uh the fifth one just came out.
• 0:59
  SPEAKER_01: 150 CEOs of the top U.S.
• 1:01
  SPEAKER_01: companies fielded in January.
• 1:03
  SPEAKER_01: And what I like is why they started it.
• 1:06
  SPEAKER_01: They launched it back in 2021 to work on why perception of marketing was failing and why CMOs kept getting fired.
• 1:15
  SPEAKER_00: Yeah, so they spent five years tracking the same kinds of questions.
• 1:19
  SPEAKER_01: And their findings this year is almost our title.
• 1:22
  SPEAKER_01: Marketing has no shortage of data.
• 1:25
  SPEAKER_01: It is just not answering the questions CEOs genuinely care about.
• 1:29
  SPEAKER_01: Despite more measurement, confidence in marketing's impact has not improved.
• 1:34
  SPEAKER_00: Yeah, if you don't mind say let's just say that again, because it's easy to skip past more measurement, same confidence.
• 1:41
  SPEAKER_01: Same confidence.
• 1:42
  SPEAKER_01: Five years running.
• 1:43
  SPEAKER_01: And the strange part is that it's not a relationship problem.
• 1:47
  SPEAKER_01: 79% of CEOs say their CMO shows strong commitment to them and to the board.
• 1:53
  SPEAKER_01: 85% say the CMO builds trust across the organization.
• 1:57
  SPEAKER_00: Which is worth sitting with because the story marketing usually tells itself is that we're misunderstood or that nobody likes us.
• 2:05
  SPEAKER_00: And that's not what this survey is saying.
• 2:07
  SPEAKER_00: The relationship is actually good.
• 2:10
  SPEAKER_00: It's it's arguably great.
• 2:11
  SPEAKER_01: We're in the room, we're sitting at the table.
• 2:14
  SPEAKER_01: Exactly.
• 2:15
  SPEAKER_01: The relationship is fine.
• 2:16
  SPEAKER_01: It's not me, it's not you, it's all good.
• 2:18
  SPEAKER_01: We're good.
• 2:19
  SPEAKER_01: It is the work they do not rate.
• 2:21
  SPEAKER_00: Yeah, so 15% get a top performance rating.
• 2:25
  SPEAKER_00: 57% are seen as execution leaders rather than strategic ones.
• 2:29
  SPEAKER_00: And then 40% are rated C or below on their ability to drive strategy and growth.
• 2:34
  SPEAKER_01: So we won the seat at the table and lost the argument at the table.
• 2:37
  SPEAKER_00: Yeah, and Boulhoff frames it as the CMO is trending up while marketing itself trends down.
• 2:45
  SPEAKER_01: And it's not only CEOs.
• 2:47
  SPEAKER_01: Gardner asked CMOs in 2024 who inside their own company is most skeptical of marketing's value.
• 2:54
  SPEAKER_01: CFOs came first, CEOs came second.
• 2:57
  SPEAKER_01: So the two people who decide what you get to spend are the two least convinced.
• 3:02
  SPEAKER_01: That's not good.
• 3:03
  SPEAKER_00: And if we sort of follow the money, uh our own report we've done with Stone Olson verifies this as well.
• 3:09
  SPEAKER_00: So when we looked at local marketing budgets in Alberta, they've been dropping since the pandemic.
• 3:25
  SPEAKER_00: And only 16% of marketers said they were very confident in our long-term metrics.
• 3:30
  SPEAKER_01: 16% is a number I would put on a wall.
• 3:33
  SPEAKER_01: It is not that marketers do not believe in long-term effects.
• 3:36
  SPEAKER_01: Most of us do.
• 3:37
  SPEAKER_01: It is that almost none of us can show one.
• 3:40
  SPEAKER_00: So the pattern holds, whether you ask the CEO, the CFO, or the marketer.
• 3:45
  SPEAKER_00: More measurement, less belief, smaller budget.
• 3:49
  SPEAKER_01: All right.
• 3:50
  SPEAKER_01: So it's real.
• 3:51
  SPEAKER_01: Now why?
• 3:52
  SPEAKER_01: Yeah.
• 3:52
  SPEAKER_01: I think it can all be boiled down to four steps.
• 3:55
  SPEAKER_01: And none of them was a mistake at the time.
• 3:58
  SPEAKER_01: Okay, let's hear it.
• 3:59
  SPEAKER_01: V.
• 4:00
  SPEAKER_01: Yeah.
• 4:00
  SPEAKER_01: So step one, measurement gave us visibility.
• 4:03
  SPEAKER_01: That's a good thing.
• 4:04
  SPEAKER_01: Digital platforms let marketers demonstrate outcomes in a way genuinely we couldn't before.
• 4:09
  SPEAKER_01: For a hundred years we had Wanamaker.
• 4:12
  SPEAKER_01: Then we had a dashboard.
• 4:14
  SPEAKER_00: And we were thrilled, which half of my advertising is wasted.
• 4:19
  SPEAKER_00: That was such a great thing to have this dashboard that could tell us exactly what it was.
• 4:24
  SPEAKER_01: Exactly.
• 4:25
  SPEAKER_01: Like we were over the moon.
• 4:27
  SPEAKER_01: Step two though, visibility became accountability at the same time.
• 4:32
  SPEAKER_01: We took those numbers into the boardroom because they made us look rigorous.
• 4:36
  SPEAKER_01: That was a choice.
• 4:37
  SPEAKER_01: And at the time, it was the right one.
• 4:40
  SPEAKER_01: It got marketing taken seriously.
• 4:43
  SPEAKER_01: Yeah.
• 4:43
  SPEAKER_00: I I did this, I've done this like it a lot.
• 4:47
  SPEAKER_00: Um and probably everyone who's listening to this has also done this as well.
• 4:51
  SPEAKER_01: Absolutely.
• 4:53
  SPEAKER_01: Step three, accountability became optimization.
• 4:56
  SPEAKER_01: Once you are accountable for a number, the organization starts managing towards it.
• 5:02
  SPEAKER_01: This is not a conspiracy.
• 5:04
  SPEAKER_01: That is just the what organizations do.
• 5:07
  SPEAKER_01: You optimize what you report.
• 5:09
  SPEAKER_01: Yeah, and step four is the one that does a lot of damage.
• 5:13
  SPEAKER_01: Exactly.
• 5:14
  SPEAKER_01: Step four, what could be observed became what counted.
• 5:19
  SPEAKER_01: Attribution windows, conversion events, platform reporting periods.
• 5:23
  SPEAKER_01: All of it quietly became a proxy for effectiveness.
• 5:27
  SPEAKER_01: Nobody decided that.
• 5:28
  SPEAKER_01: It was just simply settled.
• 5:30
  SPEAKER_00: Yeah, there's this story from a physicist, amazing what you can find online when you look.
• 5:34
  SPEAKER_00: Yeah, there's this guy, Arthur Hennington, um, that I discovered in prepping for this uh sharp cut.
• 5:42
  SPEAKER_00: And and I keep coming back to this, and it's in this book from 1939 that he talked about.
• 5:49
  SPEAKER_00: Go on, man.
• 5:50
  SPEAKER_00: Okay, so the scientist goes out with a net and he trawls the ocean and catalogs everything he pulls up, and he concludes that there are no fish in the sea smaller than two inches.
• 6:02
  SPEAKER_00: And his data collection is clearly flawless.
• 6:04
  SPEAKER_00: He his records are perfect.
• 6:06
  SPEAKER_01: But I'm assuming the net has what, a two-inch mesh?
• 6:10
  SPEAKER_00: Exactly.
• 6:10
  SPEAKER_00: So the net with a two-inch mesh, is it really it's not the point, the point isn't that he's careless.
• 6:18
  SPEAKER_00: He's actually very careful and he's very thoughtful and he's doing things the right way.
• 6:23
  SPEAKER_00: The point is the conclusion about the net, though, ultimately, it's not about the ocean.
• 6:29
  SPEAKER_01: So essentially what you're saying is that's attribution.
• 6:32
  SPEAKER_00: Exactly.
• 6:33
  SPEAKER_00: We build a very good net, and then we publish the catch as a census, and and is ultimately our proof point.
• 6:42
  SPEAKER_00: But the net itself has got some problems.
• 6:44
  SPEAKER_00: It doesn't accurately represent how things are done.
• 6:47
  SPEAKER_01: Yeah.
• 6:48
  SPEAKER_01: So I need to ask this: where does leadership come into this, though?
• 6:51
  SPEAKER_01: Because so far it sounds like it's a technology technology story only.
• 6:55
  SPEAKER_00: Yeah, and I think it comes in at the question that probably a lot of us never really asked.
• 7:00
  SPEAKER_00: And and really that whole forced-up drift that you talked about earlier, V, which I thought was really great.
• 7:06
  SPEAKER_00: There isn't any point in time where someone stopped to say, over what period of time should this investment actually create economic value?
• 7:14
  SPEAKER_01: Which is really comes back to a strategy question, not really a measurement question.
• 7:18
  SPEAKER_00: Totally.
• 7:19
  SPEAKER_00: And and the the part that I think we have to be careful about, because the lazy version of this episode is marketing blaming someone else.
• 7:26
  SPEAKER_00: And that's not what we're trying to do.
• 7:28
  SPEAKER_00: We're not saying the marketers are stupid.
• 7:30
  SPEAKER_00: It's not the CFOs don't understand us.
• 7:34
  SPEAKER_00: It's not that the platforms are evil, I don't think.
• 7:37
  SPEAKER_00: Uh, although maybe I maybe are but like then what do you think is it then?
• 7:43
  SPEAKER_00: I think it's that there's already a look back window in the settings.
• 7:47
  SPEAKER_00: If you if you open up any any look, any platform, you can look at a seven-day attribution window, 28-day attribution window, whatever you want to look at.
• 7:58
  SPEAKER_00: And but the defaults that we're we're not deciding the defaults.
• 8:03
  SPEAKER_00: We're just accepting them.
• 8:06
  SPEAKER_01: You're right.
• 8:07
  SPEAKER_01: No one signs off on that default, right?
• 8:09
  SPEAKER_01: It's just there.
• 8:11
  SPEAKER_00: Yeah, so there's no default, and there's no meeting, there's no like internal conversation, there's no like a turn internal alignment, there's no memo.
• 8:20
  SPEAKER_00: It's it's just this box that we clicked on or off, uh, and then we accepted that, and then we reported on it.
• 8:28
  SPEAKER_00: And and then from that point on, everything that we've done after we accept that window, that box, it just becomes the math that we report.
• 8:38
  SPEAKER_01: I want to sit on that a little bit though, because it feels like you're putting that that failure on us.
• 8:44
  SPEAKER_00: I ultimately I think I am, and it's probably the only honest place to put it.
• 8:49
  SPEAKER_00: Like we're ultimately leaders.
• 8:51
  SPEAKER_00: And if we run an episode where the answer is that the board just doesn't get us, then we've ultimately said nothing.
• 8:56
  SPEAKER_00: Marking's been using that line for 40 years.
• 8:59
  SPEAKER_00: At some point, that stops being a diagnosis and just starts being an excuse.
• 9:04
  SPEAKER_01: Okay, but is there a difference between owning something and beating yourself up in public?
• 9:09
  SPEAKER_01: Like what does owning it actually look like?
• 9:12
  SPEAKER_00: Yeah, no, it's a it's a good question.
• 9:14
  SPEAKER_00: I don't think it's necessarily about beating us up or beating ourselves up, but it's about self-reflection and learning more than anything.
• 9:21
  SPEAKER_00: So let me give you an example.
• 9:23
  SPEAKER_00: A few years ago, we finished a year 8% up, and I presented it as a win.
• 9:29
  SPEAKER_00: And then um turns out there was a huge portion of that that was due to factors unrelated to what we were doing in my advertising, one of which was weather.
• 9:40
  SPEAKER_00: Um the conditions that um that year were really unusual, and they happened to run at the same time as our campaign, so we took the credit, and I didn't push hard enough to know whether the growth came from the category or from something else that we were specifically doing.
• 9:57
  SPEAKER_01: And I'm guessing you weren't the only one.
• 9:59
  SPEAKER_00: No, I like I worked in retail.
• 10:01
  SPEAKER_00: Like weather was a demand trigger in half the categories that we we sold in.
• 10:08
  SPEAKER_00: Um, you know, if there's no snow, nobody buys skis.
• 10:11
  SPEAKER_00: Uh, if hail comes through the city and the roofing industry has the best quarter in the decade.
• 10:17
  SPEAKER_01: Yeah.
• 10:18
  SPEAKER_00: Um, if we get a week of smoke sitting over Calgary, and that the any that will affect anything that uh depends on people going outside to do.
• 10:27
  SPEAKER_00: So the the dynamics of the in of each category, each market really have so many external factors, whether is it just the only one, but weather is a big one for a lot of businesses.
• 10:38
  SPEAKER_01: Well, and technically, if we were to kind of look at this loosely and say, like everything that you've just mentioned, all those businesses likely has a marketing team as well.
• 10:46
  SPEAKER_00: Yeah, and none of them made it snow, and all of them could have taken credit, and some of them did.
• 10:52
  SPEAKER_00: Like literally, there's a one example where we did where we sport check.
• 10:57
  SPEAKER_00: There was one day, September 30th, it snowed a foot and a half, and all of a sudden outerwear sales skyrocketed, and we were trying to figure out why, and like we were initially taking all the credit for it, and then we realized it was because there's a lot of snow.
• 11:12
  SPEAKER_01: So one of the 35 had a stronger force than anything that you did.
• 11:16
  SPEAKER_00: Yeah, and so this 35 thing is really about forces that affect ROI.
• 11:22
  SPEAKER_00: Um, there's at least 35 forces that affect ROI.
• 11:27
  SPEAKER_00: Advertising is like a fraction of them, and there's some that are stronger than anything than the other ones.
• 11:32
  SPEAKER_00: And so that's actually why I started building that list in the first place.
• 11:35
  SPEAKER_00: I want to know which factors can push hard and ones that barely move the needle.
• 11:42
  SPEAKER_00: Because until you know that, every year looks like we did a great job or we did a terrible job.
• 11:48
  SPEAKER_01: Yeah, marketing just makes it rain.
• 11:49
  SPEAKER_01: That's what we do.
• 11:51
  SPEAKER_00: Yeah, and that's the story we tell, especially when we think about demand gen, like just drop some money in the top of the funnel, the deals come out like Plinko chips, and and really ultimately, you know, we call it like, you know, I can make it rain.
• 12:03
  SPEAKER_00: But really, Mother Nature is the one that makes it rain.
• 12:06
  SPEAKER_00: And we just happen to be standing outside at the time.
• 12:09
  SPEAKER_01: There's a name for what you're describing.
• 12:11
  SPEAKER_01: Phil Rosenvick wrote a book back in 2007 called The Halo Effect.
• 12:16
  SPEAKER_01: His argument is that when a company is performing well, we look at everything it does and decide those things must be the reason why.
• 12:23
  SPEAKER_01: Strong culture, bold strategy, visionary leader.
• 12:27
  SPEAKER_01: When the same company stumbles, the same writers look at the same culture and call it complacent.
• 12:33
  SPEAKER_00: Yeah, that's a really, really good book.
• 12:36
  SPEAKER_00: Um, and nothing about that company changed really.
• 12:39
  SPEAKER_01: Nothing changed except the result.
• 12:41
  SPEAKER_01: And the result colored everything anyone said about the cause.
• 12:45
  SPEAKER_00: And the reason I think that works is that we almost never see the counterfactual.
• 12:49
  SPEAKER_00: So we we don't get to run the year again uh without the campaign to compare what the results are.
• 12:55
  SPEAKER_01: So the halo is what you conclude when you cannot see the counterfactual.
• 13:01
  SPEAKER_00: Yeah, and this is the net that that we were talking about earlier.
• 13:04
  SPEAKER_01: Yeah.
• 13:05
  SPEAKER_00: That you can't see.
• 13:06
  SPEAKER_00: So Rosensweig is describing the conclusion.
• 13:10
  SPEAKER_00: Eddington, the net guy, the physicist, is describing the instrument.
• 13:14
  SPEAKER_00: And they're the same failure at different points in the process.
• 13:17
  SPEAKER_01: So the failure is that we confuse observability with value.
• 13:21
  SPEAKER_00: Yeah.
• 13:22
  SPEAKER_00: I think that's the bit that's the key sentence here.
• 13:25
  SPEAKER_00: And and we took what we could see and we treated it as what was worth having.
• 13:30
  SPEAKER_01: Well, let me also put something concrete on the table here because there is a study out of Norway that is one of the more useful things published this year.
• 13:39
  SPEAKER_01: Kapero uh with Anfo and the Norwegian Media Association.
• 13:43
  SPEAKER_01: Uh, 13 companies, 2024 data, mix of B2B and B2C, they look at web analytics for all of them.
• 13:51
  SPEAKER_01: And it cut they cut it essentially in two ways.
• 13:54
  SPEAKER_01: First, the standard way, so from Google's perspective, out of the box, let's just go.
• 13:59
  SPEAKER_01: And Google accounts for about 60% of site traffic, 61% of conversions, and 60% of online revenue.
• 14:07
  SPEAKER_00: Yeah, it's it's a really good study, too.
• 14:09
  SPEAKER_00: It's really interesting to look at it.
• 14:11
  SPEAKER_00: And and which is the really the number when we talk about that default setting, it's the number that ends up on every deck.
• 14:18
  SPEAKER_00: So the default channel groupings, literally, is what they're looking at there.
• 14:22
  SPEAKER_00: Um, but then they recut the same search or web traffic, um, but they cut it differently and based it by intent.
• 14:34
  SPEAKER_00: So they reclassified things like brand searches versus non-brand searches and direct traffic counted as brand.
• 14:42
  SPEAKER_00: And if you think about it, it makes a lot of sense.
• 14:43
  SPEAKER_00: Like you type in the company's name into the URL, and that's like a branded search.
• 14:48
  SPEAKER_01: Yeah.
• 14:48
  SPEAKER_00: So when they did that and they reclassified and recategorized um the web traffic, um the pitcher really did invert.
• 14:57
  SPEAKER_00: So most of the value is coming from people who already had the brand in mind before they opened a search box.
• 15:03
  SPEAKER_01: So look at this.
• 15:04
  SPEAKER_01: Like it's the same data, two different answers.
• 15:07
  SPEAKER_00: Totally.
• 15:08
  SPEAKER_00: And the that's a really important part to underline.
• 15:11
  SPEAKER_00: Caparo um doesn't say that Google's number is wrong.
• 15:15
  SPEAKER_00: They specifically call that out.
• 15:16
  SPEAKER_00: They're just saying this is a valid look at it, but this is an equally valid view of the data that Google's providing for us.
• 15:27
  SPEAKER_00: It just cut a different way.
• 15:29
  SPEAKER_00: So nobody made an error.
• 15:31
  SPEAKER_00: Um, nobody is like totally wrong for reading what was there.
• 15:35
  SPEAKER_00: They just flipped the answer, and and it's not a measurement failure.
• 15:38
  SPEAKER_00: It's really a framing choice that nobody knew we were making.
• 15:42
  SPEAKER_00: It just was provided for us.
• 15:45
  SPEAKER_01: And the sharpest number in there is the retail one.
• 15:49
  SPEAKER_00: Yeah, and in retail B2C, uh, and again, this is a small study, but it's it's interesting looking at.
• 15:55
  SPEAKER_00: And I think if you anybody's interested, you could take a look at the study and just try to reclassify your own uh web traffic.
• 16:04
  SPEAKER_00: But non-brand keywords uh take 92% of the Google investment.
• 16:09
  SPEAKER_00: Brand keywords take 8%.
• 16:11
  SPEAKER_00: And that 8% delivers 51% of the conversions and 49% of the online revenue across the whole sample.
• 16:18
  SPEAKER_00: And a non-brand conversion costs about six times more than a brand one.
• 16:22
  SPEAKER_00: So it's a really interesting look view of it.
• 16:25
  SPEAKER_01: And the reason that happens is organizational, which is the bit that got me really, because it maps onto those four steps exactly.
• 16:35
  SPEAKER_01: Marketing departments are organized by channel.
• 16:38
  SPEAKER_01: Each channel team analyzes its own impact using its own tool, and every one of them concludes that their own investment should go up.
• 16:46
  SPEAKER_01: Totally.
• 16:47
  SPEAKER_00: And every team is a net, and every net comes back full.
• 16:52
  SPEAKER_01: Yeah, every net comebacks full.
• 16:54
  SPEAKER_01: And Comparo, quote, CFO saying a version of the same thing.
• 16:58
  SPEAKER_01: Add up with all the channels are reporting, and sales should be far higher than they actually are.
• 17:04
  SPEAKER_01: I remember you and I experienced that.
• 17:06
  SPEAKER_00: Yeah, totally.
• 17:07
  SPEAKER_00: We've both been in that meeting.
• 17:09
  SPEAKER_00: We've both been guys going, look at how much ROAS I'm getting you.
• 17:12
  SPEAKER_00: Why would you ever stop giving me money?
• 17:15
  SPEAKER_01: Everybody's been in that meeting.
• 17:18
  SPEAKER_00: It's not just us.
• 17:19
  SPEAKER_00: It's not just us, Matt.
• 17:21
  SPEAKER_00: And as I mentioned, the study has limits and they themselves call it out.
• 17:24
  SPEAKER_00: Um so 13 companies is small.
• 17:26
  SPEAKER_00: It is isolated to Norway.
• 17:28
  SPEAKER_00: And so, you know, it's worth looking into for yourself.
• 17:31
  SPEAKER_00: They say directly that they're measuring performance is observed not as a casual causal lift, um, and that controlled incrementality experiments would be real ultimately the proper next step.
• 17:42
  SPEAKER_00: And they also count direct traffic as brand traffic, which could inf be inflated by a subscription business because a lot of uh the subscribers that would be logging in are going to pay a bill.
• 17:54
  SPEAKER_01: So essentially they told you that the mesh size of their own net.
• 17:57
  SPEAKER_00: Yeah, exactly.
• 17:58
  SPEAKER_00: And that's the difference.
• 17:59
  SPEAKER_00: Um and it sets up, I think, a second example um from our own industry.
• 18:05
  SPEAKER_00: So so we we sell podcast advertising.
• 18:08
  SPEAKER_01: We do.
• 18:09
  SPEAKER_00: Um, and we're about to be a critic of a metric that ultimately flatters our own medium's biggest platforms.
• 18:16
  SPEAKER_00: So, you know, keep take this with a grain of salt.
• 18:19
  SPEAKER_01: Yeah, and it should not stop us by saying the rest, though.
• 18:23
  SPEAKER_00: No.
• 18:23
  SPEAKER_00: Uh so there's a study that uh was done by Podscribe.
• 18:27
  SPEAKER_00: So more than 50 direct response brands over 30 days measuring pixel-based conversion uh by listening source.
• 18:34
  SPEAKER_00: So podcast apps came in at the number's funny to me, 0.1%.
• 18:43
  SPEAKER_00: Uh web players at 0.04 and and smart speakers at 0.024.
• 18:52
  SPEAKER_01: And what they did with that is an interesting part.
• 18:54
  SPEAKER_01: They asked whether advertisers should start excluding the lower performers, which is a completely reasonable question if conversion rate is the only thing that you're looking to optimize.
• 19:06
  SPEAKER_00: Yeah, and the lowest performer is smart speakers.
• 19:09
  SPEAKER_00: So we're still talking about the best case scenario.
• 19:14
  SPEAKER_00: 99% of people do not click through or do not convert on a click.
• 19:19
  SPEAKER_00: Uh anyway, the lowest performer is smart speakers.
• 19:23
  SPEAKER_00: And so their explanation for why, I think, is pretty good that people are passively listening, people are half attending or paying attention to this while they cook or drive or do some other thing.
• 19:35
  SPEAKER_00: It's not their objective, it is not to convert on a podcast.
• 19:40
  SPEAKER_01: No, but it is where brand memory gets built.
• 19:44
  SPEAKER_00: Yeah, it is.
• 19:46
  SPEAKER_00: Like I literally we both listen to uh smartless.
• 19:49
  SPEAKER_00: I've been listening to that.
• 19:51
  SPEAKER_00: I keep hearing Helix forever.
• 19:53
  SPEAKER_00: Actually, I'm in the market now and I'm looking at Helix products.
• 19:58
  SPEAKER_00: Um, and the Pixel can't see that though.
• 20:01
  SPEAKER_00: So if it comes back as close to zero, you know, the natural inclination is to just cut it.
• 20:07
  SPEAKER_01: Yeah, I do want to be really careful here though, because I think our side overclaims on this as well.
• 20:13
  SPEAKER_01: And what like what do you mean by that?
• 20:15
  SPEAKER_01: The usual move is to say only 5% of buyers are in market.
• 20:19
  SPEAKER_01: So a conversion rate is meaningless, but that is a B2B number, and these are direct response brands.
• 20:26
  SPEAKER_01: Supplements, meal kits, subscriptions, those categories turn over every few weeks.
• 20:32
  SPEAKER_01: A much bigger share of that audience genuinely is in market.
• 20:36
  SPEAKER_00: Yeah, it's true.
• 20:37
  SPEAKER_00: Um, it was originally a B2B number, but it's but it has been done a lot of other places, and I and James Herman's got a great book called Future Demand, and he looked at actual demand in a whole bunch of, I think it was 14 or 15 different categories.
• 20:52
  SPEAKER_00: So the 595 is a rule of thumb.
• 20:56
  SPEAKER_00: And so the argument here isn't that the 95% are of people are ineligible.
• 21:02
  SPEAKER_00: The argument really is ultimately that nobody in the analysis established the purchase cycle of a single brand in the sample before recommending which platforms to cut.
• 21:12
  SPEAKER_01: Yeah, nobody asks how long the fish take to come back.
• 21:15
  SPEAKER_00: Yeah, and I think that's a contrast between the two studies.
• 21:18
  SPEAKER_00: One essentially tells you the shape of its own net, if you want to keep going down this analysis.
• 21:24
  unknown: Yeah.
• 21:24
  SPEAKER_00: And says you got to run your own analysis.
• 21:26
  SPEAKER_00: The other one tells you which platforms to stop funding.
• 21:30
  SPEAKER_01: And technically that net, if we keep using that analogy, has real two has two dimensions.
• 21:36
  SPEAKER_00: Yeah.
• 21:37
  SPEAKER_00: So the mesh size is ultimately who you can see, and the time in the water is the window.
• 21:43
  SPEAKER_00: So both of them get mistaken for value.
• 21:46
  SPEAKER_01: Humor me for a second.
• 21:47
  SPEAKER_01: If the window is the problem, what about the number at the end of it?
• 21:51
  SPEAKER_01: Because everything we have described essentially gets compressed into, I don't know, ROI.
• 21:58
  SPEAKER_00: Yeah, and we are probably going to end up.
• 22:00
  SPEAKER_00: Doing more on ROI later, but ROI ultimately, I've said this a lot.
• 22:04
  SPEAKER_00: It's such a, I think, a bad number.
• 22:06
  SPEAKER_00: Then here's the math.
• 22:08
  SPEAKER_00: Instead of my opinion, as I mentioned, there's at least 35 factors that affect marketing ROI.
• 22:14
  SPEAKER_00: So you know I want to be careful with that number because it's not a complete list.
• 22:18
  SPEAKER_00: It's not absolute for everyone.
• 22:19
  SPEAKER_00: The factors that are strong for one industry aren't necessarily strong for the other.
• 22:23
  SPEAKER_00: So there's a lot of things that go into this, but let's just go with the 35 number.
• 22:28
  SPEAKER_01: The reality is, how many of those 35 does marketing really control anyway?
• 22:32
  SPEAKER_00: Yeah.
• 22:33
  SPEAKER_00: So if we put media and creative in one group, it's probably in the range of like five or six out of the 35.
• 22:43
  SPEAKER_00: Okay.
• 22:43
  SPEAKER_00: Um, so if you give marketing maybe a share of physical availability as well, like we say like search ads, let's say, are are like part of that, and maybe some signage in store and things like that, you might get to about eight.
• 22:57
  SPEAKER_00: So if we call it somewhere in the range of 15 to 20 percent of the inputs at the top end, I think, you know, things that aren't included would be like product inventory supply and supply chains and things like that, right?
• 23:11
  SPEAKER_00: So marketing gets graded on the whole outcome.
• 23:14
  SPEAKER_00: If your cost of good moved or competitor launch or the category shrank, all that is inside your ROI number and none of it is related to marketing.
• 23:24
  SPEAKER_01: And if we cannot even name all the inputs, then grading the output is worse than it looks.
• 23:29
  SPEAKER_01: You're not just being held to things that you don't control, you're being held to things nobody has even listed.
• 23:36
  SPEAKER_00: Totally.
• 23:36
  SPEAKER_00: And then there's the ratio itself, which is a separate problem.
• 23:39
  SPEAKER_00: So ROI is return over investment.
• 23:44
  SPEAKER_00: And so, you know, the that formula is what I'm thinking about here.
• 23:48
  SPEAKER_00: And and ultimately there's two ways you can improve it.
• 23:51
  SPEAKER_00: So you either grow the numerator or you shrink the denominator.
• 23:54
  SPEAKER_00: And shrinking is like, in other words, cutting, like your marketing investment is the easier and faster way, and it's ultimately entirely inside your control.
• 24:06
  SPEAKER_01: So the highest ROA campaign you will ever run is the smallest one as well.
• 24:12
  SPEAKER_00: Yeah.
• 24:12
  SPEAKER_00: So if you spend$1,000 and you make$5,000, that's a 5x return.
• 24:16
  SPEAKER_00: But if you cut your budget and spend$500, all of a sudden that's a 10x return on the same amount of money.
• 24:24
  SPEAKER_01: Yeah.
• 24:25
  SPEAKER_00: And the dashboard shows the small one wins.
• 24:28
  SPEAKER_00: So the but the business actually it almost did nothing to the business.
• 24:33
  SPEAKER_01: Nobody said sell to optimize for smallness.
• 24:36
  SPEAKER_01: But if that is the number on the wall, that is the direction the system naturally drifts to.
• 24:42
  SPEAKER_00: Yeah.
• 24:42
  SPEAKER_00: And there's a third hole in this, which is one we've kind of been circling around through this whole episode.
• 24:48
  SPEAKER_01: You're talking about the time horizon.
• 24:51
  SPEAKER_01: Because the R in ROI gets measured over whatever window the tools can see.
• 24:56
  SPEAKER_01: And almost every tool we have is short-term, is essentially a short-term instrument.
• 25:02
  SPEAKER_01: So the return we calculate is the return that fits inside that instrument.
• 25:06
  SPEAKER_01: The long-term return is not excluded because somebody decided that it shouldn't count.
• 25:12
  SPEAKER_01: It just never enters the calculation.
• 25:14
  SPEAKER_00: Yeah, and that's that time in the net or the for the net analogy, that's the time in the water part.
• 25:20
  SPEAKER_01: Yeah, I love that.
• 25:21
  SPEAKER_01: And then underneath all three of those, there is even a bigger one, which is that nobody has agreed what the R means in the first place.
• 25:31
  SPEAKER_00: Yeah, I'd love to hear more on this, V.
• 25:34
  SPEAKER_00: What do you think about that?
• 25:35
  SPEAKER_01: Honestly, just ask four people in a business what return means, and you'll get four different answers.
• 25:40
  SPEAKER_01: Finance means profit, the CEO might think it's revenue growth, the board might think it's share price, marketing means whatever the dashboard is reporting that month, and nobody says it out loud.
• 25:53
  SPEAKER_01: So all four can can take and look at the same number, agree on it, and be talking about a completely different thing every time.
• 26:02
  SPEAKER_00: Yeah, that's true.
• 26:03
  SPEAKER_00: And and and we've talked about this a little bit before, but revenue and profit are absolutely not the same thing.
• 26:09
  SPEAKER_00: No.
• 26:10
  SPEAKER_00: No.
• 26:10
  SPEAKER_00: You can buy revenue, plenty of businesses have, and some of them have even gone under for doing it.
• 26:16
  SPEAKER_01: Plenty have.
• 26:18
  SPEAKER_01: So defining return is not a measurement task.
• 26:21
  SPEAKER_01: It is decision about what the business is in for, which is the same failure as the window.
• 26:27
  SPEAKER_01: The definition arrived preset and nobody decided to look into it.
• 26:31
  SPEAKER_00: Yeah, and I'd push on one part of that.
• 26:33
  SPEAKER_00: You're describing it as something that's handed down to us.
• 26:36
  SPEAKER_00: But again, I think this is where the leadership part of our conversation comes into play.
• 26:40
  SPEAKER_00: It's more often that it's some, it's something that's nobody really ever asked out loud, which means the first person to put up their hand and ask the dumb question gets to shape the answer.
• 26:52
  SPEAKER_01: And I think this is where you and I do not agree.
• 26:56
  SPEAKER_01: We finally do not agree on something.
• 26:58
  SPEAKER_00: Well, maybe, but let's find out.
• 27:03
  SPEAKER_01: Let me put a room on the table first because I don't want to sound make this sound like a theory at all.
• 27:09
  SPEAKER_01: Uh, I have been in a room where we've had evidence that longer-term investment was doing exactly what was expected it to do.
• 27:16
  SPEAKER_01: We had brand search volume was increasing, organic demand was moving.
• 27:21
  SPEAKER_01: The leading indicators were all there for us to continue to go.
• 27:25
  SPEAKER_00: So the case, so you're making a case for long-term investment, is what you're saying.
• 27:30
  SPEAKER_01: Exactly.
• 27:30
  SPEAKER_01: The case was made, but we were also behind our loading targets that quarter.
• 27:35
  SPEAKER_01: And when you're sitting there with a gap that has to close within the next few weeks, nobody in that room says, hey, this is great, V.
• 27:44
  SPEAKER_01: Let us wait six months to see how the brand investment's going to mature.
• 27:49
  SPEAKER_01: We immediately moved money toward conversion.
• 27:53
  SPEAKER_01: And obviously, you fought it.
• 27:56
  SPEAKER_01: No, that's the thing.
• 27:58
  SPEAKER_01: This is the uncomfortable honesty, it's the real uncomfortable part.
• 28:02
  SPEAKER_01: I agreed with the decision.
• 28:04
  SPEAKER_01: Given the window the business was managing against, it was the rational call, which is why I don't think marketing can simply declare a different measurement horizon.
• 28:15
  SPEAKER_01: The evidence can be completely credible and still lose to the time frame that the organization is accountable to.
• 28:22
  SPEAKER_00: Savia, I want to push on that a little bit because you didn't really tell me a story where someone overruled you.
• 28:29
  SPEAKER_00: Yeah, but why would that matter?
• 28:32
  SPEAKER_00: Because the window gets chosen before finance ever sees it.
• 28:38
  SPEAKER_00: Um it gets chosen when when you set the campaign up, when you pick up the reporting cadence, when you decide which slides go in the deck, by by the time the CFO is looking at anything, six or seven of those choices have already been made by us, quietly, mostly, by accepting whatever was already there.
• 28:56
  SPEAKER_01: So you're saying the authority exists and we choose not to use it?
• 29:00
  SPEAKER_00: I'm saying that nobody's stopping you from writing down in advance the period over which you expect this to pay off.
• 29:08
  SPEAKER_00: It it actually doesn't cost anything, it and it doesn't need budget approval, and it changes, I think, the conversation because and I'm saying it's easy, but now you have made a claim that you could be held to, which isn't always about sales today.
• 29:25
  SPEAKER_01: And my counter to that is that declaring it and having it survive a bad quarter are two very different things.
• 29:32
  SPEAKER_00: I totally agree.
• 29:33
  SPEAKER_00: That's totally fair.
• 29:36
  SPEAKER_00: Um and and honestly, the best answer here is always that it depends.
• 29:40
  SPEAKER_00: Um and then that we're probably ultimately both right.
• 29:45
  SPEAKER_00: Um, and where I land on this is that marketing, I think, needs to make the first move.
• 29:50
  SPEAKER_00: I think, again, a James Herman quote, I always think about this.
• 29:53
  SPEAKER_00: It keeps he's got a great line of about marketing isn't sales done over media.
• 29:59
  SPEAKER_00: And I think we have to stop treating it like sales done over media and make a better defense around how it works.
• 30:06
  SPEAKER_00: Um, it's not like putting money into the plinkerboard and watching the cash come out at the bottom.
• 30:12
  SPEAKER_00: It does that sometimes, but not all the time.
• 30:14
  SPEAKER_00: And it's not everything that you can't force people to buy who aren't in market.
• 30:19
  SPEAKER_00: So nobody in finance is going to walk down the hall and ask us to measure over a longer period.
• 30:25
  SPEAKER_00: You don't go away, change the scorecard, and then present it after the fact.
• 30:29
  SPEAKER_01: Yeah, what you're suggesting is you bring finance in while you're trying to do that.
• 30:33
  SPEAKER_01: Totally.
• 30:34
  SPEAKER_00: Which is, I think, the actual difference between your position and mine, it it's the same action, but it's just a different sequence.
• 30:43
  SPEAKER_01: Well, yeah, I agree with that being a different sequence.
• 30:46
  SPEAKER_01: And the sequence is the whole thing, really, because my worry is credibility.
• 30:51
  SPEAKER_01: A marketer who changes the measurement on their own looks, like they're moving the goalpost because the numbers got bad.
• 31:00
  SPEAKER_00: Yeah.
• 31:01
  SPEAKER_00: And we have so many numbers, and that we can always change to a new number if we want it to be a new KPI.
• 31:08
  SPEAKER_00: And all but I ultimately think having finance the room while you do it is a thing that stops this from happening.
• 31:15
  SPEAKER_00: There's an explanation for it.
• 31:16
  SPEAKER_00: They're looking for ideas for decreasing risk.
• 31:20
  SPEAKER_00: And so I think we should go first, not just go alone.
• 31:25
  SPEAKER_01: Yeah, and at the end of the day, winning is a team sport.
• 31:29
  SPEAKER_00: Which takes us to the thing that turns this from a measurement conversation to a leadership conversation, V.
• 31:35
  SPEAKER_00: All right, which is uh that the boathouse study that's been running for five years, uh, five years or of more measurement and no improvement in confidence.
• 31:45
  SPEAKER_00: And if you've been using the same instrument for five years and it hasn't moved, the only thing you need it to move, then continuing to use it is a decision.
• 31:55
  SPEAKER_01: Sure.
• 31:55
  SPEAKER_00: It reminds me of that that commercial.
• 31:58
  SPEAKER_00: Choosing not to make a choice is still a choice.
• 32:01
  SPEAKER_01: Yeah, yeah, yeah.
• 32:01
  SPEAKER_00: Uh it's not an inheritance anymore.
• 32:04
  SPEAKER_00: We have a choice.
• 32:05
  SPEAKER_01: So it's like once you know, you have chosen.
• 32:09
  SPEAKER_00: Yeah, and I think that's ultimately the leadership act.
• 32:11
  SPEAKER_00: It's not courage per se.
• 32:13
  SPEAKER_00: It's not being braver in the budget meeting.
• 32:16
  SPEAKER_00: It's ultimately choosing the instrument.
• 32:18
  SPEAKER_00: And there's nothing wrong with that, I want to say.
• 32:21
  SPEAKER_00: No, there's nothing wrong with a net.
• 32:23
  SPEAKER_00: The net's good.
• 32:24
  SPEAKER_00: It's a good net.
• 32:24
  SPEAKER_00: It catches what swims past it this month and it tells you accurately what got caught.
• 32:29
  SPEAKER_00: It just can't tell you where the fish come from it or whether the the volume of fish is growing or whether it's moved uh to a different area or the school has moved to a different area.
• 32:41
  SPEAKER_01: There is a detail in that editing story I didn't know until recently, which is he points out somebody could work out the first law, the one about the fish under two inches, without ever going to the sea.
• 32:57
  SPEAKER_01: They do not need to see the data, they just need to look at the net.
• 33:01
  SPEAKER_01: Yeah, that's a good point.
• 33:02
  SPEAKER_01: Actually, that's a really good point.
• 33:04
  SPEAKER_01: You don't actually need the catch.
• 33:07
  SPEAKER_01: No, you don't need to catch it all.
• 33:09
  SPEAKER_01: Which is the uncomfortable version for us as marketers?
• 33:13
  SPEAKER_01: You can predict what a marketing team is going to conclude by looking at the attribution window before the campaign even runs.
• 33:21
  SPEAKER_00: Yeah, and and that should really bother everybody listening, I think, because it means the finding was decided in the setup.
• 33:29
  SPEAKER_01: So for the things that the can the net cannot catch, you need a different instrument to measure.
• 33:35
  SPEAKER_00: Right.
• 33:36
  SPEAKER_00: So things like category demand, market penetration, mental availability, category entry points, sheriff search, brand tracking, geo holdouts, and matched market tests.
• 33:45
  SPEAKER_00: None of it is exotic, really.
• 33:48
  SPEAKER_00: They're all things we've talked about before, but and and a lot of it is actually cheaper than most people assume, and definitely cheaper than what it used to be.
• 33:58
  SPEAKER_01: And a fleet that only owns nets will always conclude the ocean is the size of its catch.
• 34:05
  SPEAKER_01: Yeah, the whole thing.
• 34:07
  SPEAKER_01: Yeah.
• 34:07
  SPEAKER_01: So what does someone do on a Monday?
• 34:10
  SPEAKER_00: Well, I think there's four questions that you try and answer before you choose a measurement method, not after you start reporting.
• 34:20
  SPEAKER_00: So the first one is what economic outcome are we actually trying to create?
• 34:24
  SPEAKER_00: The second, over what period should that reasonably occur?
• 34:29
  SPEAKER_00: The third, what evidence would tell us if it was incremental?
• 34:33
  SPEAKER_00: And four, what uncertainty are we prepared to tolerate?
• 34:38
  SPEAKER_01: And I would add the one from earlier.
• 34:40
  SPEAKER_01: Does everyone in the room mean the same thing by quote unquote return?
• 34:45
  SPEAKER_01: Ask it out loud before anybody reports anything or against it.
• 34:50
  SPEAKER_00: Mm-hmm.
• 34:51
  SPEAKER_00: That's a great question.
• 34:53
  SPEAKER_00: Uh because if you pick the method first, the method answers half of those for you, and then you never notice it happened.
• 35:00
  SPEAKER_01: And there is a version of this in finance that makes it obvious.
• 35:04
  SPEAKER_00: Yeah.
• 35:04
  SPEAKER_00: So finance would never evaluate a long-lived asset without first estimating its useful life.
• 35:10
  SPEAKER_00: So you don't buy a plant.
• 35:12
  SPEAKER_00: I don't mean a like a tree plant.
• 35:14
  SPEAKER_00: I mean like a plant plant, like a factory.
• 35:16
  SPEAKER_01: Factory.
• 35:17
  SPEAKER_00: And then ask accounting software how long it should last.
• 35:20
  SPEAKER_00: But marketing routinely lets a platform's attribution window become the de facto useful life of an ad exposure.
• 35:28
  SPEAKER_01: Yeah, nobody decided 28 days was the useful life of a brand impression.
• 35:33
  SPEAKER_01: It just was set up by default.
• 35:35
  SPEAKER_01: Exactly.
• 35:36
  SPEAKER_01: So to land it, marketing's effectiveness problem is not a measurement problem.
• 35:41
  SPEAKER_01: It is a leadership problem.
• 35:42
  SPEAKER_01: And measurement is where it simply shows up.
• 35:45
  SPEAKER_01: The moment you choose the window, you are making a strategic decision about what value counts and what value does not.
• 35:53
  SPEAKER_00: Yeah, and if you choose the window, then everything after is there's the math.
• 35:58
  SPEAKER_01: Next time we're gonna do the other half.
• 36:00
  SPEAKER_01: If the window is a choice, what is simply the method?
• 36:03
  SPEAKER_01: That is the discounted cash flow conversation.
• 36:06
  SPEAKER_01: And we have an interview coming up that sets us up perfectly.
• 36:10
  SPEAKER_00: So until next time, thanks for keeping us in your ears.
• 36:13
  SPEAKER_00: Stay sharp, everyone.