Latest / Scalability School / We Tested Meta's Incremental Attribution on $200K/Month: Here's Why We Turned It Off.

Transcript

We Tested Meta's Incremental Attribution on $200K/Month: Here's Why We Turned It Off. · Scalability School

0:00
Because just due to the nature of how one
0:03
day click and incremental work, kind of
0:05
like we said earlier, one day click is
0:06
only giving credit to
0:07
people who click within 24 hours.
0:09
By default, it is substantially more
0:11
conservative than
0:12
incremental attribution.
0:13
So we don't want to be
0:14
misled by one day click.
0:16
What should we do?
0:17
That third data point that I'm alluding
0:18
to is split it down to
0:20
evergreen and promo ads.
0:22
It still ended up being the case that
0:23
evergreen consistently is
0:24
stacking up month over month.
0:25
And that's what compounds, right?
0:26
It's like if you can make more ads every
0:29
single month that earns spend in the ad
0:31
account, then that's what compounds over
0:33
time and allows you to increase spend.
0:35
I think a lot of brands
0:36
actually kind of forget about that.
0:37
They're like, hey, they didn't buy.
0:39
We did all these like whatever flows to
0:42
get them to maybe get a
0:43
bigger discount over time.
0:44
But then there was probably one reason
0:46
why they didn't buy.
0:47
So talk to that customer about that one
0:48
reason why they didn't buy and give them
0:49
a strong discount to still close.
0:51
Because like it's very likely better off
0:53
when you still have contribution margin
0:55
on that customer to be gained if you just
0:57
give them even a slightly bigger discount
0:59
to like bring in those dollars versus
1:00
them just sitting on
1:01
an email and SMS list.
1:09
And now let's take a listen to the
1:11
Scalability School podcast.
1:14
Welcome back to another episode of the
1:16
Scalability School podcast.
1:17
Andrew is allowing me to have the
1:20
absolute honor of introducing the
1:22
podcast, opening it up for reasons that
1:24
he will describe shortly.
1:26
He sounds beautiful, but he's going to
1:28
let me open this sucker up.
1:30
We have a couple of
1:30
topics to jump into today.
1:32
We're going to talk about
1:33
incremental attribution.
1:34
We're going to talk
1:34
about creative cohorts.
1:35
And if we are blessed and lucky, Zach
1:38
Stuck will be joining us
1:39
for the back half of this.
1:41
We're going to talk
1:41
about scaling ads, man.
1:43
It's going to be dope.
1:44
But look, as you can hear, I'll sing you
1:46
a little with my husky voice.
1:48
I'm in the mood for love.
1:50
That's where I'm at
1:51
right now with my bad voice.
1:54
I've lost my voice, but
1:55
it's not totally gone.
1:57
I just sound like I'm struggling to
1:58
breathe, which is true.
2:00
So and my laughs sound like I've been
2:03
smoking for 40 years.
2:04
Other than that, doing
2:06
great, feeling great.
2:07
These are aura points.
2:08
You're just stacking
2:09
aura points right now.
2:10
So much aura.
2:12
Yeah, people are going to actually
2:13
require me to do podcast appearances with
2:15
this voice from now on.
2:17
So I'll have to 11 labs
2:18
it and figure that out.
2:20
I think that, you know, one of the things
2:21
we love is tactical episodes.
2:23
And that's what we're
2:25
trying to get into here, right?
2:26
Which is around incremental attribution
2:29
and versus one day click.
2:32
I'm really excited to talk about it.
2:34
So let's go through this.
2:37
Like, I mean, we have some data from the
2:40
founders community as
2:41
well, which we'll talk about.
2:42
But Brett, frame it up for us.
2:44
Yeah.
2:45
So we've been thinking a lot about
2:47
incremental attribution
2:48
since meta rolled it out.
2:50
And I have an N of one kind of sample of
2:54
a recent experience that we've had where
2:57
we went through kind of the
2:59
disappointment, excited, back to
3:02
disappointment and trying to figure out
3:03
where we go with incremental attribution.
3:05
So if it's something that you think, if
3:07
it's something that you've been thinking
3:08
about testing, maybe you've already
3:10
tested and you've had a similar
3:11
experience, hopefully you can lean on
3:13
this to kind of decide, is it worth
3:14
testing in our account?
3:15
And should we expect that there's going
3:17
to be meaningful performance swings?
3:21
Because, you know, everybody's always
3:22
looking for, I mean, I'm kind of the guy
3:24
that always asks everybody for a hack at
3:26
the end of the episode because I like to
3:27
hear the tactical nuggets.
3:29
So hopefully you walk away with a kind of
3:30
an understanding of what to do with
3:33
incremental attribution from here.
3:35
But I can walk through, I actually have a
3:37
bunch of notes in screen shares and I can
3:38
just walk through a timeline of a test
3:41
that we ran through with a
3:42
client and their experience.
3:43
And then Andrew, you can always interrupt
3:45
me and we can ask questions
3:47
and dig into anything deeper.
3:49
Does that sound good?
3:50
Yeah, definitely.
3:51
I mean, I think I'll just mention the
3:52
community stuff out of the gates, which
3:54
is people had seen it
3:57
underperform thus far utilizing IA.
4:02
Right.
4:03
So it depends.
4:05
Like one member said, he saw that it was
4:09
especially on remarketing review through
4:11
attribution on past
4:12
purchases was inflating numbers.
4:14
Sees IA aligned well
4:16
with seven day click.
4:18
Another member said that incremental ROAS
4:21
and ads manager often lands around 90% of
4:24
seven day click ROAS
4:25
when you break it down.
4:27
And Barry Hot legendary said he thinks
4:32
it's good 90% of the problems he sees
4:35
because it solves those problems, 90% of
4:36
the problems he sees,
4:37
especially on bigger accounts.
4:39
So it's interesting.
4:41
It's like, it's interesting.
4:42
It's kind of like people said hasn't been
4:44
as good, but then like overall we're
4:45
like, I can see where it's happening.
4:47
And it's going to be better.
4:48
So yeah, I'd love to walk through it.
4:51
Sweet.
4:52
I will flip this over.
4:54
You're watching this on youtube.com,
4:56
which is a website on the
4:57
internet that shows videos.
4:59
Brad is sharing a screen, but if you're
5:01
not, you know, and you're just having it
5:03
in your ears, then, you know, I'm sorry,
5:07
but you can go and look at
5:08
the visuals on youtube.com.
5:11
Somebody the other day
5:12
I met at a conference.
5:13
This is true.
5:14
And said, who is the other guy when he
5:17
met me on the, on the podcast.
5:20
And he goes, I love
5:21
his, I love his voice.
5:22
I've listened to every episode twice.
5:25
So some people just come for the audio.
5:28
Hey, if this helps you fall asleep at
5:30
night, that is an acceptable, is an
5:32
acceptable listening method.
5:33
Kind of an honor really.
5:35
Yeah, we're just trying to, you know,
5:36
we're just trying to
5:36
smash records around here.
5:39
So, okay.
5:41
I will also try to explain what's on the
5:42
screen for everybody
5:43
that's, that's listening.
5:44
So let me, let me frame this up with a
5:46
little bit of context ahead of time.
5:47
First off, this brand
5:48
is in the apparel space.
5:50
Their AOV is over a hundred dollars, not
5:52
quite one 50, but it's
5:53
kind of in that range.
5:54
Depends on the seasonality and what
5:56
products are popular.
5:57
And they have actually a
5:58
fair amount of organic traffic.
6:01
They influencer like
6:02
posts, things like that.
6:03
So if that's helpful for context for, for
6:06
brands that are thinking about this.
6:08
So the beginning of the year, we went
6:10
through a bunch of different things with
6:11
this, with this brand.
6:12
And we started to test meta conversion
6:14
live studies directly in the platform to
6:16
try and tease out how incremental is
6:18
meta, et cetera, et cetera.
6:19
So here's what I can tell you.
6:21
So their average one day click row is,
6:23
which, which shows up, shows up here as a
6:25
3.12 for, for the account or basically
6:28
year to date up to this point is very,
6:30
very close to the actual conversion live
6:32
data results that we got back from meta.
6:34
So there's like almost no difference
6:35
between their one day click and the
6:38
incremental number that we
6:39
got back from, from meta.
6:41
So that's encouraging because that means
6:42
that we can use one day click as a, as
6:45
this, I don't know if
6:46
barometer is the right word.
6:47
We can use it as a temperature check to
6:48
say, Hey, if we're going to hold our
6:51
performance accountable to something,
6:52
this one day click
6:53
number is reasonable for them.
6:54
And one day click is probably, you know,
6:56
like I said, for the organic traffic
6:58
reasons that I mentioned, one day click
7:00
is probably more applicable to them for
7:02
them than the average
7:03
account where seven day click.
7:05
You know, I think there's this, this,
7:06
this stat that's floating around
7:07
recently, which is 70 click kind of under
7:09
reports by 15 to 20%.
7:10
We generally see that
7:11
except for this account.
7:12
So just, just kind of framing that up.
7:14
So we ran a conversion of study to start
7:15
the year, came back and basically said,
7:17
Hey, your one day click row eyes number
7:18
is basically your, your, your, um,
7:21
incremental number, which is different
7:22
than incremental, uh, attribution.
7:24
Those are, those are different things.
7:25
So just framing that up.
7:27
So what we did is we ran through that
7:28
conversional test and we started to get
7:30
through the rest of, um, the, the, the Q1
7:32
Q Q2 rolls around beginning of Q2 rolls
7:35
around end of March.
7:36
And we want to say, Hey,
7:38
incremental attribution.
7:38
We're starting to see some
7:39
success in other accounts.
7:41
We're hearing about it.
7:41
Meta started to
7:42
suggest it a little bit more.
7:43
We want to actually test if this is going
7:45
to be successful or not.
7:46
So what we do is we launch, um, we take
7:49
our top performing campaign on a one day
7:51
click basis and we split out and have an
7:53
additional cell for incremental
7:55
attribution and throw them both into the
7:57
conversion of study.
7:58
So that's the test.
7:58
It's one day click versus incremental
8:01
attribution with a
8:02
holdout for each of them.
8:03
Um, so that's kind of test design.
8:05
Uh, we go through the first week of
8:08
performance and incremental attribution
8:10
itself looks terrible.
8:12
Well, I, I have a, I have
8:13
a tweet thread about this.
8:14
If you go to my Twitter, you can find it.
8:15
And it just looks miserable.
8:16
And I'm like, should we turn this off?
8:18
Like, I'm really, I'm
8:18
really like nervous.
8:19
This is not going to pan out.
8:21
Um, but no, we decided to commit to this
8:22
and set up the test.
8:23
So like, let's give it more time week
8:25
two, three roll around and the conversion
8:27
lift results inside of meta are reporting
8:30
much, much better now in favor of
8:32
incremental attribution.
8:33
So I'm starting to feel good.
8:34
Uh, I see, Hey, one day
8:35
click on it looks good.
8:36
Incremental attribution looks good.
8:38
The commercial lift
8:38
study results look good.
8:40
They use MTA data as well.
8:41
So they use, they use triple oil.
8:43
Um, uh, but your, your North
8:44
beam would be a comparable.
8:46
It's like that also looked good.
8:47
So everything was checking out that this
8:49
was looking really good.
8:50
And then, uh, we get to the end of the
8:51
test and the results basically say
8:53
incremental attribution is substantially
8:56
more incremental than one day click.
8:58
So there's a thrown around the word
8:59
incremental, like a million times.
9:01
Um, so if it's worth
9:02
digging into that further, I can.
9:04
But, um, basically, uh, incremental
9:06
attribution was performing way better
9:08
than one day click rise.
9:10
So we say, okay, we feel
9:12
pretty confident in these results.
9:13
Um, now it's only one test.
9:15
And so maybe it's worth repeating.
9:16
It's on one skew and a
9:17
subset of hundreds of skews.
9:19
Uh, so before we roll this, you know,
9:20
maybe before we roll it out aggressively
9:22
or just like blanket account wide, we
9:23
kind of want to go campaign by campaign.
9:25
So we start to roll it out
9:27
to several more campaigns.
9:29
And what has ensued over the last couple
9:30
of weeks is I think what's, what's really
9:32
the visuals behind me.
9:34
Um, and I can go into that in a second.
9:36
Andrew, do you have any, any questions or
9:38
like anything I can clarify before I dig
9:40
into like what we've
9:40
been seeing more recently?
9:42
I mean, I think the big thing that you
9:44
mentioned is basically like it was the
9:46
patience that sort of paid off for you.
9:49
Like, and that you're using
9:51
incrementality as like a
9:52
main measurement guide.
9:55
You know what I'm saying?
9:57
Like it's so much around
9:58
like, is this truly incremental?
10:00
And, and you waited, which is really hard
10:02
for a lot of people to
10:02
do both of those things.
10:04
Yeah.
10:05
Yes.
10:05
Yeah.
10:05
Very, very difficult.
10:06
Especially after that first week started
10:07
to look, look miserable.
10:09
And when I say look miserable, it's like
10:10
triple whale looked bad.
10:11
One day clicked, looked really bad.
10:13
Um, and it just seemed to take some time.
10:14
And that's the sentiment that I've heard
10:16
in the Foxhole group from people is like,
10:18
Hey, either, uh, generally speaking at
10:20
first, it's not going to look great, but
10:21
over time it starts to get better.
10:22
Uh, and then the sentiment I've been
10:24
hearing lately is that
10:26
it starts to fall off.
10:27
Uh, and that's where these,
10:28
these graphs start to come in.
10:29
So I'm going to try to zoom in a little
10:30
bit here and I'll kind
10:31
of show you the change.
10:32
So you can see kind of Q1 and this is the
10:34
one day click versus incremental.
10:36
The dotted line is incremental.
10:37
So they're like very close in line.
10:38
Like that's the thing to take away from
10:40
this visual is that they are tightly
10:43
correlated, not only correlated, but
10:45
they're actually pretty close together.
10:46
So one day click looks increment, looks
10:48
like incremental attribution incremental,
10:49
maybe parts a little bit higher, which I
10:51
think you'd expect
10:51
because one day click is
10:52
very conservative, but then we start to
10:55
have things trend off
10:57
aggressively over time.
10:58
So, uh, I'm highlighting on the graph,
11:00
March 25th, we start to see this
11:02
deviation from one day click and
11:05
incremental attribution.
11:06
They still follow each other from like a
11:08
correlation standpoint.
11:09
They're still pretty close, but the gap
11:11
between those two lines
11:12
starts to grow substantially.
11:14
Um, and there's a couple of things that
11:16
happened around this time.
11:17
And that's where we're like, uh, from,
11:19
from then to now is what we're trying to,
11:21
trying to dig into and understand.
11:22
So the first thing that happened, um,
11:24
which was at kind of the beginning of
11:25
March meta says, we are changing the way
11:27
click attribution works.
11:29
Um, we are going to say an outbound click
11:32
is click attribution, but engagement, uh,
11:34
the kind of engaged version
11:36
of, of that is going to change.
11:38
Even though a click to a profile, a click
11:39
to comment, whatever things that used to
11:41
historically you've been counted as, as
11:42
click conversions, that's
11:44
now moving to an engaged.
11:45
Um, so that changed, um, and then, uh, so
11:48
that changed and then started to roll out
11:49
to other accounts that they announced at
11:51
the beginning of March.
11:52
I think later, later in
11:53
March, it started to roll out.
11:54
Um, that was one of the big things.
11:55
Uh, we ended our test probably at the
11:57
beginning of April, uh, and we started to
11:59
roll out the test, uh, then as well.
12:02
And that's kind of, since, since those
12:04
things have happened, um, the kind of
12:05
ensuing results have been, there's this
12:07
deviation in one day click and
12:09
incremental attribution.
12:10
Uh, so that's an insight.
12:12
Uh, and our question now becomes, uh,
12:15
well, is there, is there a third data
12:16
point that we can look at or a couple
12:18
more data points that we can look at and
12:19
ask the questions like,
12:20
well, which one should we trust?
12:22
Should we go back to one day click and
12:24
use that as our guide?
12:26
Or should we lean into
12:27
incremental attribution?
12:29
Because just due to the nature of how one
12:31
day click and incremental work, kind of
12:33
like we said earlier, one day click is
12:34
only giving credit to people
12:35
who clicked within 24 hours.
12:37
By default, it is substantially more, uh,
12:39
conservative than
12:40
incremental attribution.
12:41
So we don't want to be
12:42
misled by one day click.
12:44
What, what should we do?
12:45
That third data point that I'm alluding
12:46
to is triple whale, um, uh, MTA tool.
12:50
Uh, historically triple has like largely
12:52
followed, uh, the one day click.
12:54
And so we've built up a lot of confidence
12:55
in one day click in the platform because,
12:57
um, we've spent three
12:58
years using one day click.
13:00
It's worked extremely
13:00
well for those reasons.
13:02
Um, and that is now also deviating very
13:04
substantially from
13:05
incremental attribution.
13:06
Um, not to continue this yap, but I'll
13:08
pause in a second, but, um, you would
13:10
also expect that because triple is using
13:12
click based it's, it's click based.
13:13
So like it's not completely unexpected.
13:16
Um, and at the same time, the deviation
13:18
leaves us a little bit concerned.
13:20
The final data point that we used, um, to
13:22
kind of look at this more recently is,
13:24
well, how did the actual new customer
13:25
sales for these products look over
13:28
particularly the last couple of weeks?
13:30
We've had Memorial Day recently, so that,
13:31
that certainly throws it off, but we've
13:33
definitely noticed a downward trend in
13:36
the overall efficiency.
13:37
So something that we like to do for
13:38
brands that have multiple products or
13:40
product categories is take, um, as much
13:43
as we can look at product level sales
13:45
with product level spend.
13:46
It's not perfect, but it allows you to
13:47
see trends over time.
13:49
Um, so if I were to take new customer
13:50
sales of this, of SKU a divided by spend
13:54
for SKU a, what is that?
13:55
What is that product level MER that has
13:57
gotten less efficient over
13:58
the last couple of weeks?
13:59
So what that's leading me to say is,
14:02
okay, well, where do we go
14:03
from here with this information?
14:05
We are actually going to start deviating
14:07
back to one day click in the short term
14:09
until we feel more confident in
14:10
incremental attribution.
14:11
Maybe we need to run a
14:12
new conversion of study.
14:13
Um, but all of that to basically say, I
14:15
don't really feel confident in
14:16
incremental attribution for this account
14:18
at this point in time, because it just
14:20
seems to be heavy, heavily deviating
14:22
from, um, the
14:23
performance that we did see.
14:25
Even though we started to validate it
14:28
with an actual conversion
14:30
lift test inside of meta.
14:32
So, um, that's, that's,
14:33
that's my personal anecdote.
14:35
One other anecdote on incremental
14:36
attribution, another account, same thing.
14:38
Like it started off really slow.
14:40
It got really strong for two weeks.
14:42
Last couple of weeks,
14:42
it's been miserable.
14:43
So we're just turning it back off, going
14:44
straight back to the
14:45
seven day click over there.
14:46
How much are they spending by the way?
14:47
They'll spend 150, 200 K a month.
14:49
And what's your hypothesis?
14:52
So this is what it is now, but what's
14:55
your hypothesis about like how this is
14:57
going to change in the next six months?
15:01
Yeah, I have this, this brand because of
15:04
the organic trend of this brand.
15:06
Um, that's why I've thought that one day
15:09
click is, and we've, we've kind of
15:11
experimented in one day click has been
15:12
the more reliable thing to use.
15:15
Seven day click for them.
15:16
It's a little bit too wild because if
15:18
they have an organic moment or something
15:19
pops off, meta does not react quickly
15:21
enough to bid back down.
15:23
They use cost controls as well, pretty
15:24
heavy on cost controls, cost cap and min
15:26
row as, um, meta doesn't throttle back
15:29
down quickly enough
15:30
using seven day click.
15:32
It still doesn't with one day click
15:33
either, but we can manually
15:34
make some adjustments there.
15:36
And so, um, one day click
15:37
for that reason feels better.
15:38
The reason we'd like to incremental
15:40
because it opened up, it was the theory
15:42
was it would open up maybe the delivery a
15:45
little bit more and be a little less
15:46
conservative on who we're bidding for.
15:48
Now, obviously we could change our bids.
15:49
We can increase the budget.
15:50
We could just force more
15:51
spend into one day click.
15:52
Um, but the idea, and I
15:53
think this is why Barry likes it.
15:54
Barry had a great explanation on Twitter.
15:56
It's like in theory, it should solve all
15:58
of the one day view weird
16:01
issues that, that you get.
16:02
Um, but I think the only hypothesis that
16:05
I can really think of for this brand
16:06
right now is that incremental attribution
16:09
is not, it's not doing a great job for
16:11
this account at teasing out what's
16:12
actually incremental.
16:13
And it's still conflating that with some
16:15
of the view based conversion, some of the
16:17
engaged conversions, uh, and
16:18
some of that are organic demand.
16:20
Uh, and so what it thinks is incremental
16:22
is actually purchases that, um, I guess
16:24
would have happened
16:25
anyways, which are not incremental.
16:27
What I think will happen, what I hope
16:28
will happen is, I mean, credit to meta.
16:31
Um, they, they allow you to do conversion
16:32
of studies and don't make you spend
16:34
absurd amounts of money.
16:35
And they want this product to solve the
16:36
problems that people are having.
16:38
Like they're, they're, they're the ones
16:39
that are saying, um, they're pushing
16:41
people to test these things.
16:42
And so I think as more conversion, uh,
16:45
studies are run, um, and as, uh, more
16:47
brands are doing this, I think
16:49
incremental, incremental attribution
16:50
should only get better.
16:52
So I think three months, six months from
16:54
now, we should be
16:55
retesting it, um, for this brand.
16:57
It's going to be a wait though.
16:58
It's that with IA, they just don't have
17:01
the data now, like, or as
17:03
much data as they could have.
17:06
It's like, it's still too limiting.
17:08
Or is it sort of one of those things that
17:11
we see on meta where it's good initially.
17:13
And then like, as more people adopted, it
17:15
goes down like, or, you know, like, or do
17:17
you see that happening in the future?
17:19
Um, or like, what is it, do you think
17:22
mechanically that was holding it from not
17:24
being not performing the same way?
17:26
Or is it too generous?
17:27
Is it the opposite?
17:27
I mean, I, I kind of, I'm reading what
17:30
you're saying, but
17:30
there's a little bit of both.
17:31
It's like some of it's limiting, but some
17:33
of it's also just like, seems like it's
17:34
kind of taken, putting
17:36
it out there too widely.
17:37
And that's what you thought was going to
17:38
happen, but it didn't.
17:40
Yeah.
17:40
It's a, it's a good question.
17:41
I, it would be interesting if somebody
17:43
from meta, um, unfortunately they just
17:45
had a bunch of layoffs.
17:46
So hopefully none, uh, no people in the
17:47
incremental attribution department,
17:49
because, uh, it would be very interesting
17:50
to hear from meta how.
17:53
It actually works.
17:54
So my understanding of how incremental
17:55
attribution works is that there's tons of
17:57
brands at any given time that are running
17:59
conversion of studies on, on, on.
18:01
Meta and they meta use those to inform
18:05
building this new optimization algorithm.
18:08
I don't, I don't know exactly how they
18:09
would describe it, but
18:10
conversion of studies were used.
18:12
Historical conversion of studies were
18:13
used to build this model.
18:15
And that's how it optimizes.
18:16
I think the tricky thing about that is
18:18
like, there's,
18:18
there's so much difference.
18:20
And this is true of one day click and
18:21
seven day click as well, but like,
18:22
there's so much difference and uniqueness
18:25
to each individual brand, depending on
18:27
how much organic traffic they have, how
18:29
much spend they're
18:30
getting from other channels.
18:31
Um, the industry, like there's, there's
18:33
so much nuance in between those that
18:35
maybe for this brand, the, the mean of
18:38
what they built conversion, uh,
18:40
incremental attribution on
18:41
top of just doesn't apply yet.
18:43
Um, because the other, here's the other
18:44
thing that I didn't, I didn't say like
18:46
we've continued particularly over the
18:47
last, before we made the
18:48
decision to shut off, shut this off.
18:50
We've continued to dial back the bids and
18:53
tell meta, Hey, we don't,
18:54
we're not happy with performance.
18:55
Like we're, we should dial this back and
18:57
it's continued to
18:58
basically spend regardless of that.
18:59
It's not, it's not adjusting the way that
19:02
one day click or seven day
19:03
click does to bid changes.
19:04
Um, so my, the only hypothesis that I
19:06
really have at this point is that they
19:07
built the model on kind of the, the
19:09
average of the conversion of studies.
19:11
And just, that's not working for this
19:12
brand at this point in time.
19:14
Um, I know a couple of brands that spend
19:16
the majority of their budget through
19:17
incremental attribution.
19:18
So it's very possible that it works.
19:20
You know, I'm curious if people are, uh,
19:22
spending on multiple,
19:24
uh, uh, attribution types.
19:27
Um, and the, this, this needs to be one
19:30
of the subsets of them.
19:31
Um, and if they're reaching a different
19:32
pocket of audience, but, uh, to the
19:35
different pocket of audience, like we're
19:36
not actually seeing
19:37
really different delivery.
19:38
There's a lot of overlap between one day
19:39
click and incremental attribution.
19:41
And the, um, so there's a
19:43
lot of overlap and reach.
19:44
And then the net new visit rates was
19:45
basically the same between them.
19:46
It's not like we were reaching a bunch of
19:48
new people by using
19:49
incremental attribution.
19:50
Um, it was about the same, the same rate.
19:52
And for this brand, that's like 70, 70%
19:55
pretty consistently.
19:56
Yeah.
19:56
It's interesting.
19:56
I mean, I think it's, it's, it's sort of
19:58
in this weird, interesting phase where
20:01
you hear a lot of
20:02
people talking about it.
20:03
There's a lot of hope in it.
20:04
You know, do you think if you made this
20:06
like one day click and you made it seven
20:07
day click versus incremental attribution,
20:10
what do you think would happen?
20:11
That's it?
20:11
Yeah, that's a good question.
20:13
All right, friends, quick break.
20:14
This episode of scalability school is
20:15
brought to you by North beam
20:17
incrementality, which
20:19
honestly is so awesome.
20:21
Let me just tell you
20:21
a little bit about it.
20:22
If you're a marketer, you probably agree.
20:24
Incrementality testing is broken.
20:26
It's insane.
20:27
You got to do all this crazy stuff, but
20:28
with North beam, you do
20:29
it right in the platform.
20:31
It builds tests using your own MTA data,
20:34
launches them directly in the ad platform
20:36
and automatically monitors them
20:38
throughout the test.
20:40
And if something threatens validity,
20:41
you're alerted to it within North beam
20:44
before it ruins the results.
20:46
The really nice thing is it solves all
20:48
these problems you hear about
20:50
incrementality testing all the time, like
20:52
labor, spreadsheets,
20:53
statistical reference.
20:54
Designing tests is
20:55
normally so slow, right?
20:56
And error prone.
20:57
So with North beam incrementality, it's
20:59
automated and accurate.
21:01
And your team is able
21:02
to spend with confidence.
21:04
Give it a shot with
21:05
North beam incrementality.
21:07
Really, the future of
21:09
incrementality is here.
21:10
And North beam has figured out a way to
21:11
make it easy for you.
21:13
Do you think if you made this like one
21:14
day click and you made it seven day click
21:16
versus incremental attribution, what do
21:18
you think would happen?
21:19
Yeah, that's a good question.
21:21
I mean, that's what I'm kind of like
21:24
struggling with, generally speaking,
21:26
right now is the difference between a one
21:29
day and a seven day click obviously
21:30
changes the way that meta delivers.
21:32
Because we use cost controls, generally
21:34
speaking, the question that I would go
21:36
back to is, okay, if they're not reach,
21:39
if there's, there's overlap between the
21:41
different attribution methods and who
21:42
you're going to reach, then there's like
21:45
not, not a ton of value in that.
21:48
Because if I wanted to use seven day
21:51
click because it was going to bid more
21:53
aggressively than a one day click, well,
21:55
the alternative thing I could do instead
21:57
of switching from one day click to seven
21:58
day click is just open the
21:59
bids or spend more forcefully.
22:00
Like I could just be more aggressive.
22:03
Um, now you can make the argument that,
22:05
okay, more data in a seven day click,
22:07
maybe makes Facebook optimized a bit
22:09
better, meta optimized a bit better and
22:11
allows them to be more efficient.
22:12
Um, I've just not really seen that to be
22:14
the case, especially when you have a
22:15
brand that's consistently
22:16
getting through learning.
22:18
And in this case, it's just been, it's
22:20
just been more effective to use one day
22:22
click because it doesn't go for those,
22:24
uh, especially with the way that they
22:26
changed click based attribution to be
22:29
actual link click, as opposed to on all
22:31
kinds of clicks that are, uh, whatever
22:34
you could qualify as a click.
22:35
Yeah.
22:35
I don't know if that answers your
22:36
question, but, um, I have my
22:38
hypotheses around all of it.
22:39
I mean, what's your, what do you think
22:40
about the overall change with like the
22:42
definition of clicks and
22:44
like link clicks and stuff?
22:45
Like, do you think it's been a net
22:47
positive or do you think
22:48
it's a net positive overall?
22:49
What's your thinking?
22:50
Cause it seems like it's, I mean, I think
22:52
it should be too, but it seems like it's
22:53
in the short term, like really screwed up
22:55
some results because people were, it was
22:57
optimizing on one thing and it's like
22:59
almost had to relearn, you know?
23:01
Um, I would assume that through like gem
23:05
and lattice and all this, like one of the
23:06
signals is this
23:07
definition and that's changed.
23:10
Right.
23:11
Um, that's, uh, that's screwed, screwed
23:14
things up to a degree.
23:16
I don't know.
23:17
What do you think?
23:18
Yeah, I think that's,
23:18
I think that's right.
23:19
Um, it's, uh, it's
23:21
definitely, uh, it's definitely.
23:23
Pausing my, I think more confusion in the
23:26
short term and especially for, for brands
23:28
that are on cost controls, right?
23:29
Like you, you went from bidding on a very
23:31
specific type of, of, of
23:32
click and that changed.
23:34
And if, if the amount of conversions that
23:36
were coming through changed by 20%, you
23:39
have a completely different bid that you
23:40
need to adjust for now.
23:42
Um, and so again, I think like short term
23:43
volatility, but in the longterm, I think
23:46
it's the right thing to do because one,
23:47
it adds like way more clarity for an
23:49
advertiser setting up a
23:50
campaign from scratch.
23:51
It's like what they're
23:52
actually giving meta credit for.
23:54
Uh, and then two, you know, I think
23:56
people will, I think generally people
23:57
will agree that if you had to go rank
24:00
order, what's actually incremental.
24:02
It's a click is probably the most
24:04
incremental, most commonly, uh, of an
24:06
engagement of some kind.
24:08
So a click to a social profile.
24:10
And so I should redefine that an outbound
24:11
click is probably the most incremental.
24:13
The next range of the, what they qualify
24:15
as engagement, you know, it's,
24:17
it's video views of some sort.
24:18
It's clicks to profiles,
24:19
it's likes, it's whatever.
24:21
It's probably the next most incremental.
24:22
And the view is the thing
24:23
that's least incremental.
24:24
Um, but you know, how's that changing
24:26
with short form video content?
24:28
And apparently, um, zoomers don't click
24:30
ads or, you know, that
24:31
that's the, that's the story.
24:32
So, um, yeah, I mean, in
24:34
theory, it all sounds great.
24:35
It's just a TBD, how it all
24:36
continues to, uh, to play out.
24:38
Oh, Zach's here.
24:39
Hey, what's up, dude?
24:41
Hi guys.
24:43
So let's talk about the creative cohort
24:45
spend analysis, uh, that you pulled,
24:48
like, what's the context of this, Brad?
24:50
Um, I'm, I'm interested to hear about it.
24:53
So creative hit rates, creative volume.
24:56
That's been like a topic of discussion.
24:57
Motion put out an awesome, I don't know
24:59
if exactly, I don't remember what they
25:00
called it, but it was like the creative.
25:02
I don't know.
25:02
They put out this 2026 creative PDF.
25:05
It was like 80 pages long.
25:06
And it was awesome because it showed
25:08
like, here's what the biggest brands are
25:09
doing from a creative volume
25:10
perspective and by industry.
25:12
And then also it said like, here, here
25:14
are the types of ads
25:15
that brands are running.
25:16
Cause they have this AI tagging.
25:17
That's awesome.
25:17
Um, and so we've been
25:19
thinking about that a lot.
25:20
We've been looking at a lot and kind of
25:21
using that as a benchmark for, okay, how
25:23
much are our brands launching
25:25
on creative on a monthly basis?
25:26
And, um, what does that mean?
25:28
But I think there's been, I think we
25:30
talked about this in a recent episode.
25:31
I feel like there's like this sentiment
25:32
building in the Twitter sphere for DTC
25:35
where it's like, people are kind of
25:36
getting frustrated with
25:36
the volume game a little bit.
25:38
And they're, they're leaning back towards
25:40
hit rates, making better ads, spending
25:42
more time, being more thoughtful.
25:43
And I think that's a fair exercise.
25:44
I'm still definitely in like the creative
25:46
volume is important camp
25:47
because you have to take swings.
25:48
Um, and at the same time you want those
25:50
swings to be calculated, but I don't
25:52
think anybody would say
25:53
it shouldn't be calculated.
25:54
Uh, but that's, that's a,
25:56
that's a separate topic.
25:57
In any case, the, the thing that we've
25:59
been thinking about and trying to map
26:01
back to is this idea of viewing our
26:04
creative through a cohort lens.
26:06
And I think I'm not sure if this is in,
26:08
um, there might be like a creative kind
26:10
of report in, uh, what
26:11
you and Kurt built, Andrew.
26:12
Um, but it might be more
26:14
of a CPMR related thing.
26:15
It's like the creative cohorts in their
26:16
CPMR, which is sick.
26:17
Um, it's a really cool view.
26:19
Um, but what we basically want to know,
26:21
the question we're trying to answer is
26:22
like, where did our
26:23
spend come from this month?
26:24
So in May, was it ads that we launched in
26:27
May or is it everything that we've had in
26:29
the ad accounts historically?
26:30
And then I can go back and I can look and
26:31
I could say, okay, if our entire ad
26:33
account is made up from spend,
26:36
ads that were launched last year, all the
26:38
new shit I'm making is going to waste.
26:39
Like I, we are not doing a good job at
26:41
making new ads that are resonating.
26:43
And then you can start to
26:44
dig into the reasons why.
26:45
So this is kind of a, this is a bit of a
26:47
surface view of like, okay, um, are we
26:50
making the right things?
26:51
Are we actually progressing?
26:52
And I have an example
26:53
of what this looks like.
26:53
So I can, I'm sorry for the non
26:55
YouTubers, but I'll flip my screen over
26:57
again and we can look at this.
26:58
Um, I realized that I didn't update the
27:00
previous document properly.
27:02
So I have a different looking one here.
27:03
Sorry, you blame, blame,
27:04
Claude, Claude for the visual.
27:06
The data is accurate though.
27:07
Um, so I'll try to describe what's here.
27:10
Can you guys, guys see this?
27:11
Okay.
27:12
Yeah.
27:13
So what you'll see is
27:14
we've got spend by month.
27:16
So January 300 grand, February 400, some
27:18
grand, March three, 300
27:19
grand, April 289, May 258.
27:21
So far we've got some
27:22
time left in the month.
27:23
Um, so this cohort view is what we've
27:26
been finding really valuable.
27:27
So if we look at January, we can see how
27:29
much spend in January was made up from
27:31
ads that were launched last year in this
27:33
case and ads launched in January.
27:35
And in this case, about
27:37
10% was from ads in January.
27:39
And I guess what you're looking for when
27:40
you're looking at this is that the month
27:42
that you're currently in and previous
27:44
months are making up like a reasonable
27:46
percentage because there's, there's two
27:47
things that would be an issue.
27:48
If January here, which is 90%
27:51
pre-January, like if we're looking at May
27:53
and it was all ads launched last year, be
27:55
like, fuck, what have we been doing?
27:57
What have we been doing
27:58
this entire year so far?
27:59
We haven't made any ads that are working.
28:01
Um, and then that's, that's the trigger
28:02
to go look into that.
28:03
It's, it's, uh, it's using this.
28:05
You shouldn't just look at this and not
28:06
do something with it, but you're trying
28:08
to find insights to actually go and do
28:11
something with this.
28:11
But the other way, the other thing to
28:13
consider is, well, if all of your spend
28:15
is from ads that you launched in this
28:17
month or in the previous month, I mean,
28:19
there's some caveats to this, but you
28:21
might also be concerned that like, okay,
28:22
you're really over, over, over indexed on
28:25
stuff you've launched recently.
28:26
Uh, and you might want to continue to get
28:28
new things in the pipeline because you
28:29
don't want to become overly reliant on
28:31
just a couple of, uh, outlier ads.
28:34
I mean, all our ads are good, but at the
28:35
same time, you don't want to become
28:36
overly reliant on this.
28:38
Uh, and so this is just a view that we've
28:39
been looking at to make sure, okay.
28:41
Um, are we, are we
28:43
launching new ads successfully?
28:44
Um, are there any pockets
28:45
that we can look into, right?
28:47
Like may, this is the biggest, this looks
28:49
like one of the biggest months, um,
28:51
where, uh, in the same month that those
28:54
ads are getting to spend almost 20% and
28:55
it's not even over yet.
28:56
So like, did we do something recently
28:58
that's hitting really well that we should
28:59
be leaning more into?
29:01
Um, so just kind of sparks that.
29:02
And if you wanted to run this, like
29:03
motions, not sponsoring this, but I can
29:05
give you the prompt to go.
29:06
You can just drop this in motion and, uh,
29:07
in their little runneth chat and it'll
29:09
produce this for you.
29:10
Um, HTML graph and everything.
29:12
Um, just kind of a cool way of, um,
29:14
looking at this and breaking it down.
29:17
So basically what you're saying is
29:19
instead of doing cohort or creative sort
29:21
of like by batches or
29:24
like, Hey, this was this batch.
29:26
You're doing it by a monthly cohort.
29:28
And that's how you're looking at them and
29:30
categorizing them in order to make sense
29:33
of the performance by each
29:35
of those cohorts by month.
29:37
Like it's, it's a more linear package.
29:39
Makes it easier to understand what we
29:42
did, what we shipped in
29:43
February is still good.
29:45
And what did we, and then you were able
29:47
to dig in and look at February and say,
29:50
these are the things that gripped that
29:52
are helped to scale.
29:54
How do we go back and
29:55
replicate some of the unlocks we had?
29:56
Is that, that's essentially what you're
29:57
getting at with this.
29:58
Yeah, exactly.
29:59
And there's, there's, this is, this is a
30:01
view in a certain way
30:02
that you can look at it.
30:03
Uh, if you're a motion user, you can go
30:04
to their new launches report and that'll
30:06
give you a weekly version of it.
30:07
It doesn't look exactly like this, but
30:09
it'll give you a weekly version where you
30:10
can expand and contract.
30:12
Uh, and what you want to do, I mean,
30:13
we've talked about, um, I feel like, I
30:15
feel like scalability school needs to
30:17
take a lot of credit for like spend
30:18
velocity as a, as something to lean into.
30:20
I know Zach, that was something that you
30:21
mentioned a ton last year, like spend
30:23
velocity and things picking up quickly.
30:25
But like, if you launched ads in the last
30:27
week that became the top spender, that's
30:29
the signal to lean into is like, okay,
30:30
um, did we hit on a
30:32
seasonal moment that's relevant?
30:33
Or is there something lasting in here
30:34
that we can go recreate, um, and lean
30:37
into and do more of.
30:38
And this is just another
30:39
way of, of looking at that.
30:40
So there's the weekly view of it.
30:42
Um, but it's, it's basically just like
30:43
forcing you to, to reconcile.
30:44
If you're, if you're a brand owner and
30:45
you are hiring create, like you, let's
30:47
say you have your brand owner and you
30:48
have a creative agency, uh, and you hired
30:50
them in March and you pull this report
30:53
and all of the spend is going through
30:54
shit you've created.
30:55
Like that's, you know, there's obviously
30:56
different ways of looking at it, but
30:57
like, that's the, that's the question to
30:59
go ask and say, it's like, Hey, is
31:01
anything you're making actually, um,
31:02
producing like net new net new spend in
31:05
the, in the, um, this one has seasonal,
31:08
um, seasonal swings.
31:09
And I feel like that's why this is
31:10
changing so much, especially recently.
31:11
I love it.
31:12
I don't have any questions on this.
31:14
Zach, do you have questions on this?
31:15
No, it makes a ton of sense.
31:16
Um, we do this kind of like end of month
31:20
report every month that, um, our like
31:23
creative ops manager and our VP of
31:25
performance creative put together.
31:26
It's similar to this and then just goes a
31:28
bit deeper where we see the difference
31:30
between what we call like breakthrough
31:31
ads versus scaled ads.
31:33
Um, which breakthrough is like, does it
31:35
hit $2,000 in spend within a seven day
31:37
period at the North beam
31:40
when they click target goal.
31:42
And so we kind of look at month over
31:43
month over month where we're saying like,
31:44
okay, how, how are we performing?
31:46
How many ads are actually breaking
31:48
through or scaling this
31:50
month versus previous months?
31:52
And it's, it's somewhat, it's almost like
31:54
a, maybe one step
31:55
deeper, maybe one step above.
31:56
I don't really know like the best way to
31:57
describe that, but yeah, I
31:59
mean, this is super important.
32:01
It's like, what do we miss?
32:02
And like, what is the longevity?
32:03
It'd be really interesting to
32:04
look at it, um, over time too.
32:07
So you're like, this is what it looks
32:08
like now, but then what does
32:09
it look like going forward?
32:11
I don't know.
32:12
I don't know if I'm like explaining that
32:13
right either, but the progressive like
32:15
swing, like does the, does each month end
32:17
up turning into 10% of a spend?
32:18
Or does it actually eat up
32:19
like, ideally, is it a quarter?
32:22
Is it 25% of spend for the first month
32:23
and then kind of degradates down to like
32:25
10 and then holds at 10?
32:27
That'd also be interesting too, to see.
32:29
Yeah, that's a great point.
32:30
I mean, the reason that like the, the
32:32
thing that inspired me to do this in the
32:33
first place is we had a brand who was
32:34
like, they were like generally bought
32:37
into the idea of
32:38
creative volume is important.
32:39
Um, but they wanted to like, see the
32:42
proof that it was working.
32:43
Um, and so they started, they, they
32:45
probably quadrupled their, their output,
32:48
um, through a bunch of like very
32:50
thoughtful adjustments on the way that
32:51
their internal team operated.
32:52
So they took inspiration from kind of
32:54
like ideas that we were sharing with them
32:56
and then they converted
32:57
it into actual output.
32:58
And it was just important for them to
32:59
make sure like, Hey, did what we, we, we
33:01
produced, uh, a hundred ads this month.
33:03
And you know, that's up from the 20 we
33:06
were previously locked into.
33:07
Did those do anything?
33:08
And so it allows us to look
33:09
back and say, yes, they worked.
33:11
But here's another way to split it.
33:13
It's like 50 of the ads that you gave us
33:15
for, were for a promo
33:16
that lasted three days.
33:18
And so like, that's, that's useful, but
33:20
like, and it makes sense because promo,
33:22
promo spends harder, uh, and over a
33:24
shorter period of time, but you want to
33:25
make sure that those evergreen ads that
33:27
are still live in the ad account continue
33:28
to actually produce.
33:30
Because then, you know, you've actually
33:30
hit something and it's not just like a
33:32
one hit wonder promo ad that
33:33
you can actually repurpose.
33:35
And so it was just like validation for
33:36
them that what they
33:37
were making was working.
33:39
And when you split it down to evergreen
33:40
and promo ads, it still ended up being
33:42
the case that evergreen consistently is
33:44
stacking up month over month.
33:45
And that's what compounds, right?
33:46
It's like, if you can, if you can make
33:48
more ads every single month that earns
33:50
spend in the ad account, then that's what
33:52
compounds over time.
33:54
And allows you to increase spend, uh,
33:56
over the, over the year.
33:57
Talk about, there's this guy, Zach Stuck,
33:59
who has this brand called Marsman.
34:00
Um, and it says, we're going to go into
34:04
now how Marsman scales on
34:06
next month's repeat revenue.
34:09
Uh, which is kind of a hot topic.
34:11
I feel like everybody wants to know.
34:14
I mean, headline, I mean,
34:15
Zach, I want to go into this.
34:17
Sure.
34:17
Sure.
34:18
I mean, Brad, do you want
34:19
to like tee this up anymore?
34:20
Do you, is that, is
34:21
that enough for me to.
34:22
Yeah.
34:22
I mean, the, the, the frame, the, I
34:25
remember we were having
34:25
a conversation one time.
34:26
And one of the things that you mentioned
34:28
is like, one of the things that we think
34:29
about for pushing spend aggressively.
34:32
Now you guys are, you guys have hit an
34:33
insane, um, growth rate
34:35
and, and, um, uh, MRR.
34:37
And so, and you're pushing
34:39
growth, growth aggressively.
34:40
And I look in brands that have solid
34:43
repeat rate all the time.
34:44
And maybe they have different
34:46
profitability goals.
34:46
They have different
34:47
growth goals, whatever.
34:48
Like often it's uncommon that the brand
34:51
is not being
34:51
aggressive enough in scaling.
34:52
Cause they haven't done the financial,
34:54
uh, homework that you guys have done to
34:55
understand, like, how
34:56
much can we spend into this?
34:57
Um, and one of the things that you
34:58
mentioned that you use is like a guide
35:00
for this is like, well, okay, if we know,
35:02
and I might not be framing this perfectly
35:04
exactly the way you phrased it, but, um,
35:06
you said, Hey, if we have this, an idea
35:07
that our EP customer revenue next month
35:09
is going to be, I'm just
35:09
going to pick a number.
35:10
It's going to be 500 grand.
35:11
I'm probably selling
35:12
that very short for Mars.
35:13
Like it's gonna be 500 grand.
35:15
It's like, well, we know we're
35:16
gonna have 500 grand in cash.
35:17
We can spend into that, like at least
35:19
that much next month.
35:21
Um, because we want to
35:22
be, want to be aggressive.
35:23
Um, so I'm sure there's a lot more
35:24
context and things that go into making
35:25
the decision, but it's, it's, it's, it's.
35:27
It's at least, uh, maybe safe.
35:29
Cause you have the cash to be able to
35:31
spend into, uh, into that.
35:32
So that's the framing.
35:34
Yeah.
35:34
So at the end of the day, like this is
35:36
where these MRR businesses or
35:38
subscription businesses scale so quickly.
35:40
It just comes down to this simple model,
35:42
which is if I can forecast for this
35:45
example, 500 K in reoccurring revenue.
35:47
And I know that my OPEX is going to be a
35:50
hundred thousand for the month.
35:52
Um, I know that I need to maybe run at
35:54
like a 1.8 or two, two, let's just call
35:57
it a two, uh, MER, uh, or two blended
36:00
rows, whatever term you want to use.
36:02
Um, to still maintain
36:04
profitability with a hundred K OPEX.
36:07
I then know I can spend
36:09
that full MRR into it.
36:11
And as long as I know if I'm trying to
36:13
achieve a two MER, I have to
36:14
be at a one new customer ROAS.
36:17
And so if I can stay at a one new
36:18
customer ROAS, I can still hit my target
36:19
and I can keep going.
36:21
The nice thing about this is like with
36:22
time and with more scale
36:24
that MER can actually go down.
36:27
Um, as long as your OPEX doesn't offset
36:29
it too much, meaning that like, let's
36:30
just like ratched up those numbers a ton.
36:32
So let's say, um, we're, our MRR is 5
36:35
million a month now, for example, and we
36:39
know what our OPEX is.
36:39
Maybe our OPEX is a million a month now.
36:42
I don't know, for example.
36:43
And so with that, because we have such
36:47
good margin and we know we're going to do
36:49
5 million in returning, we might be able
36:50
to say, Hey, we can run
36:51
a 1.5 MER now in total.
36:54
So I can still spend that full 5 million
36:56
in revenue and do only 2.5 million in
36:58
revenue to average
36:59
out to a 1.5 total MER.
37:01
So again, like this is where with more
37:04
scale plus the returning customer
37:06
revenue, when you can forecast, um, that
37:09
returning it like just, that's where
37:11
things really start to compound.
37:12
I think the thing that's been really
37:14
interesting for us is, is candidly like,
37:19
there's so many things that you do in
37:20
these subscription businesses that will
37:22
have like a, uh, fundamental changes on
37:25
like forecasting that returning customer
37:26
revenue that could have big impacts on
37:28
like how you think about the business.
37:29
So let's say you run a sale at the end of
37:31
a month, one month, and your cat goes
37:34
down and you scale into it.
37:35
The next month you could assume,
37:37
depending on how many percentage of those
37:39
customers are 30 day customers versus 90
37:40
day customers, you might see
37:41
a big spike at the end of it.
37:43
But let's say like you run an offer where
37:45
your 90 day subscription versus 30 day
37:47
subscription is a bigger discount.
37:49
So out of nowhere, that becomes a larger
37:50
percentage of your
37:51
customers at the end of that month.
37:53
You now are deferring all that revenue
37:54
for 90 days on the back of the month
37:57
where then you're like, Oh shit, I better
37:59
plan my spend
38:00
accordingly around those pockets.
38:02
Also, like if you're testing different
38:04
bundles, you're testing all sorts of
38:05
stuff, like this can all
38:06
have huge impacts on it.
38:07
So I think like as much as reoccurring
38:10
revenue is like a blessing, it can be a
38:11
curse too, because there's so many
38:13
different variables.
38:14
If you are really doing a bunch of
38:15
testing, like we have been with Mars,
38:17
where we're trying like different offers,
38:18
different price points, you know, 30 day,
38:20
90 day, you know, 180 day subscriptions
38:23
to kind of see like,
38:24
what is the sweet spot?
38:26
All of those changes will then have an
38:27
impact down 30, 60, 90, 180 days from now
38:30
that you really have to
38:31
like, be careful with.
38:33
And there's a lot of like assumptions
38:35
that go into the model
38:35
until you have like longevity.
38:37
So, I mean, the smartest thing you can
38:39
really do is like, just be like, cool,
38:41
we're going to sell a
38:41
30 day subscription.
38:42
That's all we're going to sell.
38:43
We're going to scale into that because
38:45
then there's no other
38:46
crazy nuance that happens.
38:48
The second you had a discount, additional
38:49
discount or whatever, like that could
38:51
actually impact your cohorts, which then
38:53
resets your, all of your learnings on
38:55
what's expected for
38:55
returning customer revenue.
38:56
But the simple math on it is when you
38:59
know what your MRR is going to be and you
39:00
have a good, good model built around it,
39:02
like you can usually
39:03
spend into that fully.
39:04
And most of these brands that are, that
39:06
are scaling the greens of the world, uh,
39:09
IM8s, whatever, like they're looking at
39:11
the returning customer revenue and
39:12
saying, Hey, if we're going to do 5
39:13
million this month in MRR, we're
39:14
definitely going to spend
39:15
at least 5 million in ads.
39:18
Yeah.
39:18
And the, in the issue, the point that you
39:19
made that's, you have to be careful with
39:21
is when you, you add the 90 day as you've
39:24
maybe, maybe you rip a bunch of 90 day.
39:26
And all of a sudden you went from a
39:27
hundred percent is 30 days, a hundred
39:29
percent of your subscriptions are 30 days
39:30
to 50% of them dramatically are 90 days.
39:33
Well, if you're, if you don't change your
39:35
cohort math and your four year cashflow
39:36
forecast to account for that, you're
39:38
going to spend into all of that, but
39:41
you're not, you're actually not going to
39:41
realize that cash for
39:42
another 60 days beyond that.
39:44
And that's where you put yourself into a
39:45
funky cash position.
39:46
So that's what you're saying about
39:47
keeping it simple to avoid.
39:49
The cash positions, plus then like
39:50
running different discounts throughout
39:52
different times of the month too.
39:54
So like maybe you run a discount, like
39:56
4th of July is in the front half of July.
39:58
Father's day is on like the back half of
40:00
the month or like mother's
40:01
days in the back of the month.
40:02
So like, depending on when you're running
40:03
those, those sales, it's also going to
40:05
have fluctuations on the returning, which
40:06
then ideally you're looking at, okay, I
40:09
did, you know, 500 K of MRR.
40:10
I can spend 500 K.
40:12
You have to really be cautious of when
40:13
you're spending into that 500 K around
40:15
all of those moments that
40:17
you've made historically.
40:18
Otherwise you can overspend in the front
40:19
half of the month and then you don't have
40:20
enough budget on the back half when CAC
40:22
is actually really good.
40:25
Can we talk about the forecasting and how
40:27
many times you're, how
40:28
often you're reforecasting?
40:29
Like, is it built on subscription base,
40:32
repeat purchase cohort curve?
40:34
Yeah.
40:35
Like, is it a simple percentage?
40:37
Like what is it?
40:37
Yeah.
40:38
So we just rebuilt our entire model this
40:39
week because we had some issues candidly
40:41
for May of us like missing, but all
40:44
learning opportunities for us.
40:46
But we, we just rebuilt it where it
40:49
actually goes down to the individual
40:51
customer level and then is now starting
40:53
to pull in all sorts of moments around
40:55
that individual customer.
40:56
So we basically look at who that customer
40:58
is, what they paid that first day, if
41:00
there was a discount or not.
41:02
We factor that into the retention curve
41:04
of that individual based on us knowing,
41:05
Hey, sometimes when they use a discount,
41:07
they're much more of like a, you know,
41:09
they're not as the longevity of that
41:11
customer is less because they're more of
41:12
a discount shopper and
41:13
they're more likely to cancel.
41:14
So we're like, we're applying retention
41:16
curves based on the decisions they made
41:18
and the sales price point or whatever
41:20
they did on the front end.
41:21
Plus then we're starting
41:22
to pull in other things.
41:23
Like we're pulling in, we're going to
41:25
start planning things from like gorgeous
41:27
or a CX team to say like, did this
41:29
customer reach out
41:30
through our portal to cancel?
41:32
But then they stayed on because we gave
41:33
them an additional discount.
41:34
What does that mean for the
41:35
longevity of that customer?
41:36
So now we're getting down to the per
41:38
customer basis where candidly, like prior
41:40
to that, we were looking
41:41
at it more holistically.
41:42
We were saying, okay, how many customers
41:43
bought on this day, 30 days ago, look at
41:45
that monthly core, look at the churn for
41:47
that cohort and just kind
41:48
of make a lot of assumptions.
41:49
But at the scale that we're getting to
41:51
now, like we had to look at it as
41:52
granular as we possibly could.
41:54
The thing that's really interesting now
41:56
is we're taking this forecast.
41:57
We're basically saying, okay, again,
42:00
simplified numbers, 5
42:01
million in MR this month.
42:02
We want to do 10 million in revenue.
42:04
We can run in a two blended ROAS, which
42:07
would mean that we'd have to run.
42:09
We'd have to, you know, run at a certain
42:11
degree where we would still maintain
42:13
profitability, officer
42:14
OPEX, all that stuff.
42:16
We're then saying, looking at our
42:17
forecast, we're then saying, okay, new
42:19
customer target is 5 million in revenue.
42:21
How do we go hit 5 million in revenue?
42:23
We're breaking down our targets by
42:26
channel and estimated orders
42:27
by channel every single day.
42:29
So now we're looking at like not just
42:32
orders or NCROAS or CPA
42:35
from Facebook or one day click.
42:36
Like we're literally looking at did
42:38
today, did email SMS drive the correct
42:41
amount of customers?
42:42
Did organic drive the right amount of
42:43
customers like by channel?
42:45
Because again, like at the scale that
42:47
we're getting to, if like one of those
42:49
channels misses, but everything else is
42:50
hitting, we could miss our target by, you
42:52
know, six figures,
42:53
potentially even seven figures a month.
42:54
So it's been really, it's been really
42:57
cool to actually like be able to start to
42:59
diagnose it that way and actually look at
43:00
it on a per day per channel basis.
43:03
Because I don't think many brands get to
43:04
the point where they're doing enough
43:06
volume where then it actually validates
43:07
like did email actually
43:09
drive 300 customers today?
43:10
Yes or no?
43:12
Because it's just like too granular, too
43:14
segmented based on size.
43:15
But now that the numbers have gotten big
43:17
enough that we can actually like keep an
43:18
eye on the trends,
43:19
like it's really helpful.
43:20
Also, then we can plan, hey, if we're
43:22
going to run a promotion, email can be
43:24
flat for the first half of the month.
43:25
But then during that sale period, we
43:26
expect it to actually pop up.
43:27
We also, you know, have the indication
43:29
that like, you know, new customer cac is
43:31
going to drop on these channels, but
43:32
maybe it doesn't move on podcasts because
43:34
that's linear throughout
43:35
the month, things like that.
43:36
So, yeah, I mean, forecasting is
43:40
important for all businesses, but
43:41
especially for subscription, like you
43:43
have to really dive into it.
43:45
Otherwise, like you will either not grow
43:47
as quickly as what you possibly can or
43:50
you'll actually inverse, which is like
43:52
you're not thinking enough of how many
43:54
net new subscribers you need to add and
43:55
what performance needs to be on every
43:57
single channel for you to actually add
43:59
net new subscribers.
43:59
Because for the most part, most of these
44:01
subscription brands are only growing
44:03
because they're adding net new because
44:04
likely most of them are churning the same
44:06
amount as they're adding in.
44:07
So everything above and beyond that is
44:09
like the real growth path.
44:11
And the podcast point is interesting
44:13
because it's like,
44:14
those are the things you do.
44:15
And like, I think a lot of people, it's
44:16
like maybe they put together a monthly
44:18
view, but the day to day can shift so
44:20
much to your point for a promo.
44:22
Even like week or day to day trends, like
44:24
maybe your weekends are more aggressive,
44:25
maybe they're slower, depending on the
44:27
category that you're in.
44:28
I have two follow questions on that.
44:30
When you get when you see that channel
44:33
level performance, like
44:34
maybe you can use two examples.
44:35
You could use a meta example
44:37
or the email example or both.
44:39
It's like if those are off, are you
44:41
reallocating in the platform?
44:44
Is it kind of like that's
44:44
where the deep dive starts?
44:45
Like, OK, meta is off.
44:47
OK, well, is there somewhere
44:47
in meta that we can reallocate?
44:49
And if it's like, nah, probably not.
44:50
This is all shit.
44:51
We got to go to somewhere else.
44:53
It's like, are you
44:53
making that daily decisions?
44:55
Are you zooming out over
44:56
a longer period of time?
44:57
And the mini follow to that is like, what
44:59
are you using as the, I
45:01
guess, like source of truth?
45:02
Are you using just like MTA
45:03
data or using platform data?
45:05
Just kind of normalized.
45:06
Yeah.
45:06
So for the CAC by channel, we're using
45:08
last click to dictate that.
45:11
And then we kind of back into a last
45:12
click and then an MTA to kind of like
45:14
figure out for our paid channels, like
45:15
what is the happy medium there?
45:18
So that's kind of like the blend.
45:19
At the end of the day, it's like, you
45:21
know, we've had, so for example, like
45:23
email and SMS has been falling short
45:25
based on our daily targets.
45:26
And we're sort of, we're trying to figure
45:27
out, OK, like what's happening on email
45:29
and SMS that's not allowing us to
45:31
continue to like perform at the levels
45:33
and the growth that we need to see month
45:34
over month from those channels.
45:36
And so for us, it was like, oh, well, we
45:39
run a subscription offer, like a discount
45:41
50% off your first purchase and like for
45:43
life on that subscription.
45:45
Um, but we don't really have that many
45:47
like unique offers to add into email and
45:49
SMS to get those extra pops or get the
45:51
people that weren't willing to purchase.
45:53
So there's, there's one like tactic that
45:55
we're about to start testing where
45:56
basically if a customer is sitting in our
45:59
ecosystem, in our own channel list, and
46:02
they haven't converted over the course of
46:03
like a 60 day period, we're actually
46:05
going to start discounting single
46:07
one-time purchase offerings.
46:08
So normally we're not a one-time purchase
46:10
brand. We're not trying to get any
46:11
one-time purchases, but we have, we have
46:13
such big lists now that like there's all
46:15
these people that just, just don't buy
46:16
simply for the fact that they don't want
46:18
to subscribe or they haven't gone on
46:19
Amazon yet to buy on one-time purchase.
46:21
So that's just like a way for us to be
46:23
like, Hey, we have this like dead list
46:24
essentially of just people that have come
46:26
into the funnel, but haven't bought for
46:28
some reason that I think a lot of brands
46:29
actually like kind of forget about that.
46:31
They're like, Hey, they didn't buy. We
46:33
did all these like whatever, uh, flows to
46:35
like get them to maybe get
46:36
a bigger discount over time.
46:37
But then there was probably one reason
46:39
why they didn't buy. So like talk to that
46:41
customer about that one reason why they
46:42
didn't buy and give them a strong
46:43
discount to still close.
46:44
Cause like it's very likely better off
46:47
and you still have contribution margin on
46:48
that customer to be gained.
46:50
If you just give them even a slightly
46:51
bigger discount to like bring in those
46:53
dollars versus them just sitting on an
46:54
email and SMS list and then just like
46:56
dying off because you, you pull them from
46:58
the list because they
46:59
don't, don't interact.
47:00
So yeah, there's like little things like
47:01
that, that we found by digging into, Hey,
47:04
email and SMS is off for that day.
47:06
What, what can we do to actually get
47:08
those extra pops throughout the month?
47:10
Um, and yeah, same
47:11
thing kind of with paid.
47:12
I mean, paid is more of
47:13
the typical analyzing.
47:14
It's like Apple ovens off this week.
47:16
Okay.
47:17
What are we, what are we doing?
47:17
Have we not, you know, have we tested a
47:19
quiz funnel to Apple oven or are we just
47:20
running listicles or whatever?
47:22
Right.
47:22
So it's like landing page creative, the
47:23
whole gambit of normal
47:25
funnel optimization stuff.
47:27
Have you, so the thing you just said
47:29
about like the retention, you guys
47:30
running the same playbook then for people
47:32
who have like lapsed as customers.
47:33
Like they, they tried it for two months
47:35
and they, they unsubscribed and now you
47:37
have to do this whole win
47:38
back to get them back on board.
47:39
Are you guys good?
47:40
Same thing.
47:40
One time purchase or push them to Amazon.
47:43
Yeah.
47:43
Yeah.
47:44
Same thing.
47:44
Same thing.
47:45
Yeah.
47:45
I think the biggest thing this week was
47:46
just like, Hey, we're not talking to
47:48
these people that just don't want to buy
47:49
it because it's just a subscription.
47:51
So it's like, let's just
47:51
go clean that up first.
47:52
Cause that's probably hundreds of
47:54
thousands of revenue,
47:55
like every single month.
47:55
If we just give them a proper discount to
47:57
come in one time purchase at like not
47:59
quite our subscription offering, but like
48:01
in, in between our one time purchase and
48:02
our subscription price.
48:04
So yeah.
48:05
And then next would
48:06
obviously be like lapsed customers.
48:07
The interesting thing about, about
48:09
supplements and lapsed customers is like
48:10
very, very low chance of their, we've
48:12
tried things before in the past, like
48:14
free gifts and shit.
48:15
It just like doesn't move the needle.
48:16
So, um, yeah, we've, we've, we've, we've
48:19
attempted it, but I would say like,
48:20
there's probably still
48:21
more, more room to fix there.
48:22
I mean, absolute banger.
48:24
I tried not to talk very much cause I
48:25
sound terrible, but Brad appreciate you
48:28
asking good questions.
48:30
Zach, thank you for sharing.
48:31
Is there anything else
48:32
you guys want to go into?
48:33
No.
48:34
Yeah.
48:34
Okay.
48:34
All set.
48:35
Poskey Foxwell signing off.
48:37
Thank you for listening and
48:39
we'll talk to you guys soon.
48:46
This episode is brought to you by the
48:48
Foxwell founders membership that Andrew
48:50
and his wife, Gracie run.
48:51
It has been absolutely pivotal for not
48:54
just the Homestead team, but
48:55
the easy street brands team.
48:57
We've had, I don't even know how many
48:58
members are currently in there that are a
49:00
part of our ecosystem.
49:01
But when it comes to anything from
49:02
learning ads to understanding what's
49:04
going on to building an agency to knowing
49:06
retention, it's been
49:08
absolutely useful for our team.
49:10
When they get stuck or they need help to
49:11
just go there and
49:12
resource all the other experts.
49:14
So definitely would recommend it for
49:15
anybody that's looking to, you know, take
49:17
it a step deeper, try to get a little bit
49:19
more knowledge on growth marketing and
49:21
all the world D2C is.
49:22
Yeah.
49:22
I think one of the most incredible things
49:24
about it is you can just like open up the
49:25
Slack group every single day.
49:27
You can pin your favorite channels for
49:28
the topics that you care most about.
49:30
And like every day there's going to be
49:31
somebody who just like, because they want
49:33
to contribute something valuable to the
49:35
group, you can go learn
49:36
something every single day.
49:37
And it's going to be extremely useful.
49:40
There's some ballers in there that you
49:41
just get like the
49:42
benefit of learning from that.
49:43
But like for the for the costs, like you
49:45
couldn't pay them that for their time.
49:47
But through the membership, like you get
49:48
access to some incredible
49:49
people and tons of resources.
49:51
Yeah.
49:51
I mean, I think the biggest resource to
49:53
me, too, is like the events that, you
49:55
know, Foxville Founders does.
49:56
They've been able to do some even in
49:57
Wisconsin, even in the boring state of
49:59
Wisconsin, which is pretty awesome.
50:01
Getting people together in person and
50:03
able to have really just like honest
50:05
conversations of what's going on, what's
50:06
working for them now, you know, where
50:08
they're at in their business.
50:09
Knowing that there's going to be, like
50:11
Brad said, some real killers in the space
50:12
in this in this membership that can that
50:15
can help and are willing
50:16
to take the time and help.
50:17
So that's been a huge part of why a lot
50:19
of our team have
50:20
really enjoyed it as well.
50:20
And the applications are now open if
50:22
you're looking to join.
50:23
So Foxville Founders.
50:25
Yeah.
50:26
Foxvillefounders.com.
50:27
Go check it out.
50:27
Go apply.
50:28
The only way that we grow this podcast is
50:34
by you sharing it with your friends.
50:36
Honestly, like reviews kind of don't
50:38
really mean anything too much anymore.
50:39
They're really meaningful, but they don't
50:41
do a lot for the growth of the podcast.
50:43
And so sharing YouTube links, sharing
50:46
Spotify links, sharing Apple, whatever we
50:49
call it under the podcast app now.
50:51
Anything you can share,
50:52
the better we're going to be.
50:54
Guys, anything else
50:55
you want to say on this?
50:56
Yeah.
50:57
Please go check us out on YouTube.
50:58
Rack up those views for us.
50:59
We'd love to see it.
51:00
And then subscribe.
51:01
Make sure to
51:02
subscribe on YouTube as well.
51:03
And I relentlessly refresh the YouTube
51:04
comments because it dictates
51:06
my mental health for the day.
51:06
So please say
51:07
something nice about all of us.
51:10
Thank you, everyone.
51:11
Thanks for listening.
51:11
Honestly.