Latest / Scalability School / Ecommerce A/B Tests That Won (and the Best Practices That Lost) | EP 40

Transcript

Ecommerce A/B Tests That Won (and the Best Practices That Lost) | EP 40 · Scalability School

0:00
And so I'm anonymizing
0:01
these numbers intentionally.
0:02
think of like high high ish AOV appliance
0:05
that goes in the
0:06
And something that you interact
0:07
with on a daily basis.
0:09
And we did a price test
0:10
where we the price was originally
0:12
MSP was 279
0:14
and the kind of
0:16
like not business as usual,
0:17
but when there was discounting
0:18
running discounted
0:19
down to 229 so $50 off.
0:22
So there's there's two things going on
0:23
It's discounted by $50.
0:24
And also the 229
0:26
We test that against running it at 199
0:28
which is $80 off.
0:29
So 279 down to 199.
0:32
the lower list price one like an insane
0:35
revenue per visitor was up like 44%.
0:37
the more than covered
0:38
the gross margin hit
0:39
that we took by doing the discount.
0:41
So after we ran that test,
0:42
that kind of became business as usual.
0:44
And spend
0:44
has continued to grow behind that.
0:53
And now
0:54
let's take a
0:54
listen to the Scalability School podcast.
0:58
Welcome to another episode
0:59
of Scalability School.
1:00
Just Brad and I today
1:02
talking about the season
1:04
we're in, split test season,
1:06
which is really going
1:07
to be a great episode.
1:09
Super stoked to be with you, Brad.
1:10
As always, how are you doing?
1:12
I'm great.
1:13
Missing missing Zach dearly,
1:15
but he's traveling the world,
1:16
I think, or something.
1:17
You know, he's he's busy, I don't.
1:19
Not sure what he could possibly be
1:20
busy with,
1:21
you know,
1:22
that small little company
1:23
he's running, but, you know, that's big.
1:25
Yeah, exactly.
1:26
Who knows?
1:27
Yeah. Who knows? We don't we don't even.
1:29
We don't even know.
1:30
We only talk to him on the podcast
1:32
so we're not hear from him otherwise.
1:36
Look, we're talking about
1:38
a bunch of tests that you
1:40
have run running today.
1:42
Tell me about
1:44
let's talk through some of these tests
1:45
because you did like a price test.
1:48
You did a quiz test.
1:50
I mean, there's a ton of stuff.
1:51
This outlining semi is absurd.
1:53
So let's go through
1:55
wherever you want to start.
1:56
Yeah.
1:57
So I tried to
1:58
I tried to order them somewhat by impact,
2:00
but they all there's,
2:01
there's some winners
2:02
and there's some losers.
2:03
And I think there's much to
2:04
learn from the losers
2:05
as the, as the winners.
2:06
But we can kick it off with the price,
2:09
the price test.
2:09
So I will for probably obvious reasons,
2:12
going to keep these anonymous.
2:13
But feel free to ask
2:15
any follow up questions.
2:16
I'll do do what I can to answer that.
2:17
So price test
2:19
which we actually I talked about this
2:20
when Brian was on the podcast
2:23
probably a couple of weeks ago
2:24
now when this one was coming out.
2:25
You should go listen to that.
2:26
If you haven't, it's fantastic.
2:27
He's a he's a beast.
2:29
But we ran a price test.
2:30
And so I'm anonymizing
2:31
these numbers intentionally.
2:32
But think of like high high ish
2:35
AOV appliance that goes in the kitchen.
2:39
And yeah,
2:40
something that you interact
2:40
with on a daily basis.
2:42
And so we did a price test where we
2:46
the price was originally the MSP was 279
2:50
and the kind of
2:52
like not business as usual,
2:53
but when there was discounting
2:55
running
2:56
discounted down to 229 so $50 off.
2:59
So there's there's two things
3:00
going on there.
3:01
It's discounted by $50.
3:02
And also the 229 number.
3:04
We test that against running it at 199
3:06
which is $80 off.
3:07
So 279 down to 199.
3:10
And the lower list price one
3:13
like an insane amount,
3:14
revenue per visitor was up like 44%.
3:17
And that more than covers
3:19
the more than covered
3:20
the gross margin hit
3:21
that we took by doing the discount.
3:23
So after we ran that test,
3:25
that kind of became the business
3:26
as usual.
3:27
And spend
3:28
has continued to grow behind that.
3:29
So that like during that,
3:30
during that period, revenue
3:32
visitor was up 44% on Intel Jim
3:34
split test.
3:35
And we've been able to scale
3:36
into that consistently.
3:39
What could cause that?
3:40
I mean,
3:40
I think just like getting the 279,
3:43
the 229.
3:44
279 less so,
3:45
but 212 just feels like,
3:46
man, you're almost at under 200 bucks.
3:49
Like, can you?
3:49
It just feels so random.
3:50
Like,
3:51
can you just get under the 200 bucks?
3:52
Like what?
3:52
259 or 249 have done better
3:54
because it's like close to 250.
3:56
I don't I'm not a pricing psychology guy,
3:58
but maybe I should be.
3:59
But something about just like
4:01
getting it down
4:01
under the next dollar amount
4:03
I think was pretty impactful.
4:04
And obviously the discount
4:05
went up 30 bucks.
4:06
So like that's meaningful to you.
4:07
But I think a lot of the lift
4:09
actually came from
4:09
just like getting it down to below
4:12
that, that next number.
4:13
So yeah 40% increase
4:15
So what does this tell you
4:16
about the way
4:17
people price stuff generally.
4:19
Do you think that there's just not enough
4:20
price testing
4:21
generally and also testing a lower price
4:24
plus percentage off
4:25
is is like a huge part of it.
4:27
Or what does this say to you?
4:29
Yeah, I'd call
4:30
back to the episode with Brian
4:31
because I think we spend a lot of time
4:32
talking about offers,
4:33
but I think people just
4:34
don't spend enough time
4:35
doing price tests.
4:37
And then also on top of that,
4:39
I think people
4:39
are scared to play with the MSP number
4:42
maybe more than they should be.
4:44
I think what most people do for pricing
4:46
is they go
4:47
look at what other people
4:48
are charging in the market,
4:49
and you can't be tone deaf to that.
4:50
But if you've built a product
4:53
that has like some unique attributes
4:54
about it,
4:55
or your marketing is
4:56
is unique enough
4:57
that you can position it to have,
4:59
you know,
4:59
perceived, unique,
5:00
unique attributes, then I think you can
5:02
you can play with pricing quite a bit,
5:04
particularly that MSRP number.
5:06
There are definitely
5:07
like legal considerations
5:08
for what you should be doing
5:09
with MSRP and dollars off
5:10
and how long
5:10
you can be running discounts,
5:11
but I think more people can get away
5:13
with, like
5:14
playing with the MSP number
5:15
to make the offer
5:17
more compelling as a result.
5:18
And I think that that is
5:20
that's not actually
5:20
what we did in this case,
5:21
but what we could do is go manipulate
5:24
that MSRP number even more.
5:26
And our,
5:27
you know,
5:27
gross margin ends up being the same.
5:29
One of the biggest questions
5:30
that comes up inside the foxhole founders
5:31
community is around hiring.
5:33
How do you know,
5:34
and how do you find great people
5:35
that you can actually trust
5:37
to do the job?
5:37
That's why
5:38
I want to talk to you about propel.
5:41
Propel has placed more than 75 employees
5:43
for members of Foxhole Founders,
5:44
some of the hardest roles
5:46
our members have been trying to fill
5:47
for months.
5:48
They filled quickly and efficiently.
5:50
We're talking from video editors
5:51
to graphic designers,
5:53
media buyers
5:53
and more in Latin
5:55
America, to managers, directors
5:57
and heads of department. And in the US.
5:59
They have done it all.
6:01
They can help you build
6:02
that team that scales faster.
6:03
I know that this is an ad read,
6:05
and I know that it's
6:05
totally one of those things
6:06
you're like, well, it's whatever it's.
6:07
And that
6:08
honestly though,
6:08
I literally use this and have used propel
6:11
and the team over at propel
6:13
for all of these things.
6:14
They filled offshore.
6:15
They feel domestic.
6:16
Like I just said,
6:17
it's absolutely incredible.
6:18
And honestly,
6:19
the founder, Leandro, used to run an econ
6:21
brand and performance marketing agency,
6:23
so he gets it.
6:24
That's one of the things
6:24
I've always loved about Leandro
6:26
and what I love about dealing with him.
6:27
I have never even heard
6:28
a negative review
6:29
about working with them.
6:30
One time
6:30
somebody said it wasn't
6:31
working out, and Leandro immediately
6:33
replaced the candidate
6:34
that he had put in the position.
6:35
For a person
6:36
that's like the only feedback
6:37
I've ever heard.
6:37
The way that they work is simple.
6:39
It's performance based.
6:40
You only pay one off fee
6:41
once you actually hire someone.
6:42
There's no retainers,
6:43
no ongoing
6:44
costs, and every hire is backed
6:45
by a three month guarantee.
6:46
So go to the show notes,
6:48
check it out, let me know what you think.
6:49
And if you have questions about this,
6:51
feel free to hit me up.
6:52
Andrew at Fox Digital.
6:53
Com I'm happy to talk about it.
6:55
If you go to the link in the show
6:56
notes, you get 10% off your first hire
6:58
with them.
6:59
It's a great deal
7:00
really special for podcast
7:01
listeners, so
7:01
I look forward to hopefully
7:03
helping your brand grow and scale.
7:05
Yeah, I think I think that's very true.
7:07
And a cool call out.
7:08
So so let's talk about the quiz idea.
7:10
So the quiz piece
7:13
go through the arc of all that. Yeah.
7:14
So we ran this one.
7:16
This is like the most
7:17
like developed test
7:19
to go through
7:19
because it had a bunch of different steps
7:21
throughout it.
7:22
And so we basically set up a product
7:25
quiz where
7:27
we only have one product on the website.
7:28
Right.
7:28
So it wasn't like we were actually
7:30
coaching people down,
7:32
like we were finding your product.
7:33
It's not a makeup finder.
7:35
It's like we're just
7:36
just it's like
7:36
a psychological play
7:37
of getting people to say yes or
7:41
like, self-identify
7:42
as what's important to them.
7:44
And so they are feeling
7:46
there's a psychology play happening.
7:48
So the way that we tested this at first,
7:50
and I've talked about this
7:51
on the previous landing page test, is
7:53
we build the quiz out
7:54
and then we just launched it into meta
7:55
and let meta dictate spend to that thing.
7:58
And it quickly picked up.
7:59
How did you build the quiz.
8:01
Like I mean there's a
8:02
bunch of different options.
8:02
Honestly, I hate every single quiz
8:04
builder of ever used for the most part.
8:06
So caught for this one.
8:08
Okay. Love it.
8:09
So you built this quiz and Claude.
8:10
Yes, which is a great, great takeaway.
8:12
It's like we knew the questions
8:14
that we wanted to ask
8:15
in kind of the general psychology,
8:16
but actually building it out.
8:17
We just like cloud build it
8:18
and then
8:18
just put it straight
8:19
into straight into Shopify.
8:21
Caveat
8:22
there is like,
8:22
if you want to store
8:23
the answers to the quiz,
8:24
that makes it a little bit trickier
8:25
and you got to do some extra work.
8:26
But that could be for another,
8:28
another episode.
8:30
So launch it
8:31
quickly picked up
8:31
a bunch of spend was comparable
8:33
to the other
8:33
landing page tests that we had,
8:34
but we wanted to continue
8:35
to get this sucker to perform better.
8:38
So round one test.
8:40
I've talked about this
8:41
a couple of times, but frustrating test,
8:43
but it's like I
8:44
pulled up the screen recordings
8:46
and I noticed something
8:48
about the screen recordings.
8:48
People would land on the quiz page
8:50
and I watched 1015 in a row.
8:52
Nobody took the quiz
8:53
and I was a single person
8:54
to click the quiz.
8:55
They click the menu to go shop.
8:57
They click the footer to try and go shop.
8:58
They were clicking everywhere
8:59
but start quiz and I was like,
9:01
okay, what's going on here?
9:02
So we added literally
9:03
a button below that that said
9:05
shop the product name
9:07
and that increased
9:09
revenue per visitor by 20%
9:11
because it just gave the people
9:12
who didn't want to shop a path to go
9:14
and do whatever they wanted.
9:16
Now, what's interesting about that
9:17
and I thought,
9:17
I, you know,
9:18
I've thought a lot about this
9:19
and I'm not sure what the takeaway is
9:21
entirely yet,
9:21
but like that,
9:22
that business that I'm talking about,
9:24
they run almost no traffic
9:26
through their product page
9:27
because it never ends up
9:28
performing better.
9:29
So even though people are clicking
9:31
shop now,
9:32
listicles perform better,
9:33
advertorial perform better, quizzes
9:35
perform better than the PDP.
9:37
Even though people are landing
9:38
on this quiz page
9:39
and they're they're clicking off
9:40
to go to go shop immediately.
9:42
So that was that was round one.
9:43
It's just literally adding a button
9:45
that made a lot of the people
9:46
who wanted to go shop, to go shop,
9:48
and then allowed people
9:49
who want to take the quiz
9:50
to to take the quiz.
9:52
Round two was
9:54
what something I'm trying to do more
9:57
of is when we think about tests is like,
9:59
what's going to be high impact?
10:00
Is testing ten questions into the quiz
10:03
going to be more impactful than the hook
10:05
or the the the image that people first
10:07
see when they land on the quiz page?
10:09
Well, everybody sees that page.
10:11
Not many people are getting
10:12
to the end of the quiz.
10:13
That doesn't mean
10:13
that's not worth optimizing
10:14
and thinking about,
10:15
but trying to just get more people
10:17
to actually engage and take
10:18
the quiz was something
10:19
that we were thinking about.
10:20
So we the original, the original
10:24
kind of image on the quiz
10:27
was kind of this
10:27
like 3D
10:28
graphic of the product that like, showed
10:31
its components is kind of
10:32
how I'll phrase that.
10:34
We changed that to a lifestyle photo.
10:36
So it's like a real photo, like,
10:37
I'm pretty sure I actually took the photo
10:40
in my, in my,
10:40
in my office,
10:41
and I was comparing
10:42
an alternative product to ours.
10:43
And we like had key call outs for
10:45
like what was uniquely different about it
10:47
and importantly,
10:48
the the product that we were comparing it
10:50
against was like a popular product.
10:52
Like a lot of people
10:53
switched from this product
10:55
when they consider
10:55
when they consider this brand.
10:57
And so it was like
10:58
a direct, direct comparison
10:59
to something that people recognize.
11:00
But it was also representative
11:02
of how other people tend
11:04
to build this product.
11:06
So I get very quickly showed
11:07
what was unique about
11:08
what was unique about our product.
11:10
That helps conversion rate on the quiz.
11:12
And ultimately revenue per session
11:14
go up about 14%.
11:15
So that was on top of the
11:17
the original shop button.
11:20
Wow.
11:20
So just just to change in the image
11:22
on the top of the page.
11:24
Yeah, literally.
11:25
And I think you could
11:26
and yeah, interestingly,
11:29
the other thing that stands out
11:30
is that the photo that we ended up
11:31
using was one of the top statics
11:33
that we had in the ad account.
11:34
It wasn't
11:35
the only thing driving to the quiz,
11:36
but that was
11:36
it was a top performing static.
11:37
And so the idea was people
11:39
already like and click on this, this ad,
11:41
maybe it could work
11:42
well on the quiz page.
11:44
Yeah, yeah.
11:44
And it's just
11:45
and it was
11:46
you showed the competitive product
11:48
people are switching from.
11:50
And I mean obviously that
11:52
that helps a ton okay.
11:53
So so we've got that we've got
11:55
we've got the the adding
11:57
a shop now button.
11:58
We've got swapping the image
12:00
to have a comparison image at the top.
12:03
What did we do for the third one here.
12:04
So the nice thing about the third
12:06
one is as you as you asked
12:08
how do we build it?
12:09
It was built in Claude.
12:10
And so it was very easy to kind of
12:13
have this hypothesis like I was, I was
12:15
I know everybody
12:16
kind of like
12:17
is already copying Mars Man's quiz
12:19
funnel and a bunch of other stuff.
12:20
My friends
12:21
and I just like spending some time
12:22
clicking through a bunch of these
12:23
different bunch
12:24
of these different brands.
12:26
Our first round of the quiz
12:27
had like four questions. That was it.
12:29
It was very short, straight to the point.
12:32
But a lot of these other ones,
12:33
like let you
12:33
sit in the problem
12:34
a little bit longer,
12:35
like it's a little bit more diagnostic
12:37
of like what the issue is or
12:39
like why is an issue. So
12:42
so we spend some time
12:43
researching like, okay,
12:44
well what what are
12:44
what alternatives are people considering.
12:46
What else are they thinking about?
12:47
Like what makes them
12:48
feel the problem a little bit more?
12:50
What helps them understand the solution
12:51
a little bit more,
12:52
and then what helps them
12:53
ultimately understand
12:54
why we are the best solution
12:55
for this product.
12:56
And so we put together those questions,
12:58
an easy version of this.
12:59
And I was kind of alluding
13:00
to a billion cards.
13:01
Like,
13:01
you could just go to Claude and say like,
13:03
hey, I've got this quiz,
13:04
can you help me sit in the problem and
13:07
in the problem for a few more questions,
13:09
like, just add a couple of questions.
13:10
Now, maybe you want to feed it
13:11
like customer data,
13:12
maybe have some post-purchase
13:13
survey data, maybe you have some reviews.
13:15
You have some other insights that you can
13:16
you can share.
13:17
But just like tell Claude
13:18
to add a few more questions to help it
13:20
sit in the problem longer.
13:21
And I think that would
13:22
probably get you pretty far.
13:23
And I don't remember. Oh, okay.
13:25
So conversion
13:26
and revenue per visitor for this,
13:27
that ended up doubling the performance.
13:29
So 100% increase in revenue by
13:32
by making the quiz
13:33
six extra questions okay.
13:35
So six extra questions.
13:37
And when you say having it
13:38
sit in the problem longer
13:40
I mean I have an idea what that is.
13:41
But like
13:41
what are you prompting to to
13:43
to to do that.
13:46
Yeah, I mean I think
13:47
literally you could say
13:48
sit in the problem longer and Claude
13:49
will be able to figure it out
13:50
because it'll it'll have the context
13:51
of like what
13:51
your quiz is already saying
13:53
and it'll do a fantastic job of
13:55
just like,
13:55
okay, your question, question two says,
13:59
are you tired? Okay.
14:00
You say, yes.
14:01
On a scale of 1 to 10, how tired are you?
14:03
It's it's it's eight out of ten okay.
14:06
And then it's going to ask two more
14:07
follow up questions to that.
14:09
That's going to allow you to be like,
14:10
man, I really am tired.
14:11
Yes. That you know what I do?
14:13
I do act out the next day
14:14
because it makes me makes me really bad.
14:16
And yeah,
14:16
I would, I would like to
14:18
I don't want to yell at my kids
14:19
because they're not sleeping.
14:20
I'm making this up.
14:21
But like,
14:21
it just it forces the person to like,
14:23
sure.
14:24
It's kind of that psychological play of
14:25
just like making people like, sit
14:27
and think about it a little bit more,
14:28
and it's like,
14:29
we know what they're going to answer.
14:30
We're just we're forcing them
14:31
to read through and reconcile
14:32
what that answer,
14:33
what it reminds me of, is like,
14:35
you can set up automation to send you
14:36
your your business
14:37
report numbers every single day.
14:38
But you know what else is?
14:40
You know
14:40
it's better than that
14:41
is like going into your dashboard
14:42
and looking at it and saying,
14:44
okay, yeah, yeah, that's a problem.
14:46
I got to fix that.
14:47
Or like having to type it out
14:48
to somebody
14:48
and saying, yep, we're down 20% today.
14:52
So yeah, just sitting in the longer
14:54
so sitting in it.
14:55
So you're giving it.
14:56
So just making sure I understand.
14:58
So you're giving this to Claude the quiz.
14:59
It's in saying look like
15:01
what are the things
15:01
we need to add to this
15:02
to get into psychology.
15:04
What's the prompt that you're using to
15:06
like give you better questions
15:08
that you added on literally
15:10
just in the like.
15:11
That's it.
15:12
SAT in
15:13
sit in the problem longer
15:14
and like I'll let a proposal like I won't
15:16
I won't just use
15:18
I won't just use what it has
15:20
but like give me,
15:21
give me 20 additional questions
15:23
I could add to help
15:24
sit in the problem longer.
15:25
And it'll come up with
15:27
a handful of diesel ones.
15:28
And then you just pick
15:28
the ones you like the most
15:29
and the ones that it came up with.
15:31
What were different about them?
15:34
They were just
15:34
sort of doubling down, as you said, like
15:36
they were just like
15:37
sort of making you reflect on it further
15:39
and then making it be like,
15:40
like lining you up
15:41
and readying you for the fact
15:42
that you probably do
15:43
need a product to help
15:44
because you're very in need of this,
15:47
because you have sleep issues or whatever
15:48
it is. Yeah.
15:49
So think of like, like
15:51
like let's say you have like three USPS,
15:53
you have three things
15:54
that people buy your product
15:55
for like three, you know, maybe,
15:56
maybe the first one
15:57
is like the most important thing.
15:58
And then it progressively
15:59
goes down from there.
16:02
The previous quiz was like
16:03
it asked one question
16:04
about each of those things,
16:05
which was like, basically like,
16:08
do you have this problem?
16:10
And then the longer version went from,
16:12
do you have this problem to like,
16:14
does this problem cause you this?
16:16
And what would you like to be
16:18
the outcome instead,
16:19
as opposed to just like,
16:21
do you have this problem
16:22
and then moving on?
16:23
So just like it just progressively
16:24
pushed that person down,
16:26
the understanding of
16:27
maybe it was like the forced reflection,
16:29
a little bit of like
16:30
realizing the
16:31
the problems
16:31
that they experienced from, from,
16:33
from not having the problem product
16:36
as the solution.
16:38
Nice. Okay. Interesting.
16:39
Okay.
16:39
So that doubled.
16:41
So that helped so people.
16:43
So and what percentage of people
16:44
are taking the quiz
16:45
of the people
16:46
that land on the page at this point.
16:47
Yeah that's a good question.
16:48
I actually have no idea.
16:49
I don't I don't know for this one
16:50
I have to go back and look,
16:51
but I'm not sure I've stop my head.
16:53
Okay. So 20% are taking it or something.
16:55
But but it doesn't.
16:57
The conversion helps
16:58
because they're looking
16:59
at these questions
16:59
even if they're not taking
17:00
the quiz like can you
17:01
is it a progressive quiz?
17:03
Or can you see the whole quiz
17:04
from the page?
17:05
Yeah, progressive.
17:06
You have to go. One question of time.
17:07
It tells you
17:07
there's a breadcrumb
17:08
for like how many questions like
17:09
it says what you are on out of ten,
17:11
which I think is important.
17:12
So people don't feel like
17:13
they're in a never ending quiz.
17:14
But there's a progress bar
17:15
and it tells you
17:15
how many and can they
17:16
can they look at the quiz
17:18
before they take it?
17:19
Like, can I look at question eight
17:20
even if I haven't done question seven?
17:23
No, you got to go one by one
17:24
at any point.
17:26
At any point
17:27
it shows up in a different spot
17:28
on the quiz.
17:28
I'm looking at it right now.
17:30
I would love to share it,
17:30
but I can't at any point.
17:32
You can still click off to go shop
17:33
the product though,
17:34
like there's
17:34
a little bit of a menu at the top
17:36
that says like,
17:36
go shop this instead, instead of like
17:38
continuing to take this.
17:39
Interesting. Okay.
17:40
So conversion
17:41
revenue per visitor doubled.
17:43
Love it.
17:43
So round four.
17:44
You did what? To make this even better.
17:47
So we have a question in the quiz
17:50
which is like what are you using now.
17:52
And so after somebody said I'm using X,
17:57
we presented a review
17:58
from our actual customer database
18:00
where somebody
18:01
switched from that product.
18:03
Now there's a couple of things
18:04
that I'm thinking about a real time
18:06
as we're as we're saying this,
18:07
but that's in work.
18:09
Like it gets to get to the end of it.
18:11
It's like you said, hey, you're using
18:13
you're using this product.
18:14
We said, okay, here's
18:15
somebody who who loved
18:17
who switched and used this instead.
18:20
And so yeah, it
18:21
it actually made it worse by about 8%.
18:24
So he turned that off.
18:24
So I don't know if that added
18:26
too many steps in the quiz.
18:27
Like did it add friction.
18:28
The other thing that I know
18:29
now that I'm thinking about is
18:30
was it too early?
18:31
Like we asked that question
18:32
very early on.
18:33
It's like, what are you using now?
18:35
And so is it too early to say, oh yeah.
18:38
Like you should totally switch
18:39
like this person. Love that.
18:40
You should switch.
18:40
It's like, well,
18:41
they don't even understand
18:41
what their problem is at this point.
18:43
So would it be better to maybe
18:44
move the brand towards the back end
18:46
and then present that review
18:48
once they're a little bit more convinced
18:49
that they need a solution?
18:50
Maybe, maybe that's a
18:52
that's a V6 of this test
18:53
that I need to go run.
18:54
But yeah, TLDR it went backwards.
18:58
Okay.
18:59
Well hey, so these are some good ideas.
19:01
So this works.
19:01
So what do you think
19:02
are other things that you'll you'll test
19:04
next as it relates to this.
19:06
Like I mean
19:06
what are the things that, you know,
19:08
I mean switching the image
19:09
change it significantly. Like.
19:11
Yeah I mean
19:13
are you building this thing for mobile
19:14
like is it does it look great?
19:15
It looks great on mobile
19:16
I assume like
19:18
it looks terrible on desktop.
19:18
It looks great on mobile though. Yeah.
19:21
Well that makes sense. Yeah.
19:22
So you're building it
19:24
for mobile specifically load time.
19:27
Let's see. What other things
19:29
are you going to test, do you think?
19:31
Yeah, I think we could make it
19:32
a little bit more visual throughout.
19:34
Like there's no
19:36
there's our steps
19:37
along the way that aren't questions
19:39
that like pause and like tell you
19:41
like here's here's
19:42
it gives you a response
19:43
based on what you said
19:43
so far that helps you.
19:45
Like we understand you.
19:46
We know what you're going through.
19:48
So like it's not asking
19:49
for you to ask a question.
19:50
But I think on those pages
19:51
we could probably do a little bit
19:52
more of making it visual to help,
19:55
maybe on the question page,
19:56
but also on those like in between pages,
19:58
just like add a little bit more of a
20:01
quick understanding,
20:02
like there's a lot of
20:03
I mean, it's not a lot of text,
20:04
it's a quiz with very short answers.
20:06
But, you know,
20:07
people get distracted very easily.
20:08
So like,
20:09
can we add some more visuals
20:10
throughout this that make it
20:12
make it work better?
20:13
The other thing that I, that I like
20:14
is just like making the quiz
20:16
maybe a little bit more
20:18
persona specific.
20:20
Like now
20:20
that we know what people care about,
20:22
can we make
20:23
the quiz entirely about one USP or do
20:26
we need to coach people through like here
20:27
are all the things that you care about.
20:29
So I said three earlier.
20:31
Like should the quiz just be about one?
20:32
Should it be about all three?
20:34
That's the nice thing about quizzes
20:35
is it's
20:35
like it's progressive
20:36
and personalized to the person
20:38
that's taking it.
20:39
So you kind of have the flexibility to
20:41
to just like jam all your USPS into it.
20:43
And it's still build your own,
20:45
build your own adventure
20:46
like goosebumps back in the day.
20:48
But yeah, I'd be curious
20:50
like what other people are doing.
20:51
So if you're if you're if you're a
20:53
if you're a quiz guru,
20:54
please tell me what you've done that
20:55
change your quiz
20:56
because of quiz funnels
20:57
have been super fun to dig into recently.
21:00
Yeah. Cool. Okay. That's great.
21:02
Okay, so now you have another
21:06
testing stuff that you ran through,
21:08
which is the trust bucket
21:11
is what you're calling this for.
21:13
Tests.
21:14
One lesson more trust signals
21:17
did not mean more conversion.
21:19
Tell us about friends, quick break.
21:21
This episode is brought to you
21:22
by North Beam,
21:23
the marketing attribution platform
21:25
that we love over here
21:26
at Scalability School.
21:27
If you're a marketer,
21:28
I mean, you probably agree, right?
21:29
Incremental testing is broken.
21:31
They were.
21:32
These tests
21:32
require tons of labor
21:34
spreadsheets to reference,
21:36
and the designing of the test
21:38
is slow and error prone.
21:39
How are you supposed to know
21:40
what to test in the first place?
21:42
Right.
21:42
There's a million factors
21:43
that can make incremental
21:44
testing difficult.
21:45
But the good news is North
21:47
Beam has completely changed the game,
21:48
and they fix this.
21:49
Only North Beam
21:51
incrementally automates
21:52
your incremental test
21:53
design and monitoring,
21:54
while protecting you
21:55
from the major mistakes
21:56
that plague incremental testing.
21:59
North beam builds tests
22:00
using your own MTA data,
22:02
launches them directly
22:03
in ad platforms,
22:04
and automatically
22:05
monitors them throughout.
22:07
If something threatens validity,
22:09
you're alerted before
22:10
it ruins the result.
22:11
With North Beam,
22:12
you can launch tests in platform
22:14
with a single click,
22:15
saving you tons of time
22:17
and reducing any chance for errors.
22:19
And when the
22:20
test concludes, your results
22:22
flow straight into your MTA,
22:24
meaning you can easily
22:25
see results
22:25
alongside the rest of your data.
22:27
Incrementally. Was broken.
22:29
North beam fixed
22:30
So trust is
22:32
maybe not like
22:32
the best way to phrase this.
22:33
But that's like kind of the summation
22:35
that I can think of with this.
22:36
But trust guarantees
22:38
like those kinds of things.
22:39
So they all kind of fall
22:40
found into this,
22:41
fell into the same bucket,
22:43
which was like a product page test.
22:45
So this is all on a specific PDP.
22:47
So test number one, we the product
22:51
that I'm thinking of has a warranty
22:54
that is
22:55
like industry
22:55
leading, like it is substantially better
22:57
than anything else that you can get.
22:59
It is like it's an expensive product.
23:00
It means it means a lot to the customer.
23:03
Or our perception was that
23:04
it means a lot to the customer.
23:05
So it shows up on the product page.
23:07
But it didn't show up on mobile
23:10
until after the price, after the ad cart.
23:13
And like
23:13
somebody may have already made a decision
23:15
that like,
23:15
I don't know how I'm feeling about this.
23:18
So it was a
23:19
it was a bit of a progressive test
23:20
where we had three sells.
23:21
So first was no changes.
23:25
Second actually have been four cells,
23:26
but I think only three were interesting.
23:28
So normal product page.
23:29
The second one was under the price.
23:32
It's said
23:33
free shipping
23:33
and we added plus x year warranty.
23:37
We added the warranty information
23:38
right there to make it super visible.
23:39
That was the one. And step two.
23:43
Step two.
23:43
We just like bolded it
23:45
like to make it
23:46
way more obvious
23:46
because it's kind of like faint,
23:47
like you couldn't
23:48
actually see it very well. Okay.
23:49
And so just like wasn't
23:50
super obvious that existed
23:51
since we just tried to
23:52
put it in people's face like, hey,
23:53
free shipping.
23:54
Plus we have an industry
23:55
leading warranty.
23:56
They used to be really pumped about
23:57
and it's on a product page.
23:59
And
24:02
adding that warranty information up
24:04
actually dropped
24:05
conversion rate
24:06
by 7.5% compared to the other variant.
24:10
I.
24:10
Honestly, I have zero hypothesis for why.
24:12
Like I don't get it.
24:14
It doesn't feel like
24:15
it was too early to say it.
24:16
So I you know,
24:17
people are on a product page.
24:18
They're trying to make
24:19
a purchase decision.
24:20
You would think they'd want
24:21
more affirmation of the fact.
24:23
But, you know, maybe at the same time
24:25
they're they're
24:26
they're saying like, hey,
24:26
this is an expensive product.
24:27
Like,
24:27
I just expect that you're going to deal
24:29
with my problems
24:30
if I have them, no matter what
24:31
the warranty is or isn't.
24:33
Or maybe the warranty wasn't explained
24:35
very clearly there.
24:36
Maybe we could add like an info
24:37
icon that kind of explained it all,
24:39
I don't know.
24:40
But yeah, I dropped
24:41
it, dropped it by 7.5%. Yeah.
24:44
I mean it's interesting
24:45
like like you know,
24:47
so I think warranty is
24:49
and I was talking to another member
24:50
about this recently
24:51
because they had been experimenting
24:54
with a warranty
24:54
with their product
24:55
and the messaging around that.
24:56
And basically my thought was,
24:58
you know, warranty
24:59
sometimes has been diluted
25:00
by a lot of brands in terms of
25:02
like what a warranty actually means.
25:03
And it means that you have to follow up.
25:05
It means that you have to, you know,
25:07
do this.
25:08
You know,
25:08
you have to send in the issue
25:09
to an address.
25:10
And, you know, sometimes it means more.
25:12
So it also can signal
25:13
maybe it's like
25:14
has a little bit of a negative signal.
25:16
So it might might be kind of interesting.
25:19
Like
25:21
this particular brand had
25:22
had done a lot with the word protect.
25:24
So basically protect
25:26
my purchase was what they, they did.
25:28
And it was like
25:30
it was
25:31
it was essentially a warranty,
25:32
but it was a protection plan.
25:35
And that's language that they added
25:37
because they saw that
25:39
Delta Airlines was using that
25:41
in trying to sell insurance
25:42
when they purchased a ticket.
25:43
And so protect kind of interesting word.
25:47
And so I think it's
25:48
maybe not the warranty.
25:49
The warranty makes sense.
25:50
But it's also just renaming it.
25:53
And like yeah,
25:53
calling it
25:54
the protect purchase might be something
25:55
to give a shot to do.
25:57
They do they charge for that.
25:59
They did not charge.
26:01
It's basically it says included.
26:03
So it makes it sound.
26:04
So it's like
26:05
yeah like Delta
26:06
of course charges
26:07
like Allianz charges for this.
26:09
But basically they know there's this like
26:10
oh you get free product protection.
26:12
Basically that's and you know
26:17
so you can
26:18
I mean there's a lot of things
26:18
you could play with in that was my point.
26:20
Like protections. Good.
26:22
That was kind of
26:22
a suggestion that I had made.
26:23
And then
26:25
we talked about other phrasing of
26:29
functionality.
26:30
Functionality guarantee
26:33
never stops working promise.
26:35
Like, you know, so
26:36
we were like playing
26:37
with this kind of stuff of like like a
26:40
no, no worry guarantee,
26:44
you know, zero
26:46
zero issue or like,
26:49
oh, what was the thing that we know?
26:50
Was that something like that?
26:51
Yeah. No questions asked.
26:52
Like it was
26:53
we basically were playing around
26:54
with these types of language
26:55
and protect was the one I want.
26:57
So that's crazy.
26:58
It's like it's insane
26:59
how much
26:59
like small wording
27:02
changes can make such
27:02
a meaningful difference,
27:03
like protection plan.
27:05
I'm going to go test that right now
27:07
because it's like,
27:08
yeah,
27:08
it's like I wonder
27:09
how many of those things are out,
27:10
like different pages or ads
27:12
you just like don't think about, but
27:13
like inverting it slightly
27:15
makes a huge difference.
27:16
That's cool, I like that.
27:18
Cool. Yeah. I'm glad. Yeah.
27:19
I mean, it's interesting little changes.
27:21
I mean, everybody's got
27:22
looking at these landing pages
27:24
that we're building so quickly. Right.
27:25
Like anyway.
27:26
So it's like the little words like
27:28
actually make quite a bit of difference.
27:30
So review counts.
27:31
Let's talk about
27:32
that was another test that you ran.
27:34
Yeah.
27:34
This is this this also these next
27:36
two copies me off
27:37
because it's just like it's
27:38
things that you feel like
27:39
this is a no brainer.
27:40
Like it just makes sense.
27:41
Like this should just win.
27:43
And so both both losers in this case.
27:45
And so
27:47
we noticed that the review count
27:49
on the home page
27:51
for a particular brand was like like
27:54
the review count was in the thousands,
27:57
but the review rating was like
28:00
pretty low.
28:01
Like it was it was close to four.
28:02
Like it wasn't close to five.
28:03
It was closer to four than it was five.
28:05
But they had a
28:06
they had, you know, tons of reviews.
28:08
And that was prior to like some product
28:12
and branding changes
28:13
where they were pulling in
28:14
everything like it was.
28:15
It was all of the products all time,
28:17
including products
28:18
they didn't even sell anymore.
28:18
But if you remove the products
28:20
that had issues
28:20
and they don't sell anymore,
28:22
and you go to the actual reviews
28:23
of the product,
28:23
it was like way closer to five,
28:26
like a,
28:26
like a 0.4 jump in the overall rating.
28:29
But they had less reviews
28:30
so went from mid mid
28:33
to a couple a couple thousand.
28:36
Like a meaningful meaningful drop.
28:37
Like probably like
28:38
more than a 50% drop
28:39
in the quantity of reviews.
28:41
But there's like
28:42
a substantially higher rating
28:43
and the quantity of reviews
28:46
at this lower star rating.
28:48
One well, I guess let me invert that.
28:52
The
28:53
the higher rating, less reviews had 16%
28:57
less revenue per session by doing that.
29:01
So yeah,
29:02
I'm sure there's been a lot of like
29:03
testing to to be done on this.
29:05
Like if you go to
29:05
if you go to Google
29:06
my business, you're not going to.
29:07
You're not going to
29:08
you're not going to call the guy who has
29:10
you need your toilet fix immediately.
29:11
You're not calling the guy who has one
29:13
five star review.
29:13
You're calling the person that has 4.4,
29:15
but they've got 7000 reviews.
29:18
Yes, but it just felt like a big
29:19
it felt like a big difference to me.
29:21
Like it was it was meaningful.
29:22
Like it was,
29:23
if I remember correctly,
29:24
it was like a 4.1 versus a 4.6.
29:27
Like, is it.
29:28
That's pretty big, right?
29:29
That's a pretty big jump.
29:31
So totally it's a huge jump.
29:33
But it's validated that it's you know,
29:34
I think people are skeptical
29:36
when it's perfect ratings. Right.
29:37
Like it just doesn't not real.
29:38
And so people are like
29:40
it's going to be better
29:41
if it's like a real number
29:42
of if it's like over a four, it's like,
29:44
yeah, you know,
29:45
like if you're
29:46
you were also used
29:46
to looking at five star ratings now
29:49
across so much of what we do. Right.
29:50
Like if you look at a restaurant
29:53
and it has a 401 rating, you're like,
29:55
I don't know, you know,
29:56
but like I'll give it a shot.
29:57
I mean, if it's kind of your choice,
29:58
but if you do enough in the landing page
30:00
to convince like that, you know,
30:02
that's pretty powerful. So,
30:05
you know,
30:06
yeah,
30:06
there's there's a lot of nuance
30:07
there, of course, but yeah, it's
30:10
I think I think it has
30:12
the number has a lot of
30:13
and making it as real as you can
30:15
is actually quite powerful.
30:17
Yeah.
30:17
So
30:19
that's you widget is an extra.
30:22
So it's very interesting
30:23
you say that
30:23
because the real nice is like
30:25
okay so I was looking at this next one.
30:27
And on the product page there is,
30:30
there's a section
30:31
that's the review widget.
30:33
And we had an AI summary version.
30:35
There was no
30:36
you couldn't see a single actual review.
30:37
It said here's the reviews you could see.
30:40
You could see the total review
30:42
rating overall,
30:42
the quantity of the reviews.
30:44
And you could see
30:44
the individual star rating.
30:45
So it's like how many five stars,
30:47
how many four stars, whatever.
30:48
And then like a 2 to $0.03 AI summary
30:51
like very very short
30:52
with like here's
30:52
what here's what people generally say.
30:54
And I looked at this and I'm like,
30:57
I just want to see like
30:58
I want to see pictures.
30:59
I want to see what people
30:59
are complaining about.
31:00
I want to see what people like, like
31:01
I want to see the reviews
31:02
and read them for myself
31:04
because I want to make a decision
31:05
based on that information. And so
31:09
we swapped in a full review
31:12
where, you know, like it was
31:13
it showed a summary of it,
31:14
it was reviews.
31:16
So it's like a
31:16
it's a summary of it
31:17
and you can expand it and see more.
31:19
And that was horrible.
31:23
It was a
31:24
it was a 10%
31:24
drop in profit per session
31:26
from going from
31:28
AI summary
31:29
only where you couldn't
31:30
see any actual reviews to.
31:34
Full review with no AI summary at all.
31:37
And so the question to ask myself
31:39
from here is like, well, why is that?
31:42
And maybe the AI summary is, you know,
31:43
people are comfortable.
31:44
I thought that there would be
31:46
an aversion to AI like I,
31:47
you know,
31:47
I think there is a general aversion,
31:49
but I think it's context dependent,
31:50
which is when you go to Amazon,
31:53
they have an AI
31:53
summary of reviews
31:54
at Amazon, normalize this
31:56
like AI review thing.
31:57
And so we have a we have a
31:58
we have a
32:00
step two of this going right now
32:01
where all of the variants
32:03
do actually include the AI summary.
32:05
Because it didn't it didn't win this
32:07
this test that lost.
32:08
We have the AI review summary,
32:11
just the AI review
32:12
summary, the Arab summary.
32:14
But the ability to expand
32:16
but shows no reviews by default.
32:17
But you can't expand.
32:18
So it's default collapsed
32:19
and default expanded
32:20
with the ability to see like a handful,
32:22
like six reviews and you can expand it.
32:24
Read more.
32:25
And the expanded version is losing,
32:28
so it's like the other two.
32:29
The class one has no difference.
32:31
Like like those are the same.
32:33
But yeah,
32:34
the expanded version hers performance.
32:36
So it's like two much information
32:38
maybe or something might be right.
32:39
It's like people just hurting
32:41
load speed I don't know it's crazy.
32:45
And what's the A?
32:46
Is it the
32:46
same product
32:47
we're talking about 279 or whatever.
32:49
Yes. This this is yes.
32:50
This is also an expensive product.
32:51
Yep. Yep. Yeah.
32:52
It sounds around
32:53
a little bit on the brands
32:54
but that one is expensive one.
32:55
Yeah it's interesting.
32:57
So do you think like
32:58
you want to see what it looks like
33:01
in somebody else's house.
33:02
Like you want to see, you know, like
33:03
you want to see what it looks like.
33:04
You want to see how people are using it.
33:06
You want to
33:06
you want to read
33:07
real people's experiences?
33:08
Apparently not really not. Yeah.
33:10
Apparently not. Yeah.
33:12
Yeah. Interesting I don't know.
33:13
I mean, it might be
33:15
maybe maybe it
33:17
like once people get into the reviews,
33:18
it causes more questions for them.
33:19
And so they're like held back
33:21
and they're like, I don't know.
33:22
You know,
33:23
I must think about it a little bit more
33:25
because this person said this
33:26
or this said that, you know,
33:27
so it's like
33:28
instead of
33:28
sort of optimizing it directly
33:30
for a purchase more, more quickly.
33:33
Yeah, that's a good point.
33:34
It's a good point. Yeah.
33:35
It causing
33:35
more questions is definitely
33:37
could be a piece
33:37
like if you
33:39
you have the ability to see
33:40
maybe some of the lower ratings.
33:42
And that causes enough hesitation
33:45
to to wonder.
33:46
Yeah that's a good point. Okay.
33:47
Interesting okay.
33:48
So we've got so we've got that.
33:52
And now
33:53
now we're under the headline test.
33:56
There's so many I've got we've got more.
33:59
We can keep going.
34:00
We could
34:00
we could always turn it into a part two
34:01
if we need to.
34:03
But yes. So the headline test.
34:05
So shout out to Nate Lego on Twitter.
34:09
Yeah.
34:09
He's been talking about
34:10
copywriting and headline tests.
34:12
And so this is like
34:14
somewhat inspired
34:15
by some of the things that he's been he's
34:17
been putting out
34:17
because I think
34:18
like go to a high traffic page
34:21
on your website, I might be,
34:24
not explaining what he does really well,
34:25
but like go to a high traffic site
34:28
on our page on your site
34:29
and look at the headline and say like,
34:32
is this actually descriptive enough?
34:34
Is it describing my most common customer?
34:36
Like, is it
34:36
speaking to that person
34:37
or is it just a label for what this is?
34:40
So in this case
34:42
we test it on a collection page.
34:44
The collection page is like
34:45
a pretty popular destination
34:46
for this brand.
34:46
A lot of traffic ends up
34:47
going through the home page,
34:49
and then when you click the shop,
34:49
it sends you to a collection page.
34:51
Not it's not just use,
34:53
but like less than less
34:55
than, less than ten SKUs.
34:56
So it's like pretty limited
34:58
and not a ton too to shop from.
35:00
And so we ran for headlines
35:03
for about three weeks,
35:04
and there was about
35:05
$2 million in revenue
35:06
collected through this test.
35:06
So it's like not not a small test.
35:08
It was pretty meaningful
35:09
in terms of overall the control
35:12
described the product like very standard
35:15
collection page, headline
35:17
t shirts, shorts like that.
35:19
That's you can think of it like that.
35:20
It was just explain what the product was,
35:22
what the category of product was.
35:23
That's all all it said
35:25
two of the variants that we tested.
35:27
So there's four total headlines.
35:29
Two of the variants
35:30
use persona adjectives.
35:32
So basically
35:33
what we did is
35:33
we went to the customer reviews.
35:36
We ran it through Claude
35:37
Nate actually kind of gave this tip.
35:38
You run it through Claude
35:39
and you say like,
35:40
how do people describe themselves
35:42
using their actual words?
35:43
Like, what do they say?
35:45
And how do people identify?
35:46
The other thing that we did
35:47
is there's outer signal,
35:50
I think has been pretty popular
35:51
recently, is we looked at like,
35:52
who are the customers?
35:54
Are they executives?
35:56
Are they, you know, are they, you know,
35:58
stay at home parents. Like what are they.
36:01
And so we used
36:02
we used insights from that
36:03
to write
36:03
a more persona descriptive
36:05
headline that spoke to that person.
36:08
And then the final variant was,
36:11
we took the
36:11
category of the product,
36:12
but we added a little bit of
36:13
like the US piece.
36:14
So like we made it a little bit
36:15
more emotionally driven.
36:18
And it added like,
36:19
here's why people buy this product to it.
36:21
So it went from two words,
36:22
which was like white shirts to something
36:25
a little bit more descriptive
36:26
about what made the product
36:27
actually unique,
36:28
and like
36:28
why people were interested
36:29
in the category,
36:30
which was like the persona.
36:32
Adjectives were kind of
36:33
versions were a little bit
36:34
more copywriting, saucy
36:36
like they were a little bit
36:37
more exciting,
36:37
had a little more flair to them,
36:39
like they weren't as clear,
36:39
but they had a little flair to them.
36:41
And this last one was just like,
36:43
here's what it is.
36:44
And like why you're going to love it.
36:45
Very, very straightforward.
36:47
And that plain USP headline
36:49
crushed everything.
36:51
That last one buried for
36:53
by like 3 to 5% more revenue
36:54
per visitor fluctuated a little bit.
36:56
And that was on $2 million in revenue.
36:58
So like 3 to 5% doesn't
36:59
sound that exciting, but is over
37:01
a ton of revenue.
37:02
Like makes a huge difference.
37:03
And yeah,
37:05
that's the over a three week period.
37:07
So yeah,
37:08
nice little little headline test.
37:09
You can just go like doom scroll
37:11
Nate's Twitter.
37:11
He puts out a ton of like
37:13
examples of this.
37:14
They have a great podcast
37:15
that I'm forgetting the name on
37:16
right now.
37:16
Tactical and practical
37:17
and brain driven brands with former
37:21
former guest Sarah Levin.
37:24
Great content.
37:24
You can just
37:25
you can just steal that right from him
37:27
and tell him I sent you. So.
37:28
Yeah. Banger.
37:30
So what is this?
37:31
So, I mean,
37:31
what does this tell you
37:32
in terms of a headline test?
37:33
This is this is using their words to
37:37
to to sell it
37:38
essentially more effectively.
37:40
Yeah I think you could.
37:41
And that's what Nate sees
37:42
a lot of success with in this case.
37:44
If I,
37:45
if I look back on those two personas
37:46
specific headlines,
37:49
I wish I could be more more more clear
37:51
about what they,
37:51
what they specifically said.
37:53
But dates got a bunch of great examples
37:55
I think they were doing.
37:57
They were being cute and clever,
37:59
but not clear,
38:00
like they weren't as clear
38:01
as the one that ended up winning.
38:03
Like it was just a matter of fact.
38:04
Like this is the
38:05
this is the product and the specific USP
38:07
for why you're going to buy
38:08
it as opposed to like,
38:09
here's who you are
38:10
and why it matters to you.
38:11
You know,
38:12
like it was just like it
38:13
played it up a little bit too much.
38:14
And so I think it was clear over
38:16
clever was the takeaway here. Love it.
38:18
So the checkout block
38:20
assuming everybody loves a good checkout
38:22
block right.
38:23
Like it's
38:23
you know
38:24
there is not customized
38:25
enough in my opinion.
38:26
And this is something that Mr.
38:28
Stock has done
38:29
fantastically well over many years.
38:31
And we've learned a lot from him.
38:33
What did you guys do?
38:34
I and so I went into this, I was like,
38:37
man, Shopify Plus no brainer.
38:40
Like, let's go, let's go
38:42
add and customize the checkout.
38:44
And we just used
38:46
intelligence has like
38:47
a nice little checkout
38:47
checkout block tests that you can run.
38:50
And we tried it was a three sell three
38:52
cell test.
38:53
The first was just normal checkout.
38:55
The second was adding a USP block
38:57
which again just like takes the core.
38:59
A couple of reasons that people buy
39:00
and just like puts it in the checkout.
39:03
And then the the third sell was
39:06
the USP block.
39:07
Plus like some of the great reviews,
39:09
like just reviews that were sick
39:10
like really,
39:11
really good short to the point
39:12
people love the product.
39:13
Added it
39:14
and I was like,
39:15
there's no way this doesn't win.
39:16
Like, this is this is free.
39:18
And it didn't make a difference at all.
39:20
No.
39:21
Literally made no difference in any case.
39:23
Like none of them really made
39:24
a huge difference across the board.
39:26
Like I think at best
39:29
it was maybe like a 2% swing,
39:31
but like there was no, I didn't
39:32
I didn't feel
39:33
like very confident in that.
39:34
But the good news is like
39:35
it didn't go backwards,
39:36
like it didn't make anything worse.
39:37
So I just liked the fact that we had
39:40
ups in the checkout.
39:41
I just liked the fact that we had
39:42
reviews in the checkout,
39:43
so we ended up rolling it out
39:44
anyways because there was
39:44
no there's no downside to doing so, but
39:47
I just made an assumption
39:47
that I was like,
39:48
we're going to
39:48
we're going to see a least
39:50
a 2 to 3 to 5% impact by doing this,
39:52
because people are going
39:53
to get to check out,
39:54
they're going to know, like,
39:55
hey, we've got,
39:57
you know, a fantastic guarantee
39:59
you're going to love this product.
40:00
Here's a bunch of people
40:01
who are obsessed with it.
40:03
And it just yeah,
40:04
all those trust signals
40:05
made no, no difference.
40:07
And I'm sure people see different things,
40:08
but I don't I don't want to do it.
40:10
That information.
40:11
It's interesting.
40:12
Yeah. It's interesting.
40:14
So it didn't really do anything.
40:15
It didn't didn't move it.
40:16
I mean, I think, you know, there's
40:18
there's some of these,
40:19
some of these
40:19
where a checkout block
40:20
might actually be really effective,
40:21
where, you know,
40:22
you're pushing people through
40:24
on a quick sub product potentially.
40:27
Right.
40:27
And it's
40:28
and it's like if they're at the
40:29
they're at the box
40:30
and they're like, whoa.
40:31
And then you know, they're like,
40:32
what should I buy this?
40:33
And then it's
40:33
like if there's some,
40:35
you know, USP
40:36
kind of pieces there, it's like,
40:37
oh, okay.
40:37
You know what? Like yeah, I'm good.
40:39
And then they buy it and like
40:40
so it might be also product dependent
40:42
because it was this
40:43
the expensive product.
40:44
Yeah. Yeah.
40:45
Because because it's like at that point
40:47
they've already done the research.
40:48
They're like I don't it doesn't.
40:49
I don't care.
40:50
You know what I mean.
40:51
Like it
40:51
doesn't really mean anything to me.
40:52
That would be my guess.
40:53
That's a good point. That's a good point.
40:55
It's like if I'm going to buy a $280
40:58
product or 275 bucks,
41:00
you know, it's like I'm looking at that.
41:01
I'm like, okay, like
41:02
I don't it's
41:03
not going to make
41:03
a huge difference to me.
41:04
Whether you've been through Reddit,
41:05
you've been through YouTube. Like,
41:07
you know,
41:08
it's like, all right, that's fine.
41:09
You know,
41:09
like I don't need to it's invalidated
41:12
actually making it faster
41:13
for me to buy it.
41:14
It's actually
41:15
what I'm looking for, probably.
41:16
Right. Yeah.
41:17
That's interesting, I heard this.
41:19
I don't remember where
41:20
I heard this, but, like,
41:22
it was this,
41:23
like, story about people, like,
41:24
having to say like, oh, yeah, I promise.
41:27
Well, it's
41:27
like if you are just a person
41:29
who you stick to your word,
41:30
you don't have to say, I promise
41:31
until you just need
41:32
just like the
41:33
the emphasis of saying like,
41:34
oh, you can trust me
41:35
or I promise or whatever.
41:36
It's just like it's the unsaid
41:37
thing that doesn't need to be said.
41:39
It's like,
41:39
are we forcing
41:39
too much into this checkout?
41:40
We're like, we're like,
41:42
customers know the game.
41:43
Like they know what you're trying to do.
41:44
Like they're trying to butter
41:45
me up at checkout,
41:46
make me feel comfortable
41:47
with my purchase.
41:49
Yeah.
41:49
No, that's a
41:51
I think
41:51
the research
41:51
thing is a really good point.
41:52
I hadn't thought about that.
41:55
Well, let's.
41:55
Okay, so now we have more destinations.
42:00
Test I love this.
42:01
So what do you what does this mean?
42:03
This one's super easy to do,
42:05
especially if you're in apparel.
42:07
So apparel category
42:09
you tend to have like
42:09
a ton of different pages
42:10
that you can send people to.
42:11
You can send people to a home page.
42:13
If you don't have like a ton of SKUs,
42:14
you can send people to a collection page.
42:16
You can send people to a product page.
42:17
You probably have different
42:18
versions of collection pages
42:19
depending on me.
42:20
SKUs you have
42:20
which like present them
42:21
in different ways,
42:22
like fall
42:22
versus somewhere
42:23
versus whatever,
42:24
where colors
42:25
and different orders of them.
42:26
I think Metab has talked about
42:28
like even the way
42:29
like a B testing
42:30
the way you sort collections
42:31
and the impact of that.
42:32
But in general,
42:33
like you kind of default just like
42:35
have these pages.
42:37
And so for a brand, we noticed that
42:40
comfort was actually running all of
42:43
their traffic
42:44
on meta through collections pages.
42:46
And for this brand,
42:47
we were doing it all
42:48
through product pages
42:49
and like
42:49
we had tested
42:50
collection pages in the past
42:51
and didn't
42:52
really seem to make a difference.
42:54
But comfort recently was doing this,
42:55
and so I was like,
42:56
oh, okay, well, that's
42:57
that's interesting and worth
42:59
giving it a shot again.
43:00
And so,
43:01
but before we did that,
43:04
I'm getting I'm
43:04
getting a little bit ahead of myself.
43:05
Before we did that,
43:06
we all traffic
43:07
was going to the product page.
43:09
We first launched
43:10
a list of all based on the top performing
43:12
ad that helped
43:14
increase overall spend by 20%
43:15
and efficiency held.
43:17
So like huge win there.
43:18
And then we added another list
43:21
with some of the other top USPS
43:22
that bumped up spend again
43:23
a little bit as well.
43:24
And then we launched two more versions
43:26
of the collection page, all the same ads.
43:28
So like one ad,
43:30
one top performing
43:30
ad, going to five different destinations
43:32
within a CEO and letting
43:34
like individual ad sets per.
43:35
So we could just control the bids
43:36
if we wanted to
43:37
and just like figure it out one at one,
43:41
there's there was more.
43:43
But like the
43:44
most important takeaway,
43:45
I think, is that,
43:47
the top performing ad was the same across
43:49
all of them.
43:50
It still went to all those
43:51
different destinations.
43:52
And did you copy
43:53
and then put the new one in,
43:54
or would you just swap it
43:57
duplicate
43:57
duplicate the duplicate destination.
43:59
Yeah okay. Right. Okay.
44:00
So you duplicated it,
44:02
put it in the new destination
44:03
and by by allowing and then like
44:08
and it wasn't
44:08
like any bidding nuances
44:10
or anything like that.
44:11
They're all there's all the same
44:12
because it's. Yeah.
44:14
So you put it in and it
44:15
and it goes through and says like
44:18
we love the variety.
44:20
Yeah. Basically. Yeah.
44:21
It's like you're meeting, you're allowing
44:24
meta okay.
44:25
We talked
44:26
we talked about this a trillion times.
44:27
So I'm falling on deaf
44:28
ears a little bit here. But
44:31
like Andromeda is
44:32
trying to do personalization at scale.
44:34
Right.
44:34
Like they are
44:34
just trying to match you to okay.
44:37
You you in particular okay.
44:39
Here's a sick new bike.
44:40
You're going to love this.
44:40
You're gonna love this bike.
44:41
Andrew, you need this bike, okay?
44:43
You need that.
44:44
I know you should get anybody.
44:46
Yeah, yeah.
44:48
It's like they know what you're doing.
44:49
They're trying to personalize that scale.
44:51
Like,
44:51
I just think this extends
44:52
to landing pages
44:53
and at different points
44:54
in the shopping journey.
44:55
So I think this is like common knowledge.
44:58
But like a lot of brands scale
44:59
through quiz funnels.
45:00
But then some of their lower funnel
45:01
traffic goes through the product page.
45:03
And in this case we're just saying
45:04
here's five options. Meta.
45:06
Like you determine
45:06
how to spend between them all.
45:08
And the net impact of this one was like
45:11
the weekly spend basically doubled
45:13
and efficiency went up over this
45:14
over this period. So
45:16
just like
45:17
a huge net win,
45:18
I'd say what I will say
45:19
is like the first couple of days
45:20
looked really bad.
45:21
Like the collection page was horrible,
45:22
like it was not good.
45:23
And I don't know if that was better.
45:24
Trying to figure out
45:25
who's going to prefer the collection
45:26
page over the product page
45:27
because it's the same ad.
45:28
But yeah, once we once
45:29
we gave it enough time
45:30
like it, it's sat in
45:31
and it looked really good.
45:32
And it wasn't just like a week over week
45:34
thing, like year over year, this, this,
45:35
it was a specific product category
45:37
that specific product
45:38
categories up a ton,
45:39
like way more than it should be
45:41
for at this point during the year.
45:42
And like that was really the fun.
45:43
That was the only thing
45:44
that really changed.
45:45
I mean, beautiful like great.
45:47
Okay, cool.
45:48
I mean, it's
45:49
I mean, it's a good idea, right?
45:51
It's like it's using
45:52
it's using the thinking
45:53
that we know,
45:53
which is to give and give give options.
45:57
And it's giving you a structure
45:58
to do that. So it makes sense.
46:00
Somebody else in the membership
46:02
had talked
46:03
about a very similar thing,
46:04
where they had
46:05
a bunch of different types of landers
46:07
that they built,
46:08
and they actually had,
46:08
I think it was like 13 of them
46:10
running or something.
46:11
And, and had them in a CBO, same thing.
46:15
And just let it rip on the,
46:17
on a variety of top ads.
46:18
They weren't using the same one.
46:20
And it had been something
46:22
that was able to continue
46:23
to hold scale was their point.
46:25
So very similar concepts.
46:26
That's cool to hear.
46:28
Okay.
46:30
I mean, we're getting whoever's
46:31
listening to
46:32
this is fucking getting insane.
46:33
Sucks because this is good shit.
46:36
And also I,
46:39
I am slow on my bike right now.
46:41
I haven't been on the bike and like,
46:42
I've been on the bike
46:43
like four times in a month,
46:45
which is very unusual for me.
46:47
And I went up yesterday.
46:49
It was a billion degrees
46:50
in Santa Barbara was like
46:51
it was literally a hundred degrees.
46:52
It's never is here.
46:54
And I like went up the mountain
46:56
and like started up with the buddy
46:57
and like got to the midpoint.
46:59
I'm like, all right I cooled off.
47:00
I'm like, I think I'm gonna keep going.
47:02
Got up like almost got to the top.
47:03
And I was like, I'm gonna barf,
47:05
I gotta turn.
47:06
I was like, I'm going.
47:07
I look at my GPS and I'm going.
47:09
Three miles an hour.
47:10
I'm like, this is dumb.
47:11
We got it, Nick.
47:12
Normally I'll try to aim
47:13
for like five miles an hour, which is,
47:14
you know, because you're
47:15
but you're going uphill.
47:16
It's like a 3800ft climb
47:18
over seven miles, but like I,
47:20
it's a great climb. It's very difficult.
47:22
But with the heat, I was like,
47:24
I can't like, I can't do this.
47:25
And so I am slow right now.
47:27
So you're about to,
47:28
you're about to get a bunch of ads
47:29
for crazy and gummies
47:30
and electrolytes and peptides.
47:32
So I hope you're ready.
47:34
Oh yeah, I know
47:35
because I set it into the ether now.
47:36
Yeah.
47:37
Meta's listening of course. Always.
47:39
Absolutely.
47:40
Okay. Love it.
47:41
So the post-purchase swap,
47:42
what have we got for that one. Yeah.
47:44
Here's the
47:45
like the big takeaway from
47:45
this is like just go audit
47:46
your post purchase upsell
47:48
because this is like the
47:49
the actual
47:49
like tactical change
47:50
here was not like it's
47:52
not that meaningful.
47:53
I'll explain it anyways.
47:54
But I'm sure
47:56
half the people listening to this haven't
47:57
thought about their post-purchase upsell.
47:58
Although after sell
48:00
not a sponsor
48:00
but certainly could be
48:01
has been ripping on Twitter lately
48:03
given given a bunch of free game.
48:04
So maybe maybe that's sparking people
48:06
to go and audit
48:07
their their post-purchase flow.
48:08
But if you haven't looked at it,
48:10
go set something up
48:11
and there's a ton of different options
48:12
you can do for this.
48:13
You can go upselling
48:14
more of the same product
48:15
you can do trying to cross-sell products.
48:18
You can try to cross-sell
48:18
multiple products.
48:20
You could
48:20
try to upsell people into a subscription
48:22
if they didn't subscribe.
48:23
Like there's just so many options
48:24
and things that you could do.
48:25
I really I haven't done this,
48:27
but I really want to set up
48:28
like a post-purchase flow.
48:29
You've got like a nice little video
48:30
that's like, hey, thanks for buying.
48:32
You know, it's going to make this
48:33
great is if you buy this
48:34
or like it starts to like
48:35
get them ready
48:35
for their purchase in a way
48:36
and maybe tries
48:37
to sell them on something,
48:38
but actually, just like
48:39
it's got a sick video post-purchase,
48:41
maybe that helps with retention,
48:42
take rate, whatever.
48:44
In any case, here's the test.
48:47
Then this brand had like
48:48
a small subscription
48:49
service was basically like,
48:51
hey, this is like,
48:52
here's a complimentary subscription to
48:54
to your product.
48:54
Like you'll just get this
48:55
and it's going to help you
48:56
like take care of your product,
48:57
but it costs money.
48:58
So you have to subscribe on this.
48:59
But we're going to give you a great deal.
49:00
Like you pay quarterly
49:02
and you get a sick bundle
49:03
that's going to help you
49:03
take care of your your product.
49:05
Huge value.
49:06
And it worked like it worked fine.
49:07
And like,
49:08
you know, it's
49:08
nice to have
49:09
a little subscription revenue coming in.
49:11
But they launched
49:12
a complimentary product
49:14
and I just like put a Navy test in
49:16
for scientists subscription
49:18
and upselling to this way more.
49:20
That's like
49:20
way more complimentary product
49:23
that just goes with it.
49:23
And the
49:25
the revenue per visitor went up
49:27
like five x.
49:28
I went from $2 to $10.
49:30
And that's holding.
49:31
And like,
49:32
you know, realistically
49:33
it doesn't make a huge difference
49:35
like take rate and conversion take
49:36
take great actually
49:37
went down a little bit,
49:37
but AOV just went up so much
49:39
because the product pricing
49:41
was much better.
49:42
So it didn't matter.
49:43
And like the subscription
49:44
revenue is not meaningful.
49:45
Like it's not going to compound it.
49:46
Beat that over time.
49:47
You know,
49:47
maybe it increases from that $2
49:49
per person to $4 per person
49:50
over the course of the year
49:51
or even two years.
49:53
The take rate on
49:54
going to like a net
49:55
new product is a huge one.
49:57
That's hard
49:58
to make that useful for people
50:00
because like,
50:00
you have to come out
50:01
with a completely new product
50:02
in order to do that.
50:03
But yeah,
50:03
that's how I'm saying, just like,
50:04
go look at your post-purchase upsell
50:06
and just like try something different.
50:07
Even the wording.
50:08
We did this test one time
50:10
where it was like a
50:11
it was like a testing kit, like,
50:14
you know, like think of
50:15
what are the popular ones right now
50:16
like superhuman or whatever,
50:18
like a testing kit like that.
50:19
It wasn't superhuman,
50:20
but it was like that.
50:21
We added messaging in the upsell,
50:23
post-purchase upsell that was like, hey,
50:25
did you know that
50:26
97% of people hit their goals
50:28
when they have an accountability partner?
50:30
Like just added that messaging
50:32
and had an upsell to an additional unit
50:33
because, like,
50:34
you're not going to use
50:34
another testing kit,
50:36
especially not at the same time.
50:38
And that crush like that
50:39
messaging was just like helped people
50:42
think like, oh yeah, like
50:43
I'll buy this
50:43
and then I'll Venmo request my or like,
50:45
I'll get my spouse
50:46
to do this or whatever.
50:47
And that messaging
50:49
made a huge difference.
50:50
So audit your post-purchase upsell okay.
50:53
Beautiful okay.
50:54
So let's talk about the argument PDP
50:57
as it relates to a premium watchband.
50:59
Yes. So
51:02
pretty expensive.
51:03
Several thousand dollar watch product.
51:06
And we basically just took the
51:09
the product page.
51:10
And like the
51:11
the general premise
51:12
for product page tests that I like to do.
51:15
And shout out
51:15
to Konica, who's the crow goat
51:19
she like,
51:20
I probably mixing up her words here,
51:23
but the basic premise was just like,
51:25
add more to the page.
51:27
Like just add depth to it.
51:29
Now, intentional
51:30
about what you're adding.
51:31
But like, people will scroll
51:33
if they want to consume more content
51:34
like that
51:34
is kind of
51:35
one of the one of the bigger takeaways
51:36
is like just add content to this.
51:39
And so we added in, in in the old page,
51:43
it was like
51:44
the flow of the page was by box.
51:46
Hey, it includes these things for free.
51:48
Here's like your normal accordions,
51:50
which is like here's
51:50
the warranty information,
51:51
blah blah blah blah blah.
51:53
Here's the full image gallery.
51:56
Here's like
51:56
the specs of what it's made out of.
51:57
Here's how it compares.
51:59
Like in nighttime versus daytime,
52:01
here's how it looks.
52:02
And then all of the
52:03
different product reviews.
52:04
But the reviews were specific
52:06
to only that product,
52:08
not their entire collection,
52:09
which matters because like they sell,
52:11
they sell a premium watch.
52:11
They don't sell
52:12
a ton of individual units,
52:13
but they sell a lot across
52:14
their entire collection.
52:15
The new page added sitewide proof.
52:18
So that's kind of why
52:19
just what I just explained.
52:20
It added a little bit more of like
52:22
it moved the
52:23
the specs up
52:23
a little bit more
52:24
because these are people
52:25
who are buying watches
52:26
because they care about how it's built,
52:27
like they want to
52:29
like the
52:30
they're like the
52:30
person that over explains that.
52:31
She was like, did I never ask?
52:32
But like,
52:33
they want to be able to tell you that
52:34
it's Swiss made and blah,
52:35
blah, blah, blah blah,
52:36
because they're a watch collector
52:37
and they care about that.
52:39
It included what you get in the box.
52:41
It moved,
52:42
move the demo up a little bit,
52:43
and then it had like a comparison chart.
52:45
So like,
52:45
well, look, I got ahead of myself.
52:47
It had a why no two
52:49
watches are alike on it.
52:50
It said why it's different.
52:51
It added a value comparison.
52:53
Like you can buy ours for Christmas,
52:54
or you can buy this super expensive
52:56
Swiss alternative
52:57
for tens of thousands of dollars.
52:58
And here's how they compare.
53:00
It showed how
53:01
how people would wear it,
53:02
where it was built, which it was,
53:03
it was made.
53:04
It was not like made overseas.
53:06
And that was
53:06
that was valuable to this buyer.
53:08
It made the the
53:08
the risk reversal slash
53:10
like warranty and guarantee more obvious.
53:11
We added more depth of the FAQs.
53:13
We had a more reviews.
53:14
And then one other thing
53:15
that I didn't list here
53:16
is like in the box section,
53:18
you could previously like
53:19
switch to a different
53:21
there's not technically a variant
53:23
because it's like a different
53:24
it's a different skew of watch.
53:26
But we just like made
53:27
those much more visual.
53:28
Like they went from
53:29
like kind of hard
53:30
to see icons
53:31
like bigger circles
53:32
in like right at the top of the by box.
53:34
People are clicking the shit out of that
53:35
because they can just like,
53:36
actually see, like, oh,
53:37
I think I prefer that color. And so,
53:40
depth of page like went from like,
53:43
yeah, just we just like made
53:44
the page substantially longer
53:45
and then changed a couple
53:45
of those things throughout
53:46
and just like added more information
53:48
that we thought would help
53:48
answer your questions
53:49
to your point earlier about like
53:51
people are going to do their research
53:52
before they're by, well, great.
53:53
What if it's all on the product page?
53:55
Yeah.
53:56
I mean, if you're buying
53:56
like a high dollar thing, right.
53:58
The more that there's there
53:59
that there is there the better, right.
54:01
That's like
54:01
I think you're talking about it
54:03
going through it.
54:04
You're deeply getting into it.
54:05
Like the more info you have
54:07
and the more that you can consume
54:09
and be bought into the story
54:12
watches so much of it as a story
54:14
because people are going to ask.
54:15
Oh, where'd you get that?
54:16
You know, like the more
54:17
that you can tell that narrative is huge.
54:19
And, yeah,
54:20
I heard
54:21
I heard one from another watch brand.
54:24
I worked
54:24
with a watch brand for a while, and
54:27
they had to people that had bought their
54:31
watches, their watches were
54:33
average about 250 to, you know, ish.
54:38
There they sent something
54:40
like 17 emails over 45 days
54:44
talking about the story and then also,
54:46
like constantly invited them
54:47
to this Facebook group
54:48
and this Instagram group.
54:50
You could choose which one
54:51
you wanted to be a part of
54:52
or like
54:52
a Instagram group, what it was,
54:55
and it was like a owners
54:56
like cohort thing that they had built,
54:59
like anyway. And it was crazy.
55:01
Like, people want to talk about it.
55:02
It's the point.
55:03
People want to be into it.
55:04
People want to be a part of it.
55:05
And the more that you can pump them
55:07
up, the better outfits going to be.
55:08
So it makes sense
55:08
that it's on the PDP that way to
55:11
yeah that's cool.
55:12
Yeah, I like the Facebook group thing
55:13
because you're right.
55:14
It's like
55:14
even if you haven't purchased yet,
55:16
it's like you want to go see other people
55:18
that are pumped about it.
55:18
And like Facebook groups are like
55:21
you can kind of say,
55:21
I mean, I know there's moderators,
55:23
like you can kind of
55:23
say whatever you want.
55:24
So it's like you at least
55:25
get the real authentic version of
55:26
like what people think about it.
55:28
It's like it's like the, the,
55:30
the thing that Zach said
55:31
probably a year ago now, which was like,
55:33
add Facebook
55:34
comment screenshots to your landing page
55:36
or product page.
55:37
It's because like, it's
55:37
hard to fake that.
55:39
Yeah, totally.
55:40
Well that's cool. That's interesting.
55:42
So that makes sense.
55:42
So these are
55:43
I mean tons of good tests here.
55:45
So now we have bonus preorder versus
55:48
sold out.
55:49
Same out of stock product two framings.
55:52
The hypothesis is the sold out
55:54
signals demand should convert better.
55:56
What happened.
55:57
Yeah.
55:57
So you you don't have the product.
56:00
You can either say preorder or sold out.
56:02
You can probably do
56:03
some combination of the two.
56:04
But to me, to me
56:05
sold out
56:05
says like
56:06
we have this product at one point
56:07
and we had so much demand
56:10
that we couldn't keep up on it.
56:11
You could still buy it, like it
56:12
said, sold out, but you can buy it.
56:13
You can be part of the next,
56:15
the next run or next batch.
56:17
Or you can say, hey, this is a preorder.
56:20
And I thought that the
56:23
the sold out messaging would
56:25
be like a little more FOMO in that
56:27
I don't want to miss this,
56:28
and the it just made no difference.
56:30
Like it just made no difference.
56:31
Like preorder work just fine.
56:32
Even if
56:33
like the product had been at stock
56:34
at one point.
56:35
Preorder versus sold out
56:36
just made made literally
56:37
no difference at all.
56:38
And so yeah, it's L
56:39
for that type of scarcity in this case
56:42
I suppose that's interesting.
56:45
All right.
56:45
Well look,
56:46
you know, if you listen to this,
56:47
you've gotten
56:48
thousands of dollars
56:49
worth of advice for zero
56:51
thanks to Brad.
56:52
Brad, anything you want to say
56:53
before we close this out?
56:55
Not really.
56:56
Just like there's there's
56:58
so many different types of tests
56:59
that you can do.
57:00
So, you know, if you don't constantly
57:02
have like an intelligence
57:03
test of some kind
57:03
running,
57:03
you should probably consider doing that.
57:05
And if you need ideas,
57:06
download the transcript from this
57:08
and rip it through and give you something
57:11
so you have enough
57:13
ideas for the next year.
57:15
Thanks everybody for listening.
57:16
As always,
57:17
we enjoy and appreciate
57:18
any ratings that you give us.
57:20
Have a great rest of your day.
57:24
The only way that we grow
57:25
this podcast is by
57:27
you sharing it with your friends.
57:28
Honestly, like reviews
57:29
kind of don't really mean
57:30
anything too much anymore.
57:31
They're really meaningful,
57:32
but they don't do a lot
57:33
for the growth of the podcast.
57:35
And so sharing YouTube links,
57:37
sharing Spotify
57:38
links, sharing
57:39
Apple, whatever
57:40
we call it under the podcast app now,
57:43
anything you can share,
57:44
the better we're going to be.
57:46
Guys, anything else
57:47
you want to say on this?
57:48
Yeah, please go check us out on YouTube.
57:50
Rack up those views for us.
57:51
We'd love to see it and then subscribe.
57:53
Make sure to subscribe on YouTube
57:54
as well.
57:55
And I relentlessly
57:56
refresh the YouTube comments
57:57
because it dictates
57:58
my mental health for the day.
57:58
So please say something
57:59
nice about all of us.
58:02
Thank you everyone.
58:02
Thanks for listening. Honestly.