Latest / Honest Ecommerce / CRO Is Broken: Alex McEachern of Intelligems on Growth Testing

Transcript

CRO Is Broken: Alex McEachern of Intelligems on Growth Testing · Honest Ecommerce

• speaker-0: Just because you can doesn't mean you should. And I think the one thing that AI has introduced is it's gotten rid of ⁓ the barrier of entry of tr of trying new things. And so people are trying everything. speaker-1: Anyone can build anything now, it doesn't mean that it's going to be good. And like this is what I was getting at before is people can try a million different things right now. If you're trying a million different things and you never actually take the time to know what worked, you just have a million different things that you're working on. You're keeping yourself busy. You're not trying to be effective. speaker-0: Honest ECommerce is a weekly podcast where we interview direct-to-consumer brand founders and leaders to find out what it takes to start, grow, and scale an online business today. Hey everybody, welcome back to another bonus episode of Honest E-Commerce. Today I'm welcoming the show Alex from Intelligence. Alex, what is your title now over there? speaker-1: Head of marketing. I'm also glad that you didn't attempt the last name 'cause it always gets ⁓ it always gets butchered. speaker-0: Yeah, I've got some ⁓ skill in this podcasting game. Been doing it for almost a decade now, so I can know when ⁓ a a word is going to defeat me when I try to read it. speaker-1: Love it. Love it. Yeah, head of marketing over here at Intelligems. Excited to jam, man. I feel like you and I you and I always have great conversations. So excited to see what comes out of this. speaker-0: Absolutely. So ⁓ first things first, IntelliGims is a huge partner of ours over at Electric Eye. ⁓ this is not a sponsored episode by any means, but we use the product a lot. I'm gonna be heavily in favor of everything that they're doing over there. but you know, ⁓ if you want to speak offline about my direct opinion of the product, it's probably not gonna change much from what you're hearing vocally here. They sponsored a lot of our product ⁓ a lot of our podcasts in the past, a lot of our events, all that jazz. But this particular conversation. is actually just ⁓ us chatting. ⁓ I was lucky enough to join Alex a few weeks ago on their podcast called IntelliJams. ⁓ very, very ⁓ great title for that. Now you can find it on YouTube and it's also where normal podcast stuff is. Go speaker-1: Look for it wherever you get your podcast, but the best experience is always on ⁓ on YouTube for that one. speaker-0: Absolutely. And we're also ⁓ this is going out on YouTube and we're finding some success with that. And ⁓ Alex and I actually chat ⁓ about you know the YouTube algorithm and just podcasting in general, ⁓ just to so we can both try to just make awesome content for the community out there. So ⁓ Alex is a great guy, IntelliGems is a great pro product. Enough with ⁓ saying nice things. Let's let's get into it today. If you couldn't guess, the overarching conversation is going to be about conversion rate optimization. And something that came up in our jam a few weeks ago that you really loved was this the concept that ⁓ conversion rate optimization is terribly named. Yes. speaker-1: I ⁓ like I think the I said to Chase right after we were talking about him like we could put a we could put together a full episode on this, and that's what I'm hoping that we can potentially do today. But yeah, CRO, I mean, you and I have been in the space for a while. It's been the term, it's been the name for a decade in the Shopify ecosystem and the commerce ecosystem, and I think it's time that we renamed it. I have some ideas on what we rename it to, but I'd love to go through kind of like what your thoughts are on CRO. The name, why it's not the right name. I I I just feel like it sells everything we do so short and I'm getting sick of it and it needs a new name. speaker-0: Yeah, it hits really close to home because we're redesigning our website right now and the industry and by industry I mean the the the customer. So at the end of the day it'd be the brands that work with with our agency. They call it conversion rate optimization. They're typing in Google and into Chat GPT or Claude conversion rate optimization. But ⁓ maybe it's because we know too much. We're like, that's not actually That's like one piece of what we're doing. And so let's see if you can remember ⁓ you know, kind of how I explained it back on your episode on the IntelliJams, but like ⁓ you know, what does a conversion like what what did a conversion mean historically? It kind of led into why this is named wrong. speaker-1: Yeah, think from what I remember, you were saying it came from like computer programming at some point. ⁓ or I maybe, but like conversion just being like the generalized term for action was taken, but in our industry conversion has been changed to purchase has happened. I think. speaker-0: You hit the nail on the head. Yeah, that's exactly what it was. So ⁓ whenever computers in the internet started, this I don't know if this pre internet or not, but I'm assuming this is also something that would happen in like just software. But basically back in the day, ⁓ a conversion was just like the user doing whatever the heck you wanted them to do. So it'd be like clicking a button or, you know, submitting a form, you know, viewing a page, right? So actually a more modern example to like think about this is like With advertising, you can bid on, you know, a click, you can bid on a page view, you can bid on a form submit, you can bid on an actual purchase of your product, right? All of those kind of things that you're bidding on and it's optimizing towards, all of those are conversion events in the grand scheme of things. So when it's you're generalized just saying, you know, conversion rate optimization, people are only thinking about one particular thing of it being buying the product, ⁓ is a little bit, you know, it's leaving a lot of extra activity out there that, you know, specialists like us are taking care of. Products like yours are optimizing towards it's hilarious, ⁓ knowing what I know about your product that, you know, you weren't optimizing towards conversions at one point. ⁓ even at the start of the product, it wasn't optimizing towards conversions at all. speaker-1: Yeah. And I mean, this is one of the reasons that I don't love the name is because like conversion rate optimization in D to C means something very specific. And like in our industry, it means optimize for more purchase conversions. And I don't like the term because when you think about conversion rate optimization, it's like right there in the name. Like just the conversion isn't actually meaning you're getting a more successful outcome for your business. Like my favorite straw man argument is like you want your conversion to go up, drop your prices to zero. You're gonna have a great conversion on those ads coming in, but it's not gonna be good for your business. So I think like it's limiting in like it's too narrowly focused, but I think it also needs to be broadened to include the ultimate goal for a brand, for a business, which is like not just to get the most conversions I can possibly get. Everyone's gonna have their own definition of what success is, but like it's all going to be. more or less centered around growth, growth of the business. And I think like a lot more often than not now focused around profitable growth. Can I better my margins? Can I make sure that like I'm doing this in a foundationally sound way more than kind of like growth at all costs? speaker-0: Yeah. And something that we do kind of early on in the sales process is like we ask a question and it's not to be a gotcha question. It's to like understand where that are this potential client is in their kind of understanding of what CRO is, what conversion rate optimization is, is we go, Hey, like ⁓ if we increase your profit but your conversion rate actually goes down, is that a good outcome? And they always have to think about it. ⁓ you know, and we'll we'll ask it in multiple ways, right? It's like, hey, like if your average order value goes up, but your conversion rate goes down a little bit, but overall your sales have increased, is that a win? Or is it purely you need to see the conversion rate go up for you to consider this a successful partnership? Right. And, you know, it isn't a these aren't gotcha moments. It's just like trying to point out that like this is just one lever in, you know, ⁓ a dozen or so important KPIs that you can probably optimize towards but way less than that actually. We only care about three at the end of the day at Electric Eye. But it you it's just like start to talk it's like have these honest conversations up front, set proper expectations up front, ⁓ you know, and also just the perceived actions of what conversion rate optimization is is hilarious to me. I think we should get into that in a little bit. But ⁓ what are your thoughts on what I just said? speaker-1: Yeah, when you said like, hey, we optimized the three things, like the those gotcha moments, I think they're again, they're not gotcha moments. It's just there's gonna be people at a brand that all are optimizing for different things. If I am on the ad side, I might want like I'm optimizing for traffic to the site, but I also want to make sure that traffic to the site is valuable to me. So like, hey, I'm going to maybe be more focused on like actual conversion as we were talking about at the beginning. Like, does the ad lead to a sale through this funnel or through this channel. Then there might be someone who's like the direct to consumer manager who's more responsible for the site. And they might be like, I really care about AOV. It's my responsibility if people are going to be sending me all this traffic to make sure that I'm getting the most out of it. But then there's going to be like a founder or a CMO or like a VP finance or someone who's like, it's my responsibility to look at the PL of this business. And like all of these people care about like At the highest level, the same thing, even though they are all going to like micro optimize in different areas. And like, hey, if I try to micro optimize on one thing, you alluded to it, something goes up, something else goes down, something else goes up. Like, very rarely do you run a test or an experiment where only the metric you are trying to isolate moves. Everything you do in terms of changes for your site is going to be some form of trade-off. And I love that you're asking those questions at the beginning to get an understanding of like what is an acceptable trade-off? What do you actually care about? Because I bet you it's not just raw conversion. speaker-0: You know, most of the time you will get people to admit out loud that it isn't like conversions just the thing they're fixated on. They truly just want a more profitable business and a more optimized, let's call it a funnel for lack of a better term, but we could probably have a whole conversation about that. They want to improve their business and they know that their specialty lies in they created a product that solves a problem and it seems the consumers want it. They now are doing the smart thing of hiring experts that are good at looking at the data, looking at the surveys, looking at the analytics to make informed hypotheses for us to test with. I'll put an aside there ⁓ about that. ⁓ you know, and they're smart to hire people like us or, you know, many of the other dozens and dozens of clients that not clients, but ⁓ partners that you partner with over there. And why partners, I mean like other agencies. I'm we're not we're not the only people doing this in the world. I'm not, you know, but I will say we're the coolest. I don't know. speaker-1: Yeah. And like I think the the other thing too is like even for us like you on the agency side or like us on the on the software side and like people there's some brands who are going to have people with like CRO in their title. Like when you work with big enough brands, someone might have like a CRO specialist title or like they might be like speaker-0: When I see that sometimes is is it conversion rate optimization or is it chief revenue officer? speaker-1: Yes, yes. I'm talking about the conversion rate optimized. I usually the CRO doesn't have a s if they're a chief revenue officer, doesn't have specialist at the at the end. they're usually just just the CRO. But I've been having like a ton of conversations with people at some of these like larger brands that aren't necessarily on Shopify, might not even be like in commerce. And there's this kind of like sentiment right now that CRO is being put into this box and that like, ⁓ it it's It's truly just about like UX changes and trying to find like small optimizations and like, ⁓ it's my job to kind of like stack these smaller changes on top of each other and try to find like a two percent here or a five percent there. And a lot of people are getting like frustrated by this. It's like, ⁓ the walls are kind of coming in around me, but I actually feel more of like I'm a product person or I'm like a product manager where like I own something and it's my job to get the best result. out of the thing that I'm managing. And I'm like, that's interesting to me that like CRO is trying to like break away from these stigmas that exist from the past and trying to operate more like a product manager or like a growth engineer. ⁓ but like trying to shed this, Chase, you do CRO and therefore you must do a lot of like just button color changes and headline changes. And like you you just move the UX, like you move a button here, you move this here. And that will like completely change your business. Yeah, those things can stack on top of each other. But you also need to try to find like bigger, meatier things to go experiment with, to go test with. And I feel like CRO people, meaning conversion rate optimization people, are struggling with that right now. speaker-0: Hey everybody, just a quick reminder. Please like this video and subscribe if you haven't. We're releasing interviews like this every week. So don't miss out. Now back to the interview. I do want to talk about testing because it you brought it up and I alluded to it earlier. To burst the bubble of startups, you need a lot of velocity to actually do testing. Where we draw the line isn't even where the intelligence product documentation draws the line. We're definitely a little bit more conservative. But ⁓ when we're in conversations in the sales cycle with folks, we're like, are you doing a thousand conversions? AK, are your customers buying a thousand orders a week? We want a week. And the reason being, when you are to the point where you're doing testing, you need this thing called statistical significance. And that can happen in a vacuum over any sort of timeline, but when you're in the world, things change weekly, maybe even faster. But you know, you can't have politics or Facebook's algorithm, you can't have seasonality, you can't have sales change within the timeline of you running a test. So just f if you're thinking about getting into conversion rate optimization, you need to make sure that you're selling enough stuff and a quick enough kind of velocity ⁓ or you can't do testing. You can still do optimization. And again, it's badly named. One, why did conversion rate optimization just get bumped in with split testing? They're now like amalgamized together. You can still do a lot of you can still optimize your store without running split tests if you're too young to do so. And you can still get a bunch of data through other apps in the ecosystem or just other ways to then make informed decisions. to make your store better, you just can't test it. speaker-1: I like that you use the term informed decisions there, because that's something that I actually talk about a lot. And like other people at IntelGems might listen to what I'm about to say and be like, Alex, don't don't say that. Don't say that. And it's that like, yes, you need to be moving towards statistical significance, like if you're gonna be running a test. And like a lot of people think that to get to stat sig, you just need more data. You need like more swings at the thing. That's true. The more data you have, the faster you're gonna get to statistical significance. But it's also a measure of like how drastic the change is. So if you're changing a button color, you're gonna have to wait forever to try to figure out if that's stat sig or not. But like if you increase your prices by 20%, you're gonna see that mellow out faster and like give you like a more accurate read because the change is so drastic. ⁓ so like I like the informed decision angle there because like one thing I always find myself saying all the time is like ask important questions and validate important decisions. And like, You can go put yourself in a position where you feel more confident making a decision or having an answer to a question, even if it's not necessarily stats sig on a very specific thing. You just feel like you're making a more informed decision than you were before because you have more information. You've done it in a, you've done it with more rigor than like, I mean, most of the changes that happen in our world is like, I made the change and I think it worked. speaker-0: Yeah. I wanna talk about button color because it comes up a lot. Nobody listening to this podcast, nobody in my network, nobody in your network, which is definitely you guys, a lot of larger brands use your that product. Buttoncolor won't do anything, right? Button color affects Instagram with a thousand actions happening a second. It isn't something that's happening for conversion rate like at even a hundred million dollar brand. I would love for someone to check me on this and be like, you know what, it actually does at a hundred million, but for a one million dollar, a five million dollar, a ten million for a ten million dollar brand, button color, unless your button color was like breaking the site, if it was just, ⁓ we're gonna test red versus this other color of red, no, it's not gonna, it's not gonna have any really effect on it. What will speaker-1: Yeah, someone someone fight us on it. speaker-0: And ⁓ one of the my first coaches in in teaching a CRO, what one of the first things he said is like test more, test bigger things, you know, do ⁓ complete layout changes, test those against each other. That's gonna give you a better result than, you know, some micro changing one word on a product, maybe one word in a headline. Okay, that you know, if that's the first thing that I'm reading, that might be a significant test. But ⁓ you know, one one word in, you know, the the script product description, I don't think that's gonna do it. ⁓ you know, test one product page versus another product page. Test a ⁓ you know, you know, if you're getting to the point where you're sending ⁓ cold cold traffic, not cold, but I guess paid traffic from meta to landing pages, maybe try one landing page format versus another landing page format. ⁓ here side quest here, I guess, Claude or whatever your favorite AI. We'll straight up just give you a layout for your landing page based upon these formats. Just ask him what like are some of the popular landing page formats and then be Well, here's my here's my product. Give me all those you know, give me landing page designs for all these. Don't use its design though. Holy it's so bad at design. Don't tell them to design it. Just tell them to lay it out like structure wise with placeholder copy. Then you and your copywriter make it better and then give it to a real designer. And then ⁓ you can develop it yourself. ⁓ I am unaffiliated with Ryan over at Page Deck, but we have been playing with Page Deck with one of our clients right now. not a big fan of what you see is what you get page editors, but it is kind of working and all the data is there. So I'm not super mad at it right now, but I don't know. I'm done with my ⁓ tan tangential side quest here. speaker-1: My kind of like read on most things is like it or my thoughts on like AI inside of experimentation is it has become so easy to build on your own. ⁓ like, yeah, don't use cloud to design your stuff, but like there are tools out there that help you get to a point where like maybe you could port something into a Shopify template and like see how it goes. And I think like the build component of AI is kind of like making everyone, giving everyone the ability to change whatever they want. Whenever they want. And like you and I in our world is like, okay, but I also need AI to like help me figure out what is actually worth doing so that I don't just like change a bunch of stuff constantly and just like constantly build to be productive. And then I want like AI on the other side of this too to be like, I made this change. What's going on? Not just like, hey, what met like this one metric that changed, but like give me a read of what's happening in the business by doing these bigger things. And like also when you're talking about like try bigger things, like you can even think bigger. Than some of the stuff we were talking about there. Like, what if I have live chat support on the site? Can I experiment with having live chat support on and off? If I already offer live chat support, can I experiment with like having the AI bot take over versus live chat? What does that do to revenue procession? If I am thinking about introducing like a new flavor or scent, can I do like a limited run and introduce that? Because like I might think that that's going to be incremental to my business and like introducing this flavor or scent is going to get me more dollars. But maybe it just moves people to that scent from other scents and like I just have more products in my portfolio, more overhead costs of creating all this, but it doesn't actually lead to more profitability. So like when I talk when I say like ⁓ CRO is flawed, it's because like I want people to think about like all these like higher level things that you could be experimenting with. So that it's not just like, ⁓ it's CRO or it's testing. It's like A term I've been throwing around is like growth experimentation. It's like you have ideas, you have hypotheses around what could change your business, what the limiting factors are, like where you could go explore more growth. Go try some of those bigger things. And I'm not saying don't do the small stuff. Like I think you should always be trying to optimize. Like you can always find dollars in the optimizations. Do both. speaker-0: Yeah. And obviously I earlier I mentioned like, you know, we're redesigning our website right now. And the headline copy on it is we're definitely leaning more into our optimization shops and, you know, agency versus ⁓ studio versus ⁓ another one word kind of, you know, description. You know, agency one out. So, you know, we're going with Shopify Premier Optimization Agency ⁓ is probably gonna be the big the big flagship headline on the website once it goes live. ⁓ so we couldn't get too far away from ⁓ kind of conversion rate optimization. We're still using that word, but I mean right underneath it, we're gonna have to to list it. So Google likes it, likes us, so Claude pulls us up, so ChatGPT finds us in that ⁓ LLM. But the one thing ⁓ I I kinda wanna end on here, because we are talking about AI a little bit now here at the end. Is AI is stupid and I can prove it. Just because you can doesn't mean you should. And I think the one thing that AI has introduced is it's gotten rid of ⁓ the barrier of entry of tr of trying new things. And so people are trying everything. And an old analogy that I've been using for years is you know, if everything's important, nothing is important. So you still need to slow down before you can speed up. You still need to have strategy and AI. ⁓ at a surface can understand strategy. It's I'm I'm not worried about our jobs. I'm not worried about anyone that's good at their jobs being replaced by AI. AI in the tool of someone that understands what they're doing makes them infinitely more capable. But AI in the tool in the hands of a layman just creates mediocre, if not terrible results. Right. So that's the end of my rant on AI. speaker-1: Yeah, but like anyone can build anything now, it doesn't mean that it's going to be good. And like this is what I was getting at before is people can try a million different things right now. If you're trying a million different things and you never actually take the time to know what worked, you just have a million different things that you're working on. You're keeping yourself busy. You're not trying to be effective. We said at the beginning of this like if the if the goal is I'm in pursuit of more profit, did doing 16 to 260 different things get me closer to that goal? I don't like maybe I don't even know because I'm just making these changes and I'm not actually able to measure what's going on and like tell myself confidently it was a good decision. It was just two hundred and sixty decisions that kept me busy. speaker-0: Yeah, I I think, you know, a a bad analogy, but one I'm gonna be working on is like I have a hammer, I have nails, I can go get some wood. Alex, I can build you a house, man. You want me to you think it's gonna be a good house? speaker-1: ⁓ no. Actually I don't I don't know your capabilities as a carpenter or a craftsman, but I'm assuming no. speaker-0: I'll tell you right now, it's not going to be it's not going to be good. No power, no power of Claude or or or YouTube is going to offset the amount of time and skills it takes to actually learn the nuances of building a house and all the intricacies of every subsystem within building a house. So obviously that sounded dumb, but that's what people are doing when they're saying that AI can build them a successful business. It's like No it can't, but it can help you along the way. speaker-1: Yeah, man. Last thing I'll say on is like it's basically the Dunning Kruger, right? Like AI is making people think they're better at something, but like you get to a point, even like even a layman using AI gets to a point where it's like, ⁓ shit, I don't know what I don't know. But it accelerated the crap out of me getting to that high point. speaker-0: Absolutely. Alex, thank you so much for coming on the show today. ⁓ if people want to hear you t talk more, aka ⁓ check out your podcast. Where should they go? What should they do? speaker-1: Yeah, best place to find it. Go to YouTube, search IntelliJams, J A S. ⁓ if you want more of me, you can find me on Twitter at Alex ⁓ C E A or just search Alex ⁓ C E A on LinkedIn as well, and you'll find me. speaker-0: Perfect. We'll link to that stuff in the show notes. Go check out IntelliGyms if you're doing over a thousand orders a week and you want to actually have a cool product help you with split testing. ⁓ obviously I'm gonna plug my agency. If you need someone to help you do the split testing, come up with the ideas, build a system that builds you a, you know, an actual research-backed roadmap of what to test. That's what we're doing for dozens of brands all the time. reach out to, you know, us, electri.io. Alex, thanks a lot. Talk to you soon. speaker-1: Thanks for having me.