Latest / The Search Signal / Why Does AI Mention Some Brands and Not Others? Our Q2 Search Report

Transcript

Why Does AI Mention Some Brands and Not Others? Our Q2 Search Report · The Search Signal

• Michael: So we published our Q2 search report this week at Victorious, and I want to spend some time together today walking through our findings. And the report covers a lot of ground, but the whole thing really revolves around one main question, which is when somebody asks an AI about your category, ⁓ what whether your name comes up in the answer? And there's an assumption sitting underneath many, many AI visibility plans right now for a lot of brands, which is that if AI understands your company well enough, the mentions are just going to follow. And I think many marketers are betting a big piece of their budget on that assumption. So this quarter we tested it. We took 175 brands across five different industries, and we compared how accurately AI can describe each company ⁓ How often that same company gets mentioned in responses to buyer questions. So here's how want to take you through it. So first we're we'll get into what we measured on recognition and on mentions, and then the gap between the two for the brands. Then we'll dig into what does predict a mention ⁓ and I'll you through some follow-up analysis we ran after the report went out. And that's really how this research works, right? Each report that we do just raises new questions, and then we keep on pulling on those threads in between reports. So Some of what I will share today is the result of new questions that this research has ultimately raised for us. After that, we're going to talk about how when an AI answers questions about your category, it almost ⁓ the brand's own website as a source, even when the answer is actually about that brand. And then we're going to look at how the kind of question a buyer asks changes which website gets cited in the answer. And then we're gonna finish by talking about what we would do and what I suggest doing about all of this. ⁓ as always, everything that I mention today is going to be linked in the show notes. So if you want to dig in, feel free to do that yourself. With that being said, let's get to it. Okay, let's start with recognition. So if your company has had any kind of online presence for any period of time, AI most likely knows you exist. Now, whether that's baked into its training data or if it is finding you through a web search, being known honestly is a pretty low bar. And it always has been. But think about how this plays out when someone is going through the buying process. So if there are a hundred companies that do exactly the same thing that you do, AI is not going to list all of them when somebody is asking for options. It's gonna name a few, probably like three or four, maybe as many as five. But like for it to even consider naming you, it has to know you exist in the first place. So knowing you exist is obviously step number one. And that's easy. How to be one of the few companies, though. That AI is picking when a buyer is asking about your industry. That is the question that we ultimately set out to answer this quarter. And here's how we measured it. So we asked eight different AI platforms. So this was ChatGPT, Claude, Gemini, Copilot, Perplexity, Meta. And then we also looked at Google's AI overview and AI mode to describe each of the 175 different companies. And then we graded every response against how those companies describe themselves on their own websites. And of the time or so, AI did get it right, which, ⁓ know, given what we just said about being known, being a low bar, kind of makes sense, right? Now that 96% though, that was not uniform across all of the platforms. So Google AI mode and ⁓ and GPT, AI overviews and Copilot all got companies descriptions right more than like 93% of the time. Perplexity, though, only got it right two-thirds of the time overall. And in the SaaS and ecommerce industries specifically, it dropped way down below 55%. So depending on which AI tools your buyers are using, recognition is not equally dependable ⁓ each of the platforms. And then separately, ⁓ measured how often those same companies got mentioned in AI answers to standardized category and buyer questions. They're kind of like when somebody's asking a question and comparing options, right? And 89% of those brands that we looked at did not come up at all. So you've got almost every company in the data set being understood correctly. And then you have roughly nine out of 10 of them just not getting named at all. The point being is AI knowing you and AI mentioning you barely have anything to do with each other. They are two different behaviors and this quarter we are quantifying just how far apart they actually are. So if an AI knowing a company isn't what actually gets that company named, the question becomes ⁓ what is, So we expanded the analysis that we were doing ⁓ and we at a handful of other signals outside of recognition. And the first one that we looked at is referring domains, which is how many different websites on the internet link to your website. And this is not like how many total links there are, but how many unique and different websites. ⁓ And then the second thing that we looked at, second signal, was third-party web mentions, which are pages out the internet on websites that directly name your brand. So this would be like a directory listing or a review. This could also be UGC content, like a forum thread where someone you know, name drops you while answering a question. It could also be something like a comparison article on a site that you don't own. Okay, we also looked at SEMrush's Authority Score, which is a proprietary metric they use. And it's meant to sum up how authoritative a website is. And we also measured that one last quarter too. So part of where we are watching this quarter was, you know, whether its influence as metric has changed at all in the over quarter data. And then we also looked at organic keyword rankings and organic traffic. And then we checked. Whether each brand had a verified presence in Google's Knowledge Graph, which is the database that Google keeps of real people and companies and the facts that it knows about them. We did a whole deep dive, by the way, on Knowledge Graph a couple of episodes back, if you want the fuller story story there. But out of all of that, right, the two signals that showed the strongest relationship with getting mentioned in AI, referring domains came first. And then third-party mentions landed right behind it. So to be a little careful with how I say this, because I want to be clear, neither one guarantees AI mentions. And nothing we tested was strong enough on its own to predict whether a brand gets named, but both of them were meaningfully ahead of everything else that we had measured. above Authority Score and organic rankings and organic traffic, those all trailed. significantly behind. And Authority Score in particular actually got weaker the harder that we looked at it. So across the full cohort of data, it showed no meaningful relationship whatsoever, which also, coincidentally, does line up with what we found out last quarter last quarter on domain authority, right? The proxy scores ⁓ that have been historically by SEO keep falling apart under the scrutiny of these reports, while the underlying things That they were built to represent keep holding up. So part of the difference here might be what Authority Score is made of. It, from what we understand, bundles traffic estimates and spam signals on top of backlink data. And those extra inputs just don't seem to be lined up with what AI is using for mentions. Knowledge graph presence on its own showed also no relationship at all, which I personally did not expect because. That's kind of the like clean structured data you would figure a machine is going to lean on. And it probably still matters for getting your facts right. And we've talked about this before about why you want AIs to know what you do accurately. It just doesn't appear to be what decides whether or not you get named. We also checked whether the quality of the referring domains mattered more than the quantity of the referring domains. And You know, to be clear about that comparison, quality means links from well-established authoritative websites. And then just quantity means the sheer number of different sites that are linking to you and whatever authority they might be. Now, quality, to be clear, did help, but surprisingly, only a little, right? Some brands in the cohort had links from very authoritative sites and still barely appeared in their AI answers. And when we dug into those cases, The links tended to sit on pages covering topics that didn't line up with the questions that AI was being asked by users and buyers. So this looks to me like a relevance issue. And if you've been following the show for a while, you'll recognize it as something that we call co-occurrence. The links that help these brands actually sat on pages where the brand showed up next to its category. So what that means is authority without that relevance. Doesn't seem to be moving the number very much. Now, everything so far we've talked about has been about the signals ⁓ predict whether you get named. Now citations, which are different, and these are the sources that AI pulls into its own answers, have their own wrinkle, which is that the sites that AI leans on depend on you sell and what industry you're in. So in legal and healthcare AI is leaning on a small and very dominant set of sources for almost everything that it cites. And in legal, there's a handful of big directories, and they accounted for almost a quarter of every citation that we saw. And then in SaaS and ecommerce, it's the opposite, right? Citations are coming from thousands of different websites, more than 10,000 unique domains that we found in just SaaS alone. And there was not a single domain or website dominating. And that difference ⁓ what the work looks like depending on the industry that you're in. And we're gonna come back to that a little later on in the episode. And then on the third-party mention side, which the second most important and valuable metric, we found a volume threshold. And so what I mean by that is brands that had fewer than 2,000 pages out on the web mentioning them got named by AI about 3% of the time. But the brands that had more than 30,000 mentions got named 64% of the time. And right around 20,000 mentions is where the odds crossed 50-50. So, you know, picture the difference between having a couple thousand scattered pages out there mentioning your company. And then having tens of thousands of them. That is the difference between almost never showing up and showing up more often than not. Now, also, I want to clarify: this is a three-month snapshot of group of brands. I would not say this is a hard and fast rule, and I wouldn't treat it like one. We're going to continue to measure this and we're to do it again every quarter and see whether these thresholds stay consistent or whether they move as the platforms themselves, these AI platforms, start to evolve. Now, also here's where I want to go past the published report for just a couple of minutes. Because after we had finished the first round of our research, there was a question that we kept on asking ourselves. Referring domains and third-party mentions were sitting at, you know, right next to the top of each other at the top of the list. ⁓ And in the real world, those two overlap a lot, right? So, like a page that links to you also ⁓ you too. So what we wanted to know was is being talked about doing anything on its own, or is it all just the links that are also happening in that content? And that matters because the answer will change what type of work we need to fund, right? If it's all just ⁓ you just need to put money into classic link building. And these are just placements that point back to your website. But if being talked about carries its own unique signal, then PR style coverage, the kind where someone writes about your company, and whether or not they choose to link to your website or not, deserves its own share of that specific budget. So we went back into the data and we ran that test. And the short version of that is ⁓ the rise and fall together for the most part. So most of what either one measures is the same underlying condition, which is how present your company is across the web. But after you strip out the overlap between them, each one of these is carrying a significant signal its own. What I mean by that is being talked about predicts AI mentions even after you account for the links. And it also runs the other way. The links predict AI mentions even after you account for the third-party mentions. And the links' independent signal came about twice as strong as the mentions. So, point being, both of these are doing very significant work in AI performance. The links, however, are doing a little bit more of it. And I do want to give a caveat because It does matter for what we can claim and not claim here. So our 3rd-party mentions include mentions that also have a link. So there's no tool in a stack that can cleanly separate a page that mentions a brand without linking it from a page that does both. So the pure talked about but never linked effect effect could be somewhat stronger or it could be a little bit weaker than what we measured. But what I would take from it is this PR coverage that is going to get your company talked about does appear to contribute to AI visibility on top of what link building contributes. Now, what the analysis does not support also is dropping link building and going, you know, all in just on PR coverage. The links are still carrying a greater significance and more of the AI performance load. And while we're back in the, you know, while we were back in the data doing this, we also looked at one more thing, which is whether matters if those links are follow or no follow links. If you don't know what this is, just a quick refresher. A no-follow link is a link with a tag on it that tells search engines not to pass any authority through it. And for years, the industry has gone back and forth on whether the no-follow links are ultimately worth earning at all. What we found in this report is that brands with a higher share of followed links did somewhat better on AI mentions. And that result did hold up in all of the ways that we tested it. So it's definitely a real thing, but it's also a surprisingly a small effect. And it sits well below referring domains and third-party mentions themselves. So I wouldn't necessarily build a you know a whole program around it, but if you're ever choosing between two placements, the follow link is worth a little bit more. But it is important to know that no follow links are also efficacious in the performance of brands in AI search tools. And then there's also one more piece of this, and I want to walk you through it because I think I had my own theory about the links itself, and it honestly did not pan out. So when we were outlining this episode, I had a hunch about anchor text, which is, you know, the actual word that a backlink uses to link to your site. So my thinking was that links using a ⁓ name in the anchor which is the kind that you typically earn when someone's writing about the company, those would predict AI mentions better than you know, keyword or generic anchors that you get from classic link building. In in my opinion, it fit everything we talked about. And we pulled the anchor text profiles for every brand in the cohort and we ended up testing it and it did not hold up, right? Brands that had a higher share of brand name anchors got mentioned slightly less, not slightly more. And, you know, before you go and flip your link strategy on its head over this, the reason why it turned out to be this way is, I think, kind of interesting. So we looked at which brands were driving the result. And they weren't, you know, broadly covered PR savvy companies. These were more like regional chains single market businesses whose links are almost entirely local directory listings. And, you a directory pretty much always links to you using your name. So, you know, a high share of brand name anchors that turned out to be a symptom of what looks to be more more like a thin profile without much else in it, more than like a a marker of earned coverage. So you know, a metric that looks like quality on its face was actually measuring some narrowness in the link profile instead. And then the one sliver that did show a small positive signal was actually naked domain anchors, which are when the link is just like a bare URL. It's like, you know, yourcompany.com. ⁓ personally, I'd read this less as like a tactic that I would want to do and more of like another echo ⁓ the theme that's been running through this whole quarter, which is, ⁓ know, being cited plainly by name across a broad set of sites that you don't own. And the point being, I would not treat anchor text as a lever to pull an AI search performance. The thing that is worth chasing is definitely going to still be the breadth of your link building strategy. The more sites out there citing you plainly by name or by not. So, with all of that, where does that leave us this quarter, right? This quarter, we didn't we didn't find a you know one single magic ⁓ that explains who gets named and AI answers. But taken together though, they all are pointing in the same direction, which, you know, is towards how visible your company is on parts of the web that you don't own, sites that you don't control and written by people that are not you. Okay, so the strongest predictors of a mention, as we just talked about, all live outside of your own website, which will probably raise a natural next question. And it's probably the one that I would be asking if I was you, which is if AI does mention you, does your own website at least get cited as a source? Because you know, if it does, then the fix in the solution here is pretty simple, right? You just publish a lot more content on your website and just start stacking up and collecting the citations. Now, if you caught episode five of The Search Signal, you probably already know where this is headed. We've talked about this there, about how ⁓ of what AI says about you was not using sources and that was written by you. And this quarter we got to measure that thesis at scale in our own data. And the answer is: it's right, it just doesn't happen. We analyzed just under 50,000 citations across almost 6,000 AI-generated answers and and 99.99% of them pointed somewhere other than the brand's own Out of 150 companies, four got a citation to their own website. And that wasn't one platform just skewing the number. ChatGPT and Google AI Mode each cited a brand's own site only once or twice across over 25,000 combined citations. And Gemini and AI mode or AI overviews never did it once in this study. And just for context, so you know, two platforms we're missing from these numbers, keep it honest, Claude and Meta AI, we collect those answers through their APIs, which is how we can get data at this scale. And web search just wasn't available on either one during the collection. So ⁓ web search ⁓ means no sources to count. So just a measurement gap in the data, not a pattern. But to get back to it, so does this mean publishing on your own website is a wasted effort? No. And I want to definitely head that off straight on because it would be a very easy conclusion to walk away with. Your website is still what AI checks against when it is describing you. And it is still there when the buyer eventually wants to make a decision. It just isn't enough on its own to earn the citation and an answer about your category. So those citations are going to third party sources that AI already trusts in your industry, which, you know, when you think about it, lines up with what the mention data had also told us, which is that the signals that predict whether you're getting mentioned live off of your website. And the sources AI sites live off of your website too. Both behaviors are running through third-party web mentions. So the last finding I want to walk through is about the questions themselves, because you know, the same company can be ⁓ nearly in the answers to one kind of buyer question and then very present in the answers to another, depending on ⁓ where buyer is in their journey. So we tested two different types of buyer questions. And the first kind comes before somebody knows what solution that they need. And then it will this would be something like, you know, how do I know if I if my business needs a lawyer? Right. And then the second kind comes once they're actually comparing options. And this would be something like what are the best law firms in the US, right? ⁓ and to sense of what we saw, you have to ⁓ hold, you know, and citations separately. And so we'll start with citations first, which again are the sources that AI pulls in to build its answer. So both kinds of questions produced citations in the majority of their answers. But which sources got cited flipped almost completely between the two moments. So early on, before you know someone knows what they're looking for, AI is typically citing educational content like videos, the government sites, institutional sites. It was looking at we saw citations for professional and a lot of company educational content. And then there's also outside research pointing in the same direction. There was an academic study out of a university earlier this year that looked at consumer health questions, and they found that more than three-quarters of what ChatGPT was citing ⁓ came from ⁓ authoritative sources. And these like medical institutions and ⁓ government resources, kind of thing. ⁓ But once Actually comparing options, the citations moved towards things like directories and rankings and review sites and ⁓ like comparison pieces. So now that citations, now mentions are a very different story and they are wildly different. And they're depending on where the buyer is in their journey. So in those early questions, a brand that got ⁓ named, ⁓ got named and maybe one out of every thousand answers. And then in the comparison questions brands got named more than 12 times as often as that. And even within the comparison questions, you know, citing a source and naming a brand ⁓ ⁓ by platform by platform, which echoes some of the recognition versus mention gap that we talked about earlier today in this episode. So Google AI mode and Google AI overviews were citing sources at basically the same rate, but AI mode named a brand more than twice as often as AI overviews did. So this is the same company, right? Two different systems and a very different willingness to ⁓ and mention a brand name out loud. So, you know, educational content helps you start building trust early. ⁓ that type of content can earn citations before anybody is evaluating a provider. So category content, category focused content is what creates the opening for AI to introduce your name once they are ready. Both of them matter, but they do matter at different moments, which is why your AI strategy has to take the funnel into consideration. So the sources are going to be different depending on where the customer is in their journey. So, you know, one blended visibility number can't tell you whether you are missing the early conversation or missing the late conversation or missing both of them. So ⁓ what would do with all this ⁓ if I were you? The first I would do, I would look at your AI search tracking system because this quarter's data is basically an argument for a more granular setup. We talked about splitting citation rate from mention rate and also keeping the platforms separate. a couple of episodes back on The Search Signal. What this quarter adds is just another layer, which is split by the kind of question too. So citation rate and mention rate tracked platform by platform ⁓ across both the early, like how do I know if I need this questions? And then the later, like, what are the best option questions? I'm not gonna rehash the full setup here. It's one of the earlier episodes if you want to read it. And I will also link to it in the show notes. But the second thing that I would do. You know, I would go pull your current third-party mention count. How many pages ⁓ on the web are mentioning your company by name? And if you work with an agency like Victorious or if you've got an in-house search team, they can pull it from a tool. You can use like Ahrefs or Semrush pretty quickly. It's a simple number that probably isn't, but could be, might not be, on your dashboard yet. ⁓ And what would then is just check where you are performing against the range that we had just walked through. To remind you, our data found that under 2,000 mentions, brands almost never get named. ⁓ And around 20,000 is where mentioned became more likely than not likely. ⁓ So wherever you are ⁓ right now, in terms third-party mention count, is going to tell you roughly how much third-party visibility work is in front of you, right? ⁓ And you can know that before you start committing budget or resourcing to it. And then the third thing I would do, when do you start building that third-party presence? Well, once you decide to do that, I would go broad ⁓ and would go relevant before I chased a handful of placements on a bunch of big brand name sexy websites. So you know, a couple of links from the single most authoritative site in your industry is going to actually help less than you would think if the pages that they sit on are gonna cover topics that your buyers aren't asking about. Now, what broad looks like is going to depend on your industry. So if you're in legal or like in healthcare, there, you know, where a small set of sources right now are currently dominating what an AI sites, earning a place in that specific set of sites is important and is most of the job. But if you're in something like SaaS or an ecommerce, where citations like we talked about come from thousands of different websites and places, breadth is really going to win. And, you know, whichever industry that you are in, I would give honestly this work its own dedicated resources and it needs to get funded deliberately instead of getting squeezed with whatever is left at the end of the quarter after paid media and all of other marketing spend. Right. I would encourage you the full report is going to break down for you which sources are dominating each of the industries that we studied. So, you know, you don't have to guess at where to start. You can download that and look at that right now. And then I would also, you know, given the follow-up analysis that we had also talked about, I'd keep running both kinds of off-site work too. Meaning I'd still keep running link building and I'd still keep running PR style coverage that gets your company mentioned and talked about. So, like we said, the links still carry the strongest signal that we mentioned. So that's the last thing I would start to cut. ⁓ if you have been also discounting, placements that don't pass authority through link building, like the no follow links, I'd also reconsider deploying those. I said, by our data, they're worth a little bit less than follow links, but not significantly less. And then the thing that I would do is I would put ⁓ some the budget that you have towards content built to be cited in the first place. So this is things like original research and proprietary like our own here. The Search Signal and our quarterly search report, or a perspective that your company could offer that's unique. That gives another site a reason to reference you when they are writing about something in your category instead of you having to go out and earn every single mention, you know, one pitch at a time. And to be clear, also taking a step back, none of this tells you what happens after somebody reads an AI answer that mentions you, right? they visit your website or whether they are going to fill out a form on your website once they're on your website. Those are separate ⁓ problems. They have their separate fixes. ⁓ we covered actually how to approach all of those in our third episode from things like your GA4 setup to the how you hear about us field on your lead forms. So if you know measurement your gap, I would so go say go back and listen to that episode. And if it were me and I only had time for one of these this week, I'd just start by pulling the mention count. It's less than a half hour work, honestly. And it's going to tell you whether you're staring at a gap of 2,000 pages or 20,000 pages or more, right? So with that being said, let's recap where we ended up today. So taking a step back, AI almost certainly already understands your company. That was true for, you know, 96% of the brands that we had studied, and it did basically nothing to differentiate them enough to earn any mentions in AI. What did move the number was a presence on the part of the web that you do not own, with referring domains and third-party mentions out front, ⁓ with links still doing the most work of anything that we had tested. So when an AI answers a category question, it almost never cites the brand's own website. Though educational content can still earn citations earlier on in the buyer journey ⁓ they actually start comparing options. And the kind of question a buyer asks does change which sources get cited and whether any brand gets named at all. So if you want a bigger share of those answers, the work is out on the third party web. We need to be building on sites relevant to our category. That's where all of the data from our search report pointed to this quarter and that's the work where you can start this week. So you get a chance, ⁓ go check the quarterly search report, the full report on our website. It has got the industry by industry breakdown behind everything that we covered today. And it also has a few findings we did not have time to talk about. ⁓ And as if you got something out of this episode, please go ahead and follow the show wherever you are listening or watching so the next one finds you. That's it for me. Thanks for listening to The Search Signal. I will see you next week.