Latest / The Search Signal / AI Brand Reputation: Where It Comes From & How To Fix It

Transcript

AI Brand Reputation: Where It Comes From & How To Fix It · The Search Signal

• Michael Transon: Hey, welcome back to The Search Signal. My name is Michael Transon. I'll be your host. I'm also the CEO and founder of a search agency called Victorious. ⁓ before we get into today's episode, let's run through the news and see what happened this week. Let's start with Google from this week's news. So Google released a new feature in Search Console ⁓ called Platform. Properties. So this is going to let you see how to check content from your Instagram, your TikTok, your X, YouTube, and see how that is performing inside of Google Search itself. So this is going to include clicks, it's going to include impressions, and also the queries or keywords that are bringing people in. It also is going to include your top performing posts. So this is ⁓ in the same reports that you are already using for your own website inside of Search Console. This is also the first time Google has shown you performance data for content on channels that you don't own, like these social media channels. It's also opt-in. So which means you have to verify each of your platforms separately in this new feature. It's also going to be rolling out a little bit gradually. So you might not see it in your GSC property yet. But once you do, and once your social and your video properties are all verified in Search Console. And that data is flowing. You can use it to see where your impressions are generated across the various social channels that you are tracking inside of that. there are also very fresh studies from SEMrush that I want to talk about today. So the first one is SEMrush's team surveyed about 600 B2B professionals the United States specifically about how they are buying now. And to no one's surprise, it's turning out that AI is present at pretty much every stage of the buying consideration process. So ⁓ almost all of the respondents, about 97%. Said AI had helped them find vendors that they didn't already even know about. So most of the respondents said it shaped their short list. And then more than 80% them said it influenced that actual final decision, even on purchases over $1,000. So the survey responses had generally indicated that ChatGPT is the tool that they were using the most for this. ⁓ They self-reported that their selections were influenced more by a precise match to their needs and a very clear, detailed description, those were more important than how they were being influenced by like brand recognition, which is being aware of the brand already, which they said had landed near the bottom. So the takeaway here that I would give is that being a well-known brand doesn't help right now if AI systems aren't able to effectively describe your offerings in the context of a specific buyer's needs. So I would say go look at your highest value pages on your site now if you want to look at this and ask yourself whether they are doing a good job of effectively spelling what you do and how you solve problems for your customers in very plain language because that is the type of information that the model is matching against when it's deciding which of these brands to put in front of your buyer when they are using the various LLM platforms. And another SEMrush study that came out this week, and this one was across more than 600,000 different keywords. So this is really big show that overviews is pushing transactional and commercial keyword searches down. So for a long time, the AI overview answers in Google were mostly showing up in like informational searches. Now, instead, they are starting to ⁓ spread. And they're starting to spread into commercial and transactional intent queries. And this is where people and buyers are comparing options and they are getting to buy, right? And that category has grown more than 70% transactional commercial-based queries, 70% ⁓ over last six months. ⁓ And the ⁓ that are AI overviews in that category are looking to be the most valuable commercially viable terms in when you look at the PPC costs. So in some categories, the average cost per click was more than three times higher on keywords that had AI overviews than ones that didn't have AI overviews. So we need to be asking our search teams how our brand is showing up inside of these AI overviews on commercial keywords, because the ranking that you might have on that page. ⁓ might not be moving at all while the AI answer is starting to siphon off clicks. And those are clicks that are turning into revenue. Now, this should not be surprising that Google is doing this. They are looking for ways to further and further monetize these AI overviews, but we will see what happens next. So that's the news for this week. Let's get into today's episode. Okay, so as AI search has started to roll out over the last, you know, few months, most marketers have very rightly and fairly focused on two important metrics, which has been their citation rate and their mention rate. And they've also focused on share a voice, but I've seen a lot of people focus on mentions and citations. And just to remind you what mentions and citations are, ⁓ mentions are going be when an answer from an LLM engine that is given to a user. Mentions your brand name or a variation of your brand name in the actual response itself. And then citation rate or citations are when an LLM uses your website information, a page on your website or a piece of content that you wrote that you hosted on your website as source material for formulating an answer. And you can be mentioned and you can be cited in the same answer, or you could be only mentioned or only cited. Or you could have neither. So most folks have been focusing on whether or not they're being cited or whether they're being mentioned in the responses. But ⁓ the kicker: like mention rates and citation rate don't really have anything to do with the quality that ⁓ AI system will communicate about your brand, right? It won't say whether it doesn't say if you get mentioned, it doesn't say whether it was a good mention or a bad mention. If you get cited, it doesn't say if it's a good citation or a bad citation. It's just did you or did you not? Right. But what we're talking about here as it relates to the quality of your brand's representation is your brand reputation, right? And what I would say is if you're looking at your brand's reputation in LLMs and you get a favorable or positive response, that is great, right? But if you're being portrayed negatively, right, or even if you're being portrayed neutrally. Right. You need to understand the mechanism that can create that type of situation and figure out what sources could be influencing that response and what you can and also what you cannot do to help LLMs report a more positive sentiment for your brand. So today we're going to cover a couple of things today. First, we're going cover why misrepresentation in LLMs is a commercial risk for your company. Second, we're going to then talk about the two mechanisms that are behind every AI answer that help influence ⁓ how your brand is portrayed in these answers. And then third, going to look at why your reputation could potentially look different depending on what AI you ask about ⁓ your brand. then lastly, we're going to finish today's episode. We're just going to be very practical. We're going to talk about what you can do to monitor. And then what you can do to improve your AI brand reputation. So by the end of this episode today, I want you to feel empowered really to, you know, if you want to run a reputation diagnosis yourself, right? And create a recovery plan if you or if your brand ultimately do need it. So an AI agent named Fractl did a survey recently and they surveyed about a hundred different marketers and they found that about a quarter of them or 25% said that AI tool had already described their brand inaccurately. A quarter, that's a lot. And they said 14 or so odd percent of the same respondents said that an inaccuracy cost them a deal or cost them a customer relationship. Now, I do want to be like honest 100 respondents, that's not a very big data set, right? But setting that aside, just looking at the data, about one in four brands, at least from that survey, are dealing with an AI system giving their prospects and their customers just incorrect information, right? And there's no recourse for in the US yet. ⁓ But if have listened to some prior episodes. You might recall from our news segment a couple of weeks ago that there was a German court that recently held Google liable for ⁓ claims that its AI overviews had made about two different businesses in that jurisdiction. And the court the AI answer as Google's own speech and not ⁓ a neutral or just AI-generated search result, right? So there is a reason to expect that. A lot of these platforms will even right now, but might eventually even more so become, you know, more cautious about making negative claims about businesses and brands because of that liability consideration. But you know, that potential doesn't really help if we're dealing with a wrong answer right now, right? Like an AI mention that is there. It doesn't just sit passively inside of ⁓ chat window. It is being Delivered to your potential customers or your active customers. And it is an influential mechanism to whether people will ultimately choose to do business with you or choose to do business with somebody else. Right. Remember, if you can remember from a prior conversation or a prior episode, we had talked about this. We had talked about how a company, an AI tracking company Profound, had looked at this directly. And they had found that after an LLM ⁓ mentions brand, and this can be positive, this could be negative, but visits to that brand's site jump somewhere between one and a half and two a half times what you would normally expect over the following week from an individual user. That's a huge increase, right? So the point of that is if a mention is negative, right? ⁓ it does not promote your brand, it might deter people from visiting your website. It probably I would say it probably or most likely does deter someone from visiting your website to learn a little bit more about your business and what you offer, right? But if let's say that let's say it's wrong, right? So not if the mention is negative. Let's just say the mention is incorrect. It says you do something that you don't, or doesn't say something that you do, right? just say also, maybe let's just say it shares incorrect pricing ⁓ or it's outdated service, right? People are still going to click through to your website and then they're gonna land on your site. And your product pages and your reviews are probably going to give them a different story than the one that they got from the LLM. Now, at that point, what is a buyer to do? Are they going to trust your website or are they going to trust the AI system? Right. They're telling two different competing stories about your company. And at this point, you either are going to lose them outright, or your sales team, if they do come through, are going to spend the first call or two probably correcting an incorrect or wrong assumption instead of focusing on driving the buyer forward and the value of your services. Now, there's also some evidence that consumers are taking AI recommendations today with a bigger grain of salt that they have in the past. So there was a small PR firm that we saw a survey that they published recently. The company's called Idea Grove, and they surveyed some customers and found that only around like 15% ⁓ of respondents said that they fully trust an AI brand recommendation. And just 2% ⁓ said that would buy from an unknown brand that AI recommends them. Okay. So close to half of their respondents also said that the next move, this is important. They said that the their next move for half of respondents was to go to Google, to click on Google and to check out the brand on their own. So you know, people are saying that they don't trust AI answers, but they are still being influenced by them. So what I mean by that is the AI mentions are the start of ⁓ many buyers' So things like your reviews and your search rankings and your third party coverage ⁓ you know how long you've been around either corroborates ⁓ AI says about you, ⁓ it does not corroborate what it says about you. And that's the problem we need to solve. Okay, so before we dig into what to actually do about the negative brand mentions, I also think it's really worth reviewing what we know about how AI actually works because ⁓ the corrective that we are going to need to take really do depend on ⁓ where that comes from in the first place. So every AI system has two different kinds of memory. Okay. The first one is what it learns during its training. And sometimes this is called the corpus. Okay. That's everything that the model has ever absorbed before you ever typed a question. And the information is baked into the model itself. Okay. Each model's corpus also gets refreshed, and they typically get refreshed on their own schedule. So, what that means is there's two platforms or two LLMs that can be running on training data that can be months apart in age. Researchers call this parametric memory because it's stored in the model's. parameters, which is the settings on the inside, like its internal settings that shape all the answers that it gives. The second kind of memory called retrieval. And this is when the model ⁓ goes and searches web and pulls in specific documents, ⁓ usually at moment that you ask it, and then it builds an answer from what it finds. Now, this is called retrieval augmented generation or R-A-G or RAG. Okay, this is the architecture of nearly every AI system we use today and is built on everything. This is a very simplified explanation I'm giving, by the way, but I will link in the show notes to a previous episode where we dug deep into this and we talked about how it all works. ⁓ ⁓ back to it. How much a model relies on training data versus retrieval can vary widely between platforms. So there have been companies that have tested this, right? Profound did a test, and they saw with web search turned on as a setting for both options. Claude for one example and ChatGPT for the other. Claude sticks to its corpus most of the time. Okay. and it only searches live to answer about a third of the questions. Then ChatGPT, on the other hand, it was almost the complete opposite. It searches live more than 95% of the time, which that a ChatGPT answer about your brand is almost always going to be citation driven. So Point being is if an AI response is relying on training data, the answer it gave is going to be based on the information baked into the model itself. But then if it relied on retrieval, right, that means that the model pulled in specific pages from a website to formulate its answer. Because a corpus problem and a citation problem ⁓ different, two different problems, it's going to require a very different solution and approach to resolve each of these. So let's look at some data because we can get more specific about these differences. And I think it's super interesting. So BrightEdge did a study or a kind of a research study. They did a comparison where they had asked Google's AI overviews and then ChatGPT the same question about a brand. ⁓ And the two tools named different brands about 73% of the time. And then on top of that, Google's AI overviews surfaced a negative brand sentiment rate at a noticeably higher rate than ChatGPT. And then roughly 2.5% of brand mentions in the AI overviews were negative versus just under 2% for ChatGPT. So these are really small numbers, I get it, but they also illustrate, I think, a very consistent gap between these platforms. And also more importantly, ChatGPT concentrates its negative sentiment specifically at the point where a buyer is deciding whether to purchase. And we know this because there was nearly a fifth of its negative mentions landing in the consideration to purchase phase. So it's not only whether a engine is criticizing you, it's whether the buyer's journey in that criticism, in that buyer's journey the criticism ultimately shows up. So and also now, like where does that information, you might ask about your brand, that negative information come from, right? Most of what AI says about you has actually never been written by you. There a study done that found that almost 85% of brand mentions trace back to third-party pages, ⁓ not a own website. And brands are about six and a half times more likely to get cited through someone else's page than through their own content. Most of what AI says about you has actually never been written by you. There a study done that found that almost 85% of brand mentions trace back to third-party pages, ⁓ not a own website. And brands are about six and a half times more likely to get cited through someone else's page than through their own content. There there was another analysis done on a data set of over 30 million cited sources and found that Reddit was by far the single most cited domain and AI answers. And then that was followed by YouTube. And then it was followed by LinkedIn, right? If you break it down by individual LLMs, ChatGPT itself was leaning a lot more towards Wikipedia and through Reddit and also using editorial sites like Forbes. Google was leaning way more towards Facebook and Yelp. And then Perplexity had which is big on B2B questions, was Reddit, LinkedIn, and G2. So the point being is when we say that we have most of the sources online being used to talk about your brand, not being on your own website, these types of sites, LinkedIn, Reddit, G2, Wikipedia, these are some of, not all of, but some of the primary influencers being used to surface and communicate the brand reputation that you have to users that are on these platforms. So even with Google's own lineup, too, I want to say, like there's a separate report that we saw that found Gemini ⁓ is about a quarter of its citations from government. ⁓ And they're looking at academic sources and institutional sources, but that's Gemini. But then Google's AI Overviews was leaning on UGC content or user-generated content, right? This is something like I think it was something like 17 or 18% that was being used on that. So it's the same company, right? But this same company ⁓ two different AI systems having very different sourcing habits. So you put those together, right, with the the versus the retrieval split that we were talking about earlier on how AI systems present an answer, right? And the raw material that is feeding both halves of the LLM mechanism is mostly happening outside of your website. So if you have a reputation issue, right, your AI reputation issue is not going to be solved by just publishing more pages on your website. It's just not because it's not being used in the first place to determine what your reputation is. It's going to come down to trying to manage your presence around the pages that you ultimately do not control. Okay, so with all that being said, let's actually talk about how we can start to monitor our AI reputation itself. So ⁓ you can start by very simply checking whatever tool you already use to track AI sentiment or your visibility. So if you don't have one yet, you go back to episode two. that's where I talk through the differences between a couple different platforms. I think we talk about Peec, Scrunch, and Profound. Now, if your score already is looking strong, great, awesome. Keep monitoring it at that platform level because we know that you know AI systems will continue to use different systems or different pull from different resources to give answers, but that's great. If you don't already have a monitoring tool, you can still do this. it's not as programmatic, but it's possible. Just open up ⁓ an incognito with your LLM and ask a question about your brand. Try to phrase it like in the way that an actual person would ask it. And you can do it across, you know, a couple of different platforms, ChatGPT, you can do it in Gemini, you can do it in ⁓ can do it in normal Google search and say what AI overviews come back with, right? Then once you get that answer back, if it's positive, great. But if it's not, right? Let's just say if it's not what you want it to be, or let's just say it's maybe it's negative or it's neutral, or somehow in the process you might discover that it's misrepresenting what you offer, right? That happens ⁓ more frequently I think we give it credit for. So if that happens, right, just we start by diagnosing what the source is. Okay. So if you are like, for example, if you're checking Claude, right? I would say that specific channel. Don't look or LLM, don't look for a specific citation because the Claude, you know, differently than ChatGPT doesn't like list all of its citations on the right, right? ⁓ first thing I would say is look at Claude and say, did it, did it do a web search? Did it trigger a web search at all in the in the chat session, or did it just pull straight from its knowledge base, right? It's gets corpus of information, right? You can expand, typically you can expand the thinking block accordion that Claude creates to see if it ran any searches. And if there's no search activity, it means the reputation that you have sits inside Claude's corpus and it's something that the model does know. And it's not something that's being pulled from another source, so that's Claude. Let's just say that you've run your prompt on ChatGPT now. And it's giving you some citations for how ⁓ got to its answer. And allows you for the first time to say, okay, great. I asked it a question about my brand. It came back negative. Now let's go see what information was influencing ChatGPT ⁓ and the way it gave that information to me, right? So we can go and check the citation list, right? ⁓ Now that will allow us to get an idea of where negative information is being sent to or being used by LLMs about your brand. Now, I would also want to say very clearly, we cannot right now, nobody can stop or force an AI model to stop citing a page that has negative information about our brand. But we can do some things to help fix the process. From the most basic level, we can reach out to, or you can reach out to the websites directly that have that negative information. And you can ask them to remove or incorrect negative information. you can figure out how you want to go about that relationship with that site. If they respond back to you, I'm not saying that you're gonna get a particularly high rate but of response, but if you do get a rate of response, maybe there's some sort of economic agreement that you can come into place with that person or that website. What I would say though is if you are in any way tempted to instead of reaching out to these companies to make an impact by maybe creating a new piece of content on your website and you think that you can ⁓ displace negative citation, I just want to remind you that LLMs are much less likely to source information from your site when the question is about your reputation, right? They are going to treat third party sources as a more reliable and a more unbiased point of view versus your marketing copy. Okay. So point being is we can't stop an LLM from sourcing ⁓ citation from a site that has negative information about us, but we do potentially have the ability to influence the information that is on that site itself. ⁓ we don't recommend, and I think that it would be very unlikely for you to be able to publish a piece of information that would displace a negative citation from another website with a positive citation from your own website or your own marketing copy. So that's if we find citations. Now, if you do not uncover any citations, I would say you can treat that as a strong signal that your reputation issue is not related to citations, but it's more corpus driven. But I would want you to keep in mind that that's not absolute proof that the negative sentiment is going to come from the corpus. So there are some interfaces that retrieve live and just simply don't show it. So I would say double check by asking the model. You know, you can just follow up and say, hey, how did you put together that answer? What sources did you use to get that information? And it can retrieve and give you that information directly back to you. But if it holds up as corpus driven, right? The honest and uncomfortable fact is that is there's no fast fix. I'm sorry. There's just no fast fix. it's going to take time, especially because corpus refreshes happen sometimes as long as once a quarter, right? Which means that it's going to take upwards of 90 days to start to reinfluence corpus. But you can start like getting to work now, right? When's the best time to plant a tree, right? We can start today, and we can do a couple different things. So first and foremost, I would say is we need to start seeding. If you have a negative brand reputation in search and AI search that you're trying to solve, we need to seed the surfaces the engine you're diagnosing typically pulls from. So what I mean by that is if you're seeing a negative brand sentiment on ChatGPT, there's a lot of influence that we can make in Wikipedia, in Reddit, and editorial coverage and Wikidata, right? If it's Google's AI overviews, that could be a little bit more about UGC or user-generated content. You can think Facebook or you can think YouTube and Perplexity, like we talked about, that could be. and this applies to a lot of them, but it could be Reddit, it could be LinkedIn, it could be G2. So what we want to be trying to do in this situation is seeding correctly, not manipulating. ⁓ We want to be seeding positive sentiment in places where AI search engines are pulling from, right? Now, what can also tip the scale here is something that we've talked about before, which is called co-occurrence. And this is the idea that a model is going to form its impression from you from how often it sees your brand sitting next to certain ideas and how much it trusts, it trusts the places where those ideas are being talked about and where it's happening. So to influence your brand sentiment, you want the positive associations to be showing up more often and in more credible places than ⁓ whatever sources are the negative ⁓ sentiment. What I mean by that is we need to find better sources that are more trusted by AI search platforms to have positive sentiment content produced and published about our brand, which gives us the best potential to ⁓ trusted citations that is currently being used by the LLM because it has no other option or better option to choose from. So this is what I would call a genuine third-party content placement and link building strategy. So we don't know for sure how often these models refresh their training data. Some publish their information, some don't. So I can't give a general specific timeline on how long it might take to see the impact of this type of work. But if I were, you, if it was me, right, I'd probably plan for at least a multi quarter effort. To monitor like whether this work is improving your AI reputation, you need to make sure that you also have platform-specific AI sentiment tracking setup. So if you don't have a tool, this is something that I would definitely recommend that you get set up on. If you work with a search agency, like if you work with Victorious, we get all this stuff set up and managed for you so you don't have to worry about it, right? But you also might want to also set up on top of the sentiment tracking some reputational prompts in the tracking tool and maybe watch those over time as well. And these prompts should probably match how buyers in your space ask ⁓ AI about your services. So ⁓ can create specific prompts that are going to reflect, you know, or encourage potential neutral or incorrect or negative information that the AI systems have about your brand to see if you can surface that. Right. So, for example, too, if you end up in your diagnosis turning up a citation that calls you like overpriced, for example, right, that's ⁓ a reputational prompt that you can use. You can say something like, ⁓ is know, your brand and the you know, is your brand worth the cost? Right. ⁓ If it's a stale information prompt, something like, you know, what does brand ⁓ offer? Which would be, you know, phrased similarly to how somebody who was unfamiliar with your brand would would ask it, be a great way to pull out that information from the AI system. So setting up this this type prompt tracking is going to be really helpful for you monitoring a lot those sentiment shifts. Okay, so think back to the first number that we talked about when we just kicked off today's episode, which was that one in four brands have already had an AI system get something wrong about them. Okay. If there's one thing that I want you to take away from today's episode, it's going to be that every answer about your brand that happens in AI systems is going to be coming from somewhere specific. And once you can see ⁓ Where that specific place is, you've got the first step to figuring out how to fix it. So a model that is citing a page is pointing you straight to the source, you can go and get that page updated. You could reach out to that page. Or if it's on your website, you can edit the page yourself, right? A model that's working from its training though has nothing for you to edit. And that work is going to be a lot slower. You have to be able to shift the balance of what it sees about you across the web. And so at that The next time that it goes across the internet and it updates its training data, it has that new information about you. So if you're running into a situation where your brand reputation is not where you want it to be. Okay. Let's just like if it is negative, it is it, if it is neutral, or if you've got an issue where the LLMs are just not accurately portraying what you do and what you offer. Maybe it's a pricing issue or a service issue. Here's what I would do. Just to kick off the process, right? I mean, I would do this this week. Spend like 20 minutes. Just run a few prompts built into the surface of these models and figure out how AI systems are describing your brand. And if it is positive, stop. You're good. Don't, we don't need to be proactive about this at the moment, right? There's so much more to focus on. It's good to know where you stand. But if you are in a positive point, Go focus on something else. Focus on increasing your citation rate. Focus on increasing your mention rate and thereby getting to the most important metric in AI search. Focus on increasing your share of voice. Now, ⁓ it is negative, if your brand sentiment is negative, your next move is to figure out where that answer is coming from and what sources it's using, and then get to work on fixing that. So we've talked about that in this episode with reaching out to third-party sites, a third-party content and link building strategy and focusing on improving the process for corpus retrieval when the next time each LLM goes about and refreshes its training data to have more co-occurrence information out there to help influence it positively. So that's today's episode. If you found this episode helpful, and I you did, you can share it with a friend. But if you're listening right now and you are not subscribed, please, I'd encourage you to subscribe wherever you are listening or watching. And always think about leaving us a review if you like our content. It helps us make sure that more marketers get more information from us. So that is today's episode of The Search Signal. I'm Michael Transon and I will see you again for the next episode next week.