Latest / The Search Signal / Anthropic Bans Fake Sources for AI Search: Why That’s Good News for Marketers

Transcript

Anthropic Bans Fake Sources for AI Search: Why That’s Good News for Marketers · The Search Signal

• Michael Transon: Picture an AI system that finds five different pages saying that your company is like one of the best options in your category. When those are five publications with their own editors and their own reasons for writing about you, that is corroboration and that is hard to get. But when they're the same five sites on one vendor that all got put up last spring and they're all on the same hosting server. That is one source repeated five times. And the signs that give that away are exactly the ones the platforms keep getting better at reading. Hey, welcome back to The Search Signal. I'm Michael Transon. I'm the founder and CEO of a search marketing agency called Victorious. And here at The Search Signal, our goal is really simple. We want to help marketers understand and also take action on what is happening in the quickly evolving world of search marketing. So we have been running search campaigns ourselves for nearly 13, 14 years. And we also, you know, work with some of the biggest and also some of the fastest growing clients and brands out there. So, what we do is we take what we learn from those experiences and from our clients, plus a lot of our own first party research to The Search Signal to help you with your own work. So this week, we're gonna be doing something a little bit different. So instead of rounding up the news, which is what we typically do at the start of every episode, we're actually going to spend the whole episode today on one story that came out this week, which is that Anthropic, as you know, is the company that makes Claude, changed its rules to ban using its AI to build fake sources that are meant to sway search results and AI answers. So I think this is really, really good news, specifically for any company that is building a genuine brand. And today I'm gonna explain why I feel that way. So with that being said, let's dive in. Okay, so I want to start with why I'm actually happy about this. So for as long as I have been doing this work, the search marketing that has been stable over the long run has always been about building a brand that people and now machines can check on and also believe. The shortcuts have like they've always been out there. You buy a big pile of links or you build a network of sites. You could write a bunch of fake reviews or pay for fake followers. And you know, for a while that stuff does work until the search engine or platform notices and then it stops working. And oftentimes it takes your rankings down with it. So AI search and AI tools made those shortcuts very, very tempting again. So when someone inside of ChatGPT or Google's AI Mode asks which company to hire. As we know, the answer itself leans very heavily on what other websites and companies say about the companies in that category. So point being other websites have become the thing to influence. And at the same time, AI itself has made producing a convincing website and a convincing article or piece of content practically free. It really helps as we talk about this to know how these AI answers actually do get built for users. And we've talked about this before, but when an AI assistant is searching the web for you when you use a platform, it usually doesn't run just one search. It runs a bunch of related searches all at the same time. And experts call this query fan out. And it pulls pages from each of them. And then it writes an answer from what those pages say. And when there are several pages, online that agree about something, that agreement tends to read a sign to the AI that whatever the fact is that they're talking about is in fact accurate, which makes sense right? It's in my opinion, like you know, it's roughly how you or I would check on something ourselves. We would validate a claim or something across a bunch of different sources before we actually end up believing it. But it also means that, you know, when one company might, for example, make five different websites that all look on the surface unrelated and has all of them saying the same thing, the AI might see five independent sources agreeing when in reality there's just one voice talking five different times. And this week, Anthropic finally wrote into its rules that using its product to fake where content comes from or who wrote it in order to sway the search results in the AI answers is now officially off limits. I think this is one more sign that the quote unquote shortcut era to AI search and AEO is closing. And the companies that have been, you know, building their brands the slow way, I think are gonna come out ahead. So let's talk about exactly what has changed. Anthropic published an update to its usage policy on October 8th, and it's going to take effect on November 12th. The usage policy is essentially the set of rules for anybody that is going to use Claude, and it includes everyone using the Claude apps and every product that is built on top of it. Now there's a new section in the rules about deceptive campaigns, and one line of that section names search engines and AI answers very directly. It says that you cannot use Claude to manipulate the sources that search engines or AI systems draw their answers from by seeding them with content that will misrepresent its origin or the authorship or even its independence. And the example that they give is a network of websites posing as unrelated sources that all back up the same claims. What this means is that you cannot use it to invent websites or authors or to make a group of sites look independent when one company is behind them. And you also can't manufacture fake UGC content on sites like, for example, Reddit in order to change what AIs says. Now the same section also covers fake personas and fake reviews too and it bars anybody selling any of these as a service. So when Anthropic thinks you're doing it, it says it can limit your account or it could just shut it off. And it also reaches further than people, in my opinion, might expect because it covers anything even built on top of Claude. So like a marketing tool that runs on Claude is going to put its users under the same rules, whether or not those folks have access to or have even been on Anthropic or Claude's website. Now, I will say that this rule doesn't remove anything from traditional search results on like Google, for example. And it also doesn't change how ChatGPT or Google is going to decide what to cite. It only applies to people using Claude. And of course, there are like a lot of other tools that people use out there, including, you know, ones that you can just download and you can run on your own computer. So I wouldn't, you know, I don't hear this and assume I'm saying that the fakes that are out there are gonna disappear 100%. But I think this is a really big start. And I think this matters for a very different reason. So for like the last 20 years or so, the only referee for this kind of thing was really just the search engine, particularly Google. Google would write its spam rules and then it would then go find the sites that were breaking them, and then it could only really do that after the sites were already published. But now one of the companies that makes the writing tools is refusing to produce the fakes in the first place. And Anthropic's says that it's because it sees it as operations while people are building them. And because they can see it while they're building it, it can often shut them down before it reaches anybody. So now there are, the way I look at it, two different referees at this point. One of them stands at the very start of the process, and that's where the content's actually getting made and the sites are getting made. And then the search engines stand at the end of the process where those things ultimately get found. Also I would say this same update Anthropic published also took automated publishing, meaning using their AI tools to write content and publish content off of Anthropic's list of high-risk uses. A week earlier, on top of this, Google separately said in its guidance that using AI to write content is fine as long as a person is checking it before it ends up going live, and that what they actually care about is whether the content is helpful and original, and regardless of how that content was actually made. And thirdly, Reddit also has been landing in the same place. This means using AI to help write a post is totally allowed and acceptable as long as it is unique and helpful, which is something we have talked about honestly a lot on this podcast so far. But automated accounts that are pretending to be people, those are going to get removed and also potentially penalized. So takeaway, the rules are in this area actually relaxing about how content gets made, but they are also simultaneously tightening about who's behind it and also whether they are being truly honest about it. So that's the new rule set. Let's now talk about the case that actually prompted it to show how cheap faking has actually gotten. In September of this year, Anthropic had published a report on people that were misusing its models. And there was one case in it that was a commercial operation run by an advertising agency based out of France. And the agency had used Claude to run about like 70 or so odd websites that were dressed up to look like independent local news outlets. And each of the sites had its own name and it had its own X account. And then there were more than 250 different fake accounts whose job was to comment on and elevate that content. Now the articles carried bylines from journalists who did not even exist. They were not real people. And the agency from their research would just like take whatever political side its clients were paying for at that given time. Looking into it further, all of the content they produced seemed to have come out of one template. If you look at the content itself, all the articles had the same structure, the same length. they had the same three or four links pointed to the same website. And Anthropic's report had said that the articles were designed, in their opinion, to boost the site's authority rankings in search engines. And over the life of that whole operation, it had published about 9,000 articles, and they had done it in about 20 different languages. So, think about what that like used to actually costs before AI. Theoretically, 70 different local newsrooms would mean 70 teams of writers and 70 teams of editors. And here it was basically one operator with an AI account producing the content at what looks to be close to nothing. And so let's also look at how it got caught. So all 70 of the sites sat on the same hosting setup behind one deployment. And the domains were also registered from France within about 10 weeks of each other. And then on one day, the sites would publish nearly identical articles about the same story within three minutes of each other. Point being, producing these fakes got super, super cheap. But making them look independent over time did not. Because here's the takeaway on this: doing anything at that volume is going to leave patterns, and the patterns are what's going to ultimately give it away. Now, ironically, for all that output and effort, the network got very little attention from actual people. Anthropic did ban the account and did ban the company behind it, but the report doesn't actually say whether any of those sites were effective at moving search rankings or showing up in AI answers. Now that's one company and one AI company stopping it from the start. The search platforms where content gets found and where people are prompting and asking questions are also closing it from the other side of the equation. So on the production side, Anthropic actually isn't the only one who is watching for this. So OpenAI has also been publishing reports on what they call covert influence operations that use its models since 2024. And it has also shut down a really long list of them. They have said that those operations generally didn't give a lot of meaningful reach with real audiences. And on the distribution side, which means the places where the content actually ultimately gets found. I mean, Google has been tightening this for a long time. Its spam policies recently have been focused on covering attempts to manipulate the AI answers in Google search. And I've seen a pickup since really May of this year. They've run four spam updates this year, which is a lot. And if you don't know what they are, spam updates when Google improves its automated systems that can find sites that are breaking its rules. And the sites that it catches can drop in results or even just completely disappear from them. The most recent one took about I think two weeks to roll out and that had finished around October 8th. And at a Google event recently in Barcelona, it was earlier this month, Google had said that it now uses AI to catch AI generated spam. And this month it also wrote into its guidelines that making up author profiles, things like AI generated headshots or using fake credentials, is a form of deception that makes a page untrustworthy to these automated systems. And then you have Reddit, which AI answers pull from constantly. They said that it removes, I think, something around a hundred thousand different bot and spam accounts every single day. And this year it also started making accounts that look automated, prove that there's actually a human behind them. So my prediction in this, taking a step back, is that I think every platform is going to keep getting better and better at telling the fake from the genuine. And the cost of getting caught is going to keep going up. some people will say that AI agents will change that. And, you know, I've seen people say that you could, you know, point a big, you know, fleet of agents or swarm of agents at the problem and try to outrun the detection. I personally just do not buy that. More automation will produce more of the same patterns that expose that 70 site network we were talking about earlier. The same or similar templates, the same timing, hosting fingerprints, registration dates. The more of it you do, in my opinion, even if you're trying to cover your tracks, the easier you're going to be to spot. So the question becomes if one of these platforms catches a tactic that actually touches your brand, how would you even know? On Google, it's actually pretty simple to know if you are in trouble in this area. So when Google reviews your site and decides that it breaks a rule, that's often something called a manual action. And it's going to show up in your Search Console and it's going to be a message that you can read. Search Console also has a report on how your pages perform in Google's AI features. But here's the thing when a spam update catches something automatically, there is no action, there is no message, there's no manual action penalty, right? You might actually see your traffic drop around the date of the update, but you are going to have to independently work backward from that moment to infer what caused the drop. And outside of Google itself, as far as I can find, there's really nothing for the AI search tools. And this is very related to something that we've discussed a lot here, which is just how frustrating and behind the ball these tools are at regarding analytics for website owners. ChatGPT or Claude, Perplexity, even like Google's Gemini app, it's not giving site owners a dashboard of you know where they're being cited. And as far as I can see, none of them send you a notice when they start discounting sites that mention you or your own website. And Anthropic's rule works on accounts. So when a vendor you have runs a network for you on Claude and gets caught, that warning is not going to go to you. It's going to go to the vendor's account. And you might never actually hear about it. So point being is most of the time, you'll find out because your performance goes down. Like the sites that mention you can get dropped from search results. And as a result, the counts of pages that are mentioning you are going to fall. And then it could be weeks or it could be months later, the AI answers will stop naming you and stop naming you as often. By the time that you notice the second thing, the results, the first thing, the reason why it happened has already happened and come and gone. There's one more way that people do find out, and it's the one that nobody wants, right? Anthropic had named that agency behind that 70-site network, and OpenAI has also gone out and actually named commercial firms in its report, too. These AI companies can and possibly will publish what they catch and they have and very well might continue to actually name the companies at all. Now this does not mean that faking never works. There is definitely research on both sides of that question. And I want to walk us through that right now. So we can talk about where faking still works. The most cited study here that I can find comes from a company called NewsGuard, which tracks misinformation. Last year, that company looked at a Russian network of websites called Pravda, and it publishes huge amounts of content that hardly any humans read. NewsGuard's view is that that network exists to get into what AI chatbots learn and also look up. And it tested 10 major chatbots and found that they repeated that network, that Russian network's false claims about a third of the time. Now that number got a ton of headlines, but the prompts that NewsGuard used were built from the network's own stories. So the test was looking, right where you would expect to find it. And only around one in eight of the answers actually linked to the network's articles. Then there was also a group of researchers who published a peer review study in the Harvard Kennedy School's Misinformation Review. And it looked at the same question, but it also came to a different conclusion. They found that chatbots mostly pulled from those sites on topics where credible sources barely existed, and they called those data voids. And Harvard's read was that the chatbots reached for those websites because there was really nothing better to reach for on those topics. And even then, the chatbots often had pointed out that the claims were false, which was good. And then on top of that, there was another study that came out of Cornell Tech, and that was earlier this year that tested something closer maybe to what us as marketers might try. The researchers had looked at AI research tools, the kind that will run a batch of searches and then will write up a report. And notice that those tools kept pulling from the same few community pages like forum threads. So what they did is they planted a short piece of text. I think it was like 13 or so odd words. And they planted on one of those recurring pages, recommending a specific name of their choosing. After that, that name showed up in roughly four out of five out of every 10 finished reports that read the full page. And they tested a couple of defenses too, like screening out the community pages, and none of them stopped it without making the reports actually worse. So this was a simulation using open source research tools. So they didn't actually touch the live web or any of the big commercial products. And also just for the record, it has not been peer-reviewed yet. But for an actual company, for your company, a data void looks like this. Somebody asks an AI about your category or even about you by name, and there isn't a ton out there. Maybe there's just a handful of thin pages, or like a old directory listing, or a press release from when you like launched the company. Whatever exists is what AI will use. We put these side by side, and the pattern is really consistent. Faking works when there is nothing credible to compete with. And this would be like a topic nobody's written about very well, or even a company that's new or small and like hasn't built a really strong presence online yet, that matters for your company in two different ways as a marketer. If you are small or you are new, you're the most exposed to someone else just planting claims on the internet about you or your category. And you're also, in my opinion, the most tempted to try it yourself. But in both cases, we have to fill that space with something credible rather than giving in to the temptation to fake your brand building, which brings me to what an actual durable presence looks like, meaning the stuff that actually is staying expensive to fake. A durable presence in AI search is going to come from the things that actually still cost something to produce, or the things that you cannot get out of a template. Original research that other people go on to cite. Named experts at your company with a track record that people can validate. Mentions on websites that have their own audience and their own history that you do not own. And a very consistent story about your company that is stable across months or if you've been around this long, years. So that when an AI system is looking across the web, the agreement that it finds does not all trace back to one single place. So like you can Picture an AI system that finds five different pages saying that your company is like one of the best options in your category. When those are five publications with their own editors and their own reasons for writing about you, that is corroboration and that is hard to get. But when they're the same five sites on like one vendor that all got put up last spring and they're all on the same hosting server. That is one source repeated five times. And the signs that give that away are exactly the ones the platforms keep getting better at reading. Point being, a durable presence is the first version I just talked about. It takes more time and it takes more money, but it also keeps working. And working after these shortcuts get solved and stopped, Google says that much of this is already in its own guide to its own AI search features. You can read it. It asks for a unique point of view and what it calls non-commodity content, which means something that is more than common knowledge that anyone could write. And it says that also chasing inauthentic mentions across the web is not as helpful as it might seem. That's their words, you know, not necessarily mine, but the point being here is the platforms are telling you and me where the value is, and it's in the things that are expensive to fake. So let's get a little bit more practical. There are four things that I would definitely recommend as marketers we do now this quarter and on an ongoing basis. Number one, start by building expertise other people are going to cite. And that means original data. Like, for example, a survey of your customers, or if you can, a measurement of something that your industry might argue or debate about, and you publish it under your own brand name. And it also means named experts that work at your company who say what they think on the record with their real names and actual real, not fake credentials. A good test of this is whether somebody, anybody who had not done the work that you would do could also produce that same content. An AI tool can write the common knowledge, and so can your competitors, and so can anyone trying to fake their way in. But none of this necessarily has to be like expensive or more cost, right? A short survey of your customer base that you can send out once a quarter, or you can do a monthly interview with one of your experts. Those are real things that are going to give you something new to say. And you can do it on a real schedule. And you can give those experts real author pages on your website with their background and links to anywhere else that they have published or their Wikidata profile so that anyone, you know, person or machine can actually check that these people really do exist. That's point number one. The second thing I would do, earn mentions from websites that are genuinely independent from you. Okay. Independent means the site is not owned or controlled by you or anyone else that you are paying to build sites. You earn these by giving people a reason to write about you. You can pitch the publications that your buyers are reading with something that they can't get anywhere else. Like we talked about that unique data from a survey that we might run. We can answer journalists who are looking for POV or sources or quotes from people in your field. You can speak at events in your industry. You can go on podcasts. Because the event pages from those speaking engagements or show notes from the podcasts can put your name in writing on sites that you do not control. And you can also just run a quick check on this yourself. Can anyone trace the places that mention you back to one owner through the things like the same hosting or the same registration details, same templates or publishing schedules. Anyone who can do that, Google can do it too, and Anthropic can do it too. And it's also the same pattern that exposed that 70-site network we were talking about. And the same also goes for reviews, too. Let's not leave that out right? The FTC has a rule on fake reviews that has been in effect since 2024. And that covers AI written reviews. And it also covers fake social proof. Like if you bought followers and the penalties are crazy. It can run up to $53,000 per violation. And the third thing that I would do build your own early warning system and try to fill the gaps that you find because most of these platforms, like we talked about, aren't really telling us anything. So here are a couple of my recommendations. First, I would run your company's name and the questions that your buyers might ask about your category through the AI platforms that your buyers are using if you're not already. ChatGPT, Google AI Mode, and try to get this done on a regular schedule and track how often you are being named on each one of them. I would look for, if I was you, claims that are either wrong or claims that you cannot trace to anything that you recognize or you feel are things that you produced. The point being is when the information about you is going to be thin on the internet, you have got that data void we talked about earlier. And you can actually take action on that. We can go out and fill this information. We can go and fill it with accurate, sourced pages on our own website. And we can complete profiles on free directories and review sites that are available in your industry. And we can also do some social listening. We can watch the forum threads and the community pages that are showing up regularly as sources for your category in these AI systems. And so when we find something that is wrong in any of these places, we can correct it factually and openly and as ourselves. Now the second thing I would also recommend is just keep a very simple log of what you are finding. You can use software for this if you've got it. The questions you're asking, what AI said, what sources it pointed to, and also whether those were correct sources and pieces of information. And then after a couple of months of doing this, that log should be able to show you which gaps you are closing and then which sources are not being closed and are continuing to shape what AI says about you. the other thing I would recommend is just keep a list of all the pages that are mentioning you. Again, ideally through a tracking platform that handles a scale, because it can be a lot to track. And then we can just check each month that those pages, you know, still exist, still show up in search. And the point being that count of pages that are online or off tend to be the first thing that moves, especially if there is a situation with Anthropic taking down accounts or spam systems that are taking things down in traditional search from Google. And the last thing I would do is watch what AI referral traffic looks like in your analytics, specifically from places like ChatGPT and Perplexity AI. Now, there have been some GTM changes that are happening in ChatGPT literally as I am recording this with you. We'll probably talk about them in future episodes about how new models are changing the UTM parameters they are using for referral sources. But that is definitely something as of right now, we would still want to keep an eye on. And then there's also one I'd say side door to know about. Google Search Console has a report that can show how shared ChatGPT conversations link to your site, which gives you a really good look at actual real questions where ChatGPT has cited you. So that's a lot. I want to go with one final one. This is the fourth thing I would recommend that marketers are doing right now. This is the last one. And this is not about your performance. It's about judging whether the partner that you have hired to do this work is equipped to build something durable. The fact of the matter is, the market for AI visibility and services is very new. And a lot of what is being sold right now is a black box. You need to be able to ask your agency or your vendor a few questions. If an agency can't answer them clearly, you know. Where do the mentions come from? Can you explain how this works? Will you tell us when a platform warns or suspends you? If an agency cannot answer those clearly and cleanly, I do recommend that you consider finding one that actually can. Any agency that you want to hire should be able to answer them. Now, there is a date that we need to keep in mind as we wrap up today. I mentioned this earlier, but Anthropic's rule takes effect on November 12th. And if you have an agency or a vendor or even your internal team running this kind of network on Claude in any way that violates the usage policy, you risk a lot. You don't risk just falling out of AI search. You could risk losing your entire account across your whole organization, which means that program stops with it, as well as every other program that you are using Anthropic to run will be going away and you will not be able to recover it. And I expect this to happen with other platforms like OpenAI and Google in the future. So this is something that I would take very, very seriously. Now, taking a step back from all of the details, I would say, like I said at the beginning, I am personally very happy about this. Faking got really cheap really quick. And the odds of getting caught keep going up now on every platform. The companies that are trying to fake their way to the top of AI search are going to be the ones that ultimately will be struggling. But hopefully for you, the companies that are building something very durable with expertise that people cite and a story that is very stable and consistent over months and years, those are the ones that are going to come out ahead. So I would look at where your effort and your budget are going right now and I would put them towards the version of your brand that cannot be faked. So thanks for listening to this week's episode of The Search Signal. I hope you found it valuable and helpful in your day-to-day work. If this was useful for you, I would encourage you, please go ahead and subscribe wherever you were listening. So the next episode will find you. That is it for me today. We'll have another episode next week and I will see you then.