Latest / After the Demo / Is ChatGPT Sending You Customers?

Transcript

Is ChatGPT Sending You Customers? · After the Demo

• 0:00
  Kate: Say I run a business. How many customers did ChatGPT send me last month?
• 0:04
  John Arndt: If you ask your website analytics? About zero.
• 0:07
  Kate: And the real number?
• 0:09
  John Arndt: Honestly? Nobody can see it from where you're sitting.
• 0:12
  Kate: That's a terrible answer.
• 0:13
  John Arndt: It is. But the people an assistant sends you mostly don't click anything. They call you. Or they type your name into Google.
• 0:20
  Kate: So they show up, and my website never hears about it.
• 0:23
  John Arndt: Right. It just looks like somebody found you.
• 0:26
  Kate: So my dashboard's been lying to me.
• 0:28
  John Arndt: Not lying. Counting the wrong door.
• 0:42
  John Arndt: This is After the Demo. I'm John Arndt. I run about fifteen small software products out of Sacramento, most of them built and run with AI agents, plus an AI consulting practice.
• 0:52
  Kate: And I'm Kate.
• 0:53
  John Arndt: Each episode, one question business owners are actually asking about AI, answered from the middle of the work, and something you can do about it this week.
• 1:02
  Kate: This time it's the one you're already asking. Are these AI assistants sending people to my business or not? And how would I even know?
• 1:10
  John Arndt: By the end you'll know how to check it yourself, and what to fix first.
• 1:14
  Kate: And if he gets abstract, I stop him.
• 1:16
  John Arndt: She will. Start with who's asking, because it's more people than you think.
• 1:25
  John Arndt: So, who's asking. There's a consumer survey from BrightLocal, from February of this year. About a thousand adults in the US. They looked at where people go when they want a recommendation for a local business.
• 1:36
  Kate: And the answer is Google.
• 1:38
  John Arndt: Google's still first. Seventy-one percent. Facebook's second, at sixty.
• 1:43
  Kate: Okay. So nothing's changed.
• 1:45
  John Arndt: Third place is ChatGPT and the other AI tools. Forty-five percent.
• 1:50
  Kate: Forty-five percent of people asked a chatbot to find them a plumber?
• 1:54
  John Arndt: Used AI for local recommendations, yeah. And the year before, that number was six.
• 1:59
  Kate: Six.
• 2:00
  John Arndt: Six percent to forty-five percent. In one year.
• 2:04
  Kate: Okay, that's the kind of jump where I want to see the survey.
• 2:07
  John Arndt: Fair. It's in the show notes. About a thousand people. One survey, but that's a big move for any survey.
• 2:14
  Kate: And it's still third. Behind Google and Facebook. So people didn't stop googling.
• 2:19
  John Arndt: Most of them didn't. They added a step. Some of them ask the assistant first, then go look you up.
• 2:25
  Kate: Look you up how?
• 2:26
  John Arndt: Hold that. It's a big part of why your analytics say zero, and we're getting there.
• 2:31
  Kate: So the owner at the top of the show, asking how many customers ChatGPT sent them. That's not paranoia.
• 2:37
  John Arndt: No. That's paying attention. But the number I actually care about is in the follow-up. BrightLocal went back in March and surveyed just AI users, four hundred fifty-five of them. Ninety-seven percent said they sometimes double-check what the AI recommended against real reviews.
• 2:53
  Kate: Ninety-seven. So basically everybody.
• 2:56
  John Arndt: Basically everybody. Sometimes, to be fair. But think about what that means. The assistant hands them a shortlist. Then they go read your reviews before they pick.
• 3:05
  Kate: Hang on. Sometimes is doing a lot of work in that sentence.
• 3:09
  John Arndt: It is. Sometimes isn't always. The rest of the time, they don't check.
• 3:13
  Kate: So for those people, the list is the whole decision.
• 3:17
  John Arndt: Right. Which is why you want to be on it either way.
• 3:20
  Kate: So the robot gets you in the room.
• 3:21
  John Arndt: And your reviews close it. The AI builds the list. It doesn't make the sale.
• 3:26
  Kate: Which is kind of a relief. The stuff a business already did still counts.
• 3:31
  John Arndt: It still counts. But you have to make the list first. If you're not on it, that person never gets to your reviews.
• 3:37
  Kate: And if the AI and the reviews don't agree? The assistant says you're great, and the reviews say you never call anybody back.
• 3:43
  John Arndt: Then the assistant just sent a customer over to read why they shouldn't hire you. The reviews win that one.
• 3:48
  Kate: Okay, let me push on this. Say I run a dental practice, and say my patients are seventy. They are not asking a robot for a dentist.
• 3:58
  John Arndt: You're probably right about them.
• 3:59
  Kate: Probably?
• 4:01
  John Arndt: Mostly right. That same March report broke it out by age. People thirty to forty-four, sixty-four percent have asked an AI for a business recommendation. Sixty and over, twenty-four percent.
• 4:12
  Kate: So I win.
• 4:14
  John Arndt: About your patient, yes. Now, who picked the dentist?
• 4:17
  Kate: The patient.
• 4:18
  John Arndt: Sometimes. Take senior living. Say the mom's seventy-eight. Often she's not the one researching places. Her daughter is, and say the daughter's forty-two.
• 4:27
  Kate: Which puts her in the age group where it's sixty-four percent.
• 4:30
  John Arndt: Right in it. Same thing with a business that sells to other businesses. Often the owner signs, but the office manager is the one who went looking.
• 4:37
  Kate: Come on. The office manager is asking ChatGPT for a vendor?
• 4:42
  John Arndt: If she's thirty-five, she's in the same age group as the daughter, so odds are she's asked an AI to recommend a business. For work specifically, the survey doesn't say.
• 4:51
  Kate: So the customer and the person doing the research aren't always the same person.
• 4:55
  John Arndt: A lot of the time they're not. And the researcher is the one who'd be asking the AI.
• 4:59
  Kate: Okay. Split decision. My seventy-year-olds mostly aren't asking. Their kids are.
• 5:04
  John Arndt: I'll take a split.
• 5:06
  Kate: So what does somebody actually do with that this week?
• 5:08
  John Arndt: Grab a piece of paper. Write down who actually researches your business before anybody calls you. The buyer, their kid, their office manager. Be honest about it, because it's often not who you picture.
• 5:20
  Kate: Why paper? Can't I just think about it?
• 5:22
  John Arndt: In your head it's always the buyer. On paper you notice it's the daughter, or the office manager. Put a rough age next to each one, too.
• 5:30
  Kate: Because of the sixty-and-over thing.
• 5:32
  John Arndt: Yeah. If the person on your list is forty, don't plan around the seventy-year-old.
• 5:36
  Kate: And then?
• 5:36
  John Arndt: Then assume that person asked an AI first. Before they saw your website. Before they read a single review.
• 5:43
  Kate: That's it? A list?
• 5:45
  John Arndt: That's it. Once you know who's asking, you know what question they typed. And that's the question you'll want to check later.
• 5:51
  Kate: Fine. So if they're asking, why does my website say nobody came?
• 6:01
  John Arndt: Because your website can only count the people who come through your website. And a lot of them don't.
• 6:06
  Kate: You said that at the top. They call, or they type my name in. But somebody has to be tracking this.
• 6:12
  John Arndt: People are trying. At the end of September, Cody Schneider posted that most companies have no idea AI search is already sending them traffic. And he built a dashboard out of Google Analytics and Search Console that rolls ChatGPT, Perplexity and Gemini traffic into one number.
• 6:29
  Kate: That sounds useful.
• 6:30
  John Arndt: It is. I think he's right that most companies don't know, and putting it in one number is a good idea. I'd build that.
• 6:37
  Kate: I hear a but.
• 6:38
  John Arndt: But it can only count the people who clicked. So it's a floor. It isn't the scoreboard.
• 6:44
  Kate: How low a floor? How many people skip the click?
• 6:47
  John Arndt: The best number I have is about Google, not ChatGPT. It's a Pew Research study from July twenty twenty-five. They looked at real Google searches from nine hundred adults in the US, back in March of that year.
• 6:59
  Kate: Okay. And?
• 7:01
  John Arndt: When there was no AI summary at the top, people clicked a regular result on fifteen percent of visits. When the summary showed up, eight percent.
• 7:09
  Kate: That's roughly half.
• 7:10
  John Arndt: Roughly. And the links inside the summary itself? People clicked one of those on one percent of visits.
• 7:16
  Kate: One percent.
• 7:18
  John Arndt: One.
• 7:18
  Kate: The answer's sitting right there, and hardly anybody clicks through.
• 7:23
  John Arndt: On Google, with the summary showing, a lot fewer people clicked. I don't have that study for ChatGPT, so I won't pretend it's the same number.
• 7:31
  Kate: Fine. The ones who don't click. Where do they go?
• 7:34
  John Arndt: I keep a research page on this, and the link's in the show notes. For a local business, the people who get your name from an AI answer mostly call you, search your name, or find you in maps.
• 7:44
  Kate: The wrong door.
• 7:45
  John Arndt: The wrong door. None of that comes in labeled ChatGPT. The call is just a call. The name search looks like somebody who already knew you.
• 7:53
  Kate: And the people from before, the ones who double-check the reviews.
• 7:56
  John Arndt: Probably typing your name in to find the reviews. So even the careful ones look like they found you on their own.
• 8:03
  Kate: Okay, but Google has to know something. It's their summary.
• 8:06
  John Arndt: There's something there now. Google's Search Console has a report for impressions from AI Overviews and AI Mode. Search Engine Journal wrote it up in August.
• 8:15
  Kate: Impressions meaning what, in English?
• 8:17
  John Arndt: How often your pages showed up in those AI answers. You can split it by page, by country, by device and by date.
• 8:24
  Kate: That's the number, then.
• 8:25
  John Arndt: It's a number. There are no clicks in it. There are no searches, so you can't see what anybody typed. Just how often you showed up.
• 8:32
  Kate: It tells me I was at the party. Not who I talked to.
• 8:36
  John Arndt: Pretty much. Good to have. Still not customers.
• 8:40
  Kate: Okay, I'm going to be the annoying one. Say I run a business and my accountant wants a number. They called is not a number.
• 8:47
  John Arndt: You're right. It isn't.
• 8:49
  Kate: Thank you.
• 8:49
  John Arndt: And keep the dashboards. Cody's version, the Google report, whatever you've got. They're real floors, and floors are worth watching.
• 8:57
  Kate: But.
• 8:58
  John Arndt: But the number your accountant wants is at the phone and the intake form. You get it by asking one question.
• 9:03
  Kate: How did you hear about us.
• 9:05
  John Arndt: Which you might already ask. It's one of the checks on that research page, too. The change is the list of answers. Add one that says, an AI assistant, like ChatGPT. On the phone script, and on the intake form.
• 9:17
  Kate: Why spell out ChatGPT? Just put AI.
• 9:20
  John Arndt: Because the person on the phone might not think of it as AI. They might just think of it as ChatGPT. Give them the word they'd use.
• 9:27
  Kate: And then what, I count it?
• 9:29
  John Arndt: Count it for thirty days. Every time somebody picks that answer, it goes on the tally.
• 9:33
  Kate: So the number is a tally sheet.
• 9:35
  John Arndt: Yes.
• 9:36
  Kate: That's a very cheap answer to a very expensive-sounding problem.
• 9:40
  John Arndt: It's also the first number in this whole conversation that's actually yours. The surveys are about everybody. The tally is about your business.
• 9:49
  Kate: Does whoever answers the phone need to know why they're asking?
• 9:51
  John Arndt: Tell them. Otherwise somebody says ChatGPT told me about you, it sounds like small talk, and it never gets written down.
• 9:58
  Kate: And thirty days from now, I've got a real count.
• 10:01
  John Arndt: A real count. Maybe a low one, since people don't always remember how they found you. But it's yours, and it beats a zero from a dashboard watching the wrong door.
• 10:09
  Kate: Okay, but I'd rather know before thirty days. Can I just ask it myself?
• 10:18
  John Arndt: Yes. Today. It takes about ten minutes, and you don't need anything you don't already have.
• 10:24
  Kate: Okay. Walk me through it like I've never opened one of these things.
• 10:28
  John Arndt: Start with the list from earlier. Who researches your business, and what would they type? Say it's the daughter, looking for a plumber for her mom.
• 10:35
  Kate: She types something like, who's a good plumber near me.
• 10:39
  John Arndt: That exact question. Type it into ChatGPT, then into a couple of other assistants.
• 10:44
  Kate: Do I put my business name in it? Because I'd be tempted.
• 10:47
  John Arndt: Don't. That's the one rule. If you name yourself, of course it talks about you. You want to know if it brings you up on its own.
• 10:54
  Kate: And is that what you do, just fancier?
• 10:57
  John Arndt: Pretty much. I ask the same fixed questions on a schedule, never with the business's name in them, and I save the full answers so I can compare them later.
• 11:05
  Kate: Why the same questions every time? I'd want to try a bunch.
• 11:08
  John Arndt: Because if you change the question, you can't tell whether the answer moved or you did.
• 11:12
  Kate: Fair. And what you see is what my customer sees?
• 11:16
  John Arndt: No, and I want to be plain about this. I ask the engines directly, the same ones behind the apps. The apps can answer differently. And no tool can show you exactly what one person saw on their phone.
• 11:27
  Kate: Same engine, then. Not the same screen.
• 11:30
  John Arndt: Right. So what does it actually find? I keep a public leaderboard for plumbers in Sacramento. Last run was September seventeenth. Two assistants, ChatGPT and Gemini. Five questions a customer would ask.
• 11:42
  Kate: So ten answers.
• 11:44
  John Arndt: Ten answers. Eight different plumbers got named across them. And one of them showed up in five.
• 11:49
  Kate: One plumber in half the answers. And the other seven get the leftovers.
• 11:54
  John Arndt: Less often, yeah.
• 11:55
  Kate: Hold on, though. Five questions, ten answers? That's your data?
• 11:59
  John Arndt: That's the data.
• 12:01
  Kate: Because that's a pretty small pile to hang a plumber's reputation on. Say I'm plumber number eight. I'm not losing sleep over ten answers.
• 12:09
  John Arndt: You're right. It's small. I wouldn't call it a verdict. It's a thermometer.
• 12:13
  Kate: Meaning I take it again next month. One reading is today. A few readings tell me which way it's moving.
• 12:19
  John Arndt: That's it.
• 12:20
  Kate: So I win this one.
• 12:22
  John Arndt: You win this one. It's a small sample, you say so out loud, and you measure again.
• 12:27
  Kate: Okay. Got a bigger pile?
• 12:28
  John Arndt: I do. On September twenty-sixth I ran a study on credit unions. Four assistants, ChatGPT, Gemini, Claude and Perplexity. Eight metros around the US. Four hundred eighty answers.
• 12:41
  Kate: Okay, that's a real pile.
• 12:42
  John Arndt: In each metro, the biggest local credit union has about half the members. Fifty-one percent, on average.
• 12:49
  Kate: Then it should come up about half the time.
• 12:51
  John Arndt: You'd think. It got about a third of the local credit union mentions. Thirty-four percent.
• 12:56
  Kate: Being the biggest doesn't even get you your share. That seems like the headline.
• 13:00
  John Arndt: That's the small problem.
• 13:02
  Kate: That's the small one?
• 13:03
  John Arndt: One hundred fourteen of those four hundred eighty answers used a name the credit union doesn't use anymore. Mostly after a merger.
• 13:10
  Kate: Wait, that's almost one in four.
• 13:13
  John Arndt: Nearly one in four. Then eligibility, meaning who's allowed to join. There were fifty-four statements about that, each one checked against the credit union's own membership page. Eighteen didn't match.
• 13:25
  Kate: One in three. On who can even join.
• 13:28
  John Arndt: One in three didn't match the credit union's own page.
• 13:31
  Kate: Take the daughter again. Say she's finding a credit union for her mom, and the answer hands her an old name. Maybe she's one of the people who double-checks.
• 13:40
  John Arndt: Maybe. But now she's double-checking a name the credit union stopped using. Invisible is the small problem. Wrong is the big one.
• 13:48
  Kate: Okay. I ask it myself this week. Then what, I do this by hand every month for the rest of my life?
• 13:55
  John Arndt: You can. That's why I built Mentioned On: it asks the same questions on a schedule, without your name in them, and keeps the answers, so a change is something you can check. There's a free scan, no signup, and it can't promise you a spot in the answers.
• 14:08
  Kate: Good. I'd trust it less if it did. Okay, the move this week. Three assistants, the question my customer would ask, no name in it.
• 14:16
  John Arndt: Save the answers with the date. And circle anything that's wrong.
• 14:19
  Kate: Wrong like what?
• 14:21
  John Arndt: An old name, or the wrong area you serve. Anything you'd correct if a customer said it to you on the phone.
• 14:26
  Kate: And if it doesn't mention me at all?
• 14:28
  John Arndt: Write that down too, with the date. That's your first reading on the thermometer.
• 14:32
  Kate: So say I find something wrong. Do I need some new AI trick to fix it?
• 14:41
  John Arndt: Depends who you ask. At the start of October, Chris Long posted, in all caps, that people should make an AI instructions page now, because it can influence what Google's AI Overviews say when somebody searches your brand name.
• 14:54
  Kate: In all caps.
• 14:56
  John Arndt: All caps.
• 14:57
  Kate: Does it work?
• 14:58
  John Arndt: I don't know. I haven't tested it, and I'm not telling anybody to go build one. It's the hack of the week.
• 15:04
  Kate: There's always a hack of the week. And I saw the other side of that one. Stacy Muur posted around the same time that the algorithms behind ChatGPT, Claude and AI Overviews change faster than you can adapt. A trick works, people abuse it, it gets changed. So she'd just focus on SEO, which helps the AI answers anyway.
• 15:24
  John Arndt: That's a fair read of what she said.
• 15:26
  Kate: And I'm with Stacy. Say I run a business, and I've got maybe two hours a week for marketing. I'm not spending them on a trick that might get patched before I finish building it.
• 15:36
  John Arndt: Let me argue the other side for a minute.
• 15:38
  Kate: Of course you will.
• 15:40
  John Arndt: One SEO, Gagan Ghotra, says he's seeing Google test AI Overviews with no source links at all, on some best-of searches, especially local ones. Some searches, not all. That's one person's observation, and I can't confirm it.
• 15:54
  Kate: Okay. And if he's right, then on those searches, even that one percent from earlier has nothing to click.
• 16:00
  John Arndt: Right. On those searches there's nothing to click and nothing to count. If that spreads, being named in the answer is the whole game.
• 16:08
  Kate: Fine. That's a reason to keep checking what it says. I'll give you checking. What I want to know is what makes it say my name, and say it right.
• 16:16
  John Arndt: Fair. So I looked at my own data. I keep a page on where AI answers get their links, last updated October third. Thirty-four thousand citations, from about nine hundred fifty websites, scanned since June.
• 16:29
  Kate: That's a lot of links. So where do they point? Some secret AI directory I've never heard of?
• 16:35
  John Arndt: About eighty-six percent pointed at a business's own website. Or a product's own website.
• 16:40
  Kate: Their own website. The thing they already have.
• 16:44
  John Arndt: And directories and review sites, all of them combined, came to about ten percent.
• 16:49
  Kate: Hang on. Before I start gloating, what's the catch?
• 16:52
  John Arndt: The catch is those are businesses that got scanned through my own service, so it leans toward local services. Your industry might not look like that.
• 17:00
  Kate: Noted. Is there anybody else's data, or just yours?
• 17:03
  John Arndt: BrightLocal did a study in August. About thirteen hundred business locations, across Google's AI Overviews, AI Mode and ChatGPT. The single biggest cited source was the Google Business Profile. Twenty-eight and a half percent of all the citations, so a bit over a quarter.
• 17:20
  Kate: The Google profile. The one with your hours and your photos on it.
• 17:24
  John Arndt: That one. Different study, different sample, but both point at stuff you already control. And BrightLocal's own line, roughly, was that there isn't one source you can optimize for and call it done.
• 17:36
  Kate: Okay. Let me say it back to you. The new AI strategy is fix your website and fix your Google profile.
• 17:42
  John Arndt: Yeah.
• 17:43
  Kate: That's the old strategy.
• 17:45
  John Arndt: Yeah. You win that one. The work's the same. What changed is that you can check it.
• 17:49
  Kate: And Stacy wins too.
• 17:51
  John Arndt: And Stacy wins too. Credit where it's due.
• 17:53
  Kate: The stuff I circled earlier. The wrong answers. Say I'm the credit union with the old name. Do I start on my own site?
• 18:01
  John Arndt: Own site first, because that's where most of those links pointed. Make sure it says the current name, plainly. Then make your Google profile say the same thing.
• 18:10
  Kate: And they have to match because people double-check. You said that at the start. If my website says one name and my Google profile says another, the person checking just got two answers.
• 18:20
  John Arndt: And one of them is wrong.
• 18:22
  Kate: Okay. The list for this week. My homepage says, in plain words, what I do, where I do it, and who it's for.
• 18:30
  John Arndt: Plain words. If your neighbor couldn't repeat it back, rewrite it.
• 18:34
  Kate: So not, innovative solutions for a changing world.
• 18:38
  John Arndt: Not that. Then fix the wrong thing you circled, on your own site. Then make your Google profile match it.
• 18:45
  Kate: And next month I take the temperature again. Same questions, same assistants, and I see if the thermometer moved.
• 18:51
  John Arndt: That's the whole loop. None of it's a trick, so there's nothing to get patched.
• 18:55
  Kate: And the AI instructions page?
• 18:57
  John Arndt: I haven't tested it, so I'm not telling you to build one. If I test it and it works, I'll say so.
• 19:02
  Kate: Fine. Give me the one thing. Tomorrow morning, coffee in hand.
• 19:07
  John Arndt: Open ChatGPT and two other assistants. Ask each one the question your customer would ask, without your name in it.
• 19:13
  Kate: Save the answers, with today's date.
• 19:16
  John Arndt: Then set a calendar reminder to ask the same questions next month. That's the one thing.
• 19:21
  Kate: That's it? That's barely a cup of coffee.
• 19:24
  John Arndt: That's it. And if you've got two more minutes, add an AI assistant, like ChatGPT, to your how did you hear about us list. Start the tally.
• 19:32
  Kate: And send this to whoever answers your phones. They're the ones who'll hear ChatGPT told me about you first.
• 19:38
  John Arndt: Links are in the show notes. The studies, the posts we talked about, and my research pages.
• 19:43
  Kate: Back to the top, then. How many customers did ChatGPT send me last month?
• 19:47
  John Arndt: Last month, nobody can tell you. A month from now, you'll have a tally and two sets of answers.
• 19:52
  Kate: Which beats a zero from a dashboard.
• 19:55
  John Arndt: A zero from the wrong door. This has been After the Demo. I'm John Arndt.
• 19:59
  Kate: And I'm Kate. Go watch the right door.