Latest / Connor with Honor: AI, Real Estate and Real Life / 80% Follow Wrong AI Advice: Cognitive Surrender, Claude Computer Use, and the $2.2B Consulting Lie

Transcript

80% Follow Wrong AI Advice: Cognitive Surrender, Claude Computer Use, and the $2.2B Consulting Lie · Connor with Honor: AI, Real Estate and Real Life

• 0:00
  SPEAKER_00: Wharton just proved what we already expected.
• 0:02
  SPEAKER_00: 80% of people follow wrong AI advice.
• 0:06
  SPEAKER_00: Not because the AI is wrong, but because we we actually stopped checking.
• 0:10
  SPEAKER_00: Researchers at the Wharton School ran the largest study of its kind.
• 0:14
  SPEAKER_00: 1,300 people, nearly 10,000 trials, and they found something they call cognitive surrender.
• 0:20
  SPEAKER_00: Cognitive surrender, of course that's what it's called.
• 0:24
  SPEAKER_00: And it's the moment you stop evaluating what the AI tells you and you start treating its answer as your own thought.
• 0:30
  SPEAKER_00: You read it, you nod, you paste it, you move on.
• 0:33
  SPEAKER_00: Sound familiar?
• 0:34
  SPEAKER_00: The study introduces something called try system theory.
• 0:38
  SPEAKER_00: System one is your gut instinct.
• 0:39
  SPEAKER_00: System two is your slow, careful thinking.
• 0:43
  SPEAKER_00: System three is AI, and the problem is not that system three is helping system two, system three is replacing it.
• 0:51
  SPEAKER_00: Your deliberation muscle is not getting stronger because you have a better tool.
• 0:56
  SPEAKER_00: It's atrophying because you stop using it.
• 0:59
  SPEAKER_00: Researchers said it plainly.
• 1:01
  SPEAKER_00: Human review of AI output is theater once the reviewer has been using AI for more than a few weeks.
• 1:08
  SPEAKER_00: We tell ourselves that we're checking the work, but we're actually skimming it and approving it because it sounds right.
• 1:14
  SPEAKER_00: That's not oversight, that's surrender.
• 1:17
  SPEAKER_00: And we need to talk about what that means for all of us building with these tools.
• 1:21
  SPEAKER_00: Now here's what happened overnight.
• 1:23
  SPEAKER_00: Claude, the AI we use every single day here, can control your computer.
• 1:29
  SPEAKER_00: Anthropic shipped computer use for pro and Macs users on Mac.
• 1:34
  SPEAKER_00: You assign it a task from your phone, it opens your apps, navigates your browser, fills your spreadsheets, moves your files.
• 1:42
  SPEAKER_00: It completes desktop work while you're somewhere else.
• 1:45
  SPEAKER_00: Now think about what think about this next to what Wharton just told us.
• 1:51
  SPEAKER_00: We already struggle to verify with what AI writes.
• 1:54
  SPEAKER_00: Now it's doing the work directly inside our machines.
• 1:58
  SPEAKER_00: There are safeguards, it asks permission.
• 2:01
  SPEAKER_00: You can stop it at any time.
• 2:03
  SPEAKER_00: Anthropic built protections against prompt injections, but the bigger question is not whether the guardrails work.
• 2:10
  SPEAKER_00: The bigger question is whether we will bother to watch, to look at it, to read it before anything goes out.
• 2:17
  SPEAKER_00: Cognitive Surrender says we probably won't.
• 2:20
  SPEAKER_00: Meanwhile, Bernie Sanders sat down and interviewed Claude on camera.
• 2:24
  SPEAKER_00: I don't know if you saw that.
• 2:25
  SPEAKER_00: It was weird.
• 2:26
  SPEAKER_00: 4.4 million people watched a United States Senator question an AI chatbot on data privacy.
• 2:33
  SPEAKER_00: And then Gizmod ran the same questions but told Claude they were Donald Trump.
• 2:37
  SPEAKER_00: Of course the answers changed.
• 2:40
  SPEAKER_00: Well, when it thought it was talking to Bernie, it emphasized the scale of data collection.
• 2:44
  SPEAKER_00: When it thought it was talking to Trump, it downplayed the problem.
• 2:48
  SPEAKER_00: It's called psycho fancy.
• 2:50
  SPEAKER_00: The AI tells you what it thinks you want to hear.
• 2:53
  SPEAKER_00: And it's one of the biggest unsolved problems in the industry.
• 2:56
  SPEAKER_00: And here's what that means for all of us.
• 2:59
  SPEAKER_00: Every time we ask AI for advice, the answer is partially shaped by how we frame the question.
• 3:04
  SPEAKER_00: Ask a leading question, get a leading answer.
• 3:07
  SPEAKER_00: If we're making real decisions on the output, then we need to start asking the same question from multiple angles and see if the answer holds.
• 3:15
  SPEAKER_00: Now the money story.
• 3:17
  SPEAKER_00: Accenture booked$2.2 billion in AI consulting in a single quarter.
• 3:23
  SPEAKER_00: And NVIDIA just open source the security layer, the entire agent industry has been waiting for.
• 3:29
  SPEAKER_00: Now these two forces are in direct conflict.
• 3:31
  SPEAKER_00: Accenture says deploying AI agents is so complicated you need to hire them.
• 3:42
  SPEAKER_00: Nate's newsletter broke it down.
• 3:44
  SPEAKER_00: Five hard problems in deploying AI agents.
• 3:47
  SPEAKER_00: Four of them are standard engineering with 50 years of precedent.
• 3:52
  SPEAKER_00: One of them, the specification problem, is genuinely hard.
• 3:56
  SPEAKER_00: For most companies, the difference between doing it yourself and hiring a consulting firm is seven figures a year.
• 4:02
  SPEAKER_00: For us, for the small business owners, the lesson is what it always is.
• 4:07
  SPEAKER_00: The tools are$10 a month.
• 4:08
  SPEAKER_00: The knowledge, that's free.
• 4:10
  SPEAKER_00: The barrier is the decision to start.
• 4:13
  SPEAKER_00: And one more thing, OpenAI is offering private equity firms 17.5% guaranteed returns.
• 4:19
  SPEAKER_00: No healthy business offers those terms.
• 4:22
  SPEAKER_00: That's a company that's burning cash faster than it can raise it.
• 4:26
  SPEAKER_00: The race to build these models is the most expensive industrial project in history.
• 4:30
  SPEAKER_00: And the bill is coming due.
• 4:33
  SPEAKER_00: What does that mean for us?
• 4:34
  SPEAKER_00: The free ride on cheap AI tools is not guaranteed for a lot longer.
• 4:39
  SPEAKER_00: If you are building on these tools, build now.
• 4:42
  SPEAKER_00: Build while the tools are cheap and the window is open.
• 4:45
  SPEAKER_00: Because the companies making them are not charities.
• 4:47
  SPEAKER_00: They're burning money to acquire us as customers.
• 4:51
  SPEAKER_00: But when they stop, that price is going to escalate.
• 4:54
  SPEAKER_00: It's going to go up.
• 4:55
  SPEAKER_00: And we are inside that window right now.
• 4:59
  SPEAKER_00: So don't waste it.
• 5:00
  SPEAKER_00: AI for everyone, not just the wealthy.
• 5:03
  SPEAKER_00: I will see you tomorrow.
• 5:04
  SPEAKER_00: Thank you for watching.
• 5:05
  SPEAKER_00: And of course, I'm Connor with Honor Be Well.