Latest / Safe As Podcast / E66: Is AI BS'ing you?

Transcript

E66: Is AI BS'ing you? · Safe As Podcast

• 0:00
  What's the difference between a liar and a chat bot?
• 0:02
  A liar has to know the truth to hide it.
• 0:05
  A chat bot, it doesn't know the truth exists.
• 0:08
  Today we dive into why researchers say we need to stop
• 0:12
  using the term hallucination and start calling AI what it really
• 0:16
  is, a bull crapping machine, which I'll call BS.
• 0:20
  Good day everyone, welcome to Safe as I'm Doctor Ben
• 0:23
  Hutchinson. Today's study is from Hicks,
• 0:25
  Humphreys and Slater, titled Chat GDP is BS Publishing Ethics
• 0:32
  and Information Technology. They argue that we need to
• 0:34
  fundamentally change how we talk about the errors made by AI.
• 0:39
  For several years now, when programmes like Chat GDP present
• 0:42
  false information as fact, the tech industry and the public
• 0:46
  have labelled these instances as hallucinations.
• 0:49
  However, this paper argues that hallucinations is a inept and
• 0:54
  misleading metaphor. Instead, the author suggests we
• 0:57
  should turn to philosophy, specifically the work of Harry
• 1:01
  Frankfurt, and recognise that Chachiti P is not hallucinating,
• 1:05
  it's BS ING. To understand this claim, we
• 1:08
  need to be clear about what BS means, in a technical
• 1:12
  philosophical sense. Of course.
• 1:14
  Frankfurt famously argues that both liars and honest people are
• 1:18
  oriented towards the truth. The honest person wants to
• 1:21
  convey it, and the liar wants to conceal it.
• 1:25
  Crucially, lying requires an intention to deceive by
• 1:29
  asserting something the speaker believes to be false.
• 1:32
  First, let me say that I'm a big fan of agentive models, machine
• 1:35
  learning, and large language models, and these authors aren't
• 1:38
  disparaging them at all. So moving on.
• 1:41
  Rather than intentionally lying, the BS are, by contrast, has no
• 1:45
  such concern. And this distinction matters.
• 1:48
  When large language models produce false information, it's
• 1:51
  not lying. It has no beliefs, no
• 1:54
  understanding and no intention to deceive.
• 1:57
  As the authors put it, because these programmes can't
• 2:00
  themselves be concerned with truth, and because they're
• 2:04
  designed to produce text that looks truthful without any
• 2:08
  actual concern for the truth, it seems appropriate to call their
• 2:11
  outputs BS. The paper then explains why this
• 2:15
  is not just a semantic whibble. Large language models operate by
• 2:18
  estimating the statistical likelihood of the next word in a
• 2:22
  sequence based on massive quantities of training data.
• 2:25
  They're not trying to represent the world or track what is true.
• 2:29
  Their goal is to generate text that looks normal, fluent, and
• 2:32
  human like, so there's no special failure mode.
• 2:35
  When a large language model produces A falsehood, the very
• 2:38
  same process produces output that happened to be true and
• 2:42
  outputs that are completely fabricated.
• 2:44
  So when a lawyer used a large language model and received a
• 2:47
  legal brief filled with bogus judicial decisions, the AI
• 2:52
  wasn't experienced a non standard perceptual experience
• 2:55
  or some sort of memory lapse. It was simply doing what it
• 2:58
  always does, producing A plausible sounding string of
• 3:01
  text. The authors distinguish between
• 3:05
  2 forms of this behaviour, soft and hard BS.
• 3:09
  Soft BS refers to the utterances produced with a total
• 3:13
  indifference to the truth. At a minimum, they argue, LLMS
• 3:18
  are soft BSS. It doesn't intend to convey
• 3:21
  truths, and it's indifferent to the truth value of its
• 3:24
  utterances. Even when the output sounds
• 3:27
  confident or authoritative. The aim is not accuracy but
• 3:30
  plausibility. Hard BS is more controversial.
• 3:33
  It involves misleading the audience about the nature of the
• 3:36
  enterprise itself. The paper argues that if a large
• 3:40
  language model's design goal is to imitate human speech, then it
• 3:44
  may qualify as a hard BS as well.
• 3:47
  In that case, it's attempting to deceive the audience.
• 3:49
  Specifically, it's trying to seem like something that has an
• 3:53
  agenda when many cases it doesn't.
• 3:56
  Why does the term hallucination matter so much?
• 3:59
  The authors argue it's dangerous for several reasons.
• 4:02
  First, it anthropomorphizes the system, meaning attributing
• 4:07
  human characteristics, encouraging people to think the
• 4:10
  model perceives the world, and merely misperceives it from time
• 4:14
  to time. But as I put very bluntly, these
• 4:17
  large language models don't perceive, so they surely don't
• 4:20
  misperceive. Second, the hallucination
• 4:23
  framing suggests that false outputs are exceptional glitches
• 4:26
  that can be fixed with better inputs, more data, or tighter
• 4:30
  constraints. This misunderstands the system
• 4:33
  entirely because true and false outputs arise from the same
• 4:37
  underlying mechanism. Improving accuracy doesn't turn
• 4:41
  an LLM into a knowledge bearing system.
• 4:43
  At best, it produces more convincing BS.
• 4:47
  As the authors stress, these models aren't designed to
• 4:51
  transmit information per SE or represent the world at all, but
• 4:56
  to provide a normal seeming response to a prompt.
• 4:59
  Next, describing errors as hallucinations allows creators
• 5:04
  and deployers to deflect blame onto the model itself.
• 5:07
  If the AI is hallucinating, then the failure sounds like an
• 5:11
  internal defect rather than a predictable outcome of how the
• 5:14
  system was designed and marketed.
• 5:17
  The authors are careful not to replace 1 misleading metaphor
• 5:20
  with another. They reject alternatives like
• 5:23
  confabulation for the same reason.
• 5:26
  This language still suggests that something exceptional or
• 5:29
  human like is happening when the model gets things wrong.
• 5:33
  In reality, nothing special is occurring at all.
• 5:36
  This matters because fluent language systematically inflates
• 5:39
  trust. Large language models generate
• 5:42
  high confidence and polished prose or writing in an
• 5:45
  authoritative tone, which encourages people to believe the
• 5:49
  AI outputs have more credibility than they deserve.
• 5:53
  This creates what the authors see as a serious epistemic risk.
• 5:56
  Epistemic risk like the danger that will stop caring if
• 6:00
  something is actually true as long as it sounds professional
• 6:03
  and confident. The paper warns that
• 6:05
  indifference to the truth is extremely dangerous because a
• 6:09
  mindlessly frivolous attitude that accepts this growth of BS
• 6:13
  as innocuous threatens what they call an indispensable human
• 6:18
  treasure. Our shared distinction between
• 6:21
  the true and the false. When BS becomes normalised,
• 6:25
  standards of justification and trust erode and collective
• 6:29
  decision making suffers. Again.
• 6:32
  Importantly, this isn't an argument against using large
• 6:35
  language models and AI. The authors are explicit that
• 6:38
  these systems can and are useful.
• 6:40
  The danger lies in misunderstanding what they are
• 6:43
  and treating the outputs as if they were grounded in knowledge
• 6:46
  or understanding of the world. Ultimately, the paper argues
• 6:50
  that large language models should be understood as BSS, a
• 6:54
  system designed to produce utterances that look truthful
• 6:58
  while being fundamentally unconcerned with reality.
• 7:02
  And if this topic interests you and you'd like me to cover some
• 7:04
  more of the rebuttal papers for this paper, then let me know in
• 7:08
  the comments. That's it on Safe As Thanks
• 7:11
  everyone. Please remember to like and
• 7:13
  subscribe and if you'd like to support the channel then feel
• 7:19
  free to share a coffee links in the show notes.