Latest / Safe As Podcast / EP33: Is ChatGPT bullsh** you? How Large Language models aim to be convincing rather than truthful

Transcript

EP33: Is ChatGPT bullsh** you? How Large Language models aim to be convincing rather than truthful · Safe As Podcast

• 0:06
  You've luckily heard of how large language models like
• 0:09
  ChatGPT, Google, Gemini Copilot, etcetera have factual errors
• 0:15
  called hallucinations, a term that's become popular in both
• 0:19
  technical and general contexts. But what if that term is
• 0:22
  misleading? What if the hallucinations are
• 0:25
  better described as bullshit? G'day everyone, I'm Ben
• 0:29
  Hutchinson and This is Safe as a podcast dedicated to the thrifty
• 0:33
  analysis of safety, risk, and performance research.
• 0:37
  visitsafetyinsights.org for more research.
• 0:40
  Today's article is from Hicks ET al 2024 titled ChatGPT is
• 0:47
  Bullshit published in Ethics and Information Technology.
• 0:51
  The core argument is that when ChatGPT and similar large
• 0:54
  language models produce false claims, they're not lying or
• 0:58
  hallucinating, but more precisely, the activity they're
• 1:02
  engaged in is more like bullshitting in the sense
• 1:05
  explored by philosopher Harry Frankfurt.
• 1:07
  For these authors, this is a crucial distinction, as the way
• 1:11
  we describe new technology, even metaphorically, profoundly
• 1:16
  shapes how policy makers and the public understand and apply it.
• 1:20
  So why do they argue against the prevalent term hallucinations?
• 1:25
  Why not hallucinations? The term hallucination suggests
• 1:28
  that these systems are misrepresenting the world or
• 1:32
  describing what they say, which is an inept metaphor that
• 1:35
  misinforms. It anthropomorphizes large
• 1:39
  language models, attributing to them sort of a life or human
• 1:44
  like perceptual experience that they simply don't have.
• 1:47
  Large language models don't perceive, so they cannot
• 1:50
  misperceive. Crucially, this term implies
• 1:54
  that a factual error is an unusual or deviant form of the
• 1:58
  large language models of the way they process.
• 2:01
  In reality, the very same process occurs when it's outputs
• 2:05
  happen to be true. Furthermore, attributing
• 2:08
  problems to hallucinations can allow creators to blame the AI
• 2:12
  model for faulty outputs instead of taking some sort of
• 2:15
  responsibility over the behaviour of the AI.
• 2:18
  It also suggests ineffective solutions like trying to connect
• 2:22
  the AI to real rather than hallucinated sources, which
• 2:26
  according to the authors, has largely failed.
• 2:29
  If we wrongly assume it perceives things, we might try
• 2:32
  to correct its beliefs or fix its inputs, strategies that have
• 2:36
  yielded limited, if any, success.
• 2:39
  Ultimately, calling inaccuracies hallucinations can feed these
• 2:43
  overblown hype trends and lead to unnecessary consternation or
• 2:47
  misguided efforts at AI alignment among specialists.
• 2:51
  So why can't we call them lies critically?
• 2:54
  So lying requires an intention to save someone to believing a
• 2:58
  false statement. Large language models are simply
• 3:01
  not designed to accurately represent the world the way it
• 3:04
  is. They cannot be concerned with
• 3:07
  truth according to the authors, even if ChatGPT could be
• 3:10
  described as having intentions, which is a complex philosophical
• 3:15
  question, it's not designed to produce true utterances, but
• 3:19
  rather text that's indistinguishable from the text
• 3:22
  produced by humans. Therefore, it's aim is to be
• 3:25
  convincing, not accurate. So this brings us to the topic
• 3:29
  of bullshit. What exactly is bullshit in this
• 3:32
  Frankfurtian sense, and how does it apply to large language
• 3:35
  models? So, according to Frankfurt, the
• 3:38
  defining feature of bullshit is a lack of concern with truth, or
• 3:42
  an indifference to how things really are.
• 3:45
  It's not about actively trying to deceive, but rather having no
• 3:49
  regard for the truth. The authors highlight that large
• 3:52
  language models cannot themselves be concerned with
• 3:54
  truth, and because they're designed to produce texts that
• 3:58
  without any actual concern for truth, it seems appropriate to
• 4:01
  call their outputs bullshit. Their primary goal, if they have
• 4:04
  one, is to produce human like text by estimating the
• 4:08
  likelihood that a particular word will appear next.
• 4:11
  They're built on massive statistical models of text, and
• 4:15
  when these models produce text, it looks at the previous words,
• 4:19
  constructs A context, and then produces a set of likelihoods
• 4:23
  for the next word. So this process explains why
• 4:26
  large language models have a problem with the truth.
• 4:28
  According to the paper, their goal is to provide a normal
• 4:32
  seeming response to a prompt, but not to convey information
• 4:35
  that's helpful to the user. Examples include lawyers citing
• 4:39
  bogus judicial decisions and academic researchers receiving
• 4:43
  fake references. These errors can then even
• 4:46
  snowball. The problem isn't that the large
• 4:48
  language models misrepresent the world, it's that they are not
• 4:51
  designed to represent the world at all.
• 4:54
  Instead, they're designed to convey convincing lines of text.
• 4:58
  Now, this paper also distinguishes two types of
• 5:00
  bullshit. It talks about soft bullshit,
• 5:03
  which is characterised by an indifference towards the truth
• 5:06
  of the utterance, but without the intention to mislead the
• 5:09
  hearer regarding the utterer's agenda.
• 5:13
  The authors are quite certain that ChatGPT does not intend to
• 5:15
  convey truths, and so in their words, it's more of a soft
• 5:18
  bullshitter. If it has no intentions, it
• 5:21
  trivially doesn't intend to convey truths, and it's
• 5:24
  indifferent to them. If it does have intentions, it's
• 5:28
  still designed to be convincing rather than accurate, which
• 5:30
  aligns with soft bullshit. So they conclude that, at a
• 5:33
  minimum, ChatGPT is a soft bullshitter.
• 5:37
  Bullshit that is speech or text produced without concern for its
• 5:41
  truth. And it's produced without any
• 5:43
  intent to mislead the audience about the Utter's attitude
• 5:46
  towards truth. There's also hard bullshit,
• 5:50
  which they think is more of a controversial claim.
• 5:52
  Hard bullshit is produced with the intention to mislead the
• 5:55
  audience about the Utter's agenda.
• 5:57
  This involves a higher order deception about what the
• 6:00
  bullshitter is up to. But the ChatGPT is a hard
• 6:03
  bullshitter depends on whether it can be ascribed intentions.
• 6:06
  The authors suggest that there is a robust, although maybe
• 6:09
  perhaps not literal, sense in which ChatGPT does intend to
• 6:13
  deceive us about its agenda. Its goal isn't to convince us of
• 6:17
  the content and what it says, but instead to portray itself as
• 6:21
  a normal agent like us, human like qualities.
• 6:25
  They argue that Chat GPT's primary function is to imitate
• 6:28
  human speech. If this function is considered
• 6:31
  intentional, then ChatGPT is attempting to deceive the
• 6:34
  audience about its agenda by largely trying to seem like
• 6:38
  something that has an agenda when in many cases it doesn't.
• 6:41
  Ultimately, the authors conclude that regardless of whether or
• 6:44
  not programme has intentions, it clearly isn't attempted to save
• 6:48
  the hearer or reader about the nature of sort of the enterprise
• 6:52
  somewhere along the line around what the agent is, and in their
• 6:56
  view it justifies calling the output hard bullshit.
• 6:59
  So, according to paper, minimally, it churns out soft
• 7:03
  bullshit, and given certain controversial assumptions about
• 7:07
  the nature of intentional description, it produces hard
• 7:10
  bullshit. The specific texture of the
• 7:12
  bullshit is not, for these purposes, important.
• 7:15
  Either way, Cha chi PT, again their words, is a bullshitter.
• 7:20
  Next they talk about why this matters.
• 7:22
  Well, why doesn't matter. They say it because indifference
• 7:25
  to the truth is extremely dangerous.
• 7:28
  For instance, civilised life and vital institutions depend very
• 7:32
  fundamentally on respect for the distinction between the true and
• 7:36
  the false. When the authority of this
• 7:39
  distinction is undermined by the prevalence of bullshit by
• 7:42
  treating large language models as if they are concerned with
• 7:45
  truth or metaphorically suffering hallucinations, we
• 7:49
  risk accepting this bullshit and allowing this squandering of
• 7:51
  meaning. Calling these inaccuracies
• 7:53
  bullshit rather than hallucinations isn't just more
• 7:56
  accurate in their view, it's good science and technology
• 7:59
  communication in an area that really urgently needs this sort
• 8:03
  of guidance. It helps us understand that even
• 8:06
  when a large language model gets things right, it's still
• 8:09
  bullshitting as it's accuracy is incidental to its design.
• 8:13
  So in conclusion, understanding Chechi PTS output is bullshit
• 8:17
  means recognising that these models are designed to generate
• 8:20
  convincing text, not to convey truth or falsehood.
• 8:25
  This framing is vital for investors and policy makers and
• 8:28
  the public to make informed decisions about how to engage
• 8:31
  with these powerful systems. That's it on SAFE As I'm Ben
• 8:36
  Hutchinson. Please help share, rate and
• 8:38
  review and checkoutsafetyinsights.org for
• 8:42
  more research. Finally, feel free to support
• 8:45
  Safe As by shouting a coffee link in the show notes.