Latest / AI Newspaper / OpenAI Will Watermark ChatGPT Text, But Only in the EU

Transcript

OpenAI Will Watermark ChatGPT Text, But Only in the EU · AI Newspaper

• 0:00
  Welcome to the AI Newspaper, your daily update
• 0:02
  on everything happening with AI. Visit AINewspaper
• 0:04
  .org to see today's stories and sign up for email
• 0:07
  alerts. So I want you to imagine a scenario for
• 0:11
  a second. You're sitting at your computer, right?
• 0:14
  Staring at a blank screen. And you finally decide
• 0:18
  to ask ChatGPT to help draft, I don't know, a
• 0:21
  really difficult email to your team. Oh, yeah.
• 0:24
  Or maybe it's like an outline for a major quarterly
• 0:26
  report. I mean, we've all been there. Exactly.
• 0:29
  So you get this great response. You copy that
• 0:32
  text. You paste it into your document. Maybe
• 0:35
  you tweak a sentence or two. And you hit send.
• 0:39
  If you do that today, is there a hidden, invisible
• 0:43
  fingerprint tracking that exact string of text
• 0:46
  back to a machine? Well, starting very soon,
• 0:49
  the answer to that is yes. But there is a massive
• 0:53
  catch. A huge catch. Right. That is only true.
• 0:56
  if you happen to live in a very specific part
• 0:58
  of the world, and honestly, somewhat hilariously,
• 1:02
  only if you don't use a thesaurus. Yeah, which
• 1:04
  is just wild to think about. Today, we are looking
• 1:07
  really closely at OpenAI's newly unveiled text
• 1:11
  watermarking system. Internally, it's called
• 1:13
  TextGrain. Right. We're going to explore how
• 1:15
  you actually weave a secret signal into plain
• 1:18
  text without anyone seeing it, and why this feature
• 1:20
  is rolling out in this deeply restricted way.
• 1:23
  Because it really is restricted. It is. And we're
• 1:26
  also going to look at a fascinating reality here.
• 1:29
  Regulatory compliance has officially arrived
• 1:31
  before the technology can actually do what the
• 1:34
  regulators want it to do. Yeah. And it's a profound
• 1:37
  moment for the industry, honestly. Mostly because
• 1:40
  it highlights the limits of what these companies
• 1:42
  can actually control. Right. I mean, for months,
• 1:45
  the conversation has been about, you know, when
• 1:48
  is OpenAI finally going to release a way to detect
• 1:50
  their own AI's writing? And now that they are,
• 1:54
  it's not this. triumphant bulletproof solution
• 1:56
  people were expecting. Not at all. No, it's a
• 1:59
  very reluctant, highly geo -fenced deployment.
• 2:03
  Yeah. So let's start right there with the geography
• 2:05
  because the rollout strategy is, well, it's bizarre
• 2:09
  if you just look at it on the surface. It really
• 2:11
  is. Over the coming weeks, OpenAI is injecting
• 2:14
  this invisible text watermark into outputs from
• 2:16
  chat GPT and codecs, but it applies strictly
• 2:20
  to users physically located in the European Union.
• 2:23
  Just the EU. Right. If you are in the United
• 2:25
  States or Japan or Brazil, your chat GPT output
• 2:29
  remains completely un -watermarked. And the motivation
• 2:31
  behind that boundary is entirely legal. It's
• 2:34
  just compliance. On August 2nd, the European
• 2:37
  Union Sweeping AI Act actually took effect. Right,
• 2:40
  that massive legislative package. Exactly. And
• 2:43
  hidden inside that are these very specific transparency
• 2:46
  rules. The law now legally mandates that generative
• 2:49
  AI providers have to make their machine -generated
• 2:52
  text identifiable. Identifiable in a machine
• 2:55
  -readable format, right? Yes. So OpenAI is doing
• 2:59
  this for European consumers just to legally operate
• 3:02
  in that market. But wait, I mean, if they have
• 3:05
  the technology built and they are turning it
• 3:07
  on for millions of people in Europe anyway, why
• 3:10
  not just flip the switch for everyone? Like if
• 3:12
  I'm building a car and I invent a revolutionary
• 3:15
  new kind of seatbelt, I might be legally required
• 3:18
  to install it in cars sold in Paris or Berlin.
• 3:20
  Sure. But I wouldn't... deliberately ripped the
• 3:23
  seatbelt out of the exact same car if I was shipping
• 3:26
  it to New York. I mean, if the goal is safety
• 3:28
  and transparency, why are we treating this like
• 3:30
  a regional DVD code? That's a good analogy, but
• 3:33
  it assumes the seatbelt is something the driver
• 3:35
  actually wants. Oh, interesting. Yeah. You have
• 3:38
  to look at the competitive pressures tearing
• 3:40
  at these companies right now. Transparency sounds
• 3:42
  great on a press release, you know? Right. It
• 3:45
  plays well for the cameras. Exactly. But users
• 3:47
  absolutely hate it when it applies to them. Take
• 3:51
  a look at what happened back in August with Anthropic.
• 3:53
  The company behind the Claude models? Right.
• 3:56
  They actually did what you are suggesting. They
• 3:59
  rolled out an invisible text watermark globally.
• 4:02
  And how did the market react to that? With immediate
• 4:06
  visceral backlash. Really? Users were furious.
• 4:10
  Just completely rejected it? Completely. The
• 4:13
  argument from the community was basically, hey.
• 4:15
  I provided the context. I wrote the complex instructions.
• 4:19
  I supplied the critical thinking. The AI was
• 4:22
  just my intern. Oh, I see. So they felt that
• 4:24
  a permanent, invisible machine watermark embedded
• 4:27
  in their work just completely discredited their
• 4:29
  human effort. Exactly. It discredited them. So
• 4:33
  is it like a vanity issue or just a fear of being
• 4:36
  caught? It's kind of a mix of both, but mostly
• 4:39
  it's a fear of false judgment. And OpenAI has
• 4:42
  been watching that fall out. very, very closely.
• 4:46
  Yeah, but they have. In fact, they had this text
• 4:49
  watermark system built way back in 2024. Wait,
• 4:53
  really? That long ago? Yeah, they kept it locked
• 4:56
  in a vault, purely out of fear that if they turned
• 4:58
  it on, users would just abandon ChatGPT. And
• 5:00
  flock to open source competitors, I guess. Right,
• 5:03
  or rivals that promised untracked text. The EU
• 5:06
  essentially had to force their hand here. They
• 5:08
  are prioritizing legal compliance in Europe,
• 5:11
  but everywhere else? The market dynamics actively
• 5:15
  discourage them from alienating their paying
• 5:17
  subscribers. Okay, that makes the API side of
• 5:20
  this rollout make a lot more sense, actually.
• 5:22
  Yeah, it does. Because for developers around
• 5:23
  the world, the people using OpenAI's technology
• 5:26
  to build their own apps, watermarking is available
• 5:28
  today, but it is strictly opt -in. It's turned
• 5:31
  off by default. Right. So OpenAI is basically
• 5:34
  washing their hands of the decision and telling
• 5:35
  developers, you decide if you want to annoy your
• 5:38
  users. Precisely. They provide the tool. but
• 5:41
  they shift all the risk of user backlash directly
• 5:43
  onto the developer. Okay, let's get into the
• 5:46
  actual engineering of this, because this is the
• 5:48
  part that feels like straight -up science fiction
• 5:50
  to me. It is pretty wild. If OpenAI has to comply
• 5:52
  with EU law, how do you actually watermark a
• 5:55
  paragraph of plain text? I mean, you can't just
• 5:57
  drop a glowing neon OpenAI logo onto an email.
• 6:01
  Right, no. And you can't hide a barcode in plain
• 6:03
  text without making it look like gibberish. No,
• 6:05
  you definitely can't. To understand how text
• 6:07
  screen works, You kind of have to understand
• 6:09
  how a large language model actually writes in
• 6:12
  the first place. Okay. It doesn't think in full
• 6:14
  thoughts or sentences. It operates entirely on
• 6:17
  next word prediction. It looks at the text you
• 6:20
  gave it, calculates probabilities, and just guesses
• 6:23
  the most mathematically logical next chunk of
• 6:25
  a word. Which they call a token, right? Right,
• 6:28
  a token. And it just does this over and over
• 6:31
  again thousands of times a minute. So it's basically
• 6:34
  the world's most advanced smartphone autocomplete.
• 6:37
  Like the cat sat on the, and the AI calculates
• 6:41
  that mat is the highest probability word to come
• 6:43
  next. Exactly. That is exactly what it is. And
• 6:46
  what the researchers from OpenAI, along with
• 6:48
  teams from UPenn and Yale, figured out how to
• 6:50
  do is intervene in that tiny microsecond of calculation.
• 6:54
  Intervene how? So the tech screen system introduces
• 6:58
  a secret cryptographic key. As the AI is lining
• 7:02
  up all the possible choices for the next word,
• 7:04
  this secret key subtly sorts and nudges those
• 7:08
  choices. Sorts them. Okay. It essentially creates
• 7:13
  an invisible green list of preferred vocabulary
• 7:16
  words for that specific moment. So out of, say,
• 7:19
  50 acceptable words, the secret key pushes the
• 7:23
  AI to pick a word from the green list instead.
• 7:25
  Right. And if it does that enough times in a
• 7:27
  row, a specific mathematical pattern... just
• 7:30
  sort of emerges in the vocabulary. You've got
• 7:32
  it. A specialized detector holding that exact
• 7:35
  same secret key can look at a finished document
• 7:38
  and say, wait, the author of this text used a
• 7:40
  green list word 30 times in a single paragraph.
• 7:43
  Which is statistically impossible. Right. The
• 7:45
  mathematical odds of a human doing that naturally
• 7:47
  are like one in a billion. So a machine wrote
• 7:50
  this. Wow. And because the watermark is baked
• 7:53
  into the actual choice of words, it survives
• 7:57
  being copied and pasted. Exactly. Like if I generate
• 8:00
  text in the chat GPT app, copy it and paste it
• 8:04
  into a blank Microsoft Word document, the watermark
• 8:07
  travels right with it. It doesn't rely on hidden
• 8:09
  file metadata that just gets stripped away. No,
• 8:12
  it is the text itself. The vocabulary pattern
• 8:15
  is the fingerprint. But hold on. If you are artificially
• 8:18
  restricting the AI's vocabulary, like if you
• 8:21
  are forcing it to pick a green list word instead
• 8:23
  of the absolute best, most accurate word it originally
• 8:26
  wanted to use, doesn't that make the AI worse?
• 8:29
  You would think so. Yeah. I mean, we pay for
• 8:31
  these tools because they are smart. If it's being
• 8:34
  forced to use specific words just to create a
• 8:36
  tracking pattern, doesn't the quality of the
• 8:38
  writing degrade? Well, that is actually the most
• 8:41
  surprising part of the engineering here. And
• 8:43
  it's a big reason why OpenAI feels comfortable
• 8:45
  rolling this out. at least in Europe. It turns
• 8:48
  out the AI doesn't get dumber. Wait, really?
• 8:52
  Really. OpenAI ran massive internal testing using
• 8:55
  their most advanced frontier model, Astra. They
• 8:59
  threw this watermarked version against some of
• 9:00
  the most punishing, rigorous benchmarks in the
• 9:03
  entire industry. Give me an example of the testing.
• 9:05
  Like, how hard are we talking? Oh, incredibly
• 9:07
  hard. They used something called GPQA Diamond.
• 9:11
  which is a test composed of questions so difficult
• 9:13
  that even human PhDs, with access to the Internet,
• 9:17
  by the way, struggle to answer them. Oh, wow.
• 9:19
  Yeah. They also tested it on deep software engineering
• 9:22
  tasks and professional health exams. And the
• 9:25
  results showed almost zero difference in the
• 9:28
  output quality between watermarked and un -watermarked
• 9:31
  text. How is that even possible? Because the
• 9:34
  models have just become so sophisticated and
• 9:36
  their vocabularies are so vast. that there are
• 9:39
  usually dozens of equally excellent ways to phrase
• 9:42
  any given thought. Ah, I see. Yeah, so if the
• 9:45
  absolute mathematically best word isn't on the
• 9:47
  green list, the second best word probably is.
• 9:50
  And a human reader literally cannot tell the
• 9:52
  difference. That's fascinating. Yeah, on that
• 9:55
  PhD -level benchmark, un -watermarked text scored
• 9:58
  94 .44%, and watermarked text scored 93 .94%.
• 10:04
  That's basically identical. It is. On a health
• 10:08
  benchmark, the watermarked version actually scored
• 10:10
  a fraction of a percent higher just by pure chance.
• 10:13
  Okay, so the AI remains just as smart, the text
• 10:16
  reads just as naturally, and it entirely satisfies
• 10:19
  the European regulators. It sounds like a total
• 10:21
  victory. It does sound like one on paper. So
• 10:25
  why is OpenAI being so defensive about this?
• 10:28
  Because the underlying mechanism is profoundly
• 10:30
  fragile. The vulnerability isn't in generating
• 10:33
  the text. The vulnerability is in the ability
• 10:36
  to reliably detect it later. Oh, OK. Let's dig
• 10:41
  into that. Remember, we are talking about really
• 10:43
  subtle statistical nudges hidden in the vocabulary.
• 10:46
  It is a mathematical chain. And the moment human
• 10:49
  hands touch that text, the chain breaks. Which
• 10:53
  brings us to the core editorial irony of the
• 10:55
  entire situation. It really does. The EU is demanding
• 10:59
  transparency. They want a definitive binary answer.
• 11:03
  Did a human write this or did an AI? But OpenAI's
• 11:07
  own internal testing reveals how easily this
• 11:10
  sophisticated text -grained system just shatters
• 11:13
  in the real world. Right. And the first major
• 11:16
  hurdle is length. Statistical patterns require
• 11:18
  a lot of data to prove they aren't just coincidences.
• 11:21
  Sure. If you want a 1 % false positive rate,
• 11:24
  meaning you only falsely accuse a human of using
• 11:27
  AI 1 % of the time, the text needs to be long.
• 11:31
  How long are we talking? Well, a short passage
• 11:33
  of about 200 tokens, which is roughly 150 words,
• 11:37
  is only detected successfully about 80 % of the
• 11:40
  time. That's not great. No, it's not. You really
• 11:43
  need a passage of at least 400 tokens or basically
• 11:46
  a solid page of text to get detection up to 95%.
• 11:49
  So if I use ChatGPT to write a quick three -sentence
• 11:53
  email to my boss, the detector might not even
• 11:55
  be able to confidently find the watermark anyway.
• 11:58
  Correct. And it gets much, much worse when the
• 12:00
  text is highly constrained. What do you mean
• 12:02
  by constrained? So if you ask the model to write
• 12:05
  lines of Python code, or solve a mathematics
• 12:07
  equation, or perform a literal translation from
• 12:11
  French to English, there simply isn't enough
• 12:14
  flexibility in the vocabulary. Because you can't
• 12:16
  creatively use synonyms in a math equation. Two
• 12:19
  plus two has to equal four. Exactly. Because
• 12:22
  there is no room to use those green list words,
• 12:25
  the watermark just fails to embed at all. But
• 12:28
  the critical failure point, the stat that absolutely
• 12:30
  blew my mind when looking at this data, is what
• 12:33
  happens when a user simply edits the text. Oh
• 12:36
  yeah, this is the big one. We talked about how
• 12:38
  the pattern is the fingerprint. If a user takes
• 12:41
  a full page of watermarked chat GPT text and
• 12:44
  replaces just 10 % of the words with synonyms,
• 12:47
  the detector's accuracy drops from about 92 %
• 12:50
  down to 66%. It's a staggering drop. It is a
• 12:53
  huge drop. Just from 10%. Right. And if the user
• 12:56
  replaces 25 % of the words, detection collapses
• 13:00
  to a mere 17%. Which is, I mean, it's basically
• 13:03
  useless at that point. We really need to pause
• 13:06
  and absorb that. If a student generates an essay,
• 13:08
  spends 10 minutes reading through it, pulls out
• 13:11
  a thesaurus and changes every fourth word, the
• 13:14
  detection accuracy plummets to 17%. Yeah, 17%.
• 13:18
  That makes this sound less like a secure digital
• 13:21
  fingerprint and more like a polite suggestion.
• 13:25
  It essentially means the watermark is completely
• 13:27
  defeated by basic human editing. It completely
• 13:30
  defeats the purpose of the tracking. When you
• 13:32
  manually swap happy for joyful, you aren't just
• 13:36
  changing a word. You are blowing a hole in that
• 13:39
  invisible mathematical chain the AI built. Right.
• 13:42
  Because the sequence is gone. Exactly. The detector
• 13:45
  scans the text, hits that new word, loses the
• 13:48
  sequence of the secret key, and the signal just
• 13:50
  dissolves. So any professional who just lightly
• 13:53
  rewrites the AI's output in their own voice is
• 13:56
  going to wash the watermark right out of the
• 13:58
  text. Very easily. This perfectly captures the
• 14:01
  tension shaping the entire AI industry right
• 14:03
  now. Strict legal mandates from governments are
• 14:06
  colliding with the raw technological reality
• 14:08
  of how language actually works. Yeah, that's
• 14:10
  the real story here. OpenAI is deploying the
• 14:13
  system to check an EU compliance box so they
• 14:15
  don't get sued. But the technology cannot reliably
• 14:18
  tell anyone what a machine wrote the second a
• 14:21
  human gets involved in the drafting process.
• 14:23
  Which leads directly to the most controversial
• 14:25
  part of this entire rollout. Because the watermark
• 14:28
  is so incredibly fragile, and because light editing
• 14:31
  creates these massive blind spots, OpenAI is
• 14:34
  keeping the actual detector tool under absolute
• 14:37
  lock and key. Seriously? Yeah. The general public
• 14:41
  is not allowed to use it. This feels like a massive
• 14:44
  bait and switch. We've been talking this whole
• 14:46
  time about a detector that scans the text, but
• 14:48
  I can't actually go use it. Nope. If I'm a high
• 14:51
  school teacher and I suspect a student used AI
• 14:54
  for their final paper, I can't just paste that
• 14:56
  document into an OpenAI website and check for
• 14:59
  the text grain watermark? No, you cannot. Access
• 15:02
  to the detector is restricted exclusively to
• 15:05
  approved researchers and specific expert organizations.
• 15:08
  And OpenAI is granting that access manually.
• 15:11
  on a case -by -case basis. So, let me just think
• 15:15
  about this. It's like the government inventing
• 15:17
  a brilliant new way to detect counterfeit money.
• 15:19
  They print these special machines that can spot
• 15:22
  the fake bills. Okay, I follow you. But then
• 15:25
  they announce, actually, the public isn't allowed
• 15:27
  to use these machines. A slightly crumpled $20
• 15:30
  bill might confuse the scanner, so only the central
• 15:32
  bank is allowed to look at it. The analogy holds
• 15:34
  up. It really does. Especially when you look
• 15:37
  at the risks of getting it wrong. Right, the
• 15:40
  false positives. The stakes of text detection
• 15:42
  are incredibly high. I mean, imagine you are
• 15:45
  that high school teacher or a university professor
• 15:47
  or an editor at a newspaper. If an automated
• 15:50
  tool falsely accuses a student of cheating or
• 15:53
  a journalist of faking an article with AI, careers
• 15:56
  are destroyed. Yeah, it ruins lives. Because
• 15:59
  text is so easily manipulated, a false positive
• 16:02
  carries immense societal risk. And even for the
• 16:06
  approved researchers who do get access, the tool
• 16:09
  barely tells them anything, right? The detector
• 16:11
  only gives a yes or no answer, like, did I find
• 16:14
  an OpenAI watermark? Right, just yes or no. It
• 16:18
  won't tell the researcher who the user was, it
• 16:21
  won't reveal the prompt that generated the text,
• 16:23
  and it won't show the broader conversation. OpenAI
• 16:26
  is going out of its way to establish legal deniability
• 16:29
  here. They explicitly warn these researchers
• 16:32
  that a missing watermark proves absolutely nothing.
• 16:35
  Because it could just be edited. Right. The text
• 16:38
  could be too short. It could be heavily edited
• 16:40
  by a human. It could be translated. Or, honestly,
• 16:43
  most obviously, it could have been generated
• 16:45
  by a completely different AI company like Google
• 16:47
  or Meta that doesn't use the text screen key.
• 16:50
  Of course. And they also clarify that the watermark
• 16:52
  doesn't establish legal copyright ownership.
• 16:55
  It doesn't measure how much human creativity
• 16:57
  was actually involved in the final product. And
• 17:00
  it certainly doesn't verify if the content itself
• 17:02
  is factually accurate. Exactly. The fragility
• 17:05
  of text really stands out when you contrast it
• 17:08
  with how OpenAI handles images and audio. Right,
• 17:10
  because that's completely different. Oh, totally.
• 17:13
  For images generated by their dolly models or
• 17:16
  audio generated by their voice engines, OpenAI
• 17:18
  already embeds invisible SynthID watermarks alongside
• 17:22
  C2PA metadata, which is essentially a permanent
• 17:25
  digital nutrition label attached to the file.
• 17:27
  And the public can check those. Right now, anyone
• 17:30
  can go to an OpenAI verification web tool. upload
• 17:33
  an image and the tool will publicly tell you
• 17:35
  if an open ai system generated it yes they can
• 17:38
  so why can we have public tools for images but
• 17:42
  not text because of the fundamental difference
• 17:44
  in the type of data images and audio are continuous
• 17:48
  data okay continuous data yeah an image is made
• 17:52
  of millions of pixels a system like synth id
• 17:54
  can subtly shift the color of thousands of pixels
• 17:57
  by one fraction of a percent A human eye cannot
• 18:00
  perceive the color shift, but a computer can
• 18:02
  easily read it. Oh, that makes sense. And more
• 18:04
  importantly, if you crop the image or compress
• 18:07
  it or put a filter over it, enough of those manipulated
• 18:10
  pixels survive that the watermark remains durable.
• 18:13
  Ah, I see. But text doesn't work like that at
• 18:15
  all. Exactly. Text is discrete data. You cannot
• 18:19
  alter the word apple by 1%. Right. It either
• 18:23
  is the word apple or it is a completely different
• 18:25
  word. The moment you change the word, you fundamentally
• 18:28
  destroy the underlying structure of the data
• 18:30
  the watermark relies on. Wow. So releasing a
• 18:34
  public text detector that constantly generates
• 18:36
  false negatives because a user changed a few
• 18:38
  verbs, or worse, false positives because a human
• 18:41
  naturally happens to write using common vocabulary
• 18:44
  sequences, it would just create absolute chaos.
• 18:47
  So they hide the tool behind closed doors. They
• 18:49
  satisfy the European regulators on paper. They
• 18:52
  avoid the public relations nightmare of an inaccurate
• 18:54
  detector ruining people's lives. And the average
• 18:57
  user goes about their day completely unaware
• 18:59
  that any of this is happening. It is the ultimate
• 19:02
  compromise. It is an engineering marvel that
• 19:04
  solves a legal problem while completely failing
• 19:07
  to solve the practical problem. It really is.
• 19:10
  So what does this all mean for us, for you, the
• 19:13
  listener, for the people reading emails, grading
• 19:15
  essays, and digesting reports every single day?
• 19:18
  We are watching the largest AI company in the
• 19:20
  world roll out a tracking system entirely because
• 19:23
  they are legally forced to, while simultaneously
• 19:26
  proving with their own internal data that the
• 19:28
  system barely works if you do basic editing.
• 19:31
  It's pretty wild. It leaves us with a really
• 19:33
  strange thought about where we are heading. As
• 19:36
  we sprint into a future saturated with AI generated
• 19:39
  content, if the very companies building these
• 19:42
  massive models cannot reliably detect their own
• 19:44
  outputs after a human uses a thesaurus, maybe
• 19:48
  we are chasing a technological ghost. Oh, definitely.
• 19:51
  If we can never truly prove a machine wrote a
• 19:53
  piece of text, maybe the future of digital truth
• 19:56
  has to flip completely. Instead of scanning the
• 19:59
  text to find the machine, we might have to invent
• 20:01
  new ways to prove the biological presence of
• 20:04
  the human. That's a huge shift. Right. Are we
• 20:07
  moving toward a future where, to submit an important
• 20:10
  report or a legal document, you have to provide
• 20:12
  a cryptographic signature verifying your human
• 20:15
  keystrokes? Or a biometric log proving your eyes
• 20:18
  were actually reading the screen as it was typed?
• 20:21
  If the final text can't be trusted, the proof
• 20:24
  of human effort might become the only currency
• 20:26
  we have left. It's a lot to think about. It really
• 20:28
  is. Thanks for listening to the AI Newspaper.
• 20:31
  Visit AINewspaper .org to catch up on the latest
• 20:34
  updates and sign up for free email alerts.