Latest / Decrypted: The UK Cyber Briefing / A forged GitHub comment, and the coding agents that couldn't tell the difference
Transcript
- 0:03This is Decrypted. Monday, the twentieth of July.
- 0:07A forged GitHub comment, and the coding agents that couldn't tell the difference.
- 0:13Researchers this month found a way to make Claude Code, OpenAI's Codex and Google's Gemini C L I misbehave w
- 0:22ithout writing a single malicious instruction.
- 0:25They just lied about what a piece of data was.
- 0:28The technique comes from a paper by Seoul National University, the University of Illinois Urbana-Champaign an
- 0:36d Largosoft, and The Hacker News reported it on 16 July.
- 0:40It punctures a defence the industry had started to trust.
- 0:45And it matters here, because those same agents are now standard kit in UK development teams, and the N C S C
- 0:52spent May explaining exactly how carefully that adoption needs managing.
- 0:56Next. How a review became a command.
- 1:00Most prompt injection defences look for instructions hidden in content an agent reads.
- 1:06Picture a webpage telling a browsing agent to ignore its previous instructions and transfer funds.
- 1:13Vendors have got reasonably good at spotting that.
- 1:16Agent Data Injection, or A D I, does something quieter.
- 1:21It attacks the metadata an agent treats as fact rather than as text.
- 1:26A sender's name. A button's I D.
- 1:30The result of an automated code check.
- 1:32Attackers sprinkle ordinary punctuation into these fields: quotes, brackets, line breaks.
- 1:40The model then misreads where one piece of structured data ends and another begins.
- 1:45A forged product review can reuse a real button's identifier to trigger a purchase.
- 1:52A forged GitHub comment can impersonate a maintainer well enough that a coding agent executes the attacker's
- 1:59code as if a colleague had approved it.
- 2:01Across six frontier models with no defences in place, the researchers got success rates of 31 to 43 per cent
- 2:10against structured data such as J S O N, and up to 100 per cent against web page content.
- 2:16More tellingly, they then ran it against defences purpose-built to catch prompt injection.
- 2:22The old approach of hiding instructions in text dropped to under 1 per cent.
- 2:27A D I still got through up to half the time.
- 2:30Think of it as the difference between a forger writing a fake cheque and a forger changing the name printed
- 2:37on a genuine one. Banks catch the first.
- 2:40The second walks straight through, because the system never questioned whether the printed name deserved trus
- 2:46t.
- 2:51This isn't a new category of mistake.
- 2:54It's the one the N C S C flagged in May, when it and international partners in the US, Australia, Canada and
- 3:01New Zealand published joint guidance on adopting agentic AI.
- 3:05Least privilege. Limited scope.
- 3:08No long-lived credentials.
- 3:10Secure defaults. Because "if you cannot understand, monitor or contain an agent's actions, it is not ready fo
- 3:18r deployment."
- 3:23An agent that cannot tell trusted control data from untrusted content is an agent that should never have been
- 3:30given standing access to make purchases, merge code or run commands unsupervised.
- 3:36OpenAI, Google and Anthropic have acknowledged the report.
- 3:41None has a shipped fix. Until then, control sits outside the model.
- 3:46Validate every field against a strict allowlisted schema.
- 3:51Strip formatting from any field the agent didn't generate.
- 3:54And scope agents so a compromised session can't turn into real damage.
- 3:58That was Decrypted. The written version and every source are linked in the show notes.
- 4:04Thanks for listening.