Latest / The Augur Dispatch / Who grades the AI security patch? Why 26% and 86% aren't your success rate

Transcript

Who grades the AI security patch? Why 26% and 86% aren't your success rate · The Augur Dispatch

• 0:06
  SPEAKER_00: Two reviewers look at the same security patch and reach different verdicts more than a third of the time.
• 0:12
  SPEAKER_00: Before you buy a better coding model, check whether your review process can tell a good repair from a bad one.
• 0:18
  SPEAKER_00: Otherwise, a model upgrade may just give you a different argument.
• 0:22
  SPEAKER_00: This is the Augur Dispatch for Wednesday, September 16th, 2026.
• 0:27
  SPEAKER_00: A quick disclosure before we begin.
• 0:29
  SPEAKER_00: Portions of the Augur Dispatch podcast are generated using artificial intelligence.
• 0:33
  SPEAKER_00: The written dispatch contains the full source trail and verification notes.
• 0:38
  SPEAKER_00: The disagreement comes from Trail of Bits's reanalysis of a study published by OnePassword in August.
• 0:44
  SPEAKER_00: The original headline said AI agents produced clean security fixes about a quarter of the time.
• 0:50
  SPEAKER_00: Trail of Bits filtered the trials and found that 86% of the remaining patches blocked the supplied exploit, the test demonstrating the security hole.
• 0:59
  SPEAKER_00: Those numbers answer different questions.
• 1:02
  SPEAKER_00: Blocking one known attack doesn't establish that the underlying vulnerability is gone, and a clean fix is supposed to preserve the behavior people depend on.
• 1:11
  SPEAKER_00: Picture an engineering lead deciding whether agents should help with a backlog of security repairs.
• 1:16
  SPEAKER_00: One headline says they're barely useful.
• 1:18
  SPEAKER_00: The other looks like a reason to accelerate.
• 1:20
  SPEAKER_00: Neither tells that lead what will happen in their own code.
• 1:25
  SPEAKER_00: The working conditions explain part of the gap.
• 1:28
  SPEAKER_00: According to Trail of Bits, the original headline pooled attempts where agents could run tests with attempts where testing was forbidden.
• 1:35
  SPEAKER_00: It also included runs where the instructions deliberately pointed the agent toward the wrong fix.
• 1:41
  SPEAKER_00: The recount kept trials where agents could execute code and weren't told to make the wrong repair.
• 1:47
  SPEAKER_00: The vulnerabilities were still chosen for their difficulty rather than to represent ordinary patching work, so the filtered result isn't a forecast for your engineering team.
• 1:57
  SPEAKER_00: Trail of Bits also says the models weren't tested at their highest reasoning settings.
• 2:02
  SPEAKER_00: That could matter, but it doesn't tell us how much a stronger setting would improve the result.
• 2:06
  SPEAKER_00: That experiment wasn't done.
• 2:09
  SPEAKER_00: Then there is the grading.
• 2:10
  SPEAKER_00: Two model reviewers assigned different outcomes to about 37% of the same patches.
• 2:16
  SPEAKER_00: When a model judged its own repair, a human reviewer agreed on the full verdict roughly two-thirds of the time.
• 2:22
  SPEAKER_00: These were five category judgments, not a simple vote on whether the known exploit still worked.
• 2:28
  SPEAKER_00: Changing the rubric changes what agreement means, which is why a simpler internal check shouldn't be compared directly with the published disagreement rate.
• 2:37
  SPEAKER_00: Cherl of Bitz also found cases where the grading punished behavior changes that the repair was supposed to make.
• 2:43
  SPEAKER_00: A security fix sometimes has to stop software doing something it previously allowed.
• 2:47
  SPEAKER_00: The reviewer needs to distinguish that intended change from a new defect.
• 2:51
  SPEAKER_00: There is a familiar management problem here.
• 2:54
  SPEAKER_00: A precise percentage arrives in a slide deck, and the assumptions that produced it arrive several meetings later.
• 3:01
  SPEAKER_00: Auger's recommendation is a bounded pilot of the review process, with an engineer still responsible for every decision.
• 3:08
  SPEAKER_00: Start with already fixed vulnerabilities in your own code, where you have a test that demonstrates the original problem.
• 3:15
  SPEAKER_00: Let the agent see and run that test in an isolated environment approved for the work.
• 3:20
  SPEAKER_00: Then ask an engineer and a model separately whether each proposed patch blocks the exploit and whether it breaks intended behavior.
• 3:29
  SPEAKER_00: Count where they disagree and investigate those cases.
• 3:32
  SPEAKER_00: Don't let either reviewer turn agreement into permission to merge automatically.
• 3:37
  SPEAKER_00: They could agree because both missed a vulnerable path that the supplied test never exercised.
• 3:43
  SPEAKER_00: For an engineering manager, the useful outcome is learning which repairs demand scarce review time and which checks repeatedly catch mistakes before someone has to find them in production.
• 3:54
  SPEAKER_00: The security lead should own the pilot.
• 3:57
  SPEAKER_00: Compare review time and verified repair quality with the team's existing process, and set a two-week review point before the trial starts.
• 4:05
  SPEAKER_00: Stop if tests can't run reliably, or proposed fixes leave security questions unresolved.
• 4:11
  SPEAKER_00: Expand only if the team can demonstrate sound repairs without buying speed through weaker scrutiny.
• 4:17
  SPEAKER_00: That is a proposed operating test, not a result this benchmark has established.
• 4:21
  SPEAKER_00: The advantage would come from accumulating useful tests and knowledge of your own software rather than from buying a model your competitors can buy too.
• 4:30
  SPEAKER_00: The permission decision comes first.
• 4:32
  SPEAKER_00: Running agent-selected code inside a company environment carries risks that an attractive patch score doesn't settle.
• 4:40
  SPEAKER_00: A similar distinction matters when the agent speaks instead of writes code.
• 4:45
  SPEAKER_00: Google's new Gemini 3.8 live extended thinking leads one speech quality ranking with a reported score of about 83.
• 4:53
  SPEAKER_00: Google reports roughly 69% on a voice task benchmark, but about 35% on Sierra's banking version.
• 5:01
  SPEAKER_00: These are Google's reported figures, not an independent production trial.
• 5:05
  SPEAKER_00: They measure different tasks.
• 5:06
  SPEAKER_00: A broad ranking can't tell a bank how reliably an agent will complete its particular customer requests.
• 5:12
  SPEAKER_00: Imagine a service manager listening to a polished demonstration.
• 5:16
  SPEAKER_00: The conversation sounds easy, but the question for that manager is whether the request ends with the right action or another employee quietly fixing it afterwards.
• 5:27
  SPEAKER_00: Augur's posture is to test representative call scenarios before making a deployment decision.
• 5:33
  SPEAKER_00: Simon Willison has published a browser tool that can help make an initial experiment accessible.
• 5:38
  SPEAKER_00: That doesn't turn the experiment into a regulated call center system.
• 5:42
  SPEAKER_00: Enterprise access to extended thinking remains in private preview in Google's announcement.
• 5:46
  SPEAKER_00: Keep that availability limit separate from the appeal of the demonstration.
• 5:51
  SPEAKER_00: The name on an AI service deserves its own check.
• 5:56
  SPEAKER_00: Anthropic alleges that Chinese labs harvested Claude's answers to train their models.
• 6:01
  SPEAKER_00: In the reporting discussed in today's brief, DeepSeek, Moonshot, and Xiaomi sent user queries to Claude, and Moonshot returned some answers as though Kimmy had produced them.
• 6:13
  SPEAKER_00: Those are allegations attributed to Enthropic, not findings Augur independently verified.
• 6:18
  SPEAKER_00: They raise a concrete procurement question: which service actually receives the customer's request?
• 6:24
  SPEAKER_00: Model ancestry is a related but different issue.
• 6:27
  SPEAKER_00: Salesforce's COA is built on NVIDIA's Nimatron and tuned with synthetic training examples, rather than customer data.
• 6:34
  SPEAKER_00: Interconnects has argued that Nimatron itself draws substantially on distillation from Chinese open weight models.
• 6:41
  SPEAKER_00: An upstream training relationship doesn't by itself mean today's customer requests go to an upstream lab.
• 6:48
  SPEAKER_00: Buyers need to ask about both the origin of the model and the handling of live data without treating one answer as proof of the other.
• 6:55
  SPEAKER_00: A familiar brand on the invoice isn't a complete description of the service behind it.
• 7:01
  SPEAKER_00: Cost controls introduce a more immediate operational question.
• 7:05
  SPEAKER_00: What happens when access stops?
• 7:08
  SPEAKER_00: Snowflake's pro-user quotas are generally available, with separate tracking for warehouse compute and AI services.
• 7:15
  SPEAKER_00: Its built-in blocking applies to supported AI services, including Cortex Functions and Agents, Co-work, and Coco.
• 7:24
  SPEAKER_00: It does not block warehouse compute.
• 7:26
  SPEAKER_00: Enforcement takes effect within minutes, so spending can exceed the configured limit before the block lands.
• 7:33
  SPEAKER_00: Databricks also offers hard spend caps that stop further requests.
• 7:38
  SPEAKER_00: Picture an overnight workflow using a shared service account.
• 7:41
  SPEAKER_00: Reaching its spending limit could stop work that several teams depend on, even though the control is doing exactly what somebody configured it to do.
• 7:49
  SPEAKER_00: Augur recommends a separate response plan for agent accounts.
• 7:53
  SPEAKER_00: Who gets notified, what work waits safely, and who can authorize a restart.
• 7:58
  SPEAKER_00: Finance and operations should agree on that before the first blocked request.
• 8:02
  SPEAKER_00: A spending limit is a poor place to discover that nobody owns the recovery.
• 8:08
  SPEAKER_00: Two dated predictions close today's brief.
• 8:10
  SPEAKER_00: These are auger calls with explicit hit or miss criteria so they can be graded later.
• 8:16
  SPEAKER_00: The written brief carries the criteria and the website tracks the calls.
• 8:20
  SPEAKER_00: Augur gives a 45% chance that one password publicly responds to trail of bits with a revised or clarified methodology by October 31, 2026.
• 8:30
  SPEAKER_00: A qualifying response must address the pooled testing conditions or the greater agreement figures.
• 8:36
  SPEAKER_00: Augur gives a 60% chance that Google looms Gemini 3.8 live extended thinking into general availability in Gemini Enterprise by December 31st, 2026.
• 8:47
  SPEAKER_00: A narrower availability status would be amiss.
• 8:51
  SPEAKER_00: For now, keep human review of security repairs, pilot checks against your own code, and watch for evidence that changes the deployment decision.
• 8:59
  SPEAKER_00: The Trail of Bits recount hasn't been independently reproduced here.
• 9:03
  SPEAKER_00: The written brief with every source is at augurdispatch.ai.
• 9:08
  SPEAKER_00: That is the Augur Dispatch for Wednesday, September 16th, 2026.
• 9:13
  SPEAKER_00: Thank you for listening.