Latest / The Augur Dispatch / Anthropic and Accenture: who checks the AI safety evaluator?

Transcript

Anthropic and Accenture: who checks the AI safety evaluator? · The Augur Dispatch

• 0:06
  SPEAKER_00: Anthropic will pay an outside firm to examine the safety of its most capable models.
• 0:11
  SPEAKER_00: That firm, Accenture, also helps customers buy and deploy Claude.
• 0:16
  SPEAKER_00: For anyone signing an AI contract, the useful question is: who stands behind the safety assurance and what they're allowed to say when something goes wrong?
• 0:25
  SPEAKER_00: Don't reject the work because of that relationship.
• 0:28
  SPEAKER_00: Ask to see the reporting terms before treating the evaluator's name as evidence of independence.
• 0:33
  SPEAKER_00: This is the Augur dispatch for Saturday, September 19th, 2026.
• 0:38
  SPEAKER_00: Anthropic announced the arrangement yesterday.
• 0:40
  SPEAKER_00: Faculty, Accenture's specialist AI business, will lead the work, and each company expects to invest at least a billion dollars in evaluation capacity over five years.
• 0:50
  SPEAKER_00: Those figures describe intended investment, not money already spent.
• 0:55
  SPEAKER_00: The proposal would put outside evaluators inside the lab, with access comparable to an employee's, so they can follow training and deployment decisions rather than inspect only the finished product.
• 1:06
  SPEAKER_00: That can be useful.
• 1:08
  SPEAKER_00: Imagine a procurement lead looking at a presentation that says a model has been independently evaluated, while the consultancy presenting it also wants the contract to install that model.
• 1:18
  SPEAKER_00: The slide needs a footnote, possibly a longer footnote than the slide.
• 1:23
  SPEAKER_00: The funding arrangement isn't hidden.
• 1:25
  SPEAKER_00: Anthropic says directly that it will fund Accenture's work while discussing pilots with meter and other nonprofit evaluators using their own funding.
• 1:34
  SPEAKER_00: The partnership isn't exclusive, and Anthropic says it expects to announce other evaluators.
• 1:40
  SPEAKER_00: Accenture can also work with other developers.
• 1:43
  SPEAKER_00: That leaves room for different kinds of scrutiny rather than one approved inspector.
• 1:48
  SPEAKER_00: There's a reasonable case for involving a consultancy that knows how businesses use these systems.
• 1:54
  SPEAKER_00: It may understand operational problems that a narrow laboratory test would miss.
• 1:58
  SPEAKER_00: Commercial experience can make the examination better.
• 2:02
  SPEAKER_00: It doesn't by itself establish independence.
• 2:05
  SPEAKER_00: Anthropic says the field still lacks agreed standards for what embedded evaluators should see, how they should report findings, and how independent evaluation should be funded.
• 2:16
  SPEAKER_00: Auger's position is to watch the arrangement's results and reporting rights before increasing the weight you give its safety assurances.
• 2:24
  SPEAKER_00: A large budget can buy more testing, but it can't tell a buyer whether an unfavorable finding will reach them.
• 2:31
  SPEAKER_00: That is a question about the terms of the work, not an accusation that the work will be dishonest.
• 2:36
  SPEAKER_00: The distinction matters to everyone trying to buy useful scrutiny without pretending conflicts disappear when they're disclosed.
• 2:44
  SPEAKER_00: There's also an important correction to the OpenAI part of this story.
• 2:48
  SPEAKER_00: Outside investigators have already worked on-site there.
• 2:51
  SPEAKER_00: On August 26th, OpenAI named Meter in Redwood Research in its account of the Hugging Face incident.
• 2:58
  SPEAKER_00: Their own report describes six days on the premises and says they took no payment from OpenAI for that assessment.
• 3:04
  SPEAKER_00: That was a specific incident investigation.
• 3:07
  SPEAKER_00: It doesn't establish that OpenAI has named a standing embedded evaluation partner under its newer pledge.
• 3:14
  SPEAKER_00: But it does mean a prediction about naming any organization with in-lab access needs much tighter wording.
• 3:21
  SPEAKER_00: We're withholding that ambiguous call from today's package rather than presenting an earlier event as a future prediction.
• 3:28
  SPEAKER_00: The remaining anthropic call keeps its original deadline and odds.
• 3:32
  SPEAKER_00: For the buyer, the historical example is useful because it supplies something more concrete than a partnership announcement, a published account of access, scope, and limitations that readers can examine for themselves.
• 3:46
  SPEAKER_00: Ask for that level of specificity from your own suppliers.
• 3:49
  SPEAKER_00: The word independent on a presentation isn't a substitute for it.
• 3:55
  SPEAKER_00: Inside an application, the evaluator may be another model rather than another company.
• 4:00
  SPEAKER_00: OpenRouters new guide explains why an agent can pass scripted tests and still give a poor answer.
• 4:05
  SPEAKER_00: An agent is software that uses a model to carry out a sequence of tasks.
• 4:09
  SPEAKER_00: It might call the correct customer service tool, then leave an important policy condition out of its reply.
• 4:15
  SPEAKER_00: A model acting as a judge can score the answer against written criteria.
• 4:19
  SPEAKER_00: Amazon's Agent Core Evaluations applies that kind of assessment to live agent conversations, while developers are exploring narrower models, such as TypeSafe's JEV, for classification and routing.
• 4:32
  SPEAKER_00: Picture a support manager reviewing a response that sounds helpful but leaves out a condition the customer needs before making a decision.
• 4:39
  SPEAKER_00: Correct tool use doesn't settle whether the answer was useful.
• 4:42
  SPEAKER_00: A second model can help find that gap, but its scores need calibration against human judgment, and repeated runs can produce different scores for the same input.
• 4:52
  SPEAKER_00: Before using a judge to decide whether software ships, have the team rerun it against examples people have already assessed.
• 4:59
  SPEAKER_00: Treat it as an additional check while you learn where it disagrees or changes its mind.
• 5:04
  SPEAKER_00: Otherwise, you've hired a reviewer whose answer depends partly on which meeting you asked it in.
• 5:11
  SPEAKER_00: Trail of Bits offers a more substantial example of AI changing the work around an audit.
• 5:16
  SPEAKER_00: The security firm says it spent roughly six months building tools before auditing the MIDIN virtual machine, the software that executes its programs.
• 5:24
  SPEAKER_00: Agents helped create instruments, including a decompiler, which makes compiled software easier to inspect, and an analysis engine that searches for weaknesses.
• 5:34
  SPEAKER_00: The audit found an unchecked input that could allow forged signatures and theft of account holders' funds.
• 5:39
  SPEAKER_00: The work also produced 95 machine-checked proofs using lean, a system for verifying mathematical arguments.
• 5:47
  SPEAKER_00: These are the firm's reported results.
• 5:49
  SPEAKER_00: They don't establish that a general-purpose agent can safely replace an auditor, or that a proof covers every way an application might fail.
• 5:58
  SPEAKER_00: The same firm recently criticized a benchmark of AI security patching from OnePassword.
• 6:03
  SPEAKER_00: One objection was that it mixed trials where agents could test their fixes with trials where they couldn't.
• 6:09
  SPEAKER_00: For a security leader buying an audit, the useful question is what tools the team can build and what evidence it can produce about your system, rather than how impressive a patching percentage looks in isolation.
• 6:22
  SPEAKER_00: A reusable inspection tool may help the next audit too.
• 6:25
  SPEAKER_00: That accumulated understanding is a more plausible advantage than access to a model every competing firm can also buy.
• 6:33
  SPEAKER_00: Robotics brings the evidence problem back to the physical world.
• 6:37
  SPEAKER_00: Les Karpas, who leads physical AI at Nvidia's Inception Startup Program, argues that robots haven't had their chat GPT moment because there isn't an equivalent of the Internet's text for learning how bodies move through the world.
• 6:51
  SPEAKER_00: That is his assessment, not a measured deadline for the industry.
• 6:55
  SPEAKER_00: It does give a team considering a robotics project a practical budgeting question.
• 6:59
  SPEAKER_00: What task data would it need to collect for its own setting?
• 7:03
  SPEAKER_00: A better chip doesn't supply that missing experience.
• 7:07
  SPEAKER_00: If the project is already under consideration, price the data collection and validation work before relying on a forecast about when a general robot model will arrive.
• 7:17
  SPEAKER_00: Across today's stories, the useful evidence comes from watching the system do the work it is supposed to do with enough access to understand where the assessment stops.
• 7:29
  SPEAKER_00: Today's retained auger call is that Anthropic will publish an evaluation finding, or system card section, explicitly attributed to Accenture or faculty embedded evaluators by March 31st, 2027.
• 7:41
  SPEAKER_00: The odds are 40%.
• 7:43
  SPEAKER_00: A hit requires a document on Anthropic's website attributing a specific evaluation result to Accenture or faculty.
• 7:50
  SPEAKER_00: Capacity, hiring, or partnership updates don't count.
• 7:53
  SPEAKER_00: Augur calls are dated, testable predictions with explicit hit or miss criteria.
• 7:59
  SPEAKER_00: The website tracks them so the analysis can be held accountable.
• 8:02
  SPEAKER_00: For now, adopt the procurement check.
• 8:05
  SPEAKER_00: Identify who produced each safety assurance and whether that organization also sells or installs the product.
• 8:11
  SPEAKER_00: Where roles overlap, ask about publication rights and oversight rather than assuming the result is either worthless or independent.
• 8:20
  SPEAKER_00: Keep your own integration tests.
• 8:22
  SPEAKER_00: Watch for published findings before changing your confidence in the new arrangement.
• 8:27
  SPEAKER_00: The written brief with every source is at augerdispatch.ai.
• 8:32
  SPEAKER_00: Portions of the Augur Dispatch are AI generated.
• 8:35
  SPEAKER_00: That is the Augur Dispatch for Saturday, September 19th, 2026.
• 8:40
  SPEAKER_00: Thank you for listening.