Latest / The Augur Dispatch / AI safety pledges: what buyers can demand from Anthropic and OpenAI

Transcript

AI safety pledges: what buyers can demand from Anthropic and OpenAI · The Augur Dispatch

• 0:07
  SPEAKER_00: An AI vendor says independent evaluators will get desks, company laptops, and access close to its own safety teams.
• 0:15
  SPEAKER_00: That's a promise you can ask the vendor to deliver.
• 0:18
  SPEAKER_00: A promise that the whole industry will slow down is a different matter, because no single company can make its competitors follow along.
• 0:26
  SPEAKER_00: For anyone buying AI this week, Augur's advice is to ask for the evaluator's name and the terms of access.
• 0:32
  SPEAKER_00: Don't rewrite a delivery schedule around a proposed slowdown.
• 0:37
  SPEAKER_00: This is the Augur Dispatch for Monday, September 14th, 2026.
• 0:42
  SPEAKER_00: A quick disclosure before we begin.
• 0:44
  SPEAKER_00: Portions of the Augur Dispatch podcast are generated using artificial intelligence.
• 0:48
  SPEAKER_00: The written dispatch contains the full source trail and verification notes.
• 0:52
  SPEAKER_00: Dario Amade, Anthropic's chief executive, has proposed pacing the most capable systems so safety work has time to catch up.
• 0:59
  SPEAKER_00: Anthropic has committed to bringing outside evaluators inside the company, with access largely comparable to its internal risk reviewers, subject to legal and contractual exceptions.
• 1:09
  SPEAKER_00: Sam Altman has said OpenAI will follow the evaluator commitment, but its details are still pending.
• 1:15
  SPEAKER_00: A company badge is an unusually useful detail in a debate this abstract.
• 1:20
  SPEAKER_00: It tells a buyer what to ask about.
• 1:22
  SPEAKER_00: It doesn't tell them that an audit has happened.
• 1:25
  SPEAKER_00: The part that needs other people is where the plan gets harder.
• 1:29
  SPEAKER_00: TechCrunch describes three steps.
• 1:31
  SPEAKER_00: Embedded evaluators come first.
• 1:33
  SPEAKER_00: Then, leading companies and democratic countries would coordinate safety standards and limits on unchecked progress.
• 1:40
  SPEAKER_00: The third step would seek international cooperation, including possible agreements with China.
• 1:45
  SPEAKER_00: Those last two steps aren't two things Anthropic can announce into existence.
• 1:50
  SPEAKER_00: Dario wants government help, including a narrow antitrust waiver for certain safety conversations among competitors.
• 1:57
  SPEAKER_00: An antitrust waiver is a legal exception, allowing a kind of coordination that competition law might otherwise prohibit.
• 2:04
  SPEAKER_00: It isn't a safety certificate, and requesting one doesn't establish a coordinated pace.
• 2:09
  SPEAKER_00: Cohere's chief executive, Aidan Gomez, objects to letting a small group of powerful labs write rules that everyone else could be required to follow.
• 2:18
  SPEAKER_00: Cohere is a competitor, so its commercial interest belongs in the picture.
• 2:22
  SPEAKER_00: That doesn't make the concern disappear.
• 2:25
  SPEAKER_00: Whoever defines the standard can affect the cost of meeting it.
• 2:29
  SPEAKER_00: Dario has argued that Anthropic's preferred rules would disadvantage frontier labs and favor smaller rivals.
• 2:36
  SPEAKER_00: Those are competing claims about a system that doesn't yet exist in the form being requested.
• 2:41
  SPEAKER_00: Augur can't tell you which claim will hold without the actual rules.
• 2:45
  SPEAKER_00: Public support may help a proposal move.
• 2:48
  SPEAKER_00: It doesn't settle who has authority to implement it.
• 2:51
  SPEAKER_00: A crowded endorsement list still isn't an operating agreement.
• 2:56
  SPEAKER_00: Picture a procurement manager opening the latest vendor assessment before a renewal meeting.
• 3:01
  SPEAKER_00: The safety section sounds reassuring, but every supporting document comes from the company trying to win the contract.
• 3:08
  SPEAKER_00: That is where this announcement can change the work immediately.
• 3:12
  SPEAKER_00: Ask the person responsible for AI vendor risk to separate claims backed by the vendor's own testing from claims examined by a named outside organization.
• 3:22
  SPEAKER_00: Then ask what that organization could see and whether the buyer can read its findings.
• 3:28
  SPEAKER_00: A desk inside the building might allow much deeper examination than an outside test through the public product.
• 3:34
  SPEAKER_00: It can also leave important gaps if access excludes the systems or incidents that matter to your use.
• 3:41
  SPEAKER_00: The exceptions deserve the same attention as the pledge.
• 3:45
  SPEAKER_00: This isn't a recommendation to replace your vendor tomorrow.
• 3:48
  SPEAKER_00: It's a recommendation to make the next review harder to satisfy with a reassuring paragraph.
• 3:53
  SPEAKER_00: Procurement has enough paragraphs already.
• 3:56
  SPEAKER_00: Use the latest assessment as the baseline and record which material safety claims have independent support.
• 4:02
  SPEAKER_00: At the next renewal review, unresolved claims should remain unresolved in the risk record rather than acquiring an approval because someone announced an evaluator program.
• 4:12
  SPEAKER_00: The potential advantage is better supplier judgment and less repeated evidence hunting.
• 4:17
  SPEAKER_00: Not exclusive access to a model your competitors can also buy.
• 4:22
  SPEAKER_00: Keep the evidence in a reusable vendor record.
• 4:25
  SPEAKER_00: Change your assessment when the access and findings justify it.
• 4:28
  SPEAKER_00: Continue watching the broader pacing proposal until it changes an actual operating condition.
• 4:35
  SPEAKER_00: The research credit dispute around OpenAI's Navier Stokes result puts a different kind of trust under pressure.
• 4:41
  SPEAKER_00: These equations describe fluid motion.
• 4:44
  SPEAKER_00: The famous mathematical problem asks, in precise terms, whether initially smooth motion can break down.
• 4:50
  SPEAKER_00: OpenAI says an unreleased internal system produced a solution, and it has published a write-up and a formalization in lean, software used to check mathematical proofs.
• 4:59
  SPEAKER_00: The timing needs care.
• 5:00
  SPEAKER_00: OpenAI itself reports about 88 hours to reach the result, followed by another 17 hours for formalization and verification.
• 5:08
  SPEAKER_00: Those are company reported timings.
• 5:10
  SPEAKER_00: They're not an independently audited stopwatch reading, but they aren't merely a number borrowed from satire either.
• 5:15
  SPEAKER_00: The credit dispute remains unresolved in the accounts behind today's dispatch.
• 5:20
  SPEAKER_00: Mathematician Tristan Buckmaster alleges unfair treatment in discussions with OpenAI, including proposed authorship terms involving his collaborator, Levent Alpoge.
• 5:30
  SPEAKER_00: OpenAI researchers deny acting unethically.
• 5:33
  SPEAKER_00: Today's source check doesn't settle those competing accounts or independently validate the mathematics.
• 5:39
  SPEAKER_00: Imagine a research team deciding whether to share preliminary work with a lab before posting it publicly.
• 5:45
  SPEAKER_00: Written credit terms may feel awkward before a friendly conversation.
• 5:49
  SPEAKER_00: They're much less awkward then than after two teams believe they deserve the same announcement.
• 5:54
  SPEAKER_00: Set those terms before the first call.
• 6:04
  SPEAKER_00: The researcher's method, hard flow, works with flow matching models, a kind of generative model that creates outputs through a sequence of transformations.
• 6:13
  SPEAKER_00: Its technical idea is trajectory optimization.
• 6:17
  SPEAKER_00: In plain language, it adjusts the route the model takes while generating so the final output meets the required constraints.
• 6:24
  SPEAKER_00: It doesn't force every intermediate step to satisfy the final rule.
• 6:28
  SPEAKER_00: Those intermediate states are part of the model's computation, not instructions a physical robot should immediately execute.
• 6:35
  SPEAKER_00: Giving the model more room during that computation can help it find a better valid result.
• 6:40
  SPEAKER_00: Consider a robot planning how to move around an obstacle.
• 6:44
  SPEAKER_00: A shorter route has no value if it goes through the obstacle.
• 6:48
  SPEAKER_00: HardFlow aims to satisfy the constraint while also improving a goal such as travel time.
• 6:53
  SPEAKER_00: MIT reports perfect constraint satisfaction in its experiments across robotic manipulation, maze navigation, and image editing.
• 7:01
  SPEAKER_00: Computation time was comparable to or lower than most competing methods.
• 7:06
  SPEAKER_00: Those are research results from the team, not a certificate for your factory floor.
• 7:10
  SPEAKER_00: This isn't a generic checker you attach after any model finishes an answer.
• 7:14
  SPEAKER_00: It steers generation itself.
• 7:16
  SPEAKER_00: Engineers working with the relevant models have a method worth examining, without retraining being required.
• 7:22
  SPEAKER_00: A safety-critical deployment still needs validation in its own conditions.
• 7:27
  SPEAKER_00: For now, watch the research rather than treating it as permission to retire an existing safety control.
• 7:33
  SPEAKER_00: Today's Augur calls make two parts of the pacing story testable.
• 7:38
  SPEAKER_00: The first call is that OpenAI will publicly name at least one outside evaluation organization with embedded, employee-like access to its AI systems by the end of 2026.
• 7:51
  SPEAKER_00: The deadline is December 31st, 2026.
• 7:55
  SPEAKER_00: Augur puts the odds at 55%.
• 7:58
  SPEAKER_00: A hit requires an OpenAI blog post, press release, or official statement naming the organization and describing on-site or employee equivalent access.
• 8:08
  SPEAKER_00: Without that public naming by the deadline, it's a miss.
• 8:12
  SPEAKER_00: General support for evaluation doesn't count.
• 8:15
  SPEAKER_00: The second call is that the U.S.
• 8:17
  SPEAKER_00: government will not grant Frontier AI Labs an antitrust waiver permitting coordinated pacing of AI development by mid-2027.
• 8:27
  SPEAKER_00: The deadline is June 30th, 2027.
• 8:30
  SPEAKER_00: Augur puts the odds at 80%.
• 8:33
  SPEAKER_00: A federal statute, executive order, or Justice Department or Federal Trade Commission business review letter authorizing that coordination would make this call amiss.
• 8:42
  SPEAKER_00: If no such instrument exists by the deadline, it's a hit.
• 8:46
  SPEAKER_00: These are dated predictions with explicit tests so readers can judge whether Augur's expectations held up.
• 8:52
  SPEAKER_00: The website tracks the calls.
• 8:54
  SPEAKER_00: Neither forecast should be mistaken for a guarantee about what a company or government will do.
• 9:01
  SPEAKER_00: Adopt the tougher vendor review question now.
• 9:04
  SPEAKER_00: Keep watching the proposed slowdown and ask for evidence before changing a roadmap.
• 9:08
  SPEAKER_00: For researchers, settle credit before sharing early results.
• 9:12
  SPEAKER_00: For engineers, keep the distinction between a promising method and a validated deployment.
• 9:17
  SPEAKER_00: The written brief with every source is at augerdispatch.ai.
• 9:22
  SPEAKER_00: That is the Augur Dispatch for Monday, September 14th, 2026.
• 9:26
  SPEAKER_00: Thank you for listening.