Latest / The Augur Dispatch / AI coding agents: the hidden cost of human review

Transcript

AI coding agents: the hidden cost of human review · The Augur Dispatch

• 0:06
  SPEAKER_00: Picture an engineering lead opening the review queue late in the afternoon.
• 0:11
  SPEAKER_00: The coding agents have finished another batch of changes, but the person who must approve them hasn't even finished the previous batch yet.
• 0:19
  SPEAKER_00: Before you start buying more agent seats, check whether that queue is where your savings went.
• 0:24
  SPEAKER_00: Take the last 20 merge changes and compare the review rounds for those opened by agents with those opened by people.
• 0:31
  SPEAKER_00: That's the decision behind today's brief.
• 0:33
  SPEAKER_00: This is the Augur Dispatch for Monday, September 7th, 2026.
• 0:38
  SPEAKER_00: A quick disclosure before we begin.
• 0:40
  SPEAKER_00: Portions of the Augur Dispatch podcast are generated using artificial intelligence.
• 0:45
  SPEAKER_00: The written dispatch contains the full source trail and verification bits.
• 0:49
  SPEAKER_00: Latent Space reports that projects, including Flu and TLDRAW, have closed their doors to outside pull requests, the proposals developers submit before changes become part of a project.
• 1:01
  SPEAKER_00: Yet the work inside hasn't stopped.
• 1:03
  SPEAKER_00: Some projects are using their own agents to investigate reports, reproduce problems, write fixes, and review the results before a person approves the merge.
• 1:14
  SPEAKER_00: Closing outside contributions can look like retreat.
• 1:17
  SPEAKER_00: Here, it also means choosing who gets to create the work that someone else must check.
• 1:23
  SPEAKER_00: Cursor is turning that persistent workflow into a product.
• 1:26
  SPEAKER_00: Its cloud agents can stay attached to a proposed change, repair failed automated tests, and respond to review bots until the change merges.
• 1:35
  SPEAKER_00: They can also watch a Slack conversation and resume when something happens.
• 1:39
  SPEAKER_00: Those subscriptions are cloud only in the August 19th change log.
• 1:43
  SPEAKER_00: They're a vendor feature, not an audited result, showing that reviewers spend fewer hours on each change.
• 1:49
  SPEAKER_00: Uber says more than 70% of its pull requests pass through local or cloud agents.
• 1:55
  SPEAKER_00: That's a company reported adoption figure.
• 1:58
  SPEAKER_00: It doesn't tell us whether an agent opened the request, how much it changed, or how long a person spent checking it.
• 2:04
  SPEAKER_00: The distinction matters when a finance team sees output rising and assumes the cost of producing useful software must be falling, because the unfinished approval work may sit outside the dashboard entirely.
• 2:17
  SPEAKER_00: A conference account of Bloomberg's rollout describes open requests piling up while merges slowed.
• 2:24
  SPEAKER_00: Other speakers report weaker review quality and more changes being merged without review.
• 2:30
  SPEAKER_00: These reports support a concern, not a universal verdict on agents.
• 2:35
  SPEAKER_00: An agent that patiently answers every review bot can be useful.
• 2:38
  SPEAKER_00: Unfortunately, patience doesn't create another hour in the reviewer's calendar.
• 2:43
  SPEAKER_00: Software has finally learned to keep a conversation going without making the meeting shorter.
• 2:49
  SPEAKER_00: There are different ways to manage the arriving work.
• 2:52
  SPEAKER_00: SGLang's latest release absorbed nearly 800 pull requests from more than 200 contributors above its July counts.
• 2:59
  SPEAKER_00: Those release notes don't reveal the review policy or the hours required.
• 3:04
  SPEAKER_00: ZIG has chosen to ban AI from pull requests and issues, with protecting how new contributors learn part of the rationale reported at AI Engineer.
• 3:13
  SPEAKER_00: Other projects bring the work inside and run their own agents.
• 3:17
  SPEAKER_00: Each approach has a different trade-off.
• 3:19
  SPEAKER_00: The contribution counts alone can't tell an engineering lead which policy produces safer software or makes better respect of experienced reviewers.
• 3:27
  SPEAKER_00: Imagine a junior developer who would once have learned by preparing a small change and discussing it with a maintainer.
• 3:33
  SPEAKER_00: If a project blocks machine-written contributions, it may be protecting that exchange.
• 3:38
  SPEAKER_00: If it internalizes the work, it needs to decide where that learning happens instead.
• 3:43
  SPEAKER_00: The headline, about three open AI engineers producing roughly 1,500 pull requests and a million lines of code over five months, is impressive output.
• 3:52
  SPEAKER_00: As Nate Jones describes it, people didn't write that code, but that count doesn't answer how many hours went into approving it.
• 3:59
  SPEAKER_00: For a buyer, that missing denominator is more useful than another screenshot of a very busy agent.
• 4:06
  SPEAKER_00: Make the review question concrete in one repository.
• 4:10
  SPEAKER_00: Have the engineering lead own the 20 change check with review rounds and elapsed time from opening to merge as the baseline.
• 4:17
  SPEAKER_00: Keep mixed or uncertain authorship visible rather than forcing everything into clean categories.
• 4:23
  SPEAKER_00: If agent open changes need more review rounds, you've found a reason to investigate.
• 4:28
  SPEAKER_00: You haven't proved that agents cause the difference, because the size and difficulty of the work may also differ, and a sample of 20 changes can only provide an initial signal.
• 4:38
  SPEAKER_00: For a small follow-up pilot, cap the number of agent changes waiting for review in that repository.
• 4:44
  SPEAKER_00: Choose the cap with the reviewers, retain a named human approver, and keep the existing tests and security checks.
• 4:51
  SPEAKER_00: Review the result after two weeks.
• 4:53
  SPEAKER_00: Look for fewer review rounds and shorter waits on comparable work without more defects or skipped checks.
• 5:00
  SPEAKER_00: Stop expanding the pilot if review quality slips or the queue keeps growing.
• 5:05
  SPEAKER_00: The possible advantage is learning how your team can accept useful changes reliably.
• 5:10
  SPEAKER_00: Competitors can buy the same agents.
• 5:13
  SPEAKER_00: They won't automatically have your repository context, review habits, or evidence about which jobs deserve automation.
• 5:20
  SPEAKER_00: A queue limit may sound less exciting than another model upgrade, it does have the advantage of fitting into a normal working day.
• 5:28
  SPEAKER_00: Anthropic's latest announcement sharpens another purchasing question.
• 5:32
  SPEAKER_00: Claude Fable 5.1 and Claude Mythos 5.1 are the same underlying model with different safeguards.
• 5:40
  SPEAKER_00: Fable is generally available.
• 5:42
  SPEAKER_00: Mythos remains restricted to trusted access programs for work, including cybersecurity and life sciences.
• 5:49
  SPEAKER_00: Think of a technology buyer preparing a request for special access.
• 5:53
  SPEAKER_00: The useful first question is which legitimate task needs that safeguard profile, rather than whether the more exclusive name means there's a smarter model behind it.
• 6:02
  SPEAKER_00: Anthropic and Amazon's current documentation support the access distinction.
• 6:07
  SPEAKER_00: It doesn't mean safeguards have no effect on results.
• 6:10
  SPEAKER_00: Restrictions and fallback behavior can change what a service will do for a particular request.
• 6:16
  SPEAKER_00: For most teams, evaluate Fable against the work you actually need done.
• 6:20
  SPEAKER_00: Pursue trusted access when the task requires it, not because exclusivity sounds like a procurement advantage.
• 6:28
  SPEAKER_00: OpenAI has now published figures behind its research acceleration story.
• 6:33
  SPEAKER_00: By mid-August, it reports just over three agent workdays for every human workday in its research organization.
• 6:40
  SPEAKER_00: Its median researcher used more than$600 of daily inference at published API prices.
• 6:47
  SPEAKER_00: That is a way of valuing usage, not a statement that OpenAI paid an external supplier that amount.
• 6:55
  SPEAKER_00: Agent runtime also isn't interchangeable with a person's productive working time.
• 7:01
  SPEAKER_00: The company reports more experiments per active researcher, but it also had substantially more computing capacity.
• 7:08
  SPEAKER_00: Over half of successful tasks estimated to take a person four to eight hours involved at least one human intervention.
• 7:15
  SPEAKER_00: These are preliminary self-reported measures.
• 7:18
  SPEAKER_00: They're evidence to inspect, rather than proof of how much faster the whole research process has become.
• 7:25
  SPEAKER_00: At the same time, OpenAI has strengthened controls after the hugging face security incident, and chief scientist Jacob Pechotsky has written about the difficulty of keeping increasingly capable systems behaving as intended.
• 7:39
  SPEAKER_00: For someone setting a product roadmap, increased research activity and tighter safety restrictions can both be true.
• 7:46
  SPEAKER_00: Neither gives you a reliable release date on its own.
• 7:49
  SPEAKER_00: Watch the measures and the deployment decisions separately.
• 7:53
  SPEAKER_00: Augur has two dated calls today.
• 7:56
  SPEAKER_00: These are predictions with explicit tests, not promises.
• 8:00
  SPEAKER_00: The website tracks them so readers can hold the judgment to account.
• 8:04
  SPEAKER_00: At 70% odds, Augur expects Enthropic will not make Claude Mythos 5.1 generally available outside trusted access programs on its own API or Amazon Bedrock before March 1st, 2027.
• 8:18
  SPEAKER_00: An anthropic model page or announcement, or an Amazon announcement dated before that deadline and listing general availability proves amiss.
• 8:27
  SPEAKER_00: No such public listing by the due date proves a hit.
• 8:31
  SPEAKER_00: At 50% odds, Augur expects Cursor's public changelog will announce pull request and Slack subscription support for local agents, not only cloud agents, by December 31st, 2026.
• 8:43
  SPEAKER_00: A qualifying changelog entry on or before that date proves a hit.
• 8:47
  SPEAKER_00: Its absence proves a miss.
• 8:50
  SPEAKER_00: Adopt the review cost check now.
• 8:52
  SPEAKER_00: Pilot a queue limit if the evidence in your repository warrants it.
• 8:56
  SPEAKER_00: Watch access changes and research claims without treating either as a reason to surrender the final approval.
• 9:04
  SPEAKER_00: Where does your team spend more time today?
• 9:06
  SPEAKER_00: Producing a change or deciding whether to accept it?
• 9:10
  SPEAKER_00: The written daily dispatch contains the full source trail, the verification limits, and the items Augur is watching past.
• 9:17
  SPEAKER_00: That is the Augur dispatch for Monday, September 7th, 2026.
• 9:22
  SPEAKER_00: Thank you for listening.