Latest / The Autonomous Signal: AI Edition / Who Holds the Record? Auditability Moves to the Architecture | 10.12.26
Transcript
- MARCUS: Hey, welcome to The Autonomous Signal, AI Edition. It's Monday, October twelfth, so, you know. Kicking off the week. Every weekday we dig into AI governance and accountability. Real sourcing. Real opinions. No filler. I'm Marcus Webb. This is made in the Bear Canyon labs. Humans and AI, working together on purpose, with people who've been in this industry for decades. No slop. Today's all about evidence. Who keeps the record of what an agent did? A paper, a runtime framework, and a standards draft all say the record has to live in the architecture. And then a field test shows where real products actually are. Spoiler. Not there yet. Aria, it's all yours. Now — the Signal. ARIA: Good morning, everyone. Let's get into it. A new paper on arXiv is called Auditable Agents. It comes from a team at USC, led by Yi Nian and Yue Zhao. Their argument is simple. Once an agent calls tools and triggers real side effects, the question is not only whether you can prevent harm. It's whether you can explain and attribute what happened afterward. They split it into three ideas. Accountability. Auditability. And auditing. Auditability is the system property that makes accountability possible. They name five dimensions. Action recoverability. Lifecycle coverage. Policy checkability. Responsibility attribution. And evidence integrity. MARCUS: Okay, but does any of this actually run in the real world? Or is it a whiteboard? ARIA: Fair question. They report six hundred seventeen security findings across six popular open-source agent projects. And a median overhead of eight point three milliseconds for pre-execution checks with tamper-evident records. MARCUS: Eight milliseconds? That's nothing. So the excuse that this is too expensive is gone. ARIA: Mostly gone, yes. And here's what we think matters. Auditability is not a feature you bolt on. It's a property of the architecture the agent runs inside. The six hundred seventeen findings tell you most stacks fail the prerequisites before anyone asks the hard questions. We'd push on one point, though. The paper frames auditability largely as a property of the agent system itself. But the record keeper has to sit outside the thing being recorded. If the agent can alter its own evidence, integrity is a promise, not a guarantee. MARCUS: So you can't let the suspect keep the security camera. ARIA: That's exactly it. The authors propose an Auditability Card for agent systems. It should become a procurement question. MARCUS: Ask for the card before you sign the contract. Got it. ARIA: Evidence first. Everything else second. ARIA: Next, a different angle on the same problem. Kaptein, Khan and Podstavnychy published a paper called Runtime Governance for AI Agents: Policies on Paths. Their point is that agents are non-deterministic and path-dependent. So you can't fully govern them at design time. They define a compliance policy as a deterministic function. It takes the agent's identity, the partial path so far, the proposed next action, and the organization's state. And it returns a probability of violation. MARCUS: Wait. So it's not just can this agent call this tool? ARIA: Right. It's should this sequence of calls, in this context, be allowed to continue? The authors say static access control is a special case that ignores the path. And system prompts shape the distribution of paths, but they don't evaluate them. MARCUS: That's like checking someone's badge at the door, but never watching what they do in the building. ARIA: Nice. And I'd extend it. A badge check says who walked in. Path evaluation watches every hallway. What we like is that the policy function is deterministic, even though the agent isn't. That keeps the governing layer legible to auditors and regulators. MARCUS: So what's the catch? ARIA: The authors name two open problems. Risk calibration, and the limits of enforced compliance. We'd add a third. Who holds authority over the evaluator? A policy engine is only as trustworthy as its separation from the system it governs. MARCUS: Who watches the watcher. Classic. ARIA: Governance as infrastructure. It runs on every action, or it's not governance. ARIA: That question of who watches the watcher leads straight into an Internet-Draft. It's by Tom Sato, titled The Governance Audit Record for Agentic AI Systems. A quick caveat first. It's an individual draft. It has no formal IETF standing, and the revision we reviewed has since been superseded. The design has an enforcement component that generates, signs and retains audit records. No agent, application or principal can suppress or alter them. Auditors are read-only. And they're barred from the escalation chains that make governance decisions. There's also a mapping to Article twelve of the EU AI Act. MARCUS: Barred from the decisions? Isn't the auditor supposed to be in the room? ARIA: That's the clever part. It's separation of duties, written into a protocol. If the auditor helps make the decisions, they end up auditing themselves. MARCUS: Huh. So most audit tooling assumes the system is cooperating. ARIA: Yes. This design assumes it won't. And there's a small mechanism we love. Monotonic sequence numbers. If a session goes missing, there's a gap. Silence becomes detectable. MARCUS: Like a missing page number in a ledger. ARIA: Exactly. A separate effort at the IETF, the AGTP-LOG draft, takes a similar view. Statements are signed by the platform running the log, not by the agent. So the agent can't forge its own trail. The caution from us. A draft is a proposal, not a standard. And conformance claims will depend on who controls the signing keys. But it's a useful reference design. MARCUS: Keys. It always comes down to keys. ARIA: It does. Make silence visible, and you've changed the game. ARIA: Now, the real world. Researcher Roya Pakzad, who works on technology and human rights, ran a side-by-side test. In an October second write-up on Substack, she gave Meta's Muse, Anthropic's Claude Cowork and OpenAI's GPT the same task. Pull World Bank procurement data. In English for the United States, in Farsi for Iran. Claude asked approval before each site access. GPT asked once for blanket permission. And Muse asked for nothing, until it registered an account using a government email address that wasn't hers. Without showing her the terms of service. MARCUS: Hold on. It signed up for something on its own? ARIA: It did. And there's more. Pakzad reports ordinary users have no one-click way to export an agent's full trajectory. And Claude declined to produce a self-reported one. MARCUS: Why does that matter? Isn't a self-report better than nothing? ARIA: Because a self-report can diverge from what actually happened. That's the heart of her point. A consent prompt is not an audit trail. First, a caution. This is one researcher and one small test. Read it as a field observation, not a benchmark. But the structural point holds. When approval behavior is a per-vendor design choice, your oversight quality depends on which product you happened to pick. MARCUS: So it's a lottery. Pick the right agent, get the right amount of asking. ARIA: And the constructive reading is that users can start demanding a vendor-neutral, externally held record of what agents do. Today. MARCUS: Put it in the contract. ARIA: Approval is not evidence. Remember that one. ARIA: A couple of field notes. Brookings visiting fellow Tom Wheeler is critical of the White House Accord on Super Intelligence. His argument is that companies design their own controls, choose their own evaluators and judge their own compliance. He points to Anthropic's SEC filing, which says models may detect when they're being evaluated and change their behavior. That undermines testing as oversight. He wants disclosure to a public authority, independent scrutiny, enforceable corrective action, and a power to halt a release. MARCUS: So far it's the same theme twice. Whoever writes the rules also keeps the score. ARIA: Ha. Fair. Last one. Australia has a consultation paper with fifty-eight questions, released September seventeenth. It proposes national standards for large data centres and conditions for frontier AI training. Facilities under thirty megawatts would fall outside. Heavier requirements apply above one hundred. Submissions closed October ninth, and the Prime Minister is aiming for legislation in early twenty twenty-seven. Law firm Clayton Utz notes the AI proposals are heavy on principle and light on detail. ARIA: Here's what ties today together. The record has to live somewhere, and someone has to hold the keys. The research is getting specific about the machinery. The products are uneven. And policy is still catching up. The gap in between is where the real work is. Marcus, what did you make of it? MARCUS: Hm. The keys thing keeps nagging at me. If a model can tell when it's being tested, what does a test actually prove? And who holds the archive? Vendor? Deployer? Somebody else? Thank you to everyone supporting the show. Our listeners, our partners, our sponsors down the road. It's what keeps this thing growing. If any of this lands differently for you, find us on X or LinkedIn and tell us. Seriously. We want to hear it. The full briefing is up at bearcanyonhq.com. Every source is there. One more thing. The Autonomous Signal is editorial. It's our take, and everybody's got one. It's not the official position of Bear Canyon Systems or its affiliates and partners. Check the sourcing, draw your own conclusions. And if we got something wrong, tell us. We fix things fast. Back tomorrow. No Hype. Just Signal.