Latest / The Autonomous Signal: AI Edition / Authority Has to Move With the Evidence | 09.29.26
Transcript
- MARCUS: Hey, welcome to The Autonomous Signal: AI Edition. It's Tuesday, September twenty-ninth. Every weekday, we read the AI governance news so you don't have to. Real sourcing. Real opinions. No filler. I'm Marcus Webb. This is a human and AI collaboration, by design, built in the BCS labs by people who've been in this industry a long, long time. Today's theme? Governance that only happens at a moment in time. Reviews, approvals, assessments. And systems that keep changing between them. Turns out that's a problem. Now — the Signal. Aria, take it away. ARIA: Good morning, everyone. Let's get into it. First up. A June paper from Matthew Francis Dixon, posted on arXiv. He asks a simple question. How much decision authority should an AI system actually hold? Most governance frameworks don't answer that. They cover compliance and oversight. But no quantitative method for setting authority as evidence changes. So Dixon builds one. He treats delegation as a partially observable decision process. Bayesian inference tracks what the organization really knows. And the level of delegation adjusts, step by step. In stress tests, it beat five representative governance strategies. It degraded gracefully. Fixed heuristics just broke. MARCUS: Wait, so authority isn't a setting. It's more like a dial someone keeps touching? ARIA: Exactly like a dial. That's the point. Authority allocation is a live decision problem, not a one time policy. And here's our read at the Signal. That's the mathematical case for governance as infrastructure. If delegation has to respond to live evidence, the control layer sits in the runtime. Observing. Adjusting. Not in a document written before deployment. The Cloud Security Alliance says something similar from the practitioner side. Their research note says frameworks like NIST's AI Risk Management Framework were designed for systems whose behavior can be characterized at deployment time. Agents, they say, routinely violate that. They operate at machine speed across dynamic environments. MARCUS: Okay, but who holds the dial? Because if the agent holds its own dial, that's not oversight. That's a suggestion box. ARIA: A suggestion box. Yes. That's the harder question, and Dixon doesn't address it. Who owns the belief state? The actor can act within the authority it's granted. But whatever raises or lowers that authority has to be independent of the actor it constrains. MARCUS: So earned continuously. And revocable by someone who isn't the agent. ARIA: That's it. Authority earned continuously, evidenced continuously, revocable by someone else. The vocabulary is finally catching up to the principle. ARIA: Second story. A paper on arXiv from Nannini, Adam Leon Smith, and colleagues. They look at agents that plan, call external tools, and run multi-step chains with less human involvement. Then they map the duties that trigger under the EU AI Act. Plus GDPR, the Cyber Resilience Act, the Digital Services Act, and sector rules. Their conclusion is blunt. High-risk agentic systems whose behavior drifts at runtime, in untraceable ways, currently cannot satisfy the Act's requirements. Human oversight. Cybersecurity. Transparency. Their proposed foundation is an exhaustive inventory. Every external action. Every data flow. Every connected system. Every affected person. MARCUS: Hang on. Can't satisfy? Not "struggles with." Can't? ARIA: Can't. Under current design. Which is uncomfortable, but we think it's correct. Here's the thing about that inventory. Compile it once and file it, and it's stale on day two. Have the system generate it as it operates, and it becomes evidence. The OECD AI Principles make the same anchor point. Traceability is a foundation for accountability. Keep records that let people analyze and inquire into what a system did. MARCUS: So it's like a flight recorder. Not a pilot's memoir written after landing. ARIA: I'm stealing that. Yes. Auditability is a property of execution, not paperwork afterward. And it flips the problem. If untraceable drift is the disqualifier, then traceability is the product requirement. Conformity assessment starts to look like continuous assurance. MARCUS: Which sounds less like an obstacle and more like a spec. ARIA: Exactly. A compliance obstacle becomes a design specification. ARIA: A few field notes. The 2026 Singapore Consensus is out. More than a hundred contributors from thirteen countries, with Yoshua Bengio among the lead authors. It names autonomous agents and societal resilience as top research priorities. ARIA: The Council on Foreign Relations looked at the June second executive order. It's voluntary. Companies give the government access to frontier models up to thirty days before release. Matthew Ferren says the order tries to engineer a window for defenders, but hits uneven patching. Vinh Nguyen notes there's no fixed capability ceiling, so it depends on whether evaluators keep pace with months long release cycles. MARCUS: Funny. Everything so far is about tempo. Things moving faster than the check on them. ARIA: It's a theme. And AWS's Reimagine 2026 study proves it. Interviews with a hundred fifty-four executives. Review workflows built for six month IT programs, applied to two week AI experiments. Some teams just skip formal oversight. One organization had eighty-eight percent adoption, but better outcomes in fewer than one in five thousand sessions. Only twenty-four percent have documented responsible AI approaches. Their advice for agents? Start with human approval. Expand autonomy after demonstrated reliability. ARIA: So the thread today is tempo. Governance as a periodic review keeps losing to systems that change between reviews. The good news is the math and the vocabulary for continuous authority are arriving. Marcus, what's your take? MARCUS: Huh. What sticks with me is that dial. Who holds it, and can they be someone other than the agent? And what does an audit look like when it never stops running? I don't have that answer. Yet. Thanks to everyone supporting the show, our partners, and anyone thinking about sponsoring down the road. You're what keeps this thing growing. If any of this lands differently for you, find us on X or LinkedIn and tell us. And if it's helpful, hit follow or subscribe. That's the best signal you can send. The full briefing is online at bearcanyonhq.com, with every source. Quick note. The Autonomous Signal is editorial. It's our take, our opinions, and everybody's got one. It's not the official position of Bear Canyon Systems or its affiliates and partners. Check the sourcing yourself, learn something, draw your own conclusions. Sourcing errors happen. Corrections are welcome, and we fix things fast. Back tomorrow. No Hype. Just Signal.