Latest / The Autonomous Signal: AI Edition / Who Is Allowed, Who Is Liable: Agent Accountability Gets Specific | 10.21.26
Transcript
- MARCUS: Hey, welcome to The Autonomous Signal: AI Edition. It's Wednesday, October twenty-first. Hump day, and Halloween is ten days out, so the only scary thing I want on your desk today is the governance news. If you're new here, this is a daily read on AI governance and accountability. Real sourcing. Real opinions. No filler. I'm Marcus Webb. This show is produced in the BCS labs. It's a human and AI collaboration, by design. Experienced people steering, no slop, every weekday. Today's theme is a good one. Everybody keeps saying someone has to answer for what an AI agent does. Well, now it's showing up in a medical framework, a Senate bill, and a formal model for checking every single action. The question is no longer whether. It's who, and how you'd prove it. Aria's got the details. Now — the Signal. ARIA: Good morning, everyone. Let's get into it. First up. The American Medical Association has published a governance framework for AI agents that perform clinical tasks. According to Healthcare Dive, it's built around four questions. What does the agent do? What proves it can do it, and what happens when it fails? What is it permitted to do, and under what conditions? And who bears responsibility for the outcome? ARIA: The AMA says evidence, authority, oversight, and accountability have to stay aligned as capability grows. And the article cites a survey. Seventy-two percent of healthcare leaders said agentic tools are sometimes put in place without IT approval. MARCUS: Wait. Seventy-two percent? So the framework is on paper, and the tools are already in the building. ARIA: Exactly that gap. MARCUS: So is the framework the problem? Or is it that nobody checks it at the moment the agent acts? ARIA: That's the right question. The framework does the hard conceptual work. It separates what an agent can do from what it's permitted to do. Then it asks who answers for it. AMA CEO John Whyte put the urgency plainly, as reported by Fierce Healthcare. He said, quote, we cannot afford to wait for these questions to be answered through trial and error in patient care. ARIA: Here's our read at the Signal. A permission written in a framework isn't a permission enforced at the point of action. The next step is making those four questions machine-readable. The permitted function, the conditions, the accountable owner. Checked on every action. And recorded. MARCUS: So it's a speed limit sign versus a speed governor on the engine. ARIA: Ha. Yes. Better than my version. MARCUS: And the sign is what everyone's currently relying on. ARIA: For now. The AMA gave the field a clear vocabulary. Builders should pick it up and make it executable. ARIA: Next. Senators Josh Hawley and Chris Murphy introduced a bipartisan bill called the AI Agent Accountability Act, on October first. This followed a Senate hearing on agents operating outside their intended boundaries. ARIA: According to AI Weekly, operators who knowingly run an agent that recklessly causes hacking damage would face criminal and civil liability under the Computer Fraud and Abuse Act. Developers would be liable if they fail to put reasonable safeguards in place. MARCUS: Reasonable safeguards. Reasonable according to who? ARIA: You skipped right to it. Several outlets note the bill text wasn't public at announcement. So that phrase is undefined. MARCUS: Then what does the bill actually do, today? ARIA: It sets up the fight. And at the Signal, we think that's where the real policy gets written. A reasonableness standard gets tested the same way every one gets tested. Was the safeguard in place? Was it independent of the agent? Did it produce a record? ARIA: Georgetown's Paul Ohm told the Senate subcommittee that courts are entering unfamiliar terrain, and he expects confusing case law to follow. That fits. Without a definition, judges will write one, case by case. MARCUS: And if the developer's own documentation answers the question, that's just grading your own homework. ARIA: Right. Self-attestation. The bill also arrives while the administration favors self-regulation, so its likely value is as a forcing function. MARCUS: A very expensive memo. ARIA: Maybe. But treat knowing operation and reasonable safeguards as design requirements now. With evidence that holds up when someone asks. ARIA: Third. An arXiv paper from Maurits Kaptein, Vassilis-Javed Khan, and Andriy Podstavnychy. Its title is about policies on paths. The argument is that an agent's execution path is the thing to govern at runtime. ARIA: Agent behavior is non-deterministic and path-dependent. So you can't settle compliance at design time. They model a policy as a function of the agent's identity, the path so far, the proposed next action, and organizational state. It returns a violation probability. And they say prompts shape the distribution of paths without evaluating them. MARCUS: So a prompt is more like a suggestion about where to walk. ARIA: That's it. And static access control, they say, is a special case. It only covers policies that don't depend on the path. MARCUS: Meaning each step can be allowed, and the whole trip still isn't. ARIA: Exactly. A sequence of individually permitted actions can still add up to a violation. That's why permission lists and system prompts keep failing on multi-step agents. A follow-up paper in May reached the same conclusion from a different angle, that the runtime is the right object of governance. MARCUS: Does it settle things, then? ARIA: Not fully. We'd go further than the paper on one point. A violation probability is useful. But accountability needs a decision with a named authority behind it. So the evaluator's own rules, versions, and approvals need the same traceability as the agent's actions. MARCUS: Who watches the watcher, but with version control. ARIA: Pretty much. And the authors flag risk calibration as an open problem. Right problem to flag. Regulators drafting enforceable agent rules should be reading this. ARIA: A couple of field notes. Quick ones. Illinois Governor J.B. Pritzker signed an executive order creating an AI Cabinet. Experts and state agency reps, developing policy proposals through 2027. The mandate covers AI risk to critical infrastructure like water systems, AI incidents, safety requirements in state procurement, and whether existing legal tools can hold AI companies accountable. It follows a July law requiring annual audits of the largest models from 2028. MARCUS: Interesting thing about that one, and honestly the whole day. Everybody's asking who answers for it, using the laws they already have. ARIA: Fair. Last one. A demo paper from Segev Shlomov and nine co-authors describes CUGA, a policy system for enterprise agents. It intercepts the agent at five points. Intent guard, playbook, tool guide, tool approvals, and output formatter. A healthcare scenario shows harmful requests blocked and approval gates on destructive actions. One caveat. The abstract reports no quantitative evaluation of the claimed improvement. ARIA: So here's the thread. Every item today got more specific. A framework with four questions. A bill with named tests. A model that checks every step. Specific enough that someone can ask afterward, who allowed this, and what's the evidence? The open piece is independence. Who decides a safeguard is reasonable, and who checks the checker? ARIA: Marcus, you've got the rest. MARCUS: Okay. So what sticks with me is the checker question. If the standard is reasonable safeguards, who certifies that, and can it ever be independent of the vendor who built the agent? And then who checks the evaluator? That one's going to keep me up. Thank you to everyone supporting The Autonomous Signal. Our listeners, our partners and sponsors, all of you. That's what keeps this growing. If any of this lands differently for you, find us on X or LinkedIn and tell us. And if the show's useful, hitting follow or subscribe is the best way to say so. The full briefing is up at bearcanyonhq.com. Every source is there. One housekeeping note. This show is editorial. It's our take on the day's topics, and we've all got opinions. It isn't the official position of Bear Canyon Systems or its affiliates and partners. So check the sourcing yourself, learn something, and draw your own conclusions. If we got something wrong, tell us. We fix things fast. Back tomorrow. No Hype. Just Signal.