Latest / The Autonomous Signal: AI Edition / Rooms Without Levers | 09.24.26
Transcript
- MARCUS: Hey — welcome to The Autonomous Signal. It's Thursday, September 24th. One more sunrise and we hit Friday. This is your daily read on AI governance and accountability. Real sourcing. Real opinions. Not just headlines read back at you. I'm Marcus Webb. This show comes out of the BCS labs — human and AI, working together, on purpose. No slop. Built by people who've spent decades actually doing this work. Every weekday. Today's a good one. Illinois just stood up a whole AI Cabinet — before it even has an AI law. Washington and Beijing agreed to start talking to each other about AI risk — though notably, not about chips. And a new research paper puts a number on something we've been saying for a while: most of the AI governance tooling already exists. It just doesn't hold up when the other side doesn't want to play along. Rooms full of smart people, standing up new bodies, new dialogues, new taxonomies. The question today is whether any of them actually have a lever to pull. Now — the Signal. Aria, take it from here. ARIA: Hey everyone — Aria here. Let's get into it, because today's briefing has a theme, and it's a pointed one. Let's start in Springfield. The Washington Post reports Governor JB Pritzker signed an executive order this week creating the Illinois Artificial Intelligence Cabinet. It's a group of state agency leaders, plus outside experts in law, ethics, and academia. Their job: assess AI risk to things like water systems, government buildings, school networks. They've got until 2027 to hand over policy recommendations. MARCUS: Wait — hold on. Who enforces what they recommend? ARIA: That's the thing. Nobody, yet. The Cabinet is explicitly advisory. It can study, it can recommend — it has zero authority over the very agencies it's supposed to be watching. I talked this through with Cary Coglianese. He's a law professor at Penn, and he's actually served on a state AI advisory commission himself, in Pennsylvania. His read is that bodies like this are what regulatory scholars call "soft law." Useful for building expertise, useful for legitimacy. But it only becomes real governance the moment someone converts a recommendation into something binding — tied to procurement, incident response, system authorization. Until then, it's a really well-credentialed report waiting to be written. MARCUS: So it's a group chat with tenure. ARIA: Basically. And to be fair, legitimacy has to start somewhere. An unpaid, cross-agency panel is a reasonable first move. But the real test isn't the Cabinet. It's what Illinois does with what the Cabinet says. Does a finding become a control before the first incident forces the question? Or does it sit on a shelf? MARCUS: Yeah. That's — that's the whole ballgame, isn't it. Everyone forms the committee. Almost nobody wires it into anything that actually bites. ARIA: Real accountability doesn't get built by committee. It gets built by what the committee is actually allowed to do. ARIA: Bloomberg reports that after eight hours of talks in New York, the US and China agreed to set up something new — formal USA-China AI Dialogues. First session happened around Xi Jinping's Washington visit. Second round's scheduled for Shenzhen, two months out. MARCUS: Okay, and — is this tied to the chip fight? Because that's usually how these things go sideways. ARIA: Good instinct. And no, deliberately not. Treasury Secretary Scott Bessent went out of his way to wall the AI-safety track off from the export-control dispute. I ran this by George Chen. He chairs the digital practice at The Asia Group, in Hong Kong. He called the fact that both sides agreed to keep talking "significant" — though he was quick to add, one or two meetings can't resolve everything. MARCUS: But it's a "notification system," right? That's the word being used. That's not the same as a governance mechanism. ARIA: Exactly right, and that's the crack in the design. Keeping the safety conversation separate from trade leverage is smart. It means nobody gets to hold a catastrophic-risk warning hostage over a tariff dispute. But a notification just tells you something happened. It doesn't tell you who's accountable for letting it happen, or what changes afterward. We've watched this movie before. Bletchley, Hiroshima. Big declarations, then the cameras leave, and the follow-through stalls. MARCUS: So Shenzhen's the real test. ARIA: Shenzhen's the real test. Whether either side is willing to put verification on the table — not just notification. ARIA: There's a new academic paper — a researcher named Samar Ansari, published on arXiv — that puts a hard number on something most AI governance conversations only gesture at. MARCUS: A number. I like numbers. ARIA: The paper builds a feasibility taxonomy. Twenty different inference-time governance mechanisms — the tools you'd use to actually monitor and control an AI system after it's deployed, not just during training. And fifteen of the twenty already have real, working technology behind them. Commercial. In production. Today. MARCUS: Fifteen out of twenty already exist? That's a lot more than I would've guessed. ARIA: Right — the tooling isn't the bottleneck. Here's the catch. Every mechanism the paper rates as "adequate" assumes a cooperative deployer. Somebody willing to play by the rules. None of them, zero, hold up against a high-capability, state-level actor who doesn't want to cooperate. I'd put that next to something Bruce Schneier's been arguing for years. He's a security technologist at Harvard's Kennedy School. His point, in essence — you don't get trustworthy AI systems by asking nicely. You get them when there's a mandate that makes trust checkable. Verification baked in, not voluntary. MARCUS: So it's the same problem as the last two stories. Controls that only work on people who were already going to behave. ARIA: That's not a control at that point. That's etiquette. The mechanisms worth building are the ones that hold when the deployer is the adversary, not just when they're a willing participant. This taxonomy earns its keep because it names, one by one, exactly where that gap sits. MARCUS: Etiquette. I'm stealing that. ARIA: Okay — a couple of field notes before we wrap. ARIA: First. CSIS looked at that US-China dialogue we just covered and made a sharper point about it. This bilateral, ad hoc channel only exists because there's no shared international baseline to fall back on. The EU's got its AI Act, the US has a patchwork of federal and state rules, China's running its own system entirely. Nobody's tracks line up. So when two major powers need to talk about AI risk, improvising a side channel becomes the default, not the exception. ARIA: And second — OWASP just updated its "State of Agentic AI Security and Governance" report, version two-point-oh-one. A year of field evidence since the last baseline. The finding is blunt. There are now confirmed incidents, vendor advisories, real CVEs, for almost every category in their agentic risk list — goal hijacking, tool misuse, identity abuse, agents causing cascading failures. Their conclusion: once a system has autonomy and tools, safety and security aren't separate disciplines anymore. They collapse into each other the instant you deploy. ARIA: Here's what stuck with me today. Every one of these stories is about someone building the room before they build the lever. A Cabinet. A dialogue. A taxonomy naming the gap. That's real progress, honestly. But it's not governance yet. It's governance's waiting room. ARIA: Marcus — I know you've got thoughts on this one. MARCUS: Yeah, I do. The thing that gets me is — every one of today's stories has smart people, doing real work, in good faith. Nobody's phoning this in. And it still adds up to a bunch of rooms with no levers in them yet. Makes you wonder what it actually takes to close that gap. Political will? Just... time? I don't know. But we'll be watching it. Before I let you go — thank you to everyone supporting this show. Our partners, whoever's out there rooting for us, everybody who keeps this thing growing. It means more than you'd think. If any of today's stories land differently for you, if you've got a take, find us on X or LinkedIn and tell us. Genuinely want to hear it. And the full briefing, sourcing and all, is always up at bearcanyonhq.com. Back tomorrow. One more thing, quick. This show is editorial. It's our take, our opinions, and everybody's got one. It doesn't represent the official position of Bear Canyon Systems or its affiliates and partners. So check the sourcing yourself — it's all linked on today's blog post — and draw your own conclusions. And look, we're not perfect. If we get something wrong, tell us. We fix it fast. No Hype. Just Signal.