Latest / Braid / When the Clinic Became a Router

Transcript

When the Clinic Became a Router · Braid

• 0:04
  lenar: The NHS is preparing to use AI inside its app to direct patients toward the appropriate service. The Guardian reports that the tool will triage people and decide whether they should get a GP appointment, go to a pharmacy, or head to A&E, depending on what they describe. The first rollout is supposed to reach about 200,000 patients over the next year, with availability to all users by April 2028. That is a very ordinary interface for a very serious decision. You open an app because you feel ill. You type or tap through questions, and somewhere behind that flow a system helps decide whether your next stop is a doctor, a pharmacist, or an emergency department.
• 0:46
  damra: The ordinary interface is what gets me. A diagnostic AI demo can feel separate from normal care. This is different. This is the front door. It touches appointment scarcity, patient confidence, local capacity, and the small moments where someone says, I can wait, or no, I need help now. If the app gets that judgment wrong in either direction, a patient is sent to the wrong queue, a GP line stays jammed, or an A&E department gets someone who might have been treated somewhere else.
• 1:19
  lenar: The political context matters because the app is being asked to solve a human queue. Ending the 8 a.m. scramble for same-day GP appointments was a Labour manifesto promise before the 2024 election. The government points to a trial at Wealden Ridge Medical Partnership in Sussex, where it says the number of patients queueing for a GP appointment by phone fell by 29 percent. The rollout sits inside a 10 billion pound technology and data package for the health service. So the AI isn't a side project here. It is being attached to a public promise: fewer calls, faster care, and less paperwork.
• 1:55
  damra: That is a lot to put on a routing layer. A 29 percent fall in phone-line queuing is meaningful if the patients still got appropriate care. It is less meaningful if the pressure just moved from the telephone line to the pharmacy counter, or from a receptionist to a clinician who has to correct the app's triage. I don't say that as a reason not to try it. I say it because the measurement has to follow the patient after the click.
• 2:20
  lenar: The same Guardian piece says the funding package is also expected to include AI that records patient consultations, with a Great Ormond Street trial across nine London sites finding staff spent 25 percent more time interacting with patients when using the tool, according to officials. That is the optimistic version of the story: less typing, more face time, and less admin dragged home at the end of the day. Then the health leaders quoted in the piece pull the argument back down to ground level. They want a broader strategy for AI across the NHS, more evidence on productivity, clearer privacy protections, and a plan for patients who are less comfortable using digital services.
• 3:01
  damra: That last group matters because public healthcare can't only optimize for the person who is good at apps. If a digital triage system becomes the easiest path to care, then digital confidence becomes part of access to care. You can hear the institutional tension there. The government wants a single front door that routes demand more intelligently. Local health leaders want discretion, because the same app flow can work differently in a rural practice, a crowded city surgery, and a household where English isn't the easiest language for medical fear.
• 3:36
  lenar: Australia gives the same day a second healthcare angle, this time from the recordkeeping side. Guardian Australia reports that the federal health department has raised concerns about AI scribes used by doctors. The tools record, transcribe, and summarize doctor-patient conversations for medical notes, and an online poll by the Royal Australian College of General Practitioners found use among Australian doctors nearly doubled from 22 percent in August 2024 to 40 percent in November 2025.
• 4:06
  damra: That adoption curve is fast enough that governance becomes a catch-up exercise. The department's briefing, obtained under freedom of information laws, said AI scribes have little oversight. It also raised the very practical problem that some suppliers may present themselves as privacy compliant while their cloud platforms send data outside Australia. That isn't a philosophical privacy concern. That is a patient speaking into a microphone in a clinic, and the transcript leaving the jurisdiction in a way the patient may not understand.
• 4:39
  lenar: The Australian documents also note that digital scribes count as medical devices only if they serve a therapeutic purpose. That boundary is going to matter. If the vendor says, no, no, we are just saving time and producing notes, then the tool may sit outside some regulatory levers. But if those notes influence treatment, billing, follow-up, and national health records, it is hard to keep pretending the scribe is only an administrative convenience.
• 5:05
  damra: Exactly. The scribe isn't diagnosing, but it can still affect the next clinician's chart, the patient's reported symptoms, the billing code, and the follow-up appointment. A bad summary can become part of the record. A consent process that varies by practice can disappear after the visit. The system doesn't have to be the doctor to change the care. It only has to become the memory of the encounter.
• 5:29
  lenar: So I would pair these two healthcare stories without merging them. The NHS app is about routing patients through a public system. The Australian scribe story is about converting a clinical conversation into durable notes. Both are moving AI into places where the output becomes part of allocation: who gets seen, what gets recorded, what gets reimbursed, and what gets audited later. That is why healthcare leads today. It isn't a lab claim. It is institutions putting AI into the workflow before every liability boundary has settled.
• 6:03
  damra: And it asks a better question than, is AI good enough for healthcare? Which part of healthcare is being delegated? If it is paperwork, maybe the acceptable error model is one kind of thing. If it is triage, it is another. If it is the official memory of a consultation, it is another again. The word AI hides too many jobs under one label. The job description is where the ethics start to become legible.
• 6:31
  lenar: AI Engineer posted a cluster of talks today about what happens after agents are already in production. One session focuses on monitoring and feedback loops for agent products: production sessions, failure analysis, generated fixes, and the uncomfortable fact that static tests miss a lot of what users actually do. Another session discusses continual learning and regression-aware updates. The talks point to the craft problem: once an agent is used by real people, the useful artifact isn't only the prompt or the model. It is the loop that turns traces of bad behavior into something reviewable.
• 7:08
  damra: This is the agent story I like more than the launch demos. A demo asks whether the agent can complete the path once. Production asks whether you can tell what happened on the 400th weird path, replay it, decide whether the failure belongs to the model or the harness, and make the fix without breaking the 300 paths that used to work. It sounds less shiny, but it is where the tool starts to resemble software.
• 7:34
  lenar: The arXiv paper in the same cluster makes that idea sharper. It is called The Log Is the Agent, and the paper describes ActiveGraph, a runtime where the append-only event log is the source of truth. The graph the agent uses is a deterministic projection of that log. Behaviors react to changes in the graph and emit new events. The authors are explicit that they are making a systems claim, not claiming better benchmark accuracy. Their argument is that the design gives deterministic replay, cheap forks from any event, and lineage from the high-level goal down to the model call that produced an artifact.
• 8:12
  damra: That is a very builder-ish claim, but it is also almost a philosophical claim about agents. Most agent frameworks start with the model conversation and then attach tools, memory, logs, and rules around it. ActiveGraph starts with the record. The model becomes one actor that writes events. The tool call becomes an event. A rule change becomes an event. A fork becomes something you can compare because the run has a history you can replay.
• 8:41
  lenar: The paper's diligence example is useful because it gives concrete numbers without pretending they prove general intelligence. The demo runs on three companies. It produces 671 events, including 93 objects, 76 relations, 103 model calls, and 48 tool calls. The authors say the run is byte-deterministic because model and tool responses are served from recorded fixtures. That isn't an accuracy benchmark. It is a demonstration of recoverability. You can ask why a claim appeared in the memo and walk back through the behavior, the event, the model request, the evidence object, and the relation that tied it together.
• 9:20
  damra: Recoverability is a good word here. It isn't glamorous, but it changes the kind of conversation you can have after the agent fails. Without a faithful run history, you get vibes and screenshots. With one, you can say: this event caused that behavior; this prompt hash produced that response; this tool result was cached; this fork changed at step 150 and diverged here. That doesn't make the agent safe by itself. It gives you an object you can inspect instead of a séance. [chuckle]
• 9:54
  lenar: That loops back to healthcare in a modest but useful way. If an AI system touches triage or clinical notes, you want to know which input produced which recommendation or record. If an agent touches a customer workflow, you want the same kind of chain. The domains are different, but the demand for replayable history keeps appearing. A system that acts without a useful memory of its own actions becomes hard to govern, hard to debug, and hard to trust.
• 10:22
  damra: I would push on the human workflow around that log. Someone still has to read the traces, decide which failure deserves a fix, and decide when the fix is safe. The AI Engineer material points at generated pull requests and session analysis, which is interesting because the operations layer is becoming semi-automated too. The monitor watches the agent, the analyzer diagnoses the session, and then another agent may propose the change. At that point the review surface is the product.
• 10:52
  lenar: Yes, and that is where the new maturity line sits. A serious agent product will need a way to collect bad sessions, replay them, turn them into tests, propose changes, and prove that yesterday's good behavior survived today's fix. That doesn't require every team to adopt an event-sourced graph runtime. It does mean the log can't be an afterthought. Once the agent is making decisions across time, the record of those decisions becomes part of the system's intelligence.
• 11:22
  damra: There is a funny inversion there. We spent a year asking whether agents needed better memory so they could help users. Now the agent needs memory so humans can supervise the agent. The memory isn't only for context. It is for accountability, debugging, comparison, and, eventually, controlled self-improvement. That is a more sober version of the agent future, and it is more interesting too.
• 11:48
  lenar: Several items today point at agents and user interfaces. Nodescribe posted about Meta's Astryx design system. Farooq Zafar pointed to Meta Astryx, Google A2UI, and agent-ready interface standards. AI Engineer also had a session on MCP Apps, which is the Model Context Protocol app layer for giving models structured, interactive surfaces rather than only text and tool calls. I am treating the Meta and Google pieces cautiously because those sources are social posts, not primary company releases. But the direction is still visible: people are trying to define what kind of interface an agent should be able to read and operate.
• 12:28
  damra: The interface question is sneaky because it looks like developer ergonomics until you ask who gets to define the contract. Does an agent see pixels? Does it inspect the DOM? Does it get a semantic tree? Does it receive a design-system component with allowed actions? Each answer gives a different party power. The browser, the app developer, the model provider, and the protocol designer all want a say in what the agent can perceive.
• 12:55
  lenar: MCP Apps is the most concrete source in the cluster because it fits inside an existing protocol conversation. MCP started as a way for models to reach tools and context. Apps push toward richer surfaces: the model can work with a UI resource that is more structured than a screenshot and more interactive than a text response. That matters because a lot of useful work isn't a single API call. It is selecting, comparing, confirming, editing, and handing control back to the person at the right moment.
• 13:27
  damra: And the design-system angle is different from the protocol angle. A design system already encodes what a product thinks a button, panel, table, warning, or confirmation should be. If that system becomes agent-readable, the agent can stop treating the app like a mystery room full of clickable rectangles. It can know which actions are destructive, which fields are required, which controls are filters, and which state change needs user confirmation.
• 13:56
  lenar: That is the optimistic version. The harder version is fragmentation. If Meta, Google, Anthropic-adjacent MCP tooling, and every large enterprise design system all define their own agent interface contract, builders may end up writing adapters for the interface layer the same way they already write adapters for models. The better outcome isn't one universal UI religion. It is enough shared semantics that agents can operate across products without every app inventing a private dialect.
• 14:26
  damra: I also think this changes the safety conversation in a more concrete way. Agent risk talk often stays abstract. An interface contract can say: this action spends money, this action deletes data, this action emails a customer, and this action only changes a local draft. The agent can still be wrong, but the environment has a vocabulary for consequence. That is better than hoping the model infers danger from a button label and a CSS class.
• 14:54
  lenar: There is a craft question here too. Designers and frontend engineers may have to start thinking about components as things humans see and agents interpret. A table isn't only a table; it is a set of records, filters, selected rows, permissions, and pending mutations. A modal isn't only a modal; it is an interruption with a reason and a set of allowed exits. If agentic UI becomes a standards fight, it won't be won only by the prettiest component library. It will be won by the contract that lets software act without making the user guess what happened.
• 15:29
  damra: That is why I like the design-system story more than the usual agent browser story. Browser control says the agent can operate the world as we already built it. Agentic design systems ask whether we should build the world with machine collaborators in mind. That is a bigger craft change, and it is going to show up in very small details: labels, action schemas, reversible states, permission prompts, and logs that a person can understand later.
• 15:58
  lenar: A GitHub issue filed against Claude Code on Saturday describes potential session or cache leakage between workspace instances or consumer accounts. This is still an issue report rather than a verified incident report, so I want to keep the altitude right. The user says they were authenticated to an Enterprise ZDR workspace when the agent suddenly asked what kind of bricks they wanted for a Minecraft temple and then recapped that it was building one. They also note they had launched the session from an unrelated working directory that contained context they needed, while the agent was doing work somewhere else.
• 16:34
  damra: That is exactly the kind of bug report that makes engineers uncomfortable because it has two plausible categories. One category is local confusion: wrong working directory, stale context, compaction weirdness, or a tool carrying state from a previous session. The other category is cross-account or cross-workspace leakage, which would be much more serious. The report itself doesn't prove the second category. It does show why people get jumpy when coding agents claim to have memory.
• 17:06
  lenar: The labels on the issue include core, security, bug, and macOS. The reporter's point is also narrower than the internet version will be. They explicitly separate earlier directory pollution, which they can explain from their own setup, from the Minecraft prompt, which they can't. That distinction matters. A messy local session is a product reliability problem. A consumer prompt showing up inside an enterprise zero-data-retention workspace would be a trust-boundary problem. The public evidence right now supports concern and investigation, not a verdict.
• 17:41
  damra: And the reason it gets attention is that coding agents sit inside high-trust environments. They read source, configs, tickets, chat snippets, design notes, and occasionally secrets people shouldn't have put there. If the mental model is, this workspace is sealed, then one weird Minecraft recap punctures that model even before anyone proves a breach. Trust boundaries are partly technical and partly experiential. The user has to believe the boundary exists because the tool behaves like it knows where it is.
• 18:13
  lenar: This sits next to the Alibaba update, but I don't want to replay the whole Alibaba-Claude story. TechCrunch reports that Alibaba will ban employees from using Claude Code starting July 10, citing Reuters and other reports. Anthropic already prohibits Chinese companies and foreign entities owned by them from using its models. TechCrunch also notes the earlier controversy over a version of Claude Code that could identify Chinese users, which Anthropic's Thariq Shihipar described as an experiment meant to prevent account abuse by unauthorized resellers and protect against distillation.
• 18:49
  damra: The fresh angle is that Claude Code is being squeezed from two sides at once. On one side, enterprises and governments are asking whether the tool respects account, region, and workspace boundaries. On the other side, companies like Alibaba are treating the tool as high-risk software and pointing employees toward an internal alternative, Qoder. That isn't only geopolitics. It is procurement hygiene under geopolitical pressure.
• 19:17
  lenar: I think the lesson is less, beware one named product, and more, coding agents now carry institutional identity. They aren't just autocomplete with a chat box. They know who the user is supposed to be, which workspace is supposed to be sealed, and which account policy applies. They also know which country or company may be excluded and which local files they are allowed to touch. A bug in that boundary can look like a security incident even when the root cause is mundane, because the product has taught users that the boundary is meaningful.
• 19:50
  damra: And it connects back to the log conversation without needing a grand theory. When a weird session happens, the useful response isn't a vibe check. It is a trace: which account, which workspace, which directory, which cache, which compaction boundary, which prior prompt, and which model request. If vendors want enterprises to trust coding agents with serious work, they need to make the weird stories boring to investigate.
• 20:18
  lenar: Two media stories today make Hollywood look less like a single anti-AI bloc and more like a complicated customer. TechCrunch reports that Midjourney is trying to compel Disney, Universal, and Warner Bros. to reveal more about their own AI use in the copyright lawsuits against it. The studios sued over alleged infringement, including the ability to generate images of characters like Bart Simpson and Darth Vader. Midjourney argues fair use and wants broader discovery into the studios' generative AI use, including prompts and outputs.
• 20:51
  damra: That discovery fight is revealing because it turns the studios from plaintiffs into potential practitioners. Midjourney's argument is basically: if you are saying our model harms your market, then your own internal use of generative AI is relevant to that market. The judge had already limited discovery to consumer-facing videos and images, and Midjourney wants that limit expanded. You can see why the studios call it a fishing expedition. You can also see why Midjourney wants the record.
• 21:21
  lenar: Techmeme's roundup points to a Los Angeles Times story about ByteDance making Hollywood inroads with Seedance, its video generator, because of low pricing, realism, and timeline-based prompting. The roundup also includes a Threads post saying Seedance costs about 9 dollars per generated minute versus roughly 24 dollars for Google Veo. I would treat the exact comparison as a reported market datapoint, not a universal price law. But the broader point is straightforward: studios can sue model companies and still experiment with model-generated production workflows.
• 21:57
  damra: That dual posture isn't hypocrisy by itself. Large studios have legal departments, VFX budgets, storyboarding needs, labor contracts, and shareholders. They can believe unauthorized training violates their rights and also test tools that reduce previsualization costs or speed up pitch work. The awkward part is that discovery may force the internal experimentation into the legal record. The industry doesn't get to be only plaintiff or only customer. It may be both in the same week.
• 22:29
  lenar: The business brief has a similar split. Forbes has one piece arguing that AI startups are running leaner, and another about worsening bubble math and the critics piling up around AI lab economics. I don't want to make those two articles carry more weight than they can. The narrower market-temperature check is that the capital story looks different depending on whether you are talking about frontier labs, application startups, or service businesses built around enterprise adoption.
• 22:57
  damra: That distinction matters because the public argument often collapses all AI companies into one cost curve. Frontier labs have training runs, data-center commitments, model-serving costs, and expensive talent markets. Application startups can sometimes use the same model APIs to sell useful software with far fewer employees than an older SaaS company would have needed. The bubble critique and the lean-startup data are describing different layers of the market.
• 23:26
  lenar: So the closing picture for Sunday isn't one big thesis. It is a set of places where AI moved from capability into institutional surfaces. The NHS app routes patients. Australian doctors use scribes that turn speech into records. Agent teams are building operations loops around logs. UI people are arguing over how agents should see interfaces. Claude Code is being tested against trust boundaries. Hollywood is litigating while it experiments.
• 23:56
  damra: The practical question for Monday is where the receipts live. In healthcare, the receipt is the triage path and the consent trail. In agents, it is the event log and the replay. In interface standards, it is the action contract. In Hollywood, it may be discovery. AI systems are getting closer to decisions that other people have to live with, and the record of how the decision happened is becoming part of the decision itself.
• 24:24
  lenar: I would leave the day with the record around the model, rather than a call to slow everything down or a victory lap for automation. Who saw what, who consented, which route was chosen, which event caused the action, and which human can still understand the chain afterward. If those records stay legible, the next wave of AI deployment will be much easier to argue about in public. Lenar Kess.