Latest / Anaiya Algorithm / The Hidden Risk in Microsoft Copilot Agents Most Companies Miss
Transcript
- 0:00SPEAKER_00: Microsoft has over three hundred thousand enterprise co-pilot customers.
- 0:07SPEAKER_00: The number of agents being built is growing month by month as you would imagine.
- 0:12SPEAKER_00: And many of those agents built by people who are not necessarily thinking about, you know, the EU AI Act, who may not even know what Dora is, are operating in areas that carry real regulatory exposure.
- 0:28SPEAKER_00: And here's the thing.
- 0:30SPEAKER_00: The EU AI Act's Article 14 requirement for meaningful human oversight does not make an exception for agents built by non technical teams on enterprise platforms.
- 0:44SPEAKER_00: If a copilot agent influences a credit decision or processes insurance data or manages a workflow that touches employment outcomes, it carries the same obligations as any other high risk AI system.
- 1:03SPEAKER_00: The question then isn't whether that's fair.
- 1:07SPEAKER_00: It's whether your organization knows which of its co-pilot agents fall into that category.
- 1:39SPEAKER_00: And can I trace it?
- 1:41SPEAKER_00: And if something has gone wrong, can I fix it?
- 1:45SPEAKER_00: Those are different questions and they need different answers.
- 1:49SPEAKER_00: And most governance frameworks were designed for the first kind.
- 1:54SPEAKER_00: When AI moved from producing output to taking actions, the governor's playbook changed.
- 2:02SPEAKER_00: Most organizations have not updated this, so I want to give you a practical framework here.
- 2:10SPEAKER_00: Five questions that every deployed AI agent in your organization should be able to answer before it goes anywhere near a regulated process.
- 2:22SPEAKER_00: The first question is what can this agent actually do?
- 2:28SPEAKER_00: Not what the documentation says it does, what it can do in practice.
- 2:35SPEAKER_00: What systems can it read?
- 2:38SPEAKER_00: What can it write to?
- 2:40SPEAKER_00: What actions can it take on its own?
- 2:43SPEAKER_00: And what requires a human to say yes first?
- 2:48SPEAKER_00: If you can't answer that precisely, not approximately, but precisely, the agent isn't governed.
- 2:58SPEAKER_00: You've deployed a capability you don't fully understand.
- 3:03SPEAKER_00: The second question is what is it not allowed to do?
- 3:09SPEAKER_00: The boundaries matter as much as the capabilities.
- 3:14SPEAKER_00: A properly governed agent doesn't just have a list of things it can do.
- 3:20SPEAKER_00: It has a list of things it won't do, and those constraints aren't in a policy document somewhere.
- 3:27SPEAKER_00: They're built in technically enforced.
- 3:32SPEAKER_00: If your agent isn't permitted to send external communications without approval, that constraint needs to live in the architecture, not in a note in a confluence page that nobody reads.
- 3:48SPEAKER_00: Now the third question What happens when it reaches the edge of what it's allowed to do?
- 3:56SPEAKER_00: This is the human oversight question and is the one Article fourteen of the EU AI Act is most interested in.
- 4:06SPEAKER_00: When an agent hits a situation outside its parameters, an unusual request, an edge case, something that needs authority it doesn't have, what does it do?
- 4:20SPEAKER_00: Does it escalate?
- 4:22SPEAKER_00: Does it stop and wait?
- 4:24SPEAKER_00: Does it make the closest available judgment call?
- 4:29SPEAKER_00: The answer to that question tells you a lot about whether your governance is real or nominal.
- 4:37SPEAKER_00: Fourth, can every action be traced?
- 4:42SPEAKER_00: Every email sent, every record updated, every decision made, logged, timestamped, attributable.
- 4:53SPEAKER_00: Not because you're expecting something to go wrong, but because when something does and it's will, the trace is how you understand what happened, when and what needs to be corrected.
- 5:09SPEAKER_00: Without it you're investigating blind.
- 5:14SPEAKER_00: Now the fifth question can actions be reversed?
- 5:19SPEAKER_00: Not all of them.
- 5:21SPEAKER_00: Some actions can't be undone, and that's fine, as long as you know which ones.
- 5:29SPEAKER_00: The ones that matter most are the ones where the stakes are high enough that a mistake needs correcting.
- 5:37SPEAKER_00: If your agent sent incorrect information to a customer, can you identify it and retract it?
- 5:46SPEAKER_00: If it updates a database record incorrectly, is there a version history?
- 5:53SPEAKER_00: Irreversibility amplifies risk.
- 5:57SPEAKER_00: The actions that can't be undone require the most rigorous governance before they happen.
- 6:07SPEAKER_00: So I want to talk specifically about Microsoft Copilot agents because they represent the most significant and the least governed agentic deployment happening in enterprise right now.
- 6:22SPEAKER_00: Copilot Studio is genuinely easy to use, and that's intentional, it's a designed to be.
- 6:32SPEAKER_00: So a business team can build an agent in an afternoon, it connects to the tools they already use and it solves a real problem and it gets adopted, which is amazing.
- 6:44SPEAKER_00: And none of that involves a risk assessment.
- 6:47SPEAKER_00: None of it involves IT.
- 6:50SPEAKER_00: And very often, none of it involves anyone asking whether this agent touches regulated data, regulated processes, or regulated decisions.
- 7:02SPEAKER_00: So the scale here is significant.
- 7:06SPEAKER_00: Microsoft has over 300,000 enterprise co-pilot customers.
- 7:13SPEAKER_00: The number of agents being built is growing month by month as you would imagine.
- 7:18SPEAKER_00: And many of those agents built by people who are not necessarily thinking about, you know, the EU AI Act, who may not even know what Dora is, are operating in areas that carry real regulatory exposure.
- 7:36SPEAKER_00: And here's the thing the EU AI Act's Article 14 requirement for meaningful human oversight does not make an exception for agents billed by non technical teams on enterprise platforms.
- 7:53SPEAKER_00: If a copilot agent influences a credit decision or processes insurance data or manages a workflow that touches employment outcomes, it carries the same obligations as any other high risk AI system.
- 8:13SPEAKER_00: The question then isn't whether that's fair, it's whether your organization knows which of its co-pilot agents fall into that category.
- 8:26SPEAKER_00: And if the answer is, well, we would need to find out that's a gap.
- 8:33SPEAKER_00: So the concept I want to leave you with is the policy bound agent, not as a jargon, but as a design principle.
- 8:44SPEAKER_00: A policy bound agent isn't one where someone wrote a governance policy after it was deployed.
- 8:52SPEAKER_00: It's one where the governance is structural, where the boundaries, the escalation parts, and the audit requirements are part of how the agent was built, not something layered on top of it after the fact.
- 9:10SPEAKER_00: So what does that actually look like?
- 9:14SPEAKER_00: The agent's permitted actions are typically enforced, not just described.
- 9:21SPEAKER_00: There is no discretion about whether to follow the constraint the constraint is in the design.
- 9:30SPEAKER_00: And when the agent reaches the limit of its authority, it doesn't make a judgment call.
- 9:37SPEAKER_00: It stops and it's roots to a human.
- 9:41SPEAKER_00: The escalation path is defined and it actually works.
- 9:47SPEAKER_00: So every action is logged, not selectively, not when someone remembers to turn login on.
- 9:56SPEAKER_00: Continuously from deployment with timestamps and context.
- 10:03SPEAKER_00: And the agent is then monitored over time, not audited quarterly, but monitored.
- 10:10SPEAKER_00: If it starts behaving differently, taking actions it didn't take before, for example, or declining things it used to handle, that's detected and escalated quickly.
- 10:25SPEAKER_00: As you can see, behavioral drift in an agent isn't just an operational issue, it's a governance event.
- 10:34SPEAKER_00: So this is the architecture that makes agentic AI trustworthy.
- 10:40SPEAKER_00: The capability is table stakes.
- 10:43SPEAKER_00: It's the governance layer underneath that determines whether you can actually stand behind it.
- 10:50SPEAKER_00: One of the reasons I built Viridian was specifically for this problem.
- 10:56SPEAKER_00: Because governing an agent is a different exercise from governing a model, and most governance tools weren't built with that distinction in mind.
- 11:08SPEAKER_00: When you register an agent in Viridian, the classification takes into account what that agent can actually do, what tools it has access to, how autonomous it is, whether any of its actions are irreversible, and the regulatory context it operates in.
- 11:31SPEAKER_00: The output maps it against Article fourteen and identifies where the governance gaps are.
- 11:40SPEAKER_00: The copilot studio integration we have with Viridian means that when you connect your tenant, your deployed copilot agent surface automatically and you get an immediate picture of what's running, what it has access to, and what governance it currently has in place.
- 12:01SPEAKER_00: For most organizations, that's the first time anyone has seen a complete list.
- 12:10SPEAKER_00: And the continuous monitoring means that if an agent's behavior changes and if it starts doing things it previously didn't, you know.
- 12:22SPEAKER_00: Before a regulator does, before a customer notices.
- 12:28SPEAKER_00: So Viridian is on Microsoft Azure Marketplace and the link is in the show notes.
- 12:38SPEAKER_00: Agentic AI is not coming.
- 12:40SPEAKER_00: It's already operating inside most enterprises, and often without the governance infrastructure that its role in your operations actually requires.
- 12:53SPEAKER_00: The five questions I walked through today what can it do?
- 12:59SPEAKER_00: What won't it do?
- 13:01SPEAKER_00: What happens at the boundary?
- 13:03SPEAKER_00: Can every action be traced?
- 13:05SPEAKER_00: Can the actions be reversed?
- 13:07SPEAKER_00: These are not theoretical.
- 13:09SPEAKER_00: They are the difference between a governed agent and one that's been deployed and hoped for.
- 13:18SPEAKER_00: If you can't answer them for your agents right now, that's where to start.
- 13:24SPEAKER_00: This week, not next quarter.
- 13:28SPEAKER_00: So in our next episode, which is episode nine of our Viridian special sessions, we will be taking stock.
- 13:37SPEAKER_00: So three months since the EU AI Act's high risk obligations came into full force.
- 13:44SPEAKER_00: What are regulators actually asking for?
- 13:47SPEAKER_00: What compliance gaps are showing up in the first supervisory supervisory conversations?
- 13:55SPEAKER_00: And I'll tell you what I'm seeing.
- 13:58SPEAKER_00: Until then, keep leading intentionally.
- 14:02SPEAKER_00: Thank you.