Latest / CLEs You Actually Want to Hear / Agentic AI Contracting: Legal Frameworks and Emerging Risks
Transcript
- 0:02Hello, everybody. I am Mark Hoag, Chair of the
- 0:06Beverly Hills Bar Association's AI in the Law
- 0:08section, along with my co -chair Amanda Harris.
- 0:11I'm the founder of Axiomic LLC, an AI first web
- 0:14and iOS app development company. And for my day
- 0:17job, I also run, of course, hoaglaw .ai, where
- 0:21I serve as fractional GC to AI first SaaS companies
- 0:24and tech startups. John Pavlotsky is a partner
- 0:28at Stoll Reeves LLP, where he co -chairs the
- 0:31firm's AI privacy and cybersecurity group and
- 0:33technology industry group, counseling clients
- 0:36on data privacy, information security, AI, and
- 0:40related technology compliance, transactional,
- 0:42and dispute matters. Before joining Stoll Reeves,
- 0:46he served as associate general counsel at Roku
- 0:49and group counsel at Intel, and he is a past
- 0:53chair and current advisor. of the IP section
- 0:56of the California Lawyers Association. An active
- 0:59teacher and writer in the field, John has taught
- 1:02technology and privacy law at UC Davis and Santa
- 1:06Clara University School of Law, guest lectured
- 1:08at institutions including Stanford and UC Berkeley's
- 1:12Haas School of Business, and regularly contributes
- 1:15to the Stoll -Reeves Global Privacy and Security
- 1:17blog and New Matter Journal. And now, let's get
- 1:22started. Great, Mark. Thank you for the introduction.
- 1:26I could not have said it better myself. Let me
- 1:30start sharing my slides. We'll get into the presentation.
- 1:35If you have questions, you can add them to chat.
- 1:40And I will take a look at them and address them
- 1:42as we get. So let me share the presentation.
- 1:58And presentation is here. You should have it
- 2:01in front of you. And so let's get going. Agentic
- 2:06AI contracting, legal frameworks, and emerging
- 2:09risks. What in -house counsel need to know? All
- 2:16right, so let's jump in. All right. So I've titled.
- 2:26This slide, and we've got a few that are the
- 2:30initial slides. I've titled this one, that was
- 2:33then, this is now. So then we have 2018, and
- 2:39I mean, I didn't pick 2018 for any particular
- 2:41reason, but for I was passing through one of
- 2:48the gates at Oakland International Airport. passing
- 2:53by an establishment and saw Pepper. Pepper is
- 2:58a Japanese robot. I first saw Pepper in Tokyo
- 3:03in 2016. And Pepper, back then, had the same
- 3:07setup, basically a tablet mounted on it for information.
- 3:14And I've got the caption as a greeter on a stand.
- 3:17It answers. It does not act. This one definitely
- 3:21did not act. So that was that. Fast forward to
- 3:26now, and this was a week or two ago, on the streets
- 3:33of San Francisco, we have an autonomous, or at
- 3:38the very least, a semi -autonomous dog, or I
- 3:47guess robot, and you know full disclosure there
- 3:52were a couple of individuals making effectively
- 3:55their minders were behind it make sure that it
- 3:57was doing the right thing but it was navigating
- 4:01the streets of san francisco on its own and if
- 4:04you've been to san francisco recently and all
- 4:07over you know california you have autonomous
- 4:10cars and um that kind of fits uh kind of squarely
- 4:18into the theme that we'll discuss today. So they
- 4:22are now out in the world, moving, deciding, transacting
- 4:26on their own. So the gap is autonomy. And so
- 4:30that's where the focus of the contract is going
- 4:33to be. All right. So with that, let me move on
- 4:41to our agenda. We'll cover today where agentic
- 4:51AI stands, and I'll talk about agentic AI from
- 4:54a definitional perspective. I will provide a
- 5:01state regulatory update. I'll focus on California,
- 5:05of course, but Colorado too. There are other
- 5:09states that are participating in this process.
- 5:13I'll provide a... brief comparative and looking
- 5:17at laws outside the US as well approach. I'll
- 5:22talk about what makes an agent different from
- 5:25a tool. We'll cover threshold deployment questions
- 5:31and core security risks and I'll build upon an
- 5:36article that I wrote earlier this year on this
- 5:41topic. We will define the agent scope and permitted
- 5:47actions. We will cover access controls, identity
- 5:51and risk allocation. And this might be from a
- 5:55contracting kind of framework perspective, probably
- 6:00one of the most important aspects. I will cover
- 6:04service levels, privacy, liability and security.
- 6:08And then to... Wrap things up. We will go over
- 6:13kill switches and I've got I'm representative
- 6:15language and you can look for that. And suspension
- 6:20terms and then open source agents. You know what
- 6:25happens if your open source agent. Is one or
- 6:30operates on models that you. Found in a public
- 6:35repo. And I'll provide a drafting checklist.
- 6:38And then I have appendices with framework references,
- 6:44a sample term sheet outline, which typically
- 6:46for 10 technology transactions, usually it's
- 6:50overkill. But given where we are right now on
- 6:54the cutting edge, I think it makes sense here
- 6:57and some working definitions. So with that, let's
- 7:02move forward. All right, so pace of change, and
- 7:13I'll start here. This is from a bus parked in
- 7:21front of our local community college in San Mateo.
- 7:25And the caption is, Agenda AI has reached the
- 7:28billboard stage. And if anybody's been... In
- 7:32or around San Francisco, and certainly in San
- 7:35Francisco, there are many, many, many AI billboards
- 7:43or billboards for agents and other AI tools and
- 7:47products. And it seems that each is trying to
- 7:51be more provocative than the last. And so this
- 7:57one reads agents are humans too. I posted this
- 8:00on LinkedIn and got quite the reaction. Not everyone
- 8:06agreed or many disagreed with this proposition,
- 8:10which of course was intended to be provocative.
- 8:15So with that, a few things for the framework.
- 8:19One is drafting assistance. Humans reviewed and
- 8:26executed every step. And so that's where we were
- 8:28in 2018. That's where we were in October of 2022.
- 8:36And that is changing. And if anybody is, or certainly
- 8:41as part of my drafting practice, AI is an active
- 8:45part of it. It is really my assistant in that.
- 8:50And it is a collaborative approach that is used.
- 8:54And so that has changed. Tool use and integration.
- 9:00Models began calling APIs inside workflows. So
- 9:04that's sort of what's happening. And then you
- 9:07move on to kind of the next step, which is autonomous
- 9:11agents. Now systems now hold credentials and
- 9:14act across systems. And this is really the piece
- 9:18that we are focusing on in terms of this program.
- 9:22What if you're representing a customer that is
- 9:26going to be purchasing a solution that features
- 9:31or includes autonomous agents. So with that and
- 9:39the next bit and part of the kind of the framework
- 9:43is autonomy by default. And this is where vendors
- 9:48ship agentic features into existing products.
- 9:52And so big picture agreements negotiated for
- 9:56an assistant may now govern the actor. All right.
- 10:02Let me see if there's anything else. Okay. So
- 10:08that is kind of the arc that we are seeing. All
- 10:15right. So with that, I'm just gonna move something
- 10:19here. Okay. Let's talk about a roadmap, an agentic
- 10:29AI rollout. And so this is generally my thought
- 10:34process where I'm looking at a deal where AI
- 10:39agents are involved or may be involved. First,
- 10:44identify what the agent can access. And the agent
- 10:48can access potentially internal systems. and
- 10:53workflows, it might be able to access external
- 10:56systems and making sure that that handoff is
- 11:01secure. Define the agent and its autonomous actions.
- 11:07So what can it do on its own? What can it do
- 11:13or will it do after it checks in with its principal?
- 11:18And then there, We moved to three which is setting
- 11:22the scope the permitted actions and the approval
- 11:26gates so if an agent is given initial task basically
- 11:32it is when you've got an LLM control Pain and
- 11:40the agent will go out and Then they'll have to
- 11:45be exposed to you have various tool schemas It
- 11:48will go out, it will bring information back,
- 11:51and then it goes to the next step, brings it
- 11:53back, goes to the next step, brings it back to
- 11:55the next step. And the question is, with respect
- 11:59to that process, understanding kind of generally
- 12:02how it works, at what point does approval have
- 12:07to be given? I mean, not necessarily when it
- 12:09kind of goes out and brings something back before
- 12:10it takes any sort of action which could have
- 12:14consequence on the customer. And then from three,
- 12:18we go to allocating liability for actions the
- 12:22agent takes. And so with that, this then dovetails.
- 12:28It kind of goes into the documentation for the
- 12:30agent. And that's something that if you're looking
- 12:33to or negotiating an agreement, it's something
- 12:36that you will want to see. Certainly, if you're
- 12:39the customer and if you're a representative vendor,
- 12:42you want to make sure you have this as well.
- 12:46understanding what it is that the agent is built
- 12:53to do and what are the permitted use cases. And
- 12:58then if a customer goes outside of the permitted
- 13:02boxes, what is the liability for that? Five has
- 13:08to do with a build audit, so understanding how
- 13:12the agent is configured. figured and what it
- 13:16can access, monitoring it, making sure that it
- 13:22is operating as intended. And in terms of the
- 13:27monitoring piece, there's a couple of pieces
- 13:29to it. One piece would be making sure that it's
- 13:34going to other systems with appropriate authorization,
- 13:39that it's going to the appropriate other systems.
- 13:42that it's bringing back information that is useful,
- 13:46but also if it is encountering issues that it
- 13:50doesn't repeat the loop, you know, basically
- 13:54ad nauseum and in which case it doesn't return
- 13:58error, which is then presented to the user because
- 14:01if that happens then no progress is being made
- 14:04and tokens are being consumed. So that's not
- 14:07a good outcome for anybody. And then understanding,
- 14:11okay, when do I as a customer have kill switch
- 14:14rights when basically I turn the agent off. And
- 14:19those are rights that the customer should have,
- 14:22but also in as much as the vendors doing monitoring
- 14:26should have those rights as well. And then that
- 14:29goes to plan suspension or via planning suspension.
- 14:33When is... access to the agent suspended income
- 14:38or the triggers for that. And usually the trigger
- 14:40for that is violation of the acceptable use policy
- 14:43or terms for that agent. What does termination
- 14:47look like? And then in terms of getting your
- 14:51data back and the models back, they're yours.
- 14:54Any improvements to them? And then what does
- 14:56wind down look like? And so with that, really
- 15:04what we do, or at least my approach to it is
- 15:07follow the agent's authority. What is the agent
- 15:11authorized to do rather than a list of clauses?
- 15:16And for those that have been practicing in the
- 15:20data privacy or data protection space for a while,
- 15:25the approach there, I mean, for some time has
- 15:28been follow the data. Understand where it's supposed
- 15:37to go. Is it going there? Is it being processed
- 15:39the way it's supposed to be processed? The paradigm
- 15:42of the framework here is actually quite similar.
- 15:44Where again, you kind of look at what is the
- 15:47agent authorized to do? And is it doing it? And
- 15:54the authorization, kind of how it goes about
- 15:56doing it. You know, there's a couple of options.
- 16:00You can have agent orchestration, and that's
- 16:04for a flexible, unpredictable workflow. Or you
- 16:09can have it hard -coded, where there's basically
- 16:11deterministic, you know, sub -steps where it
- 16:13kind of always goes, you know, the same tools
- 16:15and brings back the same information and such.
- 16:18And again, that's something that if you want
- 16:20kind of greater visibility, the vendor should
- 16:22be able to provide that. All right. So with that,
- 16:30we've got the roadmap. I think we are ready to
- 16:33move into the core of the presentation. Well,
- 16:40actually, let's take one quick step back and
- 16:43look at the regulatory overlay. And then we'll
- 16:48dive into the actual terms. So I have overlay
- 16:52here for California and for Colorado. California
- 16:56has And there's a bunch of AI bills. A couple
- 17:01got signed yesterday. There's still quite a few
- 17:05on government nuisance table to decide on by
- 17:09September 30th. And we have laws on the books
- 17:15that have been in effect at the beginning of
- 17:17this year, if not going back a few years, having
- 17:21to do with disclosure, compliance obligations
- 17:25as such. That all aside, I think some of the
- 17:32most critical obligations will fall under the
- 17:38ADMT, the automated decision -making technology
- 17:42regulations under the CCPA. They're going to
- 17:46affect January of next year, ADMT. is any technology
- 17:51that processes personal information and uses
- 17:54computation to replace or substantially replacing
- 17:57decision -making. So it's kind of as one might
- 18:01expect. And then a risk assessment is required
- 18:06if ADMT is used for a significant decision concerning
- 18:13a consumer. And so that's the logic and the output
- 18:17of the ADMT. or processing consumers' personal
- 18:21information the business intends to use to train
- 18:23an ADMT for a significant decision concerning
- 18:26a customer. And significant decision has one
- 18:30that has to do with the provision or denial of
- 18:34financial and lending services, housing, education,
- 18:37enrollment opportunities, employment, independent
- 18:39contract opportunities, compensation, healthcare.
- 18:43And the requirements are a pre -use notice, access
- 18:46appeal, and opt -out rights. And so where the
- 18:48rubber hits the road is, again, kind of goes
- 18:52to what are the capabilities of and the planned
- 18:55uses of the agent. And so an agent is going to
- 18:57be used in the process to review and possibly
- 19:04decide on or help decide on job candidate applications
- 19:09than a understanding of the agent and its capabilities
- 19:16will be needed, and this would likely be as the
- 19:19Deployer, as the customer being the Deployer
- 19:21here, would require an assessment, and as the
- 19:30assessment would require, would require the participation
- 19:33and cooperation of the developer of the solution.
- 19:37The other very quick things to note is I have
- 19:40a few links here to guidance from the California
- 19:44Attorney General and the Oregon Attorney General.
- 19:47The Oregon guidance preceded the California guidance
- 19:50by that month. So this is actually from end of
- 19:5224, which seems like eons ago. And the crux of
- 19:59it is that this was the case for Oregon, but
- 20:03a lot of it applies for California as well as
- 20:04it. It's basically that just because we don't
- 20:07have a law that says AI in it, doesn't mean that
- 20:13there are not obligations on developers and deployers
- 20:21with respect to AI products. And they're used
- 20:26by consumers, and that can be consumer protection,
- 20:31data privacy. and such. And California had two.
- 20:37This is beginning of 2025. Application of existing
- 20:42California laws to artificial intelligence, and
- 20:45one for health care as well. So as you deploy
- 20:48these solutions, while we do have, so in California,
- 20:53more laws on the books, which are being rolled
- 20:56out. It's still useful to kind of look at the
- 21:00overall backdrop and map that against the solution
- 21:03that's being deployed. Oregon has one as well.
- 21:09OK. All right. Let's see. We had OK. Links to
- 21:18the opinions don't work are bad links. I will
- 21:23provide those. to individuals after. Okay. So
- 21:33let me kind of close this up. Hold on. All right.
- 21:39Let me keep going. All right. Colorado AI Act,
- 21:45the other concept here is similar to the... CCPA
- 21:51ADMT regulations. It goes into effect January
- 21:551 of next year. There are regulations that are
- 21:58being written right now. This replaces a 2024
- 22:03AI law and imposes obligations on developers
- 22:08and deployers as well. And so the developer needs
- 22:18to provide the deployer documentation on intended
- 22:20uses, training data categories, notations, risks,
- 22:24instructions for human oversight, and to notify
- 22:27deployers of material updates. There are consumer
- 22:33transparency rights. So notice when ADMT is used
- 22:39and following certain adverse decisions, a plain
- 22:42language explanation of the technology's role,
- 22:46review and correction rights, and this is enforced
- 22:50by the Colorado Attorney General. So that's it
- 22:59at a high level. I mean, there's a bunch of other
- 23:01AI laws across various states, but the thrust
- 23:07here is effectively its disclosure and use where
- 23:13a consequential decision is being made and, you
- 23:15know, mapping that to what an agent is capable
- 23:18of doing and what it actually does. All right.
- 23:23With that, okay, looks like we've got some links.
- 23:28Good, good, good. Let's move on. All right. So
- 23:33let's talk about Agenda .AI. It's not just another
- 23:36tool. And these are some of the main differences.
- 23:39Autonomy. It acts across steps without per step
- 23:43review. And I talked about where the concept
- 23:46is, you've got a model. It goes to a data source.
- 23:50It comes back. It goes to a different data source.
- 23:53And then it processes it. And then we have an
- 23:57output. And so it will act across steps without
- 24:00per step review. This, as mentioned, this is
- 24:04the big one, access and identity. It holds credentials
- 24:06and system permissions. And real world actions,
- 24:11it sends, modifies, transacts and pays. Now you
- 24:14can have it set up or you can certainly have
- 24:16the contract say that, okay, we are okay with
- 24:20it sending certain information. It can modify
- 24:25certain internal documents per cent perhaps,
- 24:27but it cannot transact, it cannot pay. Or it
- 24:32can do all of these, but it has to be gated where
- 24:36it brings effectively the final or near final
- 24:38product. to the individual and says, OK, do you
- 24:41approve of the actions that I'm going to take?
- 24:43And it's approved. That approval is time stamped,
- 24:47and then the action happens. But that, again,
- 24:51that goes to making sure that the contract is
- 24:55crisp and comprehends what the agent is supposed
- 24:59to do and where those checks are. And the contract
- 25:04defines authority where other rules are silent.
- 25:08we are really at a point where you will, just
- 25:11as with data ownership, which is a fluid concept,
- 25:15rely on the contract to be clear or as clear
- 25:18as you can be in terms of where the meets and
- 25:21bounds are with respect to this technology. And
- 25:26then once you have the contract relatively clear
- 25:29or clear as to that, making sure that there are
- 25:33periodic audits and sufficient access logs that
- 25:40you have to confirm that the tool and the agent
- 25:44is actually doing what it's supposed to be doing
- 25:45and not doing what it's not supposed to be doing.
- 25:48All right. So with that, just gonna move this
- 25:53a little bit. Let's talk about, yeah, from assistant
- 25:59to actor, what changes. And the familiar instinct,
- 26:03I have it on the left, and what actually changes
- 26:05on the right, and this is gonna be a good way
- 26:07to consider the framework. So familiar instinct,
- 26:15it's just another software tool. And so with
- 26:20that, you know, review before sending something
- 26:22to the owner hold, human oversight is periodic,
- 26:26not continuous. And what changes is that the
- 26:29agent holds credentials and acts under them.
- 26:34And actions can have effects before anyone reviews
- 26:37them. And the authority comes from configuration
- 26:41and contract. And so this paradigm, again, depending
- 26:49on what the agent is capable of doing, can have
- 26:55significant consequences. So the agent is one
- 26:58where you... You tell it, OK, every Friday go
- 27:01out and figure out what the weather is going
- 27:02to be before my hike on Saturday and bring that
- 27:05back to me. If it's wrong, then, or if it doesn't
- 27:10do that, the consequences are not significant.
- 27:15If the scope is broader, then there is the potential
- 27:22to have very significant consequences. And understanding
- 27:24that at the outset, becomes quite important.
- 27:31All right. Let's see, and we'll keep going. All
- 27:43right, and so with that, these are key questions
- 27:46that I ask when I'm looking at a contract that
- 27:52involves agentic AI. What authority will the
- 27:57agent have? What can it decide without human
- 28:01approval? Which systems can it reach and how
- 28:07wide is the blast radius? And so that, when I
- 28:11think of that, that's a conversation that I would
- 28:16want to have alongside, if my company has it,
- 28:21my chief information security officer. and make
- 28:27sure that I've got comfort that if it goes into
- 28:33a system, one that it can't go into a system
- 28:35that I don't want it to go into, but then if
- 28:38it does, and if it does something that it's not
- 28:39supposed to do, understand kind of what the consequences
- 28:41might be. What are the guardrails? How are they
- 28:45enforced? How is its behavior audited and monitored?
- 28:50And with that, I mean, the vendor, I mean, should
- 28:53have, Ideally should be able to give the customer
- 28:56a dashboard in terms of the operation of the
- 28:59agent, or at the very least have at least 90
- 29:04days worth of sufficient granular logs as to
- 29:09the activity of the agent. 30 days will not be
- 29:12enough. 90 probably will be okay. 180 days is
- 29:17better. And then the next question is, what is
- 29:21the failure mode and how do you stop it? And
- 29:23again, that's going to vary by solution, but
- 29:27again, this is part of the thought process as
- 29:29the agent is being onboarded. And so the approach
- 29:34would be to answer these first and then run a
- 29:36risk assessment aligned to the NIST AI RMF before
- 29:43deployment and then do it regularly as well after.
- 29:49Five core security risks. And the question here
- 29:56would be with respect to these risks, I mean,
- 29:58certainly you could put in place or put together
- 30:00a questionnaire to provide to the vendor saying,
- 30:03you know, what do you do to help prevent this?
- 30:06And the vendor should be able to answer that.
- 30:10And at the very least, I mean, have a conversation
- 30:13with the technologists of the vendor, if you're
- 30:16the customer. in terms of what is done to help
- 30:20prevent or mitigate this. So the risks aren't,
- 30:25as mentioned already, excessive autonomy, authority
- 30:29beyond what the task actually requires. And we've
- 30:34read about this in some respects in the news
- 30:38where an agent is given direction, and then it
- 30:43decides on its own effectively how to accomplish
- 30:45that task. And some of the steps that it takes,
- 30:50and perhaps the outcome as well, are ones that
- 30:53we would not think that we would take or that
- 30:56it would take. Prompt injection. Untrusted input
- 31:03redirects the agent's plans and actions. So this
- 31:06would be a vulnerability in the tool or the agent.
- 31:12Data leakage. And this would be with respect
- 31:15to credentials or records exposed across systems.
- 31:19So as it kind of goes in and grabs information,
- 31:22it kind of brings it back, making sure that's
- 31:24done securely, lack of accountability, no career
- 31:28record of what acted, why, and whose instruction.
- 31:34And then an unmanaged supply chain. So tools,
- 31:38models, and integrations are inherited without
- 31:41diligence. And so the thought here is that the
- 31:47agent is a non -human actor with its own identity
- 31:51and access profile. And so you would treat it
- 31:54as a potential insider threat. Very quickly here,
- 32:04let's make sure that we cover everything. What
- 32:07is being deployed? And this is my taxonomy for
- 32:10it. We have the agent is being deployed. It's
- 32:14basically a software which executes or accomplishes
- 32:21a task. What are the components? The model, we
- 32:26talked about that. The orchestration layer that
- 32:28kind of tells, kind of directs the agent as to
- 32:32how to accomplish those tasks. And then the system
- 32:36prompts and policies. and then cross -cutting
- 32:40overlay credentials and permissions as the agent
- 32:43goes to one tool, back to the LM, to another
- 32:47tool back to the LM or within through a loop,
- 32:53external loop or internal loop. And then the
- 32:56connections, so that's tools and integration,
- 32:58memory and state, logs and audit, the trail and
- 33:03autotrail and the human approval gates. So we
- 33:06touched upon this and... making sure that we
- 33:08understand this conceptually becomes critical
- 33:12from the contracting perspective. From instruction
- 33:19to action, yeah, how does the agent work? And
- 33:25so I've broken it down to kind of this taxonomy.
- 33:30So first it was to plan and reason. So you have
- 33:34the task, the task is decom... posed into intermediate
- 33:40steps. The next is calling tools and systems.
- 33:45So that's API and system calls. Record reads
- 33:51and writes. So read to and can write to another
- 33:54system. And then retrieved context. And then
- 33:58it acts in the world. And it can send a message.
- 34:02It can change a record. it can initiate payment.
- 34:09And I have down here, I think this is absolutely
- 34:12critical, human approval is a design choice,
- 34:15not a default. And so one might think, okay,
- 34:21if a message is going to be sent depending on
- 34:23the message, perhaps I can be maybe a little
- 34:30more lax in terms of my configuration and kind
- 34:33of... kind of strictness of the system, depending
- 34:37on the message. But for these other ones where
- 34:40you're changing a record, and again, this goes
- 34:44to the integrity of a system, or certainly initiating
- 34:49a payment, then obviously having a human approval
- 34:52date is critical, but also making sure that there's
- 34:56very high confidence in the operation of the
- 34:58system becomes very important. All right. I've
- 35:04got a few concepts and tips here, defining the
- 35:13agent and autonomous action. And so the agent
- 35:17is the configured system granted authority to
- 35:23act, and then other provisions key off it. And
- 35:26the autonomous action, I mean, there's ways to
- 35:29define it. A narrow way would be an action taken
- 35:36by an agent with that prior human approval of
- 35:39that action. So that's one way to kind of describe
- 35:43how an autonomous action might be. A broad way
- 35:47to describe the autonomous action of the agent.
- 35:50It would include tool calls, record changes,
- 35:53external communications, and such. And depending
- 35:58on the perspective on whether you're a vendor
- 36:01or a customer, you might have different preferences.
- 36:04A vendor, for example, could prefer a narrow
- 36:08agent and narrow autonomous action, only one
- 36:12to avoid potential liability, but also to have
- 36:18more control over what the agent does. And a
- 36:20customer might want the reverse. And so that
- 36:25conversation has to happen to be reflected in
- 36:28the contract. Scope and permitted autonomy, and
- 36:35this is really the core, or at least kind of
- 36:39should be the core of the discussion consideration.
- 36:44Permitted focusing on specifying the tasks and
- 36:48the systems in scope. Defining which actions
- 36:53require human approval. And specifying. spend
- 37:01volume and rate limits. And from a vendor's perspective,
- 37:11since the actions by any sort of AI action, it
- 37:18costs money to run through the consumption of
- 37:22tokens and also in terms of the requirements
- 37:29of the systems. making sure that these are specified
- 37:33become support. And then prohibited or restricted,
- 37:36for example, this would become what you would
- 37:38expect, external communications on a customer's
- 37:41behalf, irreversible record changes and deletions,
- 37:45payments, commitments, and credential sharing.
- 37:48And the recommendation here is to avoid general
- 37:54business tasks as the permitted. actions rather
- 38:02yet to name the actions. And so that way there
- 38:06isn't, there's clarity or at least as much as
- 38:08can be asked between the vendor and the customer
- 38:10as to what is permitted and within scope. And
- 38:15there isn't a question two weeks after the agreement
- 38:18is signed, can I do this with this agent? Do
- 38:21I need to go talk to the vendor? Do I need to
- 38:23go ahead and amend the agreement? Try to be as
- 38:26clear as possible upfront. I've got an illustrative
- 38:34table here, and this has permitted and prohibited
- 38:37agent actions. So for example, you could have
- 38:40it where in this column, expressly permitted,
- 38:45reading internal data and drafting communications.
- 38:48Okay. Permitted with conditions, sending external
- 38:52communication. So perhaps we have that. Maybe
- 38:55you'd have a human gate approval there. modifying
- 38:58system records, same thing. And then you can
- 39:02have it set up where, regardless if there was
- 39:05a human gate or not, it is prohibited in terms
- 39:09of initiating payments or transfers. It cannot
- 39:12do that. There's not even a step for a human
- 39:15to say, okay. And so that needs to be clarified
- 39:20up front. And that can be contractually mandated.
- 39:26So I've got language here, for example, a vendor's
- 39:28agent may act autonomously only within the approved
- 39:33task scope and may not send external communications
- 39:37and issue payments without documented human approvals.
- 39:40I mean, that's one variation. There are two,
- 39:44I mean, there are decisions to be made into the
- 39:46conversation, the client be like, okay, what
- 39:48do you... Are you okay with being done without
- 39:52human approval? What are you okay being done
- 39:55with human approval? What are you, or what actions
- 40:00cannot be done, period, regardless of whether
- 40:04or not there's human approval. Having that conversation
- 40:06up front, perhaps reducing it to a mini term
- 40:10sheet and then keying the drafting based on that.
- 40:15So that's one potential recommendation. Okay,
- 40:21access controls and identity contracting lessons.
- 40:25So I've got a few lessons here kind of things
- 40:27to think about as you draft these contracts.
- 40:31One is that the agent inherits whatever access
- 40:34you grant. And so if you provide over broad service
- 40:42accounts, you know, can turn a small error into
- 40:47a large incident. So understanding that piece
- 40:50becomes critical. Second, least privilege has
- 40:54to be contractual. So scope, credentials, and
- 40:57permission changes belong in the agreement, not
- 40:59only in the configuration. You get them both,
- 41:02but here's the thing. You've got the configuration,
- 41:06and lawyers typically will not see that. You've
- 41:08got the agreement that says, OK, it's least privilege,
- 41:12so only the minimum amount of, yep. people can
- 41:17go ahead and, I guess, want to access it, but
- 41:20also only the very limited task can be done until
- 41:29it gets escalated for broader approval. The question
- 41:34then is, okay, let's say you've got to configure
- 41:37and you think correctly. Let's say you've got
- 41:39language and agreement. How do you know that
- 41:42it has been implemented? properly and correctly.
- 41:48And so then that goes to audit rights, making
- 41:50sure that you've got access logs to be able to
- 41:52go ahead and follow that. And you could see this
- 41:59potentially on the back end. I see this once
- 42:03in a while kind of in the cyber incident context
- 42:08where it becomes known that the credentials that
- 42:13somebody had were credentials they should not
- 42:16have had, or the privileges were broader than
- 42:18they should have been, and then that caused there
- 42:21to be issues. So, great. I mean, important to
- 42:25have the language of the contract and the actual
- 42:26configuration, but it has to be a way to monitor
- 42:28it and audit it as well. Lesson three, attributions
- 42:34contested. The logs must show which actions the
- 42:38agent took, under whose identity, and on whose
- 42:41instruction. And so you could ask, for example,
- 42:45the vendor, you know, show me sample access logs,
- 42:51you know, or you can obfuscate any confidential
- 42:54information. Tell me what it looks like. Because
- 42:58again, I mean, if there's going to be, if there's
- 43:01an issue, whether it's with a customer or a regulator,
- 43:06because there's been an incident, you will need
- 43:09these to be able to show what happened and why.
- 43:12So this becomes a critical artifact. Okay. Allocating
- 43:19risk against agent actions and artifacts. We
- 43:24have these again. So for example, for autonomous
- 43:26actions, you have from the vendor position, the
- 43:31customer directs and accepts the action. And
- 43:35customer might be okay. Vendor is accountable
- 43:37for agent behavior. And so where Yeah, the happy,
- 43:43I guess the compromise, the area where folks
- 43:46meet usually is okay. I will be responsible,
- 43:50the vendor will be for the agent's behavior so
- 43:53long as it is compliant, you know, the documentation
- 43:57that I have and with how it is intended to be
- 44:00used, I mean, as documented in my documentation.
- 44:04And then if the customer uses it, for a purpose
- 44:10that it's not in the documentation. Again, assume
- 44:13the documentation is sufficiently detailed or
- 44:16in violation of the acceptable use policy or
- 44:19somehow kind of goes off the rails. As a result
- 44:22of that, then the responsibility lies with the
- 44:25customer. Logs and audit trail, the vendor might
- 44:30say, okay, I will give you some reporting and
- 44:33then there's going to be limited retention. and
- 44:36the customer might want full exportable logs
- 44:39with defined retention. For credential and scopes,
- 44:46the vendor might want broad access for reliability.
- 44:51You'd be able to monitor service system and the
- 44:56customer might want or probably will want least
- 44:58privilege and customer control. The conversation
- 45:02that happens here is, yeah, fine. If you're the
- 45:05customer, you'll get a sample vendor agreement,
- 45:08and it's got a language, let's say, that reflects
- 45:10this column, vendor position column, and certainly
- 45:15it's fine to mark up the agreement and to reflect
- 45:19the desired state from the customer's perspective,
- 45:21but ultimately there probably will need to be
- 45:23a conversation to understand what do you, for
- 45:26example, vendors minimum need to be able to service
- 45:30the tool and can that be done in a least privileged
- 45:36setting? You can read the rest of these on your
- 45:43own. I just want to keep moving. Just make sure
- 45:46that we cover everything. Moving on to then to
- 45:53authority verification and human oversight. And
- 45:58I have it here framed as customer ask and vendor
- 46:02give. So the customer ask might be a clear statement
- 46:06of what the agent may do without approval. Representations
- 46:12that actions stay within the approved scope.
- 46:15Verification, human loop gates for high impact
- 46:18actions. All of this can be engineered into the
- 46:21contract. And the vendor. might respond that
- 46:27the commitments are qualified by configuration
- 46:30and customer instruction, which I think is sensible.
- 46:34Guardrails will be implemented to the extent
- 46:36supported by the platform. The question then
- 46:38is, OK, what is supported by the platform? And
- 46:41then oversight is addressed through documented
- 46:43controls or disclaimers. And so the take home
- 46:48message here is that authority and oversight,
- 46:53I mean, the risk, I mean, it has to be allocated
- 46:55and the guardrails are negotiated, not assumed.
- 47:02All right, I want to touch briefly on service
- 47:05levels for agentic AI. And I have it broken down
- 47:08as to what to measure. And maybe taking a step
- 47:11back, I mean, service levels, SLAs, they're common
- 47:15in SAS, software as a service agreements. They've
- 47:19been around for the past. 20 years or so, if
- 47:23not more. With respect to agents, a lot of the
- 47:27concepts carry over. But there's some tweaks
- 47:33or fine -tuning that should be appreciated. And
- 47:37so with that, the measurements, I mean, there
- 47:43are a number of choices here. There's some for
- 47:45your consideration. We have, for example, API
- 47:49and platform availability, to measure monthly.
- 47:54And I've seen contracts where it's measured quarterly
- 47:57or measured annually, which does the customer
- 47:59absolutely no good. And excluding narrow maintenance
- 48:04windows. Latency and task completion, not just
- 48:10uptime of the endpoint. Maybe the uptime of the
- 48:13endpoint can be quite high, but if it takes a
- 48:16long time to... complete a task or if there's
- 48:20a delay in terms of it being actioned, that is
- 48:24problematic. Let me, I've got a question. Ooh,
- 48:32let me jump in the question now. And then I'll
- 48:36move on. The question was, my thoughts about
- 48:40the best way for lawyers who are not engineers
- 48:42or programmers to get the speed on the various
- 48:44technical issues and considerations. It seems
- 48:48that the clients need to have additional expertise
- 48:52on these issues as opposed to more lawyer -driven
- 48:55negotiation process. That is correct. I think,
- 49:06yeah, one way to look at it is this by definition
- 49:10has to be more collaborative. And I think part
- 49:18of the issue or the challenge here is you have,
- 49:25I mean, your client on the business side, the
- 49:28client could be head of finance, or it could
- 49:35be somebody else who might not have a complete
- 49:40technical appreciation for the solution. And
- 49:44so then it's, figuring out on the client side
- 49:48who are the technical folks. I mean, certainly,
- 49:52you know, there's going to be, for most companies,
- 49:55there's going to be a chief information security
- 49:56officer, a chief technical officer, a chief information
- 50:00officer, and, you know, their team that could
- 50:02be engaged for this, certainly for the security
- 50:05part. And then on the client side, If the client
- 50:14perhaps doesn't have, you know, complete kind
- 50:15of technical appreciation for this, then certainly
- 50:19with whoever they have on their technical end,
- 50:22they will have somebody, you know, having a conversation
- 50:26and want understanding of what this solution
- 50:30is intended to do. And then at that point, I
- 50:36guess pressing, might be too strong a word, but
- 50:39I'll kind of use it anyways. the vendor to provide
- 50:43documentation and show how it accomplishes these
- 50:47things. And when I show the term sheet toward
- 50:50the end, I think that might help answer the question.
- 50:54Let me see, I think I saw a couple other things
- 50:57in chat. Okay. All right, let me keep going then.
- 51:10Okay, and then the other thing to measure is
- 51:13accuracy, escalation, and human approval response
- 51:17for gated actions. If I need approval from a
- 51:21human, how long is that going to take? And availability
- 51:25targets, monthly uptime. This is typical. And
- 51:34service credits are not a remedy for an agent
- 51:38that acts wrongly. So it kind of goes outside
- 51:42of its bounds. So that should be paired with
- 51:47suspension and termination rights. And probably
- 51:51also goes without saying that a credit for there
- 51:55being an uptime violation, certainly at the end
- 52:00of the contract term, where there's not going
- 52:03to be a renewal is going to be useless. So at
- 52:05that point, a refund should be negotiated. Okay,
- 52:11privacy and downstream restrictions. I have the
- 52:15construct here. You have the customer, which
- 52:21is the data owner. You've got the agent, which
- 52:25is the solution being provided by the vendor.
- 52:31And then you've got tools and sub -processors
- 52:33and making sure that one person information is
- 52:36going to be processed. You have a purpose limitation,
- 52:41service provider and contraction terms, action
- 52:46level restrictions, and information, language
- 52:49about transfers and sensitive data. Again, follow
- 52:52the data. If you have a data processing addendum
- 52:55as part of your typical SAS agreement, maybe
- 52:57that can be repurposed for an agentic AI solution.
- 53:03And again, it needs to reflect what access will
- 53:07the agent have to parse information and what
- 53:10will it be doing to it and then making sure that
- 53:16there are appropriate protections with respect
- 53:18to processing that information. All right. We
- 53:25are getting close to the end. So let me touch
- 53:32on a few things and then I'll go through the
- 53:34resources that I've included and happy to take
- 53:37questions as time permits here or after as well.
- 53:42Liability cap, this should be evaluated against
- 53:45what the agent can actually do. So again, agents
- 53:48just kind of go into internal records and summarize.
- 53:53Yeah, you might have a summary that's not accurate,
- 53:57or it might not be timely, but you can go ahead
- 54:00and the individual customer go ahead. check that
- 54:04and so that should not lead to a normally high
- 54:07liability cap. Typical super caps or carve -outs,
- 54:12unauthorized actions, privacy, confidentiality,
- 54:15willful acts. Willful acts typically cannot be
- 54:20limited anyways by a cap, so that kind of comes
- 54:24out. Confidentiality usually is carved out. I've
- 54:28seen a super cap for that and that doesn't make
- 54:31sense at least to me. Privacy is usually a super
- 54:35cap, unauthorized actions. I think it depends
- 54:37on what they are. But depending on what the agent
- 54:41can do, what it actually does, it might make
- 54:45sense to have that fully carved out. Indemnity
- 54:50for third party claims, privacy violations, warranty
- 54:55breach, maybe. And then agent -specific exposure,
- 55:01what can the agent do that we don't want to do
- 55:04where there's potential significant liability,
- 55:06erroneous communications, transfers, record changes.
- 55:10And so this becomes, I think, in part a bit of
- 55:15a brainstorming session. But at the same time,
- 55:17one cannot, as an advisor, spend half a day trying
- 55:21to figure out what an agent can or cannot do.
- 55:23And so really it boils down to having a conversation
- 55:25with the client and looking at the documentation
- 55:28from the vendor to get clarity on that. Okay,
- 55:35I've got some information here about security,
- 55:39available standards, incident response, notification
- 55:43timelines and procedures, reporting for unauthorized
- 55:48or out of scope agent actions. So we've done
- 55:50reporting where there's been a security incident
- 55:55or a data breach. This is a similar concept,
- 55:58but it's with respect to different kind of foundation.
- 56:07Now, you could have an out -of -scope agent action
- 56:10leads to a data breach, in which case you've
- 56:13got not reporting from both. And then allocation
- 56:18continuous monitoring and log retention. Also,
- 56:24sub -processor tool and data location obligations.
- 56:27That kind of goes to the concept of data sovereignty,
- 56:29which I'm not going to cover here, but it's quite
- 56:32topical these days. Wrapping things up, I just
- 56:39want to talk about how kill switches work. And
- 56:46the question is, mapping this out with respect
- 56:50to the solution in hand. And so what would happen?
- 56:55So for example, for agent access, you've got
- 56:58a couple of options. One is credentials can be
- 57:01revoked immediately, or there could be narrow
- 57:06read -only exceptions. So you can't write anything.
- 57:13In -flight options, I mean, do you halt it? Do
- 57:16you stop it? Do you roll it back? Do you let
- 57:17it complete? Define by action type. And the membrane
- 57:21logs, do you retain them? Do you restrict them?
- 57:24Do you delete them? That's negotiated. And so
- 57:27the question of the kill switch is, yeah, who
- 57:28may trigger it? How fast does it take effect
- 57:31and what it stops? And you think about it that
- 57:35both the vendor and the customer, I mean, depending
- 57:39on the tool, it should have access to it. And
- 57:43this question needs to be answered such that
- 57:46it can be actioned and implemented. I've got
- 57:53here sample kill switch language. I could ask
- 57:56this one from time to time. So you could take
- 58:00a look at that. And the questions is kind of
- 58:02touched on. Who may trigger it? How fast does
- 58:07it take effect? And what survives it? And again,
- 58:10this is representative, but you can take a look
- 58:12at that and customize it to your deal. And then
- 58:18wrapping up, since I think we're almost out of
- 58:22time, what happens if you get or use an open
- 58:28source agent from a public repo, public repository?
- 58:34And folks have been doing open source licensing
- 58:36for some time. This is all familiar territory.
- 58:39Some respects, the short of it is with the agent
- 58:45and the tool, you get what you get as the expression
- 58:49goes. So no warranty, no indemnity, no service
- 58:52commitments. This one is a bit trickier. No vendor
- 59:01to suspend the agent or answer for its actions.
- 59:06This, you might... have you veer toward a commercial
- 59:10solution with support. Now you can get a free
- 59:14version with support to this package, so that's
- 59:17possible, but this might give you some pause.
- 59:21And then no diligence on the tools and dependencies
- 59:24it pulls in. Commercial solutions supposedly
- 59:27have that, but something to take into consideration.
- 59:32And then what you do before deployment, confirm
- 59:34the license terms, including copy left and attribution,
- 59:39pin versions and review dependencies, models,
- 59:41and tool integrations, and then run the same
- 59:45access and proof and logging controls you approve
- 59:47of vendors. So the controls have to be internal
- 59:51here. Okay, we are almost at the end. So I've
- 1:00:00got a few items, reference items for you. One
- 1:00:05is a diligence and drafting checklist. Define
- 1:00:11the agent. And again, this should be reviewed
- 1:00:13by now in its autonomous actions. Map the systems,
- 1:00:18tools, and data it can reach. So this is a data
- 1:00:20mapping exercise. You know, folks did that back
- 1:00:24in the day with the data protection space, the
- 1:00:27GPR, or CCPA. This should all be familiar. Set
- 1:00:32least privileged credentials and scopes. Require
- 1:00:35human approval for high -impact actions. Specify
- 1:00:38logging. As mentioned, 30 days will not be enough.
- 1:00:4290 days is probably okay. 180 days is better.
- 1:00:45Retention and audit rights. Secure monitoring
- 1:00:50and incident notification. Carry privacy purpose
- 1:00:54limits and flowdowns. align liability indemnities
- 1:00:58and carve outs with the agent's authority, and
- 1:01:00plan the kill switch suspension exit. So that's
- 1:01:06the checklist. This is me. You've got the slides.
- 1:01:13I've got appendix A1 is framework and authority
- 1:01:16reference. You can take a look at that at your
- 1:01:19leisure. The NIST AI RMF is still the key document.
- 1:01:24From a risk management perspective, it's currently
- 1:01:28being updated. I have definitions here. Take
- 1:01:34a look at that. And key negotiation points. And
- 1:01:42our use terms. I've got the definitions. And
- 1:01:48I have a term sheet here as well. So I will.
- 1:01:52File up on that. Maybe that was perhaps an earlier
- 1:01:55version, but I can provide that separately. And
- 1:01:59that is it. Let me see. I've got a couple of
- 1:02:03questions. I know we're a little bit over. Let's
- 1:02:06take a look. Where can insurance options be used
- 1:02:13for accuracy failures and unauthorized acts?
- 1:02:16Excellent question. Take a look. at your policy,
- 1:02:24there are some policies, and this was, I don't
- 1:02:28know, maybe half a year ago or a year ago where
- 1:02:32AI was effectively carved out. I think that's
- 1:02:37beginning to change. And so looking at the policy,
- 1:02:47I guess it would come under probably a rider
- 1:02:49under a technology and errors and emissions policy.
- 1:02:58Take a look at that and would take a look at
- 1:03:01that and make sure, I guess, one, there's coverage,
- 1:03:03second, the retention isn't sky high, such that
- 1:03:07it's effectively useless, that you've got a high
- 1:03:09enough limit in case there's an incident. And
- 1:03:16so that's kind of one piece in terms of ensuring
- 1:03:19for the customer, but also asking the vendor
- 1:03:22to understand what insurance it has. Certainly
- 1:03:24it will have, as most do, commercial general
- 1:03:28liability insurance. There's going to be tech
- 1:03:30E &O insurance. I mean, most folks these days
- 1:03:33have cyber as well. And then adding language
- 1:03:38to the agreement that says, okay. you've got
- 1:03:44your cyber policy or tech policy, and you do
- 1:03:48not have a car route or an exclusion for, you
- 1:03:54know, Gentic AI solutions, and more broadly for
- 1:03:58contractual liability. So that would be my recommendation.
- 1:04:05All right. We are over time. So thank you for
- 1:04:10joining me Oh, I did want to mention very briefly
- 1:04:13for those that are still on just love to get
- 1:04:16a few on That and I've provided obviously kind
- 1:04:20of the US approach We have yeah, we operate globally
- 1:04:26so I've got reference to the EU AI act and there's
- 1:04:30Different countries take different approaches.
- 1:04:33You know, Japan has more of a framework approach
- 1:04:36and kind of relies more on the call soft law.
- 1:04:42Australia has a flexible framework right now.
- 1:04:48There is a document called agentic AI addendum
- 1:04:51to the AI technical standard for the Australian
- 1:04:56government. And so that would be voluntary as
- 1:05:00to any sort of your private entity. But again,
- 1:05:03kind of shows that There are differences and
- 1:05:10different approaches. And if you do have a cross
- 1:05:13-border deal, understanding what might be the
- 1:05:16local limitations or whatever you've negotiated
- 1:05:19could be helpful. All right. So with that, I
- 1:05:24will stop sharing. And thank you for joining
- 1:05:28me. If there are any questions, you have the
- 1:05:31slides, please feel free to reach out to me and
- 1:05:34happy to provide sample language or discuss this
- 1:05:40topic more broadly. This concludes today's presentation.
- 1:05:46Thank you again to John for an excellent presentation
- 1:05:49today, and thank you so much to our audience
- 1:05:51for joining us. Have a great day, everybody.