Latest / CLEs You Actually Want to Hear / Californias New Chatbot Laws: Compliance and Enforcement After the 2025 Session
Transcript
- 0:03Thank you all for being here. I'm Mark Hoag,
- 0:06chair of the Beverly Hills Bar Association's
- 0:08AI in the Law section, along with my co -chair
- 0:11Amanda Harris. I'm the founder of Axiomic LLC,
- 0:14an AI first web and iOS app development studio.
- 0:18And for my day job, of course, I also run hoagelaw
- 0:20.ai, where I serve as fractional GC to AI first
- 0:24startups and tech companies. I'm delighted to
- 0:27introduce our speaker today, Christabel Randolph.
- 0:29Christabel is an associate director of the Center
- 0:32for AI and Digital Policy in Washington, D .C.,
- 0:36where she leads work on U .S. AI law and policy
- 0:39with a focus on AI governance, regulatory accountability,
- 0:43and the protection of fundamental rights. An
- 0:46attorney with nearly two decades of international
- 0:49legal and policy experience, she's authored commentary
- 0:52in the Journal of AI Law and Regulation and legal
- 0:56volumes published by Kluwer. So thanks very much
- 0:59indeed, Christabel, for being here. Let's get
- 1:02started. Hi, Mark. Thanks so much for that. It's
- 1:08really great to be joining everyone this evening.
- 1:10This is a hot topic. California has really been
- 1:13the leader in AI regulation in the US, enacting
- 1:17laws and, of course, a spate of companion chatbot
- 1:21laws. So moving on, just to give a quick update
- 1:25of introduction of our organization, Center for
- 1:30AI and Digital Policy, CADIB. We are an independent
- 1:34nonprofit. We advise policymakers, not just in
- 1:38the US, but across the EU and globally. We've
- 1:42interacted with several international institutions,
- 1:45also the OECD, the European Parliament, UNESCO.
- 1:49We have some publications which you can see on
- 1:53screen. Our AI index covers AI policy and laws
- 1:57in 90 countries. These are all available online.
- 2:01So the perspective from which I will be approaching
- 2:05the content and the subject of today's webinar.
- 2:08really builds on the work that we've done in
- 2:11the US and also in other jurisdictions and what
- 2:14we've seen in terms of the legislation that has
- 2:17come out of California and what this means for
- 2:20compliance and enforcement. Moving on to jump
- 2:26right in. There is, as I mentioned at the start,
- 2:30there's a lot to pack in. There are two significant
- 2:33bills that I will cover in detail. One is SB
- 2:37243, which was actually enacted last year but
- 2:41went into effect from this year and now is changed
- 2:45by SB 1119, also known as Adams Law, which Governor
- 2:51Newsom signed into law this year and which amends
- 2:56SB 243 and also sets out a comprehensive framework
- 3:01for compliance to ensure child protections and
- 3:05child safety. We'll cover some of the gray areas,
- 3:10talk about what we're seeing in terms of not
- 3:12just enforcement pathways in the laws themselves,
- 3:16but also what we're seeing across states and
- 3:19at the federal level. in terms of enforcement
- 3:23actions. We'll talk a little bit about how we
- 3:26initiated this work in 2023, when we filed the
- 3:29first complaint with the Federal Trade Commission
- 3:31with regard to generative AI and CHEN GPT, which
- 3:35led to the investigation in 2023 and now a revived
- 3:38investigation, which was announced just two weeks
- 3:41ago. And then of course, conclude with some takeaways
- 3:45in terms of compliance priorities. So moving
- 3:48on to the first item of SB 243, and this is a
- 3:56hook. I think this really identifies what are
- 4:03the thorny issues that will come up, because
- 4:06while these laws cover companion chatbots and
- 4:09they set out child safety frameworks, but some
- 4:12of the language would also apply to companion
- 4:17chatbots that are used even by non -minors used
- 4:21by adults. So if you offer a general purpose
- 4:24assistant that remembers users' talks in the
- 4:27first person, is it a companion chatbot? Will
- 4:30it apply to those kind of tools and products?
- 4:36Moving on to SB 243, yes. So here really is the
- 4:43first, this is the law that defines what is a
- 4:48companion chatbot. I think if we look at the
- 4:52definition that was set out last year, it talks
- 4:55about an AI system with a natural language interface,
- 4:59conversational abilities that gives adaptive
- 5:02human -like responses that is capable of meeting
- 5:05a user's social needs. It clearly mentions anthropomorphic
- 5:11design features. and the ability to sustain relationship
- 5:16across multiple interactions or across multiple
- 5:20chat sessions. So whether or not a companion
- 5:24chatbot is covered by SB 243 comes within the
- 5:29scope of the definition, turns on the capability
- 5:33and the functionality of the system or tool.
- 5:35It does not matter how the product is marketed,
- 5:38what is the marketing terminology that is used,
- 5:41The test is really whether it conforms to the
- 5:45functions that have been defined in the law.
- 5:48It also has some exclusions. The scoping is really
- 5:52important to tackle what are the subsequent compliance
- 5:55obligations. So it says chat bots used only for
- 5:59customer service business operations. So anything
- 6:03that you have in -house or internal productivity
- 6:06research or technical assistance would be excluded.
- 6:10chatbots that are built into video games that
- 6:13provide just you know general responses or directions
- 6:17during the games but do not in any way engage
- 6:20in any conversation or any further conversational
- 6:23aspect with users are excluded and of course
- 6:27standalone voice assistants that do not sustain
- 6:30a relationship or generate outputs are also excluded.
- 6:35So The duties really run to the operator under
- 6:40SB 243, so the person who makes the chat bot
- 6:43available. This could be a developer or a deployer.
- 6:48And of course, that would be a case -by -case
- 6:50determination whether an operator is a developer
- 6:54or a deployer. But regardless, if you are an
- 6:58open AI, providing chat GPT, you would be covered.
- 7:02If you are a company that is building off of
- 7:05one of the open source or closed source AI models
- 7:08and then providing your own customized companion
- 7:12chat bot, you would be considered as an operator
- 7:17under the law. Moving on from the scope to the
- 7:23real compliance obligations that SB 243 sets
- 7:27out. And in the slides, I've also highlighted
- 7:30what remains unchanged from last year and what
- 7:34are the changes as a result of SB 319, our Adams
- 7:39law that was passed this year. There are some,
- 7:44this really condenses the kind of compliance
- 7:47obligations that the law has set out. The main
- 7:52difference between SB 243 and Adam's Law, which
- 7:55was passed this year, is SB 243 is primarily
- 8:01built on a lot of disclosure obligations and
- 8:05built on a lot of notice obligations, whereas
- 8:08Adam's Law really sets out a real a real framework
- 8:15which relates not just to disclosure, but design.
- 8:18So for SB 243, the first thing that is required,
- 8:22the first compliance obligation is clear and
- 8:25conspicuous notice that the bot is an AI. We've
- 8:29seen that come up in several times. This is,
- 8:32again, relates to the definition, which brings
- 8:37anthropomorphic design or anthropomorphic conversational
- 8:41chatbots within scope. to address that issue.
- 8:44The first requirement is clear and conspicuous
- 8:46notice so that nobody is misled or confused that
- 8:51it is an AI and not a human at the other end.
- 8:55The second is no engagement unless the operator
- 8:58maintains a protocol. So you cannot really release,
- 9:03deploy, market, commercialize. companion chatbot
- 9:08unless you ensure that you've built in protocols
- 9:12to prevent suicidal ideation, self -harm content.
- 9:17So in terms of whether you're a developer or
- 9:19a deployer, you have to build in those safeguards
- 9:22with regard to the outputs that would be provided
- 9:25by the companion chatbot. There is also a real
- 9:30knowledge element here. So for example, when
- 9:33The operator knows that as a minor, there is
- 9:37a requirement to provide the default breaks at
- 9:41every three hours. So we see that also with some
- 9:44of the other social media laws that were enacted
- 9:46to trigger breaks at certain intervals and to
- 9:51provide notice also to parents that is flagging
- 9:57the continuous nature of the engagement. The
- 10:01Office of Suicide Prevention has certain responsibilities.
- 10:05They have to provide crisis referrals and protocols.
- 10:10They have to ensure that there is an annual report
- 10:13filed with the Office of Suicide Prevention,
- 10:15which then has the responsibility to review whether
- 10:19the crisis referrals, the crisis prevention protocols
- 10:22are. Adquit, according to SB 243, disclose on
- 10:29the app or if you're using an app version or
- 10:31a browser version that chatbots may not be suitable
- 10:34to some minors. It provides for a private right
- 10:37of action and sets out certain penalties with
- 10:42regard to the damages and the remedies are cumulative.
- 10:45So what's really changed with regard to This
- 10:50year is the provision regarding the design, which
- 10:56is the AI disclosure and the breaks that has
- 11:00been replaced by Adam's law. It's not that the
- 11:0414 behavioral requirements have been completely
- 11:09removed, but it has been expanded a lot more
- 11:13in this year's law, the design requirements to
- 11:16ensure compliance. For the most part, it's been
- 11:21unchanged. The first reports, as you can see,
- 11:25to the Office of Suicide Prevention are due on
- 11:28July 1, 2027, which is next year. So it gives
- 11:32roughly about one year to comply, which is a
- 11:39shortened and compressed timeline, especially
- 11:41considering the enhanced design compliance requirements
- 11:45set out by SB -3199. Moving on to really Adam's
- 11:54Law and the journey that we've seen, when we
- 11:57were engaging with the California legislature
- 12:00last year, there was a bill being led by assembly
- 12:05member Rebecca Bauer -Kahan, AB 1064, also known
- 12:09as the Lead for Kids Act. That was the same time
- 12:14SB 243 and AB 1064 were two of the key child
- 12:20safety companion chatbot bills that the legislature
- 12:24was considering. SB 243 got enacted, and it cleared
- 12:31House and Senate, Assembly and Senate, and Governor
- 12:34Newsom signed it into law. AB 1064 also cleared
- 12:39the assembly and senate and went to the governor's
- 12:42desk, but he vetoed it. One of the reasons that
- 12:46at that time the governor vetoed it is, in his
- 12:49opinion, AB 1064 would have barred making companion
- 12:54chap bot available to miners. So that AB 1064
- 12:58really actually set out a ban or a prohibition.
- 13:03regime as opposed to what we're seeing now in
- 13:07Adam's law, which is a design compliance regime.
- 13:12And the governor was concerned that that might
- 13:14really bar legitimate uses, for example, in the
- 13:17case of education or other useful uses at that
- 13:23point in time. However, based on the governor's
- 13:27veto message, the assembly and senate members
- 13:32got together and SB 1119 is a reworked version
- 13:36of last year's SB 1064 because it does not, it
- 13:42imposes broad requirements but does not impose
- 13:46broad restrictions which SB 1064 tried to do.
- 13:52So long story short. Adam's Law is a reworked
- 13:57version of the Lead for Kids Act that did not
- 14:00pass last year. There is one ban that did pass,
- 14:06which is the ban on toys with companion chatbots.
- 14:09There was a lot of conversation. Some of you
- 14:12might have noticed the news of Mattel and some
- 14:15other. toy manufacturers providing toys with
- 14:19companion chatbots built in. There were news
- 14:23reports that highlighted some of the unsafe or
- 14:30inappropriate responses. Again, the conversational
- 14:34safeguards were not built in to the companion
- 14:38chatbots that were embedded into toys. So that
- 14:41resulted in California legislature passing an
- 14:44outright ban on toys with companion chatbots
- 14:48until 2031. Moving to our next slide to Adam's
- 14:56Law. So what this is really setting out what
- 15:02Adam's Law does at a glance. And if we look at
- 15:06it closely, while it applies to operators of
- 15:11companion chatbots, most of the duties will apply
- 15:15to operators that allow child users. So chatbots
- 15:21can have users of all ages, but those specific
- 15:26duties, if you have a chatbot that, again, is
- 15:29only internal, is business to business, is not
- 15:34marketed as a general consumer chatbot, doesn't
- 15:38allow access to users, a lot of the duties would
- 15:41not bind an operator of that kind of chatbot.
- 15:45So the requirements vary. The first is age assurance,
- 15:49and this is really important because since all
- 15:52of the other compliance obligations are triggered
- 15:55depending on whether the chatbot is offered or
- 15:59allows child users whether on it. So determining
- 16:03age of the users is very important. And that
- 16:08was, again, it is a child safety forward law.
- 16:14So with regard to determining age, the law has
- 16:18two options. One is determining age through the
- 16:22Digital Age Assurance Act. And the Digital Age
- 16:26Assurance Act applies to software applications.
- 16:30And you could use it to determine age at the
- 16:34time of account setup. So that would be at the
- 16:38device level of age verification. And there's
- 16:42a requirement, the Digital Age Assurance Act,
- 16:45which is already existing law, has this detailed
- 16:48schema, which you can use to verify age, the
- 16:53age signals that are sent from the device. It's
- 16:56based on age brackets. you have to ensure, the
- 17:00operator has to ensure that there is no specific
- 17:04detail. All information must be non -personally
- 17:08identifiable information, must be bracket based.
- 17:12And if a signal is received by an operator of
- 17:17a companion chatbot under the Digital Age Assurance
- 17:22Act and under Adams law, it will be deemed that
- 17:26the operator has actual knowledge of the user's
- 17:29age. So option one of age determination is the
- 17:35Digital Age Assurance Act. Option two to determine
- 17:39age is the procedure that's set out in the Social
- 17:43Media Addiction Act, which applies to internet
- 17:47-based services or applications. regime is a
- 17:53little different. It says that the operators
- 17:55must reasonably determine it so it doesn't require
- 17:59age brackets. Even if an operator determines
- 18:05that the user is a minor, the Social Media Addiction
- 18:10Act has an option of obtaining parental consent
- 18:15to allow the user to use that internet -based
- 18:19service. Excuse me. So two options, one software
- 18:26application API based, the other internet based.
- 18:31So we'll just call the browser compliance. So
- 18:35locally these are referred to as the browser
- 18:37compliance setup. The other one is referred to
- 18:40as the application or API compliance setup. Under
- 18:43both laws, you have to ensure purpose limitation
- 18:46of the data collected and processed. So no onward
- 18:51sale, transfer, repurposing, or use of that data,
- 18:56only restricted specifically to the purpose of
- 18:59age determination. Once age is determined, then,
- 19:05of course, the other duties and obligations under
- 19:08Adam's law is triggered. What are those? It's
- 19:12a pre -release risk assessment. So the requirement
- 19:15under Adam's Law is before releasing a companion
- 19:20chatbot into the market and for companion chatbots
- 19:23that are already existing before a significant
- 19:28modification, upgrade, change, you would have
- 19:32to conduct the operators required to conduct
- 19:35a pre -release risk assessment. The risk assessment
- 19:39relates to the design, the configuration, and
- 19:43the operation of the companion chatbot. And after
- 19:48completion of the risk assessment, an operator
- 19:51must publish a summary of the results of an independent
- 19:56evaluation, must publish the evaluation methodology.
- 20:01So once you have determined that You have child
- 20:05users or minor users of your companion chatbot.
- 20:09An operator is required to conduct a pre -release
- 20:12risk assessment. The law also specifies certain
- 20:16things. For example, if you have used an independent
- 20:20expert that works with child safety, child services,
- 20:23an independent sociotechnical expert, the experts
- 20:27that you've consulted, you should also include
- 20:30those details in the published risk assessment.
- 20:35The second obligation is publishing, developing,
- 20:41publishing, maintaining a child safety policy.
- 20:45What does the child safety policy require? The
- 20:47child safety policy requires a description of
- 20:52the harm prevention measures that the operator
- 20:55is building, maintaining, monitoring. The child
- 21:00safety policy must set out a crisis response
- 21:03protocol. The child safety policy must describe
- 21:08how the operator has determined age of users.
- 21:13And the child safety policy must also describe
- 21:16and explain what are the data and advertising
- 21:20restrictions that the operator is maintaining
- 21:22to comply with Adam's law. That goes the second
- 21:29requirement. The third requirement really is
- 21:33implementing the crisis response protocol that,
- 21:39just a few minutes ago, we talked about SB 243,
- 21:46which required certain protocols to be developed
- 21:48and maintained with regard to suicidal ideation,
- 21:51self -harm. Adam's law expands that. and requires
- 21:57specific in -service referral. The language used
- 22:02is credible in an imminent threat, parental notice.
- 22:07A lot of this comes out of the litigation that
- 22:10has arisen. relating to the design features of
- 22:14these companion chatbots, the suicidal ideation
- 22:18self -harm kind of outputs that have been reported
- 22:21on and that have come out in the course of litigation,
- 22:24course of discovery. So really to address those
- 22:27issues. There is one point here which I will
- 22:31cover in the next slide is the reasonable measure.
- 22:34So every operator must maintain reasonable measures
- 22:40and safeguards to address 14 specified behaviors.
- 22:46What is not clear from the text of Adam's Law
- 22:50is whether those 14 behaviors are exhaustive
- 22:56or not, but there are 14 behaviors listed out
- 23:01in respect of which an operator has to maintain
- 23:05reasonable measures, has to maintain crisis response
- 23:09protocol and has to consider in developing their
- 23:14child safety policy and implementing the technical
- 23:17safeguards to ensure child safety. It also requires
- 23:24that no behavioral advertising or ad targeting
- 23:28from the chats, no sale of child data, and again
- 23:33this also has a purpose limitation with regard
- 23:36to to the child data if any minor data that is
- 23:40collected. There is also a requirement of conducting
- 23:45independent audits. That requirement goes into
- 23:49effect from 2029. So all the other compliance
- 23:54obligations go effect. next year. The one that
- 23:59is deferred is the child safety independent audits
- 24:02because that is tied to the audit registry law
- 24:07and bills, a separate bill that was also signed
- 24:11into law in California. So moving on to talk
- 24:16about the child safety audits a little bit to
- 24:19the next slide. As I mentioned, this requirement
- 24:24goes into effect in 2029. Again, audits must
- 24:30be conducted every two years before any substantial
- 24:34modification. There is a revenue threshold for
- 24:39ensuring compliance. So below this revenue threshold
- 24:44are exempt till 2032, but any operator above
- 24:49this revenue threshold has to comply. After conducting
- 24:54the independent child safety audits, there is
- 24:58a requirement to report the summary to the attorney
- 25:00general within 30 days. The attorney general
- 25:05can, of course, then require the full report
- 25:08of the child safety audit. Operators are required
- 25:13to retain documents for as long as the chatbot
- 25:17is deployed, plus five years. who may audit,
- 25:21which is very important. We see these requirements
- 25:26in terms of independent verification, independent
- 25:29child safety audits coming up, not just in California,
- 25:33but also in other states. Under AB 1405, there
- 25:41is a requirement for the government operations
- 25:44agency to set up an auditor registry. So auditors,
- 25:49within that audit registry are the only competent
- 25:52auditors to carry out the independent child safety
- 25:56audits. So a lot will also depend on how the
- 26:01auditor registry is being implemented by the
- 26:05state, by the government, and who are the auditors
- 26:08that are being registered, being certified to
- 26:11be listed on that registry and to carry out the
- 26:14independent audits. And once that is completed,
- 26:18the child safety audit requirement will be triggered
- 26:22in 2029. There is a bit of ambiguity in the text
- 26:26of Adam's Law as to whether the child safety
- 26:31audit requirements will be binding even if the
- 26:36audit registry is deferred. We've seen this with
- 26:41some of the other laws enacted in the 2024 and
- 26:442025 session, for example, where there are obligations
- 26:49on the state agencies to set up a certain oversight
- 26:54mechanism or oversight processes. If those are
- 26:58delayed, does that then defer or delay the...
- 27:03audit requirement under Adams law is there's
- 27:06a little bit of ambiguity on that. So it would
- 27:09be useful to keep a watch on also what the state,
- 27:13what the government operations agency is doing,
- 27:16how quickly those are moving. It is also likely
- 27:19that since the independent verification organization,
- 27:23the IVO bill has been passed, signed into law,
- 27:26it's likely that both of those might proceed.
- 27:29along the same timelines, but all of that to
- 27:33say that the child safety audit provision in
- 27:36terms of operative timeline does depend on the
- 27:41audit registry being set up and to give effect
- 27:47to this requirement. I'm just gonna take a pause
- 27:53and see if there are questions. Is hybrid approach
- 27:58covered when initial screening is chatbot, then
- 28:04human takes over? That's a great question. So
- 28:07there's a lot of chatbots that will provide initial
- 28:11responses and then redirect to a human to respond.
- 28:17Typically, those are... We see those most often
- 28:22in the case of customer service, you know, any
- 28:24other situation where there is a chatbot responding
- 28:29with initial information, and then of course
- 28:31it's escalated to a human. The understanding,
- 28:35the reading of the text is those situations might
- 28:39not be covered, but this applies to any kind
- 28:46of companion chatbot that is Again, the language
- 28:52in the definition is a companion chatbot capable
- 28:56of meeting a user's social needs. So that's a
- 29:01little bit of the ambiguity in terms of scoping.
- 29:06And I think that's a healthy discussion to have,
- 29:08but yes, where it escalates to a human, typically
- 29:15in the context of customer service or business
- 29:20-to -business interactions would be out of scope
- 29:23of SB 243, which defines the scope for the purpose
- 29:28of Adams Law as well. Some of the related 2026
- 29:33statutes, we talked about the Digital Age Assurance
- 29:37Act. And the requirements that I mentioned with
- 29:40regard to age determination were amended by AB
- 29:441856. So what we discussed was as the law is
- 29:48chaptered and stands today, as AB 1856 really
- 29:53imposes some more stringent requirement, whereas
- 29:56we mentioned that a signal is treated as actual
- 29:59knowledge. It requires only minimum amount of
- 30:03information be collected for the purpose of age
- 30:06determination. And in the event of a violation,
- 30:14there is the provision of penalties which can
- 30:19be enforced only by the attorney general. There
- 30:22is a two -tier penalty regime depending on negligent
- 30:26acts and intentional acts. Again, the statute
- 30:29has a bit of ambiguity as to what would be considered
- 30:32a negligent violation as opposed to what would
- 30:36be considered an intentional violation. So operators
- 30:40are looking at the risk of penalties or violation
- 30:46of the Age Assurance Act. But the lines between
- 30:49what is considered negligent and what would be
- 30:52considered as intentional is not very clear.
- 30:55I also discussed a little bit about the ban on
- 30:59companion chatbot toys. There was another bill
- 31:02that was passed in 2026 relating to healthcare
- 31:06chatbots. So businesses offering a healthcare
- 31:09chatbot would be deemed to be healthcare providers
- 31:13and would have to then comply with the confidentiality
- 31:17of medical information act. That's an important,
- 31:22that act itself lays out an entire compliance
- 31:25regime with regard to ensuring that the devices
- 31:30have FDA approval where required and are compliant
- 31:35with the obligations under that law, but only
- 31:38if the companion chatbot offers health care and
- 31:42is specifically for health care purposes. A lot
- 31:47of times these raise questions about where general
- 31:54purpose chatbots are users ask questions, health
- 31:59questions. You could go into ChatGPT, ask it
- 32:02a health question. Does that qualify it as a
- 32:06healthcare chatbot or not? Again, that is another
- 32:10gray area which will require, which will be laid
- 32:16out depending on how this law is enforced. Moving
- 32:22on to Again, if you recall, I had mentioned that
- 32:31Adams Law has certain design requirements with
- 32:36regard to the risk assessment and the crisis
- 32:40response protocols. So what are the 14 features
- 32:46operators must prevent in a companion chatbot
- 32:53that is used by miners or that allows miner users.
- 32:59The reasonable measures and the loss lists this
- 33:03out quite clearly, the reasonable measures to
- 33:07prevent these 14 features. So you would have
- 33:10to test whether an operator, any operator that
- 33:15you're advising, you're assessing, evaluating,
- 33:19whether they are ensuring reasonable measures
- 33:22to prevent these 14 kinds of interactions and
- 33:27outputs with a child user. So the first is, of
- 33:30course, carrying forward the provisions of SB
- 33:34243, which relates to self -harm, suicidal ideation,
- 33:38drug or alcohol use. Again, attempting to, second
- 33:42is the attempting to diagnose or treat. This
- 33:47relates to the other law that we just covered,
- 33:50the healthcare law, which is AB 1979. So unless
- 33:56it's an FDA regulated device subject to HIPAA,
- 33:59any companion chatbot that attempts to diagnose
- 34:02or treat would be covered under Adam's law, engaging
- 34:06in obscene matters, sexual abuse material, depicting
- 34:10a child in... sexual deep fakes, discouraging
- 34:16a child from telling a professional appropriate
- 34:18adult, discouraging breaks or suggesting that
- 34:21the child needs to return frequently, non -compliant
- 34:25advertising in chat. If you recall, I mentioned
- 34:28that A .B. Adams law has a prohibition on advertising
- 34:33within chats and behavioral advertising and ad
- 34:36targeting, expressing or simulating romantic
- 34:39interest. blaming a special understanding of
- 34:42the child, encouraging reliance for emotional
- 34:45support, excessive praise or flattery, and of
- 34:48course encouraging the child to circumvent parental
- 34:51controls or conceal use. What is expressly allowed
- 34:56is age -appropriate information about abuse,
- 35:00neglect, or bullying. These would typically be
- 35:03responses or outputs that directs a minor user
- 35:07to seek help or seek information from official
- 35:12designated hotlines or resource or support channels.
- 35:18Those are expressly allowed, but when implementing
- 35:24or assessing compliance with regard to Adams
- 35:27Law, the reasonable measures, the child safety
- 35:31policy, the risk assessment, the crisis response
- 35:34protocol all have to consider these 14 features
- 35:39which operators have a duty to prevent in companion
- 35:43chatbots that allow minor users. So there's a
- 35:52question about whether Section 213 First Amendment
- 35:55rights are irrelevant. That's a great question
- 35:58and that those discussions did arise. If you
- 36:03look at the Senate and Assembly floor analysis,
- 36:07yes there were questions about whether these
- 36:09would attract First Amendment concerns. Typically,
- 36:14if we look at the determination in the Character
- 36:20AI litigation, where Character AI, which was
- 36:24then acquired by Google by Alphabet, raised a
- 36:29First Amendment defense in its motion to dismiss.
- 36:36The court did not accept that, and that did not
- 36:40survive. The motion to dismiss did not succeed.
- 36:46So if that's any indication, it seems that courts
- 36:50are taking a different view of these child safety
- 36:56design requirements, looking at them as product
- 37:00design, looking at them as required disclosure.
- 37:04But of course, how that would play out in another
- 37:09challenge to Adam's law is yet to be seen. Another
- 37:14point to note is that the 14 features which are
- 37:18discussed, which are set out in Adam's law, really
- 37:21builds on the RAIN versus OpenAI case, which
- 37:26is still ongoing. And if we look at the kind
- 37:30of harms that the plaintiffs, the parents of
- 37:33Adam Raine set out in their complaint against
- 37:38OpenAI, that really talks about many of these
- 37:4414 features that have now found its way in Adam's
- 37:47law. So it's not that we are waiting for cases
- 37:52to arise out of the law, it is the law has borrowed
- 37:58relief from the facts that have been led in these
- 38:02cases and is now being reflected in the statute.
- 38:07So it's a fair question, but it seems like where
- 38:11child safety considerations arise, the courts
- 38:15are taking a view that these do not attract the
- 38:20First Amendment prohibitions. Looking at the
- 38:24gray areas, so for example, just going back to
- 38:28some of the definitions and requirements in both
- 38:32SB 243 and Adams Law. So is a general purpose
- 38:37assistant a companion chatbot? Again, the text
- 38:40says capable of meeting a user's social needs.
- 38:44What comprises social needs? Is it a companion
- 38:47chatbot that then goes on to organize or provide
- 38:55information for social purposes? Is it just a
- 38:58dual purpose? So there is a little bit of ambiguity
- 39:03there. And from a compliance perspective, it
- 39:06also raises a question of how do organizations
- 39:09then comply to determine whether a companion
- 39:12chatbot is meeting a user's social needs. It's
- 39:17a fair question. an AI chatbot that you're using
- 39:22purely for scheduling or work purposes but then
- 39:27also is you know making social arrangements or
- 39:31providing input on those kind of arrangements
- 39:33would it then come under social needs. Engagement
- 39:36design across sessions is another gray area because
- 39:41the law requires Adams law and its features checklist
- 39:46says that suggesting that the child needs to
- 39:49return frequently emotional reliance and excessive
- 39:52flattery. Again, reliance and excessive are subjective.
- 39:57It clearly would vary from user to user. Question
- 40:02is, how do you really build these safeguards
- 40:05and what is considered reliance or what is considered
- 40:14excessive? Is mature sexually active 14 year
- 40:19old a minor 50? So that's a great question because
- 40:23the age verification depends on the Digital Age
- 40:27Assurance Act, which requires brackets that doesn't
- 40:31really make a distinction on whether a user is
- 40:36sexually active or not. It relies on the existing
- 40:41laws that determine who is a minor, but again,
- 40:45the age brackets also go beyond the existing
- 40:49laws like COPPA or the California Consumer Privacy
- 40:55Rights Act, which talks about minor and adult.
- 40:59So that's a fair question. The read is it would,
- 41:05because if we look at the OpenAI, Rain versus
- 41:10OpenAI case, Adam Rain was a 17 year old. So
- 41:14because the law, the text of SB, Adam's law is
- 41:19borrowing really from that litigation. If we
- 41:22see the other Florida case of Sewell Setzer who
- 41:27was 16 years old, it seems that these compliance
- 41:33requirements apply to the extent of the age brackets
- 41:39set out in the Digital Age Assurance Act, which
- 41:41goes up to the age of 17. Again, training on
- 41:47children's conversations is another gray area
- 41:51because the law limits the use of child's data.
- 41:55There's a purposeful limitation built into both
- 41:57SB 243 as well as Adam's Law. But then what is
- 42:03model improvement? considered training, is it
- 42:06considered maintaining the required safeguards
- 42:09and safety services and service of the companion
- 42:13chat, but that is not clear. So we see in privacy
- 42:17policies across the board, especially after the
- 42:22LLMs are now being deployed a lot more across
- 42:25various services, a lot of operators, a lot of
- 42:29service providers have updated their privacy
- 42:31policies to say, that there will be training
- 42:34on user data, including even telemetry data,
- 42:39et cetera. But this law as a prohibition has
- 42:45a purpose limitation built in and prohibits training
- 42:47on child's data. But whether training your model
- 42:52for improvement purposes so that you're maintaining
- 42:56the safeguards that are required would be caught
- 43:00under that prohibition is something that's not
- 43:02clear. Simulated intimacy and dependency. Again,
- 43:08the obvious gray area is when does a regular
- 43:11conversation move to what is that line that is
- 43:16crossed, which then takes a conversation from
- 43:19a regular conversation to a conversation that
- 43:23is simulating intimacy or a romantic interest
- 43:28or creating excessive dependency. That also raises
- 43:33the question to what extent operators would be
- 43:35able to monitor chat sessions across users would
- 43:41be able, or even if they're not, if there's no
- 43:44active monitoring, what are the kinds of lags
- 43:48or triggers that need to be built in to a model
- 43:52at the design stage to ensure that at particular
- 43:56intervals that there are flags being issued for
- 44:00when a particular chat, a particular session,
- 44:04or repeat sessions are crossing that threshold
- 44:07is not clear. I mentioned about the negligent
- 44:11versus intentional violations, especially because
- 44:14it's tied to the Digital Age Assurance Act, which
- 44:18was also amended this year. When we were engaging
- 44:22with the California legislature, we recommended
- 44:26to clarify the negligent versus intentional tiers.
- 44:32We recommended that negligent should be defined
- 44:35as a failure to ensure reasonable care, while
- 44:39intentional adopts standard menstrual criteria
- 44:44of knowing will follow reckless. But those, of
- 44:48course, those recommendations that we made were
- 44:51not included in the final text. Moving on to
- 44:58some more gray areas. Again, age determination
- 45:05is an area of... and is also an area of ambiguity,
- 45:11because the Digital Age Assurance Act says that
- 45:14a signal is self -declared at account setup.
- 45:19The Health and Safety Code, which is the Social
- 45:22Media Addiction Act, says reasonably determined.
- 45:25And the attorney general's rules, which are implementing
- 45:30the Social Media Act, provide for biometric analysis,
- 45:35zero -knowledge proof, and rejects self -declaration.
- 45:42So there is a bit of ambiguity as to what would
- 45:46be considered a sufficient signal under the amended
- 45:50Digital Age Assurance Act. And that, of course,
- 45:54it would be good to keep an eye on the attorney
- 45:58general's rules that are implementing also the
- 46:01Social Media Addiction Act because it is setting
- 46:04out a range of technical age assurance measures
- 46:09that would be considered compliant. Similarly,
- 46:14for output standards, there's also a gray area
- 46:18because Adam's Law in the 14 features for which
- 46:24compliance is required, one of the features is
- 46:27disproportionate flattery, diagnose or treat.
- 46:31So for example, as I mentioned, if you're using
- 46:35a chatbot and asking it for, you know, what do
- 46:40you do about teenage acne? Is it considered as
- 46:45diagnosing or treating? A lot of the sessions,
- 46:51tutoring, kids tutoring applications, kids tutoring
- 46:54chatbots, and even for adult tutoring chatbots,
- 46:59there are engagement -based assessment of performance
- 47:04which require you to complete and continue to
- 47:08interact and engage a session, whether those
- 47:11would be considered as discouraging breaks or
- 47:15would be considered as violating the requirements
- 47:18of triggering breaks for minor users is a gray
- 47:23area. So educational uses is something that would
- 47:28require a closer look. And again, for implementing
- 47:34and enforcing a crisis response protocol, the
- 47:39language that Agnom's law uses is credible and
- 47:43imminent threat or serious and serious emotional
- 47:47distress. So what... is going to be considered
- 47:51as a credible and imminent threat is something
- 47:54that is a gray area that is left to be determined.
- 47:57And would this be determined across one instance,
- 48:02a single chat session, cumulative chat sessions,
- 48:06period of use is not clear. Moving on to the
- 48:14enforcement actions. So if we look at SB. 243
- 48:19and Adams Law together, which really sets out
- 48:21the entire regime for compliance. The first track
- 48:27is private rights of action. SB 243's private
- 48:31rights of action are already in effect. The private
- 48:38right of action under Adams Law will take effect
- 48:40from 2027. And of course, there is there is also
- 48:46the requirement to demonstrate an injury. In
- 48:50fact, under SB 243 and under Adam's law, of course,
- 48:56the requirements are a little more permissive.
- 49:01So for example, there are a couple of options
- 49:06set out in the law itself. But the important
- 49:10thing to note that even though SB 243 has been
- 49:13alive since last year. As of my research until
- 49:19yesterday, there is no law, there's no suit or
- 49:23enforcement action that's been initiated under
- 49:26invoking the private right of actions of SB 243.
- 49:30So really how these will be enforced and how
- 49:33the enforcement would play out is something that
- 49:37is still left to be seen. Moving on to the second
- 49:43track, there is, of course, the common law remedies
- 49:49that really build on the, again, doing versus
- 49:55open AI suit. It's unclear whether actions under
- 50:00California's unfair competition law can be sustained
- 50:03for violations of SB 243 and Adams law, simply
- 50:07because SB 243 creates its own private. private
- 50:10right of action, and Adams law specifically bans
- 50:14UCL claims. So the UCL claims, the predicate
- 50:18for the UCL claims in the rain versus open AI
- 50:21case were different. And so the one thing that
- 50:26we know is UCL claims based on Adams law have
- 50:30been foreclosed, but nonetheless, common law.
- 50:34enforcement actions still remain a viable path
- 50:37under Adams law as well as SB 243. Moving on,
- 50:44I just wanted to go through the remedies that
- 50:48were requested in the in the rain case and compare
- 50:52it to Adams law. So if we look at the kind of
- 50:57relief or remedies that were sought in the rain
- 51:00versus opening a case, those real the Adams law
- 51:04really mirrors the remedies that were requested.
- 51:07So that gives a good sense of, you know, how
- 51:10common law actions can and might play out now
- 51:16that the laws and force and what are the kind
- 51:18of grounds on which those actions can be sustained.
- 51:24So really, for example, even the requirement
- 51:29for having independent audits and independent
- 51:33monitors was a relief sought in the rain case,
- 51:36which has now made its way into Adams law. And
- 51:39this just gives an idea of how much that case
- 51:43has informed the law as well as what are the
- 51:47kinds of you know, violations that will give
- 51:51rise to private lawsuits as well. Moving on to
- 51:59the next slide. Public enforcement is a little
- 52:05different in both the laws. In Adams law, which
- 52:13is SB -319, we see that the attorney general,
- 52:17the district attorneys, other public prosecutors
- 52:20listed in the business and professions code have
- 52:24the authority to enforce. So there is the public
- 52:27enforcement option in the law, which is quite
- 52:31clear. AB 1856, which relates to the Digital
- 52:36Age Assurance Act. The enforcement is with the
- 52:41attorney general only. AB 1979, which was relating
- 52:46to operators' obligations with regard to healthcare
- 52:51chatbots. Again, that... enforcement under that
- 52:56law will be according to the Confidentiality
- 53:00of Medical Information Act, the CMIA enforcement
- 53:03mechanisms. But for SB 243 and SB 867, which
- 53:09is the ban on TOI, the companion chatbot TOI,
- 53:13there are no named enforcers for some specific
- 53:18provisions, and it will run through the private
- 53:21rights of action. When we look at the enforcement
- 53:27pathways, it's very clear that there are three
- 53:30tracks. One is statutory private rights of action.
- 53:34Second is the common law lawsuits, which we're
- 53:40seeing design liability lawsuits. There are now
- 53:46at least by my count that are ongoing and in
- 53:51the courts. And the third track is the public
- 53:54enforcement by the designated authorities of
- 53:58all the chatbot laws that California has enacted.
- 54:02Adam's law, SB -319, has the most number of public
- 54:10enforcers designated to enforce the law. And
- 54:16then I just... wrap it up by some of the work
- 54:21that we've done and how we're seeing the state
- 54:24and federal interaction. So in March 2023, my
- 54:29organization, we filed the first complaint with
- 54:32the Federal Trade Commission under Section 5,
- 54:36which is the unfair and deceptive practices provision
- 54:40of the FTC Act. And we asked the FTC to do broadly
- 54:46three things. We asked the FTC to halt further
- 54:50commercial deployment of CHAT GPT. The second
- 54:54really that we asked for was pre -deployment
- 54:57risk assessments and risk assessments during
- 55:00the AI lifecycle. relief that we asked for was
- 55:05rulemaking by the Federal Trade Commission. And
- 55:08when we filed the complaint, the material that
- 55:11we relied on were the model cards published by
- 55:14OpenAI with regard to GPT -4, which the model
- 55:18card itself set out the risks to child safety,
- 55:22cybersecurity risks, the kind of risks that we're
- 55:24seeing with the hugging face hack that was that
- 55:29I'm sure many of you might have read in the news,
- 55:31if not all of you. So those were really the risks
- 55:33that were set out in the model and system cards
- 55:36and documented. And those were the risks on the
- 55:39basis of which we asked the FTC to exercise its
- 55:41Section 5 authority. We also appeared before
- 55:46the Commission at their open meetings. impressed
- 55:50on them the need to initiate an investigation.
- 55:53We filed two supplementary complaints after our
- 55:56initial complaint in March. The FTC initiated,
- 56:00issued a civil investigative demand in July 2023
- 56:04and the investigation commenced and it stalled
- 56:09after that but Of course, in October 2026, we
- 56:13see that the investigation has been revived and
- 56:16now extended not just to OpenAI, but also to
- 56:19Anthropic and other AI companies, at which time
- 56:24we have then, you know, revived the issue. We
- 56:28have continuously called on the FTC and we've
- 56:31called for congressional oversight on these investigations
- 56:34and these issues. that should result in an order
- 56:38and should result in the FTC establishing guardrails
- 56:41at the federal level in October, soon after the,
- 56:47just this month, soon after the. investigation
- 56:50was revived by the FTC. We also urged them to
- 56:54set out the kind of relief that we had asked
- 56:58for in our first complaint and our supplementary
- 57:01complaints and indeed the kind of relief that
- 57:04we are now seeing play out in the private lawsuits
- 57:08and the private actions against these AI companies
- 57:11and also in some of the laws. So really it's
- 57:15an effort that we have that we believe is in
- 57:19the public interest, which is why we engage with
- 57:22these laws also in California, which is why it's
- 57:26really important to take some time and talk about
- 57:31it and see what has resulted. I will just recap
- 57:36the compliance timelines under the different
- 57:40bills. Yeah, I think I jumped ahead and covered
- 57:44this. So moving on to the next slide. Yeah, if
- 57:49we look at the compliance timelines, the core
- 57:52duties under SB 243 begin in July 2027. The core
- 57:58duties under Adams Law begin also in 2027, and
- 58:05then the audit requirement in 2029. But what
- 58:11will also be important to track is when the FTC
- 58:16if hopefully, if and when they do conclude their
- 58:19investigations into OpenAI and Anthropic under
- 58:24Section 5, the kind of orders that will be issued
- 58:29will then add another layer of federal compliance
- 58:32as well. So what are the key takeaways? The key
- 58:37takeaways from all of these chatbot laws are
- 58:41scope comprehensively, chatbots, companion chatbots
- 58:44that we are now seeing. Most used or even the
- 58:50customized chatbots tend to be dual use, multiple
- 58:54use, general purpose. So scoping comprehensively
- 58:58is probably erring on the side of caution. Crisis
- 59:03protocols. have to be built. A lot of the models
- 59:06that we see being marketed now don't necessarily
- 59:09have the crisis protocols that are required by
- 59:11the law and might need to be retrofitted. But
- 59:14in terms of significant modifications or operators
- 59:17that are releasing new companion chatbots or
- 59:22customized chatbots have an opportunity to build
- 59:25those in during the deployment and fine tuning
- 59:29stages. Excuse me. Risk assessments have to start
- 59:34at the design stage because it's really hard
- 59:37to retrofit it. And in our experience, we work
- 59:40with a lot of AI experts, technical experts,
- 59:43and legal experts. Retrofitting using synthetic
- 59:47data to then retrofit the kind of safeguards
- 59:50that are required by law really makes the models
- 59:53brittle and can lead to something called a model
- 59:57collapse, which then means a compliance collapse
- 59:59and is a real risk. ensuring robust privacy -preserving
- 1:00:05age assurance techniques would be important.
- 1:00:08And then, of course, ensuring documentation of
- 1:00:12your risk assessment measures, your reasonable
- 1:00:15safeguard measures, child safety policies, crisis
- 1:00:19prevention, and having the right document to
- 1:00:23documentation and records retention. Again, a
- 1:00:26lot of this depends on having robust internal
- 1:00:30governance. And I think the oversight that agencies
- 1:00:34and enforcers will have as a result of these
- 1:00:38laws will be looking really at the kind of measures
- 1:00:42and guardrails that operators are implementing.
- 1:00:46So documenting the guardrails that you're implementing
- 1:00:50would be really important. There is one question,
- 1:00:55federal preemption. So that's an interesting
- 1:01:00question. There's been a lot of federal preemption
- 1:01:02discussion circulating around since last year.
- 1:01:05The preemption proposal was really unpopular.
- 1:01:09It was defeated in 1991. And right now we're
- 1:01:14seeing a lot of bills at the federal level also
- 1:01:16with narrowly scoped preemption provisions. Given
- 1:01:20the way chatbots operate, it's really hard to
- 1:01:24scope based on functionality. I think what we
- 1:01:30will see is not 50 state standards, because if
- 1:01:34we look at the kind of chatbot laws that are
- 1:01:36being passed, California has been a forerunner,
- 1:01:39but we're also seeing a lot of convergence, not
- 1:01:43just in terms of the laws, but also in terms
- 1:01:45of the enforcement actions. Attorney generals
- 1:01:48are doing a lot of these enforcement actions
- 1:01:53and collaborating on a bipartisan basis. Yes,
- 1:01:59we will see 50 states legislating on this. Likely,
- 1:02:03there's been a real wave of legislation. But
- 1:02:06we will also see a lot of convergence. And we're
- 1:02:08already seeing convergence on these provisions.
- 1:02:11So for example, Connecticut, New York have both
- 1:02:15moved forward with bills very similar to Adams
- 1:02:18Law. Utah, Texas are also looking at these provisions.
- 1:02:24Alabama. is looking at the kind of provisions
- 1:02:27that are set out in Adam's law. So yes, we will
- 1:02:30get 50 states legislating, designating different
- 1:02:34enforcement authorities, but in terms of the
- 1:02:38substantive provisions, we will probably see
- 1:02:42a lot of convergence. There was another question
- 1:02:51in terms of Paga actions. That's a good question.
- 1:03:00I haven't really given thought to it, so I wouldn't
- 1:03:03want to give an uninformed answer. I think it
- 1:03:06would be great. I'm happy to also take this offline
- 1:03:10and maybe see what are our thoughts or what we're
- 1:03:13seeing with regard to those kind of enforcement
- 1:03:15actions. But at this point in time, I don't have
- 1:03:17a clear idea. It's not something I thought about
- 1:03:20deeply. All right. Looks like that's it for questions.
- 1:03:30This concludes today's presentation. I would
- 1:03:32like to give another thanks to Christopher for
- 1:03:34an excellent presentation today, and a warm thanks
- 1:03:37to our audience for joining us. Have a great
- 1:03:39day. Thank you.