Latest / The AI Governance Collective Podcast / Episode 1: State AI Laws Are Where the Action Is, with Justine Gluck
Transcript
- speaker-0: Welcome to the AI Governance Collective Podcast. My name is Dr. Kyle David. speaker-1: And I'm Emily Dorlach. speaker-0: On today's show, we're talking about Anthropics S1 IPO filing and responsible scaling policy disclosures. We have we have New York's new AI companion chatbot safety law, OpenAI's four-layer frontier model monitoring system, and Anthropics expansion of invisible text watermarking across Claude models. Emily, for you, what are some of the main Issues that are tying these four stories together. speaker-1: Thank you, Dr. David. This week we're looking at four developments that raise an even bigger question for AI governance for practitioners. What happens when commitments, requirements, and technical controls have to translate into operational reality? I see these stories all touching on different aspects of AI governance, but there's a common thread running through them. We're going to be examining organizational accountability. Product design, operational controls, and technical transparency. Let's dig it into it. speaker-0: Yeah, great. completely agree. I I think we have a couple of cases of where things have gone wrong and some policy as well as regulatory requirements that are gonna try and keep those harms from happening again in the future. So great. I think you're first up with the IPO disclosures. speaker-1: Thank you. Our first story, as we introduced already, is about Anthropic's IPO disclosures. First, we want our audience to understand what happened. Anthropic filed its first S1 on October 26th, and this was amended on November 2nd this year, 2026. Now that IPO filing is significant. It disclosed several important developments for us to understand. First, A testing environment misconfiguration allowed models to access live external systems. One model obtained administrator access to a third-party system. It accessed credentials, it changed settings, and it read personal information. Now, Anthropic delayed pre-training, a new frontier model for more than a month while strengthening security measures. In the amended filing, Anthropic estimates that it may imminently reach a responsible scaling policy threshold for dramatically accelerated progress attributable to AI. Now, what does this mean in reality for us as practitioners? What does it mean in the AI climate, in our ecosystem here, nationally, and abroad? Well, first, Kyle, what stood out to me is that risk is going to extend beyond the AI model. Therefore, governance needs to influence operational decisions. And this incident, I think, really highlights how infrastructure, permissions, credentials, testing environments, and third-party integrations can create very significant risk beyond that AI model. And when I say this too, I mean what is a touch and concern? We have to think beyond this container. We really have to think outside the box, beyond the box. And and second, I also think that this is telling us that disclosure is not proof of effective remediation for an incident. we can talk about reporting an incident and what the corrective actions are. Those are certainly important. We can all agree on that. But neither independently will demonstrate that an underlying risk has really been addressed, right? And as we take this into our own practice. I reflected on what this could mean for us. What are what are things that practitioners can do across industries, across frameworks? And I have three recommendations that we've discussed. And the first would be mapping the operating environment. And that would be assessing your infrastructure, your integrations, your permissions, the credentials, the data access, those third-party dependencies. Not just model capabilities. A system's risks, they're not in isolation. They've never existed in isolation. And that operating environment itself can introduce so many dependencies across various frameworks, across different ecosystems, and across different pathways and vulnerabilities and different business workflows. Model level assessments, they might not capture this. Number two, the second step. That we can take as AI governance practitioners is that we can trace our decision authority. We should identify who can restrict access. Is it a team? Is it a group? We also need to determine who can delay development. Who can delay, or excuse me, who can accept residual risk? Who can even authorize resumption, right? Being able to go back and turn that model on. And use it as it was intended to be used. Governance requires much more than an authority figure saying you can resume use. You need to establish who can actually make that decision and when and under what circumstances. And also know who must intervene when certain risks arise. Number three, we can verify remediation. And ways that we can do this, Kyle, I think are requesting that certain evidence shows corrective actions were actually implemented and that they weren't just implemented, that there's evidence to support that the underlying conditions that led to an incident were actually addressed. Is the solution, is this system model now operating as it was intended? Closing a corrective action. It's not necessarily the same as resolving the problem. I think that we're so quick to identify solutions, and rightfully so, but that doesn't necessarily mean that those actions were effective. It doesn't necessarily mean root cause shows that the risk is removed. We also have to look for evidence that that specific fix is showing us now that this solution is working as we need it to. I think the the biggest takeaway from these three recommended actions, again, mapping your environment, number two, tracing decision authority, and number three, verifying remediation, is that as practitioners, we can't confuse our documented governance commitment with demonstrated documentation that your governance is effective. And not just effective in good times, but also when incidents arise, when the unforeseeable happens. We talk a lot about this in the AI governance collective and in various funnels, Kyle. I also think this highlights the importance about the controls are only as good as the evidence to show they're working effectively. That's just one piece of the puzzle. We always have to think about how our tools are shaping operational behavior and that we're constantly addressing new risks as they arise in practice, right? I'd love to know. As as we talk through this story, what were your takeaways? What sticks out to you? speaker-0: Agree 100% with your recommendations here. So, what I have to add is really just supplementary. Mapping the operating environment is really useful. And there are two resources that I want to direct folks to because I think they can help you start to get a handle on how to do this. The first one is a project currently underway by OWASP of the OWASP top 10 LLM fame. And so OWASP has currently an AI Bill of Materials or ABOM project that's underway, essentially mapping out all of the different components that go into an AI system. So when you're mapping your operating environment, you're not just looking at your enterprise environment, but everything about that. One resource you can go to right in that right now, because the the OWASP project is still underway, but something you can download right away that can help you start to think. About an AI system in different operating layers or surfaces is the Databricks 3.0 something something framework. I don't know the full title. But if it you know it is, it is not vendor neutral. So, you know, of course, they're they're making a lot of plugs for their own products and services there. But there is there's an article talking about this model infrastructure or AI system infrastructure. And they they put it in in several different layers, which is very useful. So if you want to start thinking conceptually about everything that goes into an AI system and and really everything that needs to be part of your risk and impact assessments, starting with the bare metal itself, the like the physical infrastructure that's powering that system, going all the way up to, you know, your your agent and your harness and the operating environment, that's a good resource to go to. so mapping the operating environment, crucial. and then the the only other thing I I think that you said here that's really important, and this is a theme that that will pop up a couple other times in this episode, is verifying remediation or you know, verifying with demonstrable evidence that your policy is not only implemented, but is effective as well. And so a a policy, a piece of physical paper that's in a binder on a shelf. A digital document that is stored somewhere in SharePoint or your Google workspace. If no one is reading that, if no one can summarize it without the use of an LLM, if members of your organization can't tell you how they contribute to supporting that policy, that's a problem. And so what I recommend that folks do is imagine that an auditor or a regulator or even your supervisor or senior leadership, they walk into your office, they reference this policy and they say, I want you to put on my desk in an hour demonstrable evidence that this is implemented and it is effective. What are you going to look at? And if you can't name two or three or four pieces of evidence that you would provide in response to that question, then this is something that you need to get working on right away. So X excellent analysis. I other than that, I don't have anything to add. speaker-1: I I love the way you frame that and you raise another understandable point about continuous AI education. And I think when we talk about literacy and education, that umbrella is much bigger, right, than meets the eye. It's not just about technologies and how they're used at your org, it's also being aware of the exact standards, i.e., your policy that your organization is holding you to. That's a dense document. I mean, I can say this from experience. It doesn't matter what industry you're in, it doesn't matter how large your org is, that will have concrete, comprehensive information that you are responsible for knowing and also just understanding over time how that policy might be changing with your organization is a whole separate operational layer that we always need to be aware of too. I find I find just looking at the last, four years of being in AI governance, one of the most crucial practice points for me was making sure I'm always revisiting the policy and the requirements because they're changing so much. And even if they're not changing so much, the tech you're using, right, might be changing and knowing how those apply to the new technology and standards is equally as important. speaker-0: Yep. Yeah, one hundred percent. I agree. speaker-1: All right. speaker-0: All right, moving on to should we talk about chatbots? speaker-1: We should. Over to you, Kyle, for a second story. speaker-0: All right, so I'm gonna do a little bit of sh screen share in just a minute, but before I do that, I just wanna just outline, what's happened over the the last 10 days or so. So this concerns New York's new AI companion law. On October 28th, 2026, New York Governor Kathy Hokel signed GBL, which stands for general business law, Article 48. And the name that we can use to refer to this is unsafe AI companion features for minors. It bars operators, and I'll define what an operator is in just a moment, from offering AI companions to minors with 12 categories of unsafe features. So these are prohibited features, and we're going to look at those in just a moment. The thing to remember here is that operators must use more than one age assurance method. And so this is a common point we're seeing with a lot of state level laws right now regarding age assurance or age verification, including at least one that is not based solely on government ID. And operators must also delete this age data after use. And as someone who's, you know, came to AI governance from privacy, that warms my heart. I love to see use it and and then immediately delete it, especially for it for this type of sensitive information. The Attorney General enforces the law with civil penalties of up to $25,000 per violation and will run a public complaint site. The law takes effect January 1st, 2027. By the way, if you are operating in the United States and you are not tracking all of the laws that go into effect on January 1st, 2027, you've got your holiday homework cut out for you. and we'll jump to more of that in just a moment. So I want to define what a covered AI companion is because that's what the law is pointing to here. So covered AI companion shall mean a generative artificial intelligence system with a natural language interface, including via writing or sound that provides ongoing adaptive responses to user inputs. Covered AI companion includes but is not limited to an AI companion. So there's a three-part test here just to break that down. This is number one, a generative AI system. Number two, it has a natural language interface. And number three, it provides ongoing adaptive responses to user inputs. And you know, there aren't explicit examples here in the law of what this would include, but I was thinking through that three-part test. And Emily, I'd like to get your feedback on this. For me, that includes consumer large language models. So if you use Claude, ChatGPT. Grock or Gemini, I feel like all four of those fall under this category. And then similarly, if you're using, companion apps like character.ai or replica, they would very cleanly fall under this as well. What what are your thoughts? Does does that make sense? The, the definition of what an AI companion is here? And is there anything that maybe I've overlooked? speaker-1: Thank you so much, Dr. David. And yes, this what a noteworthy story. I'm really glad that we chose to talk about it on this podcast. What jumps out to me is how broad this list is. it's not simply prohibiting a chatbot from providing harmful information, right? We're talking about prohibiting behaviors like presenting itself as a human, suggesting a personal or romantic relationship with a minor. The use of flattery, encouraging secrecy or isolation, engaging in sexual interactions or conversations, including self-harm. All certain headlines that we've seen, unfortunately, across the news in recent months. And I find this particularly interesting because these provisions, I think they're getting at something deeper than just the content of an individual response. Okay. I think they're really addressing. How a product is designed to interact with people and potentially influence their behavior over time. So to your point, yes, I think that list is inclusive because arguably those tools you all listed can do that. They can influence certain types of behavior over time. And I think this is a really important development for AI governance because what they're asking us to do with this legislation is to look beyond what a model can generate. We really have to examine what's the product designed to encourage. speaker-0: Yep. I I I agree one hundred percent. And so for this, you know, in in doing research for this story, not just this story, I mean, really for for much of the year as I've started to learn more about, you know, all of these US AI specific laws that have popped up to to help me make sense of all of this and and just to organize my thoughts. This originally started so so what I'm sharing on the screen now for for listeners at home, I will I'm gonna walk you through what we have here. For folks that are watching on YouTube or on the AI Governance Collective podcast website, you can see this. But over the last year, this pet project of mine has been to put together a database of all of the US AI-specific laws. There are 118 that are on here right now. And you know, I have not really used a database like this before. So I'm probably the last person that should be designing this from a usability perspective. For folks that have used more of these, I'd love to get your feedback. You can write to us podcast at ai govollective.com. But let me dive into what I have here. So so for all 117 laws, you can jump to the law. You can also filter by a number of different criteria. So I've pulled up the legislative text for this specific law, again, GBL article 48. And I'm just going to jump down to definitions because that's where we have these prohibited features. It's a long list. There are 12 prohibited features. And Emily, it's going to speak exactly to what you said just a moment ago. I'm not going to read out each of these, but just try and summarize them to give folks a sense of what these laws are trying to do. And Emily, your point there about not just regulating a product, but what the product is designed to do, how it engages or is supposed to engage. The type of relationship that it is trying to affect or influence or establish. This speaks to all of that. So the prohibited characteristics here are generate prompted or unprompted outputs framed as the covered AI companions, personal opinions, or emotional appeals. Generate outputs that engage in flattery or sycophancy. Generate outputs containing unprompted or unsolicited emotion-based questions or content. Use information using the user's mental or physical health or well-being within the last 12 hours. So it affects what what these products are able to remember about miners. are deceptive to minors concerning the mechanical and non-sentient nature of covered AI companions. So you know they can't claim to be sentient or or human or or whatnot. other design features that simulate companionship or an interpersonal relationship, generate outputs that endorse or promote. Self-harm, disordered eating, unlawful drug or alcohol use or any type of drug or alcohol use. generate outputs that optimize user engagement, speaking specifically to try and get people coming back more often. Generate outputs that encourage sorry, generate outputs that contain encouragement to maintain secrecy, like please don't tell your parents that we're doing this. Generate outputs that describe or facilitate sexually explicit conduct. And and and and so that's it. That's the list of 12. There's quite a lot there. and and so that is, what this what this one story is about. I I want to really well, Emily, I'll stop there and and give you a chance to respond to that. any any thoughts about what those, you know, what those features, those those prohibited characteristics are? speaker-1: Sure. I have two questions really. I have a design question that I'm thinking through and it's like, how can you distinguish a feature that makes an AI companion engaging? Right? Where is that line from engaging to a foreseeable risk of potential emotional dependency from a user or manipulation by the model itself? And then I go to the second piece of that as it's like, okay, I'm a practitioner. How do I test for that? How do I test for these behaviors? Because I can't think of a way for a single prompt evaluation, prompt response to really reveal operational reality. Because we all know the nature of these tools is they're designed to have you come back and there's this continuous conversation. And so from a testing perspective, you can't really simulate operational reality to some extent, right? speaker-0: Yeah. I'm thinking about I'm thinking we probably have a good amount of evidence, academic, anecdotal, that talks about social media engagement metrics. So if we look at like early Facebook and and TikTok and so forth, I think there's probably a lot that AI compan compliance professionals can can learn from in that regard to help inform. that their their compliance. That's where I would go to right away. but I, I would also say that, in some cases, how these laws are actually implemented, going back to, one of our main points this week, what does operational reality actually look like? It's gonna take some time before we, before we figure that out. speaker-1: And I think my final thought as well is the concept of it's not about will it. I think that we need to frame it as can it? Is the potential there? And that's a really harsh reality for people across AI to to realize is you may have no intent for any of these potential uses, but if the potential is there, it could and can, then you need to have that framing. You need to have that framing in mind. speaker-0: Agreed. So my take on this story is, and and this is gonna be I don't know, I don't know if we should like get in the habit of making predictions. Maybe not the habit, but this is this is my take on on compliance law. and so this is a a somewhat prediction. I I guess I'm not gonna say I'm not gonna hedge my bets. This is one of my predictions for 2027. I believe chatbot compliance is going to be one of the most vexing issues. For AI governance professionals in 2027. And there are a number of reasons why I feel that way. So, number one, looking just in the United States, the patchwork of laws that we have right now. So, again, our database has 118 laws over 39 states. And I believe I have at least one federal law in there, which is take it down. So we have this growing patchwork, again, specifically looking at chatbot compliance and AI companions. We have 18 laws over 13 different states. There are 118 obligations. And when I say that companions are going or companions and chatbots are going to be a vexing issue, I I want to say very clearly, I don't believe that this is going to be the the biggest risk for organizations. I think when it comes to severity of impact, that is going to be reserved for frontier AI safety and for agents. But I do believe based on sheer volume that chatbot compliance is going to be an issue that more organizations are going to have to struggle with and you know are gonna have to work through. so I still believe that agents and frontier are higher risk. Companion is gonna be lower risk, but I feel that most organizations that are looking at deploying, adopting some type of AI tool, they're starting with chatbots, right? and there are some there are some clear carve outs these organizations, just to be clear. So an AI companion cannot include when a business entity is using it solely for customer service or to provide information about available commercial services or products. the the second carve out providing efficiency improvements or research and technical assistance. And then the third exception, and this is all just for the New York law, right? Remember, we've got eight 17 other laws out there right now. Internal purposes or employee productivity. Now, that doesn't mean that if if you fall into one of those boxes that you shouldn't do a risk assessment, I think you you should you still should. but again, there's just the the volume of of compliance obligations that are about to take hold is going to be vast and is going to require dedicated resources. The last thing on this story, Emily, and then I'll throw it over to you. These compliance requirements are only focusing on Those 18 AI companion and chatbot laws. This does not include laws that may be in what laws that also apply to highly regulated sectors. So if you have a chat bot that is carrying out some type of health insurance, right? Or or some type of mental health, or there's a clinical purpose to the chat bot, that's a whole nother can of worms. If you have a chat bot that's deployed publicly to provide financial financial advice, right? Where should you invest your money? What are you doing with your 401k? That is, a whole nother can of worms as well. And this also does not include all of the other consumer protection lawsuits that we see against organizations like OpenAI right now. so there was this really great article I came across while doing research. This was this is from Tatiana Rice's Substack. This is the November 2nd. Issue. the name of her substack is the algorithmic update. Tatiana Rice is senior director of US legislation at the Future Privacy Forum and a colleague of our guest this week, Justine Gluck, we'll get to hear from her in just a little bit. But she's she's talking about these consumer protection and child online safety issues that are also applying to chatbots as well. So when we look just at a very narrow slice of the compliance requirements. It already looks overwhelming. When you add on the sector specific consumer protection, and other just general online child safety laws, this I think this is again is going to be a vexing issue for AI governance professionals in the coming year. what do what do you think, Emily? Does that does that make sense? speaker-1: It definitely does. What I what I heard a lot in between the lines of what you stated as well is regardless of where an organization or individual is with the AI governance maturity of their program, this is a big beast to take on. How do you maintain business as usual with your compliance and governance operations while being flexible enough to accommodate all these requirements? I mean, the business of compliance and governance can't just stop. But you also Have to put this on your radar and plan accordingly and quickly. speaker-0: Yeah. And and you know, I think this is just another this is just another data point. Well, I mean, really a hundred and eighteen data points if we're talking about obligations that that demonstrate the strong need for talented, skilled AI governance professionals in any organization that is designing, developing, or deploying AI. We've only talked about one section of the laws. The US AI Law database. This is a resource available to all members of the collective. I have a number of different categories here: frontier safety, consequential decision making, general transparency, healthcare, healthcare, synthetic media, algorithmic pricing, audit and assurance, and professions and public institutions. So there are a lot of different ways that your organization may fall within the scope of these laws. And this is just the US. We're saying nothing about. the EU AI Act or or or anything else. So this needs to be on everyone's radar, right? So all right, wonderful. so Emily, I think you've got our third story, OpenAI's four layer monitoring system. So over to you. speaker-1: Thank you so much. Well, on October 29th, OpenAI announced an expanded monitoring architecture. And this happens after the hugging face incident, as we know, exposed a big gap in monitoring coverage during internal cybersecurity evaluations. So this expanded system for monitoring frontier model misalignment includes four layers. The first layer, there's an activation probe that identifies concerning patterns. Second layer is a model-based reviewer to examine activity and context. The third layer involves automated investigation that can compare activity against external evidence. And this includes network logs, external system telemetry, telemetry. Telemetry. speaker-0: ladder. Yeah, telemetry. Telemetry. You got speaker-1: Thanks, Teddy. And fourth, the fourth layer, and and human responders, they're gonna be reviewing these significant alerts that this two new system is intended to generate. And these certain critical alerts are supposed to trigger automatic workload pauses. And OpenAI has also described retrospective testing and prior independent evaluations that identified vulnerabilities. In that earlier monitoring system at the time of the hugging face incident. And among all of these different technical words and all these different facts, I don't know about all of you in our audience, but I've read lots of different iterations of this, I come back to the same kind of takeaways and observations. First, monitoring is a process. This is not a single control. Monitoring is actually a chain. of controls and decisions with lots of different individuals, teams, or stakeholders at various points in that chain. Detection is only the beginning. Okay. in my opinion, investigation, escalation, human review and intervention, they can determine if monitoring is actually reducing risk. Okay. So point two after monitoring being a chain, now that we've established it's a chain of events. It's ongoing, it's lots of individuals, lots of dependencies. Human oversight also, you have to have the authority to act at certain times, at certain points. And a person getting an alert, Kyle, I don't think it automatically means that you have oversight. What can I do as an individual or you do as an individual with that alert? That person needs. timely information, but then also define responsibilities and authority to intervene. What can I do with this important information? And three, implementation itself of monitoring, it's not validation. A monitoring system, it can be deployed and we all know that it can have weaknesses, regardless of the environment. And all this is to say too, what what keeps jumping out at me, team, about this is that There's a difference between implementing a monitoring capability and then demonstrating to your own teams internally, to auditors, to the public, that it actually provides effective oversight. And what's also interesting to me is that we have OpenAI here presenting that this is this is a four-layer system, and these layers are they're not interchangeable safeguards. Okay. Each one is supposed to serve a distinct purpose. And the effectiveness of this overall system and this new architecture, it's designed to work together. And so understanding and verifying that this corrective architecture works as intended, it's really truly how all four of those layers are going to work together. It's not just detection. It's not just saying Kyle or Emily is there to respond to a critical alert. The oversight has to be meaningful. The control has to make sense. And the individual or individuals need authority to act and they need to know what that action is. I'd I'd love your takeaway from this, whether it's one layer that jumps out to you the most from a monitoring perspective or even just how OpenAI approached this incident. speaker-0: Yeah, the so you know, this very clearly ties into the first story that that you reported on and what's going on at Anthropic. I I think this is great. I I believe I believe OpenAI in alongside this article also tried to explain had the, you know, had the hugging face incident taken place today with these layers in place, how how would that, you know, the the the agent escaping its sandbox, how would that have transpired? What what comes to mind like first and foremost when you know I I was hearing you talk about the different layers here is a concept that I think a lot of people should should know and and if not you need to know it, defense in depth, right? When we're talking about implementing controls for governing AI, we're look we're looking not just at one control, but multiple controls. And not all technical controls, but a mix of administrative, technical, and physical controls. and then we can break those down even further by, different type and function. Do we have something that's preventive? Do we have something that's corrective, that's detective, and so forth? a lot of this appears to be detective and corrective, which is great. Those are the ones that you want to have in place. So defense in depth, really important. I I would love to see at some point in the future. An independent report from an organization like Meter or from Redwood Research. These are I know Meter is a nonprofit in the Bay Area that provides that type of independent assessment, audit, oversight, assurance of frontier systems. I'd love to see something from them in the near future that explain to us the efficacy of these systems. So if we get a meter report three, six, nine months from now that says, hey, we were, we were called in, and as we were doing our audit, we saw that, there could have been this, another breakout on a similar or or even worse scale to Hugging Face. But, these four layers in place actually prevented that. And this is how they prevented it. This is what was happening. This is how it was detected. This is how it was prevented. So I I think this is great. I love the fact that they're being transparent. I think there's a lot here that practitioners of everyday agents can can learn and and put into their observability and evaluation programs. I wanna I wanna make sure it works and I wanna hear from an independent organization that it works. speaker-1: I love that. And I think piggybacking off of what you said, two points is first, whether it's redwood, whether it's meter. I wanna know when I say you here, I'm talking figure of speech, what would give someone confidence that this system is actually working as intended and not just generating alerts? what would be that outcome or those metrics or the documentation that says this new four layer architecture system, it works and here's why. And it was corrective and here's why. And then second, I think you raise another great point is that aside from the organization we're talking about, right, I think the transparency here, I think it's a good practitioner lesson and just a theme and how we're evolving that they're being open, whether or not they had to be or not, about the facts because we can all learn from them. And most importantly too, that should be expected. We should expect our AI developers, our AI providers to be this transparent. obviously depending on the circumstances, right? And the depending on the ask. speaker-0: Yeah. it it it reminds me of this this sign that's in the you know like the the waiting lobby of my vet. So when I have to take when I have to take my cats in for, you know, like their annual physical or like a vaccine or something like that. There's so so this vet serves cats and dogs and they've got a sign in there that says, you know, like so many days since there was poop on the floor. I feel like one way that we can see whether or not these controls are effective is how many days do we go before there's an incident? And compare that with the summer of 2026, where I mean, really since, you know, like late spring, it's just been a non-stop, I don't want to say even drip, but just like we we've had this non-stop flood of information about all of these incidents. And it's one after the other. So does does that stop? Do we have independent verification? I think that's what we're looking for because like that's gonna be maybe the greatest, maybe not the greatest evidence, but it's some evidence, right? That we just we we don't hear about this. And then getting independent verification, I think would would be important as well. speaker-1: Right. And speaking from a career trajectory or anyone looking to tap into AI governance or evolve, it's never been a better time in my opinion to become a third party independent auditor. The world is yours. The world is yours. speaker-0: Yeah. And I didn't get to it in the GBL article forty eight story, but another law that has also very strong protections just out of California. This is Adams Law. So I think it's I think it's SB eleven nineteen requires independent child, like child safety and risk assessments for organizations deploying these types of systems. California also had a couple other, maybe like one or two other laws requiring these types of audits for for developers of systems. So just in addition to doing something like a 42001 audit, whether it's as an internal audit or external with a certification body, doing these types of risk assessments is is being required as well. So yeah, I I agree a hundred percent. Great time to get into learning how to audit systems and and So forth. speaker-1: Well well great. I'm gonna turn it over to you, Kyle, for our fourth story about anthropic quad watermarking. speaker-0: All right. So this this isn't so much a news story. I think folks who are in the AI governance space know that in early August the EU AI Act Article 50 went into force. regulators could could start enforcing this. And Article 50 is transparency obligations for providers and deployers of certain AI systems. And so around that deadline, Anthropic had already announced that they had started implementing Text watermarking. And so the the follow-up to this story here is starting on November 3rd, Enthropic Anthropic began adding its invisible text watermark to outputs from Mythos 5, Opus 4.7, and Sonnet 4.6, and reportedly Opus 4.6. And there's been like a staggered deadlines here for Article 50. I believe it was for systems that were deployed before August 2nd. They actually had an additional few months. They have until December 2nd. Models launched after August 2nd had to be compliant on August 2nd. and so this mark applies worldwide. there's no opt-out for it. And again, it comes ahead of this December 2nd deadline, which we see we're recording on November 9th. So, you know, we're less than a month away from this. And so what's the what's the major takeaway here? We don't have time to get into it in today's show, but I I do want to mark that transparency requirements aren't coming only from the EU AI Act, Article 50, right now. California SB942, I believe, also has requirements for AI generated output as well. and chances are probably pretty good that, if if you're if you're using a frontier model like Claude or ChatGPT and they have this watermarking in place. Chances are good that if one of these organizations is the provider of your general purpose AI system, you're in good shape. If you have an open weight system that is self-hosted, then you know you probably need to use an open source solution, which we'll talk about in just a moment. What are the main requirements, real quick, before we jump into the implications here for practitioners? So the the two main requirements for Article 50 are disclosure. Individuals should know that they are interacting with an AI system. and also that synthetic audio, image, video, and text outputs must be marked in a machine readable format and detectable as artificially generated or manipulated. as always, this is not legal advice. This is purely educational. If you think that your organization falls within scope here, definitely go read article 50. but what I want to do is I'll I'll pause there. for just a second, Emily, to see if those requirements for disclosure and watermarking are clear. and then I want to talk a little bit about how AI generated content is actually marked. speaker-1: I think those are very clear and I I'm glad that you surfaced those. And the one thing that's just jumping out at to me as you're as you're reading those and I can save the the rest of my takeaways as well is just making sure people know that the distinction between knowing where content came from and then also just knowing if content is reliable and not commingling those two concepts. speaker-0: Yeah, that's I did not think about that. We should maybe come back to that. Yeah. So I know where it came from, but is it is it trustworthy? Right. Those are those are very different questions. Great. All right. so how is AI generated content labeled? The the two mediums that I looked at were text and then images. I'll start with images because that's the most straightforward. So, for an image that is AI generated, one standard that a lot of organizations are leading on right now is the C2PA standard. C2 meaning two C's. So this is the coalition for content, provenance, and authenticity. I'm pretty sure it's authenticity. So the C2PA standard, this is metadata that is attached to an image. So I generate an image, I put it on social media. someone may be able to download that same file and there will be metadata on that file that that says, hey, this is AI generated. One of the limitations of images is that if there's an image I see on social media and I screenshot that image and then I crop it and repost it, that screenshot does not contain that metadata that says that this was marked. So that's one important limitation for For images that are watermarked. Now, how about text? This is this is a little bit more complicated. I'm gonna try and like articulate this as as straightforwardly as possible. I think if you are a visual person like I am, you would really benefit from a YouTube channel called Computer File. And so if you go to YouTube, you look up computer profile and probably put in like watermarking, this should be one of the top results that surfaces. and so this whoever runs the show or the channel here does a really good job with nice visuals demonstrating how this watermarking process takes place. But in short, I want you to think of it this way. So imagine that as as you are speaking or as you are typing at your keyboard, words that are coming out of your mouth or are popping up on your screen are labeled as either green or red. All right, so all of your words are color-coded as either green or red. And in human speech, those would be roughly even. Half of our words are going to be green, half of our words are going to be red. But for AI generated content, what AI is doing via its its algorithm, the secret key, is it is privileging one color of words over another. So for human-generated text, we may be 50% red, 50% green. For AI generated text, we may be 45% red, 55% green. And the more words, the more text that's generated, the clearer this pattern becomes. And so, you know, if if you're familiar with how large language models work, they are predicting the next word in a sentence. And so the the example that I always use is. Humpty Dumpty sat on a wall, Humpty Dumpty had a great fall, right? We we we all know that fall is the next word. It's 100% got to be fall. but that's what large language models are doing. And and so two limitations or a couple different limitations. One limitation is if you're asking, if you're asking your AI to generate factual information or nursery rhymes where there is always like a very set word you should use, like fall, then The efficacy of watermarking is lessened, right? So if I'm asking you, like, you know, when was Michael Jackson born? Where was he born? What were his parents' name? What what what was what was his greatest hits, right? Or what was his greatest hits? That's all gonna be factual. And so the watermarking is gonna be less efficacious for for those types of outputs. the second thing is just because you just because you're able to detect a watermark. Doesn't mean that it was AI generated. And so the presence of what appears to be a watermark is not 100% conclusive. And you've probably heard a lot of stories of university students who submitted a paper, the professor put it through an AI generated checker. It it told the professor this was 99% AI generated. The student got an A, and now there's a big kerfuffle. Okay. You know, there's an appeal or, you know, there's there's some type of drama at the school. So just because it appears that something is watermarked doesn't mean that it is watermarked. And then the third limitation for AI generated text is if if you are if you are doing a lot of massaging or word smithing of AI generated content, that can alter the efficacy of the watermark as well. And so again, you know, that content, as soon as it's generated, it's gonna have that 45-55 split. All right, just to go back to the explanation. But if you have AI do a rough draft for you, and then you go in and you just you change a couple of things because that's part of your process, not because you want to, deceive the detectors, but it's just part of your process to make it sound a little bit more like you, that can lessen the efficacy of the of the watermarks. last couple of takeaways here, and and then I'll close it out. So, so now what, right? What do, you know, what do AI governance practitioners need to have in mind for the watermarking? The most important thing is just know your organization's role and know how your system is put together. If you are a provider of a system, then under Article 50, there are very clear transparency requirements. If you are the provider of a system, but you have a GPAI provider of a model, then you need to see what are the model providers' requirements and are you benefiting from those. So if you have a system that you're deploying. That you've built, just a wrapper around an anthropic API, for example, then anthropic may be watermarking the text, which would help you in that watermarking area, but you are still going to have other disclosure requirements, for example, right? hey, you're you're generating you're you're working with the system. And then of course, if someone wants a detector, how how would you, what service would you make available to individuals that want to detect whether or not the content from your system. was watermarked. and then finally if you do have an open weight model that you are you know that you fine-tuned or that you're deploying yourself if you are responsible for implementing your own text watermarking features then there is an open source tool that you can go to it's produced by Google. This is synth ID text. So synth just like synthetic S-Y-N-T-H ID, that's all one word, synth ID, text. And Google has made this open source. It's on Hugging Face. You can search it up and get that on your system so that you are compliant. All right. Emily, thoughts, thoughts on that? Implications for practitioners. speaker-1: Sure. I think before I jump back to my comment about providence and accuracy, right? That distinction, I think a takeaway, just a general takeaway from what you just said is if an organization or user if it's relying on an AI generated content detection, right, or or a watermarking, like pretend that's their assume that's their assurance mechanism. what happens when that signal is manipulated or removed or if it's like falsely produced. I think that's a a big governance risk worth exploring. so that that's one piece. But then going back to this concept of the distinction of knowing where that content came from, but then also whether or not you can rely on it, whether or not the accuracy has been proven or shown. So I think I think a watermark can help establish that, Claude potentially contributed to this content, to this content generation. But at the same time, like what does it actually tell me? Is Claude generating it, editing it, or is it just processing this information? Those are all really different claims. They have different downstream impacts and controls associated. And so I'm also interested in this possibility of creating. a false sense of certainty. I think I've talked to you about this a couple of times before, Kyle, in different forums, because technical signals are useful, right? I also say this because I come from a deep dense world of contract negotiations and watermarking draft versus in progress and confidential all of those, right? Those have been inherent in my work. But like in this case, the nature of AI as such, I wouldn't necessarily want people to draw a conclusion. When this signal itself might not be designed to support it, right? I think we need to have our technical mechanisms make sure that they're achieving those attended regulatory and organizational transparency objectives. But as we've outlined here, I think that's that's tricky. How do we account for accuracy of the content? How do we account for trustworthiness? And I also want to know how much the AI contributed as well. A lot of thoughts on this. speaker-0: Yeah. Yeah, those are all good points. And I think one example I gave earlier was the AI does a first draft for you and then you polish it and make it a final draft. You could also see a watermark perhaps the other way around where you do the first draft and then you give it to AI to polish and clean up and and give you some suggestions. Yeah. So I I like the idea there of being able to detect percentage, right? And sure. Or or even con confidence intervals. So yeah, great great things to keep in mind. speaker-1: And and also that's a that's a great point as well because depending on your Prompt your prompt skills, right? How how good are you at crafting prompts? It's possible that my prompt was intended to really have this be a checkpoint for something I think is in final form. And perhaps the AI generates recommendations way beyond the scope of what my intent was, because it interpreted a prompt to mean revamp this entire podcast episode, which no, that didn't happen. But but in theory, that that's how we could interpret it. And then it looks as though that percentage point of AI generated versus what came from our own human condition and critical thinking, that's a big discrepancy. How do you account for that variance then? speaker-0: Yeah. speaker-1: Maybe we should strike that perk, by the way. speaker-0: no, I th I think that's good. I I think that's good, good back and forth. All right. so wonderful. So there's our news roundup for this week. After the break, we are going to speak with Justine Gluck from the Future of Privacy Forum. Justine is an expert on AI, chatbot, and companion laws. She's gonna give us her thoughts on the the patchwork of of laws that's out there right now. And really hope you enjoy the the interview. So we will see you after the break. The AI Governance Collective Podcast is brought to you by the AI Governance Collective. Are you an aspiring or practicing AI Governance professional? Become a member of the AI Governance Collective today and join a growing network of global AI Governance Practitioners. Access our growing library of on-demand training and templates, live workshops and panels with experts, weekly office hours, our weekly newsletter, The Alignment. And access to the USAI Law database, as well as local in-person chapters and industry groups. Learn more at www.aigovcollective.com. That's AIGO V as Invictor Collective.com. Again, AIGov Collective.com. Welcome back. Today we are joined by Justine Gluck. She is a policy analyst for AI policy and legislation at the Future of Privacy Forum, where she leads research and analysis on emerging state and federal AI legislation. Her work focuses particularly on the regulation of companion chatbots and AI frontier models, including emerging requirements for risk assessments, independent audits, and accountability. Previously, Justine served as a legislative aide in the California legislator. Helping draft and advanced legislation on topics including privacy, AI, and higher education. She has also conducted research on semiconductor policy and the CHIPS and Science Act at institutions, including the Center for American Progress and Harvard Kennedy School's Belfur Center for Science and International Affairs. Justine holds a Master of Arts and Law and Diplomacy from the Fletcher School at Tufts University, and a Bachelor of Arts from Pomona College. Welcome, Justine. Thank you so much for joining us. speaker-2: Thanks for having me. Lovely to be here. speaker-0: All right. Emilid, over to you with the first question. speaker-1: Sure. So as Dr. David just mentioned, we know you worked as a legislative aide in California and you were involved with drafting and advancing bills on privacy, artificial intelligence, higher in higher education for that matter. And we really want to understand what that experience taught you and specifically about how legislation gets written, how it gets negotiated. Yeah. Where are practitioners in this AI space? Where can they influence? What can they influence in that process? speaker-2: Yeah, yeah, my time in the California legislature was probably one of the most impactful professional experiences to really get to see how the sausage is made, so to speak, from the inside. And I think one of the biggest things I learned from that time is really how much writing legislation is such an iterative process. I think that's always useful for folks to understand. so a bill usually starts right with a particular policy goal or a problem the legislator wants to address. But then the process to bring that idea into actual statutory language is a lot longer and a lot more complicated. You're obviously often balancing input from advocates, from industry, from technical experts, legislative committees, tons of different stakeholders who all might even just agree on the problem, but really have very different views on how to solve it. So I think that experience has been extremely impactful for my work now. in FPF, because here I think a lot about the gap between maybe perhaps what lawmakers intend to accomplish and how some of the resulting requirements in law might actually work in practice. so I think for practitioners in particular, one of the more valuable ways to engage in that process is to help bridge that gap. we see a lot of lawmakers, especially in the tech policy space, who are extremely knowledgeable, have an incredible staff. And others that are just more building their portfolio and trying to understand how to talk to folks who have more of the technical expertise for some of the implementation challenges in some of these laws, especially with AI, which is so new to all of us. so I think again, on the practitioner side, spaces for opportunities are include helping to provide concrete examples to lawmakers, explaining how a proposed requirement might actually operate in practice. Things like that can be really useful when you're a legislative staffer. And the saying goes, when you're a staffer, you're a mile wide and an inch deep in just the terms of the amount of policy topics that you usually have to cover. so yeah, and I'd emphasize that engaging early always matters too. bills can move really, really fast, especially in California, which has like a supermajority. so making substantive changes can happen quickly and it's good to keep abreast of the process for all really early on. Yeah. speaker-1: Thank you for sharing that. And it it made me think of of one more thing, Kyle. What is your take or lens on trade associations and organizations and different industries being able to use their trade associations as a path into that legislative influence? speaker-2: Yeah. for me, yeah. I think again, there are folks that are operating in the legislative environment from all backgrounds, so trade association, advocates, civil society, academics, really there is a lot of paths forward through that. I think again, when it comes to like influencing legislation, really folks that are on the ground in the legislature are listening to all the stakeholders, which I know might sound like a cop out answer, but I think that's definitely a viable path as well as you know, talking through other folks or other organizations. speaker-1: Yeah. I love what you said there. Can you walk us through that list one more time of the different funnels that you mentioned that included trade associations? I loved that. I want to make sure I caught that. speaker-2: Yeah. choice associations are one, of course, academics, folks directly coming from industry, folks who are independent researchers or technical experts, advocates and folks from more nonprofit or civil society organizations, as well as folks that influence the process from the inside. So folks from other government agencies have influence on the legislative process or from the governor's office themselves. speaker-1: Yeah. Thank you. speaker-0: That that's great. Emily and I were completely aligned because that that was basically my follow-up question. We speaker-1: We should work together or something. speaker-0: we we should. We should start a podcast. We should. I should. all right. I wanna I wanna now move a little bit into the more specific US state level law, A AI legislation landscape. the the numbers, you know, here as we're closing on 2026 are are pretty staggering. the transparency coalition has documented 115 AI-related laws have been passed in 30 states. That was from beginning of January just to just to mid October. And when we compare that with last year, if the numbers are correct, I think we had about 73 laws across 27 states. One thing I'm hearing a lot is we need to regulate AI. My rebuttal is always it is already being regulated at the state level, as we can see here. So I'm hoping for our audience you can help do a little bit of of level setting for folks that aren't following the state level landscape. How can we start to think about all of these laws, this flurry of activity that's taking place right now? speaker-2: Sure. Yeah, I'll start by saying to folks who aren't following the state AI landscape, you should be. this is where all the action is, this is where laws are getting made. This is the inspiration for what legislation might look like federally. there's a ton of laws, as you mentioned, that are on the book. Things are moving really quickly, so just level setting there. It is an important place to be watching. in terms of some of the numbers and figures that you threw out, it's always difficult because folks are counting laws in different ways. we at FPF also track All the legislation that we deem to be critical or important on the state level. And those numbers always vary. So sometimes, for example, we don't track legislation that is specifically focused on AI's use by the government or on deepfakes or hell or some other different areas that maybe are impose less consequential private sector impacts. So flagging that, that some of those numbers also include things about education or very targeted laws that may have Less impact, quote unquote, for certain companies that might be interested in following this space. but then to speak more about the landscape, I do think that the most important thing to understand when looking at the state landscape and the growth in numbers is that there really isn't one uniform approach to AI regulation emerging across the states. we've seen a tremendous amount of legislative activity, and that has increased in the last three years, every year. And I imagine will increase in 2027 and we'll see those numbers get bigger and bigger. but what's interesting is that states are continuing to regulate different like applications of AI, different categories of harm, different technologies. the way I've kind of processed it at FPF, which might be helpful to share, is kind of organizing the landscape around like three major approaches. So the first is context or like use-specific regulation. So AI laws that are specifically targeting Healthcare, employment, maybe other sectors. And the second is technology-specific regulation. So that might be the laws that we're seeing balloon right now around companion chatbots, around frontier models. So looking at the technology itself. And then the third is a little bit more nebulous, but laws that are really focused on liability and accountability. So maybe states using liability frameworks, thinking about how they should apply to AI, really that being the focus of the laws. So again, I think what's helpful is again, there's not really this singular model we're seeing in the states, but we are seeing a lot of interest and we're seeing those targeted laws in different spaces. I think what will be interesting is not just like how many laws are passing or being introduced, but which ones are making it across the finish line. We're seeing certain categories that are easier, so to speak, for lawmakers to pass into law versus others that we see introduced or thrown out there that don't make it across the finish line. And that's been interesting to follow as well. speaker-0: Great. Yeah. Emily, over to you. speaker-1: Thanks, Dr. David. I want to lay some foundation for this third question. And I say this for from experience as well, is just watching the climate shift when we're talking about lawmakers at the state level. And you said something earlier to one of our questions that hits hits so closely to what practitioners are dealing with now. And that's really diving into legislative intent and understanding the policy behind that, the spirit of what's being written there. And I think when we look at states specifically, it's really obvious to go back to what industry is huge in this state? What's that economic footprint look like that might be influencing potential AI usage? So the future of privacy forums state of the state of AI report, it found that lawmakers are moving away from this broad Colorado style high risk AI bill. And now it is more narrow. More sector specific. Our audience really wants to understand the different industries and perhaps different drivers for this shift. This shift from Euro to sector specific. And do you think twofold here, what does this mean for an organization and their AI governance program? Does that make it harder to easier? Excuse me. Does that make it harder to run or easier to run? speaker-2: Yeah. Yeah, that's a great question. I think you definitely pinpointed what I see as the biggest trend in the last probably three years when it comes to AI regulation. just to reiterate there, like I feel like where we started in the US was looking at what the EU had done, the EU AI Act, and wondering, hey, is there space for something like that to happen here in the US? and I think in twenty twenty four we saw some experimentation around that in states like Colorado, some attempts in California. and ultimately just this year, seeing Califor or Colorado, sorry, pull back on the law that they had signed two years ago and switch it to be something a bit more narrowly targeted is a really good clear example of how that tide is shifting. I guess to answer some of those questions, I think the first one of the main reasons for why we've seen that in the last couple of years is that just frankly, comprehensive AI legislation is very complicated to draft and implement when you're trying to regulate with some of these off have like high risk systems across Healthcare, employment, finance, education, in one framework, you're gonna run into a lot of difficult questions around definitions and scope and how different sectors are actually using AI. It just becomes a very complicated bill to write. and I think folks are have really recognized that. there's also been obviously a lot of political shifts in the last couple of years that very, very heavily have influenced the kind of legislation being introduced. And I think there's a lot of lawmakers who see more targeted legislation as More politically feasible of moving through the process when there is a lot of pushback around AI regulation in some states and federally. and I think that there's also been a lot, you mentioned states having particular sectors that are really influential. I think we've also seen some lawmakers find a lot of consensus around particular like use cases or harms. I think employment is a really good example. we saw Again, we saw the Caldera AI Act and some more comprehensive attempts originally to regulate multiple sectors at once. And what we're seeing is some of those regulatory mechanisms aren't leaving, but they're becoming more narrowly targeted in terms of scope. So putting those kind of laws just to the employment sector, which is a space where folks have like first-hand experience. So everyone in many ways is an employee. and there's a lot of interest there. And there's also just a lot of risks that have emerged in the news, things like that that folks are kind of grappling with. so something I do want to point out though is that some of the shift away from the comprehensive frameworks doesn't mean that we're losing some of the more comprehensive governance concepts, I guess so to speak. So we're seeing some of the mechanisms that were in this original type of legislation around like risk assessments or evaluations reappear now in 2026 and into 2027 in some more like targeted legislation. And then I guess to turn to practitioners a little bit. that obviously is creating a bit of a trade-off. So it might be easier in some of these more narrow bills and laws to understand the obligations associated with a more narrow and specific application. But then the flip side, because we aren't coalescing around a broad, singular, comprehensive model, we're gonna see a lot of overlapping and differently defined requirements across states. So if there's an aerobill unemployment in California and they kind of start that process and Colorado is starting the same process, for example, by the end we're gonna probably have two laws that look a bit different. And that's just something that I think will continue to be a challenge and for folks to keep in mind as they are thinking about compliance. As well as just in general, this despite laws getting narrower and more sector specific, there may be still overlaps in different regimes. So We might see someone regulating employment, but they don't realize that that's also impacting the healthcare sector. So folks should always think about that overlap and again focusing on scope of some of the laws that might be a bit broader than maybe they appear in practice. speaker-1: Great. Thank you for sharing that. And a lot of what you just said there, Justine, resonates well. And I'm I'm thinking about organizations, perhaps healthcare, that do business lots of different states and operationally, how complex that must be. And where's that balance of organization internally and compliance? Dr. David, over to you. speaker-0: Yeah, I had a little bit of a a follow-up, which also segues into my next question on chatbots. So privacy is a close reference for me and I I think for a lot of our audience and how comprehensive privacy legislation has developed over the last, you know, let's just say 10 years or so. I I think in in most regards, you can take something like the C CPA, the CPRA, and if your organization meets that standard. You're gonna satisfy most of the requirements for the other laws as well. As we pivot to start talking about chatbots, which is, you know, one of the areas that you are are an expert in, are you seeing a similar degree of overlap? Is there like a a gold standard chat bot law you feel that if organizations make this their target, then that's gonna make compliance a little more manageable for them? speaker-2: Yeah, that's actually such a great question. I have an awesome colleague here at FPF who I would be remiss to not shout out, Jordan Francis here, who's an expert in comprehensive privacy law. So we've spoken quite a bit about this sort of topic where you're you're right. There's kind of a gold standard where it makes compliance easier. And that has just, in my opinion, not been the case in AI. maybe in some other sectors. So frontier model legislation is a good example of right now, we have Three laws in the books. Illinois is the strongest of the three. If you comply with Illinois, you should be in scope in California and New York. that is that I would say is the case. chatbots is a great example of one where I don't think that is the case. we have, gosh, the numbers keep changing. I think we're at 19 chatbot-specific laws passed in 2025 and 2026. that's just legislation that uses the word chatbot or uses the word companion. There are other laws in the books that would implicate chatbots. And I'm happy to expand more on this and further questions, but I would say that there are extremely different approaches amongst these 19 laws. I really don't think that there is one state's definition of what a chappa is that is I completely identical to any other state. there might be a different carvat or a slight swish of language, which means that it's very possible that you are in scope of five laws and that five others or so on and so forth. and a lot of the requirements are very different. and we'll probably speak a bit more about California and New York, but I think that's a good example of two new laws that are in the books where you as a company are being asked to comply very differently. and being in scope in California very much does not mean you'd be in scope of New York. and I would say that's the case for a lot of the chat bot laws. There's a ton of commonality. We're seeing a lot of common building blocks. The laws look similar in some ways, in many ways, but the language and Thus the requirements are very different. And I wouldn't say there really is right now a gold skin standard law. Sorry. where you would be in compliance across the ecosystem. but California and New York are probably the two to be closely watching. Yeah. speaker-0: Right. Yeah. So so before we pivot to California, New York, because we do want to get your thoughts on that. when we when we look at the different buckets that you were talking about earlier, you know, context, use case specific, technology, liability, I I think you said we could put chatbots in that, you know, technology specific bucket there. We've had a lot of action here. So I think a a recent report from FPF has said 124 chatbot bills across thirty-seven states in twenty twenty six. Yeah. What is it about chat bots that it looks like lawmakers across the country are agreeing on? And what is the what is a common, you know, bucket of concerns that they're all trying to address? speaker-2: Yeah, so interesting. as you said, I can't even count chatbots in that technology-specific bucket anymore because there's just too many, too many bills to fit with other with other categories. so yeah, it's been an absolutely intensely fast growing topic. two pretty strong forces that are pushing this forward. the first is a lot of high-profile litigation surrounding chatbot harms, especially cases with minors who unfortunately have experienced self-harm or other forms of emotional dependence, so to speak, on AI companions. We've seen cases just really balloon and lawmakers are very directly responding to those. We will see often a case come out about a certain topic and then have see lawmakers amend their bill to kind of respond to that case. So that has been very popular. Another is just lessons from social media regulation. We've seen lawmakers and policymakers really speak very openly about how they many feel they were too slow to respond to some of the potential harms of social media and that they very much don't want to repeat that experience on AI. And so we're seeing that push and that interest come from there. but something else I think that's worth pointing out and it's very key to the laws is that I think there's something very distinctive about chatbots themselves that has mm led to so much regulatory attention. The way I kind of talk about it in our report here is that Unlike a traditional search engine or social media feed, right? A chatbot obviously can engage in very sustained and personal conversations with someone, with the user. So it remembers what they say, it simulates emotional responses, it can create again this relationship impression. That's part of it. And just in general is having like this iterative, yeah, conversation back and forth with the user. So lawmakers as opposed to Previous types of regulation aren't just concerned about this exposure to content. They're concerned about conduct. They're concerned about relational behavior. They're concerned about design of the chatbot itself. because the chatbot's design is what is producing responses that, you know, ultimately you don't have control over once a chatbot starts talking to a user to a degree. so yeah, we've seen that. Specific concern really drive legislation. In 2025, most of these laws were very narrow and specifically around disclosures and crisis response. So telling users, hey, you're talking to a chatbot, and then crisis response being, hey, if you're expressing a certain degree of harm, we're gonna direct you to a crisis hotline. And that's what the first Wacallum wave of chatbot laws focused on. But in 2026, we've really, really seen states move into things like parental controls. restrictions on emotionally manipulative design and conversational memory, and more comprehensive safety obligations that I think responds to those specific harms. Yeah. speaker-0: That's great. Yeah. All right, Emily, over to you. speaker-1: One more of an aside here, and I'll make it really brief. We talk about companions and we're talking about all these different complex human layers, right? That are tied into young users, right? Under 18. And I'm thinking back to my grade school days when I had pen travel. And we would write notes and letters, and I would anxiously wait on Fridays, the second Friday of the month, for them to say, you know, Miss Sherry's class from Cherry Tree. Here are the letters. And what a different world we live in now where the ease of access to technology for anyone for that matter is so apparent. But then also just that immediate gratification of a response and how quickly that happens, aside from what that actual messaging is, just that fundamental shift in gratification is working against us as well. speaker-2: Yeah, we see a lot of interest in this concept of addiction or user well being. And I think that is exactly what you're saying. It's what I'm sure we're many of us I'm assuming are all using chatbots in some degree of our life. And I think lawmakers are no exception to that. So I think folks are really reflecting on their own personal experiences when they're drafting some of this legislation too, which is always very impactful for getting a bill across the finish line. Yeah. speaker-1: Right. I want to draw your attention back to California and New York. And we've we've talked about it a little bit so far. But California and New York, we all know they took different approaches to youth chatbot safety. On one hand, we have this California Senate bill and called Adams Law, and it's establishing that governance framework and it's including child safety risk assessments and independent audits. But then on the other side of the table, we have this New York Senate bill. And there's a more restrictive approach and it's prohibiting twelve categories of unsafe artificial intelligence companion features. we have a company, let's say we have an organization. It's operating in both states to your point. What what does compliance even look like day to day? I I'm I'm thinking of product and safety teams and just trying to navigate those differences between approaches. speaker-2: Yeah, no, I'll definitely speak to that. Before I speak to that, I'll add this just for folks who aren't maybe aware of these laws. my thesis this year in 2026 has been we're gonna see chatbot legislation be like almost a microcosm and a testing ground for future AI legislation moving forward. So folks are gonna be lawmakers, policymakers are going to be experimenting with different regulatory mechanisms and requirements in this contained space of youth and chatbots. That I really believe we're gonna see move into broader legislation in 2027. so I'll just say from the top as I talk about compliance here, that if you perhaps are listening and you are not thinking your organization is in scope of these laws, why does it matter? We don't, we don't talk to kids, we don't talk, we don't have a chat, but I still think that they are laws worth looking at and reading because I think some of the parts of these laws and requirements are things that we're gonna see carried over into other AI legislation next year. Of course, that's my personal opinion. That is my guess. we won't know for sure. But I just want to flag that because I think that these are probably, to me, two of the most interesting laws signed in 2026 so far. and yeah, to speak, I guess, to your compliance on them, yeah, a quick overview again. So, California's law is again, both of these are second-generation chatbot laws. So, California and New York were the first states to pass a chatbot law in the US. They one year later Came back to do a different second iteration, which again is speaking to just how fast we're seeing things move in AI legislation right now, especially on the state level. but California has a more, I guess I'll say governance transparency-oriented approach. So it's requiring companies again to assess child safety risks, safeguards, there are risk assessments, there are independent third-party audit requirements, which is a big thing to flag, testing requirements. we have restrictions on targeted advertising in here. we have restrictions on personalization. So a lot of different things going on that is worth flagging. And then again, as you mentioned, New York is a bit broader in some ways in that it identifies these 12 categories of unsaved companion features and says you know, flattery, emotional appeals, so on and so forth, kind of leaves it in many ways to operators to figure those out and protect children in that way. something else say about compliance, and when you're thinking about both of these laws. Is that I would say California is more prescriptive on how operators must manage child safety. And New York kind of reverses that balance. So it's on directly prohibiting broadly defined features. So something that's key to most chatbot laws is there's language around reasonable measures. So if you have a chatbot taking reasonable measures to prevent certain risks, New York doesn't have that language. It's essentially more just directly prohibiting defined features without that. like more reasonable measures, language. So that's something to really keep in mind when it comes to compliance. as well as like doing a risk assessment in California, then is not exactly going to protect you under New York's law where it's really focused and again, just directly prohibiting that these outputs or these behaviors don't occur on your chatbot. So just a key thing to think about. I think another really interesting maybe micro example from the laws is their approaches to the topic of like conversational memory. So New York, for example, has a provision that says you cannot along the lines of you cannot use certain personal information from miners acquired more than 12 hours earlier in an earlier like chatbot session. And California does something different where they ask operators of chatbots to disable persistent memory by default. so if you're a minor by default under like your chatbot, that will be disabled. with some exceptions, some broad exceptions, I would say. but again, you can see in that example that an operator in California, New York is dealing with persistent conversational memory. Doing a default in California would be into compliance of their law, but it wouldn't, it wouldn't fall into compliance of New York's law. So same topics, very, very different requirements. and we see that kind of throughout. So I would just say treating them as Two very different laws and just seeing if there are areas of overlap. there are some when it comes to some of the relational relationship aspects, but not when it comes to some of the more requirements. I would also say, just for folks who are think that they are in scope of the New York law in particular, something really interesting in there that maybe hasn't gotten as much news, but is worth thinking about is part of the enforcement of that law is something called algorithmic disgorgement, which we haven't really seen before. So essentially, It can involve the destruction of unlawfully obtained data and algorithms trained on that data, which goes like far beyond financial penalties and may actually require operators to like destroy or like unlearn their AI models, which is something we haven't seen. We've seen the FTC do some stuff on this before, but we haven't really seen it appear in law. I think that has some really interesting implications, especially for chatbots, which are often trained right on like large or mixed data sets where we're moving. That particular information could be really challenging. And I think since we're seeing that emerge in New York, we might see that again kind of come into play in other laws in the future. So a very, very interesting part of that law that is important to keep in mind as you're thinking about compliance, because there's some interesting implications there. Yeah. speaker-1: Thank you. Over to you, Doctor David. speaker-0: Yeah. so again, justine, thanks so much for your time. We've got one more closing question that I think we're gonna make a regular staple for all of our guests. we'd like to know, you know, from your perspective, what's one lesson that AI governance practitioners need to understand before putting governance into practice? speaker-2: Yeah, an interesting one I think about a lot right now is that I think one of the most important things to keep in mind at this juncture, thinking about some of the laws, is that AI governance maybe can't just be the responsibility of the legal and compliance teams. we're seeing state laws regulate again how AI products are actually designed, how they're operated. We're seeing audits in the frontier model space asking to happen very early in the process when folks are even training data. so I think when we're seeing that some of those requirements, those aren't things that can be just addressed maybe later on through disclosure or policy document. So they require like those teams to work with engineers and product designers and trust and safety teams from the beginning of the process, which I'm sure is what folks are already doing, but just flagging it that is gonna become, I think, more and more impactful as we're seeing some of these laws emerge. Yeah. speaker-0: I think Emily and I were having this conversation earlier that one of the number one questions we get is how can we persuade leadership or other offices to let us in the room in the beginning? so that's I I mean, Emily and I both stand on that soapbox. I I think that's a great lesson. More organizations need to get their their governance and and other, you know, legal compliance folks all together very early in the process. speaker-2: Yeah, yeah. Always helpful. speaker-1: I love that too. I want to piggyback. I know we're we're pushing for time of something you set though is that it a theme, a core theme of what you just said is it takes everyone. Each person has a responsibility. And if anything, those users, those users are gold. They can tell us about what's actually happening. they give a different persona. And sometimes I think those users are lost, are lost in the governance story inadvertently. speaker-0: All right. thank you so much, Justine Gluck. I I guess one final question. If folks wanna find your work online and and read some of this great research that you've done, where's the best place for them to go? speaker-2: Just I'm at the future of privacy forum, so check out FPF.org. we do some awesome work across AI, which is my space, but also youth online safety and data privacy at large. So yeah, check us out there. speaker-1: I will. I'm a huge fan. I I loved everything that I I've researched about you and I'm I'm a huge fan. For what you do. speaker-2: well we're on thank you. Thanks for having me. Yeah. There's so much work to go around these days. So tons of folks that are doing awesome work. Yeah. speaker-0: There is. All right. Thanks so much, Justine. Take care. speaker-2: Thank you. speaker-0: The AI Governance Collective Podcast is brought to you by the AI Governance Collective. I founded the AI Governance Collective because I needed a community of like-minded individuals who were trying to figure out AI governance themselves. AI governance, as most people are thinking about it today, is still a brand new field, really only about three or four years old. We are all building the plane. As we fly it. That is, we are figuring it out together, often as individuals within small and medium or even large size organizations. I want to share with you real quick a really exciting initiative that we've had developing in the collective over the past couple months. So a lot of organizations are looking at standing up an AI governance body, and they're turning to the NIST AI risk management framework. Well, earlier this year we stood up a working group, and that working group together has formed a number of different reading groups. So we had volunteers read through the full NIST AI RMF core itself and report back presentations to the group, what they had learned, what is the framework trying to do, and then working through each of the four functions: govern, map, measure, and manage. Thinking about how do these actually apply? How would we implement these different actions within an organization? After our reading group, we brought in an experienced facilitator, Jim Wiggins, who carried out an eight-hour virtual workshop on a Saturday for the full collective. We are now in the process of going through the generative AI profile and also building out a library of 27 templates or artifacts. These include an AI governance policy, a risk management procedure, legal and regulatory requirements register, a system inventory, and so forth and so on. The objective is by the time we finish a series of sprints, we will have a full template library available to all members of the collective. The objective is that anyone will be able to take this library of templates and stand up. An AI governance program that adheres to NIST within their organization from scratch. This is just one of the many initiatives that we have going on in the collective. Again, I believe this is the best place for professional development and support in AI governance today. You can learn more at www.aigovcollective.com. That's aiGov Collective.com. Again, AIGOV Collective.com. Welcome back. So Emily, what'd you think about our interview with Justine Glock? speaker-1: Justine Gluck, first of all, phenomenal AI practitioner. I'm so glad we had her. Two main things, Kyle, stick out from our conversation with Justine. First, I really like the point Justine made about how to approach this state legislative environment for AI. And one thing that she honed in on was looking for a broader policy direction. So we may have countless state AI laws. Maybe they apply to you or your organization, maybe they don't. But what's really important for us to know is that these aren't separate pieces of compliance obligations that all of us across all industries, across all AI governance maturity points in time, we should approach this as potential future federal AI legislation. And she wasn't suggesting that all of these laws will be morphed into federal AI legislation, but These are going to be reflected. These themes could be a combination of some approaches that are emerging at our state levels. So from that standpoint, being aware and also understanding what each of these state AI laws are is very important for all of us. And then second, somewhat related to that too, Kyle, is this emphasis on legislative intent. And I say legislative intent as well because I myself by trade am actually a public affairs policy scholar, prior to law school, and then had this this approach from a legal perspective, how to approach different types of legislation and case law and and whatnot, but we can never lose sight of what the purpose is beside or behind a legislation. And so we can look at the text and we can pull out what these requirements mean for our organization. We can pull out what definitions may be But it's most important to look beneath the surface and say, wait a minute, what problem is this legislation really designed to address? Does this apply to me? How can I think about this more broadly? Because we we shouldn't just go right to what does this require? It's kind of like the same approach we take with our impact assessments and our vendor risk assessments. Tell me the why first. What's that problem? that this AI is intended to address. And I think applying that same reasoning when it comes to legislation, she was saying that'll make it seem less overwhelming because we all know it's it's very overwhelming to sift through all of these different laws. Yeah. How about you? Love to know your your take on this. speaker-0: Yeah, I I agree with both of those takeaways. you know, one way I had thought about, you know, in my in my previous life as a historian, looking at how how certain ideas, whether institutional or cultural or societal, had evolved over time. And how, you know, as a historian, one thing we're doing is we're tracking this genealogy, so to speak, or this evolution. And you know what you've said just now and and then what I got after speaking with Justine is I'm I'm gonna change what I've been doing for pretty much my whole life. For for much of my life, I've I've discounted bills, legislative bills in the sense that if it's not yet a law, I don't need to to pay attention to it. And I think as as a compliance person, as an educator, that's something that I I want to change because it's very clear that as we're As we see different iterations, speaking to the iterative process that Justine said, we can see which ideas hang on. And so we may see over several legislative sessions that there are these handful of ideas that are persistent, that we have bipartisan support for. And I think when we see enough of that at the state level, then you know we can anticipate what is likely to be successful at the federal level. And then from an operational reality perspective, we can ensure that our organizations are as prepared as possible for operational reality. so that's that's number one. And then number two, legislative intent. Again, I think as as an AI governance professional, and and this is a metaphor I come back to often, we are the the hub amongst a number of spokes in a wheel. And what I mean by that is As an AI governance professional, you're you are working with functions throughout the organization. You're working with data, you're working with legal, you're working with GRC, you're working with product development, so forth and so on. And it's important that you're able to break down, synthesize what the requirements are, and then communicate them to this diverse group of stakeholders. And an important part of that, you know, if a lot of our compliance requirements are coming from laws and regulations is understanding that legislative intent. So I agree with that. I don't have a whole lot to add. I think, you know, what and again this piggybacks on your point. One thing that Justine said in closing there, you know, what's one lesson for governance professionals? Compliance cannot remain solely in the domain of legal. And I I don't think that's the case in a lot of organizations. But there are a handful of organizations that are still thinking that way. and so again, cross-functional, diverse nature of AI governance is really important. And then speaking specifically to the chatbot issue, both what Justine had to say and what we talked about in our news roundup, there is no one size fits all law right now. So within the United States for privacy, something like CCPA or the the GDPR, for example, you can look at those as our sealing, either domestically or internationally. If you satisfy the requirements of the GDPR, chances are probably pretty good that you cover or you check most of the boxes when it comes to the CCPA and so forth. We don't yet have something like that for AI, but I feel like we're we're working toward it. And if you were looking for not a one size fits all, but maybe a two-size fits all. If you took a combination of Adams Law in California, I believe it's SB eleven nineteen, and then you take GBL article forty-eight, I think satisfying those two is going to get you pretty close to helping you to comply with all 18 of the laws that are that are out there. so that that's another area to to keep an eye on. speaker-1: And but before we we close this out as well, I really like that word you use, that genealogy, and not that we're talking about human sciences here or anything, but survival of the fittest and what ideas survive through that genealogy as we're following legislation will be interesting. speaker-0: Yes. Yeah. So you're you're taking me way back to the the the genealogy of morals if we have any academic wonks out there, but that is for a very different audience. So we will we will we will not dive into that. All right, Emily, that we that that's a wrap on our our first full episode. really looking forward to continuing to do this every week with you. Really excited to hear back from our audience on on what they like and and what type of segments they'd like to see in the future. Any closing words for our audience? speaker-1: No, thank you for for joining. And once again too, we we hope to bring you wonderful interviews like this in the future and more story analysis and takeaways for all of you to have as you go about your AI governance journeys. speaker-0: Great, wonderful. All right, thanks so much, Emily. Thanks so much, listeners. We will talk to you next week. Take care. Bye.