Latest / 80,000 Hours Podcast / #246 – Sneha Revanur on how a small team of activists helped pass America's landmark AI safety laws
Transcript
- Sneha Revanur: Looking out to 2028, it’s no doubt that this very well could be the most important presidential election year in history. Whether or not AI timelines are shorter or longer, there’s a whole range of outcomes where this could either be the person who will be in charge of the intelligence explosion, or in any case, this will be a person who is making some very major world-historic decisions about AI. And what will it mean to empower this person with superintelligence potentially and who would we trust to hold the keys in that moment? Zershaaneh Qureshi: Today, I’m speaking with Sneha Revanur, who pretty remarkably founded a US nonprofit at the age of only 15. So this nonprofit, Encode, it’s focused on AI advocacy and has done a lot of exciting things, like helping to pass landmark AI-safety laws in California and New York, successfully fighting against federal preemption, and helping to criminalise the sharing of deepfake pornography. Now, Sneha has been doing this for about six years now, so I’m just excited to get to talk to her about what she’s learned from trying to influence AI policy from the outside through advocacy, and through lobbying. Sneha, thank you so much for joining us. Sneha Revanur: Thank you for having me. Zershaaneh Qureshi: So when you founded Encode, it was focused on immediate societal harms from AI — things like criminal justice, for example — but nowadays you’re particularly concerned about existential-scale risks from future, more advanced AI systems. Can you talk us through how that happened? Sneha Revanur: Yeah, I love telling this story. I feel like it’s a bit of an outlier in AI safety circles. I didn’t grow up reading sci-fi. It took me a really long time to feel in my bones this was the most important century. I had all the hallmarks of a Bay Area teenage striver who just wanted to spend all that time prestige-farming or whatever. If you were tracking the trend lines in deep learning, then you would have seen it coming, once we had transformers and learned the bitter lesson, that it was only a matter of time before we had much more powerful AI. But that wasn’t me yet. In 2020, I do think that I had one pretty good general instinct though, which was that I thought that AI was still very underpriced politically. I thought that it would eventually be a top voting issue, and as a result we needed a massive investment in politics and advocacy around AI. And I think the basic story that I had in mind was that society would over time start to delegate more and more authority to AI — criminal justice was the motivating example for me here — and kind of gradually hand over the keys in scary and far-reaching ways. At the time, I would have told you that the main risk from this was that AI wouldn’t be good enough. That humans would systematically overestimate AI capabilities and systematically defer to biased or otherwise unreliable systems for the sake of efficiency or whatever. And now what freaks me out more is that society is greatly underestimating and underpreparing for AI capabilities. So what gives? Honestly, it wasn’t a specific argument that changed my mind. In fact, I had friends who approached me and tried to reason with me because they knew that I was doing AI advocacy. And that did not really do much to break down my cognitive walls at first. Honestly, it was just like ChatGPT being released my freshman year of college. It has been very interesting to form my core views on the world with the arc of AI progress as my backdrop. And as soon as I was interacting with LLMs every day, I couldn’t help but feel humbled by how wrong and how dismissive I had been before. Because what really motivated my dismissal before was that I was fundamentally a capabilities sceptic who was imagining the future via basically the wrong mode of thinking. I think it took experiencing LLMs for me to realise that to avoid being blindsided again, I would have toggle into a very different mode altogether, one that is very tolerant of a super high and initially uncomfortable degree of extrapolation and imagination. Zershaaneh Qureshi: A lot of people though who we come across who, like you, are anticipating that AI is going to get much more capable and pose these catastrophic risks — so things like losing control over AI systems or AI extremely concentrating power within society — a lot of the people who are worried about these things are particularly concerned about the prospect of shorter timelines. The thought here is that AI will advance rapidly enough that we might be in very serious trouble in just a few years. Now, the politics world moves notoriously slowly and unpredictably. So I want to know how confident you are that this legislative route that Encode is taking can actually be successful if we do get to AGI in just a few years, or if in fact your efforts are mostly targeting worlds where we have more time to act. Sneha Revanur: The way that I think about this is there’s “politics” in the narrow sense of the word — the slow grind of bills and hearings, and sort of strictly thinking about things as like passing legislation — and there’s “politics” in the broader and maybe more ambitious sense of the word, where it’s about building power. And I think we really strive for the latter. We want to find creative and strategic and oftentimes totally experimental ways to build power for the cause of AI safety. This is obviously hard, as you pointed out, because we’re working against the clock. And oftentimes we sink a lot of resources into things that we don’t know aren’t going to work until after the fact. But I think there is one philosophy that tends to work for profoundly uncertain situations like this, which is: if you do everything, you will win. I think Encode takes a lot of swings. Some of them were successful. Others we, like I mentioned, didn’t know would fail until we tried. For example, I was really proud to see the California and New York laws passed with the leadership of our determined sponsors — Scott Wiener, Alex Bores, and Andrew Gounardes. These laws are obviously first steps, and I can see how a sceptic could dismiss their impact on the margin. But the real win here is not the literal first-order effect of the fact we are now requiring companies to submit safety plans — because obviously many of the companies were already voluntarily doing so — it’s that we are building regulatory capacity in nimble and well-resourced states that are able to potentially activate on short timelines if the federal government doesn’t act. Zershaaneh Qureshi: It occurs to me that in trying to tackle the AI issues that you care about, you could have taken a more kind of conventional policy route for doing this. So you could have worked in a government agency or become a congressional staffer or something like that. And it does seem like there are some downsides to trying to change things from the outside through civil society. You don’t get to be literally in the room when the details of a bill are being decided, for example. And we’ve talked about these state laws that you’ve helped to pass in California and New York as two examples — SB-53 and the RAISE Act. I’m wondering whether you think you’d have been able to shape these bills more effectively if you’d been, say, working as a staffer for the legislators behind them? Sneha Revanur: There are two things that I would say off the bat. The first is that in California, there’s actually a pretty interesting feature of the way that state bills work, which is that organisations actually can be in the room when the details of a bill are being decided, in the form of acting as cosponsors. So when we cosponsored SB-1047 and cosponsored SB-53, we were in the room for a lot of those negotiations. Obviously, so many of those decisions were downstream of our bill sponsor, Senator Scott Wiener. And he did an excellent job steering the ship there and sort of exercising very good judgement on how to think about all of these various tradeoffs, and negotiating with all of these stakeholders, and standing strong in the face of a lot of industry lobbying — and, especially in the case of SB-1047, this absolute firestorm on Twitter. So definitely hats off to Scott Wiener for his work on that. But I think that we were fortunate, given the nature of how California politics work, to actually get to play quite an inside-facing role that I think organisations usually don’t get to play in other states, or more broadly, when working on legislative campaigns. And I think one more thing that also really works in our favour is that we specifically take an approach that involves overseeing the entire full stack: owning the entire process from AI policy development to implementation. And this is what we’re trying to do a little bit of in California and New York, where now that these laws have passed, the critical question is, “Are the offices that are enforcing them going to be staffed with experts? Does the state government actually have enough resources to execute on this effectively?” One inherent limitation of state laws is that they’re oftentimes a lot more expertise-constrained or don’t have the information security practices that the federal government does have, when it comes to handling some of this very sensitive information from the labs. So I think it’s really exciting to also get to be in the room and shape some of the implementation and enforcement things towards the very end of the process. Zershaaneh Qureshi: What do you think the downsides are of the route that you’ve taken versus a more conventional inside-track route? Sneha Revanur: Yeah, I think one thing that is really frustrating is that oftentimes you can get derailed by forces that are just entirely outside of your control. There have been so many cases where we thought that we’ve made a very compelling case: we have mapped out all the people, we have built up relationships, we have tried really hard to craft a pitch that was extremely sensible, and navigate all of those tradeoffs — and you’re still going to get clobbered by industry at the end of the day. And there is still going to be some army of lobbyists that descend upon Sacramento or Washington or whatever and really get in your way. And it sometimes feels like you just can’t really do anything about that, and you can feel kind of powerless in the face of just the enormous amount of money and political capital that is on the other end. And the other thing is that obviously we have just gotten in the game in the past couple of years. Whereas a lot of these Big Tech lobbyists, these are familiar faces who have been building up relationships over a bunch of different fights — even predating a bunch of AI stuff — over the past five years, 10 years, 15 years. So sometimes it can feel like the odds are stacked against you, and things can go sideways, and not even as a result of anything that you’ve done wrong. So I think that there is that element to it as well, and I don’t want to downplay how frustrating that can be. And of course, right now I’m riding the wave of some of our recent political success, but this has definitely not always been the case, and we have had plenty of things that have failed. And of course, for us to pass a bill in California, it took getting the first one vetoed. So I think it’s important to recognise what some of these limitations are. But I think that, for someone who is tolerant of these setbacks, for someone who recognises that failure is oftentimes the precursor to success, advocacy can still be a great way to make an impact. Zershaaneh Qureshi: Yeah, it’s interesting to hear you talk about this, because it feels like quite a contrast from the Encode that was set up in 2020. Back then you were this extremely, genuinely, very grassroots, outside organisation. And over time you’ve kind of been growing into this more professionalised, more insider-facing organisation. I think something that I’m curious about here is: in the current world that we find ourselves in, where the public is just incredibly angry about AI, there’s this narrative that Big Tech is just totally out of control. I would have guessed that the original approach, the more grassrootsy kind of approach — the outside game — might be actually better placed to capitalise on that kind of sentiment. So I’m kind of wondering why you made this decision and how you’ve been thinking about the tradeoffs here? Sneha Revanur: Yeah, I think one thing that I would say off the bat is: building up those networks is just really, really hard. Especially what we were doing before, trying to motivate all of these uncompensated high school and college-age volunteers. I think it was really hard to translate that into genuine political outcomes. And even though it was super exciting to be able to summon people to do a lot of things in a specific state, for example, or get a lot of people to call up their member of Congress, or activate in very specific ways, I think that overall the cost-benefit analysis just wasn’t really making sense for us in terms of actually translating into our desired outcomes. I think also one point that I would make is that the way that people think about AI is not monolithic whatsoever. I think it’s extremely hard and almost impossible — and this also just wasn’t true of the way that we had things set up before — that I can say, “I am the voice of the people and I represent all people and how they feel about AI” — just because people’s views on the issue are so diverse and oftentimes ill-formed. And sometimes they have a vague sense of how they feel about something, or they care a little bit about this and not about that. And it’s really important to me that what I’m saying when I speak on behalf of a group of people is credible and authentic and sincere, and not that I’m trying to play up support that might not actually be there. And so I think that the strategy that we’ve taken now of just bringing together groups that already have formed perspectives on individual issues and letting them speak — for example, going to kid safety groups that obviously already are very deeply passionate about the issue, and talking to mums who have literally lost their children to addictive AI companions or whatever, and letting them say their word and speak their stories and not putting words in their mouth or claiming to be an ambassador on behalf of those people — I think is not just the right way of doing politics, but also just significantly more effective, I think, in actually getting things done. Zershaaneh Qureshi: So when you look at Encode’s track record, I’m wondering if there’s any one example that you’re especially proud of, where you just feel like advocacy has really moved the needle? Sneha Revanur: I think one thing that has been pretty surreal to see over the last year especially is just the sheer number of strange bedfellows that have banded together for the cause of AI safety — like extremely strange bedfellows. You have these forceful coalitions of family-first conservatives who think that we are gambling away the human spirit, and moms who have lost their children to addictive AI companions, and singers in Nashville and actors in Hollywood who want to protect their likeness from voice cloning. It is exactly this team effort across people with very different terminal goals, very different political backgrounds, that has helped us stave off, so far, federal preemption of state AI laws — this attempt in Congress to block states from regulating AI, and also override existing laws like the ones that we helped pass in California and New York, whether or not the federal framework that’s replacing it is better than what we’ve already gotten. And it seems to me that without some degree of active coordination, without a lot of mutual trust building, not all of these different preemption stakeholders would have found their way to each other. So I’m really proud that we’ve been able to contribute to this alongside allied orgs like Americans for Responsible Innovation. My colleague Adam, for example, leads these super cool monthly coordination calls to engage hundreds of allies. And if you want to know how hard coalition building is, you can take a look at the fact that the side in favour of preemption has clearly failed at aligning any coalition of their own. There are these massive disagreements among industry groups and the White House and all these trade associations, all this infighting about exactly how much to yield and what affirmative position to actually take besides “preempt everything we don’t like” — which is why they haven’t been able to come forward with any sort of framework that would satisfy any key parts of our coalition or get anyone to peel. And so I think that coalition building is one thing that I’ve been really excited about as a force for getting things to happen in politics. And we think about this all the time at Encode. We are always trying to make new friends and keep them. And it is one of our top axes for measuring our counterfactual impact. Zershaaneh Qureshi: So you’ve described a really wide range of people banding together here, and I think that’s a worry that some people have with this coalition approach, right? Which is that everyone here has pretty different interests, broadly speaking, and that might end up potentially diluting the thing that you’re trying to actually achieve. So because you kind of have to target the thing that everyone can agree on, that might push you to be focusing on the lowest common denominator. How do you navigate that? Sneha Revanur: That’s a really good question. I think that first of all, in the case of preemption specifically the great thing about this is that obviously it’s a lot easier to rally people who have very different terminal goals against a common enemy than it is to get them to support the same thing. And so I think that there is that calculus that works in our favour. But I think more generally, when it comes to coalition building, this is a fair concern. And there are two ways that we try to mitigate some of those costs. The first thing is just keeping very clear internal bright lines around what we will and won’t support, and having a very clear sense of when something has gone too far, and how to communicate with stakeholders when we’re not willing to get on board with something. So, for example, my colleague Nathan, who’s a lawyer — who’s our general counsel and also is a person who brings this fine-toothed comb to all the bills that we’re looking at — he’s someone who, as we all do, cares very deeply about the First Amendment and thinks really hard about free speech. And so every single time we’re taking a look at a kid-safety bill that one of our partners wants us to support, Nathan will be there making sure that we are thinking very hard about how this trades off against free speech, and how we think about these goals that we obviously value equally. And so I think that having those limiting principles in place really, really helps when you’re weighing in on complex legislation. I think also one more thing that people really underappreciate from the outside looking in is just how much relationships matter. Like we have been in the trenches with these people. This is our third preemption rodeo, and that means we have these built up relationships and a lot of mutual trust that we’ve accrued over the span of a year. And this is again really thanks to the efforts of my colleague Adam, who has been doing a lot of this coalition coordination. What this means is that when someone gets to know you, when someone trusts you, when someone generally believes that you have good takes and analysis and have your heart in the right place, and you aren’t a sellout or whatever, they will tend to give you a lot of benefit of the doubt. And if they have an objection to a specific line of bill text, you can just get on a call with them and they will just ask you questions and you can explain why you are doing things a certain way, and they won’t nitpick as much. And they’ll just be willing to have that conversation and oftentimes come around to your position because they ultimately trust where you’re coming from as a result of having that relationship. And so I think that oftentimes for a lot of these coalitional dynamics that seem very insurmountable from the outside, trust and relationships are king and can really help you mitigate a lot of those downsides. Zershaaneh Qureshi: Pushing on here: probably Encode’s most-cited win is SB-53, which is California’s landmark AI safety law. It requires frontier AI companies to publish safety protocols, protect whistleblowers, and so on. But SB-53 did only pass after its more ambitious predecessor, SB-1047, was actually vetoed in the previous year. And I think at this point, a sceptic of Encode’s work would say the bill that passed was actually just much weaker than the original. And it also passed in an environment where attitudes had kind of already shifted. I mean, people were more alarmed about AI at this point in time after the release of reasoning models like o3. So yeah, somebody might say this was just a much easier win, and Encode’s advocacy efforts just weren’t the thing that moved the needle here. What do you make of that argument? Sneha Revanur: I think you make a really good point. I think that those are all fair critiques. A lot of the climate shifted. Sacramento was already feeling some ambient anxiety about the release of reasoning models. After SB-1047 was vetoed, Governor Newsom convened a high-level commission of experts like Fei-Fei Li and Tino Cuéllar that did great work and eventually put forward a report recommending transparency requirements and whistleblower protections, and those ended up being the core bones of SB-53. This wasn’t an against-all-odds victory. And of course there were aspects of the situation that ended up not working out in our favour. We had to negotiate away, for example, the third-party auditing requirement that I think was very central to the bill originally. But I also would make a few points to push back here. First of all, we were also cosponsors of SB-1047 and that bill passed the legislature. We did a lot of work wrangling labour unions and Hollywood actors and coordinating these diverse letters of support. And Elon Musk even tweeted about the bill. Having that coalition muscle, priming the legislature, building up our working relationship with Senator Wiener’s office, socialising the idea of frontier AI regulation in Sacramento, getting on Governor Newsom’s radar to the extent that he said the bill created its own weather system — and he ended up convening that commission that really paved the path for SB-53 — these were all residual impacts from SB-1047 that made it much easier the second time around, such that a lot of things that we invested in early on just ended up paying off at a later date. And so I would not really disentangle these two things as being entirely disconnected, because I think that in a lot of ways, things that we did during SB-1047 just had a delayed payoff. And you could even make the argument that we actually needed SB-1047 to get SB-53 to pass in the way that it passed. So that’s one thing that I would say. The second thing that I would say is that it’s really important to have advocates in the room who have a lot of context on safety-specific things. For example, one win that we were super excited about when it came to the final version of SB-53 was that we were able to actually get the bill to cover internal deployments of AI models. And I think that especially on the heels of Mythos, a lot of people are especially concerned about the risks that might emerge from internal deployments and the lack of transparency around that by default. Especially because at this point, the most capable models that exist aren’t available to the public and aren’t externally deployed. Because the law actually creates this secure reporting channel to the California government where lab employees can raise the alarm about risks from internal deployments — that is not a thing that existed before. And that is primarily the result of us really negotiating really hard, being in the room during those conversations, pushing back against industry that wanted to remove that aspect of the bill. So I think that having people who are high-context advocates really matters, and that, to me, seems to move the needle. So yeah, maybe you would have gotten some kind of bill passed, but it would have maybe been different in a lot of very fundamental ways from what we eventually got. And I’m very proud of — through this entire process — the outcome that we accomplished with the final version of SB-53. Zershaaneh Qureshi: But of course, with SB-53, you will have had to have made some compromises to get this bill to actually pass, right? So one thing that happened was that third-party audits were taken out of the bill, and then also between SB-1047 and SB-53, I think that the idea of kill switches got taken out and some deployment restrictions were dropped. This seems, I think, pretty normal in the advocacy world; you do just have to pick your battles. But I’m wondering if there’s any sense in which you feel like the weakened version of SB-53 actually ended up being harmful overall? My reasoning here is that the law that gets passed in California here is something that in theory will eventually kind of translate into other state laws and into federal frameworks. So if SB-53 isn’t strong, then it maybe sets a lower bar for what legislation is sufficient. Does that thought concern you? Sneha Revanur: Yeah, I mean, the fact that California didn’t pass third party audits clearly didn’t stop Illinois from just passing third-party audits yesterday, right? So I think that the standard can totally keep rising. And in fact, actually what happens in practice is that once one state does something, there is some second-mover advantage, where other states feel a lot more comfortable following suit. And they’ve seen how this other governor, for example, has got a bunch of credit for signing a really important AI law, or this other legislature has gone through this entire process and weathered the storm of all of the AI lobbying and come out with this very strong piece of legislation. And I think that there obviously is this dynamic at play where states feel a lot more comfortable doing this when there are role models to follow. And so I actually think that our job was easier in New York and our job has been easier in Illinois because of the work that happened in California. And now we have a standard that is meaningfully higher than what we had in California, as a result of the Illinois third-party auditing bill being passed by the legislature. I think that also, when it comes to Congress, it’s totally possible that had no bill passed in California — and to be clear, I think that SB-53 probably is the strongest possible thing that could have passed at that particular time, and that would have been signed by the governor at that particular time — had no bill passed out of California, then maybe Congress would be like, “Oh, there is no political momentum behind AI safety. We can come out here with a really bad federal framework.” And maybe that framework would have been even worse than what you have right now in SB-53. Maybe it wouldn’t have covered internal deployments. Maybe it would have been entirely voluntary. There are lots of ways in which this could have been worse. And it’s only because we have the leverage of California giving us this actually pretty decent law, and New York giving us this pretty decent law, and now Illinois moving the needle forward, that Congress now is incentivised to give us something even stronger than t hat. And stakeholders who are against preemption are not going to come to the table unless we’re getting something stronger than what already exists out of Congress. And the only reason why we have that leverage in the negotiation is because we have these state laws that obviously set a really strong baseline. Zershaaneh Qureshi: But you do agree, right, that SB-53 in its current form is not sufficient for what you would like to be seeing happening at a federal level, right? Sneha Revanur: Oh, totally, yeah. This is the reason why we’re fighting so hard on preemption, because we don’t want the easy way out of just codifying what we already have in California. We think there is a lot of room for improvement. We think that even from the Illinois bill that just passed, there’s a lot of room for improvement. We want a federal framework that builds upon what we’ve seen in the states and makes sure that The Center for AI Standards and Innovation (CAISI), for example, is well resourced. There are several asks that we would have that go above and beyond what we’ve already seen in California and Illinois. Zershaaneh Qureshi: Moving on, I do want to talk about a clash that Encode had with OpenAI last year. During your efforts to pass SB-53 in California, your colleague Nathan Calvin received a subpoena from OpenAI. Now, OpenAI did describe this as “routine litigation,” but I’d be interested if you could just sort of take us back to that time. What happened? How did that week play out for you? Sneha Revanur: Yeah. I mean, that was a pretty crazy time in our lives, I can definitely say. The context on this is: last year, early 2025, Encode filed an amicus brief in the Musk vs Altman lawsuit surrounding the restructuring of OpenAI into a for-profit. We were among a chorus of voices that had a bunch of concerns about this, and weren’t certain that this would lead to the optimal outcome in terms of board governance or allocation to the OpenAI Foundation, and kind of just felt very contrary to the founding charter of OpenAI. We had been making some noise about this for a couple of months, and also, of course, we were in tandem advocating for this bill in California, Senate Bill 53 — which OpenAI [had ceased to be] formally opposed to, but that we knew was actively lobbying against, based on our knowledge of some of the conversations they were having in Sacramento. So I think that there was that very interesting confluence of factors, where the lawsuit was really coming to a head, and it was this decisive stretch over late last summer; and the bill was also undergoing some pretty intense negotiations, and it seemed as though it was all but poised to pass at this point and would soon be heading to the governor’s desk, and things were in a very critical moment for the bill. And it was right then that my colleague Nathan Calvin, as you said, pretty much had a sheriff’s deputy show up to his house while he was having dinner with his wife and serve him the subpoena in real time. And we were subpoenaed both via Nathan individually and also at our corporate address — which is nonstandard: it is very much nonstandard to serve the general counsel individually; usually these notices are served only to the corporate address. I mean, overall, the biggest takeaway from the entire experience was like, damn, I guess we’re kind of playing in the big leagues. You know, for the first time we had all this attention on us: Nathan Calvin’s thread went viral on Twitter, got 6 million views, and prompted Jason Kwon — who is a chief strategy officer and a very senior executive at OpenAI — to publish a thread detailing OpenAI’s response to this. And it really felt like we were receiving all this support from different people, but also kind of all this attention and visibility that we had never gotten. And I’m sure there are a lot of people in the broader AI world who didn’t know what Encode was and kind of had their prior formed on us via this experience. So in that sense, it was definitely stressful, it was definitely anxiety inducing. Not just the experience of going forward about this publicly, but also the actual experience of getting subpoenaed. This was taxing in a lot of ways: we had to worry about our texts being accessed, we had to worry about conversations that we were having with legislators who were on the bill being leaked or things of that nature. So I think it was very concerning for us from an infosecurity perspective, what this might mean for the bill, and we had to take all of these measures that I think were totally unpredictable to us in advance of that. So I think in that sense it was very stressful, but in another sense, it was also a really exciting validation of just how far we’d come as an organisation: that were kind of playing in the big leagues, had visibility on our work for the first time, and clearly our work was being taken seriously by people in high places. And going forward, in terms of how this impacts how we’ll operate and who we’ll work with, I think that actually our relationship with OpenAI has improved quite a bit. We remain super optimistic about new future avenues for cooperation. I think the biggest thing that I remember from that was just the outpouring of support from OpenAI employees who very clearly were sympathetic with Encode and wanted to support us. And I think that consistently those relationships have been very positive for us. Zershaaneh Qureshi: I mean, in the end, it did seem to me that you’d been pretty intimidated and shaken by this. You put out this statement on Twitter, where it kind of looked like you were letting them off the hook a little bit. You said things like, “it was reasonable for OpenAI to have concerns about collusion.” You said, “OpenAI wants to be a force for good,” that its success “is critical to American dominance,” that it’s “criticized disproportionately.” Looking at all of these things, I mean, it seems like you were pretty intimidated in this moment. It kind of reads as though you were sort of trying to appease them. I’m curious why you didn’t put more pressure on them in this moment? Sneha Revanur: I think that honestly, what I really wanted in that moment was to make the public aware of this very specific incident that we were extremely concerned about — obviously the potential use of intimidation tactics against advocates who were raising concerns about the restructuring. Obviously well below the standards that OpenAI should be holding itself to, which I also said in the statement. But I think that what I also wanted long term was to make sure that we could still work together down the line. And you know, this is a company that we are going to be running up against every single time we’re advocating for a state bill. This is a company where we’re going to be encountering their lobbyists in Sacramento and Springfield and Albany and Washington, and it’s important for us to have some inroads there. I think that as an organisation that was doing this specific instance of public-facing advocacy and public pressure-building, but is also doing a lot of the behind-the-scenes coalitional work and doing a lot of the behind-the-scenes advocacy, it was important for us to kind of balance those interests and make sure that we weren’t closing the situation in a way that would close off future possibilities for us to work together. And I think that if you look at what happened with Illinois, if you look at the fact that they’ve obviously been making some positive updates, it hasn’t been perfect and there are still lots of things that we have complaints about. But I think that sentiment has been somewhat vindicated, of like, this company actually is willing to come to the table on certain things, and more importantly, they are responsive to pressure and they are responsive to changing incentives. And now that these laws are passing and like there’s clearly momentum on the side of AI safety, they kind of are realising that they have to make some concessions and that’s actually happening in real time. So I feel very grateful in hindsight, and I was very confident back then that that was the right thing to do, and I think that some of the recent momentum has proven that too. Zershaaneh Qureshi: Yeah. It’s interesting, actually: since all of this went down, in fact, there are some signs, at least to an outside observer, that OpenAI’s attitudes towards regulation is maybe shifting. They’ve backed this safety bill in Illinois which includes third-party audits; that’s something that they opposed at the time of SB-53. They also recently called for an international AI governance body. I’m really curious what you make of this, and how your relationship with OpenAI is now shifting. Sneha Revanur: Yeah. Of course, I wouldn’t overclaim and say that all this is downstream of our individual relationship with OpenAI. I think that there are shifting winds here, and I think that very broadly it’s just a sign that companies are responsive to pressure, so we should keep putting pressure. For example, when it comes to the Illinois bill endorsement or some of the other things, I expect that some of that has to do with researchers internally making noise and putting pressure on the Global Affairs team. It’s been pretty interesting for me to see over the past couple of months that it seems like a lot of employees at OpenAI just aren’t really aware of the company’s pretty shady political activities and have only woken up to that over the past couple of months. So as more OpenAI employees who sit outside the Global Affairs arm have realised some of the stuff that the company is doing — and that Chris Lehane, who runs the Global Affairs team, is doing — they have made some more noise about this, which has obviously led to the company probably shifting its position. From what I’ve seen, it seems like the company actually has a pretty good culture of internal dissent, and you see employees speaking up and saying pretty critical things about their employer all the time, which seems like a very positive sign. So I would say first thing is it’s probably just a product of employees paying attention to the politics side for the first time, and making some noise and putting pressure on the relevant people internally. The second thing is, as I mentioned, civil society pressure from the outside maybe is actually beginning to make a dent. I think that because Encode and a couple of other organisations have tried to position ourselves as good faith critics, it maybe makes it more likely that these companies are actually willing to hear us out and have a conversation and in some cases come to the table on things that they may not have come to the table in the past. I think that’s the second thing. And the third thing is honestly just that they’re realising that their strategy from earlier is really not paying off. It’s very obvious that this short-sighted approach of “let’s bludgeon all AI regulation” and “let’s give [New York state representative] Alex Bores a really hard time via this super PAC [political action committee] Leading the Future that we are kind of associated with but also kind of not associated with” — they’re doing this funny act of trying to have it both ways. I think that people are realising internally and in this broader orbit that that is not a good strategy, and the public is getting really upset about AI, and there’s a lot worse out there on the scales of Ludditeism than Alex Bores — so it’s probably not a good idea to come on too strong in the early innings, and it’s probably better to cooperate. I think that there’s probably some people in OpenAI who are having that realisation right now, and I think that’s also one feature of the situation. Zershaaneh Qureshi: Now, I believe that AI regulation is now one of the most heavily funded single issues in American politics. You’ve got the super PAC Leading the Future, which is pretty anti-regulation and it’s raised $125 million for the midterms this year. It’s the biggest spender in this year’s midterms, I believe. You’ve also got the emergence of a counter PAC that’s backed by Anthropic. I’m interested in how this influx in cash and attention actually shapes your attention agenda? Sneha Revanur: I think when I take a step back and analyse this, there is this interesting misapprehension in Silicon Valley that capital eats the world and is fully fungible into political outcomes. Like there’s some market where you can just go buy your preferred reality, right? Like when all you have is a hammer, everything looks like a nail. It is definitely true that money can influence elections, but you either want to have the tailwinds of public support for you to amplify, or be spending on an issue where no one is paying attention. This is part of why the pro-crypto PAC Fairshake has been so successful. I think they had like a 90% candidate success rate in 2024, which is absurd and honestly very impressive. But crypto is a low-salience issue that is not driving people to the polls anytime soon. So in that case, the spending actually does make a huge difference on the margin and can reliably spook candidates into submission. And a16z is one of the top funders of Fairshake. OpenAI’s Chris Lehane, who runs the Global Affairs team, literally helped set up Fairshake after his time at Airbnb. Josh Vlasto, who runs Leading the Future, was an advisor to Fairshake. So it’s the exact same architects who seem to think that they can just get the gang back together and run the same playbook and get the same results. This is fundamentally not an AGI-pilled strategy. Honestly, it’s funny how often you can completely demystify why some person is acting a certain way if you just model them as not being sufficiently AGI-pilled. Because the stuff that LTF is doing is not what you would do if you saw the writing on the wall that AI is going to be a big deal politically; if you saw the writing on the wall, from the fact that the public is souring on AI already and wants more regulation. And there are very obviously going to be politicians coming to ride that wave who are way more adversarial to the industry than the extremely sensible and high-context Alex Bores. As you would expect, it seems like there are already some cases of candidates who have actually seen a boost in name recognition and fundraising from all of the LTF spending against them. So I would say that it actually seems like some of these super PACs that are spending big against regulation look more and more like a paper tiger day by day, and we’re not too threatened by their involvement. Zershaaneh Qureshi: Yeah, I’m also interested in how Encode is thinking about the midterms and the 2028 elections. With AI becoming this really big electoral issue, do you have a strategy around that? Sneha Revanur: It is the fastest-growing issue right now in terms of salience, but there still is that delta there and I think it will take a while before voters are super motivated to go vote for one candidate or the other based on differences in their positions on AI. I do think we’ll get there at some point, but we’re definitely not there yet. And I don’t think that we’ll be there this year for the midterms. I do think though, what’s been really interesting is that candidates are feeling all of this pressure to stand out and to take positions on the issue and to differentiate themselves. AI is becoming a really interesting political opportunity for candidates who are competing in crowded primaries, for example, to differentiate themselves. It’s becoming a really interesting opportunity for the Democrats and the Republicans to differentiate themselves. I think the jury is still out on which party will do a better job of seizing that opportunity. But I think it’s just been so surreal to see viral campaign ads that are centring AI data centres as the enemy, and to see candidates really taking seriously the possibility of AI-driven job displacement and proposing things like a token tax. It definitely feels like it’s the kitchen table set of concerns that has taken off so far in public messaging, but overall it’s super interesting to see a lot of people talking about AI who obviously just weren’t talking about AI before. As for how Encode is thinking about this, I think that we are obviously keeping an eye out for candidates who both are going to be champions on addressing catastrophic risk, and also have really ambitious plans for more near-term issues like job loss and kid safety. This is a combination that’s been hard to find so far, but I think there are some people who are talking about this. Already it’s been really exciting to see candidates explicitly talk about things like loss-of-control risk in their policy platforms, and things like this actually come up in the political conversation for the first time. Looking ahead to 2028, it’s no doubt that this very well could be the most important presidential election year in history, right? There are a lot of reasons for this. Whether or not AI timelines are shorter or longer, there’s a whole range of outcomes where this could either be the person who will be in charge of the intelligence explosion, or, in any case, this will be a person who is making some very major world-historic decisions about AI, and we want this to be a person that we trust to have a lot of context and to be a person we would really trust with the keys. One thing I’m also thinking quite a bit about is power-concentration risk: what will it mean to empower this person with superintelligence, potentially? And who would we trust to hold the keys in that moment? One thing I’m really concerned about is it’s very likely, based on what we’re seeing so far, that the executive branch is going to be the first to wake up to just how powerful AI is, and is really going to leave the judicial branch and the legislative branch in the dark, which will just really mess up the balance of power. I think there’s already been some concern about this, and I think this will potentially get substantially worse once we actually have much more powerful models and much more national-security-relevant models. So one intervention that I’m really excited about is speeding up AI adoption among the judicial branch and the legislative branch, to kind of counteract some of that balance-of-power issue and make sure that we don’t have all of this power singularly concentrated in the hands of the executive. Zershaaneh Qureshi: Something Encode’s been working on recently is whistleblower protections — which, as I understand it, are relatively popular from an industry per spective, at least within people who work at AI companies. One example of this is there was this “right to warn” letter that came out a couple of years ago, where a bunch of current and previous employees of AI companies wrote this public statement calling for more whistleblower protections. I do definitely see the value in targeting some of these easier wins, which do have real genuine value. But I’m interested in how you think about targeting these sorts of easier wins versus the strategy of targeting interventions that are more controversial but potentially even more important. So things that spring to mind potentially here are things to make companies financially liable for the harms that their systems cause, or tougher restrictions on deployment — both of which I think are especially controversial, but also really important. How do you think about the strategy of balancing these different focuses? Sneha Revanur: Yeah, that’s a great question. I mean, obviously there’s a context of SB-1047 where it wasn’t even really proposing strict liability. There are so many ways in which that bill could have been substantially stronger. There are so many ways in which that bill could have just been way more aggressive. But even the word “liability” being in the room I think seems to be politically toxic and obviously incited a bunch of opposition. We are trying to think about how to tactically sequence the things we’re working on to build support over time. One pipeline that has been incredibly effective, from our perspective, is: you take something that a couple of labs are already voluntarily doing and you figure out, “How can we codify this into law, and how can we use this to realign incentive structures across the entire industry?” And obviously that works in cases where it’s about submitting a safety and security plan. But of course, it’s entirely possible that by next year, by the year after that, we’re going to have such high levels of risk that you need to do more aggressive things, and we’re going to be ready for that. We’re going to see what needs to happen in that moment. But I think that politics is really all about sequencing and coming out with the right things at the right time and knowing when you’re actually making things worse by being too aggressive too soon. And I think there’s some story that SB-1047 might have had some of that effect, so we’re being really cautious about that going forward. Zershaaneh Qureshi: Yeah. If you’re willing to speak about this, what do you think actually needs to happen for the moment to be right for some of these more aggressive interventions, say, on the side of liability? Sneha Revanur: That’s a good question. I mean, obviously I don’t want this to happen — I would love for our politics to operate in a way where it’s not always responding to crisis — but I think that a lot of people in the advocacy world seem to think that there could be some sort of warning shot that catalyses a lot of political will. Maybe this is a really disruptive cyberattack or something else. Hopefully it’s not a really bad bioweapon, because that seems potentially a lot less recoverable, versus cyberattacks can maybe cause just enough damage to wake people up but not be entirely catastrophic. I think that there’s a sense that maybe there will be this moment where the Overton window totally just bursts open. Because the way that government functions, especially the federal government, is that it really responds in times of crisis, and oftentimes it over-anchors to whatever that specific kind of crisis is. So right now what I feel really excited about is that post the announcement of Mythos, there is some broad sense that AI is actually a big deal. You know, we should think a lot about cyber, but it’s very focused on cyber. And there’s potentially a bunch of low-hanging fruit for bio preparedness, which is also a threat vector that I’m extremely concerned about that people are not attempting, just because it’s not the thing that feels most salient to them right now. So I think that it could be a warning shot. Hopefully it’s not a super bad warning shot. Hopefully there’s no warning shot at all, and we’re just able to organically build this political will over time. But I think that people tend to underestimate how much politics is responsive to specific things that are happening in the world, and responsive to specific galvanising events that make politicians feel a certain way, versus just being about having really good arguments and being able to reason with people. That’s kind of not how it works, even though I wish it was. Zershaaneh Qureshi: OK, moving on: if somebody listening wanted to do sort of what you do — like maybe they want to go into advocacy or lobbying, or maybe they even want to start their own organisation like you did — what advice would you give to them? And specifically, is there anything that you think has really driven Encode’s successes that you’d recommend they try to replicate? Sneha Revanur: Oh my god, yeah. Come join us! I’m super excited about more people working on AI advocacy. I think it’s one of the most underresourced and talent-constrained ways to make AI go well. And I’m really excited that I think post the successes of the last couple of months, people have been taking AI advocacy more seriously and giving it its due. I think there were a lot of people who potentially wrote this off after the SB-1047 veto who are now coming to realise that it was only a matter of time. So yeah, super excited about the space growing and having more awesome people to work with. For the right person, it’s literally the coolest job in the world. But I think I have a couple of pieces of advice that are also drawing on some of my own reflections from working on this. I think the first thing is a very specific challenge that comes with AI advocacy: you are signing up to practice two opposite habits of mind at essentially the same time, and getting good at one really tends to erode the other. On the one hand, politics rewards fluency in persuasion: you’re building relationships and managing coalitions, you are learning how to distil messages and make different audiences tick. And there is one overdeveloped version of this skill where you can be the master of some message that you never actually stopped to ask yourself if you genuinely believe. On the other hand, what’s normally rewarded in AI safety circles is very different: it’s sharp, unsparing analytical thinking; very careful, sometimes even fastidious epistemics. You’ll sit down and you’ll make an offhand comment, and someone will be like, “I don’t see why that would be true,” and proceed to really carefully interrogate what you said as if it were some very important substantive claim. And this is good, because it’s why the community has been so right about so many things when everyone else was wrong. But it doesn’t automatically make things happen in the world. So there are failure modes to both of these approaches, and the challenge that I have navigated, and that I have been thinking a lot about recently over the past couple of months as well, is knowing when you’re a “soldier” — because as you’re an advocate, you’re kind of playing the role of a soldier — how do you also know when to be the scout, and how do you know when to toggle between these two modes? And as someone who went into that soldier mode from a very young age and was kind of jumping into advocacy from a very young age, I think it’s been really helpful for me to sometimes take moments to step back and just take stock of, “Am I getting anything wrong here?” Because keep in mind, it took me a while to even realise that catastrophic risk was a big deal and I should be working on this. And it only came from me taking that moment to step back and toggle into scout mode for a second and just ask myself if I was fundamentally wrong about things in the world. And that was game changing, and that was the unlock of everything that I’ve accomplished since then. So I think knowing how to toggle between those two main modes, knowing when you should step back. Also kind of just actively in the background of the work that you’re doing, working to carefully build your knowledge base and surround yourself with smart people who value unmotivated disagreement. I think that these are things that might not be instantly rewarded when you’re doing politics, but are really important and really crucial for making sure that your actions are actually aligned with making things go well. That’s one thing that I would say. I think what also really helps with this is that one thing I’ve noticed about people in AI safety is that they’re just incredibly approachable. It doesn’t really feel like there is some sense of hierarchy. You can kind of just walk around Constellation and find some extremely smart person and just pick their brain over lunch, and they will just be very charitable with their insights and be willing to talk to you. There’s not really a sense of, “You’re junior, so you’re not worth my while” or whatever, which is very much the case in other industries. So I think that’s a really, really important thing to take advantage of. Don’t be afraid to reach out to people. Don’t be afraid to go meet people that you admire and pick their brains, ask questions. People are so willing to help each other and really want to share what they know. Zershaaneh Qureshi: I think that’s all we’ve got time for. But Sneha, you’ve been wonderful. Thank you so much for coming on the show. Sneha Revanur: Thank you so much. Thank you for having me.