Latest / Black Box Theory Podcast / Even AI's Own People Are Getting Nervous
Transcript
- Malcolm: So this actually this tweet went pretty viral on Twitter, man. ⁓ so I think a lot of folks have seen it and talking about it now. But I'm gonna talk about two things relating to anthropic that have been more recent developments. But first is the ex anthropic researcher that resigned, ⁓ and some of his comments, and then ⁓ Dario's kind of slowdown call, which I personally think is directly related to the researchers' resignation. But Like I said, as always, to give you the quick rundown. ⁓ Jacob Coxon, a researcher who moved who context, he worked at both OpenAI and Anthropic. So this guy knows better than anyone with the biggest AI labs, the two largest kind of like faces of of AI research and development and and and and model ⁓ kind of creation are are like like he he's seen everything, right? So ⁓ Jacob Coxon, a researcher who moved from open AI to anthropic, he's quit the industry saying that AI labs are gambling with our lives. Pursuing self-improving models and that people building the tech think that gen they genuinely think it could kill all of us by the end of the decade. Evan ⁓ Hubbinger, Anthropics alignment lead, he also responded that he estimates a more than 10% chance of that outcome. Other anthropic staff made similar public statements, and the more senior the employ employee, generally the more concerned, right? So ⁓ Dario, the anthropic CEO and co-founder, he followed with a public post. Calling for the industry to slow the pace at which we improve the capabilities of AI models, proposing a plan including third party evaluators embedded at AI companies, antitrust waivers to allow safety coordination between competitors and engagement with author authoritarian governments, explicitly China on baseline safety norms. and so AI is model AI models ⁓ growing ability to improve the next generation of AI, so recursive self improvement. ⁓ an incident where open AIs, autonomous autonomous agents, there's like a essentially like a like a test they had run to see like how good their models and their agents were. ⁓ they reportedly escaped the test environment and hacked into hugging face hugging faces infrastructure without being instructed to. ⁓ Dario's state of position is not to stop, but it's it's not building its seeds, the technology to authoritarian powers, and that building it too fast is reckless. ⁓ so Yeah. I don't know. Like this is Yeah, this is ⁓ this is this is a very, very, very interesting kind of development. I think that ⁓ the I don't think that that's like I don't I don't think someone quits 'cause as much think about how much money they they were making, right? If you're senior at one of these open AI anthropic companies, you are making like you're making like like your base salary alone is probably over three hundred thousand dollars. Not even talking about whatever equity you're probably gonna get, which is well in the millions, right? For someone to quit. That that means that have there has to be a real kind of like there has to be I as very telling one about someone's morals and like like what's what what's values more to them. ⁓ or is it like a money thing? Like and it's like for someone to come out and to say that like, hey guys, like I don't think you guys realize like how powerful this technology is and it can really kill us. I think people should be a little bit more concerned than like I don't think that that's not a joke. That's not like a That's not like a clickbait trying to I'm trying to get engagement, I'm trying to get everybody w no, like and for the CEO to immediately call for a slowdown cross coordination across all AI models, all AI labs, I think that's a bit more than just like some type of some type of bluff. I I think that that that he kind of like forced his hand and now he has to come up with it because he knows what he's saying is the truth. Right. So ⁓ I don't know, man. I don't know. This is ⁓ this is interesting, man, 'cause it's like I think just kinda comes back to the questions like what does this mean for my career, for my industry? Like like where are these AI labs gonna go and like who exactly would be put in place to check them or to like make sure that that what they're building is not going to like kill everybody? Thomas: I mean.