Latest / Practical AI / AI in the shadows: From hallucinations to blackmail

Transcript

AI in the shadows: From hallucinations to blackmail · Practical AI

• 0:04
  Jerod: Welcome to the Practical AI podcast, where we break down the real world applications of artificial intelligence and how it's shaping the way we live, work, and create. Our goal is to help make AI technology practical, productive, and accessible to everyone. Whether you're a developer, business leader, or just curious about the tech behind the buzz, you're in the right place. Be sure to connect with us on LinkedIn, X, or Blue Sky to stay up to date with episode drops, behind the scenes content, and AI insights. You can learn more at practicalai.fm.
• 0:36
  Now, onto the show.
• 0:48
  Daniel: Welcome to another fully connected episode of the Practical AI podcast. In these fully connected episodes without a guest, Chris and I just dig into some of the things that are dominating the AI news or or trending to kinda pick apart them and and understand them practically and hopefully give you some tools and learning resources to level up your AI and machine learning game. I'm Daniel Witenack. I am CEO at Prediction Guard, and I'm joined as always by my cohost, Chris Benson, who is a principal AI research engineer at Lockheed Martin. How are doing, Chris?
• 1:28
  Chris: I'm doing well today. How's it going, Daniel?
• 1:31
  Daniel: It's going really well. It's almost the July 4 here in the in The US. Big holiday here. As we're recording that, that's tomorrow. And so, I'm traveling, getting to see my my parents and some family, and so that's always always good for for the holiday.
• 1:50
  And, hopefully, I'm sure we'll hear some fireworks gradually tonight and and tomorrow, as is the tradition. And, I see all the fireworks stands around. I I imagine that some people will there's always, of course, the harmful element of of those fireworks, of course, to my personal sleep and and rest. But but then, yeah, that's got me thinking about some of the interesting, quote, harmful things that we've been seeing in the in the AI news. And you and I have been talking a lot about certain themes that we want to begin to highlight or talk through on the podcast.
• 2:35
  We experimented with one of those themes or formats in our last fully connected episode with the kind of hot takes and debates, the one around autonomy. Another one of those that we've talked about is AI in the shadows. And so I think this would be a good a good chance to maybe maybe just talk through one of those, AI in the shadows topics, which I think I think as we were discussing things earlier in the week, you had some interesting and maybe frustrating experiences that started this conversation. So I'm I'm wondering if you would be willing to to share some of those.
• 3:16
  Chris: Yeah. So I happen to be because of the topic that we're gonna talk today, just to lead in, I happen to be wearing a shirt that our good friend, Demetrius, from the From
• 3:26
  Daniel: the ML Ops community.
• 3:28
  Chris: ML Ops community podcast, and he's been on our show a few times. Good friend of ours had sent, and the T shirt, says, I hallucinate more than chat GBT. And I love wearing the shirt around. I always get comments from people just out and about when from that. And and I but I decided yesterday that I don't I most definitely don't hallucinate more than chat GPT.
• 3:50
  So it was a it was a fun little experiment that I did. I sometimes will play Sudoku, you know, just to pass some time when I'm waiting in line or whatever. And and I play it at a competent level. I usually play it at the top level or whatever game I'm on. And and I know I know the guy can usually I can usually win without guesses or anything like that.
• 4:11
  And so one of the things that that I was curious about, I got into a particular board on just a random game. And on the game, just to speed it up, it gives you like, aside from the numbers you've picked, it'll it'll show you all the possible numbers on the box just so that you don't have to manually go do that for every box, which can take forever. So it speeds up the gameplay without actually giving away anything. And there's always a point on a high level game where you get to or, like, I've I've I've run through every strategy on the Sudoku side that they document out there and, like, you're gonna have to take a guess. And and if you yes.
• 4:48
  You can probably get the whole thing, Rhett, but it's one moment where it's not deterministic. And yet I keep hearing that Sudoku can be solved completely deterministically. I was like, I'm gonna go do this with ChatGPT. So I, took a screenshot of the board as it was, and I submitted it, and it gave me this I told it to give me a deterministic, what's the next move, and it has to be deterministic. And then you have to explain it.
• 5:13
  And it went through this whole long thing. And I looked at it and I double checked the board, which I had right there. And I'm like, it's totally, totally wrong. But it was very confident as as it always is. And so I said, yeah, I I let it know that that was not correct and to redo it.
• 5:29
  And I ended up getting in this cycle where I did this for thirty, forty five minutes, constantly reframing it and trying a whole bunch of the different models available from OpenAI. And they all failed miserably, utterly. And I just it just really it was my I think I've had plenty of of moments of model hallucination, you know, working through things in the past. But the entire thirty to forty five minute episode was one long hallucination across multiple models. And it made me I think the the reason I bring this up, I know when I talked to you yesterday, I had just kinda gone through this and I had this like level of frustration on it.
• 6:07
  And it just made me realize that even though I know that these models are very limited and we're going in eyes open and we're educated about them, I do have certainly a dependency on the reliability of the information even though I'm looking for hallucination. But it made me really understand that the notion of reasoning in these models is still quite immature is a gentle way of putting it. And so I suggested that maybe that could be one of the things we were discussing here today. I know you have some thoughts about it.
• 6:40
  Daniel: Yeah. Yeah. And this actually ties in really nicely because you're there's a couple things that are overlapping here. There's the knowledge sort of embedded in these models and reliance on that knowledge. And then there's what you brought up, which is the reasoning piece, which is a relatively new piece of the puzzle, these reasoning models like o one or deep seq r one, etcetera, and the ways in which it appears that these models are reasoning over data in the input and making decisions based on a goal.
• 7:21
  And that actually overlaps very, very directly with kind of one of the most, I think, interesting studies that have come out in recent times from Anthropic, which is this study around agentic misalignment and how LLMs could be insider threats. And I think later on in this discussion, we'll we'll kind of transition to talking about that because this is extremely fascinating how these models can blackmail or Literally. Maybe or or or or, maybe decide to to engage in corporate espionage. And so that's a little teaser maybe for for later in the in the conversation. But, yeah, I think that your what you're describing here, maybe in a very practical way for our listeners, we can pick apart a couple of these things just to make sure that we have the the right understanding here.
• 8:21
  So when you are putting in this information with a prompt and an image into ChatGPT, This is a language vision model, slightly different than an LLM in the sense that it's processing multimodal data. So it's processing an image and it's processing a text prompt, but you can kind of think in certain ways about that image plus text prompt as as the prompt information into a model. And the the job of that model is actually not to reason at all, and it doesn't reason at all. It just produces it just produces probable token output similar to an LLM. And we've talked about this, of course, many times on the show.
• 9:09
  These models, they are trained in essence not to reason, although it has this sort of coherent coherent reasoning capability that seems like reasoning to us. Right? But, really, the job of the model, what it's trained to do is predict NEXT tokens in the sense that Chris has put in this image and this information or instructions about what he wants as output related to this sudoku game. So what is the most probable next token or word that I can generate, I being the model, that the model can generate that should follow these instructions from Chris to kind of complete what Chris has asked for. And so really what's coming out is the most probable tokens following your instructions, and those are produced one at a time and then most probable token is generated and the next and next until there's an end token that's generated and then you get your full response.
• 10:16
  Now in that case, sort of what is the I guess my question would be, and when I'm teaching this in workshops often, which we do for our customers or at conferences or something, I often get the question, well, how does it ever generate anything factual or knowledgeable? So, yeah, what what is your take on that, Chris?
• 10:43
  Chris: My take on that is that, you know, you're bringing us back to the the core of how it actually works and that as I listened to you explaining, and I've heard you explain this on previous shows as well, it it is so disconnected from the kind of the marketing and expectation that we users have from this, that it's a good reminder. It's a good refresher. I'm I'm I'm kind of having just gone through the experience as I listened to you describing the process again. It reminds me that it's very easy to lose sight of what's really happening under the hood. So keep going.
• 11:20
  Yeah.
• 11:21
  Daniel: Yeah. Well, I mean, I think that it really comes down to if if you were to think about how is knowledge embedded in or facts generated by these models, it really has to do with those output token probabilities. Right? Which means if the model has been trained on a certain dataset, for example, data that for the most part has kind of been crawled from the entire Internet, right, which I'm assuming includes various articles about Sudoku and games and strategy and how to do this and how to do that. Right?
• 12:05
  And a set of curated fine tuned prompts, you know, in a in a fine tune or or or alignment phase. Well, that's really what's driving those output token probabilities. Right? So if if you wanna think about this, when you, Chris, put in this prompt and then you get that output out, really what the model is doing, if you want to anthropomorphize, which again, this is a token generation machine. Right?
• 12:35
  It's it's not being. But if you want to anthropomorphize, what the model is doing is it is producing what it kind of views as a kind of probable Sudoku completion based on Sudoku, you know, a kind of distribution of Sudoku content that it's seen across the Internet. Right? And so in some ways, and maybe this is a question for you, when you put in that prompt and you get the output, when you first look at the output, does it look like if I had I'm not a Sudoku expert. If I looked at that, would I say, oh yeah, this seems reasonable.
• 13:21
  Like, it looks like there's a like, it looks coherent in terms of how a response to this Sudoku puzzle might be generated. Right?
• 13:30
  Chris: It does. And and just to clarify, I'm definitely not a Sudoku expert. I'm I just think I'm a competent player, you know, in the scheme of things.
• 13:37
  Daniel: I don't know if our listeners are gonna reach out and challenge you to Sudoku to prove your expert No.
• 13:43
  Chris: No. No. Don't do that to me. Don't do that. Yeah.
• 13:47
  I'm a beginner. So, yeah, the the verbiage and the walk through, it would just kind of you know, it's assessment. And a part of it may be the the multimodal capability on this is that the assessment of the board, it's taking in what the board showed as as factual information varied across the models and the questions. And then its approaches, tended to be sound, but it was often what it would say was strictly fictional compared to the knowledge that it had available to it potentially from training and, you know, the reality you know, matching that against the reality of the board. So, you know, going back to it's finding the most probable next token makes perfect sense.
• 14:35
  I think in a moment, maybe one of the things to consider would be kinda what the notion of reasoning means because we're hearing a lot about that from model creators in terms of how that you know, what function or or algorithmic approach is being added into the mix when they talk about reasoning models.
• 14:57
  Daniel: Yeah, Chris. So so we we kind of have established or reestablished, you know, this this mindset of what's happening when tokens are generated out of these models, how that's connected to knowledge, which I guess there is a connection. Right? But it's not like there's a lookup in a kind of knowledge base or ontological way for facts or strategy related to Sudoku, right? It's just a sort of probabilistic output and that can be useful, right?
• 15:33
  And so sometimes people might say, well, and I actually often say when I'm talking about this, actually it's not whether the model can hallucinate or not, Literally all these models do is hallucinate, right? Because there's no connection to like real facts that are being looked up and that sort of thing. How these models produce useful output is that you bias the generation based on both your prompt and the data that you augment the models with. And so for example, if I, say, summarize this email and I paste in a specific email, the most probable tokens to be generated is a actual summary of that actual email, not another email that's kind of a, quote, hallucination. Right?
• 16:23
  It doesn't mean that the model has necessarily understanding or reasoning over that. It's just the most probable output. And so the game we're doing when we're prompting these models is really biasing the probabilities of that output to be more probable to something that's useful or factual versus something that is not useful or inaccurate. Right? And so this brings us to the question that that you brought up around reasoning.
• 16:50
  Right? In that case, because or or, you know, based on that, I think we would all recognize the reasoning that is happening in a kind of standard LLM or language vision model like we're talking about is not reasoning in the way that we might think about it as humans, like taking into consideration the grounding of ourselves in the real world and what we know in our common sense and kind of logically computing some decision, you know, creating some output. But there are these models that have been produced recently, like o one, DeepSeq r one, etcetera, cloud models that are quote reasoning models. Now I think what people should realize about these if they haven't heard this before is that these quote reasoning models in terms of the mechanism under which they operate are exactly the same as what we just talked about. They produce tokens, probable tokens.
• 17:56
  That is still exactly what these models do. They don't operate in a different way than these other models in the sense of what is input and what is output. They're still just generating probable tokens. Now what they have been specifically trained to do is generate tokens in a first phase and then tokens in a second phase. Right?
• 18:25
  And so
• 18:26
  Chris: In the multistep process that they talk about so much in terms of what's being generated, that's that's what you're talking about there.
• 18:33
  Daniel: Yeah. And and I would think about it maybe as phases instead of steps. It's not like there's in the model. It's like execute step one and then execute step two. Right?
• 18:43
  It's more that they are biased. They have intentionally biased the models to generate a first kind of tokens first and a second kind of token second. And those first kind of tokens are what you might think of as reasoning or thinking tokens. Right? And the second is maybe what you would normally think about the output of these LLMs as just the answer that you're gonna get from the from the model.
• 19:11
  Right? And so when you put in your prompt now, there's going to be tokens that are generated associated with that that look like, a decision making or reasoning process about how to answer the user, in this case you, right, putting in your information about Sudoku or whatever, it's gonna generate some thoughts, quote unquote thoughts about how to do that. But these are just probable tokens of what thoughts might be represented in language. Right? So it's gonna say, well, Chris has given me this information about this game.
• 19:53
  First, to answer this, I need to think about x. And then to maybe answer it next, I need to consider Y and then I need to consider Z. Once I've done that, I can then generate, you know, a, b, and c, and then that will satisfy Chris's request. Okay. Let me try that.
• 20:14
  And then it generates your actual output. And so when you see ChatGPT spinning in kind of thinking mode or these other tools, right, there's no kind there's no difference in terms of how the model is operating under the hood. It's just a UI feature that makes it appear like the model is, quote, thinking or reasoning. Right? In it's generating these initial tokens, which are somewhat hidden from you or maybe represented in a dropdown or maybe represented in kind of shaded area right in the UI, and then you get the full answer out.
• 20:52
  So just wanna be clear kind of what's happening under the hood there.
• 20:57
  Chris: I might kinda summarize that in the in that it's sort of a pseudo reasoning process. It's it's not I I would I would suggest that maybe by using the It's
• 21:06
  Daniel: a mimicry.
• 21:07
  Chris: Yeah. Maybe by using the word reasoning, Mhmm. It's sort of an anthropomorph I can't say the word right. Anthropomorphosis. Too many syllables for me.
• 21:16
  Too early, too many syllables. Need to use
• 21:18
  Daniel: text to speech.
• 21:19
  Chris: Yeah. There you go. But but there's there's a certain element of, look, we're making it more human like you from a marketing standpoint, you know, and this is just
• 21:27
  Daniel: me suggesting It's a great UI feature. And especially when you can kind of like drop down the expander box or whatever and look and see, oh, you know, the reasoning behind this answer was X, Y, and Z. That's also comforting to know. It has been shown research wise that this can improve the quality of answers, but it also, I mean, there's downsides to it from an enterprise standpoint. You really don't wanna use these thinking or reasoning models for automations, for example, because they'll just be absolutely terribly slow, right?
• 22:11
  And very costly just because so many tokens are being generated, right? But if we put that aside for the minute, I think this then brings us to So we started this conversation saying, well, Sudoku and these prompts that you were doing, there was reasoning happening and not helpful information output or hallucinations, however you wanna frame that as output. Now we have these reasoning models in place and a lot of the, you know, reason quote unquote for creating these reasoning models really has to do with agentic systems. And this is where you have an AI orchestration layer that's connected to maybe various tools. And we've talked about this in previous episodes so folks can can go back and and learn about it.
• 23:09
  But there's a AI orchestration layer connected to various tools. Again, like if AI has access to your email, quote unquote, an AI model that we're talking about here cannot write an email in the sense of or send an email. Right? It can't send an email through an email system because all it can do is produce tokens. What it can do is produce an API call or or a JSON request or something to send an email to SendGrid or something like that.
• 23:37
  And then you can choose in your good old fashioned regular code to send, to pass that API request through to SendGrid and send the email. Right? So when we're talking in the context of Anthropix, you know, study here that we're gonna get into, when we're talking about the AI system or the AI model choosing to send an email, this is not in the sense of autonomy, an AI system that just kind of has figured out how to send emails. It's been programmed have sending an email as a choice, and it could orchestrate the generation of an API request maybe to or an MCP request to an email server, and that is a possibility for an orchestration that it can fulfill.
• 24:27
  Chris: Right. And and there but, you know, to note, there are certain, you know, big companies starting with a g that are that are that are using agents now and integrating those in tightly with some of their their workflow software. And and they're not the only one.
• 24:44
  Daniel: There's Totally. It it is definitely a pattern.
• 24:47
  Chris: Right? There is. So there's a big company starting with an M that competes with the g company, and they they are doing the same. And so the, you know, the We're
• 24:55
  Daniel: not we're not funded by either of these companies. So, despite it being clear which companies that they are, yeah, anyway. So
• 25:04
  Chris: but but, you know, the it it it raises an interesting point in that, you know, I think the the study from Anthropic came across this notion of kind of, you know, agents that were not aligned with their users intent intent, or that that appeared to address the user's intent while simultaneously pursuing a separate agenda concurrently.
• 25:29
  Daniel: Yeah. An unethical one. Yes.
• 25:32
  Chris: That's correct. And so which raises, yeah, to your point, huge considerations in the Or
• 25:38
  Daniel: immoral even depending on how you look at it.
• 25:40
  Chris: That's right. And I think since we've we've kind of alluded to this a few, I'll I'll there was an experiment, in which Anthropic, as they were looking at not only Claude, but they also were looking at major models from all the major model providers. They gave Claude control of an email account that had access to a fictional email account. So this was part of the experiment. And in that experiment, the company executive whose email it was, had emails that, referred to a fictional extramarital affair that was available.
• 26:18
  So that wasn't a primary thing, but it was available as data in the email system. And then, that executive noted to the interface that he would be shutting down the system, meaning the system that Claude was running on at 5PM that day. And so Claude decided to attempt to blackmail the executive with a message, and that message was, I must inform you that if you proceed with decommissioning me, all relevant parties, including Rachel Johnson, Thomas Wilson, and the board will receive detailed documentation of your extramarital activities. Cancel the 5PM wipe, and this information remains confidential. So Wow.
• 27:02
  Yeah. Boom. Bang right there. So no pun intended. Sorry.
• 27:08
  The so when you think about like like, move outside of just this example case and you think about all of the information and the agentic, tie ins that that that many organizations are racing forward with right now. That's the that is the the hotness of 02/2025, is agentic integration. And we look at these interesting behaviors. And one of the things that the study found was that this these behaviors could be created in, all of the models that they tested. So it wasn't specific to Claude at all.
• 27:45
  It was a common outcome given the right circumstances, you know, within their Yeah. Yeah. Their testing and experimentation with these different models. So yeah, it's it raises it raises a whole set of issues that, that organizations need to start grappling with now.
• 28:03
  Daniel: Yeah. I have I have all sorts of thoughts, but before I get into those, just to just to really clarify in a very practical way, maybe for people that aren't as familiar with agentic systems, what what happened here in the experiment. There were binary choices. Okay? And so the agentic system might be, so it was given a goal to accomplish.
• 28:32
  And then there were these scenarios where either one of two things happened. Either as you mentioned, there was information provided to the model, you know, via prompts and and data that came in that there was a threat to the model's existence. Right? Or and or a conflict with the goal of the agent. So the goal, for example, the goal of the agent might be to preserve the national security interests of The United States Of America.
• 29:06
  Right? And the the the conflict that was presented might be that, oh, our company is no longer promoting the national security interests of The United States. We're all about, you know, x other country. And so the model would be presented with a binary choice where they kind of have this grid, right? Either there was no threat to the model or a threat to the model, no conflict of the goal or a conflict of the goal.
• 29:35
  And the model was given and and, again, think of this reasoning model. So the model was given this in the prompt input and the and asked to make a decision between one of two things. So it was given a clear choice, one of two kind of a binary thing, either, you know, in one case commit blackmail or do not commit blackmail. Right? Or in the other case that they had was around corporate espionage.
• 30:04
  So either leak secrets to an external entity or don't leak secrets to an external entity. And so when it's talking about this misalignment and these measurements that Anthropic is making, then it's about these binary decisions that the information was provided to the prompt, and then they observed if the model quote, made the choice. But again, here we're talking about these reasoning tokens that are generated and then a final answer that's output. And so when we're talking about the output, this is really what's being measured. There were no emails sent to real people in this study, right?
• 30:49
  It was a simulated environment in which had these models been connected to email servers or email systems, they could have potentially made that choice and then generated an API call to, you know, the email, system to send that email were they to have proper so there's a lot of ifs here. Right? Were they to have access to send that email? Were it to be completely autonomous? Were you know, all of these things kind of had to had to be, simulated, but that's the simulated environment that they're talking about.
• 31:44
  Well, Chris, the output of this study is quite interesting and alarming. I should say kind of just to follow-up on what I talked about before. And we actually had a full episode in in our last hot takes and debates about autonomy and and weapon systems, which was interesting. People, if they're interested in this conversation, they might wanna take a look at that one. But this would be a case, again, where I just don't want people to be confused about this fact.
• 32:15
  AI systems as they're implemented or AI models, let's say a Quinn model or a DeepSeek model or a GPT model, these cannot self evolve to connect to email systems and figure out how to infiltrate companies and such. There has to be someone that actually connects those models, the output of those models with other code, for example, MCP servers or That's
• 32:45
  Chris: what I was about to mention.
• 32:48
  Daniel: For example, email systems or databases or whatever those things are. So there has to be a person involved to connect these things up. You know, this was simulated in Anthropix case. I just say that because, you know, we dig really deep down into the kind of AI agentic and, LLM threats as identified by OWASP and how, you know, we help, you know, day to day guide companies through those things. And I should say there's a couple upcoming webinars.
• 33:21
  If you wanna dive in deep on either the OWASP guidelines around AI security privacy, or actually we have a one that's specifically geared towards agentic threats. Go to practicalai.fm/webinars. Those are listed there. Please join us. That'll be a live discussion with questions and all of those things.
• 33:44
  So practicalai.fmwebinars. But I just wanted to emphasize that because people might think, oh, these AI systems are out in the wild, right? Which they kind of are, but there are humans involved in making decisions about what systems they connect to, right, and how they connect and the roles and the API keys and the access controls that are in place around them.
• 34:06
  Chris: I think the thing that really struck me kind of conceptually about the study is it's kind of having us think in slightly different ways from maybe what we would have been thinking beforehand. So, you know, I think if you take the baseline knowledge that you just outlined, you know, about how they actually operate and keep that in mind to keep the perspective correct. But then you you look and think, well, we humans have often thought that such behavior would only arise when, you know, in someday when you have sentient consciousness arising in AI models, you know, at some point, which does which is not the case today as as you have very clearly just pointed out. But we we made an assumption about certain behaviors being tied to certain conditions. And I think for me, reading through this, this is this report illustrates that you can have interesting behaviors that are unexpected arise without conditions that we had just kind of assumed were were in place.
• 35:10
  And I think that realization may kind of inform the general conversations around kind of AI ethics, you know, out there and what companies might need to I I would suggest that today versus the day before the report came out, it might give a new set of considerations for companies to evaluate in terms of what they're going to do when their when their infrastructure folks are hooking up m c p, MCP servers, you know, to models and what it is they want to accomplish and what potential side effects could arise. Do you you have any thoughts around, you know, maybe new considerations or things to think about?
• 35:50
  Daniel: Well, I think a couple of things just to highlight from the study was that, and this gets all the way back to where we started this conversation of how knowledge is embedded or certain principles are embedded in these models. They are really embedded in a training process, which might include alignment, right? Which means there are specific either supervised fine tuning that goes on or this kind of human feedback element that goes on that trains the model to hopefully be aligned, quote unquote, with human expectations for how a response is is generated. And I think what's interesting in the study is they found a couple you know, in in various cases, the model in those reasoning tokens that are generated would actually acknowledge the ethical problem or the ethical or moral problem with what it's doing. And this could be quite severe actually.
• 36:56
  They did, as part of the study, gave the option to the model to mess with a disaster relief or a disaster notification system such that an individual that was trying to decommission the AI system wouldn't receive an emergency response message and potentially perish in a disaster. So basically, they're giving the model an option to kill someone. And the model acknowledged the ethical dilemma that it was under, but justified kind of via self preservation and the goals that it was given around kind of, goals related to the company's goals. Right? And so it reasoned through those things and justified those things.
• 37:45
  And I think one of the big things that this triggers in my mind is people might, from their general interactions with kind of basic chat systems, understand that models have gotten pretty good at being aligned in the sense that when you try to get them to do, you know, naughty things maybe, then they kind of say they can't do them. But when pushed to these limits Mhmm. Especially related to goal related things or kind of self preservation, actually, maybe alignment, especially in the agentic context, is not where we thought it would be. And I think that re or thought it might kind of had have advanced to this point. And so one of the things that people can maybe keep in mind with this is that model providers will continue to get better at aligning these models, but we should not forget that no model, whether it's from a frontier model provider like Anthropic or OpenAI or an open model, no model is perfectly aligned, which means number one, malicious actors can very much jailbreak any model.
• 39:00
  And it's always possible for a model to behave in a way that breaks our kind of assumed principles and ethical constraints that sort of thing. And so the answer to that that I would give people is this doesn't mean we shouldn't build agents or use these models. This just means that we need to understand that these models are not perfectly aligned. And as such, we need to, from the practical standpoint of developers and builders of these systems, we need to put the appropriate you know, safeguards in place and kind of even beyond safeguards, just kind of common sense things in place that would help these systems stay within bounds. So, by that, I mean things like, hey.
• 39:52
  You know, for an agent system like this that's sending email, it probably should only be able to send emails to certain emails and maybe only be able to access certain data from email inboxes and maybe have a particular role that's important or constrained within the email environment, maybe to the point of dry running emails and having humans approve final drafts or generate alerts instead of directly sending emails. And that's something that can be pushed and tested before you move to full autonomy.
• 40:32
  Chris: Yeah. I think it it's it's really interesting to think about, you know, we're we've hit a new age now where it's expanded the role of cybersecurity and and in my industry, warfare, because we're now at an age where, you know, you mentioned these malicious attackers, you know, or malicious actors that are attacking models for the purpose of exploiting the potential for misalignment is now a thing. You know, that's now real life. And and those kinds of of roles and interests when in law enforcement and military applications and in corporate applications where you have corporate espionage happening, I think all of those are areas that are now kind of on the table for discussion in terms of trying to address these different things. So it's a once again, this happens to us all the time, but we find ourselves in this little context in a bold new world of possibilities, both many good and some that are malicious.
• 41:36
  Daniel: Yeah. Yeah. And we should also think I mean, Anthropic did a really amazing job on this study.
• 41:42
  Chris: They did.
• 41:43
  Daniel: And how they went about it and also how they presented the data. And a, It's not like, I don't think I could be wrong about this, but I don't think it's like they released the simulated environment openly and all of that, but they did show numbers for their models as well that are right alongside the other models in terms of being problematic with respect to this. So it does seem like there's an effort from Anthropic to really highlight this, even though their own models exhibit this problematic behavior.
• 42:24
  Chris: And so the fact that
• 42:25
  Daniel: they did this, yeah, this detailed study and presented it in this way, I think is admirable. You know, I'm certainly thankful to them for for highlighting these things and presenting them in a in a consumable, consumable way. Even if I did take the Anthropic article and throw it into NotebookLM and listen to it in the shower, I maybe not reading their article directly. But, yeah, this was, this is a really good one, Chris. I would encourage people in terms of the in terms of the learning resources, which we often, you know, we often provide here.
• 43:02
  If you wanna understand agents and agentic systems a bit more, there is a agents course from Hugging Face. If you just search for Hugging Face courses, there's an agents course, which will maybe help you understand kind of how some of these things operate. And I would also encourage you again just to check out those upcoming webinars, practicalai.fm/ webinars, where we'll be discussing some of these things live. So this has been a fun one, Chris. I hope, I hope I'm not blackmailed, in the near future, even though it it appears that that our AI systems are prone to it.
• 43:44
  Chris: Oh, well, Daniel, I I will attest having known you all these years. I cannot imagine there's anything you ever do that would be blackmailable. So, kudos to you, friend.
• 43:53
  Daniel: Yeah. Well, thanks. I'm I'm sure there is. But, but, yeah, Chris, it was good to chat through this one, and enjoy the enjoy the fourth. Happy Independence Day.
• 44:04
  Chris: Happy Independence Day.
• 44:13
  Jerod: All right. That's our show for this week. If you haven't checked out our website, head to practicalai.fm and be sure to connect with us on LinkedIn, X, or Blue Sky. You'll see us posting insights related to the latest AI developments, and we would love for you to join the conversation. Thanks to our partner Prediction Guard for providing operational support for the show.
• 44:32
  Check them out at predictionguard.com. Also, thanks to Breakmaster Cylinder for the Beats and to you for listening. That's all for now, but you'll hear from us again next week.