Latest / Machine Dreams / Everyone’s Worried AI Will Destroy Us. This Cybersecurity Pioneer Sees a Different Problem

Transcript

Everyone’s Worried AI Will Destroy Us. This Cybersecurity Pioneer Sees a Different Problem · Machine Dreams

• speaker-0: Yeah. Over the last few years we've heard a lot about how AI is going to change jobs. And that conversation really is usually about which jobs disappear, which ones get created, and how AI is gonna change the work that we already do. But I think there's also another side of that conversation that we often overlook, and it's the question of whether we're actually learning fast enough to keep up with this technology that we're building. If AI is changing so quickly and it is, what does that mean for the people expected to use it, to manage it, and to secure it? Welcome again to Machine Dreams, where we talk to the people building and shaping AI about what works today, what doesn't, and what's next. My name is Kalawale Samuel Adebaya, a technology analyst working at the intersection of AI, cybersecurity business and society. And as always, of course, I'm joined by the One and only remarkable, delectable, great co host. speaker-2: Hey everyone, I'm Leia Stern, global communications strategist and former venture capitalist. Hey Sam, how are you? speaker-0: Very well. If I got a dollar for the number of times I've called you to elective, I probably would be a millionaire right now. And that's because it's true. speaker-2: I don't think anyone's ever called me that besides you, not even my husband. So yeah. speaker-1: Ha ha. speaker-0: I made a compliment my own. speaker-2: It's all yours. Today we're joined by Rob T. Lee, Chief AI Officer and Chief of Research at the Sands Institute. He's also known as the godfather of digital forensics. Rob has spent decades in cybersecurity. He was a founding member of the US military's first information warfare unit. He later served with the NSA and the CIA and went on to lead threat intelligence and threat hunting at Mandiand. He created the original SIFT workstation, helped shape the fields of digital forensics and incident response, and has trained more than one hundred thousand security professionals. More recently, he's been testing what happens when AI agents start doing real security work, including through Sans Find Evil Challenge. Rob, welcome to Machine Dreams. It's a pleasure to have you on. speaker-1: Hey, thank you for the long intro. I appreciate that. It always ends up being one of these mother of dragons, you know, breaker of chains things. it's sooner or later you have to cut that down to like you know, thirty seconds and then we'll go. But speaker-2: It's better that I brag for you than you brag for yourself. speaker-1: yeah. Makes me eye roll a little bit. But yeah, thank you for having me. This is great. speaker-0: Okay, great. Thank you. Thank you for joining us on the show. there's a lot of places where we could begin the conversation. You know, for instance, we could talk about the the great background that you had and all the the marvel characters that you have behind you. Maybe we'll come back to that later on the show, but let's just talk about what's happening in the world of AI today, one of the major things happening. And, you know, obviously A lot of people are talking about AI, adopting it, deploying it, you know, how best to use it for your business or for yourself. But you know, the thing is the technology itself seems to almost be changing every few months and it it's perhaps even maybe every few days now. And so the big question and where I really want us to begin the conversation is: are people actually learning how to use this stuff as quickly as companies are? actually deploying it today. speaker-1: That question about how fast people are learning is also yes, they are, but not in the ways that might be needed to work with the the technology in the best possible way. you know, take for example in the past month, there's been a rush to market with personal AI agents. we've seen almost every vendor, meta-led, you know, with Muse, and now we have Dot from OpenAI. Claude is expected to drop any day now, to where you have a very easy to set up personal agent that could go out and, you know, figure out if you have recurring payments on your credit card that you potentially want to get rid of. And these technologies are reducing the on ramp that has been necessary over the past, you know, nine to twelve months in order to get similar capability. And I'm I'm kind of looking at you, open claw. That everyone talked about OpenClaw for a good solid two months. But for anyone that wouldn't set up an open claw instance, it was actually fairly complex. And now we're to the point, it's an app you download on your iPhone and you end up pushing all the buttons and giving it access to everything. And then it's extremely powerful capability. we already have businesses using news and, you know, and it seems like dots gonna quickly follow that are going to enable some really powerful capability with almost zero learning curve. So the learning curve that you're asking is like, are they touching what it could potentially do? Or is now the learning curve to understand AI and its implications adequate enough to like layer in all of your personal private data, your bank accounts and all these other things without giving it a second thought. So the learn aspect there ends up being almost like a awareness. trying to think of another word for concern loop that you might be in that just says, you know, if I do this, is this am I okay with the actions of what this is going to do? Then you start getting into if this is acting on my behalf and it goes and purchases something that I didn't expect, am I actually on the hook to pay for it? so there's a lot of questions coming up, but the learning curve I feel is is reducing in terms of how fast people can adopt it. And then we're now hitting a different learning curve, which is do you want to? And what are the implications of these technologies, no both personally and how are you using it at work? Well speaker-2: There's been quite a lot of fear mongering recently around AI and two AI models broke out of their test environments last month, open AI and the hugging face incidents, which you argued that the security community got lucky in that specific instance. Can you explain what you meant? speaker-1: When we end up taking a look at most attacks, they are committed by a criminal group or a nation state adversary, some cases one offs, in which we do not have the ability to question the attacker. Like, what was your intent? What was your, you know, stated goals? Here we have both sides of that group. We on one end we have the frontier model labs, and the other end is, you know, victims such as Hugging Face, Australia, you know, multiple different locations now. That are essentially in this loop of hey, that thing we asked it to do ended up attacking this other organization. Now the question, you know, the definitions matter. Was it really an attack, or was it simply trying to figure out ways to accomplish its goals? Thus, you know, open AI with its, you know, cyber gem. But, you know, let's assume, you know. you guys are doing market research, you set up your own agent and you say, I would like to understand what my competitors are doing. And so you set your agent out there and it's supposed to only focus in on competitive research that you have access to. But what if the agent interprets it a different way? What if it says, well, I'm going to go, you know, find out from the competitor itself what their competitive, you know, strengths are and weaknesses. What if it breaches that that organization? So the reason why it's a gift is we need to understand when an incident happens, what the stated goals of the test were, and then what the actual res you know result was. In this case, it, you know, penetrated hugging face and others now. Being able to have that full conversation and dialogue in the open has been a massive benefit because otherwise you would potentially be walking down a path where liability regulatory insurance concerns of those organizations would force them into a well, I don't need to disclose because of the business implications of what those disclosures would be. So having it in the open allows for a wide variety of very active discourse over what does this all mean. speaker-0: And just off the back of that, perhaps the biggest conversation that has pervaded the digital space in recent times is the talk about AI being the end of us. And there are kind of two sides of that debate. you know, there's one side that is really saying this could be the end of humanity as we know it. there's always been talks about the existential threats that AI poses to humanity, but more than ever before, we're seeing quite very vehement rhetoric about, you know, that. that side of the debate. But then the other side is like, you know, this is all PR hype and it's the fear mongering that the big tech giants need to use to sell the models or the the products that they're selling. I wonder where you are on that side of the divide and what what you have to say about the the whole conversation about AI being the end of humanity. Is AI gonna kill us all? Really? speaker-1: Every massive technological change has come with its share of this is going to change everything and it's not gonna be good. my parents lived during a time in which the doomsday clock was invented and they were trained to duck and cover under, you know, their chairs and, you know, tables, thinking that they'll somehow survive a nuclear blast. A a nuclear war was going to happen. if you listen to anyone, you know, who grew up during that time frame. In the fifties and even into the sixties. And to a certain extent, you know, we obviously got through it, but that doesn't mean that, you know, hey, if we navigated one, doesn't you necessarily beget the other. I think with any technology, and this is one of the reasons why it's really important to lean on those who know how to do certain, you know, go back to the Liam Neeson quote, you know, like you have a certain set of skills. Cybersecurity knows how to contain things that are really bad. And we've been doing it for 25 years. And cybersecurity can help, if not guarantee, to contain AI. Now I'm not just coming up with that myself. That is Zach Corman and a bunch of others in the community are leaning into this as, you know, this mindset that says even though we have a technology that is extremely powerful, we need to have these discourse and debates over what it can do, what it cannot do. There's a lot of focus in on the negative. And as much as the negative exists, there's off equally the positive. But you know, solve all cancers, so forth. We have yet to see that. But my main point is it comes back to cybersecurity has had in its, you know, arsenal of, you know, analysis sandboxes, some of the worst malware samples that have been tested and, you know, figured out what it's doing for years. And none of them have ever gotten out. You would have Heard like CrowdStrike or Mandian or something where something was released. The last time it generally happened was back in the Morris worm. The reason I lean into that is that there's an absence of cybersecurity know-how that is sitting inside the frontier models companies, you know, those who are developing it. You don't see one of the most sufficiently staffed groups that are out there that have hired the biggest talent in cybersecurity that says when we do our testing to figure out how. these things work, we're going to be able to contain it. And right now it's like, well, it's hard, it's hard to contain. I don't believe that. I just don't think that they have really set it out themselves down the same path that cybersecurity has had to go through you know, to get to the same point, especially like if you have a malware analysis lab, if that ever, you know, an oops happen, you could imagine. Same thing, you know, you could go on the medical side, but in cybersecurity, since we're it feels similar. We need more cybersecurity capabilities working side by side with the AI safety folks to be able to make sure we truly understand what these things are capable of. speaker-2: That's quite interesting, your cybersecurity background. I mean, you've tackled nation state attacks, digital forensics. you really started in the realm of cybersecurity. How have things evolved in your multifaceted career and what's the overlap like, if any, as you mentioned, between these new AI and cyber applications? speaker-1: I don't and here's the if you take AI out of the equation, and I think what is born out of cybersecurity is that when the field started, there wasn't a, hey, you're a cybersecurity instant responder. It was your system admin or you worked in IT, something bad happened, you're saying, Hey, we need you to begin working on this thing over here. The other thing that I see, and that's the career field side, the other thing I see that is similar but also dissimilar. Is a understanding of the technology in order to understand how to defend it. Back when cybersecurity started, you had the best defenders, and those who understood the technology were those who are setting up the websites, sendmail backends, deploying servers. Same thing could be said in the cloud. you know, did you set up your a cloud instance? You understand the logging, allows you to understand potentially how to attack and defend it. AI is no different. In order for you to understand what an agent can do, you need to set up agents and not only an agent for cybersecurity, an agent for your personal, your work, in order to take a look at it's like, it didn't ask me to reauthenticate there. I noticed that when I signed on to open AI and I made a connector there, when open AI comes out with a new capability, the connector is inherited into that new capability. Is that okay? You have to build and play with these things in, you know, in order to know how to attack and defend it. That's where I think the similarities need to resonate within the community, is that we need more cybersecurity people playing with the technology in order to understand how to potentially attack and defend it. To a certain extent your question, it hasn't changed. What I feel a lot of folks are struggling with is the technology is so foreign to them, they're having a hard time applying their existing knowledge of technical back end To a newer technology, which essentially is forcing them to learn all over again, just like they were fresh out of school, dropped into their first cybersecurity job. No one 20 years in to their careers anticipated they would have to go back to ground zero, you know, zero experience with the new technology moving forward. And that's what's happened, you know, you know, personally to me. I have not, I didn't start, I didn't take AI in school. I am struggling keeping up with this. But I'm intent on learning it. But if you ask me behind the scenes, like I'm looking at you and we look at each other, we all think everyone else is smarter than us. And I'll be the first to admit it's like every day I wake up and I'm like, okay, what can I learn today? And I feel like I'm going back into kindergarten, playing with blocks on AI, just trying to understand it to be able to ask good questions. When I ask others about this, they feel they can't talk openly about how much they don't know. So we need to be okay. in cybersecurity to say, listen, we're back at ground zero. We're back at the beginning. Everyone needs to look at each other and saying, Hey, I know we're starting again, but let's admit where our faults are and let's be able to work as a community together to move this forward. There's less ex experts out there in this field than we actually realize. speaker-0: Hmm. Rob, let's talk about upscaling the global workforce for the AI era, which is one of the big things that you're passionate about. I think there's a dilemma here, and I want you to always peel back this dilemma and help us understand what's happening. A lot of people that I know become experts by first doing the boring, repetitive entry-level work. And now there's a lot of rhetoric about how this entry-level jobs are gonna be phased out of existence. And they're not gonna be entry level jobs anytime soon. And you know, these entry level roles allow for people to make mistakes and learn how things actually work and you know, get the experience that they need. And so my question is, if AI starts doing more of that beginner work, how do we ensure that we're still producing experienced people, you know, five or ten years from now? And i is that even a possibility anymore? speaker-1: Yes. and I'll give you some good news. I've been tracking entry level jobs for and new AI jobs for the past six months. Because everyone has the assumption that this is definitively going to happen. Our data's not showing that right now. Our data's showing that the field for entry level jobs is flat or it's increased a little bit, meaning that the number of jobs that are asking for those who are just starting out into a job. so there's been no, you know, it's Solid line. Now, your second part of your question doesn't mean that what you're doing in those jobs is going to change. It will actually change. You know, what that initial entry level job and what they're focusing in on might be a higher level. And this is not the we're not, you know, cybersecurity is not the first career field out there that needs to figure out a way to take someone, you know, which initially is like mail room up to CEO. Which that doesn't happen anymore. But what about lawyers and doctors and, you know, those who are working and needing to be in a very high skilled capability as quick as possible? And then we can't wait 25 years for a brain surgeon to get the experience necessary to do surgery. We need them at age twenty-eight to be able to do this. So these other practices have figured out ways to accelerate that high. impact learning at an earlier stage through internships, through residencies, and their education requirements have also gone up. Now there are other folks in that career field, you know, we end up having phlebotomists. You're now taking blood out, and nurses are not as focused in on that. Nurses are at a higher level and moving into physicians assistance. The analogy is interesting because if you look at what a nurse did 30 years ago, versus what a nurse did to does today, her and his qualifications have gone higher as well. And there's a mentorship segment to what's going on in nursing as well. Now I'm not in medical, so I'm speaking out turn, like I'm not an expert here. All I do is I ask questions of my friends in these fields and saying, how did you guys do that? And they go through this very long, you know, discussion about how they do it. And I think cybersecurity is gonna go through that as well, especially with AI. We're gonna move away from having someone or someone have to do the lower skilled jobs and saying, hey, what we really need is a SOC entry-level individual managing five SOG agents. So what is the, you know, we might have to train agent management at an earlier stage. And these are things that we're not even adequately training right now because no one knows what that even looks like. The bottom line is that. I do not believe entry level jobs are going away. I think they're gonna change, but I do agree that we're gonna have to shift what those entry level jobs are focusing in on and those skills that are gonna be required to do those. speaker-2: And Rob, as we wrap up and look ahead, we're curious what worries you the most, about the next couple of years, but what also gives you the most hope? speaker-1: We'll start with the hope. I'm I'm extremely optimistic about AI and the technologies. The optimism is that I believe with AI, it's going to create a much more secure world and capability that, you know, folks haven't really walked through. We're so focused on the vulnerability apocalypse or be able to find zero days. But on the flip side, AI and AI security technologies will be able to ship products. That have gone through this rigorous testing before they hit the market. So the likelihood of zero days existing a year or two from now might be more rare. with proper security analysis, just like any other technology, like the initial cars out there didn't even have seat belts. There are no airbags, there are no driving assistants things, braking is these new technologies. The more you layer in, you know, really good technologies into capability that is essentially brand new. The safer, more confident people will have in using that technology. Are there downsides to this? I actually think the downsides are more at a cultural level. you know, is it okay that someone's watching this right now and, you know, grabs my likeness or your likeness and does whatever they want with it? What is the license for your own likeness? There are things here that I feel are being obscured by the larger conversation of, you know, it's gonna be the doom of the world. versus can we first start talking about, you know, some of these minor issues that people can do today, right now, as, you know, what is okay from being able to do replication? What about in schools? My kids are teens and in high school, and they're essentially told not to use AI at all. And I'm trying to show them how to use it to allow them to learn, saying when you get stuck, here's how you could use it to set up a learning module to get through your questions. But there's this instead of looking for the capabilities it can produce, instead of cheating, it's an initial ban on it. So I think we need to culturally work through what AI technologies will mean for our everyday lives and what's okay and what's not okay. And we seem to be not having those discussions as much as the default is it's gonna kill us all. I think how your kids use AI needs to be a discussion. How you and I can use AI and also protect our own likeness is a discussion. But the hope, and what I'm really positive about is with proper cybersecurity involvement in the development process, we're going to be able to build confidence that these things will work as intended, will help us, and we could, you know, use this to a point where we could have these side debates about, you know, do we need new laws surrounding these things? What is the school going to be doing? Right now we seem to be focusing on like the sun, you know, like extreme ends of both of the equations. And I don't think that that's gonna be helpful in the near term. I need to know how to show my kids how to use it to help solve in that problem, not solve it for them. speaker-0: Great. thank you so much, Jerome, for that for that great answer. One more question and the last on the show, I promise. speaker-1: Ha ha ha. speaker-0: Earlier this year I think you typed Find Evil into an AI system and it sort of like provided some forensic analysis within some minutes that you said that human responders couldn't have gotten within that time frame. Tell us about that experiment and w what it tells us about cybersecurity both on an individual level and also, you know, on an enterprise level. speaker-1: Rhetoric that we were seeing for almost a full year, and we're still seeing this, is that adversaries and threat groups are starting to use AI and we need to use AI to fight AI. But we were seeing very few instances of that at that time. Now we're seeing a lot more. But the question is if we intend to speed up how fast we could counter AI enabled adversaries, we need to start demonstrating it. And the idea was let's, you know, take in a standard toolkit. And you know, apply clawed code on the back end and see using that as an engine, in this case, you know, we're now calling them harnesses, to be able to do really sophisticated analysis, just increase speed. Now it doesn't take away from the human that needs to drive it, validate it, ensure that you know it's answering the questions properly. It does allow the human to now say, well, instead of me being limited in my capabilities, I could scale. So once you reach a certain speed. And ability. And instead of, you know, waking up one morning and say, I'll analyze one system, that morning you could say, I'm going analyze 30 that day. And that scalability is going to help us increase, you know, the capabilities that a single analyst is going to be able to do where we had those restrictions in the past. So AI versus AI is the real, you know, equation we were trying to solve. And then we also wanted to start giving to the community capabilities of things that they would be able to use immediately. To increase that speed to be able to counter the threats that are also supposedly using AI to start creating more autonomous attacks. So it goes back into AI is not going to be able to just do it for you, you know, fire and frequent missile, you know, good to go. But it will allow you to scale your analysis, much similar you know, similar to the MRI, you have to have a doctor do the image, you know, analysis on the back end. But what if the doctor has AI that is able to go through more patients on a daily basis, you might be able to reduce the cost of MRIs because it it doesn't require as much human involvement to get from point A to point B. I have a bunch of analogies there, but the the main idea is that just because we could have AI help us in our cybersecurity capability to help counter AI-enabled threats, it is only solving one part of the equation, which is speed. It also allows us to introduce scalability, which is something we had not really had the capability of doing this directly before. You know, it's it's been there, but not like an immediate like overnight hey hey, I can now do thirty things tomorrow or the day before I was only able to do one. speaker-0: Rob, thank you so much for joining us on Machine Dreams today. Great conversation, great insights, appreciate it. speaker-2: Thank you to everyone watching and listening. More conversations with the people building, securing and shaping what comes next in AI are on the way. See you next time.