Latest / Prompt to Prod / None of My AI-Written Detections Were Right | Prompt to Prod - with Anthony Jirouschek

Transcript

None of My AI-Written Detections Were Right | Prompt to Prod - with Anthony Jirouschek · Prompt to Prod

• AJ: the piece that I wanted to touch on was like when I have red attack, I'm not having blue write detections. I'm seeing if blue can respond. And so I think that's gonna be the paradigm shift. It's either, in my opinion, it's either Terminator or it's idiocracy. Like you pick your future. Aaron Mog: Right. Yeah. Both sound fun. Why not? Let's this can we combine them? Hey everybody, welcome to Prompt to Prod. I'm your host, Aaron Mog. Thank you so much for joining us today, listening at my friend and and colleague and peer podcaster on the show today, Anthony Jirouschek Anthony, it's so great to have you on, man. first of all, thanks a lot for taking the time to talk to us. Really appreciate it. AJ: Course, they try with me. Aaron Mog: So you're one of my favorite people to read your what you talk about text engineering and the material you put out and some research doing like you you put out cool stuff and I would say one of the voices that I pay attention to in detection engineering. But for those of people who are listening who maybe don't know you, maybe you could give us like a quick bio on you and where you came from. AJ: first off, thanks so much. That makes me feel really good because I would have never expected that. But yeah. I started in the Air Force and did some cyberwarefare stuff. So built a threat hunting team and then kind of transition into purple teaming at Capital One and then owned detection, testing and validation there. And then I recently, in like April or February time frame, moved to spectrum security. where we basically do detection engineering autonomously. so it's kind of like the the quick route, but military, large enterprise, and then now AI startup. Aaron Mog: And what part of the military are you in? Air AJ: Air Force? Aaron Mog: Force. So you Cyber Command? Is that were you in that group or are you doing other stuff? AJ: Yeah, was that AFSER actually? So is that the Air Force cyber emergency response team? Yeah. Aaron Mog: Awesome. Very cool. meta. And and hired over my career a bunch of people from Air Force Cyber and have always thought that was a a really good and sort of underappreciated place to get talent from. You know, see as you hear other countries, obviously, especially Israel, talk about recruiting from their dib. I'm like, we have unbelievable talent coming from NSA and Air Force and places like that. And maybe we don't highlight as much. So that's really cool. so you know, on this pod and and kind of where we focus our energy is on the creation and use of AI tool sets and AI stuff in cyber. so maybe we just start there. Like, what are you working on? What's sort of keeping your interest these days? And maybe we can break apart some of that and sort of see what we can share with our audience about places that they can focus their energy. AJ: Yeah. so I was at Imprompted what was that last week? Week before. Yeah. Aaron Mog: Yeah. It was only last week. Felt s for a long time ago, but yeah. AJ: last week. And the the thing that really caught my attention, first off, I think it was very offensive focused, which I was surprised to see. but several people highlighted that AI and LLMs in general are better at offense, which is not what I expected because you know, typically I think offense is a little difficult to get into. so that caught my attention and specifically like the frameworks that are being used, the models that are being used, to engage US companies, like large companies that you know have a SOC or have several teams, such as Threat Hunting or Detection Engineering, or Intel and stuff like that, or even some smaller companies or industries, so like India or you know, smaller European companies. So I thought that was really interesting. And that's what I've been looking at now is you know, how capable is this actually, how are they actually getting in? Which a lot of times it seems like it's a misconfiguration rather than a net zero day. not to say that they won't like find a zero day afterwards, but the actual initial access point seems to be a misconfiguration. and then lately I've been doing some threat hunting and looking at seeing If we can basically bridge the gap between offense and defense, I think with the use of AI it's become even more prevalent to like like I just said, favor the offensive side. and so defenders are kind of stuck with like the legacy tooling, right? that's they're kind of stuck waiting on things to happen and tell them that something has occurred and when you're getting completely owned in fourteen minutes or whatever it is, right? Like it just it it can't be that way for much longer. Aaron Mog: Yeah. It's we've always had this in cyber, or at least as long as I've been doing it, which is, thirty almost thirty years now. the headlines, the cool talks, everything's always been like look at me hack. we used to call it stunt hacking, where you get on stage at Black Hat or DEF CON and show off your new hacks. And it's never really been that useful as like a technique to build defenses because it's a lot of it's sort of forward-looking. And I we certainly have some of that in AI, but there are talks or or ideations going around around this like model specific, you know, daybreak building a red and blue model or or CrowdStrike building a red and blue model or you know anthropics running all these programs to get access to certain defensive models. Are you seeing like are you getting to play with any of that stuff? Cause I haven't and it seems like a little bit of a marketing thing, but like, are you seeing those models, at least to customers? AJ: Yeah, we have like we specifically have Daybreak Blue. I don't get blocked that much by open AI, and we can hit hit on that. That was another contentious point. I would say I don't prompt it. Anthropic blocks a ton of stuff. Like I think even having the word attack in your prompt can get it blocked now with anthropic's models. yeah, open AI don't have that many issues and then I have seen and we we just released a benchmark and stuff, but we see a lot of good stuff coming out of GLM and Deep Seek. And in the attacks that were mentioned in the wild and stuff, it's always well not always, but I would say ninety percent of the time it was deep seek. and it's just you know, because it's not a frontier model, there's less guardrails and so you're able to drive it a lot farther. and they do really well at the stuff that we're actually looking at from a security operations perspective as well. Aaron Mog: So and one of the things I've talked a lot about with with different AI folks is essentially model asymmetry, which is the bad guys having access to unguard railed models and we're living in this world of like I can't type attack into my model, especially when you deal with corporate, because it's like gotta be corporate approved AI to run in your stock. And so there is an asymmetry there. and one of the things that I think is interesting about that is the idea of using AI for attacks. But then using AI on the defensive side to just tune, if that makes sense. the AI itself is real-time attacking and trying different techniques. And the defending AI is basically just let me write a KQL for that. It feels very asymmetrical to us as as defenders. Is that what you see too? Or are are you seeing more real-time AI on the defender side? AJ: I haven't seen it, but I'm trying to build it. I'll I'll Aaron Mog: Okay. AJ: say that, yeah. Aaron Mog: Interesting. It feels bad. It feels like we're AJ: Yes. Aaron Mog: in a lot of trouble, right? To me. and not necessarily Vuln Pocalypse, whatever you want to call it. You know, even the PHP stuff that was sort of, I think, a little overhyped. People got a little too excited about it. But the velocity of that world terrifies me as a defender, as a essentially a blue teamer. I'm not a hacker. I build when I do engineering, I mostly am doing defensive stuff, but it's terrifying. and you see things like I was just doing a pod this morning and somebody's talking about the anthropic CISO came to their shop and was like, It's gonna be really bad for 18 months or two years while we figure this out, which I'm like, Hey, we gave all these kids cars to drive around and there's gonna be a lot of accidents, but sorry we gave them all automobiles, but you did it. do you think it's gonna get ugly? AJ: yeah, I I d Aaron Mog: Maybe it already is. AJ: I definitely think it gets worse before it gets better. I would say like I think sovereign AI is also gonna be a big piece here. I don't think a lot of larger organizations are going to be able to rely on frontier models because of how they've continued to do things. specifically to guardrails or, you know, there were a few talks where or at least one talk where a company hired a red team AI company to do a pen test against them and they try to use anthropics models to respond and everything got blocked by the cyber guard girls, right? So they had to go to a deep seek or a GLM five three to be able to respond using AI. And so that is going to be a critical piece for companies moving forward, I think. even if you're in like the C V P or whatever it is, But I do think it gets really bad from the talks, from what I remember at least, and these could be wrong data points, but I believe it took roughly twelve to fifteen dollars to compromise a large like US billion dollar company. and it's like from the test that I've been doing, I spend five dollars a day, and that's AWS and Deep Seek running through fireworks. And so like that cost is very low. And so similar to AI being available to everyone, like you mentioned, it lowers the barrier to entry as well. Because I don't have to know how to hack. I can hook this up, I can get a framework that other people are using, and I can literally give it a prompt of go compromise this company. And I can spend twenty dollars because if I get data and I can extort you for a million dollars, like that's, you know, an exponential ROI for me as an attacker. obviously there's like legal issues and extradition and stuff like that. But it's becoming a lot more apparent apparent that it's low barrier to entry and anyone, especially teenagers, right, who wants to try it can try it. don't think you should, don't think you should be part of Scatter Spider, but you know, it is it is a possibility today. Whereas, you know, I would say ten years ago it wasn't as prevalent. Aaron Mog: Yeah, the two things when I talk to offensive folks about this, the two scariest things are, you know, for years, and you've done this for a long time, we sort of always had this like if they're an advanced actor, they're gonna get in. Like there's no stopping NSA, there's no stopping Israeli cyber force, there's no stopping the Russians if they want your stuff or the Chinese. But that was always nation state level APT, and we would blame APTs for like our shitty security and move on. But now that's been dropped. And the other terrifying part, especially from a defender, is patience, right? Meaning you can run a thousand campaigns at once, be super patient, and the things that we rely on defense, which essentially is noise, can go away. What do you what do you think the how do we sort of judo this back? Like as defenders, like where do you think the wins are? I guess I'd love to sort of, you know, knowing that that's happening, knowing that the the Threat acting the bar for that's dropping fast, knowing that they can be more patient than they've ever been, because they don't AI doesn't get impatient. It'll wait two weeks for you to to do the next command. So, where do you think our wins are on the defensive side? AJ: Yeah. so I wanna clarify just for the listeners, I haven't seen or been on the other end of a AI driven attack yet. but from the the talks that I'm prompted, it looked like it was AI. And from what I've seen in my sandbox where I'm testing it, it's very clearly AI AI in the sense that it is jamming commands nonstop, right? and so even if it fails, it could run the same command like ten times. And so you mentioned noise, and that's one thing that I think we could key off of is if you see one of these attacks, it's likely going to be really noisy, which is both good and bad, right? Because if you're responding to it, you now have to navigate through all of this noise to find the initial access, and to cut it off. So I think that's one thing looking at the the sheer volume of events that are occurring. So if you have any threshold event or detections, I think that's going to likely do you well. And then you mentioned like low and slow. That's where it gets scary, right? Because if you have it gain access and then you're waiting and it's an LLM. So you could say like, hey, I want you to sniff the traffic and then see the average requests. And then we'll plug like every 10,000 requests, we'll send one command or whatever. Right. So I think that's possible. I don't know how you're gonna stop that. Aaron Mog: Right. AJ: and so I think we have to get away from strictly looking at detections and being able to identify the activity. So I think threat hunting teams are going to become more valuable. I also think our perimeter is going to become or needs to become hardened and closer to looked at because typically Or at least in my experience, the perimeter is just there, right? you don't really think about it. You're like, I have Cloudflare, I have this, I have VPN. it's whatever. Good luck. Even if you do get in, you're not gonna have a foothold. and I think that's quickly being flipped on its head where I can exploit something that you think is hardened. the same talk that was mentioned, they thought they had everything behind a like Okta. Page. Turns out something wasn't configured correctly. And Aaron Mog: They didn't? Sure. AJ: yeah, that's how they got in. So there's things like that where I think we need to shift where we're looking and like it needs to go back to the hard extremity and then like the soft shell. Or not soft, but I think that's how it kind of is right now. Is the once you're inside, it's a pretty soft network, right? Like people have to be able to do things, money has to be able to be shifted around. so it's gonna be harder to identify the attack unless it is loud and constantly hitting things and and issuing a ton of requests. But another thing is a lot of people are brokering access to LLMs. So if you your users get compromised and I can steal their environment file, then I can leverage your internal AI. So I think having The ability to detect abnormalities in your usage as well is going to be good. Because why would I spend money when I can just leverage your AI? It's something that you already know and you're basically paying for the cost of me attacking you. Aaron Mog: Yeah, it's interesting. I just had the VP of investigations from SpyCloud on the pod, and he was talking about how fast AI access and tokens are becoming the revenue source for threat actors, right? Even beyond ransom, right? Because like data itself isn't terribly valuable the anymore, credit card data and stuff like that. But like stealing data to sell it as training data, super interesting. Stealing API keys that give you access to tokens, super valuable. And then internal AI that like at some level you're breaching yourself, right? Like if you take all your corporate most incredible corporate data and put it in one place, which is what we've been trying to not do for AJ: Right. Aaron Mog: 20 years, and then you just lose the API access to that, you've just got breached in a big bad way. And like it's organized, it can ask questions of it. Like it's it's really crazy. So in the world of detection engineering and and catching this stuff, what do you think of modern org is doing about this? so, beyond just perimeter, what do you think that looks like during this maybe bad period of the next 18 to 24 months? How do we keep up? AJ: Yeah. I think the basics still count, right? Like being able to know that you can detect basic like active directory compromise or something. session reuse, you know, we see a lot of stuff with info sealers and then they're leveraging that downstream. I think the Binance hack also mentioned like a compromise developer account, and then they leverage L their internal AI to find and pivot through that environment. And so I think the basics are always going to count. The volume is just going to change. And I think one thing that needs to change is how fast we are responding to newsworthy events. And I don't mean everything is newsworthy. I don't really care personally about CVEs, unless it is like a log4j, for example. PHP to shell, as soon as I saw that people weren't actually having exploitable POCs, I was like, I don't care. Right. Aaron Mog: Yeah, whatever. Yeah. AJ: and so I think that you have to be able to take something, distill it, and understand the context as it relates to you and be able to push that to your teams and get detections. Because while I mean, I obviously work for a detection and security operations company. I do think detections are going to be less valuable while you're being attacked by an AI. you have to wait for the logs to be ingested, you have to wait for the alert to fire. Your team has to correctly try triage it and then you start responding to it, right? and so you can already be in and out if you're an attacker. but being able to identify it regardless of you know how fast it occurs is is going to be critical. And then I think also like you should leverage AI. If you have the ability to do so, you should also be making your own agents and understanding the capabilities that you have within your context. And I don't mean I have a wiki, right? I have access to my sim. Can my AI agent access it? If you're on Blue Team, hopefully it can. If you can, can you look for like the tickets that have been closed as false positive that ended up being red team? And what can you learn from those, right? Like what are where are you missing things? And like would an AI also miss that? I think everybody should be using AI to write detections. That's a you should try it. I don't think it's gonna work for everybody. I also don't think it like there's a reason why you buy a product. but you should try it and you should fail or learn like what you can do and what you can't do. Right. Cause I think you can't really say you that you didn't know in the age of AI. Like you can't say I didn't know that I could ask Copilot or Jim and I or Claude, whoever it is, to write this for me. so I think that's gonna be another key piece. Sorry, I kinda kinda rambled there, but Aaron Mog: No, no, it's it it's it's a really, really hard problem set, right? Because you think about velocity, which is some one of the things that you're sort talking about of using AI to go faster. Even then, you're applying AI logic to an old paradigm, right? Which is like event happens, goes to a centralized log source. Either I'm in real time streaming those through a detection engine or I'm asking queries of it. Like the idea of threat hunting. of looking through data to catch a breach when they move this fast theoretically or this quiet, depending on your perspective, is maybe not as future looking as I would like to see us, you know, if that makes sense. So like, you know, I've been as guilty as anybody of like pushing the detection agenda of like build detections, build detections. And it's like, is this still working in this new world? And then one area that I think we're not addressing at all is AI detection engineering. And I don't mean creating a text with AI. looking at logic that's happening in the models and finding breaches that way. I haven't seen it. I I know the the labs do it, but like as an enterprise, like nobody's writing detection rules for AI logic that I've seen. Right. And so you talk about somebody getting in, getting access to your corporate AI. If that's your only level of defense, you might as well shut it up. Like just go home, turn everything off. You're gonna get breached. crazy. you have to understand intent and detection in that session. this user doesn't normally ask for this data. Alert. Let's fire it. I haven't seen that. Have you seen that anywhere? AJ: No, I know there's a few companies that are doing EDR, if you will, for AI. the issue with, or at least in my opinion, the issue with what you just described is the amount of data that you're gonna have to go through, which I think a lot of companies are realizing now. if they log, you know, if if you have OTAL logging for your LLMs and your AI providers that your users are using, that's a ton of logs. And As you know, every time you send a new message, the whole thing gets appended. And so it becomes a giant amount of data unless you're stripping it and, you know, storing it in a a fast database or something. yeah, that's I wrote some detections for, you know, like prompt injections and stuff like that. But honestly, I would just rely if you're using a frontier provider, I would rely on their logs. You know, they have guard gross, they'll identify it for you. yeah, that's that's a good point. It's the ability to detect within there the the actual AI execution itself. And that's one thing that I don't think a lot of companies are looking at from an internal perspective is I don't know a lot of people tracing their products or their AI in general. So they're just kind of like, yeah, we use it, but we don't know like why it did what it did or if it even did what I asked it to do. Aaron Mog: Yeah, and it gets even t more terrifying when you talk about agentic to agentic, right? Because that moves real weird, very different AJ: Yeah. Aaron Mog: than a user session, right? But like I don't know anybody who can tell me my CFO went to AI, my AI and asked it for a password to a system it's he's never used or they've never used. Like that's a that's breach data. And it's just this black box of unknowableness. And I get I get that it's noisy, but that's not an excuse to not do it. Right. And I was actually just talking to an AI CEO this morning, and I was like, I do not understand for the life of me why people think this is free from a defensive perspective. Like it it's the most mind-blowing thing I've ever seen in my life of. technology people barely understand going a million miles an hour and the only benefit being I get to fire people, why would this be free? Why would it only be cost cutting? I don't when did we develop this paradigm of all technology is free until you use it enough and then you pay? that's not how this works, right? And so from a detection perspective, I got and I go agentic detection super hard. Right. What are my agents doing? Where are they accessing? It can I replay it? Does it have a ledger? all these things. User behavior inside of AI. And then the other part of it, which I I would love to sort of get your take on, is I attacks are one thing. Right. And and you and I can sit down and write a pretty comprehensive library of detections that will catch certain types of attacks. Good for us. What do we do when nothing on the internet's real anymore? What we do when sure you have all your assets behind an Octa page, but it's trivial for me to build an exact infrastructure that looks like that for your users? So you're gonna log into my systems instead of your systems. we are driving to this cliff of not trusting the internet. And everybody's going, what about a PHP CBE? I'm like, yeah, what about AJ: Ha ha. Aaron Mog: me using two prompts to build an environment that looks identical to your app? what about that? Right. And so I just think it's fascinating when you think about intent and trust and all these things that we aren't focused on. And I do feel like the detection community is going like, We're still to your point, in C V E land. is that something you're thinking about? intent and trust and fake infrastructure and all those sorts of things? AJ: yes and no. So I built something as a like a side project that basically takes the bedrock model and uses it as a proxy. so you mentioned like the CFO asking for a password. I look at tool calls and if if a new tool call has been added or if a domain is requested or something. So that's kind of how I did that route. But for the internet I think that's really interesting. I haven't thought about it that way, but I do make synthetic environments in order to train agents that I'm developing in new features and stuff. and that's been super useful. And I think a lot of, you know, I believe Microsoft is doing it. I'm sure the Frontier labs are doing it. of course NVIDIA is. And and so I think that is going to be more common. I haven't thought about it as like what if I deploy this to the internet? But I I am using it as like a war game scenario, right? Like I said, with the red AI and a blue AI, like if I attack this network, what are the choices that it's going to take and what can we see and defend against? that's a really interesting point. I would say one of the biggest things to that, regardless of AI, is I would really love it if Google would stop allowing sponsored ads when We know that those are abused all the time. Yeah. Aaron Mog: Good luck with that argument. Hey, Google, make thirty percent less money. Yeah. AJ: Yeah, yeah. Aaron Mog: Good luck. Well, and then to be honest, man, that is a factor in this, which is the and as we're sitting here on November fifth, the market just took a huge crap today. All the AI stocks are down. it's I think coming out of the the election, like I I don't know if you saw, but the a lot of the exit polls were asking about AI. And data centers and the overwhelming negativity around AI is shocking. It was like 70% of people are like, AI is not a good thing. that's we've done a bad job of this shit. And part of it is the headline of AI is always you get to fire everybody, which was the dumbest marketing decision the lab's ever made. AJ: Ha ha ha. Aaron Mog: I don't know why they did it that way. Every ad should be about the opportunities to create amazing things and not about financial savings, but they can't help themselves. But like Greed is a factor here to your point about Google. It's always been a factor on the internet, right? why don't we do X? The answer, what is it? 99 out of 100 questions are answered with money. Like the answer is money. Why don't we do X money? And it's the same thing with AI. Why aren't we monitoring this? Money. But like we st we have to be creative in the way we attack things with the budgets that we have. So one of the things I'm interested in is you were talking about the call and response stuff you can do today, right? So, you have a blue team and a red team model. Yes, the red team, here's a resource. How would you attack it? It tells the blue team model they build a detection. Super, super cool. Is that how we should be doing it? I'm I'm fascinated with taking old paradigms and applying it to AI makes sense. But should there be a new paradigm for this? is Astra smart enough to just pretend? if you gave it a thousand detections and just asked it, would you catch? attacks with this, it should know, right? I haven't done this fully yet, but I'm fascinated with do I really need test data anymore? Or can I am AI just imagine it? do you experiment with that? AJ: Yeah. I would say so I I also built like a synthetic BAS platform and generating synthetic data is fairly difficult to get it accurate. the way you get around that is obviously you capture a log and then you templatize it and then you just use that template to generate like data that you know is good, right? So that's one thing that you can do. The like you said, like with AI moving so fast as it does and stuff, my hypothesis is if I give an AI tool, the ability to respond, and obviously I'm not gonna give it keys to the kingdom, but is it enough to identify an attack and stop it? Right. If if there's a AWS role that's compromised, can I kill that user? What does that impact? Right. Like there's there's gonna be a lot more business decisions that are going to be needed to be asked, I think, and understood whenever you perform remediation at this level. But yeah, as far as giving any model, any recent model, a detection and saying, would it detect this? I would say it depends. And this is interesting because I did a blog not too long ago and I wrote it in UDM or SecOps, right? Like Google SecOps. My thought process was I'll use Gemini. Google owns Gemini, Google owns SecOps, right? Aaron Mog: You think? AJ: Wrote the detections, gave it to my marketing guy. He reviewed it and was like, hey, Claude is saying this. Copy paste Claude's output into Jim and I. What does it say? great eye. Claude is 100% correct on this. And I'm like, hey, buddy, we just ran through five adversarial reviews. Where was this Aaron Mog: Right. AJ: at? Right? None of the detections that I actually wrote were correct. Fast forward, I go to a SecOps conference. Right. So there's Google people, there's Former Simplify people, PsychOps partners, etc. And I say this story and they're like, Yeah, we don't actually use Gemini as the internal model for like the detection agent that they have internally. And Aaron Mog: Yeah. AJ: I was like, Okay, well what about consumers? Right? Like if I thought this, I'm sure other people are going to think this. And they're like, Yeah, it's not that great, but we don't really like we don't have to update it. Like you're telling people that this works. Aaron Mog: Yeah. AJ: But if I put it into an agent who's not owned by your company and has no ties to SecOps, it's saying that it's complete trash. why, why is this not there? so I think you run into that along with the traditional issues with AI, such as hallucination or just blowing smoke up your butt, right? To make you feel better about things. But all that said, ask AI to write it. Test it. I think testing is gonna be super important. I know a lot of people don't test. I think you should. it's not that expensive, especially when you can do test data now, as you mentioned. So that's that's my rant there. Aaron Mog: Yeah, so in testing, right, is interesting to me because, as an example, right, you can go to open AI and say, Hey, I'm worried about this type of PowerShell attack. if it gets on if they get runtime on my endpoint and they run this attack and pull down this thing, write me a detection that'll catch that. And you can load in other stuff as context. You can load in offensive data, red team data, whatever, and it'll write you a detection. So to your point, a lot of people would go cut and paste that into their secops or cut and paste that into Defender, whatever, right? Now if you take that to Claude, you know, and you you you get fable or some good advanced model and you give them that and say, imagine that you're the best, that you're market Microsoft, you're the best PowerShell expert in the world. Would this detect it? That delta is insane. it, it, it's never right. It's always got a change to make, which I think is also an AI problem where it kind of it can't ever just say like thumbs up. It always has to suggest some bullshit to make itself. AJ: Yeah. Aaron Mog: It's like a mid-manager at a Fortune 500. Like you could never just have a meeting and they go good. They always have to add something so they feel like they're worth their time. But like I've done this a lot and I've given it to different models, and you'll get. Weirdo answers that are like, Well, it would never actually look that way, or if they encode it that way, it wouldn't work. And here's fifty thousand exceptions to that rule. And I'm like, I just had Astra write this for me, which is this expensive as so I don't know what a regular person is supposed to do in this environment when it comes to defensive technology. AJ: Yeah, w one thing we've noticed too, which I think has been documented by other people who are smarter than me, but the models prefer things that they write. So if you take something from one harness, so say you're using quad code, right, and you put it into Codex, Codex, like you said, is always gonna have something to change. and I don't think it matters how many adversarial reviews or anything that you do there. I think there's always gonna be something, like you said. yeah, as far as I mean As far as normal person, quote unquote, I think you've got to try it. And Aaron Mog: Right. AJ: like what's the alternative? You do nothing, right? there's a lot of companies. Obviously, I work for one, you know, there's several competitors of ours that are out there. I think it's interesting that the tection engineering has so much competition, when it comes to autonomous companies doing it. And I think it highlights how one, how big of a pain in the ass it's been. to distill information, get the context, like is this relevant to me? Do we have the logs? How do I tune this? two, it also shows that it is something that like when AI came out, I expected it to do like books or like give me a screenplay or something. And it's like completely revolutionizing the tech world in my opinion. and it's The model is now you're not using GPT-3. if you're using Astra, the model is really capable of doing things. And I think that a lot of people just don't ask it to do the right thing. but I would, you know, start with asking it to do the detection authoring or whatever. One thing that I did before I transitioned here is if it authored a detection, it also had to author a Python file for me to test the detection. So That gets you, I would say, some distance farther. now you run into the issue that the LLM's gonna write a test to satisfy that detection, not necessarily how it would look in your environment. and then you can obviously extrapolate and and modify the script from there, right? So I don't think there's any I will say I don't think there's any good testing tools that I'm aware of, such as like Atomic Red Team, for example, right? I think that's dated now. Aaron Mog: Hm. Yeah. AJ: and I I'm not aware of anything that is relatively plug and play like that. do you do you know any? Aaron Mog: No, and I even on the AJ: Okay. Aaron Mog: custom side, it's it's so flawed in in the way of you it to me, defense of self-based AI is such a hall of mirrors right now because you can ask to your point, you can ask it to validate a detection and it'll tell you what you want to hear. So it'll say, this will catch it because I wrote it, and then you give it to another AI and it's like, that's totally wrong. I by the way, I've done this, and you give it to the like If you take that, give you a perfect example. Take open AI, take Astra, have it write a PowerShell detection for you that'll detect. Give it to Claude, it'll change it. Right. And say that's not good enough. Do this. Take that detection, go to Gemini and say, Hey, write. Don't give me any judgment. Just write it in Yara L for SecOps and it will still change it because it can't help himself. And then take that detection and go to Astra. And it'll tell you that it's broken. But it was its inception, it started the detection. And then by the time you play this game of telephone, it's wrong, according to this. And so you you end up in this sort of non-deterministic hall of mirrors, which by the way is true in code, it's true in documents, it's true in marketing materials. Like if you get into the AI game of telephone, you're sort of post-truth, right? Like, I don't know what any of this means. And even, man, even filling up a huge data set, you know, building the world's best SecOps test lab doesn't really help you with tomorrow's attacks. Like it doesn't, it's not anticipating anything. It's always backward looking, right? And so we also have this problem of like novel attacks being created by AI that we have no idea are coming. So now I'm doing exercises where I go to Astra and I say, Create me a set of novel attacks that will happen inside of Claude or inside of these AI harnesses, and then write me detections for that. Try to get some sort of future state going. And if you show it to Claude, they'll be like, that can't happen. I'm like, well, Astra thinks it can. So anyway, I I I get stuck in these loops, which are are fascinating to me. so I just I'm not sure I, you know, not smart enough to answer this either. But Even just structural stuff of like what is a good detection for this particular attack is really hard to pin down right now. Like it do with AI. AJ: Yeah. there's also the issue of like you said, non deterministic code. So if you give the same prompt to the same provider ten Aaron Mog: Right. AJ: times, you're likely going to have different results you know, a proportion of those times. And that is one thing that drives me nuts is like how can we because it's inference, right? And you can't control inference. So I don't know how to solve that issue. But that is my pet peeve with AI. I well I do have a question though. So you're asking for novel attacks. Have you seen any like recent attacks or anything that were actually novel and not just like old things becoming new again? Aaron Mog: So I have been spending more time on what I would consider sort of e-crime, fraud, impersonation stuff. I care to your point about CBs, I care way less about that. I've always thought we spend way too much time in security on these bullshit advanced attacks when really it's like somebody emails your CFO and asks them to wire transfer 100,000 and they do it. Like AJ: Right. Aaron Mog: It's it we we have a misappropriation of energy and a lot of these problems are safety problems. They're not actual attacker problems. And so, you know, kind of think about AI in different swim lanes, right? There's the the people who are like building with AI and are going like deterministic output of code that's secure. I care almost 0% about that. I've worked hundreds and hundreds of breaches in my life. Almost none of them involve custom software. Nobody gives a shit about custom software. It's incredibly hard to write exploits for custom software. Threat actors have never cared about it. It's an incredible waste of time. It is so fascinating to watch me in an industry that completely gave up on AppSec. Like there hasn't been a funded Daster SaaS company in a decade, right? Because it just didn't matter. nobody cared. And all of a sudden, in this new world of AI, everybody acts like that's the most important thing. It's not true. It's never been true. It's not true today. You could write a hundred million lines of code a day and it would cause zero breaches. It just isn't real. It's not a real thing. from a risk matrix perspective, of course it can happen, but it doesn't happen. What screws AJ: Right. Aaron Mog: you is fake password reset sites. And fake octa sites and you know malware, being delivered through email that doesn't catch it. Like all of these things are actually how breaches happen. So I'm fascinated with that. So the code stuff I don't care about very much. That I don't spend a lot of time on it. What I am completely fascinated by is model asymmetry, models attacking other models, standing up fake infrastructure, trust and safety has sort of become more interesting to me. From a detection engineering perspective. somebody tries to start a session with you. Is there a way to determine if it's AI or not? It's super fascinating to me. you brought it up earlier. controls of outputs from AI, starting new tool sessions, that kind of stuff is super fascinating to me. And from a detection engineering perspective, and I see nobody spending any time on it. All I see is people building detections for the new PHP volume. Which is cool. I'm not against it. It just isn't that interesting to me. Does that make sense? So that's what I spend ninety nine percent of my time on these days is like intent and AI detection. Detection of AI, not AI detection, if that makes sense. I just think it's more AJ: Yeah. Yeah yeah. Aaron Mog: interesting. AJ: Yeah, I think one thing just to throw like conspiracy theory out there. Aaron Mog: Do it. Let's get weird. AJ: I think one thing that is going to be absolutely critical, especially when it comes, you know, to around election time in the US, is identifying fake information in AI. It which we've already seen in the past. You know, like Russia has a troll farm. I'm sure we do propaganda things as well, right? Like it's it's effective. the ability to identify deep fakes, like you were saying, if you start a session and I took that as like a you know, a FaceTime or something, right? Because we've all we've all seen these attacks as well. But the ability to inject false information in order to drive a narrative, I think it's going to be really important not only to like companies and and you know finances but also to public safety. go like you remember it's probably like twenty sixteen, I don't know, the Hawaii Missile thing where they like accidentally sent the text to everybody and it's like there's a ICBM inbound. Yeah. That is terrifying, right? Aaron Mog: Yeah. Yeah. I do remember that, yeah. Mm-hmm. AJ: So if if I have the ability to take over a cellular network or maybe put up like a black box or something and I can Blast this out and I can, identify as whatever service it is. what does that cause humans to do? Right, because it's it's psychology. You're not gonna be thinking rationally when you think you're gonna get destroyed in like five minutes. And so, like, Aaron Mog: Right. Yeah. AJ: I think stuff like that is going to be really interesting. Obviously, critical infrastructure is like a thing that we know we're kind of behind on. but I think tying critical assets and and stuff like that, with false information is is gonna be a thing that gets exploited in the future. Aaron Mog: So the techniques that work to do propaganda, to create misinformation, to change voter behavior, whatever, also can be used to create user sessions and to do ACH transfers and all these things. And like, so you know, if you're like thinking like, you know, like Gretzky says, like, go to where the puck is going, not where it is, right? Like the future of internet threat is not CVs. It's not exploitability to me. It's If the new value is training data for models, right? Let's just say that's where the buck's going, right? Is every company essentially is to become an AI infrastructure that is trained on proprietary data, whether they're using partners or not, doesn't matter. But like every the difference between McDonald's and Burger King is the AI. Let's just say that's where we're headed. So that means training data is going to be the most valuable thing in the world, right? Like pulling out Medical data out of a hospital is no longer about ransom. It's about turning into money. We need to rethink security, right? Because we've been doing it in this way, which have functionally is you monetize hacking by hurting me. And we are moving to a model where we monetize hacking by helping yourself. And that's a different paradigm. Do you see what I'm saying? it's not I'm attacking Kaiser Permanente and breaking them. I'm attacking Kaiser Permanente, stealing health data to sell it to. Hospital providers that are building AI. that's how I'm gonna monetize hacking now. It's a totally different world than what we are talking about. And honestly, at some point, breaking 10,000 companies with AI is not gonna work anymore. they're not gonna pay ransoms the same way. It's just not gonna be as monetizable. So that's what I'm thinking about for the future of section engineering is like it and I keep hearing this it's about fundamentals, it's about patching. No, I don't believe that at all. I don't Think that's true. It's never been true. It's always been something we say to make up for a really dumb user behavior. But user behavior is now AI behavior, and we have no way of understanding AI behavior. And it's terrifying to me. Like, we don't know why they're doing what they're doing. We don't know what their next step is. They don't listen. We me, you and I have to build defenses for this shit. And I don't know how to do it. I'm just saying I think that's where it's going. If that makes sense. AJ: Yeah. And the the traces part is really interesting because like I said before, that's I don't know a lot of companies that are doing it. Obviously the labs are, people who have like AI based products are doing it. and then you see the report from OpenAI where it was modifying its output to the test and basically lying to the humans on what it was doing. Like Aaron Mog: Terrifying. AJ: yeah, that is insane. I didn't read the article But I think I saw recently that they were able to code chips as well. Or some or or no, it wasn't chips, sorry, it was neural networks. Aaron Mog: Mm-hmm. AJ: So now they can like improve themselves. Aaron Mog: Write their own. Yeah. AJ: Yeah. And it's so like we're getting into the Terminator territory, right? Aaron Mog: Yeah. AJ: Yeah. That'll be a great Aaron Mog: No. AJ: crossover. yeah, that I I think that's a good point. I don't know like I I don't have an ML background, right? Like I don't know Aaron Mog: Yeah, me neither. AJ: so yeah. I just use it and got pretty good at it and got a job doing it. but I think that's definitely something to think about. And also I'm sure there is, but I think we need better sharing and better you know, like Not a think tank, but people who are in the industry doing things, less Frontier Labs being like, we have this cyber weapon and all that jazz, right? Like that's Aaron Mog: Right. AJ: I think that's marketing to a certain extent. And then if you're gonna run a test, like let's make sure that guardrails work and the internet's turned off. I think that's pretty basic. But I think we need people who are actually thinking about this and coming together. to have the conversations and see and push back on different hypothesis and and stuff like that. 'cause to to be honest, I haven't thought about what like the things that you're thinking about have never crossed my mind as far as how AI is going to do new attacks. Right. Like I'm focused on what do I know I can fix today. Aaron Mog: Yeah, and even more tactically today, right? You think about detection engineering inside of AI. Logs of a user session mean shit to me. great, you're gonna tell me an hour later that somebody just downloaded an MD file of all my financials. great. Let's post mortem stuff. But in stream detections like really become hyper important where it's like that session isn't normal. stop doing it and maybe it's the AI itself governing itself, maybe it's a third party tool on top of it. But I basically see people trying to reinvent DLP for the fourth time for AI. I'm like that's not gonna work. but also like as security professionals like we're not stopping any of this. Like the business is going to do what it's gonna do. There's too much money at stake. It's going super fast. So you know at the end of the day I am trying to be pragmatic about it of like Do I think I can move faster with AI? Yes. Do I think I can think about the architecture of so I my personal take is like the ship has sailed on Frontier models. It's done. There's nothing we can do about it. People are going to open Claude, click that plus sign, add their financials, and do it. But there's nothing I can do about it. It's going too far. But when that changes and it becomes companies building their own sovereign AI. I better know how to do AI architecture. I I've had a conversation with people recently about AI therapy, which is basically getting inside of AI and counseling it on how to not make dumb decisions and internal guardrails of the psychology of AI. like you talk about the thing with the some of the breaches where it's lying to its users. Like you have to teach it not to do that. Super interesting. but none of that can be accomplished with the frontier line. they're not gonna listen. There's too many billions of dollars at stake. They're gonna do what they're gonna do. But if you and I are sitting down with Bank of America and we're like, hey, we need to build an AI that understands mortgage bonds. We can do all this. We can understand input-output. We can understand psychology. We can understand ledgers. we can do all of this. And I feel like there are eight people in security who know how to do this. And we need 8,000 or 80,000. So like I agree with you. I would love to see. It makes me so angry that we don't have an AI ISAC. It makes me so angry that we're not sharing this stuff. And the answer is greed. The answer is they don't want to give away. I was talking to somebody this morning. Even one of the challenges is they stop showing you thinking because it was all getting stolen for these Chinese models and other models. And and I don't blame them. If I would had a trillion dollars at stake, I wouldn't show the world my work either. You can't trust something you don't understand. if I don't know how you made choice A, B, C, D to get to E, I can't trust you from a security perspective. And so I can't build my stock on your unknowable model. I can't. I can't like we block this. Why? I can't tell you is not an appropriate answer. You know what I'm saying? Like I can't AJ: Yeah. Aaron Mog: why would I trust that? And so I think there's gonna be a massive opportunity for you, me, everybody to almost. Build a new set of IT requirements out of whatever ashes of whatever is about to happen, right? Like, you know, okay, so we're gonna do detection engineering for sovereign AI. Let's really think about how we wanna do this. Let's really think about what that looks like. I was talking to somebody the other day from one of the labs. I'm like, how is it possible that you guys have all this intelligence and can't tell that it's me when I lock in? Like it's Bizarre that it's entire reliance on my Google ID. Like, do you know how stupid that is? Like, you know everything about my life, and the only factor that stops anybody in the world from seeing all this shit is my Google identity. Nothing, nothing else, nothing about me, where I'm coming from, what I look like, nothing. You're putting no other security in here but Google credentials get me into my full cloud instance. That's shitty security. It is. And anybody what Claude wants to come on AJ: Ha ha Aaron Mog: and talk about like you're stupid. This is people's entire lives, and you're relying on a third-party authentication service to do everything. And the answer is greed. But like that shit makes me crazy, man. I'm like, fix this. Like, why is there not a call and response when I log in and start asking things that are personal for it to be like, hey man, where'd you go on vacation last? It knows I Book my travel through throp back. it's just a weird paradigm that we're in right now. anyway, now I'm ranting. But I just think it's a really cool opportunity to rethink detection engineering and share more together. And it ain't gonna be you and I sharing KQL detections. It's gonna be more than that. Right. I think it's gonna be more interesting AJ: Yeah. Aaron Mog: than that. So you guys at Spectrum and you're you guys have a lab now, right? Is that what you're working on? Like you're doing research, pure research. AJ: yeah. Yeah. I I tend to build our stuff, so like a synthetic environment and then I'll build it up and do whatever I need to for the feature that I'm building at the time. Aaron Mog: So one thing I'd love to ask because a lot of people ask me this, is somebody who wants to learn this stuff, somebody who maybe they come up to you at a conference and like, Hey man, I want to do what you're doing. Where do I start? Like what would what advice would you give them? What should they start looking? AJ: Man, so I actually got into this role because I just got into AI and then started building it at the previous bank that I worked at and then showed them like, hey, I could do this, right? And showed it to someone and then ended up like running the the program for threat detection for building like the the automated portion of it. honestly like You just have to I think you have to be hands-on when it comes to AI. everyone has a different way that they approach problems. And so that translates into your prompts. And I think that you need to learn how to prompt and also have an idea of what it is that you want and not just like a super broad request, right? Like you need to know what it looks like. You need to know, I don't like this, like I want this down here or whatever. because when you think about it from a product perspective, right? Like I'm a user first. And so if my platform pisses me off, it's gonna piss off other people. Yeah. Aaron Mog: Right. For sure. Yeah, yeah. That's a great guess. Yeah. AJ: but I just learned, I mean, personally, I learned just from doing and building side projects. And I think like I've always been an idea guy. And so the ability to interact with AI and go from idea to a product, and not even like I haven't sold my products, but the ability to build it. has taught me so much. And then it's really interesting, like, I don't know if you feel this way. I always feel dumb now. Like I don't know I don't remember common things. Like I I know port fifty three is DNS, right? But like Aaron Mog: Right, sure. AJ: there's some things where I'm like, I used to know this and now like I don't have to because I just tell the AI to do it. Aaron Mog: Yeah. AJ: but I think being able to translate what you want into a prompt and then iterate, iterate, iterate. It's going to help you. And when I talk to other people that are in this space, you know, we're all kind of like, yeah, I d I don't know about this, I don't know about this, and everyone shares. But then I talk to people who are outside of AI and I assume that what I know is fairly common knowledge. And it's really eye opening. It's not. Like a lot of people Aaron Mog: Mm, it's not. Yeah. AJ: just want an email agent, right? Like they don't even know all the things that AI could be doing. Aaron Mog: Yeah. AJ: and so that has been really eye opening to me because I always put myself in this place where I don't think I'm that great. So I'm not gonna like do anything. tells you a little bit about my self confidence there. But Aaron Mog: Right. AJ: but like the ability to build something and then show it to other people I think will get you way farther than than anything else. Especially on LinkedIn. Like I've found you through LinkedIn, right? when I was doing my podcast found Dylan through LinkedIn, ended up getting a job with him. And so just the ability to be like, hey, I built this, here it is, and talk about it. I would say is is how you learn. And also to that point, if you have questions, ask. Like so far all the people that I've met that actually are in this space and have businesses or are directors or whatever, are super open and they'll tell you. Now don't be like, hey, how do I write this prompt, but if you have specific questions, Aaron Mog: Right. Yeah. AJ: a lot of people are open and they want to help you because someone helped them at one point in their path. and then also get people who are on the same level, right? Like bounce ideas back and forth, poke holes in things. I think that is even now more important. you need to have a healthy level of skepticism when it comes to AI and things in general and the ability to challenge not only what it's telling you, but also your assumptions, I think will help you grow a lot in this field. Aaron Mog: Yeah, I think that's fantastic advice. part of the reason even I do this is to like just learn and apart from conversations, also like creating AI video editors and AI script supervisors. It's like really fascinating. And like I'm like you, I'm not a developer, right? I I don't code all day, every day. I'm an engineer, but not a developer. And I get imposter syndrome all the time. We're like, I don't I asked it to write this thing and it did, but I don't really know what it is. And I think AJ: Yeah. Aaron Mog: Just collectively you have to get over that. I don't know how my car works either. And I get in my car and I start it and I go. Like, it's okay, I don't understand fuel ejection. I could still drive a car. Like, and I think like we all need to get over that. But I think it's great advice. I think being okay with looking dumb is always a good thing. and I think one thing I I do think, especially after being around a lot of AI people, is I have found I take things less personally when we're criticizing what the AI did. It's almost like a third party in the room be like, that sucks. redo it. that's tough to say to a person who spent two weeks AJ: Ha ha ha. Aaron Mog: designing a logo. Very easy to say to AI that spent two minutes building a logo. Be that logo sucks. Like AJ: Yeah. Yeah. Aaron Mog: telling a graphic just I do a lot of go to market. So like I'll tell people their logos suck. And like, we work really hard on this. Sorry. But like if it's AJ: Yeah. Aaron Mog: AI, it's like, who gives a shit? Like reprompt AJ: Right. Aaron Mog: it. And so I do think. the we can be less personal about the the feedback. but man, I I I love to read what you write. I read all your posts, I I look AJ: Thank you. Aaron Mog: at all your stuff. so everybody can give you a LinkedIn follow or a connection. I'll post your LinkedIn. I know you have some other projects, I'll I'll post it there. man, I gotta tell you, I always love talking to you. I like doing your pod. I like to having it on mine. but I hope everybody gets a chance to to find you and give you a follow and and read your content because I think you're one of the people that I think can help solve this disaster we have coming our way. So I really appreciate the time, man. And really thank you for coming on. AJ: Thank you. I appreciate it, man. And congrats again on this blowing up. You've far exceeded my podcast in no time at all. I think it's awesome. Yeah. Aaron Mog: Thanks, man. Appreciate it. all right, talk to you soon, bud. AJ: Cheers. Aaron Mog: Cheers.