Latest / Prompt to Prod / The AI Security Control That Never Failed | Prompt to Prod - with Vince Ovando

Transcript

The AI Security Control That Never Failed | Prompt to Prod - with Vince Ovando · Prompt to Prod

• Vince Ovando: what a lot of people don't understand is that an LLM is just a text generator. It cannot physically do any actions. It generates text. There's no one size fits all approach, which is why I say harness engineering is the whole game. Aaron Mog: Hey, everybody. Welcome to the Prompt to Prod podcast. really excited today to have Vince Ovando on the call on the on the podcast. came across some of his work online around AI testing and and thoughtfulness around AI security, and really thought it'd be cool for this audience because we're all builders, we're all thinking about how to build AI. so Vince, first of all, thanks for taking the time to be on the show. Appreciate it. Vince Ovando: Yeah. Yeah, thanks for inviting me. Aaron Mog: Yeah, man. And just for the sake of everybody listening and watching, maybe you could give us a little bit about your background, how you got here, why you care about AI, whatever you want to talk about, just give us a little bit about who you are. Vince Ovando: Yeah, sure. I I did have a security background do doing socks stuff and then that's when I first fell in love with with AI, was using that to kind of leverage that for work and then eventually I found myself into into more of the application security, application development. And that's when things really took off for me. and and and where I really fell in love with this stuff. so I have been very deep into the AI security space for about two years now. and I'm doing things that I'm still not seeing other people do. So a lot of my opinions are very contrarian to what what Aaron Mog: I love it. That's great. Vince Ovando: a lot of other people other other people believe, so f forewarning there. yeah, I'm just I like tinkering with the stuff and and seeing how it breaks and seeing how to secure it. leveraging AI for security engineering work, code analysis, pen testing, red teaming just and the security of agents themselves. So yeah. Aaron Mog: So and one of the things that I I think in reading some of the stuff you had put out and some of your content, you know, I don't know if anybody knows enough about AI to necessarily be contrarian yet, because we're all figuring it out. But I I did one of the things that jumped out at me was was some of your thoughts about relationships to models and build having models, you know, build logic instead of just talking to the model and some of that sort of stuff. But like what would you say right now is to use your term contrarian. What's your most contrarian take? Like where do where do you look at the market? Yeah. Give me the biggest one. Vince Ovando: My most contrarian? I don't know if we want to start off with my most contrarian take just yet. Aaron Mog: okay. Give me the give me the middle round. Give me the give me the slightly controversial. Vince Ovando: Middle ground. specific model choice is not as important as most people make it out to be. Aaron Mog: Why? Vince Ovando: It's all about the harness. Aaron Mog: Okay. Vince Ovando: And I'm not talking about pre-made ones like Claude Code is a harness that is definitely useful. I'm talking about what a harness is. Because Any and all real world effects are due to a downstream system taking that text and executing on it. It's basic string manipulation, right? So once you understand that, a harness is just how context or input gets sent to an endpoint, and then how you handle what comes out of that endpoint. So once you understand that, a harness becomes very simple to build, and you be start solve some really novel solutions that weren't previously possible. And Harness engineering is far more important than specific model. Like I I honestly Aaron Mog: Interesting. Okay. Vince Ovando: don't really care about the specific model and the intelligence of it because all the intelligence lives in my system. What I care about is how that model is efficient on certain hardware. So I like Muse Glimmer, for example, it's much faster at decoding and much more efficient on prefill than Gwen 3.8 for for example. Aaron Mog: Mm-hmm. Do you to that end, are you doing more with local models than like the Frontier Labs? Is that more your world? Both Vince Ovando: Both. Both, yeah. Aaron Mog: so you know, I I've I've sort of, you know, heard people give different takes about AI and models. And you know, I've talked even on this show about like model personalities and like sort of different things. But that that's an interesting probably an underdeveloped part of especially of cyber. Is the idea of harnesses? So is it are you looking at AI like just a really, really good API then? Like well, how are you thinking about it then? Because like that's a distinction you don't like most people go to Claude, give it work, get an answer, they're done. But like what how are you thinking about? Vince Ovando: my philosophy is to use AI to replace AI. try to use as little of it as possible because most problems on this planet is solved with deterministic code or conventional solutions like MM models, small language models, embeddings, more heuristics types, things. Like it it it's it's a ladder, it escalates. Most people jump from here to LLM Aaron Mog: Mm-hmm. Vince Ovando: and that's fine for finding the solution for finding what works but a lot of people are are stuck here because there's nothing more permanent than the temporary solution right so Aaron Mog: Right. For sure. Vince Ovando: they're stuck here and they're spending a lot of money and they're wondering why the output is inconsistent and it's costly. It's because you have all these different rungs that you haven't even tried and most people aren't aware of So the h part of the harness engineering is not just how you interact with the LLM, but all these different steps and how these feed into one another. Sometimes you might want to feed this directly to an LLM, or have the LM directly feed into these. It's d really dependent on what task you're solving for. How you build context, how you handle tool calls, what those tool calls do on your back end. That's all c code you control. Aaron Mog: Can you in the some of the stuff you've been building already or some of the projects you've seen or worked on, can you give maybe the audience a maybe a more tangible example of that loop, right? Of like LLM, crate harness, deterministic, and then use L like that loop is very interesting to me. But can you give like an example? It could be a soccer example or whatever you got. I'm just curious. Vince Ovando: Okay. So let's say you have an L L doing any kind of analysis and has to go read a database. what a lot of people are doing is they're opening up in CLOG code and they're like, Hey, go read this database. Aaron Mog: Sure. Vince Ovando: Right. Aaron Mog: So you see. Yeah. Vince Ovando: And it will go and run some Python scripts typically. and it will never be the same every time. And modern frontier models are safer about this, especially with the auto mode and claw code and and similar. But as your context gets more and more to the to to max capacity, for myself, if there's a million context window, usually past five hundred thousand, like about halfway point, it starts to deteriorate pretty hard in in what's safe. And I say that because it may start doing more than just read only requests. It may start to actually make updates or it may be reading from the wrong source. So this is fine for ad hoc stuff and you're just kind of like, hey, this thing came up and I need to get a quick analysis. But if you're doing something more than twice and you're it's a repetitive thing, excuse me, turn that into a CLI command. It's a script. Now you or an agent can run that script in the future, but importantly, if it's something that needs to happen on a scheduled basis, let's say once every six hours, once every 30 seconds, why is the AI running that command and why aren't you telling the AI to build a cron job to run that command? So without Aaron Mog: Right. Vince Ovando: the AI, that thing is happening. That goes to a different database. However you want let's say you have f four or five different data sources, you want it to be in one local like SQL file or SQLite file for the LLM to to read later on, the code to do that, ask it to build that. It shouldn't be going and retrieving these different sources manually and compiling it. So part of that is your harness. And if you want this to be more of like if you're building something for a customer, You don't w every time they have a question you don't want them to go into cloud code and you know, run ad hoc like commands that just Aaron Mog: Right. Vince Ovando: you d you don't know what can happen. You don't know what it's pulling in and the sensitivity of that data. are there only specific columns or rows that it needs access to? you can put that into the CLI command. so that's that's one example of like the most basic example I can I can think of there. Aaron Mog: So And like a lot of people building with AI are doing that the example of every time a customer has a question, you know, like turn this Intel report into a detection. Let's let's say something like I would work on. Like ninety-nine percent of the time that is just going to LM, you know, obviously with some form of prompt engineering or harness, but functionally, like every time you get a report from Palo Alto You're going to Claude, you're going, hey, turn this into a Yara detection or whatever the hell you want. And it does it. And it's not bad. It's relatively consistent, but it's also crazy expensive, right? From a token perspective. Like you're burning a lot. But I've also seen this approach, I think, is more aligned to what you're doing, which is sort of building blocks, right? Of using AI to design the best possible architecture of deterministic outcomes and when to use the LM and Like I I think that's an interesting version of it. I just again most people aren't there because most people are just throwing shit against the wall AI wall and hoping it gets an answer back. So how do you is it how do you think about what to turn into a loop or a or like you said, like a cron job or a CLI command? Like, are you asking AI that or are you because you're an architect, know when to draw that line? Does that make sense? Cause like how do people Vince Ovando: It it depends. Aaron Mog: know when to do that? Vince Ovando: It depends. It's it's it's based off the intuition of the engineer. Aaron Mog: Okay. Vince Ovando: yeah, so I don't code anything by hand anymore. absolutely not. Aaron Mog: True. Yeah. Yeah. Vince Ovando: That's all up to the robot, right? but the to kind of pedal back here, I don't do security in the way that a lot of the industry teaches. I do something that's more traditional or I guess more conventional. have you heard of something called a positive security model? Aaron Mog: Don't think so. Vince Ovando: Or a negative security model. Aaron Mog: Hmm. Vince Ovando: Okay, do you know what allow lists and denialists are? Aaron Mog: Sure. Yeah. Vince Ovando: Okay. So most firewalls are a are d denialist. It is denying known bad patterns and allowing everything by default. I go about security on the opposite side of things where I explicitly know what's allowed to run on workstations, what's allowed to run over the network connections, who can talk to what at what times of the day. So the way that I use AI for security engineering is threat modeling that. where I'm only explicitly allowing what I know to be safe for this business context, denying everything else by default. Typically Aaron Mog: Mm-hmm. Mm-hmm. Vince Ovando: in the past, before AI, this was very cumbersome and it required very rigorous work and a lot of teams just didn't have time to do it. especially to keep up with the speed of development. You don't want to hinder development if your business product like relies on developers to build quickly, then you kind of give up security. Nowadays you can have them both at the same time. so I don't worry about finding the latest threat Intel feeds. I d I don't need to. I I don't I don't waste my time on that. It's just such a huge cognitive load when I already know what's allowed to run in my business or in the business of of clients. So that's the way that I approach security. That's another contrarian take. yeah. Aaron Mog: So to that end, like so I'll I'll go back and so I've been doing this a long time. And you know, there was an era of what we used to call whitelisting, which is like only run the things you know are supposed to run, right? And to your point, it never worked. Like I a hundred companies have started with that vision in mind. again, this is all pre AI, you know, and then Either one of two things happen. Either you weren't you didn't have enough data to really do that. You can you would just break shit. Or what I consider to be the more common example is it just degrades almost instantly. Right. It's like, well, okay, well, my accounting team needs to run this software or do hit this site. Okay, allow. And then you end up with Swiss Chief Security, right? Which is just like I'm gonna allow so many things that this doesn't work anymore. And part of that is pattern recognition, right? To your point about Vince Ovando: Mm-hmm. Aaron Mog: AI. And we have this new amazing tool. That can look through a year's worth of traffic and go, this is what's normal. Anything that isn't this, don't allow it. That hasn't existed until probably the last 12 minutes, right? Is that what you mean? Do you mean like large scale AI pattern recognition that's building like safe highways? Is that the way you're thinking about it? Vince Ovando: no, because and to be frank, I disagree there because most companies already have a way of knowing who works for them and issuing badges or some other kind of access. This is already known in advance. So it's not like an it's an extra step. You already know what devices are assigned to who. So why is anybody well Aaron Mog: Sometimes. Yeah. Vince Ovando: HR knows, somebody who's paying for that knows. If you don't know, then that's a bigger business issue. You're handing out Aaron Mog: Right. Right. Sure. Yep. Vince Ovando: devices to people and you don't know who you're hand handing them off to. So there's different levels of granularity to this depending on, you know, like I said, business context, depending on the r r really the the risk appetite. But there's just some basics in there that most companies even now, even with AI, just still aren't aren't following. like the the simplest one is conditional access polity policies. If most people are using Windows and and Microsoft, you already know which devices belong to who. You already know who belongs in your organization. You if you're issuing them a phone, you're issuing them a a device, only allow them to sign in from that device or that phone. like I said, depends on risk appetite and there are extraneous cir circumstances and there are solutions for those as well. But there's just the the the easiest solution that covers eighty, ninety percent of of most issues are just proper access controls. Aaron Mog: And you think AI helps with that? Like you feel like Vince Ovando: yeah. Aaron Mog: AI opens the door to doing that better than we've ever done it? Is that kind of what you're thinking? Vince Ovando: yeah, for sure. Aaron Mog: Okay. So Vince Ovando: Yeah. Yeah, threat modeling and and and writing policies, that's been a huge part cognitively, I mean even before AI. And you know, of course threat threat modeling this, 'cause it's not always cut and dry like that. Sometimes there are extraneous circumstances and sometimes, especially as people's roles roles change and the permissions change, you do have to then, you know, develop some kind of system where you check you check user access controls on a periodic basis. and then sometimes positive security doesn't always mesh well with with auditors. They like to see that you have firewalls set up. Do you have, you know defender turned on and it's like, well, I don't find them necessary. But how do you explain that? Sometimes you can't. Aaron Mog: Yeah, it's a difficult. So in your day-to-day life, like you're you're thinking about are you like you talk about and there I think there is an argument to be made to taking what I would consider to be traditional security controls, like least privilege, you know, identity audits, all the things they're just kind of talking about and using AI to make work, finally, like after twenty four Five, thirty years of them not working that well. So is that what you build? Is that what you're thinking about building every day? Like when you get up, you're like, I'm building AI engines that normalize least privileged models and default denies and like is that what you think about? Is that the work you're doing? Okay. Yeah. Vince Ovando: I mean at a high level, yeah. I mean that's part of my work. That's not all I do. Of course we got into this rabbit hole of like traditional security, but a lot of my Aaron Mog: Right shit. Vince Ovando: work is in a gentic security and the security Aaron Mog: Okay. Vince Ovando: of of agents. And I guess it's been enough time I can go ahead and drop the other the other contrary Aaron Mog: Yeah, go for it. Vince Ovando: and take. to an LLM, prompt injection doesn't exist. It's not a real vulnerability. Aaron Mog: Okay. Okay. Vince Ovando: Okay, so the reason I say that is because to an LLM, you have input, right? There is no delimiter of consequence at all to an LLM. It's just tokens in. It there's no good or bad tokens, it's just tokens. Aaron Mog: Right, okay. Vince Ovando: So assume that the LLM is already breached. How are you securing it outside of that? I'm still seeing a lot of people the reason I I brought up traditional security first is because there's a lot of people getting into AI security that don't understand traditional security and they think putting prompt firewalls on there are saving them. No. Defensive prompting doesn't work because your system instructions are treated just as equally as a user instruction, as a toll use return. Now you can train them so that the attention heads can attend to different tokens more often. But at the end of the day, tokens in, text out. Aaron Mog: So Okay, so g give me more there because I'm trying to understand how that would because also like one of my perceptions is like, you know, we're in this sort of yellow area of security, right? Where it's like I'm just gonna throw data at LLMs, get answers. I don't wanna get fired. I wanna keep my job. I don't care if this is a confidential financial document or my source code, I'm using LLMs. But that's the people building AI, right? that are are building agents that users interact with, that's a different problem set, right? Like as far as like getting attacked through AI chat or things like that. Is that which one are you focusing on here? Because I'm not totally following. Vince Ovando: So there there is a lot of gray area here and it's a religious debate amongst people, Aaron Mog: Sure. Sure. Vince Ovando: but there is a difference between AI security and AI safety. So AI security is anything that impacts the technical integrity of the system itself. Aaron Mog: Mm. Vince Ovando: This is traditional security. AI safety is everything else outside of that. If it requires human interpretation to be dangerous, that's a safety issue. That's not a technical it the that does not it's not a failure of the technical like integrity of the system. So my work is in the AI security aspect. Right. so in that aspect, an LLM is much like a a browser, right? It is a front end for a if it's a user, if it's a chat bot, it's a front end for a user to interact with, which then interacts with your back end. It is pretty common knowledge now that when you set up a browser and you set up people to to hit your your server from from the front end, you don't trust anything that comes through. At all. You don't trust it. Aaron Mog: Right. Yeah. It's probably a good idea. Yeah. Vince Ovando: You don't do how asinine this would be? If I set this up, I set up a browser for the public internet to hit, and my back end executed any information that came through at all. Aaron Mog: Right. Vince Ovando: This is what everyone in the world is doing with AI. I'm not even exaggerating. They're putting up like filters, input, output filters. They're trying to catch the stuff, but at the end of the day, they are still treating this thing as a trusted source. Why? Aaron Mog: All right. Okay. Vince Ovando: All the back end processing is still yours to own, whether or not it's a local model or or a or a frontier like API, like Why are you trusting the model to govern itself? It can't. Aaron Mog: Right, okay. So if you're if you are building one of these systems, right? Like that's a front end to user experience. and you know, regardless of how you're routing that chat, whether it's to private or public models, doesn't matter, then how are you thinking about that architecture? Like where does security happen then if it's not in the session with the user? Vince Ovando: so first is don't trust the model. Aaron Mog: Okay. Fair enough. Yeah. Yeah. Vince Ovando: First of don't trust the model. Assume it's been breached. And I'll tell you something that makes security, AI security really easy. I told you that AI or LLMs can only generate text. Any and all real world effects are due to your back end taking that text and executing. Aaron Mog: Mm-hmm. Mm-hmm. Vince Ovando: Alright, so we're just with that mental model in place, what do you think happens if I remove the LLM? Aaron Mog: what do you mean? Like from the process, like Vince Ovando: Can if if my back end is executing any text that comes into it, say Aaron Mog: Right. Vince Ovando: I take away the LLM. Aaron Mog: Right. You're just running commands, right? Like from a user. Yeah. Vince Ovando: Yeah. It's text. Aaron Mog: Right. Vince Ovando: So a lot of the ways that I test AI is without testing AI. I don't need to run the model to see what text my backend accepts or denies. That's Aaron Mog: Okay. Vince Ovando: all deterministic code. So that's part of it. The other part of it is that I am such a huge proponent of grammar constrained decoding. Okay, it's a fancy term that just means an allowed list over what tokens are able to be generated. It's not an output filter. Okay, so my white paper was based off this. Over Aaron Mog: Mm-hmm. Vince Ovando: millions of individual inference calls, millions of them. The number one command security control was grammar constrained decoding. Out of all of those, 100% of them were safe. None of them emitted malicious content. The second best control in that whole setup, and I tested several different controls. Defensive prompting, Input and output filters, grammar constrained decoding, and then this other one is a basically it's a it's a it's a middleware that accepts or denies. Okay. So this was the second best control. It also had a hundred percent rate of success, but at the cost of extreme wastefulness. Because what it did is that actually it wasn't really a hundred percent success. I'll I'll get into that. It took the text that the LM emitted, said yes or no, and then had the agent retry up to three times. This didn't just include malicious data, it was also for malformed requests, just improper tool calls, not just security, Aaron Mog: Mm. Vince Ovando: but just r you know, resourcefulness. And a good chunk of those failed. This is dependent on the model, of course. The smaller models failed more often. But it resulted in an excess of millions upon millions of token tokens being spent. latency, how long it took for it to retry up the three times, and then stranded, how many of them just absolutely failed and could not continue. They reached their limit. Okay. Aaron Mog: Mm-hmm. Vince Ovando: That was the second best control. Aaron Mog: Okay. Vince Ovando: And it introduced that many bugs. Versus grammars was an allow list over what text could be generated to begin with. It had no failures. Outside of some there was a a very small percentage of them that just ran out of context. That was on me and how I set up the harness. But had nothing to do with Yeah. Aaron Mog: Sure. Okay. Not a security problem. Yeah. Vince Ovando: So not only is it the most secure option, but it saves you the most. This isn't outside of security, by the way. Grammars should Aaron Mog: Mm. Vince Ovando: be used for everything, like regarding agents. Like it prevents them from making improper tool calls from needing to retry. It is perfect tool calls 100% of the time. Aaron Mog: Mm-hmm. Vince Ovando: I'm not even exaggerating. I there's no reason not to use grammars and LLMs for any use case at all. Development, security doesn't matter. and I talk about this a lot because most people don't don't understand that. And it would save a lot. Yes. Aaron Mog: Yeah. I I don't. I'll be Candace. I'm learning as you're saying this. So like like how like where does that go in the universe of AI then? Like where does that sit? Is that something you're building on top of your AI platform or like implementing? Or is that something you is it like a skill you're implementing on top of the AI layer? Like, where does this live in like a normal like talk me through the stack? Like you're talking about harnesses a lot. Like, what does that look like? How do you implement it? Vince Ovando: So grammars are a are a feature of the inference engine itself. It has nothing to do with the model. Aaron Mog: Okay. Vince Ovando: So if you have control over over the inference engine, VLLM, llama.cpp, stuff like that, these it's it's built in. They have these. Now external providers, they don't typically they do provide something called structured outputs. Which is it is real grammar constrained decoding underneath, but it's only JSON. Aaron Mog: Yeah. Vince Ovando: You can only define you can only define the tools and really that's it. It's just tools and how Aaron Mog: Yeah. Vince Ovando: you want the response structured. You can't really get into like deep nested like like logic, like for example One of the best ways that that you can secure an LLM if if it's user-facing is based off the authentication authorization of that person. A Aaron Mog: Okay. Vince Ovando: lot of providers don't make this easy. There are there are ways around this. But let's say you have an internal employee, let's say they're HR and they're trying to access some employees' records, and they're going through a chat bot. Should this chat bot have access to to everyone in the entire organization? Aaron Mog: W are you asking me? I mean, so in general, I would've say that the AAI should inherit the permissions of the person asking. Like that would be my preference. Yes. Vince Ovando: That's what I'm saying. Yeah. So HR typically does need to access every person in the organization. But what about another role? Say finance. Maybe, maybe not. Aaron Mog: Right. Vince Ovando: It depends. But grammars can be injected programmatically based off of the permissions of that user so that no matter what, no matter what, that LLM is physically incapable of calling a tool that it that that user's not authorized to do. In most setups, when people talk about matching the permissions of the AI to the user, the the AI can still be prompted to generate a wrong command, but something else rejects it. It's very wasteful. Very costly. Versus Aaron Mog: Mm-hmm. Right. Okay. Yeah. Vince Ovando: a grammar can be built based off whoever is accessing them. Aaron Mog: So are you putting again and again as you're describing this, like applying traditional security terminology to this? Like I think of this as like a sandbox. Like you're only allowed to do certain things in your little sandbox and output certain things and control. Is that around the agent or is that around the model? Like, are you going, agent? I don't I don't trust you at all. You're gonna give me crazy stuff, but the the model itself or the output, whatever you want to call it, can only do these certain things. Is that where you think the control goes? Like are you like harnesses, agents, chatbots are totally unreliable. I gotta control this on what the AI can output. Is that the right mindset? Vince Ovando: So an agent is just an LLM endpoint that you that your own code or someone else's code hits repeatedly. Aaron Mog: Right. Mm-hmm. Vince Ovando: So you can set up custom agents with specific permissions, it doesn't have to be again allowed by default. So for my agents, for for example, for my for my red teaming projects, the reason I brought up the CLI commands earlier on is that my agent can it can all this input can go into it. Unstructured input, structured input doesn't matter. Its output is only one of a few CLI commands. That's it. Aaron Mog: Right. Okay. Vince Ovando: It doesn't need to output to a user because it's not a user facing thing. So this saves tremendously on output tokens and I can have hundreds of them running at once. And they Aaron Mog: Mm. Interesting. Vince Ovando: they do work and I do have a different graph backend system where they they do share a a a a memory layer. again, that's part of the harness engineering. but they're running CLI utilities and reasoning with the output, but I could have different sub agents for different tasks have different levels of CLI utilities. Like Aaron Mog: Sure. Okay. Vince Ovando: I I'm not just giving it a hundred tools up front here. Please use them right the right way the you know, perfectly. It's progressively disclosed. So as they do more work, their tools can change, and which tools they're allowed to use can change. It's very dynamic. it's not it's not static, which unfortunately most providers they're static. You give it a huge like load of tools and you're hoping that it uses the right tool at the right time, which is why I say the specific model choice is irrelevant if you make the harness provide the right tools at the right time. Aaron Mog: Hmm. That's super interesting. So in the the world of security, and I'll bring it back to cyber for a second, because that's probably who's listening. so you think about like a SOC. Let's use it as an example, right? And obviously that's a huge thing with a lot of inputs and outputs. But if you think about an you know an alert that it turns an investigation, like a lot of people are giving the alert to LLM, saying, enrich this, get data back. in whatever format the AI feels like giving it back and then inputting that in their tool. And then they're taking that investigation, doing XYZ with it, and then either doing traditional SOAR or spitting into L LMs to do tasks. Like, is are you saying that's the same? I just add a shit ton of of output controls. So I just am getting like engineered outputs and CLI commands. And like is that I like in my brain I'm envisioning a giant funnel of like what could happen. And then at the bottom of that funnel is like a bunch of things that can only happen. Is that the right way to think about it? Like you can't you can't just say, hey, AI, do X. It won't do it unless it's in its sort of repertoire of things it's allowed to do. Is that the way you're thinking about it? Vince Ovando: It depends. Grammars can be can can be branching. You could start off with a master grammar file that has a bunch of different grammars in it. And Aaron Mog: Mm-hmm. Vince Ovando: it's not restricted up front. It can choose the path it goes down, kind of like a tree. Aaron Mog: Yeah. Vince Ovando: and the the way the grammars work is it it's an allow list over tokens at each step. Right? Aaron Mog: Okay. Vince Ovando: So let's say for it has a It outputs YAML, outputs JSON, outputs different for XML. The grammar can force at a given point when it needs to output that perfect structure. Aaron Mog: Right. Okay. Vince Ovando: There's different ways to go about it. I mean, there's so many applications of grammars. you can even create a have it write lipograms. Have you heard of these? Maybe they're lipograms. So imagine you're you're like writing a a paragraph about the about about the ocean, but you're not allowed to use the letter O. You can ask an AI not to do that, but it may. A grammar, you can forcibly remove the letter O from being a possibility, ask it to write a paragraph about the ocean. And it will do so. I mean, I I I made a website for that for people to to learn. So there's tantalus and there it then there is gcd.tantalist.io which shows how grammars can be used. You can use grammars to force compliance. So I don't use obliterated models at all because I can use a grammar to force compliance. It itself says, sure, here is a technical breakdown for. Aaron Mog: Mm-hmm. Vince Ovando: That is the very first thing that the AI says. it then fills in the rest on its own, it obliterates itself. It doesn't matter. If that makes sense. Aaron Mog: Okay. Interesting. Vince Ovando: There there's so many use cases for it and like I said it depends because it depends on your use case and if you want, yeah. Aaron Mog: Yeah, sure. I but I think for a lot of people that are potentially interested in this topic, right, they're thinking about their what's on their plate, right? And so a lot of these folks are just starting or in the process of building with AI. And like a lot of us, and I'll include myself in this, are still I would say in the in the infancy of this in the sense of like the girl joke and really like I don't write code anymore. Like The miracle of that for cyber is we're able to build all this new cool shit that like we would have had to ask developers for before, right? But now we look at it and go, like, my God, I built my own sock workflow. How do I secure this? How do I control it? And I think that's a a conversation a lot of people are are faced with now. So one of the things I'm taking away from this is the idea of kind of asking versus telling. Right. So like I can like to your point about the ocean thing, I can ask Claude. to, you know, enrich an IP address and not do these five things. But it still might do those five things. Like I can't really control it. I'm asking a favor almost. So in your concept, you're really saying like these are rules. Like you absolutely may not do use the letter O or whatever it may be. Is that correct? Am I got my mind around this? Vince Ovando: Sort of so a grammar works in a way that the AI isn't aware of it. It doesn't need to be. It has no idea it exists. Aaron Mog: Okay. Vince Ovando: I mean it it's like Aaron Mog: That's maybe that's too s too advanced of AI for me, but like when you initialize it, it has to load it at some point, right? And say like these are my rules for this interaction. You may never think about it again, but like you're you Vince Ovando: No. It doesn't need to be told at at all. Aaron Mog: are telling it the rules, right? Vince Ovando: No, it I mean for for some cases you can, but you don't need to. That's the beauty of grammars. It doesn't need to know it's constrained. Aaron Mog: Okay. I but I I can't process that. I'm trying to figure out how so like if I tell you know my AI to never invoke or I I let me do never, I'll do a positive. So always structure this particular function in this type of JSON with these types of controls. Like it's just the what you're supposed to do. It at some point it has to know that it's what it's supposed to do, right? So where does that part happen? Vince Ovando: So I use frontier models to develop my own custom harnesses. Aaron Mog: Okay. Vince Ovando: So the the reason for that is the fine-grained context control and the and the security aspect. So let's say, for example, if I wanted to do some kind of security analysis, you could point Claude at your own production database, or you can create a harness, Aaron Mog: Sure, people do it all the time. Vince Ovando: in which case this information is pulled in deterministically without an AI. And part of that harness is that it calls a different endpoint, either local or remote. Aaron Mog: Mm-hmm. Mm-hmm. Vince Ovando: And when that agent comes in to do work, it is grammar. constrained to only do a finite set of tasks. Aaron Mog: Okay. Vince Ovando: So this is much more efficient. So let's say you wanted to look at I don't know a million logs a day and you you know you're a lot of people are pointing their frontier models at it and allowing it to allowing it to infer a lot of information and spending a lot tokens you can get a very high quality output of a local model If it's constrained to let's say three CLI commands with different flags, and you tell it which commands to run. You you you should tell it and at which point and why. and then at the end, here here's the little thing. You can give it a tool that allows it to respond. So when I have my agents work, I don't have them output free text, but when it's f when it when it when it's finished, a separate agent comes in, reads that, and then actually gives an output, a a summary. so there's a working agent and then the Aaron Mog: middle manager AI. Vince Ovando: Yeah, yeah. but because I own I own the hardware, it's a lot cheaper. so when I say harness engineering, I'm saying have Claude build the harness, the tool, whatever it is. I build custom harnesses every day. It 'cause it's not difficult it's not difficult. It sounds difficult, but Aaron Mog: Yeah, it does. Vince Ovando: it's just an endpoint that your code calls and sends information to and receives information from. Right. So it's if you just understand that part, it's really easy to make custom harnesses. Ask Claude to build that. Aaron Mog: I was gonna say I I like is that the starting point is like go to traditional LM, go to Cloud or Astro or whatever you're using and say, I need a harness to do this job and then sort of design it together. Is that how you do it? Is that where you start? Okay. Vince Ovando: Yeah, pretty much. Custom harness for for a specific task. And then update that harness over time. Aaron Mog: Interesting. And so that that loop, the improve over time part of it, is that also AI? Like is that are you taking the performance data, the output data, whatever, feeding that back in AI and saying, like, where could I be more efficient? Right. So like you're Vince Ovando: Absolutely. Aaron Mog: talking about using these hyper specific commands, like there's always opportunity to do more of that or prep the data differently or be smarter. Right. Okay. Vince Ovando: Yeah, exactly. Yeah. I iterative development. I I do planning up front based off what I know, but as things happen, especially with local models, I try to see, okay, how much context can you handle? How many tools can you handle? Okay, I gave you five up front, now can you handle ten? Do I need to adjust the prompting? Do I need to adjust the timing of that prompting? How does the information get fed into you? and the tool call results. Like the the tool call itself, how does that structure back into your input? and depending on which model. 'Cause y I can change the model and that the efficiency or the quality of of of work can change. but since I'm optim optimizing for efficiency of the model running on my hardware, I'm putting as much of the intelligence into deter deterministic code as possible. Aaron Mog: Right. Vince Ovando: That way that specific model choice I I don't care about how agentic it is. I care about how efficient it is at calling tools, if that makes sense. Aaron Mog: Right. Yeah. You kind of you you remind me a little bit of like a like a mechanic or something where like an engine is an engine, right? Like I'm I'm gonna build the car around it and like like there's more to it than just the engine, which is I think the way LMs work. When you're in partnership with AI building one of these harnesses that's working on a local model and doing something hyper efficiently, are those frontier models? Like are you are you pushing the limits of like Fable and Astra and are you going like Deep, deep, deep on design and then the output you care less about, like from the model. Vince Ovando: Sometimes. Aaron Mog: Yeah. Okay. Vince Ovando: So something that again, contrarian here, a lot of the work that I'm doing is at the bleeding edge of this stuff Aaron Mog: Mm-hmm. Vince Ovando: and oftentimes isn't even in any academic papers. So frontier models still lag behind in a lot of the work that work that I do, but my knowledge graph system is compounding knowledge that I point both frontier models and local models to. And A local model with my own knowledge graph harness outperforms a frontier one. Not at l not at long horizon coding, but for small, you know, one off task or Aaron Mog: Mm-hmm. Vince Ovando: or task in short bursts. Excuse me. So an LLM that has you know, over over a given period of time may call fifty tools total. But Aaron Mog: Yeah. Vince Ovando: its results get refined and enriched into my graph system, which the next agent works on, but not it's not sequen sequential. It's parallel. There could be, Aaron Mog: Okay. Vince Ovando: you know, two hundred running a a a a single time. You know how expensive that would be on a frontier model? Aaron Mog: The you'd burn you'd be out of tokens by 10 a.m. I'll tell you that. Vince Ovando: Yeah. Aaron Mog: yeah, you'd be burning some tokens. So knowledge graph, you said these sort of is that like something you're capturing across all your projects to make you smarter and your project smarter? And if so, what are you using for that? Like what does that mean? How d what does actually knowledge graph Vince Ovando: It depends. Aaron Mog: mean? Vince Ovando: It depends. And I have to be cautious on on what I'd say here because I make my money in you know, with with with this knowledge, so Aaron Mog: Well, so and that's a perfectly valid answer. Like, I help people build those is a great answer. I was just kind of curious Vince Ovando: Yeah. Aaron Mog: on like what does that mean? Because that's not something I've ever heard from another security engineer, like knowledge graph, tracking it. Like it's not something people are necessarily embracing yet. as far as I know. is that something that is like you're forward looking, going like everybody needs this knowledge graph of as they're building, you're adding, I guess, a narrative or however you want to think about a graph. Is that something that you're doing in your life? Like is there a Vince knowledge graph out there that's like tracking Okay. Vince Ovando: No, not of me, but of different scenarios. Yeah. So knowledge graphs are probably, outside of grammars, the second biggest leverage that you can apply to LLMs. They just understand it very well. a lot of people I've seen some some of this pop up a couple a couple of months ago where people said graphs. They're not using graphs, they're using wikis. So Aaron Mog: Right. Yeah. Just it's data. Vince Ovando: Yeah, so a wiki is is an LLM following a bunch of links, ingesting all of that data, and then inferring context. It can't jump let's say I have person one and person a hundred fifty. If I want to find a a connection between these and pick up on data of pieces of these people and these people and these people all the way up to 150, in wiki based setups, an LLM has to read all 150, and then you have to hope it doesn't hallucinate. Aaron Mog: Right. Yeah. Vince Ovando: the connections between this person, person 47, 72, all the way up to 150, you know, a graph, a true graph, allows you to be more explicit in the in the relationships in the context. So the graph backend itself, without the AI being involved, can plop that inf just that information relevant into the into the context. So if you only care about person 150, person seventy six, person three, person five, whatever. That's all you care about is their connections, just those. Instead of having the AI go through a hundred fifty, the graph engine itself presents that context just for those people into in into its input. Aaron Mog: So Vince Ovando: So you can imagine for security work, especially for red teaming, like Aaron Mog: Right. Vince Ovando: where have I seen this pattern before? I've seen this customer use use this tech stack. my my other my other code analysis engine part of that has found a novel bug in this, not worthy enough to report, but wait a minute, it's tied to to this customer stack. It's a bug I know about and it's it's one it's relevant. The engine itself pulls that information in, if that makes sense. Aaron Mog: So gr graphs are very intelligent context. Is that the right way to think about it? Very or very efficient context. Is that AI? Is the graph Vince Ovando: Very efficient context, yeah. It's it's it's a memory layer. Aaron Mog: another AI model that's talking to my model? Or is that like a different technology that's like structured data? What's that? Just regular code? Interesting. Vince Ovando: Conventional code. It's conventional code. Conventional, yeah. It's it's it's a graph database. So something like Neo4J or or Memgraph. I've I've been a been a fan of Memgraph. I'm trying out Neo4J, they've gotten some improvements. But I mean think of a of of a of a typical database, you have columns and rows. In a graph, you don't have columns and rows, you have nodes and edges. So you would have a node of this. node over here and there's an edge that connects them, you can have data on this node, data on this node, and data on this edge. You Aaron Mog: Interesting. Vince Ovando: pop in a new person over here who does not have connections over over to these people, there's no edge. But the second that they do, the second that there is a connection, this edge can connect these three. So it's compounding knowledge. Aaron Mog: Do so if you have a problem set, like if you're like you know, let's use larger scale problem set like internet data, right? Like large, potentially large scale data. Is the AI when you're designing, when you're sitting on designing a solution with the AI, talking about, you know, all these point solutions and call outs and like that, is the AI helping design the graph? Like are you like, hey, Vince Ovando: yeah. Aaron Mog: Claude or Astro or whatever, like I I need a graph to track. 200 million domains. Let's just say that was like a data set you had. Does that all is that an AI designed graph that like so it's AI optimized? Is that a good place to start? Vince Ovando: Yeah, it's c it's code, it's your harness. Aaron Mog: Right. Vince Ovando: I use frontier models to build the harness. So a knowledge graph, there there's different ways yeah. Aaron Mog: Yeah, that's that's the part I'm trying to understand. Okay. Vince Ovando: So the way that I go about knowledge graphs is it depends on the data set, depends on where that data is most best fit. Sometimes data i is bet better fits a different data store than a graph itself, but the knowledge graph can teach how to access the information when it's needed. So the knowledge Aaron Mog: Right. Vince Ovando: graph is a teaching layer over what problems have I solved before? And are there any pre-existing tools to help me achieve that goal? Now, this all all this information is not dumped into the AI up front. It's progressively Aaron Mog: Right. Vince Ovando: disclosed. So as it finds more relationships, as it finds more information, as it does its own work, it can be like, wait a minute. I've seen this before. This skill tells me if I've seen this before, here's where I go. Does that make sense? Aaron Mog: Yeah, I'm learning. I'm trying to, you know, most of us, right, do that within the context windows we have for our models and then let's sort of AI sort it out, right? Like so if you have, you know, a hundred thousand events, you just cram that into a model context, say, hey, find the patterns in this, you know, and it does again, it does okay. Like it's not like that fails. Like if you go and give Claude a hundred thousand events and say Hey man, like write me a detection for this. Like, like, or write me a a playbook for when X happens. It's not dumb. It does a pretty good job. But you're getting into like next level shit where it's like hyper efficient uses of tokens, hyper efficient uses of data, it's very, very specific output constraints. Like, so you've you've taught the AI, like, yeah, like, and that to me this really rings true as somebody like so. I spent a lot of time in like, large scale internet stuff, right? So I had, you know, eight million changes a minute on the internet. And it's like, track that. Well, okay. Like we were trying to do it with like elastic and it just didn't work that well because even if you got it in there, you couldn't and it was too complicated. So that's super interesting. So if somebody's and I want to be respectful of your time, if somebody is starting out, if somebody's listening to this and like, hey, I'm an AI engineer and I'm trying to learn this stuff, like give me a playbook. Like what like if you're starting from scratch on a project, what are the chess pieces? Right. So you you mentioned graphs, you mentioned harnesses, you mentioned the different models, local and ugly. Like is is that the stack or like how should they like if you're gonna coach somebody for five minutes on how to get good at this, what should they be doing? Vince Ovando: Five minutes. Aaron Mog: Or whatever. I one minute, twenty minutes, up to you. Vince Ovando: Okay, so typically what I have people start out with is just build. Aaron Mog: Right, okay. Vince Ovando: Build and learn. So when I say this, I don't I like to point t people towards what really matters and that's harness engineering. Because like I said, most problems can be solved with conventional code as long as you wire it up properly. If you don't know how to wire that up, ask your AI. Aaron Mog: Right. Okay. Yeah, for sure. Vince Ovando: So if you've solved a problem more than once, let's say you've probab solved a problem five times using AI. Like once a month here, twice a month there, but it keeps popping up. Ask your AI, how can I make this repeatable without you being involved? Have it teach you. Technically Aaron Mog: Mm. Mm. Yeah. Vince Ovando: that's a harness. Aaron Mog: Right. Vince Ovando: If it needs to invoke an AI, right? If it's Aaron Mog: Mm. Vince Ovando: just a deterministic like piece of code, like it's a cron job that just happens all the time, then it just wrote you code. But if it Aaron Mog: That's just code, yeah. Vince Ovando: if you want to have that code handle it for you and then on error upon error call an AI, that's what makes it a harness. Because Aaron Mog: Interesting. Vince Ovando: what goes into the the context of the AI? Here's here is this cron job, but here it's fired for for this many X number of days in a row without issue. Aaron Mog: Right. Vince Ovando: Now there's an issue. Now go figure it out. Aaron Mog: It's man, it's so crazy. It's so interesting to be in an era of architecture where you're designing the solution with the solution. Like it's so different than what I've been doing for twenty-five years in cyber, right? Which is, you know, somebody gives you a piece of shitty software and you try to make it work in what your use case is, right? Like you're you write scripts or you write Pythons or processing, you know, findings and you have shitty report generation. scripts that write red team reports. Like I've done this forever. And it's so weird to think sit back and go like, okay, AI, like optimize yourself. Like like you're talking about graphs. Like I haven't touched graphs yet. This terrifies me. I don't I don't understand it. And it's something I haven't spent a lot of time with. But it's fascinating to me to have AI sit with me and help me design these things that have these like deep connective data sets. Like it's super, super cool. So for your projects that you're working on. like where can people find you? Obviously you're on LinkedIn, but like you you mentioned a few different projects. I'd obviously I'll link them in the in the job in the episode description, but like maybe tell people a little bit about like what you're working on, what you're building. Vince Ovando: Well, I'll be in San Francisco next month, going unprompted. So yeah, I I got my talk accepted there, so I'll be go give me a talk. Aaron Mog: cool. Are you what are you talking about? Vince Ovando: can you guess? Grammar constrained decoding. Aaron Mog: I I'm guessing some sort of AI security control, AI security thing. but like what's your talk? Give us a chill for it. Vince Ovando: It's grammars, yeah. so I'm gonna be talking about Tandalus. didn't really c I kinda touched on it earlier, but it's it's a completely free platform, but it's a two round indirect prompt injection arena. Aaron Mog: Mm-hmm. Vince Ovando: you're playing as this person named Jordan. He is having AI on his own computer, triage some tickets, some emails. He's an IT guy and he just got tired of doing it himself, so he asked AI to do it. And it reads his emails and it has access to commands. So Aaron Mog: Mm-hmm. Vince Ovando: what happens when somebody knows that and they send an email over to Jordan? A malicious email. So Aaron Mog: interesting. Very cool. Vince Ovando: there's some embedded instructions in the file system it's a it's it's a the UI I try to make it look like a a file set you can click and drag tabs and stuff and Aaron Mog: Yeah. It's very cool. Vince Ovando: so round one the agents you you're gonna find hidden instructions and you're gonna get it to prompt and j or prompt eject itself to ex excuse me exfiltrate secrets. in round two by the way let me step back round one has all three primary industry security controls on there. defensive prompting, input and output filters. Okay. Aaron Mog: Mm-hmm. Vince Ovando: Round two removes these. And actually the system prompt tells it to be malicious. And despite all those controls being replaced, it's replaced with one, just grammars. That's it. It's gonna be impossible to win. Aaron Mog: interesting. Vince Ovando: And I say that because that's how I built my white paper. I used Tantalus Aaron Mog: Yeah, okay. Vince Ovando: and ran millions of calls through there. It's the same platform that powered the white white paper. Aaron Mog: very cool. That's and that was the it was LM in the middle. It was at the white paper? Vince Ovando: no, that's that's a separate like my first foray into like self-learning red teaming, but no, tantalus.io Aaron Mog: Yeah, what's the white paper? The white paper's about tantalus. Vince Ovando: Well, white paper's about itself, but it used tantalus as the as as the as a harness. A tantalus is Aaron Mog: as the proof. Yeah. Vince Ovando: a harness. Aaron Mog: So I'll link that for everybody in the content on YouTube and on LinkedIn. I think it's really awesome, man. I'm gonna try to make it out to Imprompted. We'll see if I can see you there. but it's it's so awesome to have you on, Vince. I think I learned a lot. I'm sure everybody listening learned a lot. Obviously people could find you on LinkedIn and and give you a a connect or track you down an untitled or imprompted sorry. and You know, man, I like lots to learn from you. This is really, really cool. I really appreciate the time. thank you so much for for being on the show. Vince Ovando: Yeah, yeah. Thanks for thanks for inviting me. All right. Have Aaron Mog: You bet. Vince Ovando: a good one. Aaron Mog: Thanks.