Latest / Prompt to Prod / Treat Your LLM Like a Threat Actor | Prompt to Prod - with Assaf Kipnis

Transcript

Treat Your LLM Like a Threat Actor | Prompt to Prod - with Assaf Kipnis · Prompt to Prod

• Assaf Kipnis: I treat the LLM as a threat actor. Not a malicious threat actor, but it's a threat actor because I need to I tell it do one and then I need to have guardrails to what if it does one point one? it's not hallucinations that is the problem for when you build at scale. The problem is the agent makes decisions. The agent makes decisions with whatever context it has. Aaron Mog: Hey everybody, welcome to the Prompt to Prod podcast. I'm Aaron Mog I'm your host. really excited today to have my buddy Assaf Kipnis on the podcast. been somebody I've I've known for a long time, has a lot of experience, a lot of areas of security, and excited to talk to him about AI today. so first off, thanks for being here. Thanks for so much for taking the time to talk to us. Assaf Kipnis: for having me man. I appreciate it. I'm excited to be here. Aaron Mog: So for anybody out there who who may not be as familiar with you and your career, maybe you could give like a quick update on who you are, where you come from, what you're working on, and we'll go from there. Assaf Kipnis: Sure, so for the last thirteen years I've been working both in information security, trust and safety, integrity. I spent most of my time doing social media investigations and large scale investigations. I spent time at LinkedIn as an investigator. I managed and was a investigator on e-crime, which is the main financially motivated actor. investigations team at Facebook, that was about five years, spent a little bit time of time at Google doing scaled investigations across their products and did some voice investigations at 11 labs. now I'm consulting both on AI, threat intelligence, fraud, security, working with multiple companies on how to streamline their work, how to provide deterministic workflows and kind of just have trusted AI or not AI workflows and automations. Aaron Mog: Awesome. So as part of of that, one of the things that I know you've been doing is building a lot. some ideas that you're working through, I think you build for your customers. and so maybe you can start there. because I think most of people listening to this are interested in taking ideas from concept to execution with AI. both in the sense of like securing AI and doing that type of work, but also just building regular stuff with AI. so Maybe give me a little bit about what you've been building, what you are I guess excited about, what was disappointing, and we'll we'll tear that apart. Assaf Kipnis: So it's hard because I'm excited about everything I build until I'm not excited about it anymore. Aaron Mog: Right. Assaf Kipnis: but the I can start with a couple of things that are at the bedrock of everything I'm building now. So when I started building with AI, I was I had the idea of a startup and I was saying how do I extract threat intelligence and make it in a deterministic way, make it something that you can deploy in your environment? and everything that I've been building since has been in the vein of how do I take pros or information even in a spreadsheet and transfer it to something to to somewhere else with the ability for me to say, Yeah, I know this is not scramble, this is not adding things, and this is I can give this to someone and then they could say, yeah, this is actually correct. Essentially so it doesn't blow up in my face. and one of the the main thing that I built, and for some reason I I struggle with names. I struggle with making names up for stuff. That was always my problem. So back in the military or in in in school, my last name is Kippne's. Everybody called me Kippy, which is Sesame Street character from Israel. It's a it's a hedgehog. I always hated it, but I was like, okay, I'm gonna call this the Kippy system. So essentially what I built for myself is a system that works within Claude Code and it's everything I do goes through that system. It starts it started with I don't I don't remember anything. I have ADHD and I don't remember anything. So the first thing I wanted it to do is if I work with a client or if I work on a project, I want to open cloud code in the morning and say, what do I need to do today? What what am I missing today? What did what is the last thing that person said? So I started by building a system that essentially for every project opens a bunch of folders with MD files, and that's where it started that then is able to retrieve as I ask questions and have nothing actually saved in context. Everything is in the files. and as I kept building and adding more projects, the thing that I noticed is okay, I have this system, but how do I ensure that every new thing I build, every project I start actually has this? So I went back to the old idea of the the concept of is of instantiation. So I took my system, turned it into a skeleton, and then essentially, if I say, I'm gonna start a new project with Erin today and we're gonna work on X, open a new instance. It creates a completely new folder with a folder structure inside of how everything is supposed to be structured and specifically for Whatever it is that we're doing, does the investigation. So if I say Aaron has Company X, it will go investigate everything about who you are, who your company is, what are we doing together. And I can just, once that's set up, I can say, here's the document that I set up with Aaron. What do we need to do now? How do we need to work from that? So that's one of the first things I was working on. and I could talk to all the failures I've gone through with that one because it blew up in my face so many times. and the other thing that I'm really proud of is is my investigation system. So essentially what it can it it's an esta it's another institution of the Kippi system. So it's as deterministic as I can make it, and what it does is it just runs open source investigations completely for me. an example is yesterday someone gave me a very high level of malware investigation that they found. they found this malware on this website and said, okay Asaf, what's going on? it took several hours and then as I'm I'm guiding it but it runs through every OSINT tactic, every OSINT site, every MCP that I have and crawls everywhere and finds patterns and connection between data and then Four or five hours later, it has a complete breakdown of what this malware is, who is hosting it, where they're hosting it, what are the other websites that are hosting it, what is it built like, blah blah blah. And then I can continue asking questions. So as far as things I built, those are the two main things that I'm the most proud of, and I can I still today, it's been months, I still today continue to add to them and change them as I'm learning things. Aaron Mog: So that's super interesting. So you have, I would kind of I'll put it in the category of like personal assistant life management sort of workflow, Assaf Kipnis: Yeah. Aaron Mog: which you know, we've seen a rise of this recently from meta from other places where it's like manage your life through AI. Now I don't want to talk about that. And the other side is is sort of the investigation. I'll call it cyber, more cyber focused, like enrichment and things like that. The personal assistance side of things, and and you know, this is an area I'd love to get your take on because somebody who's building it. Like it seems cool, it seems pro productive, but also like there's two parts to this. One is like it's not as easy as people make it sound because there's still so many problems with auth and MCPs and these guys don't have that. And you know, and it's like, you know, this idea that you can have this magic agent that like manages your money, and it's like it does it because you know, my bank doesn't allow it, and then it loses its connection and it loses like it's so difficult. Because everybody has their own little kingdom to make it talk. So I think that's it's interesting. And then the second part of that obviously is just security of the model. And like, do I really want everything I own to be in Claude? Maybe I do, but like taking those two swim lanes, right? With the part like the integrations piece of this, like it feels like the market's way ahead of what is actually available to most consumers from a usability perspective. Do you find yourself frustrated with that or are you building a lot? Like so if you're like, I need, you know, you know, this to talk to my Google Calendar, but also talk to my LinkedIn, like it that's not as easy as it sounds. Assaf Kipnis: No it isn't. It is so frustrating. And the most Aaron Mog: Right. It's so infuriating, right? Assaf Kipnis: so the most infuriating part for me is I'll look online and being neurodivergent, when I see someone pasting posting something online of I made this, I one-shotted this, I immediately believe them. And I'm like, hold on. If they're doing that, then I don't have any idea what I'm doing. Because this Aaron Mog: Right. Assaf Kipnis: doesn't work for me at all. Aaron Mog: Right. Assaf Kipnis: and then I start digging into it. there was there was this thing that this guy one-shotted like a first-person shooter. And I'm like, did they? And I went and researched it. Yeah, there was a very, very, very big prompt, but behind that he did a ton of back end Aaron Mog: Right. Assaf Kipnis: of of the images and everything like that. And and and the the the movement. Like it's just not true. And the same as for these assistants is like, yeah, I just connected it and it goes to my to my calendar and it goes to LinkedIn and it just posts for me. So actually I'm I've today or last night it finished an the iteration a thousand and one to my how do I post on social media Aaron Mog: Mm-hmm. Assaf Kipnis: tooling and it's a nightmare because Aaron Mog: It is a nightmare. Yeah. Assaf Kipnis: I worked at LinkedIn and I worked in anti scraping and anti automation in some of the times. So when someone says to me, there's just an MCP that you can just post on LinkedIn or we can just find all the all the posts really easily, it's not true. Aaron Mog: No, it doesn't. It doesn't exist. And if you try, they'll ban you immediately. Yeah. Assaf Kipnis: Yeah, they'll ban you. there's ways to do it with the browser, but it takes forever. There's ways to do it with like Appify, but it's janky. So the problem that I'm seeing a lot of the times is like is someone says, I build this assistant and it da ta-ta-ta-ta-ta-ta. And then I start breaking it down and yes, it kinda does that, but it rep it it doesn't respond reliably. It's it it logs into stuff that you didn't ask it to log into or the thing that I actually was thinking about this recently people are really worried about hallucinations and I think that's how I would c I was categorizing it in my head, but It might not have all the context, it might choose specific context, and it might make decisions that are not what you want them to be. So that's where it becomes difficult when you ask it to do something, it tells you it did it, and then if you really look into it, it went through the process, but at some point it made a decision that just was a fork in the road. That took it completely somewhere else. And and that's where this oversimplification makes it really difficult. B also talking to clients, and they would say, yeah, I have Claude, I've been doing this on my own. Okay. Are you sure it works? It looks it Aaron Mog: Right. Assaf Kipnis: looks like it works, I bet it does. But is it gonna work in a week? Is it gonna work when you have 10 X customers? Those are the things that that really keep me up at night. Aaron Mog: So it's interesting because then you translate that into let's say the the real world of like cyber. I can't ask Claude to pay my bills. It's not possible yet. Like it's too many different sites, too many different things. It doesn't talk to Chase. It doesn't talk to Discover, whatever. It's like broken all the time. Like in it, like just Assaf Kipnis: Yeah. Aaron Mog: out of the box. Like I can make it happen with an app, but like Assaf Kipnis: Yes. Aaron Mog: it just doing it at a login level is very, very difficult. So I see the same level of hand waving in cyber where it's like, just have AI do your investigation. And I'm like, okay, but like. I'll have 35 tabs open Assaf Kipnis: Yeah. Aaron Mog: when I'm building a detection or doing an investigation. It's very, very difficult to synergize that into like a an agent workflow. And I and I I hear a lot of how easy it is. Like just let me do it and I'll do it. And and also doing it to the standards of a Fortune five hundred company is also part of the challenge. So when you when you move to the security world, how like how are you dealing with that? Are you being more specific in what you build? Are you building with more code? Like obviously your agent, like your day-to-day agent, you're probably not writing that much code, right? But do you find that when you go into cyber you are? Like what's the delta there? Assaf Kipnis: So having worked in in threat intelligence my entire career and seeing the Well, more in threat intelligence and actually more in the safety and integrity side, the concept of a false positive in integrity and like social media investigations and stuff and stuff like that is a lot more important than I've seen it in other fields. So I honest I what happens to me is I get stuck on it and I said I said, Whoa, whoa, whoa, I can't have false positives. And if actually I was talking to a director from Google a couple days ago and we were talking about Jeff. He's Yeah, so there'll be false positives and we'll go through them. And I it gives me the willies. Like I Aaron Mog: Right. Assaf Kipnis: I no no no no no no no no if if I'm giving you a product, it should not have false false positives because you're making decisions on false positive information. And when you do it with AI, you trust it a lot more inherently, because you think AI lives in your head. That's kind of what it feels like. And also you're not really reviewing it in the same way, and and on an enterprise level. So the way I so you were saying something like, you probably live a lot more in in AI than in code. And honestly, most of what I pr like yes, AI writes all the code, but most of what I produce is code. Because Looking at it after a long after playing with AI for a really long time, a lot of things have to be decided with a yes or no. A lot of things have to be decided with was this done? What yes or no? And if you ask AI yes or no questions, you're not it's not gonna work. It's it's it's even if it does work, the my problem is not if it works or it doesn't work, actually. Is it worked 90% of the time? What about the other ten percent? So instead of chancing that, just write code. Like if and I'll give you this the dumbest example. So I was doing something for my daughter's bath mitvah and I was it's celebration when she turns 13 and I wanted to take text from one document and have it Fit into a different type of styled document. Basically take the it was songs and psalms and and things for her to say, and and there was a a note, a pamphlet that had it really stylized for someone else, and I wanted to say, just put it here. This was this is a perfect example for just do it, it's easy. I said, take this, put it here. my god. And and I was really shocked by it because I'm not used to it because everything I do is with code, but this one I said, okay, LLM do it. It added a bunch of ideas that I'm what the fuck? Where did this come from? mess this translation, transliteration, it looked horrible. And then I shared it with someone and they're like, who made this text? And I was mortified. So this is a very simple example to how do you do this deterministically? Like, I want this text to be here. I don't want you to make anything up, and I can't tell the LLM, don't make anything up, because that's that doesn't work. So the way I do it in my work and in my investigations is the same way I did it. I said, write a script, scrape all of the words from the doc, put them in a doc in a text document, take the other document, strip it, identify what are the the types of text design in there and put that in a in a in a knowledge base and then take the document that he just scraped, put it into that document, into that that type of design. Then a script scrapes that entire document, puts it in a new text, and diffs it. It sounds extremely extreme, but for me, If one word is different is a problem. So that's how I work with an example I made for a customer is how do they identify attacks, fraud attacks, security attacks on their on their data? So they gave me a bunch of spreadsheets. I will never have the LLM run through the spreadsheets. The LLM writes me a Python script to to scrape all this information and then reasons on it and then scrapes it again to identify that it's exactly the same as the information that was brought in. So that's how I do that's and when you say when people say you just just investigate with Claude, just put the information in, blah blah blah. the way I do it is it has multiple scripts that takes the the incoming information in whatever type of information it is, puts it in a file and only then operates on it. So when you take the structure away from the LLM, it has it's it the probability of it making stuff up is less because it's not trying to figure out, what's gonna be in the next cell in the Excel. Let me just let me just do that. So I think it might sound a little extreme, but that's how I do Everything. Aaron Mog: So you're this is one of the fascinating things about AI to me right now. And and the more I talk with people that are highly technical, especially obviously in cyber, which is my part of the world, I'm hearing this narrative more and more, which is I'm not actually using AI to do the job. I'm using AI to write things that do the job. Justn't totally what I think the market understands and what customers understand and what the AI companies are selling, right? They have if you go Listen to Anthropic talk or OpenAI talk, they're not talking about using AI to solve problems by building software that now has tech debt and needs to run and instantiate places and all. They're not saying that. They're saying click the little plus sign in CloudWork, add your spreadsheet, and magic will happen. Right. But Assaf Kipnis: Yes. Aaron Mog: I don't know a single security person that trusts that, right? If that makes sense. Like I just don't, that isn't how people are doing things. But like what Then what are we doing here? Like we're gonna create just an insane amount of code, right? To replace human beings that now needs to be maintained. I don't like when you hear you're replacing analysts with AI, like if you're like you and I are sitting here, we're running a fraud department, like, hey, let's replace somebody's analyst with AI, we're not actually doing that. We're writing code with AI that replace analysts to do the job efficiently and deterministically. But like, that's not AI. That's code. That AI wrote to solve a problem. And that's like the biggest gap I see right now when I talk to people building is that right? It's not take a Windows log, add it to an LLM context with Claude and get an answer back. It's train, use Claude to analyze a set of logs and write a Python script or whatever to then do this going forward. I they're not making Super Bowl commercials about that. That's not what Assaf Kipnis: No. Aaron Mog: you hear from the model lab. Like it's it's a totally different approach than what I think people think it is. Assaf Kipnis: And this is something I've been thinking about recently about the how AI is marketed to people. And actually Aaron Mog: Yeah. Assaf Kipnis: the what there was an article I saw recently that said, the normal person doesn't use AI. And they it was kinda like, they're not people are not using AI, it's not and and also the almost the the concept of, well, the consumer's not using it, it's not gonna last. And my perception oz on it is almost that AI is marketed to the wrong people. AI is infrastructure. AI is a is a way to build things faster, at scale, and also be able to replace reasoning when like when when necessary. Like for example Yes, I can write code, but at some point I'm gonna have to have a person look at this. So as AI continues to get better, AI is gonna be the person looking at it in the middle, right? But Aaron Mog: Right. Assaf Kipnis: you still have to constrain it from not just letting everything through or or et cetera, et cetera. But the the idea is the the the day-to-day uses of AI, yeah, th they're really cool and really helpful. I just got a 3D printer. So it's like the 3D printers, right? It's like, I can make this and I can make this and I can make this. But in the end, even if all consumers stop using AI, it will still be infrastructure. It will be in everything that you use because we're it just allows us to build things faster and a lot more complex than we could in the past. But Yeah, we still we still are kinda there with okay, but how do we maintain this code? Especially if the concept is we're gonna get rid of people to because AI is going to replace them, which is a concept I don't agree with. It just makes no sense Aaron Mog: Right. Yeah. I Assaf Kipnis: to me. Aaron Mog: yeah, 'cause I think people, you know, they open up, you know, Chat GPT and they click the plus sign and they add a picture of their kid and they say, put my kid in, you know, a football game or whatever, and it happens and it feels like magic. It is magic. It's incredible. Assaf Kipnis: Yeah, it is. Aaron Mog: But that makes them think that's how we're doing things in the real world with AI. And it's not. Right. That that plus button is functionally useless in the business world. Like to create predictable outcomes that like are repeatable auditable things, right? But I also don't know that we want to write a trillion lines of code, literally. Like that's not good either. And so I don't like again, like in c in cyber in specific, you know, I talk to people who build a lot with AI, and almost everybody's using it as a copilot to build reliable code-based systems. I I don't know that that's sustainable. I don't know if that's a real thing. And because the amount of people understand apps is still a very small percentage of people doing business, right? you know, CFO uploads, you know, a spreadsheet to Claude and says, What are my top 10 biggest expenses last quarter and how do I minimize them? It'll give you an answer. But he doesn't know that if you do that again or with a different AI, you're gonna get a different answer. They don't know that, like they think it's just What it's just what the engine told me to do. So I know you do a lot of help for other companies trying to build AI, things like that. How do you have that conversation? When you're like, I know how to use AI because I click on the plus sign, I add a spreadsheet, it gives me an answer. And you go, I can't run investigate you can't take a customer complaint email, throw it at Claude and get an answer. It doesn't work that way from a fraud perspective. Is that a conversation you're having? Assaf Kipnis: Exactly. That is a conversation I am having and it's it all really relates back to me for s to cyber and threat intelligence because what we do as cyber security and threat intelligence pro professionals, which we all hate doing, is we educate the other person. Because we have to educate the other person because even you think about it from a from a security perspective, why do I need to patch that? Well Let me tell you a story. and why do I need Aaron Mog: Right. Right. Assaf Kipnis: to spend time on this? why do I even need a security team? Why why do I care? Well, mostly it's let me tell you a story of let me let me show you how the world's gonna explode in your face. And I think building responsibly with AI ends up being the same thing. You so and that's the experience I had. Like, yeah, I have a customer, they said, build this for us. And I go down to Okay, what happens if the spreadsheet comes out and like whatever got was in the spreadsheet is incorrect. yeah, I don't I know not to trust AI. Okay, but how do we do this? Like w what what would happen? what what are the what are the guardrails that you need? And it is the most difficult part to then s come and say, okay, we can build this, we can build this. I can build this with AI, and the cool thing is that as a as a consultant, I can build this myself while I build four other things, right? but the discussion needs to be: you need to understand that this is code. You need to understand that I'm not gonna give you a project in Claude and then you can just mess around with it. because if you start changing things, the logic is gonna change. And then you need an engineering team that's going to take my PRs and Plug them into your infra. and it's a struggle because the the the commercial world is like, yeah, we're we're just gonna connect this and it's just gonna work. And then the issue is that and I I don't know a way around this yet. It's just you have to be what we all don't like doing is you have to be a part educator when you talk to people and like, okay. AI is doing this, but it's not really a like it's not really doing what you think it's doing. It it all has to go through these hoops. And talking to security professionals is easier, but the way I think, and this is something I solidified recently, and this is why kind of jumping around, this is why I think The people who are going to be really good or who are really good with working with AI are threat intelligence and security professionals because What if it does one point two? What if it goes there? What if it goes there? What if it goes there? That's all code. What if it decides to to do do an RM command because it thinks that this is not right? What if it makes the wrong decision? So the way I build with AI is literally consistently trying to figure out how is it going to do the thing that I don't want it to do? And I have to learn from exper from from getting punched in the face consistently. Like, why'd you do like why'd you do that? Let's let's figure out how you never do that again. and the issue with working with clients is that they don't think of anything as a as a as a malicious actor or as a threat actor. They're like, this is just gonna work. My clo my customers are all cool. Nobody's gonna do weird stuff. So it kinda goes into the space of going back to we really need to have these conversations with customers and clients of The perception is it's not gonna just work. It's actually the opposite. It's going to try to do all kinds of stuff that you're not planning for. Aaron Mog: Yeah, it's it's it's difficult because we're also fighting this narrative, right? That's got billions of dollars behind it, which is this is magic. Like it's going to change the world. Which I'm a I think we're both AI pro AI for the most part. Assaf Kipnis: Mm-hmm. Aaron Mog: you know, the it just isn't being explained correctly. Like it just isn't being you know, I I listen to a lot of startup pitches and It's like input, magic AI, output. And it's like I'm the part in the middle is the part that I'm not sure about, you know, of of you know, how how are you doing this? Are you really using AI? Or do you have an app that has no AI that happened to be written by AI with you? Because those are two completely different things. And I would say 90% of what I see in cyber is regular little apps. That were written with AI, probably mostly, and maybe a little bit of enrichment somewhere along the way, right? Especially when it comes to threat intelligence stuff or whatever. It's like, hey, AI, summarize this Palo Alto threat article for me, and it does it. Cool. Like, but that's not AI. That's not, there's no decisions being made. It's just summarizing for you, which is something AI is good at. So, and I just don't think it's being sold that way. Like, it's still SaaS. It's just SaaS with AI written. code. And so when you're dealing with customers, and and I'm interested in this because like I don't get as much I get more opportunities on the go-to-market side than I do on the build side. But you know, the you know you show up and customers like, I want a magic investigations box. And you're like, cool, doesn't exist. I'm gonna write you an app that's custom to you that really solves your problems and I'm gonna do it faster and better with AI. How do they How do you negotiate that? Or like I'm sure a lot of people listening to this are interested in that conversation because they're trying to do things with AI and explain it and sell it. So I I love to like, what's the pitch like when you have that friction point? How do you get around it? Assaf Kipnis: So as you were talking, I was thinking about something. So I don't think the SaaS landscape changed that much. you probably remember automagically. Love that word. We Aaron Mog: Well great word. Assaf Kipnis: would get vendors and they would say, because it's a sales engineer, and they would say, it does it. You put it in and automagically it does the things. So automagically was replaced with AI. Everything is AI. I constantly see things on social media that I'm like, someone just coded this wrong. This is not AI. This is someone's human messed up. so the way I look at it, so and and I'll give you an example. I just spoke to someone and they said the about the malware investigation, and they said, Well, we have we're really interested in what's going on with this malware. So I'm not telling them I'm going to make you a piece of software and I'm not telling them I'm gonna make you an AI thing. I say I already built something. It's real fast. It if they ask if it has AI, yes it does. If they want to know what it does, great. But in the end nobody does. All they care about, as far as I've seen, is the result. How are you are you giving me an investigative report that I can trust? In a day instead of three to four weeks, because that's how long that investigation would have taken me. there's a very small universe, which doesn't feel small to us, of people who actually want to say, Hold on, how does this work? most people are like, Cool, just give me the result, I'm just gonna get them faster, and it's gonna look good. So the way I try to do it is because I Constantly build stuff for myself. I can constantly package those things and say, you have that problem? Cool. I already built something that does 90% of that problem. Let me just change it up for you and I'll have you t I'll I'll I'll have it solve it for you. so I have less of the conversation of is this AI or not? another example that I have is I have a very small customer that what they needed in the beginning was our reps put stuff into a chat app and we need it to go into a spreadsheet. Someone did that for them with AI and I looked at it and it was a N8N and Gemini. Aaron Mog: A lot, yeah. Assaf Kipnis: Why? This is this is regex, dude. Just copy. Like I always ask myself, could I have done this 10 years ago? If I could have done this 10 years ago, I'm gonna do it that way. I tried to explain it to them, they don't care. I said, Yeah, it's gonna cost you less. It's gonna cost them a dollar less, and it's gonna be faster and it's gonna be deterministic, and they're like, Is it gonna work? Is it gonna be fast? Cool, we don't care. Aaron Mog: Right. Assaf Kipnis: but Yes, when you go into areas where people actually ask these questions, then that's where you need to again tell the story of this is what I've seen, this is how I saw it break, and this is how I saw it this is what could happen if we only make it with AI. to say that that sticks, it's Honestly, it's the hardest to stick with people who are in the middle between I don't know anything to I'm very, very good at this is the people in the middle who think they're really good at what what they're talking about and have been like hearing all the the hype and that's that's where the education gets really hard because they think they know everything and they're not security people and you try to explain they're like y yeah, but it just works. I did it on on cowork and it just worked. That's the place that I struggle and the way I do it usually is I I just need to do a demo of like, look, this is where it goes off the rails, and this is where it doesn't go off the rails, and these are things that could happen, and really correlate it to their data and their concerns. Because on some data they might not care at all. But on some data, like, this goes back to a customer, this needs to be accurate, like, okay. Aaron Mog: Right. So I haven't been in security consulting since the AI change. Like I I sort of made my way. So I spent a lot of my career in consulting, you know, hourly building. You know, I do a PCI audit, I do a pen test, I do a Assaf Kipnis: Yeah. Aaron Mog: a strategy roadmap, whatever. And the leave behinds in that world when I when I came up were heavily document based, right? It was like here's your deliverable. You're maybe paying $100,000 for a 10-page PDF. Like who knows, right? Now we're in this weird new hybrid era of like I'm working for you and maybe I'm doing some internal building and I'm building scripts and I'm building apps and I'm building AI. Like you talk about like the idea of like I was talking to somebody the other day and I'm like, if you're doing a pen test, an app sec review, and you're not leaving behind skills to make it so that that app mistake doesn't happen again, you're a dinosaur. Like you you're not doing app sec the right But like that's different. That's different than it's ever been. So for you being out there in that world, how has that been? Like I I'm super fast because like you can't just show up and be like, hey, I'm a fraud badass. Pay me you know a hundred hours and I'll fix your program. Like, because it might take you two hours plus something you already built. Like, how does that work for you? How does the hybrid work? Assaf Kipnis: So I never understood the consulting that just provides a document. So I'm like, w then what? Then they're not gonna do anything about it. So yeah, build do it again. And Aaron Mog: Come back next year. Bill again. Yeah, sure. Yeah. Assaf Kipnis: I'm doing stuff like that on something that I'm doing, and I'm like, okay, they're just gonna come back next year, cool, that's all they want. So first identify what they want. And that's the hardest part for me because I'll they'll they'll show me something and say we want to know this. And my mind is okay, but what does this do? What would this do tomorrow? What what is the meaning of what I'm doing? And I'm working, I'm actually freelancing for someone, and the answer I'm getting is whoa whoa. They just want to know the answer. That's it. Aaron Mog: Right. By the answer, yeah. Assaf Kipnis: Yes, just okay, just give them the answer because they don't care. And working within a company, I've worked in these large companies that as an employee, when your leadership just wants to know the answer, is extremely disheartening. Because I want to fix the problem and they don't care to fix the problem. Jesus, they just want to write a sc document. So at some point, you just have to really sit yourself down and say, what do they want? Do they just want an answer? Great. Stop it. Just give them the answer. And they're gonna be super happy and they're gonna come back to you again. but the most fun clients are the clients who care, or clients who are kind of also systems thinkers and say, Okay, what else do I need to know about this process? About what will it what will break if we did this and we did that? So I think it's kind of having the understanding to underst to figure out where you where you are. But then a thing I've been learning the hard way is also when you start talking to a customer, you really have to get down to what they want and let's build a spec. Let's so I can come back to you later and say if that's the answer, great. But if it's not, you wanted a tool to do a thing, great. Let's talk through it. Let's build a spec. Let me let me iterate on it. Because one of the biggest and I think I said it multiple the biggest multiple times, but for me is the biggest challenge with AI is that I just wanna go. I just wanna go the next next iteration, next let me fix it, let me fix it, let me fix it. And then you s you lose the client because they're like, Aaron Mog: Yeah. Assaf Kipnis: Whoa, I'm in phase one, where are you? I just fixed Aaron Mog: I Assaf Kipnis: all the problems I already saw. So Aaron Mog: Right. Assaf Kipnis: it's pacing and saying, Okay, you wanted A, here's A, let's talk about it, let's figure it out. I already have B and C in the in the chamber in my head. but the other thing is I think billing by hours is dead because I work on four different c clients at the same time. even if I'm building something new, I have multiple projects open and they're all building for multiple clients. And if I charge for my actual time, I would charge nothing. Aaron Mog: Right, yeah. Assaf Kipnis: So I have been moving into charging by project. What do you want done? Let's figure it out. Aaron Mog: Right. Just outcome. Assaf Kipnis: Exactly. That's one of the the things that I really liked working at at Facebook. It was when it was Facebook, it was impact. I wanna work by impact. I wanna work by how I'm affecting your product. And I think that also helps for the longevity. I'm not here to just Fix a small problem. I'm initially here to understand this problem and then say, You hired me because I know how to look at the system. If you wanted to just solve the problem, go on upwork. So and someone would say that to me. I'm Aaron Mog: Right, yeah. Assaf Kipnis: just gonna go on upwork. Cool, dude, go on upwork. This will not be interesting to me anyway. So I solved your problem and then I come back to you with, okay, and this is gonna interact with this and this and this and this and this, and these are gonna be the other things that we need to think about if we want to grow your business. Or I found something and within fraud, within within secure, within the cyber issue problems that I'm seeing, and I can say, now I understand your space better, and now we can start identifying forward. How do we sell your your product better? How do we get analytics on your product? How do we find attacks that we didn't know are even happening in the future? So kind of creating this relationship of I can continue building for you almost like a fractional executive in the company if that's what you need. And that's what I like doing most, like being the fractional, like you brought me in to fix a problem, but now let's see how we can make your business better. Aaron Mog: Yeah. I and I think I agree. I think that's the future. I think we're all moving to this world of buying outcome and sort of building software more and building scripts more and you know, less here's the PDF, right? Kind of stuff. So to that end, and I and I I you we'll be respectful of your time, but I I will I'll to I'm gonna attack this from a slightly different angle because I'm super Assaf Kipnis: Mm-hmm. Aaron Mog: curious in your opinion on this. So I mean you're one of the top guys I know in e crime and fraud and you know social media hacking and all these sorts of things. The ability for the other side, you threat actors, just people being pains in the asses, whatever maliciousness you want to assign to it, like AI has got to be terrifying, right? Because the ability to create fake images, the ability to create misinformation, the ability to stand up totally legitimate looking infrastructure, as we're sitting here talking about building, like the bad guys have this ability too. Assaf Kipnis: Yeah. Yeah. Aaron Mog: What where do you think this is going? Like that part of it. Because it terrifies me because I I all I hear about is Volumpocalypse. All I hear about is outside in attack, attack, attack. But you and I also have a lot of experience in the world of drawing people out, right? Making people give you information, making people contribute to things, making people share things because you look like you're the real deal. What are we gonna do, man? Like how are we gonna solve this problem? Assaf Kipnis: So I'm gonna go get my soapbox. this is Aaron Mog: Yeah, let's do it. Give me a give me a good one. Assaf Kipnis: I can talk about it more from the world of integrity, trust, and safety and less fraud, but integrity, trust, and safety, and threat intelligence. And I am, for the lack of a better word, I'm very jaded. this is and you this is something that also kind of feeds into how I build. trust well corporate. has always been, and it's his insecurity too. I saw something bad, block it. Make block lists and allow lists. And as a person who's a systems thinker, being even at my most favorite job, and I really enjoyed working face at Facebook, there were times that I'm like banging my head against the table. Are you serious? We're just gonna block an image? Do you know what hashes are? Like, like, yeah, we're gonna block This image with this hash. Cool, they'll change a pixel, and that's it. So the world of integrity and security has has really revolved around we find something bad, we block it, we move on with our lives, we say we won, we took down a bunch of bad accounts, and we move on. And that never worked. because there was a never an understanding that and I really like to pick on scam for example scams because there was never an understanding that scams is a business and that in general all threat actors mm ninety percent of threat actors are businesses and they operate like a business. and in instead the the corporate world operates like it's a guy in his grandma's basement just trying to think And like, we blocked them, we took we we we took down their account. And who cares? Like as a business, do you care if something small happens? No, you just like work around it. so That's let me get my my my my thread back. The the going back to the thing that scares everybody is images, voice, video. Honestly, if the industry and this is multiple industries continue working in the same old way of let's block this, we are gonna continue losing and the scale of losing is just gonna get bigger and bigger and bigger because it Aaron Mog: Yeah. It's terrifying. Assaf Kipnis: has. Aaron Mog: Yeah, it has already, Assaf Kipnis: And Aaron Mog: yeah. Assaf Kipnis: and I see this all the time. I see this on okay, let's identify that this is an AI generated video. Let's put a a watermark, let's have an algorithm that identifies anything AI generated. And that works today and and you consider the the AI as an an asymptotic growth, it's not gonna work tomorrow. You're constantly going to try to block the next thing. And Aaron Mog: Yeah. Assaf Kipnis: this is the scary part. The scary part for me is that we have companies that are very used to working in a specific way, and I think that's it's been everywhere. And then they try to bolt AI on top of it, and they'll say, We'll I I'm sure you've heard this, we'll fight bad AI with good AI, which is bananas. and the Like we're just gonna block things faster. We're gonna identify things with faster with AI. The thing is, a defender will never be faster than the attacker. Because an attacker needs one way. A defender needs n ways to block. Any ways to in n directions to look at. and For if you ask me, okay, I saw so you you you've been on your soapbox, you're extremely jaded and and everything's bad. What do we do now actually? So what do we what we need to do actually just talked to someone about this. I think that what we need to do is what we always needed to do. We need to understand threat actors. We need to appreciate where they're coming from, understand what they're doing, and then have a plan. A a plan cannot be Something bad happened, block it. That's not a plan. A plan is how do I make bad thing A never ever ever happen again on my platform? And how do I make any other iteration of that difficult? how do I increase it's basically the opposite of a customer? How do I increase friction while I I decrease friction on my on my actual customers? So The problem that I'm seeing with it now is that it's really it's time consuming and expensive to learn everything you can about every threat actor. I'm seeing it when I'm building AI, treating AI like the threat actor because something will happen and I'll say, stop, stop everything. How did this happen? This just blew up in my face. Do a complete root cause analysis, figure out why that happened, and build code to make sure this never happens again. And any iteration of this happens. And if you try to do that at very, very large scale, you're gonna need a very, very big engineering organization to constantly do that. The thing is, if you do that, what's gonna happen is that you're going to dissuade attackers because, from a financial business perspective, if you're Facebook and an attacker is trying to do s a specific Exploits, whatever it is, or exploit a specific set of exploits, if you make it extremely difficult for them to even start thinking in this direction, at some point they'll say, Facebook is annoying, we don't want to do this anymore. There's other social platforms, let's go there, maybe we can do it there. And I've seen this happen. so Now that we have like the the multiplier of AI, it works both ways, right? They can throw MN things at you forever, but that costs money. And you as a the companies usually have the resources, and then you can just continuously make these attacks more and more and more difficult to do, and then it's I don't know if everybody appreciates this perspective, but the perspective is get off my platform. Go somewhere else. I don't care where else you Aaron Mog: Right. Yeah. Assaf Kipnis: go. And if everybody thinks like that, then i you you continuously getting kicked off of platforms, and you'll find somewhere else. The the perspective I've seen at some large companies were, well, if we stop them, they'll just gonna stop. Aaron Mog: Right. Assaf Kipnis: Crime you crime doesn't stop, just goes to the Aaron Mog: Yeah, good luck. Assaf Kipnis: to the different opportunity. That's just how they make money. It's a business. And if you make it if you make the ROI crap, then they're just gonna s eventually. I don't like, let's not say just, eventually they will stop doing it. The problem is. that I think everybody struggles with in the space is when you're an executive or a leader at a company, you have to show numbers. And this is very much a security problem. I can prevent everything forever. But if I go to my leadership and they ask me where where's the numbers, I'll say, I don't have numbers. I things are not happening. And it's like, how do you prove something that didn't happen? And I think it's just now with AI we have the ability to do this at bigger scale, have the ability to measure this differently, but the simplest answer I have is you need to stop thinking in the way that you were thinking about it before and start from scratch with how do we implement AI into this to make this something different? because just applying AI to things that don't work anyway is not gonna work. Do I think that's possible for larger companies? Highly unlikely. Because people are already doing Aaron Mog: Yeah. Well Assaf Kipnis: things the way they are doing. Aaron Mog: And you know, I think part of the challenge of somebody and I'm not any crime expert, but like, you know, from a business perspective is so many legitimate things are acting like threat actors these days, Assaf Kipnis: Mm-hmm. Aaron Mog: right? With manipulation and targeting and, you know, you can't there is no more that's fake, that's real, we lean towards real. It's all fake. It's all AI generated. Assaf Kipnis: Yes, exactly. Aaron Mog: you know, counting on meta to police the world's intent is damn near impossible. Right. And I just don't like, is this possible? Like Facebook is already a hall of mirrors. Like you don't know what's real. Like disinformation is everywhere. You know, I was I was just doing a little bit of studying this week on like the upcoming elections and like what kind of AI manipulation. It's already out of control, out of control. By the way, by the campaigns themselves, not even including Assaf Kipnis: yeah. Aaron Mog: the threaters. Of inserting people into videos weird ways and putting them next to things they were never next to. Like the campaigns are doing this. What do you do? Like, what do we do when nothing's real anymore? And I, you know, again, that's a societal problem that neither one of us is gonna fix, but like I worry about it in cyber of like the intent becomes super important. And I don't I don't know how to manage intent. I've never known how to manage intent because Your CFO uploading a document to his Gmail is X fill. By definition, it's an actor Assaf Kipnis: Yeah, yeah. Aaron Mog: X filling data that shouldn't be X filled. But like, did they mean to do that? Did their coworker bot do that? Did their stupid grok thing do it? Like, who the hell knows? And and so, like, I just I worry about our ability to manage this stuff, you know, at its scale. And also to your point, creating friction slows growth, and that just can't happen right now. Like the greed meter being spiked. is like you can't possibly stop anybody because you might cost yourself a fraction of revenue and God forbid that we do anything to help anybody that costs us money. Right. So I don't it's a weird time to be doing e crime. Assaf Kipnis: Yeah, I've always looked at it like that though. and and I remember telling my team, it's like we need to understand that if the business doesn't make any money, we don't have a job. So they have to make money, we have to Aaron Mog: Right, yeah. So it's a balance. Assaf Kipnis: understand we can be frustrated with growth. this is where leadership in general I feel f failed after twenty twenty one, twenty two, just failed. In by trying to just, hey, provide metrics. Here's metrics. And that didn't work. The way I think about what you're saying is we have to appreciate that we are in a different world. And we can't treat the world as if it's the same world it was a year ago. I see that in education too. I was just talking to my mother-in-law and Students don't want to learn anymore and they just want to copy. Dude, students never wanted to learn, they always wanted to copy. Nothing happened, Aaron Mog: It's the same it's always been. Yeah. Assaf Kipnis: nothing changed. But if you sit there and instead of trying to figure out how to work with the new world, you just say that the new world is bad and cheating. Or you going back to the the the example of the work, if you say we just need to block all the things that are scary, instead of saying stop. The new world is where people generate videos. The video generation we need to appreciate we're not getting on top of that. It's just the the I see it in fraud when it's like, someone took a picture of their shipment and it's an AI generator. How do we make sure that that it's a real picture? Let that go. You're not going Aaron Mog: Well right. Assaf Kipnis: to. We need to And and and the thing is, that never changed. We need to go to intent. And yes, I understand you're saying, well, intent could be really squishy, but that's that's the only direction we need we that's the only useful direction because if we continue relying on understanding if content is real or not, we're we're it's Aaron Mog: Right, who cares? Yeah. Assaf Kipnis: never gonna happen. The the thing is that I was I was talking to some to to my mother in law about this of all things is she was saying, they're making these like exactly what you said, these videos and every people believe it. Dude, I worked in this in twenty eighteen. People made crap videos with that I was like, dude, really? People believe Aaron Mog: Right. Assaf Kipnis: this? And people reposted it forever and believed it. People will believe whatever they want to believe. Aaron Mog: Yeah. Yeah, for sure. Assaf Kipnis: this does make it more difficult, but in the end people are gonna be people. and the only way I see w with stopping or not stopping, hindering attackers and adding friction is by understanding how they operate. And maybe it's time to let go of the one-offs. Maybe not. Maybe some people are gonna yell at me about this, but but I feel like right now all we're doing are the one-offs of like someone created videos. Yes, but if someone created a video and I found who this account belongs to, and then I found all the other accounts that are related to this and all the other things that they're doing, and then I start profiling the whole network, and then I find the access where they started and I block that or I make that more difficult, that's a lot more effective. That's always been the case. But Aaron Mog: Yeah. Yeah. Assaf Kipnis: it's it's it's it got so amplified with AI because it we kinda used to be able to control content. We can't control content anymore. Aaron Mog: Yeah, it's gone. Assaf Kipnis: So stop trying. But people don't want to stop trying because it it's it's worth Aaron Mog: Yeah, well, and also to your point earlier, it happened super fast. It happened in a year and a half, like maybe two years, where it Assaf Kipnis: Yeah. Aaron Mog: like overwhelmingly got good really, really fast. And people have a really hard time telling the difference. Like there's always been, like you said, suckers, but like even me and you, like we live and breathe this shit. And it's sometimes Assaf Kipnis: yeah. Aaron Mog: I'm like, Wait, what? Is this real? Like are augmented or totally AI generated. Or, you know, and like a lot of it's on purpose, like they are doing it on purpose. So I I agree with you totally. I think I am I am fascinated with intent. I'm fascinated with you know, a a further push into identity and intent and normality and some of the things that we weren't able to do very easily before, like looking over mass data sets and finding abnormalities like the AI is really good at, right? Like AI is pretty awesome at pattern recognition that you know. You and I, like I was talking to somebody the other day who works in one of the Frontier labs, and I'm like, you know, what like you and I can look at a domain and just get a gut feeling if it's garbage or not, like almost immediately, right? Like this good this domain's trouble, this email address is trouble and and Assaf Kipnis: Yeah, yes. Aaron Mog: sort of gut instinct. Teaching AI to do that's super, super hard, but possible. So if all of a sudden every user in the world has me or you sitting over their shoulder helping them. essentially navigate this crazy new world. That's useful because we know. We know. Don't click what the fuck are you doing? Don't click on that. And like we can we can AI that to a certain extent. So I'm not like giving up. I'm just saying it's an interesting problem set. so that's that's awesome. So from a you know finding you perspective where people obviously can find you on LinkedIn. Where else can they find you? Assaf Kipnis: I'm on X what is it? Kipnis underscore A, if I'm not mistaken. I I forget. Yeah, yeah it's Kipnys underscore A. Keep forgetting that. Aaron Mog: Okay, had that for a while, I bet. Yeah. Awesome. I'll link to it in the description too, but Assaf Kipnis: Yeah, and I I I'm trying to write I I've been writing on Substack a little bit, but right now Aaron Mog: awesome. Assaf Kipnis: yeah, LinkedIn and X is w where you can find me and my website is Ask Consulting dot IO where you'll see me continuously changing my website because Aaron Mog: AI. Assaf Kipnis: I don't be telling like, let's change this. it's the worst. Aaron Mog: I know the feeling. so I'll link to those in the show description. anybody who wants to find you Man, I'm such a huge fan of the work that you do. I'm so happy Assaf Kipnis: Thanks, man. Aaron Mog: you spent some time with us today. everybody should follow you and and read what you're writing and because you are incredibly talented, experienced in this world. and a very important part of where we're headed. So thank you so much for being here, man. It's so great to connect with you and I'm sure I'll talk to Assaf Kipnis: Appreciate it. All right. Thanks, Aaron. Aaron Mog: you soon.