Latest / Machine Dreams / What Happens When AI Starts Making Decisions Without You?
Transcript
- speaker-1: Hi everyone and welcome back to Machine Dreams. This is the show where you get the best insights from the people actually building, securing and deploying AI. What's working, what's broken and what comes next and how that might even shape our future. I am Kolaolei Samel at DeBio, a technology analyst covering AI, cybersecurity and enterprise technology. And as always, I'm joined by my delectable co-host. speaker-2: Hey Sam and hi everyone. I'm Lea Stern, Global Communication Strategist and Venture Capitalist. How is everyone doing? Sam, where are you? Where do we find you today? speaker-1: Today you will find me in the very heart of Silicon Valley and in San Jose where I've been for the past couple days getting around with the business of AI and cybersecurity. And then I'm in my makeshift studio. speaker-2: Cool. And how are the vibes out there? Positive? speaker-1: Yeah, it's been great. It's been rainy, ⁓ but it's been great. It's interesting to see the innovations and the big talk. mean, you know, when you're in Silicon Valley, you're going to see a lot of these things and the movements and the motions in the AI world particularly. So it's been great so far. speaker-2: Amazing. Enjoy. And thanks for ⁓ still recording the podcast. Appreciate it. Today we're getting into one of the most urgent and underestimated problems in enterprise security right now. And that's the question of what happens when AI agents start doing real work inside your organization and nobody's watching. Our guest is Shachar Tal, a veteran of Israel's elite 8200 unit and a former senior executive at both Checkpoint and Celebrite. He's the CEO and co-founder of Siata, which he built from the ground up in 2024 with a team of fellow 8200 alumni. He raised $8.5 million in seed funding and just had the company acquired by Checkpoint Software Technologies, the very place where part of his career began. Siata's answer to the agentic security problem. If you can't see what your agents are doing, you can't govern them. And right now, most organizations are going in completely blind. Shachar, welcome to Machine Dreams. Thanks for joining us. speaker-0: Thank you so much. Very happy to be here. speaker-1: All right, great to have you here, Shaha. So let's just get right into it. Everybody seems to be building agents right now. Everybody wants to get on the agentic AI train. And everybody wants to build first so that they don't get on the train late. But even before the agentic era, there was always the question about whether organizations knew about the applications within the network. And now AI agents seem to be making that even worse. What are you seeing in the wild today? What do you have to say to this trend of agentic AI and how to secure AI agents? speaker-0: No, absolutely. Listen, shadow IT has been around and we have seen that in the last couple of decades where usually people would have ⁓ more devices or unsanctioned software. And there's a lot of problems around how do you manage that? How do you secure that? With AI, it's happening much faster and at scale. ⁓ But one thing, a lot of people are really concerned and rightly so for these new digital employees that are now employed by them, right? So their own workforce has a small sort of like a team of a workforce around them that helps them do their work. They have their own sets of permissions, they touch company data and they perform ⁓ actions. They perform actions on behalf of that employee or on behalf of a business function. But the thing here is this is not a rebellion. This is not like, ⁓ no one's letting me work. You know, I want to make sure, you know, I want to use this tool in an unsanctioned way. They're doing what they're expected to do, right? Because they're being ⁓ asked to utilize AI. They are being expected, you know, to utilize AI in this, this brave new world. And so it's not about a rebellion. It's, it's more about demand that is outrunning outpacing governance. So that's exactly what we're seeing. Everyone is adopting AI very quickly, but there is a growing gap between the controls that companies can use to have in place and the actual actions that are taking place in their environments everywhere. speaker-2: And you were quoted as saying that AI agents represent the biggest leap in enterprise technology since the cloud. What is so exciting about this point in time in tech innovation and what is most scary? speaker-0: Yeah, I think this is well felt by everyone in the industry. And if you kind of have read any piece of news in the last few months, you have read about agents and about how they threaten to take away some of the workforce or some of the human capacity and what they're doing. But this is an amazing opportunity for humanity. And I don't know how to understate that. ⁓ It's a very grand sentence, but it is true because we are changing how we perform knowledge work. So everything is changing around us. And it's like, there was the internet, there was the cloud, which kind of changed how people think about things. And this agentic AI movement is also changing how people think about things, but it's doing it in a much faster way. And this, I guess, this gap in control and in governance and understanding what's happening is still out there. So everyone is like, they feel they are lost, like CISOs and security teams and governance teams. They know there is an issue and they don't know exactly how to solve that yet. It's not about just missing a tool because like, you know, the entire market, everyone in the industry is kind of. figuring out what the right tooling should be, right? What should we do? What kind of practices, what kind of standards do we want in place? And it's part of this challenge because it's all happening very, very fast. the threat, of course, the challenge is if we let this happen, watching from the sidelines, we're going to see a lot of breaches, compromises, data lost. and we've, started seeing that we're going to see more very naturally. ⁓ but, know, it's part of our journey as helping organizations prevent that go into this new world, knowing what they're doing, knowing what's operating and what permissions does it have. speaker-1: Shahar, tell me about the concept of agentic identity and how important that is to deploying AI agents securely and safely and solving the challenge of trust because there's still a lot of distrust that people have with using these agentic systems. speaker-0: Yeah, no, you're absolutely right. And I think a lot of people kind of take this problem maybe a bit too academically, right? So I've seen all the, and we've taken part in some of the working groups are trying to define this and like, let's have this scriptographic identity and let's create new protocols that relay one another. While everyone is busy with that reality happens and agents are already out there. ⁓ And so our approach very early on was to make a best effort ⁓ kind of identity assignment and attribution to a corporate identity, which are the two key factors in understanding kind of agentic activity, right? So what is this agent? Can I track it over multiple runs? And I track this identity. For example, if I have chat GPT on my desktop, that would be an agent. right, for us, because now you can ask it to do things, you can provide MCP tooling, et cetera. So it's an agent and it's sitting on a computer. As far as we would call it, that's a single identity. Okay, and it's also attributed to me. If that's my computer, it's attributed to me. Now, these two attributes are very important as you go to define policy. What an agent can do, what data sources is it allowed to access and with what tokens. So I know that's a lot, but it's something that we took, again, great effort into ⁓ making practical very early. Even if we don't have the perfect academic solution to everything being crypto-grade, we still have policies in place. Hopefully that makes sense. speaker-2: celebrate Checkpoint, Siata, three success stories. What is your secret sauce? And how did it feel when you got that call from Checkpoint that they were coming for you? speaker-0: Well, okay, so there is no secret sauce. It's about hard work and it's about having the best team, the best people around you. And I've told this over and over again at Celebrite, I did nothing except for bringing the best people I knew around the world, right? So, you know, I just got the world-class researchers to solve the problem for us and that made the whole difference. And we same goes for Sciatica. make sure that we have the best people in place to solve the problem. When I got the call, ⁓ it was, I guess, partially surprising because we've had, we've had conversations, we've had a few conversations with, you know, with a few, let's say big corporates and we've spoken to corporate development teams about different opportunities and maybe partnerships. ⁓ But we did not imagine that would be acquired so fast because, you know, kind of seven months out of stealth is faster than what we had imagined. ⁓ But, you know, I got the call and I was called in to the office. Like, you know, don't sign anything and come into the office. Yes, so it was the same building where I worked a decade ago. So I kind of, you know, I knew how to get there. speaker-2: the office where you previously were. speaker-0: Um, so that was easy. And I got in and I spoke with the executive that I also knew from the army, by the way. Um, and then, you know, we had this conversation and it was an amazing opportunity for us. And immediately, like we knew it was a great place for Sayada to execute on the vision that we, you know, that we had in mind. Like this is a place to bring this solution. at scale because as far as we can reach with our small startup operations, we can get to 10 and 20 and 50 customers. But here's an opportunity to have 2,000 salespeople in front of the entire world delivering our solutions. And that's already happening. So we're super, super excited about that. speaker-2: That's so amazing. speaker-1: Great. Shahar, your research researchers have been ⁓ mapping ⁓ the attack surface for these agentic systems and you're talking about prompt injection and data exfiltration and memory poisoning and, ⁓ you know, hallucinations and even jailbreaking, which I sort of tried to attempt a short while ago. ⁓ So I'm curious to know which of these things are keeping you up at night for this. agents. speaker-0: Okay, so I guess the only thing keeping me at night is my kids when they're loud and they're running around. it's sirens. Well, some of it, yeah. But honestly, what's been keeping us busy is we've been conducting a lot of research around agentic frameworks, around how they work, around what the permission model is like. And we come with a lot of expertise, like deep expertise with vulnerability research. That has been kind of our main profession for a lot of the team in the last few years. And as we review and audit these AI agent or agentic frameworks, we find a lot of issues. We find a lot of problems that are very impactful because everyone... rushing to adopt these frameworks very, very quickly. ⁓ And so we found a lot of major discoveries that we intend to make public, hopefully at Black Hat this upcoming August. ⁓ And we found very, very deep issues with all of the big ones. And it's like... big seven, like magnificent seven issues. So that's been keeping us very, very busy because we understand that this is an opportunity for us to really impact how security happens around the world in agents right now. speaker-2: And do you fear for other cybersecurity companies that maybe soon will become obsolete as the industry is so rapidly changing, players like Mithos might be making them obsolete? How do you stay relevant in this fast moving market? speaker-0: Yeah. So I don't think it's going to make them obsolete, but we definitely have to be aware of that. And so like what I mean by that is, you know, we've all, we've, read through them, the mythos preview, ⁓ you know, research, we also have our, our own conversation, ⁓ with anthropic and access to mythos and Chagi PD five point four cyber is also kind of, you know, really changing how we think about these areas. I think what's happening here is we're moving away from these, know, model, model launches, like what the model can do to how do we mobilize defenders? And this is, this is what cybersecurity vendors need to learn and very quickly adopt and do, right? So we want to make sure that we put these models capabilities into defender use. as soon as possible or at least faster than attackers would. Because what's happening now is everything that's like, ⁓ this, you know, the public exploits so-called end day exploits, things that are made public, but there's no public exploit yet. And now the theory says, and I don't think that's a theory. I think that will happen is that once there's wide enough access for mythos and mythos-like models. we're going to see much faster exploitation in the wild. And so this is the opportunity for defenders to make use of the same models, okay? And to mobilize into defense. So how do you do that? You create your own exploits very fast and create defenses based on these exploits. You make sure that you put these models into use ⁓ in triage. in triage and in analysis of active attacks. We also need to make use of this narrowing window, okay, for defense, not just attackers. speaker-1: Great. Sharad, you've talked about some of the vulnerabilities that you're discovering or uncovering in some of these tools. You didn't call out the names, but I'm assuming that might include Cloud, ChatGPT, ⁓ Cursor, and maybe even Langchain. speaker-0: We already found Blankchain ⁓ and we've had a big release around that. We're going to talk more about that specific research at the Blue Hat Conference ⁓ in Israel, which was postponed to June, but we're going to be on stage. We're going to discuss that. There are more names. just being very responsible right now about our disclosure processes. We don't want to shame anyone in public or anything like that. want to make sure that everything is fixed and we want to make sure that we do it together. speaker-1: On top of that, my question really is, what are these vulnerabilities telling you or showing you about the state of security awareness among the people that are actually building these systems? speaker-0: I don't think it's about awareness. ⁓ It's about agility. Everyone's rushing to do this very fast. Again, rightly so. It's a competitive space. Everyone wants to build agents and agentic frameworks as fast as they can. ⁓ But it's also a very vulnerable surface as is, right? So everything, we know prompt injection. And prompt injection isn't going to solve itself, it's not going to fade away as kind of a minuscule problem because it's going to stay. And at the core, it means that you can influence inner thinking for agents, right? So you can influence the end results, the end actions ⁓ that it ends up taking. And with this core vulnerability, it's an intangible vulnerability, but it becomes ⁓ our regular, even memory corruption vulnerabilities, we've been able to trigger based on changing something in the behavior of an agent. Right? So I would say it's not an awareness question. It's a surface question. It will continue to be that way as these frameworks mature, but it's going to take a while. It will be a few years, I think. speaker-2: Shahar, what is your biggest takeaway from being in 8200, besides perhaps that you met and brought your incredible co-founders there? What did you learn there that really stayed with you in the course of your career following? speaker-0: I would say, I mean, it's a very good question. I think one of the things that you learn very quickly when you're there is that everything is possible. Everything is possible. Break what you think is possible and then, you know, great things start happening. And, you know, I think it's a really interesting landscape, right? So having this 8200, which is crazy funnel when you think about it, right? So you have all these 17 year old bright kids who've been growing around computers. They get screened around this 10 million people and they capture the best and they put them and they tell them, ⁓ no, you're going to be security researchers. And so, well, I mean, what do you get a few years later after a few years of intense training, intense kind of execution? ⁓ unlimited environments, like where they educate them to break what they think is possible, you get great cybersecurity companies, which is exactly what we're seeing, right? That's what we get out of Israel. ⁓ So I really think it's an amazing school, but also in a powerhouse that keeps and enables Israel's powers. speaker-1: Thank you, Sherham. My last question really is, if you were a CISO right now with a budget and a team, you know, to build a different threat model, what is the internal conversation? What is that conversation that you need to have internally before any of this becomes actionable? I'm sure a lot of CISOs and a lot of our viewers would like that question. speaker-0: No, no, listen, it's a... So I'm aware every organization has their own processes and pace and methods of doing things. ⁓ I think the first thing you have to accept is this is already happening, whether you are on top of it or not. Right? So whether you already defined a policy, people are already using that. So instead of being kind of... You know, the brake pedal security is used to being like, here are your stop signs. Here are things you shouldn't do. You have to get in front of it and tell them, well, here's what's allowed. Let's do it together in this way. And that's exactly what we've been, you know, trying to build and around this entire agentic security posture management, right? Which is like, I understand what's allowed in my organization, I get to define it. And I also get to define, you know, actions once this policy is violated. So, and I think that's really critical to put in front first. And it's outside the usual paths for AI governance, right? So people have, in the last few years, there have been AI governance committees that put in a lot of policies around models or on how you train them or on how you use AI. But it's very, let's say, not right for the pace of 2026 now. So that's something that I definitely understand is how do we move from stopping to enabling as fast as possible. ⁓ Don't overdo it. Don't be academic about it. Be practical about controls you can deploy today. speaker-1: That's it from Shahat Tal, C &C co-founder of Seattle, the company that is shaping the future of agent identity and trust. Thank you so much for joining us on Machine Dreams. This was a really great conversation. speaker-0: Thank you so much. hope you have a great rest of your week. speaker-2: Same to you. And thank you to everyone listening. If this episode resonated with you, please share. We'll see you next time on Machine Dreams.