Latest / Noise2Signal / EP 4. Past, Present & Future of Risk Based Vulnerability Management with Ed Bellis

Transcript

EP 4. Past, Present & Future of Risk Based Vulnerability Management with Ed Bellis · Noise2Signal

• Speaker 2: Ed, did you know you have a child outside of your marriage? Speaker 1: I'm sorry. Speaker 2: Yeah, if you ask Google Gemini, you're the f Speaker 1: Yeah. Speaker 2: You you've created essentially the R B VM category way back in two thousand ten. Speaker 1: We did raise a seed round of funding ⁓ after talking to I at least forty, maybe fifty VCs. Speaker 2: Was Kenna the original name or act Speaker 1: Actually technically we had three names. Really? Yeah, the very first name almost nobody knows. We were a company called Yeah, don't don't Google that. Speaker 2: No way. No, th did that lead to the product market? Speaker 1: ⁓ when I think back at the early days of the company there were two big moments. One was that on the product market fit side. The other was what what I call market market fit. Speaker 2: Were there times where it was not certain if can I survive? Speaker 1: Yes. Probably multiple times. But we were down to probably less than two weeks of funding left in the bank. Like we ran into orgs that were even tying bonuses to their their scores for their assets. And that's when it got a little concerning for me. Speaker 2: I mean once you have people's jobs on the line, then your product could become like the villain. The whole mythos thing was farcical in the sense that in the sense that they didn't get the defenders into the program. Speaker 1: Having access to certain things doesn't make I'm not gonna become a criminal tomorrow because LLMs make it easier for me to become a criminal. Speaker 2: Ladies and gentlemen, please welcome Ed Bellis, founder and CEO at Empirical Security. Okay, we are live. Hey, welcome Ed. Ed, do you know you have a child outside of your marriage? Speaker 1: I'm sorry. Speaker 2: Yeah, if you ask Google Gemini, you're the father of RBBM. Speaker 1: Embarrassing. Speaker 2: It's so it's so it's so nice to have you here on the interview series. You you've created essentially the R B BM category way back in two thousand ten. You've been a CISO, you've been a founder multiple times. And it's been a fun and exciting journey from from my perspective to watch you grow and scale companies. And I'm curious what got you started with Kenna? What was going on in the early days of Kenna? Because this was way back in two thousand ten when you created RBBM or Kenna and there was no market demand for it. And you saw something and you built something there. Speaker 1: The no market demand for it is is probably one of my lessons learned. But ⁓ yeah, absolutely. So back in it was the end of twenty ten, ⁓ we were just kind of coming out of the finan the two thousand eight financial crisis. Everybody was just starting ⁓ early recoveries from that. But ⁓ I was the CISO at Orbits, the online travel company back then. And the issue we were having was ultimately what I was trying to to solve with Kenna. And and admittedly, I think in some cases we were on the bleeding edge of things, which also led, you know, kind of to your comment of, you know, there was no real market demand back then for it. ⁓ we had a number of vulnerability assessment scanners doing different things, whether it be on the network and host side or on the application side, we had dynamic scanners, we had static analysis tools that were looking at our code. And all these things and and really ⁓ just a large number of vulnerabilities, in fact, more vulnerabilities than we could possibly fix. and I was starting just talking to my peers, going, what are you guys doing to solve this? Because this seems like a ⁓ a difficult issue at at best. And honestly, a lot of them were doing ⁓ the same things that we were doing, right? Which was ⁓ trying to manually pull things together, put them in a massive spreadsheet, had the teens of analysts pouring through things to figure out, okay, is this real or is it a false positive? If it is real, how important is it? What applications does it affect? You know, is how what's the likelihood that we're gonna see bad things if we don't get this fixed? And it was all a very manual process. And by the time we got all the way through that process, you know, the pile was just growing bigger and bigger each time. So ⁓ after talking to all of those folks, I was like, all right, well, I can't find anything on the market other than Excel to do this, which is clearly not cutting it. ⁓ and and a team of analysts. So I went out and co founded Kenna back in to your point in the end of twenty ten ⁓ to solve that problem, right? And it took us probably longer than I thought it would. Speaker 2: You bootstrap can I d in those days or was it ⁓ did you get funding initially? Because it's very hard to convince. My sense is it's very hard to convince somebody, hey, give me all your vulnerability data, I'm gonna put it in the cloud. Speaker 1: Yeah, yeah, absolutely. So ⁓ first question, ⁓ bootstrapped or funded. We did raise a seed round of funding after talking to I probably talked to at least forty, maybe fifty VCs, right? ⁓ and ⁓ a whole lot of nos and a whole lot of maybes. The maybes I think are worse than the no's in my opinion, but ⁓ ended up raising A very in a very different time, ⁓ a seed round of just over one million dollars. ⁓ and we went out and got started on on building this. We had kind of built an initial kind of proof of concept, worked with ⁓ some friendlies in terms of kicking the tires on it and giving us some feedback. ⁓ but it was ⁓ definitely a a VC backed, if you will, ⁓ startup, although back then, you know, th those were different times for sure. ⁓ To your point around vulnerabilities in the cloud, 100%. Right. ⁓ in fact, we ⁓ I brought on a number of advisors back then and and one of them ⁓ was Jeremiah Grossman over at White Hat Security at the time. And he basically said, Are you sure you just don't you want to do this on prem instead? Because you're you're gonna get so much pushback for for vulnerabilities in the cloud. And they were they he said that because he was experiencing that as well. ⁓ I was fortunate in the sense that, you know, there was ⁓ folks like Qualas were already blazed some of that trail. So there were a lot of vulnerabilities in the cloud that were in Qualis's cloud at least. ⁓ but it was still a slog, right? There was for for everybody that was okay with it, there was probably 10 orgs that weren't okay with it. ⁓ and it was just this slow thing. And I remember also talking to in the early days. To Doug Song, who was one the founder over at at Duo Security. And he said something to me about this that stuck with me to this day, which is be on the right side of history. ⁓ and we knew where things were going. It was early and there was going to be a slog in the early days, but we didn't want to build something that was on prem and then five, ten years later be having to redo everything because everything had moved to the cloud, right? We were Speaker 2: And was was Kenna the original name or did the name change later on? Because I remember there was something RISCIQ or RISC IO or something on those lines, right? I mean Speaker 1: Yeah, that's that's part of the part of the reason why the name changed. So actually technically we had three names. Yeah. The very first name almost nobody knows. ⁓ we had basically our private Speaker 2: No way. Breaking news. Speaker 1: Breaking news, breaking news for the the the ten people that that didn't or outside of the ten people that knew. in the very early days when we were ⁓ when we had our friendlies kicking the tires and we raised that seed round of funding, we were a company called Honey Apps. ⁓ and that didn't work for many, many reasons. ⁓ but we quickly changed our name. Speaker 2: How many negative connotations? Speaker 1: Yeah, yeah. Nothing nothing good's gonna come of that. And we kind of realized that pretty early on and changed our name to your point to Risk I.O. Speaker 2: No, so this is a funny anecdote I want to share. ⁓ yes, you've probably seen the CSAM. ⁓ cybersecurity asset management. Speaker 1: Yeah. Yeah. Speaker 2: In one of the fr companies I worked at is they call the product CSAM and CSAM is C you know what that means. Speaker 1: Yeah, don't don't Google that. Speaker 2: So yeah, you know, you come up with these names and later on you realize too bad. Speaker 1: Yeah, one of one of the hardest things ⁓ to date is is naming software, right? So so we ended up changing our name to RiskIO. We were Riskio for ⁓ several years and then you know, a few years I should say. ⁓ and then as we started to get bigger, and this other company out there that you mentioned, RiskIQ, also was getting bigger, we started to see confusion in the marketplace where ⁓ people who were looking for you know, risk-based vulnerability management were going to them. And then people who were looking for more of like the attack surface management stuff were coming to us. And it's like, no, that's that's not right. So ultimately we had a a discussion with RiskIQ. and we decided to change our name to Kenna at that point. And we've been Kenna we're we're Kenna ever since obviously until the Cisco acquisition. Speaker 2: You know, one of the one of the things I want to give credit to the the OG in the VM space is Philippe. Philippe trailed the blaze in two thousand one and he had this headstrong view that we're gonna put vulnerabilities in the cloud way back in two thousand one when cloud was like one point ⁓ the cloud one point ⁓ with the sales force around the world. And I remember talking to Philippe and he told me he told me it took me five years to convince people. Okay. That you know, and it o and he didn't give up on that. It was so easy to go on prem in two thousand one. Speaker 1: For sure. Everything was on prem back then. Speaker 2: Yeah, like you know, tenable was on prem, so easy to go on prem. But the upside of that for COLIS was they got all the big enterprise customers. Because you because once you go to the cloud, you get scale. With on prem, you don't get that level of scale. So it was a you know, like you said, in what was the thing that you said? Solve the right problem, you know, regardless of hard it is, and then you know, just have the conviction to ⁓ to go with it. And was RBBM like s was R B BM an established category by then, or did you have to like force your way, this is not BM, this is RPBM? Speaker 1: Yeah. Yeah. There was there was definitely no such thing as RBVM for a number of years actually, even after founding. You know, we we were kind of, you know, pounding this, hey, prioritization is the next big problem, right? So first you have to get through ki I always called it like the the stages of grief for volume management back then, right? And the first is ignorance is bliss. I don't even know where my vulnerabilities are. I'm not doing any sort of scanning or assessment and that sort of thing. Then you get to that next stage where you you bring in the qualists, tenables of the world or whomever, and you start scanning everything. And then you're like, ⁓ my God, vulnerabilities are everywhere. What do I do next? Right. And then you get to the stage where, okay, I gotta start to prioritize these things. That's where we came in. And I felt like, you know, to your point, we were early. So a lot of people were still either in the God forbid ignorance is bliss stage, or they were just getting into the You know, ⁓ my God, we're we're starting to see vulnerabilities all over the place. What do we do next? ⁓ so yeah, we we long ⁓ it's kind of a long slog, but we we ultimately created a a category and obviously that was not only working with customers and the market and talking to a million analysts and and all of that thing, but ⁓ it's it took some time. Speaker 2: You know the hard part the hard part in those days was you guys there was no established Threat Intel for VM. Right? Like so you had to literally create the threat Intel framework. Because you know, what does RISC BSVM mean? I mean it means is there in Metasport exploit available? Like that is not threat Intel. You know, like that was like the early stages of like is there ⁓ exploit available true? And C V S is greater than nine? Right? That was like I didn't tell. You know what I mean? So you have to create the whole framework of risk to make sense of it. Speaker 1: We actually ran into some of this by lock and accident. ⁓ so we knew starting the company is like we gotta figure out a way to prioritize it. The honest answer is we didn't know exactly how we were going to s ⁓ end up using the the algorithms to ultimately prioritize these these vulnerabilities. ⁓ but so you you mentioned no vuln threat until there. We we talked to so many vendors and they're like, Why would you want that? What would you do with that? I don't get it. ⁓ and then we actually so I had met Roger Thornton. I mentioned that that one million dollar seed round, which was from ⁓ Tugboat Ventures ⁓ back then. And Roger Thornton actually was associated with those guys. In fact, he sat in on the due diligence meetings ⁓ to kind of assess what we were doing. He was at the time, he was one of the founders of Fortify Software. ⁓ But later on, he left after they got bought by HP. And then he ⁓ was the CTO over at Alien Vault. And we stayed in contact and and ultimately he ended up ⁓ being on our board for a while at Kenna. But I was talking to him and he's like, Hey, we've got this thing. We're calling it the open threat exchange. And it's basically just all of The telemetry and stuff, all of our customers can opt into this and we can collect all of the telemetry of all of our sims that are deployed ⁓ around the world. And it turns out a lot of this stuff is ⁓ attacks on vulnerabilities and exploitation attempts and all of this stuff. Is that anything that would be interest to you? And I'm like, Yeah, that's that that's on amazing, actually. So we kind of fell into that, and that was our first ⁓ real data source outside of, you know, just the Standard metasploits and exploit DBs and stuff like that, which wasn't really exploitation. It was more exploit code. ⁓ and we started pulling in all of that data. And then later on in a ⁓ after a board meeting, I was talking to Roger and he goes, Hey, you keep saying exploitation attempts, but I want you to know like what you're actually looking at there is it's a combination of things. It's where we collect and see, yes, a signature gets tripped for some sort of exploitation attempt. But it's also married up against an asset that has that actual vulnerability. And in some cases, IOCs posts post that attempt. So a lot of these are actually successful exploitations, not just attempts. Speaker 2: That's what he really wanna focus on. Speaker 1: Yeah. Yeah, exactly right. So that was that was our first of many data sources. We then spent years just like pounding not only the industry to set up this the this this market, but started talking to as many security vendors as we had that we knew did similar types of things and saying it a and trying to figure out if they had data and and honestly they would oftentimes sell the data or give the data away for nothing because they didn't think there was any value in it. Speaker 2: Did that lead to like no that did that lead to the product market fit where you know you have these hundred thousand vulnerabilities, but then our telemetry tells us that these are the CVs that we are getting hitched for exploitation and you have these assets that have these vulnerabilities. Instead of ten thousand, these are the ten you really need to focus on. Basically like that was that like the aha moment for your like prospects or like the free trials or whatever you were doing to get your first customer to get, you know, the money in the door? Speaker 1: 100% spot on. So one of the things they know up until then there was a whole lot of, ⁓ this is great. I don't have to log into fifty different consoles, but you're really showing me a bunch of stuff that I already know if I logged into 50 different consoles, right? So now with this new data that we are injecting in and kind of correlating against all of their environments and then prioritizing based on that, that was definitely the product aha moment for us where there's like, ⁓ you actually can show me why. I can narrow this big pile of a million down to, you know, 10,000 or whatever it is. ⁓ and that that was definitely our first ⁓ product market fit moment for sure. I will say ⁓ when I think back at the early days of the company, there were two big moments. One was that on the product market fit side. The other was what what I call market market fit, which is where ⁓ this is ⁓ a a partnership that we had established really early on as well. Where ⁓ we had actually hired ⁓ a guy from Qualas at the time he was ⁓ in sales and he headed up our sales at at Kenna. And he had just ⁓ established when he was at Qualas, he established a partnership with ⁓ Dell Secureworks, the managed service provider over there. So basically Qualas was gonna do all of the vulnerability scanning for them. Well, good news there is once people start doing the vulnerability scanning, that stage two of grief is okay, how do I prioritize all this? So he would then went back ⁓ as part of Kenna, part of RiskIO, I think actually at the time, and went to SecureWorks and said, Hey, we'd love to work with you again on the prioritization aspect of all this. And we were tiny still at the time. And I remember we ended up ⁓ agreeing to a deal. They invited us into their SKO to basically train all of their different sales teams on ⁓ Risk IO and and how to use the product, how to sell the product, etc. And we w went down to, I think it was in Atlanta and we went down there and each sales team would come in. They'd have the Asia Pacific team, the SMB team, the enterprise team, all these different sales teams. And every single sales team that came in that we trained was bigger than our entire company. and so we would go and did all that. And then we finally, I think turned that was back in January, February time frame. We finally turned up the the service through SecureWorks. I wanna say it was the first week of May. And in the first week of that service, they had already signed more customers than we had signed in the entire life of Risk IO before. So that was kind of our market market. Speaker 2: period was how long? Like a year and a half into the company, or how long was that period? Speaker 1: That's a good question. I wanna say that was closer to t almost two and a half years in by the time we put that. Yeah. Speaker 2: And then ⁓ were there were there times where it was not certain if can I survive? Speaker 1: Yes. Probably multiple times. But before that moment, before actually probably both of those moments, definitely before the the secure works ⁓ moment, we ended up we were down to probably less than two weeks of funding left in the bank when we ultimately closed our series A. ⁓ and there was ⁓ obviously a lot of back and forth ⁓ amongst that. One of the things that stands out about ⁓ raising our Series A is at the time I mentioned we had a number of people that were kind of kicking the tires and trying out our s our service. One of them we had absolutely at our sa size and stage, no business being in, but Bank of America at the time was was kicking our tires. And the guy who was leading that effort at at the bank, ⁓ agreed to be a reference for us during our Series A. ⁓ so he talks to to the investors. They, you know, were also as excited as we were that Bank of America was was an early ⁓ potential customer in our pipeline. And ultimately we closed that. But the funny thing is is that was in 2012, I think, when we closed our series A. And it was probably Four or five years later before Bank of America actually became a paying customer. Speaker 2: So then what were they doing? They just doing the free trial, yeah. Speaker 1: Man, it was there are there's so many things. I mean, th that th that would be like an eight hour podcast, I think, if we talked about the the lo the the pipeline of ⁓ that is large banks. Speaker 2: I yeah, I think it is true with many at least in my personal experiences, like this enterprise deals take forever. You feel like you have it in the door, this is coming definitely next quarter, this is definitely definitely ⁓ yeah, CISO on the ⁓ you know ⁓ speed dial, ⁓ the CISO is calling, you feel everything is coming, it never comes in though. It just takes a long period of time. There's a lot of decision makers, there are a lot of stakeholders, ⁓ a lot of security reviews, compliance reviews, everything you go through and then you get like I don't know, like fifty thousand dollars. ⁓ Speaker 1: It's it's funny, but ⁓ I that that you know that brings up some memories too. It's of the early days of Kennaslash Risk IO. We've kind of started in mid market for the most part, and partly because it's my background where I came from, from orbits and things like that. You know, they're they're not a large enterprise by any means. ⁓ and so we did that, but then ultimately ⁓ we kind of came to the realization with the mid market is it's it's it's a little bit of an easier sales kind of. The renewals are harder. It's not as big of a problem for them as we came to that realization that look, the bigger the company is, the more assets they have, the more vulnerabilities they have, the bigger of a problem this is. So we kind of went the the way of of enterprise. So we ran into that a lot, obviously. Speaker 2: And one thing I realize and you know, I don't know if you if you agree with this, but but it is in some ways it is easier to sell into mid market because they are the ones who have the least amount of bandwidth to do any of these any of these things. So ⁓ for them the problem is very acute. I don't have the money, I don't have the resources, I need help. And if ⁓ who like can I come say and you know, instead of focusing on this ten thousand, we give you this hundred and you have like one guy who's doing IT and security, he can probably do it in a weekend. You know, in some in some ways it is easier to do the mid market, ⁓ th because the enterprise deals, obviously the reviews and everything take a long period of time. Speaker 1: They do. They do. I will say though that with mid market, the the the tough part I think about the mid market is is sometimes, not always, the process takes almost as long, not quite as long as enterprise, but the deal value is significantly smaller. So the you know, it's just not as profitable in the in the end. And I've worked with Speaker 2: You get your logos but not much. ⁓ Speaker 1: ⁓ yeah. Speaker 2: ⁓ you know, all the case studies and whatnot, but not necessarily the revenue that you need to scale them. Speaker 1: Yeah. It's it's certainly helpful and it definitely in the early days you also want as many people using your product to get feedback, but then you run into entirely different issues when you're in the enterprise than you would at the mid market anyway. So Speaker 2: One of the best things Kenna did, in my humble opinion, is this concept of scoring, gamifying, giving you ⁓ trying to move customers ⁓ away from C V S, which in my humble opinion was always like the a rating of the technical severity of the vulnerability, not the risk of that vulnerable. You know what I mean? And you know, Kenna came in, hey, we're gonna give you a good score and it's gonna be between one two hundred, I think one or ten and then group your ⁓ you know assets in like which are which groups of assets are really bad, which groups of assets are decent enough and so on. I'm curious, but you know, it also came in my senses because I built this tourist thing at college and it was a struggle to get people out of C VSS. It was a big struggle to convince people to use a different score ⁓ from a different vendor that they had not tried before. But that was your ⁓ tip of the spear. Hey, we come in, we prioritize, we give you a better score and so on and so forth. I'm curious how does how did that land or or how was that transition telling people or customers, hey, you should use our score, not the C VSS? Speaker 1: Yeah. I think ⁓ actually convincing security practitioners, again, we're we're we're attracting at their in the early days, to your point, the tip of the spear, kind of the the the people that are more forward thinking b before the laggards and and everyone else, which is the the large portion of the market. but for us it was not that difficult to convince them that C VSS wasn't working because they're coming to us to say, hey, we're prioritizing by C VSS. And it says we have to fix, you know, half of our vulnerabilities and we've got two million of them. So I I can't fix a million vulnerabilities either. what what do you have? And then we would load their data in and then show the di the difference, but more importantly, show them the why. ⁓ and in and in our case it was, you know, hey, you're we're seeing tons of exploitation attempts on this particular vulnerability or or whatever the case might be. ⁓ so that Convincing the security practitioner was a little bit of work, but I wouldn't classify it as a lot. But then once we got in there, it's like, okay, hey, by the way, can you talk to our auditors, our regulators, all these people who are coming in and saying you have to fix everything that's a CVSS seven and above? ⁓ and so we would actually, we ended up having to produce like lots of content, lots of research ⁓ reports, how the scoring works, why it's better than C VSS, et cetera. You know, one of the things, or I I guess a couple of things there's ⁓ that that always bugged me about C VSS. One was to your point, it's it's severity, it's it's it's really kind of a how bad could this be? Speaker 2: Technical Civility. Speaker 1: And and the other thing is it's really subjective, right? You could you could hand somebody and say, here's here, go score this vulnerability with a C V S S score. Here's all the criteria. And you could hand it to five people and get five different scores. So it's it was ⁓ not great. Speaker 2: Did you did you like you know, once once these scores went out, did you have like a lot of success with like the executive teams in terms of like being able to gamify their organization and see and you know, basically drive drive down the risk? ⁓ Speaker 1: It that was definitely one of the things that I think di our product did really well at and actually kind of made things a bit more sticky ⁓ within these orgs as well is the aspect of there were there were I remember specifically actually getting a text from a CISO at one of our customers saying, Hey, I I just wanted to, you know, let you know that we finally got to and he had basically set this level of a score that he wanted all of his orgs to get to. And they finally got there, right? And they're having a big party and just texting me a pictures and stuff like that. And I was like, ⁓ this that's a great feeling, right? ⁓ so that that was super helpful. And I think it actually drove ⁓ it kept I don't know if it drove ⁓ new business, but it certainly drove renewals for us. ⁓ a lot of people really kind of stuck to that, which was great. Now there's There's difficulties in in that too. We s we s we found later on it's like, hey, actually, geez, you guys like we ran into orgs that were even tying bonuses to their their scores for their assets. And that's when it got a little concerning for me. ⁓ because I'm like, Hey guys, this I I I firmly believe in kind of ⁓ measuring things by risk and and not C V S S, et cetera, but keep in mind that risk is not entirely controlled by the team that's responsible for remediation, right? There's if a new campaign exploitation comes out this morning and suddenly a volume that they had that was low is now high, it's not their fault. They they need to react to it, but you they they shouldn't not get a bonus because of something like that, right? Speaker 2: Yeah, I mean once you have people's jobs on the line, then your product could become like the villain. Speaker 1: Yeah, exactly. And and and on intent does it's Speaker 2: I don't think well. You're just doing your job. You're just helping go. Speaker 1: It was written all over it. Yeah. Absolutely. Speaker 2: Were there any ⁓ like so you you mentioned that your go to market ⁓ started with like the main market, you went to enterprises and then ⁓ once you went to the service providers, like the secure works, the floodgates opened. Like, you know, they went in and they were like, Hey, we just can use this. Did you then pivot to the MSSP model or serving their service providers rather than enterprise? Like d were there any business challenges in scaling, Kenna? Because you know, you're a s small startup at that point of time and suddenly you have influx of I want this. Speaker 1: Yeah. Yeah. Absolutely. So a couple of things there. One is the SecureWorks business, they had some enterprise, but I would say there was a lot of mid market and even some SMB in there. ⁓ SMB's tough ⁓ in security. ⁓ churn rates are high. their budgets are obviously not not great. ⁓ and sometimes they don't even know that they have a problem. ⁓ I always I'm envious of the again going back to like Doug and John over at Duo, they had this great business that could go all the way down markets to SMBs and be successful. I don't think we really were that successful on the SMB front ⁓ at Kenna. ⁓ I would say for just in terms of scaling and challenges and things like that, I would say for the most part, we became an enterprise ⁓ security business for the most part, which has ⁓ we discussed, you know, some of the challenges there, including You know, you could have ⁓ a great ⁓ quarter, one quarter, and a terrible quarter the next all because of, you know, where procurement fell, whether it was the 31st or the first. ⁓ but the like the MSSP business, we did we were successful with some MSPs. I mentioned SecureWorks. It was a great kickoff start for us. We learned later on that The churn rates became really high within SecureWorks specifically. and we found out, you know, this was a business ⁓ where they were just they were selling lots of SecureWorks products. And in some cases, the customers didn't even really know that they had Kenna, right? So that makes it really difficult to renew them when they don't even know who you are. And the fact that we were also kind of shielded, like we couldn't necessarily directly interact with the customers, made that more difficult too. So there was challenges. Speaker 2: Yeah, you didn't have a personal connection with the customers. They didn't know you. You didn't know them. It was all handled to this ⁓ provider. One I don't know ⁓ if this was a good thing or not, but even though you were in business for a long period of time, you didn't have real competition. In the sense that like even though you had like you your playbook was open, everybody knew knew what you were doing, there was no real competition for a long period of time. Speaker 1: Yeah, we had like a couple a couple of of ⁓ folks out there at the time. They're very yeah, very they were smaller than we were. and then I'd say like the last couple of years, just before you know Cisco bought us, like the market just kind of exploded. All right. And and then there was just like and now it's even today I feel like there's a there's a million companies all doing that. Speaker 2: We have a funny anecdote to share with you is ⁓ you know so ⁓ so all the VM vendors were trying to sell VM and they were trying to sell VM and Kenna comes on along and you know let's say the VM vendors are charging a dollar per asset. And the perception in the market was Kenna is able to charge four dollars per asset. And the the executives are splitting their hair, like what is happening? Like we are doing all the hard work, we're scanning. you know, we're putting all this data together and everything. Kenna comes in, hey, just give us your API key. You know, we're going to charge you four dollars for the same asset. And that in some respects led to some of the competitive solutions from the VM vendors. Whenever we came out with Lumen, ⁓ COLIS is still ⁓ trying to get it out, the ETM is kind of like the closest ⁓ you know, that's like ten years later. ⁓ but it's amazing to see for such a long period of time, you know, you didn't have like a real competition. But as compared to now, like that space is flooded with VC funded startups. There is it is all over the place. Everyone wants to basically connect into your data, make it more agentic and whatnot, and and and and go from there. You touched upon something ⁓ that is near and dear to my heart is the Cisco, Kenna, acquisition, and ⁓ end of life. I'll I'll tell you why. Why it is near and dear to my heart. ⁓ in some respects, I got into in some respects I got into product management because of Kenna. So I was leading the research team at Tenebo. I was leading the research team at Tenable. And our mission, purpose of life was just ship more plugins. Ship more, f you know, go and find more things. Just go find more things. And ⁓ and you know, my research team, all the Nessus plugins, they would just go and find more things. And Kenna comes along and says, Hey, you guys, you need to you need to really prioritize. That's the real problem to solve. It's not the f detection, you know, we can detect all these things, but then you're just creating more work for the customers. You're not actually helping them get to the point. ⁓ And Tenable had this, you know, at in those days, Tenable had this ⁓ Innovation Week. you know, where engineers, researchers, product people, they would come together and they would craft a new feature and and and ship it into the product by end of the week. You come in, you get together, you plan two weeks in advance, then you you build the product and then you ship it by end of the week. And I call the feature caught with pants down. Literally, there is a blog there is a blog out there still on Terminal's website. So it's caught with pants down. There was literally these are the kinds of things like default password. Vulnerability is getting exploited by malware, vulnerabilities in the news, all those things. And then that feature won the ⁓ that that thing, that feature won the competition for that year. And I was like, I was so psyched. I was so psyched by that. And like, dude, I don't want to be doing research in engineering anymore. I want to be building product. That got me started into ⁓ building products. So Kenna, in some respects, is very near and dear to my heart. Like, you know, I valued, you know, you brought something to mark to the market that didn't exist before. Was ⁓ it was a need that customers wanted, but they never expressed. It's very hard to find out the product market fit when no one tells you, like, you know, if you went, for example, if you went to the people in the 1920s and 30s, hey, what do want? I just want a faster horse. But they don't say, hey, well, if you could give me a self-driving car, that would be awesome. And in some respects, Kenner did, hey, here's the self-driving car. You don't have to hire Speaker 1: Yeah, yeah. Speaker 2: Fire horses. You know what I mean? That was the equivalent of what that that was the equivalent of finding the problem and solving it better than anybody else. So when the Ken I was happy when the acquisition happened, you know, more resources, let's go. This is going to be awesome. But I felt like the product stagnated ev after six after the Cisco acquisition. Because I was I was on the outside integrating with Kenna. You know, at SaltStack and all these other companies I've been at, and I was integrating with Kenna. And dude, this is the same comp this is the same product. I I saw like four years ago. Nothing has changed. And then one fine day I realize I I read in the news, hey, can I zy? And it was a sad thing for me to see can I you know can I go Speaker 1: You and me both. You and me both. Yeah. So ⁓ I all of those same feelings, right? So we kind of got plugged in, started talking to Cisco as part of the acquisition, you know, one of the their pitches, which ⁓ is very attractive to a founder, is imagine your product now being plugged into thirty thousand sellers, right? So which is even way bigger than any of that secure works stuff that I talked about in early on. ⁓ and obviously I wanted to one of our goals in life was to get our product as into as many orgs as we possibly could. And Cisco has as big a reach as anybody out there. ⁓ so we got in there and you know, there there's lots of things that you expect that look, the this big company, you're not gonna move as fast as as you did as a startup, and and that's fine. That out you get out all the benefits kind of outweigh that. ⁓ but unfortunately for us, we you know, it's a kind of a combination of a few things. But one of the things was the the the sponsoring team that that actually did the acquisition inside of Cisco, within about six months of us being there, they had all peeled out and left and went to other companies. ⁓ so in some ways we were kind of orphaned in there and the the folks that we were there with probably didn't quite see the the the same value as as the sponsors did. And then combination with that, kind of the process of, you know, the sausage making inside is they start to instead of one plugging into thirty thousand sellers, which they kind of do, but they're generalists, they start peeling away your resources, right? They take your sales and they move them into the sales org. They take your marketing, they move them into the marketing org. They take your product people, start to distribute them out a little bit. ⁓ and you end up having a much smaller team that's dedicated to just Kenna. ⁓ So that was difficult. And then those 30,000 sellers, the vast majority of them are generalists and they don't know anything about you or even bulb management, right? So if a customer tells them that they want Kenna, great, they'll sell it. But they that's it, right? So you end up in a bad situation, or at least we did. ⁓ you know, some acquisitions do better than others. I th I my theory is the companies that were Honestly left alone the longest, did the best. You know, who knows. Speaker 2: What do you think was the cause for the ⁓ cause for the failure in terms of Cisco not seeing the value of Kenna? Was it like the lack of like the vision and executing the vision with Kenna? Because so I mean I I have a point of view on it, but I'm curious what do you think? Speaker 1: I think ⁓ largely they were there was a group of folks that were like, Hey, imagine if we take Kenna and then we combine it with they were building out and had a lot of resources that were kind of dedicated to XDR there. What if we also started to do like risk-based, taking that risk-based approach across our products, right? We're gonna do a risk-based X XDR, we're gonna do all these things where Look, we're we're flooding the the market with noise and alerts and things like that. What if we took this risk-based lens to all of these different things? And that was kind of the vision that was sold in as part of the the acquisition, from my understanding, at least, at least from the outside coming in. ⁓ obviously all of those people that were, including the product people that were kind of working on that, they they laughed, right? ⁓ and so there was just it was it was a really kind of Awkward and unfortunate situation. Speaker 2: Yeah. It's it's very unfortunate because they had something that was working. I mean, but they had to build new I I guess they would have had to build new scanners, mean new things to actually make it a full vision so that it is a it they they didn't have to depend on the colosses and the tenable. Otherwise, you know, if you're depending on the VM meters for the scanners and the signatures, then you're in a in a you you're in a very tough spot. Maybe they didn't want to make the investments. Speaker 1: That's right. Speaker 2: Let's switch gears. ⁓ you created this category, you know, from its inception. Where do you think ⁓ I mean a risk based VM has transitioned into exposure management, exposure management has become CTEM, CTEM is now agentic CTEM, ⁓ and there is like, you know, the the category keeps evolving. It's the same thing. It's the VM, RB VM, CTEM. It's like they just come up with a new name for the same category. Yeah. I'm curious, like, where does this go from here? Speaker 1: A great question. ⁓ obviously, so ⁓ I've since left Cisco, ⁓ co-founded another company called Empirical Security. Still in kind of that exposure management slash hygiene side of the house. ⁓ we have, you know, very strong opinions on on where this is going. Our core competency at Empirical is we are model builders, model trainers. ⁓ and we think exposure management as a whole, whether it's CVs and vulnerabilities or application security or config or cloud. All I I always think of security as kind of there's there's two big broad halves of the house, right? There's the the hygiene prediction prevention side of the house, and then there's the detection and response side of the house. ⁓ and and oftentimes you go into these orgs, they're very separate like that too, right? The the the the vault management team never talks to the SOC and never talks to whom you know. Whoever's managing the Sims, et cetera. but there's valuable data over on that side of the house that would really do well to feed the, the, the hygiene side of the house, right? So we come in and think, hey, this has become basically a machine scale problem. ⁓ you can't hire your way out of this. And obviously, the things in the news with Anthropic and Mythos and the all the doomsayers that are out there now about how security is coming to an end, ⁓ you have to basically use these machines to help on that that hygiene prediction prevention side of the house. So what we do is we build and train models that take data from all of your environment, including that kind of detection and response side of the house, to start to get ahead of this so that you can be proactive before those bad things happen. Our w we say internally, we don't mean this and God don't I I shouldn't even say this on a podcast, but we're our our job is, you know, to to put the detection and response out of business, but that's that's not the case. You you need both sides, but the more I can do on this side of the house, the less noise I'm gonna see on the right on the other side of the house. Speaker 2: One thing we one thing we didn't ⁓ talk about in your ⁓ Kenna journey is you also created or your team created the EPSS code, which is now getting ⁓ mentioned in the methos and the entropic things as the as the as the metric to ⁓ Basically chase and close vulnerabilities. I mean, you know, that ha that must feel epic. Like you know, you did something maybe 10 years ago. One day you wake up in the morning and you see EPSS flashed all over again. And you're like, what is this going on? ⁓ it's related to mythos. and obviously all the model work, model building, training work that you're doing. ⁓ there's one question I wanted to ask you. And, you know, I f my felt the mythos, the whole mythos thing farcical in the sense that, in the sense that they didn't get the defenders. into the program. They didn't get the responders in the program. They had a bunch of infrastructure players like Nvidia in the program. And I'm like, what are what is NVIDIA doing? I mean great company. I I have nothing against them. Nothing against that. Speaker 1: They they they probably ⁓ have an investment in anthropic, would be my guess, but Speaker 2: Regardless w of what you're doing. I mean let's say you you come out and say, hey, we found this new ⁓ you know, we trained this new model and it can find ten thousand new vulnerabilities. Who's gonna help them detect it? Who's gonna help them remediate it? I feel like there's no consideration. There's no consideration for the the people who are actually going to be tasked to do this work as soon as your model drops. I f I I thought it was like odd. What's your w I'm curious what do you what do you think about that? Speaker 1: So I think there are some people that are in the project glasswing which ⁓ make total sense. It's like ⁓ the mystery. Yeah, they but they have they're gonna find a lot of vulnerabilities and hopefully develop a lot of patches prior to it being released so that you don't have a bunch of zero days on your hands. ⁓ at least you have the ability to fix that as a practitioner. There are others that were, you know, I think Oracle was left out, right? I Speaker 2: Well then. Yeah, makes sense too. Speaker 1: There is an a an awful lot of vulnerabilities in ⁓ across the the Oracle landscape, right? So ⁓ Speaker 2: Ask tenable. They didn't ask Tenable. They didn't even ask callers. You know, I there is narrative in the industry. VM is dead. Cybersecurity companies are dead. And my counter to that is these guys are going to be so busy over the next 40 years. They're going to be so busy finding these things. Maybe they're dead in two years, but not now. Now is not the time. Now is the time to just make more, you know, Speaker 1: I had to guess, if I had to guess on what and this is totally outside and having absolutely no knowledge whatsoever of what what's going on in the inside of Anthropic, but ⁓ you know, not inviting Qualest, not inviting Tenable, th if you think about what their model is doing, their model is finding vulnerabilities, which is the same thing that Qualus and Tenable do. So why am I gonna invite these guys who are my future competitors to create detections for this? I'll I'll be the I'll be the model that detects all this stuff, right? Speaker 2: Yeah, and maybe or maybe ⁓ there is some level of regulatory capture that is going on. You know, we are the safest we are the safest model, trust us to do all these right things and then and then go from there. Speaker 1: We could speculate for hours. Speaker 2: ⁓ there is one ⁓ you know, you recently wrote a ⁓ on the topic of BM is dead, cybersecurity companies are dead, all these, you know, narratives, fake narratives that are going on. But I I found one of your recent blogs to be interesting. You said AI is eating your scan. AI is eating your scanner. There is some truth to it. I didn't believe it first, but now I see it. There is a chance. There is a chance where the scanner can be eaten by AI in some respects. Not everything, but there are some aspects of it where the scanner can do or the AI scanner or a L L ⁓ powered scanner can do a much better job than the rule-based scanners. of the of the past, where they can reason with the data, when they're probing a port, they can see the response, think on the response, probe it differently, rather than writing like a bunch of signatures, that are hardcore. Speaker 1: Yeah. And I can and I can write the exploit and sometimes write the fix depending on what I have access to, right? So if I'm like at the source code level, I can probably, you know, we we've seen this as well. Like static analysis tools are running into LLMs quite a bit now too, right? Where they're ⁓ you have the LLMs that are detecting the vulnerabilities in your own code and then issuing pull requests to fix those vulnerabilities. ⁓ I think we'll see more and more of that. I think there's There's certainly areas where I'm not I don't know when or if we'll ever replace it. There's really unique environments. There's network devices which are complicated or IoT and some of this other stuff. But it's I just feel like the the models themselves are gonna be get better and better at detecting more and more. ⁓ which again gets back to stage zero of the problem all over again, right? You you thought you had a lot of vulnerabilities before. Now imagine how many vulnerabilities you have, right? It's just ⁓ it's I I I I no pun intended, but it's it's an untenable situation for these folks, right? Speaker 2: Do you think do you think these a new do you think the new version of these companies come to life? To new iteration, like you know, like because the very h if you have been in this industry for twenty years, it's very hard to evolve to the new AI stack. You know, re architecting your entire product, re architecting your Speaker 1: I think they have Speaker 2: You know, first of all going back to your customers and telling them, Hey, by the way, we are going to be much more heuristic now than than deterministic. You know, there are some things we might just make up that so ⁓ you know so Speaker 1: You don't want your vones I mean, I w I was about to say you don't want your vulner scanner to be hallucinating, but some would argue there's already a lot of hallucination going on as well. Speaker 2: Because false positives have been there ⁓ for ⁓ forever. You know, I I don't think maybe there is like a confidence score when the LLM comes out and says, Hey, you know, maybe I'm wrong on this, but there is a good chance this can be exploited. and the last the the c ⁓ the thing I want to touch upon is where do you think the value is concentrating? This that's w one of the last things that you were saying in your blog where ⁓ you know, the value is not concentrating in the detection. That's a solved problem. Detection is a solved problem. But the value is going to concentrate concentrate in some other part of the business. Speaker 1: Look, I I still don't think that prioritization is a solved problem today. ⁓ especially as ⁓ we discover more and more and more. ⁓ I I hear a lot of arguments of yeah, but we're also gonna use these these same technologies to fix more and more and more. And I think that's only partially true. ⁓ I think we're gonna get a a bigger volume of things. I think we'll get a bigger volume of fixing things too, because everything is becoming more and more machine scale. And I think everything is also becoming faster, right? Time to discover, time to exploit, and time to fix will all become faster. ⁓ what I don't think we're s gonna see is like, ⁓ you know, there's a lot of Doomsayers because of the Mythos announcements and things like that that, ⁓ you know, they're They're gonna be attacking everything. There's gonna be zero days everywhere and we're all just going to lose and there is no security. You know, the the the security nihilists are back in full force. But ⁓ I don't think you're going to see necessarily a n the same level of increase in terms of the number of ⁓ attackers and certainly threat actors than we saw before, right? I don't I'm not going to start attacking people just because it's easier, right? It's the same reason why, you know, the having access to certain things doesn't make I'm not going to become a criminal tomorrow because LLMs make it easier for me to become a criminal, right? But there are going to be things that are more automated on the attack side as well. So you will see a volume increase. I don't think it will be at the material level that we will see in terms of the number of vulnerabilities, et cetera, and the number of amount of exploit code that's out. Speaker 2: Do you think these ⁓ do you think these ⁓ crutches that we have been using in the past, like ⁓ s you know, C V S, CSACav, EPSS, these are all proxies for exploitation. Like, you know, we don't know, you know, what to focus on, so let's use these as a as a way to basically guide ourselves in terms of prioritization. Do you think these will be relevant or would it be more on the sense that exploitable by AI or non exploitable by AI? Like those are the two real metrics that we really care about and then we go from there. Speaker 1: So so to be clear, the I am biased in the fact that we we at Empirical are the the trainers of EPSS. So we we still believe in that, but to your point, that's just one piece of the puzzle, right? And part of in order to kind of measure likelihood of something happening on the exploit side, I might also want to be able to do things like, well, how easy Is it for this to be exploited by an LLM? That should be a factor in my overall prioritization, right? So am I doing that as part of ⁓ example at empirical? Am I training models that include data like that? Right. All the to me, there's a everything is a data point, and I basically need to pull all that together and then use a machine to ultimately understand what's the likelihood of this happening to me and what's the likelihood of this happening to me soon. Speaker 2: ⁓ Ed, this has been this has been a fun interview. What's the next big thing dropping from Ed Ballis? Speaker 1: Well ⁓ it it's ⁓ hopefully l a whole lot of empirical security ⁓ is is coming out. So Speaker 2: And less fear mongering, less fear mongering, ⁓ Speaker 1: But yeah, th that's ⁓ that's goes without saying. I've I've I've ⁓ been all the way going back to putting my CISO hat on twenty plus years ago is like ⁓ the the fear, uncertainty and doubt thing. I I I hope it's coming to an end. It I I see signs of it coming to an end. It's certainly ⁓ more data driven than we've ever been before. I but in fact I remember back in the old days when I was at orbits, everybody all of my peers are complaining there's just not enough data out there for us to be able to make a decision. Now it's the opposite, right? There's so much data that everybody's just flooded and overwhelmed with it. So these are all good problems to have, and I think we can solve.