Latest / Expert Insights Podcast / #80. AI, Automation, And Risk-Prioritized Intelligence: A New Era Of Pentesting (Eoin Keary, EdgeScan)
Transcript
- 0:09SPEAKER_03: Hello and welcome to the Expert Insights Podcast,
- 0:12SPEAKER_03: your go-to source for cybersecurity insights from industry experts.
- 0:17SPEAKER_03: Is traditional pen testing enough to catch modern cybersecurity risks?
- 0:21Does your team need to be thinking about new ways to catch
- 0:23critical vulnerabilities?
- 0:25I'm your host,
- 0:26Joel Witts,
- 0:27and today our guest is Owen Keery,
- 0:29the CEO and founder at EdgeScan.
- 0:32SPEAKER_03: We're going to be talking about the limits of traditional pen testing,
- 0:35SPEAKER_03: how to defend against new and unknown attacks,
- 0:38and why businesses should be thinking about risk-prioritized
- 0:41vulnerability
- 0:41SPEAKER_03: intelligence.
- 0:43SPEAKER_03: Let's get into it.
- 0:44SPEAKER_03: Owen, it's great to have you on the Expert Insights podcast.
- 0:47SPEAKER_03: Thank you so much for joining us.
- 0:48SPEAKER_00: Thanks.
- 0:49It'll be great if we could start with just an introduction from
- 0:52yourself to your
- 0:53SPEAKER_03: background and kind of how you got into cybersecurity.
- 0:56SPEAKER_00: Yeah, sure.
- 0:57SPEAKER_00: My name is Owen Kiri.
- 0:58SPEAKER_00: I'm the CEO and founder of Edgecan.
- 1:01SPEAKER_00: I've been involved in cyber for about 20 plus years.
- 1:06I used to lead the pen testing team for Europe for a big four
- 1:13prior to founding Edgecan.
- 1:15So I've been everything in my career from software developer to
- 1:20pen tester and
- 1:20hacker type of guy to software security person and currently
- 1:25sort of CEO type of
- 1:28SPEAKER_00: guy,
- 1:28SPEAKER_00: which has many different fingers and many different pies.
- 1:33SPEAKER_00: So, yeah.
- 1:34SPEAKER_03: Fantastic.
- 1:34And it'd be great to kind of dig into the story of what led you
- 1:38to found EdgeScan.
- 1:39What were the sort of key challenges that you identified and
- 1:42what were you trying
- 1:43SPEAKER_03: to help teams to fix?
- 1:45SPEAKER_00: Yes.
- 1:46SPEAKER_00: Prior to being an edge scan,
- 1:52SPEAKER_00: the challenge we were faced with was delivering high volume,
- 1:59SPEAKER_00: continuous penetration testing.
- 2:02SPEAKER_00: At the time,
- 2:03SPEAKER_00: and it's still accelerating,
- 2:07SPEAKER_00: it's the way software is developed,
- 2:10SPEAKER_00: everything from software methodologies to deployments on clouds.
- 2:14So the idea that software is changing rapidly all the time,
- 2:18organizations are deploying quicker.
- 2:21The traditional approach to offensive security or penetration
- 2:25testing were to do a pen test.
- 2:28SPEAKER_00: Pen tests are great in terms of accuracy.
- 2:31SPEAKER_00: If you have a decent company delivering the service, it's human driven.
- 2:37SPEAKER_00: But there are challenges when you get to a high volume,
- 2:40SPEAKER_00: high scale or continuous or frequent assessments.
- 2:45SPEAKER_00: And in modern society, we need frequent assessments.
- 2:50SPEAKER_00: we need possibly continuous, we need accuracy.
- 2:55And to do that with traditional approaches such as humans doing
- 2:59it on their own
- 3:01SPEAKER_00: results in a very expensive,
- 3:04SPEAKER_00: rather slow,
- 3:05SPEAKER_00: and it's pretty much impossible to deliver continuous testing.
- 3:10So we built Edgecan to address the idea of a client having the
- 3:15ability to assess
- 3:16SPEAKER_00: either tens or thousands
- 3:18SPEAKER_00: of web applications,
- 3:20SPEAKER_00: cloud or network endpoints,
- 3:22SPEAKER_00: or even APIs,
- 3:24SPEAKER_00: and to be able to deliver this in a continuous way,
- 3:29SPEAKER_00: but also be assured of accuracy.
- 3:31So,
- 3:31you know,
- 3:32the EdgeGAN mantra internally is scale massively,
- 3:36but accuracy can never suffer.
- 3:39SPEAKER_00: So EdgeCon has always been built around the ability to provide,
- 3:43SPEAKER_00: in effect,
- 3:44SPEAKER_00: continuous constant validation,
- 3:48SPEAKER_00: constant offensive security.
- 3:50SPEAKER_00: But when our clients receive alerts or vulnerability intel,
- 3:54SPEAKER_00: they can be reassured that they're real.
- 3:56So we don't have any false positives,
- 3:59which is sort of a very important part of what we do.
- 4:02SPEAKER_03: EDGCAN recently released its 2025 vulnerability statistics report.
- 4:07I'd be really interested to know what your key takeaways were
- 4:10from the report and
- 4:11what it tells us about the state of the vulnerability
- 4:14management landscape.
- 4:16SPEAKER_00: Yeah, we're still finding vulnerabilities.
- 4:19SPEAKER_00: So,
- 4:19SPEAKER_00: yeah,
- 4:19SPEAKER_00: the report is based,
- 4:20SPEAKER_00: just to context the report,
- 4:22SPEAKER_00: what the report is,
- 4:23SPEAKER_00: it's based on the previous 12 months of assessments.
- 4:27SPEAKER_00: Last year, we did north of 2000 penetration tests.
- 4:31We did hundreds of thousands of assessments in total across
- 4:36lots of verticals,
- 4:37SPEAKER_00: everything from energy to banking to media to government.
- 4:43um and it's a snapshot of all the vulnerabilities we found
- 4:47everything because we
- 4:48could we can do the support in such a way because it's all the
- 4:51vulnerability data
- 4:51is validated so we're pretty happy that it's real uh it's
- 4:55decent even though
- 4:56statistics and lies it could be very uh you know similar things
- 5:00in our industry but
- 5:01SPEAKER_00: um we're pretty happy with that but you know key takeaways um
- 5:06We're still seeing lots of the same vulnerabilities year on
- 5:13year in terms of
- 5:13SPEAKER_00: critical and high severity issues.
- 5:15SPEAKER_00: A lot of these are very preventable.
- 5:20And I think a lot of the cause of that primarily is that people
- 5:26aren't testing what
- 5:28SPEAKER_00: they should be testing.
- 5:29Because once you have a system with some of those
- 5:32vulnerabilities present,
- 5:34SPEAKER_00: it's not very difficult to find them.
- 5:36um so so so in some ways people aren't testing as frequent as
- 5:41they should be or
- 5:41they may be not testing the right things i think one of the
- 5:44takeaways is to
- 5:45understand your landscape as i said you know the very top of
- 5:48this call um you can't
- 5:51SPEAKER_00: secure what you can't see and i i i think it's not rocket science to to
- 5:59SPEAKER_00: to have a solution that provides visibility across your landscape.
- 6:03SPEAKER_00: But now,
- 6:04SPEAKER_00: you know,
- 6:04SPEAKER_00: the last few years,
- 6:05SPEAKER_00: we call this thing called attack surface management,
- 6:07SPEAKER_00: which ASM,
- 6:08SPEAKER_00: which is all of a sudden a big deal.
- 6:11But it's something that it was sort of a core,
- 6:14a keystone aspect of security for 20 years.
- 6:20SPEAKER_00: But I think visibility comes first.
- 6:23SPEAKER_00: I think frequent testing as well is very important.
- 6:27And I think a lot of the vulnerabilities we're finding in the
- 6:30stats report are
- 6:32results probably of systems not being tested enough or people
- 6:42maybe not even
- 6:42SPEAKER_00: knowing that the system was there.
- 6:44SPEAKER_00: And then when they got it tested, those vulnerabilities were there.
- 6:46SPEAKER_00: I think that's the first piece of it.
- 6:50SPEAKER_00: other aspects, you know, of the stat support.
- 6:54SPEAKER_00: You know,
- 6:55SPEAKER_00: it's interesting as well,
- 6:56SPEAKER_00: I think,
- 6:56SPEAKER_00: is that a lot of ransomware attacks and cybercriminal attacks,
- 7:00SPEAKER_00: they're leveraging...
- 7:02SPEAKER_00: They're leveraging vulnerabilities which are not particularly sexy.
- 7:06SPEAKER_00: That's the truth.
- 7:09And some people say ransomware is always about clicking an
- 7:12email or a phishing
- 7:15SPEAKER_00: attack,
- 7:15SPEAKER_00: clicking an email,
- 7:16and then something loads into your laptop and spreads into your
- 7:18internal network or
- 7:19SPEAKER_00: whatever.
- 7:20SPEAKER_00: A lot of ransomware attacks,
- 7:21SPEAKER_00: though,
- 7:22SPEAKER_00: can also be used by maybe having an insecure VPN,
- 7:25SPEAKER_00: something as simple as that,
- 7:26SPEAKER_00: an exposed database,
- 7:27SPEAKER_00: an exposed port or something.
- 7:32SPEAKER_00: And I think that's not discussed enough.
- 7:34And it's something that if you look at the report,
- 7:37it maps vulnerabilities to ransomware.
- 7:40It maps vulnerabilities to other ways of looking at
- 7:44vulnerability data as well,
- 7:45SPEAKER_00: such as the CESA-KEV and EPSS scoring.
- 7:50SPEAKER_00: You know,
- 7:51looking at those different ways of taxonomies and methodologies
- 7:55for looking at
- 7:56SPEAKER_00: vulnerabilities,
- 7:56SPEAKER_00: you'll see that sometimes a vulnerability can be rated,
- 8:01SPEAKER_00: you know,
- 8:01SPEAKER_00: in the traditional sense,
- 8:03SPEAKER_00: which is called CVSS,
- 8:04SPEAKER_00: can be rated as a high,
- 8:07SPEAKER_00: but the exploit predictability is low,
- 8:11SPEAKER_00: or it isn't on this catalogue called the C-Secure,
- 8:16SPEAKER_00: which is maintained by the US government.
- 8:18SPEAKER_00: Right.
- 8:18so so you need to you need to pull um severity information from
- 8:23multiple sources in
- 8:24order to make a decision i think if you pull severity
- 8:27information as if it's a high
- 8:28risk but it has a low epss score and it's on the cc kev or it's
- 8:33not on the cc kev i
- 8:34think you need to combine those things in order to make a
- 8:37decision on what issues
- 8:38you should be fixing um so i think i'm a bit of maturity in
- 8:42terms of prioritization
- 8:44SPEAKER_00: you can't
- 8:45um you can't make a decision on priority of a vulnerability um
- 8:51apart from the fact
- 8:51that where it is what's it what's it sitting on how important
- 8:54is the way it's the
- 8:55system the vulnerabilities on but but if all of that was the
- 8:58same it's very hard to
- 9:00to to make a decision on which which of these will be used
- 9:03against me which these
- 9:04SPEAKER_00: vulnerabilities will be used against me if um
- 9:09SPEAKER_00: If you're relying on one source of severity,
- 9:12SPEAKER_00: so multiple sources of severity,
- 9:14SPEAKER_00: I think is very important,
- 9:15SPEAKER_00: which is something that's coming along reasonably well in the industry.
- 9:18But I think if you're not doing it,
- 9:20you should definitely have a look at that.
- 9:22So you mentioned,
- 9:23you know,
- 9:23the one recommendation there about looking at multiple sources.
- 9:25SPEAKER_03: I wonder,
- 9:25do you have any other recommendations for security teams
- 9:27looking at the report and
- 9:28SPEAKER_03: thinking,
- 9:29SPEAKER_03: you know,
- 9:29SPEAKER_03: how can we protect ourselves better against some of these issues?
- 9:33SPEAKER_00: Well, it's simple.
- 9:34SPEAKER_00: There's a couple of things.
- 9:36None of it's groundbreaking,
- 9:39but it ultimately is to set up a continuous visibility system.
- 9:47SPEAKER_00: Once you have that, it's also then...
- 9:51SPEAKER_00: Everybody has lots of vulnerabilities.
- 9:54SPEAKER_00: Some of them are more important than others,
- 9:56SPEAKER_00: and that changes over time in terms of how severe they are.
- 10:00But the idea is to maybe go after maybe specific types of
- 10:04vulnerability and do it
- 10:07SPEAKER_00: in a continuous way.
- 10:10SPEAKER_00: But more than that,
- 10:11something we've done in our AI Insights product is we analyze
- 10:16the vulnerability
- 10:16SPEAKER_00: data our clients have.
- 10:18And what we do then is we start to recommend that certain teams
- 10:22or certain business
- 10:23SPEAKER_00: units within the organization,
- 10:25because they own the applications and we have all that data in
- 10:27the platform,
- 10:29we recommend that such and such a team should probably do some
- 10:33training in secure
- 10:35SPEAKER_00: application development in a very focused manner.
- 10:39SPEAKER_00: So preventative security is still important.
- 10:42SPEAKER_00: So it's like problem-based training.
- 10:44SPEAKER_00: If we can show you all the weaknesses that you have,
- 10:47SPEAKER_00: and then we can also let you know that these weaknesses keep reoccurring,
- 10:50SPEAKER_00: because we keep detecting them across one system or a million systems,
- 10:58SPEAKER_00: Maybe that's where your development team should focus on.
- 11:01Maybe that's where your cyber team should try and eradicate
- 11:05that type of vulnerability.
- 11:07I think leveraging analytics and looking at training from the
- 11:13point of view of the
- 11:15SPEAKER_00: vulnerabilities you have in the real world,
- 11:20SPEAKER_00: as opposed to delivering generic security training,
- 11:22SPEAKER_00: which people tend to switch off about.
- 11:24SPEAKER_00: Yeah.
- 11:25SPEAKER_00: Yeah,
- 11:26SPEAKER_00: I think pointed focus,
- 11:27SPEAKER_00: problem-based training based on your vulnerability data,
- 11:30SPEAKER_00: I think is very powerful.
- 11:32SPEAKER_00: I don't see enough of it,
- 11:34but I think something like that could move the dial for a lot
- 11:37of organizations.
- 11:38SPEAKER_00: Fantastic.
- 11:39SPEAKER_03: Good stuff.
- 11:40SPEAKER_03: Okay,
- 11:42so I wanted to talk about an article that you wrote for Forbes
- 11:45back in January,
- 11:46which was on this topic of the evolution of penetration testing
- 11:50and the movement
- 11:50towards penetration testing as a service and continuous threat
- 11:56exposure management.
- 11:57I wonder if you could talk through what are some of the
- 11:59limitations of traditional
- 12:01SPEAKER_03: pen testing and why is this shift to...
- 12:04SPEAKER_03: All right,
- 12:07SPEAKER_00: so it comes down to,
- 12:09SPEAKER_00: how would you say,
- 12:11SPEAKER_00: it's momentum,
- 12:12SPEAKER_00: it's acceleration,
- 12:14SPEAKER_00: it's the idea that
- 12:16SPEAKER_00: You know,
- 12:16SPEAKER_00: when I started doing security many,
- 12:18SPEAKER_00: many years ago,
- 12:19SPEAKER_00: most of my staff were probably children when,
- 12:22SPEAKER_00: you know,
- 12:22when I started in cyber and software was built in a different
- 12:26way and deployment
- 12:28SPEAKER_00: speed was slower and we didn't have a thing called a cloud really.
- 12:31We had,
- 12:32you know,
- 12:32we had hosting environments,
- 12:33but not cloud and we couldn't deploy rapidly.
- 12:36We didn't have the idea of iterative development where we're
- 12:39deploying daily or,
- 12:40SPEAKER_00: you know,
- 12:41SPEAKER_00: agile development or whatever you want to call it,
- 12:42SPEAKER_00: where you're deploying all the time.
- 12:44SPEAKER_00: and updating our application.
- 12:46SPEAKER_00: It was very monolithic and waterfall based.
- 12:49SPEAKER_00: And so at that time, a penetration test was a good thing to do.
- 12:53SPEAKER_00: So the idea is company,
- 12:55SPEAKER_00: a group of people go off for six months,
- 12:57SPEAKER_00: build code,
- 12:58SPEAKER_00: deploy code before its deploy gets pen tested.
- 13:01SPEAKER_00: and maybe it gets another one once a year for compliance.
- 13:04That code doesn't really change a whole lot until the next big
- 13:07deployment in six months time.
- 13:10Now,
- 13:10it's like organizations are deploying every day,
- 13:13every hour,
- 13:14every month,
- 13:14whatever.
- 13:15SPEAKER_00: Every time you introduce new code,
- 13:18there's a possibility of introducing more risk because it could
- 13:21be a bug and that
- 13:22SPEAKER_00: bug can manifest itself into a vulnerability.
- 13:25um so so pentesting worked at the time because it suited the
- 13:30way the world is but
- 13:31it doesn't work anymore um so we had to do something different
- 13:36so what do we do we
- 13:37built a whole industry on automated scanners automated scanners
- 13:41could do it they're
- 13:42cheap you could do it they can run 24 7 they get it wrong they
- 13:46produce quite a lot
- 13:47SPEAKER_00: of noise a lot of false positives but they do something and
- 13:51and that worked as well apart from the poor people that had to
- 13:55weed out the noise
- 13:56and figure out what was real and that most people that do that
- 13:59for long enough
- 14:00won't be working for you you know it's a very stressful thing
- 14:04it's very frustrating
- 14:05thing but we were conditioned as well for a long time in terms
- 14:09of accepting that
- 14:11all scanners produce noise false positives and and and you know
- 14:15it's fine it but
- 14:16SPEAKER_00: If in any other walk of life,
- 14:18SPEAKER_00: in any other sort of discipline of engineering,
- 14:20SPEAKER_00: you probably wouldn't accept that.
- 14:22Like if we built bridges and 40% of them fell down within the
- 14:26first month,
- 14:27SPEAKER_00: we probably would look at it and say,
- 14:28SPEAKER_00: we're not doing this right.
- 14:29SPEAKER_00: But in cyber, it seems to be acceptable.
- 14:33SPEAKER_00: So the idea of,
- 14:34SPEAKER_00: you know,
- 14:35SPEAKER_00: the evolution then is,
- 14:37SPEAKER_00: okay,
- 14:37SPEAKER_00: well,
- 14:37SPEAKER_00: let's use automation to find things that we know it's good at finding.
- 14:41We call those our technical vulnerabilities,
- 14:43vulnerabilities that you can find using technology,
- 14:45SPEAKER_00: you can confirm very easily,
- 14:48SPEAKER_00: even a lot of AI,
- 14:49SPEAKER_00: fantastic companies do this,
- 14:51SPEAKER_00: but you don't actually need AI to do it,
- 14:53SPEAKER_00: is you need strong rules and validation.
- 14:55SPEAKER_00: Automation is perfect for that.
- 14:57It can find vulnerabilities that you could confirm technically
- 14:59that they're an issue.
- 15:02SPEAKER_00: But then we have another sort of strand of vulnerability,
- 15:05SPEAKER_00: which is the business logic,
- 15:08SPEAKER_00: the logical vulnerability.
- 15:09SPEAKER_00: And logical vulnerabilities are unique per application,
- 15:13SPEAKER_00: per system,
- 15:13because every application and every system and every API is
- 15:16unique and different.
- 15:18SPEAKER_00: And it takes a bit of human expertise to do that.
- 15:22SPEAKER_00: So the idea would be,
- 15:24SPEAKER_00: well,
- 15:24can we use clever technology to find all the technical issues
- 15:28on a continuous way?
- 15:30SPEAKER_00: And then can we somehow...
- 15:33SPEAKER_00: Get the human,
- 15:35SPEAKER_00: or in this case in Edgecan,
- 15:36SPEAKER_00: there's a sprinkle of AI,
- 15:38SPEAKER_00: there's some human aspects,
- 15:40SPEAKER_00: there's some cyber analytics.
- 15:42SPEAKER_00: Can we use that to dig deeper on demand and look at specific things,
- 15:47which means the human only needs to look at the complex stuff
- 15:51that automation can't
- 15:52SPEAKER_00: do well.
- 15:54SPEAKER_00: And that for me is the evolution of PTAS.
- 15:57It's not automated penetration testing,
- 16:01but a lot of the penetration test is automated anyway.
- 16:04SPEAKER_00: A lot of the penetration test is around scanning,
- 16:09SPEAKER_00: finding stuff,
- 16:11SPEAKER_00: and then it's handed over to the human to exploit.
- 16:15SPEAKER_00: But the idea with PTAS is let's use automation where it's good,
- 16:21SPEAKER_00: but there is a subset of very critical vulnerabilities like authorization
- 16:25vulnerabilities and authentication or maybe some way of
- 16:29breaking a wizard and
- 16:30SPEAKER_00: stealing,
- 16:31moving floating points or stealing money or inflating the price
- 16:34or stacking coupons
- 16:35SPEAKER_00: when you're buying something online,
- 16:37SPEAKER_00: whatever it may be.
- 16:38SPEAKER_00: And all of those things,
- 16:40SPEAKER_00: automation doesn't even know what a scanning,
- 16:41SPEAKER_00: it doesn't know it's a pizza shop application,
- 16:43or it's a high network banking application doesn't know that a
- 16:47human does.
- 16:49SPEAKER_00: So the evolution of p tests is is to be able to give a deep testing,
- 16:53or try and keep keep pace with with the speed of development
- 16:58and the speed of
- 16:59SPEAKER_00: change.
- 16:59SPEAKER_00: And I think that's where p how p tests has evolved.
- 17:03SPEAKER_03: Yeah, absolutely.
- 17:04And you mentioned in the article about how organizations can
- 17:07get started and your
- 17:08SPEAKER_03: recommendation was around mapping,
- 17:10SPEAKER_03: mapping the attack surface.
- 17:12SPEAKER_03: Yeah,
- 17:12it'd be great if you could dig into a bit more how
- 17:14organizations can kind of,
- 17:15SPEAKER_03: I guess,
- 17:15SPEAKER_03: get up and running with p-testing and,
- 17:17SPEAKER_03: you know,
- 17:17SPEAKER_03: continuous threat exposure management.
- 17:19SPEAKER_00: Yeah, yeah.
- 17:21SPEAKER_00: So as a term,
- 17:23SPEAKER_00: continuous exposure manager,
- 17:24SPEAKER_00: CTEM,
- 17:26SPEAKER_00: you know,
- 17:26SPEAKER_00: it's also called proactive security by other analysts.
- 17:30It's the idea that,
- 17:32you know,
- 17:32firstly,
- 17:32what you need to do is understand scope.
- 17:34And there are simple tools,
- 17:37free tools like Nmap,
- 17:39you know,
- 17:39they may not be
- 17:42SPEAKER_00: specifically built for this, but they can certainly be used.
- 17:45SPEAKER_00: There's lots of ways of doing it.
- 17:46SPEAKER_00: You don't have to use EdgeGAN,
- 17:47SPEAKER_00: but we'd obviously love you to do that,
- 17:50but there are tools you can use to map what's on every IP in
- 17:57your CIDR range,
- 17:58SPEAKER_00: in your IP space across your internet,
- 18:01SPEAKER_00: what's there,
- 18:02SPEAKER_00: what's exposed,
- 18:03SPEAKER_00: what services are exposed.
- 18:05SPEAKER_00: And you can do that.
- 18:06SPEAKER_00: Automation is great at that.
- 18:07SPEAKER_00: Perfect.
- 18:08SPEAKER_00: And it's step one to understanding what do I have exposed?
- 18:12SPEAKER_00: Maybe everything I find shouldn't be exposed.
- 18:14SPEAKER_00: What do I do?
- 18:16SPEAKER_00: I turn off.
- 18:17SPEAKER_00: I turn on firewall rules.
- 18:19SPEAKER_00: I take the system off of the external exposure.
- 18:24SPEAKER_00: You know, I sort of do that type of thing.
- 18:25SPEAKER_00: Again, not rocket science.
- 18:27and then doing that continuously so as your development team or
- 18:31a third-party
- 18:31development team is building code is doing stuff for you or an
- 18:35administrator just
- 18:36screws up a firewall rule or something um you'll detect that
- 18:40within four or five
- 18:41hours of it happening right you know depending on the frequency
- 18:43of of how you're
- 18:46how you're how you're doing this um and it sort of gives you an
- 18:51additional pair of
- 18:52SPEAKER_00: eyes um
- 18:53And you'll find that an awful lot of breaches and horror
- 18:57stories are in relation to
- 19:00something that's been sitting out there for a while and people
- 19:03didn't do anything
- 19:03SPEAKER_00: about it,
- 19:04SPEAKER_00: right?
- 19:04SPEAKER_00: Or didn't know about it.
- 19:05SPEAKER_00: And maybe if they knew about it, they would have done something about it.
- 19:09SPEAKER_00: And then moving on from that,
- 19:10SPEAKER_00: if you can identify that,
- 19:11SPEAKER_00: well,
- 19:11SPEAKER_00: then,
- 19:11SPEAKER_00: yeah,
- 19:11SPEAKER_00: look,
- 19:12SPEAKER_00: simply having an automated scanner...
- 19:16Out of the box,
- 19:16it'll probably produce tons of noise and won't be very useful.
- 19:20SPEAKER_00: So take some time,
- 19:21SPEAKER_00: maybe only focus on certain categories of vulnerability,
- 19:25SPEAKER_00: maybe try and tune it down,
- 19:26SPEAKER_00: you know,
- 19:26because most scanners give you the ability to tune a little bit
- 19:31and to reduce and
- 19:32SPEAKER_00: increase various aspects of what it does.
- 19:35SPEAKER_00: maybe do that as well.
- 19:37SPEAKER_00: So what you have then is you've continuous visibility and then you can do
- 19:41SPEAKER_00: continuous vulnerability detection across the same estate,
- 19:45SPEAKER_00: right?
- 19:46SPEAKER_00: And then moving from that,
- 19:48SPEAKER_00: you know,
- 19:48SPEAKER_00: the idea,
- 19:49SPEAKER_00: all right,
- 19:49SPEAKER_00: when you're getting into the PTAS space,
- 19:51SPEAKER_00: obviously engaging with a traditional consultant would be expensive.
- 19:55SPEAKER_00: It isn't really PTAS.
- 19:56SPEAKER_00: It's contract statement of work, time and materials.
- 20:00You know,
- 20:00in our PTAS model,
- 20:01for example,
- 20:02you don't have to pay for retesting.
- 20:06uh you know time and materials over consultant model will be
- 20:09that you have to pay
- 20:10for a retest where where with edgecan you can have as many
- 20:13retests i know there are
- 20:14other companies that will give you a limited amount of
- 20:16retesting but for edgecan
- 20:18it's unlimited so you can retest until you get it fixed if you
- 20:22can't fix it
- 20:22properly the first time and we retested and we find a different
- 20:25way to exploit the
- 20:26SPEAKER_00: issue go get it fixed and retest it again it doesn't we don't care um
- 20:32SPEAKER_00: But there are ways to try and start this journey of CTEM,
- 20:36SPEAKER_00: this journey of trying to at least continuously see what you have.
- 20:40SPEAKER_00: And it doesn't have to be particularly expensive.
- 20:42SPEAKER_00: And you don't have to use a tool like Edgecam.
- 20:45But at least it will be a start for most organizations that are
- 20:48simply just opting
- 20:50SPEAKER_00: for a one-off pen test or something like that.
- 20:52SPEAKER_03: Yeah.
- 20:53And I'm curious for your thoughts on kind of what sort of size
- 20:56or scale you have to
- 20:57SPEAKER_03: be before you start executing this.
- 21:01SPEAKER_00: It's interesting, right?
- 21:01So in our client base,
- 21:04we have very,
- 21:05very big with thousands of web applications.
- 21:07SPEAKER_00: And we're very,
- 21:08SPEAKER_00: very small because when we were a scale up or we still are,
- 21:11SPEAKER_00: but when we were a startup,
- 21:12SPEAKER_00: we wouldn't turn down business.
- 21:14SPEAKER_00: And we have some clients still in that space.
- 21:18Smaller clients benefit from PTAS because it's obvious that the
- 21:24validation is there.
- 21:25Smaller clients may not have a cyber person in their business
- 21:28to validate.
- 21:29SPEAKER_00: So the validation is there.
- 21:31Smaller clients benefit because there's a massive cost saving
- 21:34on a PTAS solution
- 21:36SPEAKER_00: versus hiring expensive consultants.
- 21:38SPEAKER_00: Being an ex-consultant, I know.
- 21:41SPEAKER_00: I'm not throwing mud, but it can be in certain cases.
- 21:45Most PTAS licenses are static in terms of cost,
- 21:50regardless of the size of the application.
- 21:53SPEAKER_00: So you can manage cost very easily and you can budget very easily.
- 21:59SPEAKER_00: So on the smaller size, they're sort of the concerns.
- 22:01It's either they haven't got the skill set to validate their
- 22:05own vulnerabilities.
- 22:08SPEAKER_00: they may want to ask somebody,
- 22:10SPEAKER_00: in EdgeGAN's case,
- 22:11SPEAKER_00: something that stands out from a lot of our client feedback is support.
- 22:15SPEAKER_00: So if you find a vulnerability,
- 22:17we don't expect clients to be experts in cyber and they can
- 22:22just contact one of our
- 22:23pen testers directly and talk about the issue and talk about
- 22:26how they can fix it or
- 22:27SPEAKER_00: how bad is it or can they show me,
- 22:29can they recreate the issue and show me how to recreate it
- 22:32because maybe they
- 22:32SPEAKER_00: couldn't recreate it themselves to understand better.
- 22:36SPEAKER_00: If you're a smaller business,
- 22:37SPEAKER_00: if you're using your consultants,
- 22:38SPEAKER_00: that would cost you additional fees to do that.
- 22:43SPEAKER_00: On the bigger side, it's more around the enterprise side.
- 22:49SPEAKER_00: It's more around the idea that I've got 1,000 or 2,000 web applications.
- 22:54I can't use a scanner because we don't have the bodies
- 22:57internally to run scanners
- 22:58SPEAKER_00: continuously on 2,000 web apps and process the data.
- 23:02SPEAKER_00: I need this continuous, and I need it consistent, and I need accuracy.
- 23:08Because if you don't have accuracy,
- 23:10unless you're getting four false positives per assessment,
- 23:16SPEAKER_00: And you've 10,000 applications,
- 23:18all of a sudden you've got 40,000 false positives to weed
- 23:21through before you get
- 23:22SPEAKER_00: rid of that stuff alone,
- 23:24SPEAKER_00: apart from the stuff that matters.
- 23:26SPEAKER_00: So when you're at that scale,
- 23:29SPEAKER_00: it's more an accuracy and being able to scale and maintain accuracy.
- 23:34And that's the type of thing that our enterprise businesses
- 23:37need as opposed to,
- 23:38SPEAKER_00: obviously there is cost savings,
- 23:40SPEAKER_00: but then at that level,
- 23:42SPEAKER_00: penetration testing companies,
- 23:43SPEAKER_00: traditional can't do this anyway.
- 23:45SPEAKER_00: They can't do it at the scale we're doing it.
- 23:47SPEAKER_00: So we're not really competing with them.
- 23:50SPEAKER_00: So there's different sort of needs depending on the size of the business.
- 23:55SPEAKER_00: Smaller is more
- 23:57Budget orientated at skills,
- 23:59maybe skills deficit or,
- 24:01you know,
- 24:01we can help with all that.
- 24:03SPEAKER_00: On the bigger sides of the wedge,
- 24:06SPEAKER_00: it's more around accuracy,
- 24:07SPEAKER_00: consistency,
- 24:10SPEAKER_00: speed,
- 24:11SPEAKER_00: and all those things that happen.
- 24:13SPEAKER_00: obviously integrations into ecosystems all that's important as well for
- 24:17particularly for larger businesses because generally your
- 24:19vulnerability data is
- 24:21sucked into something else that they use and you know you you
- 24:24know but as long as
- 24:25they're assured that the stuff that they're getting from
- 24:28edgecan is accurate you
- 24:29SPEAKER_00: know they're pretty happy right so absolutely yeah yeah
- 24:35You're listening to the Expert Insights Podcast,
- 24:38home of leading cybersecurity conversations.
- 24:41Subscribe now to get insights from top cybersecurity leaders
- 24:45straight to your inbox.
- 24:47SPEAKER_03: The sort of final segment I want to talk about is around,
- 24:51SPEAKER_03: I suppose,
- 24:51SPEAKER_03: AI security testing would be the best way to put it.
- 24:54SPEAKER_03: So we're hearing a lot about Vibe coding and people using AI.
- 25:00Edgecam recently published a blog around understanding LLM
- 25:05penetration testing,
- 25:07SPEAKER_03: which highlights some of the risks of AI systems.
- 25:09So it'd be great to dig into that and talk about some of the
- 25:11risks that you're
- 25:12seeing actually in using AI and some of the vulnerabilities
- 25:15that can create.
- 25:16SPEAKER_03: Yeah.
- 25:16SPEAKER_00: Yeah, very true.
- 25:19SPEAKER_00: So first of all, it needs to be taught that AI is still software, right?
- 25:24SPEAKER_00: And so therefore,
- 25:25SPEAKER_00: AI systems are vulnerable to lots of the same vulnerabilities that a web
- 25:30SPEAKER_00: application is vulnerable to,
- 25:31SPEAKER_00: right?
- 25:33SPEAKER_00: Which is sort of not discussed enough, but it's true.
- 25:37SPEAKER_00: Um,
- 25:37SPEAKER_00: so it's not as,
- 25:38SPEAKER_00: as exciting as some people think,
- 25:39SPEAKER_00: but there are a whole class of other vulnerabilities,
- 25:43SPEAKER_00: which AI is unique to AI,
- 25:44SPEAKER_00: right?
- 25:45Um,
- 25:45in terms of confusing the AI,
- 25:48trying to convince it to give you internal backend data,
- 25:51um,
- 25:53trying to give data for some other user outside of your
- 25:59sandbox,
- 26:00shall we say.
- 26:01There's terms prompt injection and breaking out of your
- 26:06environment attacks and that.
- 26:07SPEAKER_00: A lot of those attacks though,
- 26:11we find are proven to be,
- 26:13and if you look at the blogs on Edgecan,
- 26:14you'll see them.
- 26:15SPEAKER_00: A lot of the attacks are English, right?
- 26:19SPEAKER_00: It's actually English language.
- 26:20SPEAKER_00: It's not code.
- 26:21SPEAKER_00: It's not weird characters and stuff like that.
- 26:25SPEAKER_00: And therefore,
- 26:26SPEAKER_00: with Edgecan,
- 26:28SPEAKER_00: we test for all that using our automation,
- 26:30but then we have humans validate the responses because the
- 26:33responses are super
- 26:34SPEAKER_00: contextual and you can't really get AI to validate AI in any way.
- 26:40accurate way so so so we use humans to validate their responses
- 26:46and determine are
- 26:47we breaking out of our sandbox are we injecting prompts that
- 26:50shouldn't be there um
- 26:51SPEAKER_00: are we you know um so so
- 26:55SPEAKER_00: So AI pen testing, yeah, it certainly is a big thing.
- 26:58SPEAKER_00: I think a lot of AI protocols and models were built.
- 27:05Like everything on the internet in history,
- 27:09there was a lot of security that wasn't considered.
- 27:13And clever people come along and try and confuse AI and break
- 27:17it because AI isn't
- 27:18SPEAKER_00: as clever as people think in many cases.
- 27:20SPEAKER_00: Right.
- 27:20SPEAKER_00: Yeah.
- 27:22SPEAKER_00: So, yeah, it's certainly a growing thing.
- 27:24SPEAKER_00: And particularly when people start to build their own LLM.
- 27:28SPEAKER_00: So you give your everything from your GPTs and your, you know,
- 27:34SPEAKER_00: your geniuses and all this type of thing.
- 27:36But when people start building their own LLMs and teaching
- 27:39their own LLMs,
- 27:41I think the problem will only get bigger because people want
- 27:44custom LLMs to do
- 27:46SPEAKER_00: specific things for their business.
- 27:50SPEAKER_00: Yeah.
- 27:51SPEAKER_03: And so the classic one is that sort of, you know,
- 27:54help,
- 27:55my friend's had an accident,
- 27:56I need the source code or the password to help them.
- 28:01And that sort of thing has historically been very effective at
- 28:04tricking the GPT
- 28:05SPEAKER_03: models,
- 28:05SPEAKER_03: as you say.
- 28:07So is this just an issue where it's customer-facing or where
- 28:10you've got your pizza
- 28:12SPEAKER_03: restaurant,
- 28:13SPEAKER_03: like you mentioned earlier,
- 28:14has got that LLM for taking the order and the customers are
- 28:16interacting with it?
- 28:17Or is it also an issue more for the back-end AIs,
- 28:20I suppose,
- 28:21that are maybe more internal?
- 28:23SPEAKER_00: Well, that's a scary thing.
- 28:25SPEAKER_00: It's definitely an external piece, particularly if you can talk to AI.
- 28:29SPEAKER_00: With Edgecan, with our AI, you can't talk directly to it.
- 28:33SPEAKER_00: Our engineers build the prompts and we did it on purpose.
- 28:36So we have a bunch of insights that we think are important
- 28:41based on client feedback.
- 28:43SPEAKER_00: And then when you use our AI, it will give you those insights.
- 28:46SPEAKER_00: You can't ask it any additional questions.
- 28:49SPEAKER_00: That may change, but there's a reason for it, right?
- 28:51SPEAKER_00: We don't want...
- 28:52SPEAKER_00: We decided to reduce the attack surface on our AI by doing that, right?
- 28:56SPEAKER_00: Yeah.
- 28:58SPEAKER_00: But yeah,
- 28:59SPEAKER_00: well,
- 28:59SPEAKER_00: like the scary thing is,
- 29:01is that if you have an aggressive AI or as it evolves and it
- 29:05starts attacking other
- 29:06SPEAKER_00: AIs internally,
- 29:08SPEAKER_00: it starts asking other AIs for...
- 29:12queries or maybe understanding how to do it to bypass any
- 29:16protections to other AI
- 29:17SPEAKER_00: because it's an AI itself.
- 29:19And a lot of that sounds like something you might see on an
- 29:22episode of Black Mirror
- 29:23SPEAKER_00: or something.
- 29:23SPEAKER_00: But the reality is that I think five years time, that could be the case.
- 29:29SPEAKER_00: There's very little constraint.
- 29:32SPEAKER_00: And there's some very good podcasts and YouTube videos that
- 29:38in relation to this type of problem that has completely been
- 29:41ignored by a lot of
- 29:42SPEAKER_00: the large AI players in terms of AI safety.
- 29:46Then when you get the idea that AI starts to replicate itself
- 29:52and Vibe code,
- 29:53SPEAKER_00: Vibe coding,
- 29:54SPEAKER_00: as you mentioned,
- 29:55SPEAKER_00: Vibe coding is very effective for proof of concept.
- 29:58When you get into complex architectures,
- 30:00it tends to fall down,
- 30:02most people would say.
- 30:05But in some cases,
- 30:06you'll see it has risen productivity of human developers
- 30:10dramatically.
- 30:11But then you can read other articles that are saying that the
- 30:15amount of setup required
- 30:18the amount of setup required to get the AI to do it before the
- 30:22actual work was done
- 30:23SPEAKER_00: was more than the saving of the actual using the AI.
- 30:28SPEAKER_00: So there's lots of,
- 30:29SPEAKER_00: you know,
- 30:31there's lots of arguments from both sides in terms of the
- 30:35effectiveness AI.
- 30:36SPEAKER_00: But I'd say like, look,
- 30:38trillions of dollars have been pumped into this it's it's the
- 30:41new steam engine it's
- 30:43the new you know it's the new uh revolution it's not going away
- 30:48it has its uses
- 30:50there is a trend now towards smaller specific ai uh agents as
- 30:54opposed to big ais
- 30:55and i think they realize because it's they can't they don't
- 30:58work the big ones too
- 30:58SPEAKER_00: well um is the truth um but yeah look who knows
- 31:04SPEAKER_03: Yeah,
- 31:05SPEAKER_03: you mentioned an interesting point around it being,
- 31:07SPEAKER_03: you know,
- 31:08SPEAKER_03: it's hard to predict what they're going to say.
- 31:09You don't know what's going to come out and it's hard to
- 31:11predict how they react.
- 31:13SPEAKER_03: So I think that point you mentioned around AIs poisoning other AIs,
- 31:17SPEAKER_03: you know,
- 31:17SPEAKER_03: how can you,
- 31:18SPEAKER_03: you know,
- 31:19SPEAKER_03: without a human element of observing that,
- 31:21SPEAKER_03: I suppose,
- 31:22SPEAKER_03: or some very good automated system,
- 31:23SPEAKER_03: it's going to be very tricky to...
- 31:25SPEAKER_00: Well, there's also human bias.
- 31:26SPEAKER_00: So the people that built the AI, and there's lots of examples.
- 31:30SPEAKER_00: I don't think we'll get into it today,
- 31:31but there's lots of examples of AI being told not to answer
- 31:35certain questions or to
- 31:36SPEAKER_00: answer certain questions in a certain way,
- 31:38SPEAKER_00: as opposed to just go and grab the data and make it,
- 31:41SPEAKER_00: you know,
- 31:42SPEAKER_00: and combine the data and make a decision and give you some,
- 31:44SPEAKER_00: you know,
- 31:45SPEAKER_00: evidence.
- 31:45SPEAKER_00: Some of them are told just don't answer this question or whatever.
- 31:48SPEAKER_00: So...
- 31:51SPEAKER_00: So there is an element of human bias in the back end of every AI system,
- 31:55SPEAKER_00: which people may or may not be aware about.
- 31:58But if that was in terms of an AI lawyer or an AI accountant or
- 32:06some sort of AI
- 32:07SPEAKER_00: economist system,
- 32:09SPEAKER_00: and we decided,
- 32:10SPEAKER_00: oh,
- 32:10SPEAKER_00: all our legal now is going to be managed by this AI agent over there,
- 32:15SPEAKER_00: so we'll fire all the lawyers in the company.
- 32:18SPEAKER_00: It really would depend on any of the bias that's built into the AI.
- 32:23SPEAKER_00: There's been lots of studies in terms of the ghost of the machine,
- 32:26SPEAKER_00: which is the human taint of how the AI has been built.
- 32:35SPEAKER_00: I think that can be across all types of AI, regardless of what it does.
- 32:42yeah it's fascinating stuff so just in terms of a final i guess
- 32:46actionable piece
- 32:47for the for organizations on this topic um you very much see a
- 32:53like testing ai and
- 32:54testing ai systems as part of that continuous pinterest pen
- 32:57testing strategy and it
- 32:58needs to be thought of as any other sort of software system
- 33:00you're using in terms
- 33:01SPEAKER_03: of looking for vulnerabilities
- 33:04SPEAKER_00: Yeah.
- 33:05SPEAKER_00: So,
- 33:05SPEAKER_00: yeah,
- 33:07SPEAKER_00: I think if there was anything I'd recommend,
- 33:09SPEAKER_00: you know,
- 33:10SPEAKER_00: obviously I use Edgecam,
- 33:12SPEAKER_00: but regardless of all that,
- 33:14SPEAKER_00: like,
- 33:14SPEAKER_00: you know,
- 33:14SPEAKER_00: think about how you secure your systems,
- 33:17SPEAKER_00: think about frequency,
- 33:18SPEAKER_00: depth and accuracy.
- 33:21SPEAKER_00: If you don't have accuracy,
- 33:23SPEAKER_00: you'll wind up with a lot of noise amplification when you embellish those
- 33:27SPEAKER_00: vulnerabilities in terms of put them into other systems.
- 33:31SPEAKER_00: And it even causes more work.
- 33:34And it removes the loyalty of the developers when they're faced
- 33:38with fixing issues
- 33:39SPEAKER_00: that really don't show any risk or have any risk to the business.
- 33:42SPEAKER_00: Yeah.
- 33:44SPEAKER_00: it's just wasting resources and time um so accuracy is very important a
- 33:48prioritization and it's very important continuous do the thing
- 33:54is you need to be
- 33:55continuous um you need to look at it from the point of view
- 33:58that that cyber
- 33:59criminals are continuous but they're not using commercial
- 34:02scanners um to find these
- 34:04SPEAKER_00: issues um
- 34:06obviously they probably are you know to some degree they might
- 34:09be but but the idea
- 34:10is most um of the famous hacks you've seen are a little bit
- 34:14more complex than that
- 34:15a lot quieter and they wouldn't be detected otherwise they
- 34:17would have been
- 34:18prevented before they happened scanners are noisy um but but
- 34:23but to look at uh
- 34:26SPEAKER_00: to look at sort of offensive security as a treadmill,
- 34:31SPEAKER_00: not a point in time,
- 34:32SPEAKER_00: because the world's changing too fast now for point in time assessments,
- 34:36SPEAKER_00: we need to get you know,
- 34:38you need to get a product that does this on your treadmill and
- 34:42keep it going.
- 34:43And hopefully it will produce actionable and useful stuff for
- 34:49you in order to make
- 34:50SPEAKER_00: your business more secure.
- 34:52SPEAKER_00: Fantastic.
- 34:53SPEAKER_03: Thank you so much for joining us on the podcast today.
- 34:56SPEAKER_00: You're very welcome.
- 34:58SPEAKER_01: Thanks for your time.
- 35:17This podcast was produced by Expert Insights,
- 35:19home of leading cybersecurity conversations.
- 35:22Subscribe now to get insights from top cybersecurity leaders
- 35:26straight into your inbox.
- 35:27Why not check out our weekly decrypted cybersecurity briefings
- 35:31and our Game
- 35:31SPEAKER_02: Changers conversations with innovative individuals.