Latest / Identity at the Center / #420 - Sponsor Spotlight - GitGuardian
Transcript
- 0:04This is identity at the center. Welcome to the Identity at the
- 0:11Center podcast. I'm Jeff and I'm flying solo
- 0:13today. Jim's off.
- 0:14He's on assignment, as we say in the biz.
- 0:16But today we've got a great episode.
- 0:18Today's is sponsored by Git Guardian, so we're going to
- 0:21introduce Dwayne here in a second.
- 0:22But if you want to learn more about them, you can find them at
- 0:25gitguardian.com/LP's/IDAC. Just to make it clear, right?
- 0:31This is a sponsored episode. This is how we support the
- 0:33podcast. And we're very thankful that GIT
- 0:35Guardians, you know, one of those sponsors that have stepped
- 0:37forward to help us spread the gospel of identity and access
- 0:40management. So let me go ahead and introduce
- 0:42Dwayne McDaniel. He's a principal developer
- 0:44advocate at Git Guardian. Welcome to Identity at the
- 0:47Center, Dwayne. Thanks very much for having me,
- 0:49Jeff. I'm excited to be here.
- 0:50Yeah, thanks for joining us. And I don't know what it is
- 0:52about principal developer advocate that I just it seems
- 0:55like it just gets blocked up in my mouth and I don't say it no
- 0:58good. As Charlie Kelly of it's always
- 1:00any Philadelphia would say. Tell me a little bit about your
- 1:04kind of your role. How did you get into identity?
- 1:05Is it something that you chose or did it choose you?
- 1:09I actually agree with you. I trip over it all the time.
- 1:13I misspelled it the first time I ever put it, wrote it down in my
- 1:16signature, and thank goodness I have Grammarly.
- 1:20They told me, hey, that's not right.
- 1:22No, I've been a developer advocate for a little over a
- 1:26decade now. So my background comes from the
- 1:30world of Java middleware a long time back, platforming our
- 1:35platforms and tools, dev OPS focus for a good stretch of my
- 1:39life. And that's actually how I went
- 1:41from sales. I have a sales background to
- 1:44developer advocacy. Was working for a platform
- 1:48called Pantheon dot IO and I had AVP of sales who said, hey,
- 1:53you're not the sales guy I need, but you're really good at
- 1:55explaining stuff and I need you to help people figure stuff out.
- 1:59And that's literally I took that as a mission and it became kind
- 2:02of my advocacy, like for myself for what I do.
- 2:05Like that's how I describe what I generally do.
- 2:08So a few jobs later end up having a chance to work for
- 2:13someone who had worked for a previous company that I'd been
- 2:15at. And that's how I ended up here.
- 2:18And this is really my first job in the security realm, but I've
- 2:21been here almost four years now. It's hard to believe, but three,
- 2:26you know, three quarter or four years by the time this comes
- 2:28out. And yes, this is my first role
- 2:32where I've had the chance to look at identity and access
- 2:36management and the depth that I have.
- 2:38It's one of those things I just kind of always took for granted
- 2:40in previous roles that now it's somebody else's burden that's
- 2:42somebody else's problem or that's baked into the platform.
- 2:46I, I mentioned Pantheon a little bit ago.
- 2:49They were the company in 2013, 2014.
- 2:53They had environment variables like secret environment
- 2:56variables and told you do not hand configure your vars like do
- 3:01not hand configure these. Let us do this and then we can
- 3:03swap these out at intervals and we'll handle all that for you.
- 3:07So your database credentials, your Redis credentials, like we
- 3:11got that New Relic credentials. And at the time it was like, oh,
- 3:16OK, that's just how it works. And that was in 2014.
- 3:19And then it comes to find out I end up working for a company and
- 3:22our entire premises, people leak secrets, people hard code these
- 3:26things left and right and by the 1,000,000 per year.
- 3:30And it just left me scratching my head when I first joined,
- 3:33like, is this that bad of a problem?
- 3:34And yeah, turns out it actually is.
- 3:36It's amazing, you know, how far this industry has come in yet,
- 3:40you know, here we are in the year 2026 that we're still
- 3:44finding companies that have like hard coded secrets into
- 3:46applications, you know, they've left open, you know, different
- 3:49repositories or repos or whatever want to call like all
- 3:52these different resources we're pulling into, you know, with you
- 3:55kind of getting into this role here with your guardian and you
- 3:58know, really just starting into the identity spot, I guess
- 4:02officially. Is there anything that surprises
- 4:06you from your past background and kind of like, hey, this is
- 4:09kind of interesting, I didn't expect this.
- 4:12Oh man, there's a lot of things. The the first, like I said, it
- 4:15just people are doing this like you're hard coding credentials
- 4:17like you, you're trusting that these secrets are one, not going
- 4:22to be stolen and two, that you're going to be able to
- 4:25remember where they all are and you're going to be able to
- 4:26rotate them yourself. Like that was how we viewed it
- 4:29back in my old company Pantheon was, look, this is a burden and
- 4:34we're going to take that operational burden on and take
- 4:36that off of your hands so you can focus on doing what you
- 4:39actually want to do, which is deliver your actual applications
- 4:42out there. The bigger surprises have
- 4:47actually been more subtle than that.
- 4:49Like I've come to personally believe that we mixed up access
- 4:54or, I'm sorry, authentication and authorization and a really
- 4:58terrible way the way we've done it with passwords and API keys
- 5:02and standing privilege and everything associated with these
- 5:05hard coded secrets, everything associated with, you know, a key
- 5:09that anybody that holds that key can go through the door and then
- 5:12do whatever they want on the other side within a certain
- 5:15scoped level. Recently I've been actually
- 5:18having the conversations like do identity and access management.
- 5:22Is that one thing? These are, there's an and in
- 5:25there for a reason. Where are those lines?
- 5:28Where do we have those conversations?
- 5:29So that's where I'm at right now with it.
- 5:32But just the scope of how much we have gotten it wrong.
- 5:37And we're seeing the ramifications of that throughout
- 5:40the supply chain attacks these days and just story after story
- 5:44after breach after breach. We've been saying it for a while
- 5:48here at Gate Guardian, but the attackers aren't bothering to
- 5:51break in anymore. They're just logging in and
- 5:53that's the easiest path in, which is sad, but I was
- 5:57personally a little surprised by that when I first came to fully
- 6:00realize that's the extent of the damage.
- 6:03Preach on, brother. I think there's so many people
- 6:06who think of I am as just authentication, but it's so much
- 6:08more than that. You know, I've I've talked to
- 6:11hundreds of people, you know, on this podcast, been very
- 6:14fortunate, but very few what I would call, you know, your kind
- 6:17of role that principal developer advocate.
- 6:19I even said it right that time, what is, you know, kind of like
- 6:22the normal day look like for that type of role, just kind of
- 6:25maybe peel down the the 4th wall there and kind of take people
- 6:28behind the curtain. I wish there was a normal day.
- 6:32That's kind of the joy of the role.
- 6:34I see myself as one of the last generalists.
- 6:37In a world where we're moving, the pendulum is swinging back
- 6:40toward the generalist. I technically report to the CMO
- 6:44and other organizations. Devrel reports to operations or
- 6:49to engineering or to sales. It doesn't really matter where
- 6:54we report up in the chain. The matters how we interact with
- 6:59all of the departments. So I interact with product as
- 7:02much as I do with sales. I'm a function of marketing,
- 7:05interact with our customer support, interact with
- 7:07documentation, interact with basically all of the teams.
- 7:10I am the person in the room saying, wait a minute, a person
- 7:14who doesn't have this insider knowledge might not know that
- 7:17this is hard. I'm the person who breaks
- 7:20things. They will give me early access
- 7:23to things and I will be the loudest in the channel saying,
- 7:25hey, this doesn't work. Hey, this doesn't work.
- 7:27And like it was a beta preview with, of course we know it
- 7:29didn't work, but thanks for telling us.
- 7:32The person who was like, we're going to release the script.
- 7:34And I try to run it and I follow the structures letter and like,
- 7:37here's the errors I get. And I just assume that naivety,
- 7:42naivete that a new developer, I, I want to advocate for them that
- 7:46they're going to have the best possible experience.
- 7:48At the same time, I get to spend a lot of my time out in the
- 7:51field at conferences. I went to like 30-4 conferences
- 7:54last year alone. I've spoken to over 100 since I
- 7:58started a Pantheon. I'm sorry, since I started a git
- 8:00guardian, I got Pantheon on the brain.
- 8:02I was talking about it earlier. And I get to bring that
- 8:06knowledge back in house. So what are the latest trends?
- 8:10What did the person who actually wrote this report that we just
- 8:14read last week, what did they actually say about it in person?
- 8:16What did they present at their session?
- 8:18What are the trends going on out there?
- 8:21And try to bring that knowledge back and close that loop.
- 8:23So I'm advocating both ways internally.
- 8:25I'm advocating, hey, this is the actual situation on the ground
- 8:29and what people are facing out there and why they can't just
- 8:32jump to a better solution tomorrow and why they can't just
- 8:36adopt us like right the second based on their other priorities
- 8:40in their stack and the other way around.
- 8:42Like the community needs to know what the problems are, how we
- 8:46can address those. I try not to do it in a Selly
- 8:48way or salesy way. I mean, I'm old, old school
- 8:52sales, so it always shines through a little bit.
- 8:54But trying to explain the bigger broad strokes technologically,
- 8:57this is what we need to be addressing.
- 8:59This is where we need to be going as an industry.
- 9:01And yeah, of course the Guardian fits in there somewhere, but I
- 9:05also put us up on the same screens that I do.
- 9:08People would consider direct competitors or open source
- 9:10products that can do a fraction that handle part of the use case
- 9:14and try to explain like in the bigger sense, this isn't just a
- 9:17get guardian can save you issue. This is a we need some
- 9:21fundamental governance conversations at a higher level.
- 9:24And yeah, you need better tooling along the way.
- 9:26But the more important thing is process and people and how you
- 9:30put those plans together. So what is it about like the
- 9:33secret space that kind of pulled you in with all the different
- 9:35things that we have and identity going on?
- 9:38Like what is it about secrets? It's like, Oh yeah.
- 9:40Like that's that's where I want to be right now.
- 9:42This actually comes from a couple jobs ago.
- 9:45I was at a place called Git Kraken.
- 9:47I've worked for two companies with git in the name and a Git
- 9:49Kraken. I love Git I I absolutely adore
- 9:56teaching people git basics, explaining how it in turn works
- 10:00internally. I read the good the git book
- 10:02years ago. You can too.
- 10:03It's a little a little dated, but all the foundational stuff
- 10:06is still there. It's just the git-sem.com/book I
- 10:10think or just look up the git book and it's free and explains
- 10:14how this all works, the plumbing level versus the porcelain and
- 10:18the all those fun hidden commands and like what Lib git
- 10:21see is and like how we're implementing these things at
- 10:24scale across platforms without having to put the CLI in place.
- 10:28Anyway, Long story short, I'm looking at git.
- 10:32One of the most powerful things built into git is the hook
- 10:35system. Linus Torval needed a way to run
- 10:38a script when a thing executed, so they put 17 various hook
- 10:43points where an event happens inside of git and something
- 10:46executes. Whatever that script is, it
- 10:48could be in anything you want as long as you have a local
- 10:50executable environment. And I came across this idea
- 10:55literally after I left Pantheon of like, hey, people are hard
- 10:58coding secrets. That's nuts.
- 11:01What if, what if we just put hooks in there?
- 11:03So that I gave started giving a talk back in 2021, I believe
- 11:07first time I gave it was stop committing your secrets, get
- 11:10hooks to the rescue. So specifically what about
- 11:13secrets is it's one of those things, if you can nip it in the
- 11:17bud is they say you can stop it before it gets committed.
- 11:23You solve an entire class of problem if you put it into your
- 11:27source code and you push it and that source code gets public.
- 11:31Wow, now everyone in the world can log into whatever that thing
- 11:33is. You put it into your source code
- 11:35and it doesn't leak publicly, but somebody breaches.
- 11:39Well guess what? You just gave them an open door
- 11:42into anything. Like you can look at so many
- 11:45examples. The Uber breach from 2022 is one
- 11:48of the most egregious because after they got through Pam, they
- 11:52fished a super admin password, got through, flooded them with
- 11:57alerts, got through that, realized, hey, there's
- 12:01PowerShell scripts everywhere, chock full of all the
- 12:03credentials. Like they taunted hacker one
- 12:05from within their own interfaces because, well, they just had the
- 12:08keys. They just could just March in
- 12:10and say, all right, I'm here, I can do whatever I want.
- 12:13So it just seemed even back then before I worked for Gate
- 12:17Guardian, that was just this solves an entire class of issue.
- 12:21And I wasn't even thinking really from a pure security
- 12:23standpoint. And like I wasn't thinking in
- 12:26terms of IAM, I was thinking in terms of DevOps.
- 12:29Like DevOps has very aligned goals with security.
- 12:32We want uptime. We want no one to come in and
- 12:35mess with the integrity of our stuff.
- 12:37And at the end of the day, we do not want auditors or lawyers
- 12:39breathing down our necks saying, hey, you did this wrong.
- 12:43And that's kind of the same general goal that the CIA triad
- 12:46for security. So if we can align those, and I
- 12:49think identity is where they align perfectly.
- 12:53So that's how I got so passionate about this stuff.
- 12:56And then we're going to forget Guardian.
- 12:58Just seeing the amount of research and seeing what's out
- 13:02there and like the horror stories and paying attention to
- 13:04news cycles as close as I do, like I'm still, I am more
- 13:07convinced now than ever that if we dealt with this one aspect,
- 13:11we would, wow, we would solve so many problems around security.
- 13:16Like so many supply chain attacks right now are just get
- 13:20their credentials and pivot from there or get through an initial
- 13:25misconfiguration to get their credentials and move on from
- 13:27there. And you just see that time after
- 13:30time after time. I did a, an independent search
- 13:32the other day and it came by like 37% of all articles in the
- 13:37last year from Bleeping Computer were about credential abuse.
- 13:42Like, yeah, it's it's it's a problem.
- 13:46That puts the bleeping and bleeping computer.
- 13:48You think about it. Let's let me, let me learn more
- 13:52here about git guarding because you know, I, I'm, I will say I'm
- 13:54not as familiar with it as probably, you know, other
- 13:57developers might be. I'm not a developer and I'm sure
- 13:59we have lots of people listening who are not familiar with.
- 14:01So let's give me the overview. Like what is git guardian?
- 14:04You know, what is the mission? What are the problems that
- 14:07you're looking to solve? And and start start start with
- 14:10that. So our Co founders started in
- 14:122017 looking at what can you glean off of the API from the
- 14:18public API for for GitHub, you can look at it today, it's still
- 14:23public. It's in the name of the product
- 14:24api.github.com/events. It's a firehose last year's 1.9
- 14:30billion commits across the thing and it's everything that happens
- 14:34in public. So you change the name of repo,
- 14:36you make something public, you push a commit, it's all there.
- 14:40And one of the things they immediately discovered on just
- 14:43doing some cursory grep and some regex looking through what's
- 14:47there is, hey, why are there all these production environment
- 14:50keys laying around? So they started making phone
- 14:53calls. And the story I've heard is most
- 14:57people hung up on them or got extremely mad or threatened to
- 15:00sue because, hey, why are you looking at a private repos?
- 15:03Like we're not like, this is how we got this.
- 15:05It's literally in public. Like you should do something
- 15:07about that. And it was one company said,
- 15:11hey, this is great how much we owe you.
- 15:13And they said, hey, there's something here.
- 15:17And that's kind of how it started with public monitoring.
- 15:19So we still to this day look at every new event that hits GitHub
- 15:23public. If it is a new commit or
- 15:25something that has become public that wasn't public before, we
- 15:29scan it for a secret. And we started with a handful
- 15:33like your common, most common patterns.
- 15:35And right now we have over 600 detectors that can pinpoint
- 15:39like, hey, this is a secret. So we do generic detection with
- 15:43contextual analysis. If we look at the long weird
- 15:48string, if it's in a header and the app fails, if nothing comes
- 15:53back from that call, we're pretty sure that's a secret.
- 15:55Like no matter what form that takes, we call those generic
- 15:59detectors. So that's how we started with
- 16:02public monitoring. We still do that.
- 16:04That's still probably our most popular thing that people talk
- 16:08about out there because it's tied to our report that we do.
- 16:11That's how we build our annual report, which we'll we'll talk
- 16:14about later. But then people started asking,
- 16:18can we hook this up to internal repos?
- 16:20And the answer to that is, well, gets Git if we can handle the
- 16:23scale of watching everything and get out public, sure.
- 16:27And that's what we started doing.
- 16:28So originally it was just your Git repos no matter where they
- 16:32were hosted. And then it became, well, we
- 16:34have ACLI, we can scan anything anywhere.
- 16:38Just give us a path, give us like Sol's point is at the repo
- 16:40no matter where it's hosted. But then it's quickly spread to
- 16:45secrets don't just leak in code, they leak in the entire
- 16:48ecosystem around the software development life cycle.
- 16:51So Slack, Jira, Teams, Confluence, S3 buckets, we're
- 16:57actually just about to release that and private beta right now.
- 17:01Let's scan all your buckets, see what's in there, Docker images,
- 17:06registries, and the list just goes on.
- 17:09And then we recently added the ability to bring your own
- 17:13source, which means if it's got text that's scannable, we can
- 17:17make it a source that you can scan on a regular repeated
- 17:19basis. Just a little bit of wiring and
- 17:21off you go to literally anything on your sun logs, your home
- 17:25grown seams or whatever. We can make it happen.
- 17:29And then about a year ago or a little over a year ago, we went
- 17:34down the path of NHI governance. So NHI governance is the
- 17:38outcome. Like if you have a plan of we
- 17:41need our non human identities, which is the vast majority of
- 17:46keys are associated with non human identities versus humans.
- 17:50Like it's 150 to 1 guesstimate right now across the industry
- 17:54for every one person that has a key, there's 150 non humans that
- 17:57have keys. So that's the way we leaned into
- 18:01this NHI side and not even identity governance is really
- 18:05what state are these things in? Who owns them?
- 18:08What policy breaches have occurred?
- 18:11Like is it, has it lived too long?
- 18:13That's by far the number one breach that people have with
- 18:16these entities. Is it over permissioned?
- 18:21Should you have? Do you know this as admin
- 18:23privileges? Like we can highlight and
- 18:26surface that in a coherent way across not just the secrets that
- 18:30you've leaked out there in public or in charge, your repos
- 18:34and like whatever they appear in plain text.
- 18:36But now we can hook into your vault systems and give you pure
- 18:40inventory view of all of your identities across the thing
- 18:44holding your keys. And as of the end of last year,
- 18:48we started hooking into identity providers.
- 18:51So Okta and A WSI AM and start being able to inventory so
- 18:57anthropic open AI, we can say, hey, these are all of the
- 19:02accounts that got made even they're even though they're not
- 19:04used, even if they're not in the wild quite yet, but they've been
- 19:09provisioned. We can give you that insight
- 19:11say, hey, this person provisioned this at this such
- 19:13such time and we don't see it used anywhere yet or hey, this
- 19:18same identity is being used by these like 15 endpoints and
- 19:24we've extended that just recently and this is just coming
- 19:27out now. We're happy to work with folks
- 19:29out there to move it to endpoint protection, to move all of this
- 19:33on to the developer machine as an MDM solution.
- 19:36So I think what you've kind of described there is not only like
- 19:39the secrets themselves are a problem, but you start getting
- 19:41into a lot of the secret sprawl, as it's typically called, right?
- 19:44You've got all these different spots that, you know, things can
- 19:48be stored in, kept in, and each of those are a vector that could
- 19:51be potentially misused. Talk to me a little bit about
- 19:54where do you typically see those things kind of manifest?
- 19:58Is there like a common area that you found where it's like, yeah,
- 20:01we know about this, but oh, by the way, people listening out
- 20:03there in the world, you really should be checking this thing
- 20:06because this is kind of like the common thing that we see a lot.
- 20:10One of the most common files we find out there as you can guess
- 20:15is EMB files. It's very easy with get to use
- 20:21get ignore. It's also very easy to mess up
- 20:23get ignore. You know 1 bad line, 1 bad
- 20:26comment and wow you blown out your get ignore or accidentally
- 20:31excluded to get ignore file. Actually recently somebody told
- 20:34me a story that they were using Claude code and multi agent
- 20:39setup and one of the agents decided that get ignore was in
- 20:42the way and it just simply removed it.
- 20:44And the next commit, they didn't notice that and then they pushed
- 20:47all of their environment various public.
- 20:49They know that because we sent them an e-mail.
- 20:51So I didn't talk about that. That is part of our service and
- 20:55that's something we've been doing the entire time is we find
- 20:59a secret and one of those commits publicly, we send an
- 21:02e-mail to the committer right then and there.
- 21:04It's an automated system. It's completely sealed off from
- 21:06the rest of our business. It's not a a marketing thing for
- 21:09us. It is literally like this is a
- 21:11public good. You did this publicly.
- 21:13Shouldn't do that before I worked for the company, I got
- 21:16one of those because I had tried to be clever and I was got a new
- 21:20laptop and I just tried to bounce my SSH key over through a
- 21:23public or a private GitHub repo like, yeah, that's that'll work.
- 21:26You'll save me a few minutes. And no, it actually cost me a
- 21:30lot of time because I got an e-mail immediately saying, hey,
- 21:33you leaked this private SSH key in public.
- 21:36And the first words out of my mouth were no, I didn't that's
- 21:38not public repo. And I went and looked like, Oh
- 21:40no, I have leaked to public in public.
- 21:42So right then and there I had to go through everything that was
- 21:45associated with with I was super lazy.
- 21:48I was reusing that SSH key in multiple platforms should not
- 21:52have done that. It's a bad practice.
- 21:53I know that now. But I nothing bad happened.
- 21:59I was able to catch it so fast and able to invalidate it so
- 22:02fast on like GitHub and the other platforms I was using.
- 22:04And on that, I never saw any negative effects because they
- 22:08saved me on that. So that's what we're hoping to
- 22:09do with everyone out there. If you see an e-mail from us,
- 22:12we're not picking on you. We're just trying to keep
- 22:14everybody safe. We call that our Good Samaritan
- 22:16program. Yeah, I was going to ask you
- 22:18about that a little bit later on, but let's talk about it
- 22:20right now. So what is he kind of talk about
- 22:23giving me an e-mail, but how does that effectively work?
- 22:25Is it something that well you described as something like you
- 22:28guys automatically do? But tell me more like is it can
- 22:32I opt in? Can I opt out?
- 22:33Like how does this program work? You can definitely opt out.
- 22:36We would not recommend it though.
- 22:39And if you are not signing your commits with an actual real
- 22:43e-mail, that's problematic in and of itself for a number of
- 22:46reasons, but also means we can't e-mail you, which we're starting
- 22:50to get into some weirdness there around cosign commits because
- 22:55who do we e-mail if there's like multiple commit committers?
- 22:57But if it's obviously clutter GPT, then yeah, it's pretty
- 23:03obvious what what happened. But no, we don't have it as an
- 23:07opt in. This is, again, we're looking at
- 23:10a public feed. This is public information.
- 23:13We know we're not the only people doing this out in the
- 23:15world. So the way Git works and the way
- 23:18it's worked for wow, it turned 21.
- 23:21I just realized that this month. Git just turned 21.
- 23:24It's old enough to drink now. April 2020, April 2005.
- 23:29It was the first time Linus subscribe get, I think this is
- 23:35the week it got named too, where we're recording this just weird
- 23:38fun fact. But the one of the ways it works
- 23:41is you have to sign your commits.
- 23:42There has to be an e-mail address so Linus can get in
- 23:44touch with you to figure out what the heck is going on with
- 23:46his patch if he needs to like that.
- 23:48That was the original idea. And we still benefit from that
- 23:51because, well, there's a way to contact the person who made that
- 23:54change, who made that commit. We know we're not the only
- 23:57people looking at that feed. We're the people who are trying
- 24:00to keep everyone safe. And we're doing it as a public
- 24:03good so we can alert you hopefully before the attackers
- 24:07get a hold of it. If you're a nation state actor
- 24:12out there, you have caches of everything that happens.
- 24:16It's not that hard to set up, just most people don't have the
- 24:19bandwidth or resources to try to deal with the almost 2 billion
- 24:22commits a year and then sort out what can they can do with it.
- 24:26The big nasties definitely do the cybercrime syndicates.
- 24:30I would not be surprised if there was a private server
- 24:33somewhere on the dark web that shiny hunters and the other
- 24:36threat actors like collectively just hey, I noticed the sun this
- 24:40part of the repo or so weird. The other danger to it is people
- 24:47have this misunderstanding of what GitHub is and isn't and
- 24:52what it can do and what it can't do.
- 24:56There is a belief and and it it is true for a certain class of a
- 25:00secret that they have this program.
- 25:02It's been going on for a few years now.
- 25:04And if you register your API key as a provider, if you make API
- 25:10keys and you register with GitHub, you can set up an
- 25:13automatic invalidation or attach policy to it.
- 25:17So the classic is AWS credentials your secret, Yeah,
- 25:24your AWS password in secret or API key in in secret combo.
- 25:31They will shoot off an e-mail immediately.
- 25:33When GitHub, it gets uploaded to GitHub, it will immediately
- 25:36shoot an e-mail to AWS and AWS will attach user policy.
- 25:40I think we're up to 3 or 4 now. And that policy is just disallow
- 25:45anything that would cost money. It's not a security policy.
- 25:48It's actually a cost savings measure from AWS if you actually
- 25:51look at the policy itself that gets attached.
- 25:54So that's what one of those misunderstands like, hey, well,
- 25:56GitHub will just take care of that, right?
- 25:57Because we pushed it on to GitHub.
- 25:59Well, not really. It works for some things, not
- 26:02everything. And then it doesn't always
- 26:04invalidate. It just attaches some kind of
- 26:06policy to it. The other part is the push
- 26:09protection. Push protection is awesome on
- 26:11GitHub for a certain set of things.
- 26:15And assuming that you use it, it's very easy to turn off.
- 26:20It's very easy to end around on a per push basis and just say,
- 26:24yeah, I know what I'm doing. I'll I'll do it later.
- 26:26And last time I checked, it wasn't free for a private repos
- 26:28that might have changed. Don't, don't hold, don't hold me
- 26:32on that one. But it's something that's very
- 26:34easy to get around that push protection.
- 26:36The other piece is we get a lot of feedback and I'm always
- 26:40surprised by this to this day. Like this happened to me this
- 26:44week. Someone said, oh, I've taken the
- 26:46commit down. It's all good now.
- 26:48It's like what they meant is they overwrote that commit with
- 26:52another commit or they added another commit onto the chain
- 26:55and Oh no, it's gone. It's like, no, that that other
- 26:59commit's still there in public. Like that's how Git works like
- 27:01that. Just because you put another
- 27:03commit in front of it doesn't make that commit not be there.
- 27:06And like the fact that Git is kind of forever.
- 27:09And even if you try to prune it, orphan commits still exist.
- 27:12There's a giant shadow cache that is GitHub.
- 27:15And if you do not access it, you have access to all of these
- 27:18commits that people thought, well, no, I deleted that commit
- 27:20like that. I cleaned that up.
- 27:21Like it's gone. It's like, well, if it ever,
- 27:23ever, ever made it to public, if we sent you an e-mail about it,
- 27:26assume it's forever. Assume that on a server that is
- 27:30somebody else's on another platform somewhere, that commit
- 27:33will live forever. And we've seen that in the wild.
- 27:37Like we've seen multiple attacks where it's like we see the repo
- 27:41get taken down immediately, like turned to private again, and
- 27:46then there's still a security incident related to that
- 27:48organization, you know, a few days or weeks later.
- 27:51Because it takes time for them to go back in and fix all the
- 27:54different underlying mechanisms that are associated with that.
- 27:58The hope is that they will. We have evidence that they
- 28:02don't. This is directly from the
- 28:05report. We haven't really talked about
- 28:06our state of secret sprawl report, but that is the big
- 28:09glaring number that bothers me personally the most out of it is
- 28:16so for a class of our secrets that we find that we can detect,
- 28:20we can do a validation stuff. Can't do it for everything under
- 28:23the sun, but we're working on that where we do a non of truth,
- 28:27non intrusive API call to the platform or some kind of call to
- 28:31the platform say hey, does this key work?
- 28:33Does it authenticate? And if it's valid, we tell you
- 28:36it's valid. If it's not valid, we say it's
- 28:38not valid. If we don't know, we tell you we
- 28:40couldn't find out. Whenever we're trying to mislead
- 28:42you. The vast majority of secrets are
- 28:45falling that generic camp and we can't do that.
- 28:48We took in 20 or for 2025 report looking at 2024 data.
- 28:53We took a sampling of secrets that we found to be valid back
- 28:57in 2022. You know, they're still on
- 28:59public, they're still there. We went back and checked like,
- 29:02yeah, these are still there. So 11,000 + 11,700 or something.
- 29:06The exact numbers in the report, the old report, and we tried to
- 29:10revalidate them, exact same process.
- 29:13And in January 20, 2570 percent came back as still valid.
- 29:19Years later. We did the exact same
- 29:22experiment, the exact same data set and it's dropped to 64%,
- 29:28which means only 6% of those have been dealt with in the last
- 29:33year or not then in the 2025 now dealt with could mean that
- 29:38service just went away or they finally did rotate that that
- 29:42secret or something. We don't know.
- 29:45So what we do know is that there's evidence that if it is a
- 29:49long lived secret that hits public and it's valid, it's has
- 29:55a tendency to stay valid and that's terrifying.
- 29:59Why do you think that is? Like why aren't if, if, If
- 30:01people are being made aware that they have a problem, why do you
- 30:05think they don't fix it? I think it's a combination of
- 30:08factors. The 1st is they don't have a
- 30:10good solid plan and governance. I, I used governance earlier,
- 30:14and that's what I really mean by governance, is there's no good
- 30:16governance in place to say here's what you do instead.
- 30:20If you just swap out secret one for secret 2 and it's both in
- 30:24plain text, you have done nothing.
- 30:25There is no point in doing that exercise and people logically
- 30:29know that. Well, if I don't have a better
- 30:31way to handle this, I'm just going to keep the old one there.
- 30:35There's also just the fact of inertia.
- 30:38The risk of something bad happening is the probability
- 30:43that an attacker will eventually do something nefarious.
- 30:47The danger of downtime is dollars and cents.
- 30:51It's plain as day. If this system goes down for an
- 30:55hour, we lose X amount of money. Let's leave the system run for
- 30:58as long as we can without making any changes, and there might not
- 31:03be any issues. And that's comes down to a
- 31:06business decision, not a technical decision, not a
- 31:08security decision. Like, yeah, let's see what
- 31:12happens. It hasn't happened so far.
- 31:15Nobody's knocked at these companies completely offline
- 31:18yet. Yeah.
- 31:20What's the worst that's going to happen?
- 31:23We'll take the risk and go for just keep the thing running.
- 31:28I think we are starting to see better governance out there,
- 31:31especially as supply chain attacks have ramped up a lot in
- 31:35the last year, the targeting developers and targeting
- 31:37credentials specifically. But we're still stuck in this
- 31:41paradigm of, well, what do we do instead?
- 31:45And if you're not focused on this, like I'll admit I'm hyper
- 31:48focused on this because where I work and what I, I read day in
- 31:51day out, most developers are not, most DevOps teams are not.
- 31:56They're gotten way other so many other things they need to pay
- 32:01attention to uptime, throughput, network bandwidth, so many
- 32:06issues that from the security side, it's easy to say, well,
- 32:09it's going to happen and we take those for granted.
- 32:11But dev OPS teams, developers are under pressure on all of
- 32:14this stuff all the time. And it's like, do I fix our
- 32:18throughput? Do I fix the number of customers
- 32:20that can check out in an hour? Or do I fix this hard coded
- 32:24secret that's been in there for a year that nothing bad's
- 32:27happened to yet? What do I do on my next Sprint?
- 32:29What am I going to get a bonus on?
- 32:30What am I going to get paid on? What am I going to get yelled at
- 32:34by my manager the loudest about? And they guarantee you it's not
- 32:37the secret most of the time, unless there is an active
- 32:41incident and then it's all drop out, what you're doing, all
- 32:45hands on deck. We lived through it and we're
- 32:46never going to do that again, right?
- 32:49And then everybody just go back to work, don't put anything
- 32:51better in place. But I think that's going to
- 32:55change. I think that is changing now.
- 32:56And the technology is evolving very quickly.
- 32:59And I think we're going to get to a point where hard coded
- 33:01secrets are going to be laughable in just a few years.
- 33:06Not overnight, maybe not tomorrow, maybe not next year,
- 33:08but there'll be a point in the not too distant future where
- 33:12we'll say what? You hard coded an API key?
- 33:14What were you thinking? It's like using Explorer,
- 33:18Windows Explorer 8. Like what?
- 33:20Why? Why would you do that?
- 33:22That's archaic. Well, that's why we do this
- 33:24podcast. It's a little bit of a time
- 33:26capsule. You know, the people in the
- 33:28future will be looking at these, at these cave people I am who
- 33:31are trying to figure out, you know, how to do things.
- 33:34I was going to ask you about that report that you mentioned.
- 33:36So the state of secrets for all 2026.
- 33:38I'm looking at the the cover of it right here and we'll have a
- 33:40link in our show notes for people to check out.
- 33:42It says 5th edition. So I'm assuming you've been
- 33:45doing this for at least five years.
- 33:48But it also has A tag line that says the year software changed
- 33:50forever. Take me behind that that tag
- 33:53line. So one of the things we great
- 33:57joys we have at Git Guardians, we get to watch technology in
- 34:00real time evolve that evolution. The way I like to phrase it is I
- 34:06can't tell you the exact traffic patterns and exactly what cars
- 34:09are on the road by the number of accidents, but I can get some
- 34:13pretty good information about makes and models and how people
- 34:16are generally behaving on the road.
- 34:18And that's kind of what we can extrapolate from this.
- 34:20So the, the, the elephant in the room always is AI.
- 34:26It has been for a couple years now, but 2025 is the year we
- 34:29watched or from the 2025 report to 2026 report.
- 34:34It became not a experiment, not a test or for companies.
- 34:42It became the mainstream how we develop and it goes well beyond
- 34:46the enterprise. This is just how developers in
- 34:48general are approaching the field now.
- 34:52Things that might have been done a few years ago with like, I've
- 34:55tied together these scripts and maybe got N8 N in there now is
- 34:59like, well, no, this is agenic from the ground up.
- 35:01This is just literally how we're doing it.
- 35:03And that's evidenced from the sheer number of services we're
- 35:08seeing with leaked keys was open router had the largest increase.
- 35:15Now to be fair, open router did not exist four years ago.
- 35:18They didn't exist three years ago, I'm pretty sure.
- 35:21I think they're they're fairly new though.
- 35:22The platform might have, but not with like the full breadth of
- 35:25them doing what they do where they get free tokens for certain
- 35:28things. And they're a really cool
- 35:30platform that allows you to do multiple LLM models.
- 35:34So you can have a fall back, a natural fall backs.
- 35:36Instead of calling directly to applaud your calling or
- 35:39Anthropic your calling, it will just fall back to the next thing
- 35:44you picked on the list. And yeah, that's not available.
- 35:46Not a big deal. We'll go down the next thing on.
- 35:48It's a really cool system, but we watched 48 X increase in the
- 35:51number of leaks secrets from 2025 to or from 2020 or to 2025.
- 35:57Like that's massive. And 12 of the top 15 most leaked
- 36:03secrets are the highest. Secrets with the highest change
- 36:07rate of leaks were AI platform services.
- 36:12So not the elements themselves. Deep seats definitely on there,
- 36:15but things like Brave Search, MCP, yeah, I've been running,
- 36:23not remembering all the names of all the things, but there's a
- 36:25lot of them. And if you look at the report,
- 36:27you can run down that list and say like, hey, this is a serious
- 36:32jump. And what kinds of curves were
- 36:34seeing, like Super Bass? Super Bass really showed up for
- 36:37the first time last year's report of the 2025 report in a
- 36:43significant way, and it's still showing up in a significant way
- 36:45on this report. It's not growing as massively
- 36:48and rapidly as it did the previous time.
- 36:51But Super Base we call out because it's how people do cheap
- 36:55vector databases like pretty quickly.
- 37:00Is that the only use case for Super Base?
- 37:01Absolutely not, but that's predominantly the one that you
- 37:04see people talk about in public. So again, we can't tell exactly,
- 37:08exactly what cars are on the road, but we can tell you the
- 37:10general shape of traffic and how people are behaving out there.
- 37:14And we didn't see the big number.
- 37:15The, the we, there's a big number at the beginning of this
- 37:18report that that's the thing that we always get the news
- 37:23about. And that's something, if you're
- 37:24listening to this, you probably have seen at least a mention of
- 37:27this out there because it gets picked up by a lot of sources.
- 37:31But the total number of secrets we saw added to GitHub in public
- 37:36in 2025 was 28,649,024. Yeah, but I'm just going to say
- 37:4428.65 million. I'm going to round up a little
- 37:47bit because I'm just take that liberty.
- 37:49But over 28.6 million secrets, that's a 34% increase year over
- 37:56year. And this is a continuing trend.
- 38:02We have seen it rise every year since we've started putting out
- 38:05this report. And really from before that,
- 38:07before we put out these numbers publicly, the first report, we
- 38:12went back and we looked at that data set and reran basically all
- 38:18the numbers. And we were very conservative in
- 38:22our first reports because there's a lot of, we want the
- 38:25highest precision and recall we can have, but we need to be
- 38:29sure, especially around generic secrets and contextual analysis.
- 38:32And we simply didn't have the capabilities around LLMS that we
- 38:36have today. So going back looking at all the
- 38:40numbers that first report, if we ran into the exact same tools we
- 38:45had today, it's 11,000,000. Secrets got added to GitHub in
- 38:47public in the year 2020, 2020, 2318 million, 21 million in
- 38:532024, and then here we're almost 29 million in 2025 across 2025.
- 39:01The interesting trend on that isn't just that, hey, we're
- 39:04leaking more secrets, obviously there's more developers.
- 39:07But when we did the comparison, and this is one of the numbers
- 39:10that jumped out to me, GitHub reported growing 98% over that
- 39:16same time period and we're reporting 152% growth of leaked
- 39:22secrets. So this is not just a new person
- 39:26problem. This is not just a new developer
- 39:28is going to leak secrets. This is serious.
- 39:30Like we continue as an industry to keep leaking secrets at an
- 39:34alarming rate. And we've never done it faster
- 39:36and we've never done it with more lines of code than we did
- 39:38in 2025. So it's really is the year
- 39:41everything changed when you look at it from just a pure numbers
- 39:43perspective, just it's ramped up not an order of magnitude yet,
- 39:47but it feels like it's headed that way.
- 39:49Yeah, I feel like that yet is the keyword there because, you
- 39:52know, we, we, we went, we went almost 35 minutes, I think
- 39:55without mentioning AI, which I think might be a record for us,
- 39:58at least in our recent episodes. But I feel like we're on this,
- 40:02you know, exponentially upward trajectory of, of this problem
- 40:06because now it's, it is becoming mainstream.
- 40:08I mean, I'm vibe coding, you know, stuff on the weekends and
- 40:11I am definitely not a developer, which means I'm relying on other
- 40:14tools to help, you know? Help me think about security and
- 40:17I'm learning stuff, you know about that, especially when it
- 40:18comes to identity. But do you feel the same?
- 40:20Or it's like, OK, you know, AI really wasn't doing much
- 40:24development type stuff. But now we get into this world
- 40:27of NH is agentic identities, MCP servers, agent to agent, right?
- 40:32And it becomes this exponential, you know, increase in the
- 40:36developer footprint at large. Yeah, we actually have the
- 40:41numbers to back that up. The we we specifically looked
- 40:46and did a a very close up view of Claude code.
- 40:50And there's an interesting trend that happened where January of
- 40:532025 there were no commit signed by Claude Code or cosigned by
- 40:57Claude Code or co-authored by Claude code is the proper way to
- 40:59say it. That's one of the notations in
- 41:02get you can add additional, additional signers to a commit.
- 41:06So Claude Code allows you to do that.
- 41:08They started in earnest and very early.
- 41:11So we see the first ones show up in February and like there's two
- 41:16and then by the middle of the year it is like 4X.
- 41:22You're 4 times more likely to have committed a secret if you
- 41:26Co signed or co-authored with odd code then if you didn't.
- 41:33And then around September they release a new model and that
- 41:37number starts ticking back down and never gets to parity and
- 41:40never gets back to that baseline.
- 41:42So for every thousand commits, we're seeing like 1.5 secrets
- 41:45and it's 2.4 I think for Claude across the entire year.
- 41:50So you're more than twice as likely across the entire year on
- 41:53average to have leaked a secret if you are letting Claude
- 41:58co-author. Now what we're inferring from
- 42:00that is you're more likely to have just let Claude do all the
- 42:06work if you're letting it co-author and that non
- 42:10scientific little surveys we have done always point back to
- 42:13that exact same thing. It's like, yes, I sign my own
- 42:16commits after Claude gives me the thing.
- 42:19Those are at least reviewed in some capacity, even though we're
- 42:22pushing thousands of lines of code per PR at this point, per
- 42:26per commit at this point versus hey, that looks good enough.
- 42:31Claude, go ahead and do the get thing for me and vibe code it
- 42:35all the way to production. Like that's what we're seeing.
- 42:38If you're not putting yourself in the loop, if you're not doing
- 42:41those reviews and you're not taking full responsibility of
- 42:44like this is on my shoulders completely and I'm not letting
- 42:48Claude say it co-authors, then you're less likely to have
- 42:52leaked a secret. So will we see the models get
- 42:57better? We are really hoping so.
- 42:58It's, it's strange to say I'm rooting for Claude here, but I
- 43:02really am seeing the improvement from September to the end of the
- 43:05year and now seeing what there will be coming out with this
- 43:07year. We might get to a point where it
- 43:09will be safer overall to commit with claw code to let it go
- 43:16ahead and handle these things for you.
- 43:18It's not today. We have the evidence.
- 43:20It's not in the new year 2025. But yeah, we're in April already
- 43:24of 2026. That's hopefully cross our
- 43:27fingers that will be something we'll we'll be able to help
- 43:30solve or new developers as the models get better, as the
- 43:33training gets better, as we realize, hey, we can't just keep
- 43:36training on templates and old data and old ways of doing it
- 43:41because that's all insecure and terrible.
- 43:43I feel like that's table stakes now for any of these, you know,
- 43:47AI coding agents, right? Either helpers or fully
- 43:49autonomous. It strikes me that they have to
- 43:53get the security right. They might get a little bit of a
- 43:56grace or a free pass here over the last couple of years while
- 43:58this has kind of been being built.
- 44:00But at some point security teams are going to come along and say,
- 44:03no, you cannot use that. It is not secure.
- 44:06And so I feel like it's in their best interest, right?
- 44:08Whatever company is developing, you know, any of these, you
- 44:11know, coding AEI agent tools that they have to get the
- 44:14security right. Am I, am I being fair with that,
- 44:17with that observation? I think you're being optimistic
- 44:19with that observation. That's fair.
- 44:22Putting security in a place where they can literally say no.
- 44:26I'm not sure that's reality anymore.
- 44:28We just saw this week the the news from Versal.
- 44:32It was 1 developer used a compromised agent and all of
- 44:41their internal secrets just got leaked.
- 44:43It's amazing. That's what it takes, right?
- 44:44You could do everything right by the book and then one person
- 44:47goes, you know, One Direction that maybe is or isn't
- 44:50sanctioned and there your your castle starts to crumble.
- 44:54And I don't know if it was their their report or someone else's
- 44:57report that I read this week. I read a lot of reports as part
- 45:00of the job of developer advocate is to stay up on what's going
- 45:03on. That said, when they brought
- 45:06that up to their internal developers, not a single
- 45:09developer knew that was even a possibility.
- 45:11They hadn't heard. They're not keeping up with the
- 45:14latest, greatest attacks. They're not keeping up with like
- 45:17the horrors that can be done through prompt injection.
- 45:21Like they know it exists. It's something out there that's
- 45:23going to affect someone else in the world.
- 45:27But you know what? I'm going to keep developing the
- 45:28way I've been developing because that has kept me employed and
- 45:31that has kept me active and happy and healthy all this time.
- 45:36And now we can't trust our tools.
- 45:38Like I'll give you an example of the the advice we're giving
- 45:41people now is local environment variables are no longer safe.
- 45:48That's if you'd have told me a year ago I would be giving that
- 45:52advice to developers now, I wouldn't have believed you move
- 45:56to local vaults. Key pass is free 1 password.
- 45:59Most companies provide some kind of password system and they all
- 46:02have documentation and guides on how to do this properly.
- 46:05But moving away from at any point, anything on your machine
- 46:09can reach that text file can reach that your, your history
- 46:13can reach your, your logs can reach.
- 46:16You know at this point there's no difference between the data
- 46:19plane and the control plane for most AI instances or for most
- 46:22agentic instances. So you can simply open up a file
- 46:27and them will arbitrarily execute this thing that will
- 46:30call open claw that will then run a scanner to look at all of
- 46:36your credentials, like dump all of your environment variables
- 46:40and then report those out to some kind of thing on the
- 46:43blockchain. Like that sounds sci-fi, but
- 46:45that's literally happening right now.
- 46:46That's canister worm from team PCP.
- 46:49Like there's these are real attacks that are happening in
- 46:52real time and we in security are like living by the seat of our
- 46:57pants moment to moment. What horrible, new terrible
- 47:00thing is going to emerge today in the news?
- 47:02Like I read multiple news sources a day that like are not
- 47:05mainstream news by the time it hits like your CNBC or your
- 47:10whatever news source you're reading out there, your ground
- 47:13news like, Oh no, no, this has been going on for a while.
- 47:15It's like it is already too late to do anything about that
- 47:19version of the attack. Just you hope you didn't get
- 47:21popped. You hope you weren't one of the
- 47:23people that downloaded the wrong version or opened the wrong file
- 47:26with the wrong tool. All right, well, shout out to
- 47:28ground news there. I also think it's a cool thing.
- 47:31I love ground news. Check it out for people who are
- 47:34interested. So we've spent the last 45
- 47:37minutes kind of talking about all these problems and stats and
- 47:40measures. And I'm sure we got a lot of
- 47:42people listening that are like, OK, So what do I do about it?
- 47:47And a lot of the people that are listening may or may not be,
- 47:49well, say they're probably not developers.
- 47:50They're probably more focused than I am just by the nature of
- 47:53this podcast. What are some of the things that
- 47:55people can take away and say, OK, these are the questions I
- 47:58need to start asking either my, my dev team or my dev SEC OPS or
- 48:02information security or whatever right groups might be in the
- 48:07role of application development organization.
- 48:09Like what are some initial questions for me as an I am
- 48:13practitioner out in the world to say, how do I get my arms around
- 48:16this problem? How do I, you know, get the
- 48:17scope and start to talk about it with those folks?
- 48:21I, I think for I am folks particularly asking, does this
- 48:28fall under my scope of what access means?
- 48:31If an auditor comes to your door and says unauthorized access
- 48:35happened and that's described under NIST and under NIST 2,
- 48:38like no matter what PCI unauthorized access happened and
- 48:41customer data got breached. Are you in the position to say,
- 48:47well, that was a machine credential?
- 48:49Like that's clearly not me or look, you're in charge of IAM
- 48:54and that's access and that machine has an identity.
- 48:57Like how do you align that? So if that's the first like
- 49:02actually come to Jesus moment here of is this something I can
- 49:06deal with? Is this something we can
- 49:07approach as an organization? The answer has to be yes.
- 49:10The answer has to be yes. But how do you do that?
- 49:13And that's where governance really comes into play.
- 49:15What is your plan around this? Are you so sealed off from dev
- 49:19OPS and from Devsec OPS that those conversations aren't
- 49:22happening? I don't know really what to tell
- 49:25you if that's they're closed door and you're siloed and you
- 49:28don't talk to each other. So have those conversations.
- 49:31What you can ask them for in practical terms are inventory is
- 49:37an inventory are a list of identities and just start there.
- 49:41Like, hey, what has access to what?
- 49:45In a perfect world, you'll be able to pull that up in a
- 49:47dashboard and say, OK, this key exists.
- 49:51It goes to this, it was issued here, this person owns it.
- 49:54It's being used in these places. Last time it was rotated was
- 49:58this time. That's kind of an ideal
- 50:00situation, and that's totally technically possible.
- 50:04I, I, I know this partly because I work at a company that makes
- 50:06this technically possible. We're not the only player in the
- 50:08world that can help with this, but we're the only ones that see
- 50:12it as a secret first problem. I've gotten some really
- 50:17interesting conversation the last couple years of my life of
- 50:19what identity means, and I've come to the conclusion that it
- 50:24doesn't actually matter. That sounds insane.
- 50:28I know saying that out loud. Because the attackers don't
- 50:32care. The attackers care about access.
- 50:36Whatever they can take to get access, assuming whatever
- 50:41identity means they're going to take.
- 50:43Can I get to that database? Not oh, who issued that?
- 50:46Attacker did not care. When was this last rotated?
- 50:49Does it work or not? That's the only thing they care
- 50:51about. So that's the pragmatic
- 50:53approach. What is the access mechanism
- 50:55itself we can hone in on? So that's first, do you have a
- 51:01list, an inventory of all of these?
- 51:05Do you have visibility into all of your secrets?
- 51:08And then start mapping from there.
- 51:11Like what state is it in? Like what does this go to?
- 51:13Where? Where was it issued?
- 51:15Who issued it? But also you can start turning
- 51:19on its head a bit too and say, like, look, what services do we
- 51:21have? What identity providers are even
- 51:24available across our organization?
- 51:27Maybe just start there. Like, hey, look, we have Okta
- 51:30and we depend on that day in, day out.
- 51:32How do our developers manage access for the stuff they're
- 51:38building? Is it just Willy nilly all over
- 51:40the map? They can make whatever API keys
- 51:42they can stamp and good luck. That's a much different
- 51:45situation than look, we have policy and procedure and
- 51:48governance in place that says developers need to go through
- 51:51this portal and give me a direct very specific answer on why does
- 51:57it get an issued, What's the like lifespan of it going to be?
- 52:01Who is in charge of it in case of contingency, like what that
- 52:05person goes away, who inherits that role, Who inherits the
- 52:10responsibility for this thing? But you can't do any of that
- 52:14without at least having that conversation first to like who
- 52:19owns this in the larger sense? What's the plan there and then
- 52:22working for an inventory on your way out?
- 52:24I think that's great advice. I think the other thing I'll add
- 52:26is don't ignore the e-mail that you got from Get Guardian saying
- 52:29hey, you've got something out in the public that shouldn't be
- 52:31there. And actually I'll, I'll put a
- 52:34quick plug for it here at the end.
- 52:36I spend a lot of my time out in the field talking about where
- 52:39we're going next. And it's not going to be long
- 52:42lived secrets standing privilege standing access is awful.
- 52:47It's the root cause of so many of our evils.
- 52:50And yeah, if we solved that magically tomorrow, we'd have a
- 52:52whole new world of evils to deal with in security.
- 52:55I understand that. I'm not saying it's a panacea
- 52:57for all of our problems. But like the crowd strikes 2026
- 53:02global threat report. It's like 89% of breaches were
- 53:07malware free, 82% used a abused A credential for something that
- 53:13wasn't a human. I I don't hold me to that one
- 53:16either. Like I.
- 53:17It seems like every report seems like around that 80% mark is
- 53:20it's an identity related breach of some sort.
- 53:23I think 82, that's the number I want to have my head.
- 53:25But, you know, read the report yourself, Tell me if I'm wrong.
- 53:29But it's still, yeah, it's a terrifying number.
- 53:31So everyone is kind of in general agreement that we need
- 53:35to move away from the piece of data should grant access.
- 53:40That's the EV from teleport defines the CEO Teleport defines
- 53:45secret. That way it's a single piece of
- 53:47data that by itself grants access.
- 53:50That's that's it. That's very simple.
- 53:51Like no matter what form it takes, you have it, you can get
- 53:53in and we need to move to a world of get to a trust boundary
- 53:57and say, Hey, who are you? Oh, I'm me.
- 54:01Can you prove that? Here's my certificate that
- 54:03proves my identity. Let me go check that against.
- 54:06Let me validate that against the issuer.
- 54:07I trust this issuer. They're already on my allow
- 54:09list. I trust this endpoint because I
- 54:12was expecting it. Hey, I have a secret here.
- 54:15Can we exchange that public PKI real quick?
- 54:17OK, now I trust you to to tell me that you trust them.
- 54:20That's them. Great.
- 54:21Cool. Cool, cool, cool.
- 54:23Then come back to the transaction boundary and say,
- 54:25all right, what do you want to do?
- 54:30Now I can check the list to see if you're on my allow list for
- 54:32like, you're even supposed to be here and you want to do what?
- 54:35Hey, that totally checks out. Your roles say that's totally
- 54:39cool. Go ahead and make that
- 54:40transaction happen. And then and only then grant
- 54:43access. Actually grant the key.
- 54:45The working model I have in my head is grant the key to the
- 54:48transaction, not any of the entities inside the transaction.
- 54:52So if the transaction is not happening, there is nothing to
- 54:54hold the key. So there's nothing to stand.
- 54:57And it sounds, again, like maybe I'm talking science fiction
- 55:01here. No, this is literally called
- 55:02Spiffy Inspire spiffiest secure production identity framework
- 55:06for everyone. It's a graduated project and the
- 55:10CNCF, it is just as mature and their world view as Kubernetes
- 55:13itself is. It's been around for over 8
- 55:16years now and it's been cloud native like how Kubernetes
- 55:20should have been working the entire time for a while.
- 55:23But the great hope is we're now seeing this adopted much wider
- 55:27across the industry and things like whimsy, the workload
- 55:31identity and multi system environments.
- 55:34It's a fun name for a really serious project.
- 55:36And there's a lot of Internet engineering task force IATF
- 55:41projects out there trying to deal with this right now.
- 55:43But let's make an operational standard that that transactional
- 55:46dance can happen automagically. Just that's how the protocol
- 55:51works. The same way you don't think
- 55:52when's the last time you actually sat down and thought
- 55:54about it, how MTLS actually works.
- 55:56You don't you just like that's, that's just a thing that
- 55:58literally MTLS is just a way to deal with transport layer
- 56:03security that we're going to get there sooner than later.
- 56:07I know this for a fact because we are seeing this widely being
- 56:11pushed by the platforms themselves.
- 56:13But this was just like a couple of little players, niche players
- 56:16here and there that were trying to solve this.
- 56:19Like I'll throw out a couple names.
- 56:19Ambit Teleport the hash core is working on something like this
- 56:25cyber arks working on something like this.
- 56:26That's why they bought venify. I would be like OK, that's
- 56:30that's still like a futuristic, but the platforms themselves
- 56:33have made some major, major advancements in 2025.
- 56:37And the biggest one I can point to is AWS security token service
- 56:40AWSSTS that that service has been around for a while.
- 56:44It lets you call the issuer of an identity certificate like
- 56:51just with an API call inside of AWS.
- 56:53So hey, I'm going to make this call as a entity, as an
- 56:57identity, give me this certificate I can take with me
- 57:01across platforms. And it always worked internally
- 57:04with AWS, but now it works across boundaries.
- 57:07So you can take it to Azure, you can take it across to Google
- 57:10Cloud, you can take it to data bricks, you can take it to a
- 57:12growing list of providers and say, no, we're expecting STS and
- 57:17the way they build the protocol is really, really cool.
- 57:18I'm not going to get into the meat of it.
- 57:20I don't work for AWS, but it makes it very simple to the
- 57:24verification process is very straightforward.
- 57:26It's very standardized feeling. And I think we're going to see
- 57:29more and more of that where the developer will say, look, I need
- 57:33to authenticate to this service. I think we're going to see it
- 57:36involved in the MCP ecosystem sooner than later as well.
- 57:41Like what is the thing calling? Why is it calling?
- 57:44What's it trying to do? And let's build all of that in
- 57:47together to like, let's verify that, verify it, and then issue
- 57:52a trust boundary agreement just for this transaction.
- 57:58Let it go away. It's not free.
- 58:00I mean, it's actually no STS itself is free for everything on
- 58:03AWS. This is the, the big
- 58:05announcement I made in November. That's why I'm so excited about
- 58:07and that's the one I talk about. It's like AWS believes in this
- 58:09so much that every developer on AWS could be doing it this way
- 58:12instead of API keys for everything they do.
- 58:14Well, at least for things they can work with those particular
- 58:18SAS providers. But like operationally, and
- 58:22that's where I'm a developer advocate.
- 58:25Yeah. If I'm doing a million
- 58:26transactions a day and you're telling me I need a 10 extra
- 58:29milliseconds for this handshake, 1,000,000 * 1010 million
- 58:33milliseconds, that's actual real time.
- 58:35That impacts bandwidth. That impacts a lot of things.
- 58:39Where is the line? Do we do it tinfoil hat mode
- 58:42where we do this for every transaction?
- 58:44Or do we allow like O off type? Like, hey, if as long as it's
- 58:49being the same identity, let's keep extending it.
- 58:51That's where the conversation should be and that's where I'm
- 58:54always happy to engage. It's like, well, let's figure
- 58:57out what you actually need. Let's figure out what the limits
- 58:59of that are. I mean, let's tune that.
- 59:00But wow, that is such a better plan than I'm just going to put
- 59:04a key and if an attacker gets it, they can just do whatever
- 59:06they want. It's really hard to take over a
- 59:09certificate authority. It's really hard to take over a
- 59:11platform. It is really hard to impersonate
- 59:14a specific stack and IP and make everything matched with that
- 59:17credential was expecting or what the certificate authority was
- 59:20expecting you to be. So if you come back, it's like,
- 59:23hey, is this them? Like, no, that's clearly not
- 59:25them. Don't, don't let them in.
- 59:27Do not do anything with that. That's a much safer world in my
- 59:30opinion. I love your passion and
- 59:32enthusiasm for this and I'm going to have your LinkedIn
- 59:35profile in our show notes so people can reach out debate
- 59:38respectfully of. Course.
- 59:40You know, I want to ask you a final question unrelated to
- 59:44identity and get and all that stuff, but if I want to learn
- 59:48more about everything you just talked about, right, Spiffy
- 59:50Spire whimsy. We've done a few episodes, but
- 59:52it's been a while on those. So people kind of listen back on
- 59:55on the podcast about that. But how do I get up to speed on
- 59:58this? Is there like a you have a
- 59:59couple of resources you just kind of share with the audience?
- 1:00:02The number one book I have told anyone to read in the last two
- 1:00:06years of my life has been the Spiffy book spiffy dot IO slash
- 1:00:10book. It's 198 page PDF and explains
- 1:00:14all of this. There's even a section about how
- 1:00:16much developer time you could potentially save a week and
- 1:00:19their estimate is on average about 5 hours per developer a
- 1:00:22week. Like just don't have them deal
- 1:00:25with any of the nonsense of permissions or API keys or
- 1:00:31secrets in general. Just like figure out this better
- 1:00:33way to deal with identity. Let the AP is take care of it
- 1:00:36automatically. I say automatically a lot, but
- 1:00:39it feels like that from the developer perspective.
- 1:00:41It's like, what do you mean? I don't have to manually set
- 1:00:43this. I just got to allow list it.
- 1:00:44What what are you talking about? Like that sounds magical.
- 1:00:47So I would start there. It's a it's a great read.
- 1:00:50There's a lot of great resources whimsy.
- 1:00:53I would read the standard. You can just look up WIMSE and
- 1:00:58i.e. TF there's a working drafts.
- 1:01:01I wouldn't tell you where to start on that, but start I guess
- 1:01:05on their front page and look at the mission statement and like
- 1:01:07what they're actually doing. It's always in debate what we're
- 1:01:10doing with that or tongue up around attestation and what
- 1:01:14falls it within and without the spec as far as, you know,
- 1:01:17certain things. But you know, that's that's the
- 1:01:19ITF in general. It's fun, fun to watch.
- 1:01:21I'm not a member. I just, you know, fanboy outside
- 1:01:23looking at all that. I've written about this a few
- 1:01:26times. If you want to get your hands on
- 1:01:28it and like actually get going, Spiffy is a set of ideas the
- 1:01:32same way Oauth is, but they're in their Spire, which is the
- 1:01:35Spiffy runtime environment. So if you want to download and
- 1:01:38just start actually testing and seeing what you can do with it,
- 1:01:41it's completely free and open source like everything else.
- 1:01:45Cloud native, like they made it open source for a reason, so we
- 1:01:48can all get our hands on it. Tons of other reading you can do
- 1:01:51out there on the subject, but that's where I would generally
- 1:01:53start. Oh, and check out a WSSTS.
- 1:01:57There's a really, really great blog announcement from November
- 1:02:01that explains everything, basically.
- 1:02:03I know about it. What I was saying came directly
- 1:02:07from that blog post and it explains the architecture a
- 1:02:10little bit better than I could articulate.
- 1:02:13Yeah, that's right. Those are the resources I would
- 1:02:15start with. Of course, we write about this
- 1:02:16on our blog as well, so you can always check out our blog
- 1:02:18blog.getguardian.com and happy to send people down that path.
- 1:02:24I appreciate that I'll have some of those links in our show notes
- 1:02:26for people to check out, but it's always good to have more
- 1:02:28resources we can learn from. So let's let's pivot to ending
- 1:02:33this on a lighter note because I feel like we have all these
- 1:02:35problems in the world. Let's, you know, figure out how
- 1:02:37we're going to get to 0 standing privilege that we've been
- 1:02:39talking about forever. It's like it feels like it's
- 1:02:40always on the horizon somewhere, but it's a mirage.
- 1:02:43Some What do you do for fun when you're not being a principal
- 1:02:49developer advocate? It is.
- 1:02:52It is springtime here in Chicago and I live fairly close to
- 1:02:55Wrigley Field. I live about a mile from Wrigley
- 1:02:57Field. So love the Cubs.
- 1:02:59Go, Cubs go. Just put on a winning streak
- 1:03:01right now and touch wood. It's the same gym as that here
- 1:03:03because he is a huge baseball fan and I am not, but I am
- 1:03:06definitely a Cubs fan. No worries, no worries.
- 1:03:08So that's how I spend right now. That's how I spent last evening.
- 1:03:11Spent the day before evening as enjoying the Cubs with some
- 1:03:14friends of mine. But outside of that, like I cook
- 1:03:18a lot. I been cooking for a number of
- 1:03:22years, but I got really into making hot sauces during the
- 1:03:24pandemic and that has evolved into other spices, dry rubs and
- 1:03:30tahines and whatnot, like spicy, sour, salt stuff.
- 1:03:37Started messing around with like really really hot Peppers at one
- 1:03:41point where it was a little too intense for actual sauces.
- 1:03:44For my friends, I do it all as a hobby.
- 1:03:46This is like not a side business or anything, it's just something
- 1:03:48I enjoy doing. But I, I love hot food and
- 1:03:54sharing food with people And it's a, it's pretty good way
- 1:03:57because if you're a hot sauce with someone and I like, I don't
- 1:04:00like hot sauce. I don't know, I don't know, what
- 1:04:01do you put this on? How do I store it?
- 1:04:03But if you start making like dry rubs, it's like, well, that's
- 1:04:05never going to go bad shelf stable forever.
- 1:04:07And if you don't like it, you can use a little tiny bit and a
- 1:04:09big thing and that's a little bit of a kick to it.
- 1:04:12That's what I was telling people.
- 1:04:13So I got a few that I have refined down that I'm really
- 1:04:16proud of, that I've shared around with my friends here and,
- 1:04:19you know, be carrying around with me.
- 1:04:20If you see me at future events, I might have some in my bag.
- 1:04:23All right, so if you're going to be at ideniverse, maybe where
- 1:04:26this year, that'll be a good spot to try it.
- 1:04:28I, I am not a spicy food person. I, I, I do not want to have an
- 1:04:33experience while I'm eating. That makes sense.
- 1:04:36I want to enjoy the food, but I do like a little bit of
- 1:04:39something, like if you were going to create something for
- 1:04:42me, you know, or recommend something, you know, that I
- 1:04:44should start with in this world, like what would be like, OK,
- 1:04:48start with just a pinch of this. Oh, Scotch bonnet, buy dried
- 1:04:53Scotch bonnet Peppers and just start there.
- 1:04:56It's the base heat of anything jerk like jerk chicken that that
- 1:05:02jerk spice from the Caribbean and it's just it's a really
- 1:05:05mellow spice. It's hot, but it's nowhere near
- 1:05:11like ghost pepper that level. It's it's a reasonable pepper.
- 1:05:15And if you put a little tiny bit in a dish, you'd be like, oh,
- 1:05:17that really perked that up. And it added a nice fruity
- 1:05:21flavor underneath it all that you can't really get any other
- 1:05:24way if you can get to this. Like part of the reason I love
- 1:05:27ghost pepper is it is the juiciest, fruitiest pepper on
- 1:05:31Earth. It is.
- 1:05:32There's no other pepper taste like that.
- 1:05:34Like, yeah, it's one of the hottest Peppers on Earth, but I
- 1:05:37don't know how else to get to that, to that taste.
- 1:05:39So I actually do want experiences.
- 1:05:41I do want to have AI. Don't know how I would
- 1:05:43experience this any other way. And yeah, I'm sweating a little
- 1:05:46bit. Sure, I'm turned bright red, but
- 1:05:48I'm alive, you know? I'm living it.
- 1:05:51What's the hottest thing you've ever eaten?
- 1:05:57Straight up scorpion. I did a bit of a scorpion pepper
- 1:06:01once and then I in my trophy case behind me, my you can't see
- 1:06:06it because it's way too small, but I have two coins from pepper
- 1:06:09con on there. Pepper Con happens every year at
- 1:06:12the Toxic BBQ at Defcon, which is badly named.
- 1:06:17Badly named on purpose because like everything else is a joke.
- 1:06:19It's the friendliest, nicest part of Defcon.
- 1:06:22It's like we gather at a City Park in Vegas in blazing heat
- 1:06:25and it's free food for everyone and we make donations and people
- 1:06:28bring craft beer and then pepper Con happens.
- 1:06:31So spice enthusiasts bring they're Peppers that they have
- 1:06:35grown throughout the year or a spice blend they have.
- 1:06:38There's always a challenge coin from the people organize it that
- 1:06:41if you take, you know, lick the back of your hand and put a
- 1:06:44bunch of their spice on it, if you can survive that, then you
- 1:06:47get a coin for the year. But other years people have
- 1:06:49brought straight up pepper. They're like little slivers of
- 1:06:51pepper. Yeah.
- 1:06:53But the only one that I can remember eating and it was like,
- 1:06:56I do not like any part of this experience was straight
- 1:06:59scorpion. There's crossbreeds of scorpion
- 1:07:02which are way better tasting like chocolate.
- 1:07:05Scorpions are delicious but straight old school original
- 1:07:08like when they made scorpion Peppers.
- 1:07:10Like that's a little too bitter and it's one of the hottest
- 1:07:13Peppers like known to man. Take me through that scorpion
- 1:07:16experience. Like when did you know?
- 1:07:17Was it immediate or is it one of those that, like sneaks up on
- 1:07:20you and after a few seconds it's like, oh, oh, I'm in trouble?
- 1:07:23It was pretty immediate. Well, because it was a fresh
- 1:07:27pepper. Like you get that juicy little
- 1:07:30like crunch and again, the first thing is like, that's a little
- 1:07:35bitter and then it hits and then it hits exact point on.
- 1:07:40For me anyway, it hit the exact point on my tongue that first
- 1:07:43countered it and then like exploded out to the rest of my
- 1:07:48mouth. And from that point on, I can't
- 1:07:50accurately describe what happened other than I started
- 1:07:53like crying, nose running and then a few seconds I was
- 1:07:57coughing. But where the spice hit, like I
- 1:08:00can describe most spices I I eat like how they hit the tongue,
- 1:08:04like where it rolls over the palate like a wine connoisseur
- 1:08:06might. I'm not a wine person, but it's,
- 1:08:09it's kind of like that. I can say, oh, that's very
- 1:08:10forward on the tongue or that hits the very back of the the
- 1:08:13the sides of the tongue and then like spreads out to the back of
- 1:08:15your throat. This was just everywhere all at
- 1:08:17once. But yeah, I'm not planning on
- 1:08:21doing that again. Some Peppers are really good to
- 1:08:23add to something. I actually have a scorpion mash
- 1:08:28and I use in chili, like not much like an eighth of a
- 1:08:32teaspoon and a giant pot of chili will do you.
- 1:08:34But it adds like that subtle warm heat.
- 1:08:38And if it bounces out with, you know, the rest of the sweetness
- 1:08:40of the dish, then it works really well.
- 1:08:42But on its own, I wouldn't recommend just going out and
- 1:08:44biting into a raw pepper like that's what banana Peppers are
- 1:08:48for. That's what you know, jalapenos
- 1:08:50are for. I eat jalapenos like they're
- 1:08:53good. They're not hot freak some some
- 1:08:54of my friends out, but like love going to the restaurant and
- 1:08:57like, oh, they got jalapenos on. It's got a giant side of
- 1:08:59jalapenos eating like potato chips.
- 1:09:01I find jalapenos hot. So I'm not one of those people.
- 1:09:04I like a little bit like, you know, maybe like a 1/4 of a
- 1:09:07jalapeno in in the environment. But more than that, it's like,
- 1:09:11all right, we got problems. Just scoop out the the inside.
- 1:09:14Next time you get out, buy a jalapeno.
- 1:09:16Just make sure you carefully scoop out the inside seeds and
- 1:09:21that cartilage in there and you'll be fine.
- 1:09:24Like that's where all the heat lies is in in the seeds and in
- 1:09:28the that internal internal oils if you cut.
- 1:09:32Half a seed that's that's about my tolerance.
- 1:09:35If you cut a pepper in half and it ever looks oily, avoid.
- 1:09:38I would avoid that if I were you.
- 1:09:40I look for that like you cut a pepper and it looks oily.
- 1:09:43Like there's a reason it looks oily, and there's that Inside
- 1:09:46that oil is where all the capsaicin is sitting, like ready
- 1:09:49to spring. All right, well, I don't know
- 1:09:50how we can, you know, top that. Look for the oil in a pepper.
- 1:09:54Dwayne, thank you so much for spending time with us and thanks
- 1:09:57again to Git Guard and for sponsoring us.
- 1:09:59I'll have links in our show notes again for everything, but
- 1:10:01go to gitguardian.com/LP's/IDAC. Reach out to Wayne with, you
- 1:10:07know, pepper fax or recommendations or git
- 1:10:12recommendations or, or things like that.
- 1:10:15Last words, Dwayne. I'll let you have them.
- 1:10:17I am always on the road. If you check out my website,
- 1:10:20dwayne-mcdaniel.com, you can see my upcoming events.
- 1:10:24You can see actually past conferences and my blog writings
- 1:10:27and my videos for certain things that have videos for them.
- 1:10:32The appearances I've made and the big encouragement out there
- 1:10:37is talk to your other teams. If you're in security out there
- 1:10:42and you've never sat down with the developer and talk through
- 1:10:44their workflow, that's a really good place to start.
- 1:10:48Like you can't defend what you don't understand.
- 1:10:50You can't defend what you can't see.
- 1:10:51So having that communication layer of, hey, we're just
- 1:10:56talking. I'm just trying to help you do
- 1:10:58your job. I don't want to get anybody
- 1:10:59fired. I don't want to get in trouble.
- 1:11:00There's no emergency right now. But hey, to spend 1/2 hour and
- 1:11:02you just walk me through how you push, walk me through like what
- 1:11:05tools you use, like maybe you can find better ways to improve.
- 1:11:09If you're I am folks and you've never talked dev SEC OPS, that's
- 1:11:12a problem. Go have those conversations.
- 1:11:15Organizationally, we have a tendency overall, I feel to kind
- 1:11:19of silo up like I'm going to get my work done and I'm going to
- 1:11:22let them worry about their work over there.
- 1:11:25And with the current threat landscape, with currently what
- 1:11:27we're going through with machine identity, especially with agenic
- 1:11:30AI and all the identity stuff there, like we can't live on
- 1:11:35islands anymore. And the better relationships you
- 1:11:38have with your other teams, with the organization as a whole,
- 1:11:44that's going to help you better position.
- 1:11:45Like if you're the lone voice out in the wilderness saying,
- 1:11:49look, we got to deal with this because massively bad things can
- 1:11:52happen and no one else saying that, well, you're probably
- 1:11:54going to listen to. But if the heads of development,
- 1:11:57your infrastructure team, your IM team and basically anyone
- 1:12:03else that will listen is like, yeah, this is a hair on fire
- 1:12:05situation. We need to put this fire out.
- 1:12:08That gets listened to, that gets board approval, that gets, you
- 1:12:11know, people want to be resilient.
- 1:12:12People want to be robust. So that that'd be the advice is
- 1:12:15like in a world where we're pushing more and more to build
- 1:12:19everything you can as fast as you can and then manage these
- 1:12:21robot armies. No, I think the opposite needs
- 1:12:24to be true right now. Like, connect with your teams,
- 1:12:26connect with them as people, like, understand what they're
- 1:12:28working on and how you can help them do stuff better.
- 1:12:32That's great advice, Dane. We'll we'll put that in the bank
- 1:12:34for sure. That'll do it for us this week.
- 1:12:38Thank you so much. For people who are watching and
- 1:12:40or listening, you can find us on the web, IDC, podcast.com, like
- 1:12:43share, you don't subscribe. All of that fun stuff helps us
- 1:12:46great guests get great guests like Dwayne and yeah, leave it
- 1:12:49there. We'll talk with everyone the
- 1:12:51next one. Thank you very much.
- 1:12:55You've been listening to Identity at the Center.
- 1:12:58We hope you've enjoyed the show. Make sure to like, rate and
- 1:13:03review, and we'll be back soon. But in the meantime, hit the
- 1:13:06website at identity@thecenter.com.
- 1:13:10See you next time on Identity at the Center.