Latest / The Application Security Podcast / Dustin Lehr -- Culture Change through Champions and Gamification

Transcript

Dustin Lehr -- Culture Change through Champions and Gamification · The Application Security Podcast

0:00
Chris Romeo: Dustin Lehr transitioned from a 13 year career as a software engineer and application architect into cybersecurity leadership.
0:08
Chris Romeo: With experience in retail, DOD, and video games, he fosters collaboration between dev teams, engineering leaders, and software security advocates to design engaging security programs.
0:20
Chris Romeo: Currently, he's the Senior Director of Platform Security slash Deputy CISO at Fivetran, and the Co Founder and Chief Solutions Officer at Katilyst Security, focusing on building security culture and champion programs.
0:34
Chris Romeo: He also co leads the Let's Talk Software Security remote meetup.
0:40
Chris Romeo: Dustin joins us to discuss the perspective of security champions from a developer's Exploring what makes a developer champion unique.
0:48
Chris Romeo: The motivations for becoming one and the benefits of participation.
0:52
Chris Romeo: We conclude with insights into effectively implementing gamification that resonates with those same developers.
1:37
Chris Romeo: Hey folks, welcome to another episode of the application security podcast.
1:41
Chris Romeo: This is Chris Romeo.
1:42
Chris Romeo: I'm the CEO of Devici and co host of the podcast.
1:46
Chris Romeo: I'm also joined by my good friend, Robert Hurlbut.
1:48
Chris Romeo: Hey, Robert.
1:50
Robert Hurlbut: Hey Chris, yeah, Robert Hurlbut, and I'm a principal application security architect and threat modeling lead at Aquia, and really excited about the podcast today.
1:58
Robert Hurlbut: Always excited about it, but, you know, diving in to some more application security topics.
2:03
Chris Romeo: Yeah, we've got an old friend, uh, which I'm realizing I've only known for three years, but I feel like I've known Dustin Lair for a lot longer than that.
2:11
Chris Romeo: Um, who I think of as one of the brilliant, uh, people in AppSec and, uh, always challenges my thinking.
2:17
Chris Romeo: And sometimes I challenge his when he's on stage at a conference and I ask a question from the audience that I probably was maybe harder than I thought it was going to be, but, uh, he handled it brilliantly as always.
2:31
Chris Romeo: So, Dustin, it's great to have you back on the show.
2:33
Dustin Lehr: It's great to be here.
2:34
Dustin Lehr: Yep.
2:34
Dustin Lehr: So I'm Dustin Lair.
2:36
Dustin Lehr: Like Chris said, I've been around AppSec for quite a bit of time and spent a lot of time talking to Chris and Robert in the past about, you know, unique approaches and different approaches to AppSec.
2:48
Dustin Lehr: So happy to be here.
2:50
Chris Romeo: Yeah, and you were, uh, on a previous episode entitled Advocating and Being on the Side of Developers.
2:56
Chris Romeo: So, if folks want to look up and, and hear your origin story, they'll have to go to the archives to find it.
3:01
Chris Romeo: Because we don't, we don't repeat our origin stories a second time, but it also gives us the ability to jump right into the topic, which is always fun as well.
3:09
Chris Romeo: So, Dustin, you spent a lot of time focusing on security champions, gamification, um, security culture, the intersection of all of these things.
3:19
Chris Romeo: And so I know I've heard, I've heard you on a number of other interviews and talks and things talking about champions, but one of the things I realized is a lot of times those conversations are going from the side of security.
3:31
Chris Romeo: So I thought it'd be a little fun today if we flip this around a little bit and we started thinking about this and we, and we collected your answers from the perspective of the developer, the development person who's out there potentially wanting to become a champion, potentially being asked to become a champion.
3:49
Chris Romeo: And so to kick this off, go ahead, put your developer hat back on.
3:53
Chris Romeo: I know you were a developer for a good part of your career as well.
3:56
Chris Romeo: If you're a developer.
3:58
Chris Romeo: What's the problem from your viewpoint?
4:04
Dustin Lehr: Yeah.
4:04
Dustin Lehr: And I think, uh, this'll be good to expand on a little bit as well, because I don't think it's necessarily.
4:10
Dustin Lehr: Any developer.
4:11
Dustin Lehr: I think there has to be a little bit of a certain mindset that a developer has that would attract them to something like a security champion program.
4:21
Dustin Lehr: And what I mean Really lines up with sort of my approach in my career.
4:27
Dustin Lehr: And one of the reasons that I got into security to begin with is because I always had a focus on kind of the bigger picture, kind of the, the quality elements, not just make the feature work as quickly as possible, hack it together.
4:41
Dustin Lehr: But can we make something that's maintainable that other people can read?
4:45
Dustin Lehr: Right.
4:46
Dustin Lehr: How do we refine our process to be more efficient when it comes to our development?
4:50
Dustin Lehr: Uh, and then obviously other things like security and performance and that sort of stuff.
4:56
Dustin Lehr: So I think, you know, finding developers who have more of that mindset.
5:01
Dustin Lehr: They're already thinking about these things.
5:03
Dustin Lehr: Um, I think those are your initial allies.
5:05
Dustin Lehr: Those are your initial sort of security champions that you could, um, start to work with.
5:11
Dustin Lehr: Um, because I think they already get it to a degree.
5:13
Dustin Lehr: And I think one of the biggest benefits of a champion program as well is, You know, finding advocates who can help spread the word, right?
5:22
Dustin Lehr: So you can sort of connect with folks that already sort of get it, uh, and then go from there.
5:27
Dustin Lehr: And in my view, I think a lot of those, um, developers who do get it, they don't necessarily want to become security people.
5:35
Dustin Lehr: Like, that's That's not the goal is to like shift them to become security people.
5:39
Dustin Lehr: They're happy.
5:40
Dustin Lehr: They're happy as builders.
5:41
Dustin Lehr: They just maybe want to get a little bit more involved in, you know, thinking about security.
5:47
Dustin Lehr: And, and I think the other thing I would add to this is I do think security helps to drive a lot of the development quality practices that should probably be.
5:58
Dustin Lehr: Driven by engineering in a lot of cases, things like a good asset inventory system, uh, you know, better, more efficient SDLC processes.
6:07
Dustin Lehr: You know, we talk about securing the SDLC.
6:10
Dustin Lehr: Sometimes the first step is building an SDLC and you'll find developers who are trying to actively pursue that.
6:18
Dustin Lehr: You know, can we have a better process?
6:20
Dustin Lehr: They're trying to think about that stuff.
6:22
Dustin Lehr: So, Again, I think finding those allies and working with them, uh, is kind of the way to go.
6:29
Chris Romeo: So I need you to tap back into your old development days here too.
6:33
Chris Romeo: And think about this, like, what was security pushing you for when you were a developer?
6:39
Chris Romeo: Like, that's what I'm trying to get to as far as what's the problem.
6:42
Chris Romeo: If I'm a developer, like, we've all got decades of security experience.
6:47
Chris Romeo: And so I feel like we have, Can you have security colored glasses?
6:50
Chris Romeo: Security is not really a color, but it's, the metaphor works, security colored glasses, because it changes, like, we use terminology, we, we have concepts and things that we all just believe in, right?
7:01
Chris Romeo: We don't have to, we don't, you don't have to convince Robert or I about the benefits of a champions program, but if you're a developer who has really not had any exposure to security before, Like, is there a problem that you're seeing when you're looking at, when you, when you're listening to security, say, now you have to do this.
7:19
Chris Romeo: You have to do this.
7:20
Chris Romeo: You have to run this tool.
7:21
Chris Romeo: You can't, uh, how dare you have a bug that's been this long in the queue and not been fixed?
7:26
Chris Romeo: What's, what do you, do you see, if you're a developer, do you see a problem or are you just sitting there going, I don't know what these folks are talking about?
7:34
Dustin Lehr: Yeah.
7:34
Dustin Lehr: And I think trying to realize that there's different approaches is important too.
7:41
Dustin Lehr: I do think you'll have people that get it immediately.
7:43
Dustin Lehr: I think you will have people that say, yeah, that is a problem.
7:47
Dustin Lehr: I've been trying to get my engineering leadership to think about these types of issues for a long time.
7:52
Dustin Lehr: Thank you for coming along and saying the same thing.
7:54
Dustin Lehr: Maybe they'll listen to you better than they listen to me.
7:56
Dustin Lehr: Right.
7:57
Dustin Lehr: But I'm not sure that that's the typical developer mindset.
8:00
Dustin Lehr: Uh, it could be, but I also think there's another class of developers who are like, look, I'm on tight deadlines.
8:09
Dustin Lehr: I want to build stuff.
8:11
Dustin Lehr: What is this that you're bothering me with?
8:13
Dustin Lehr: Like, is this really a problem?
8:15
Dustin Lehr: And, you know, and I think back from my development days, it's like, I'm very short on time.
8:21
Dustin Lehr: I've got a million things to do.
8:22
Dustin Lehr: Unless you can demonstrate that this is, like, an exploitable actual issue, not this, you know, kind of, uh, I don't know, like, some sort of, uh, almost esoteric type thing, like, like this imaginary thing, right?
8:43
Dustin Lehr: Um, if you can, if you can demonstrate that it's actually an issue, that's what I need to actually take action on it.
8:49
Dustin Lehr: Otherwise, why would I waste my time?
8:52
Dustin Lehr: You know, and I think devs are just generally very short on time and that's, that's, that's their mindset.
8:58
Dustin Lehr: Like I've got deadlines to hit.
9:00
Dustin Lehr: I've got features to develop.
9:02
Dustin Lehr: Why would I waste my time on this?
9:04
Chris Romeo: Cause a lot of times developers say you're living in an agile world.
9:09
Chris Romeo: Right?
9:09
Chris Romeo: I know there's no real such thing as an, there's no defined Agile world, but you're following an Agile like process.
9:15
Chris Romeo: Let's call it that.
9:17
Chris Romeo: You, at the beginning of a, of a sprint, You, maybe you have, there's story points or something that are being used, but you're getting a block of work.
9:25
Chris Romeo: And the expectation is, by the end of that sprint, you will have moved that block of work to whatever state.
9:31
Chris Romeo: Sometimes it's not complete, sometimes things cross over, but a lot of times it's, you have these three tasks, Dustin, and two weeks from now, during, on demo day, you're gonna need to, do a demonstration for us.
9:42
Chris Romeo: And so you're, you're, and so that's really what's driven.
9:45
Chris Romeo: And from what I've seen in a lot of organizations, that's what your bonus is keyed on as well.
9:51
Chris Romeo: It's not keyed on, it's keyed on, you know, how, how well did you meet your sprint deadlines?
9:55
Chris Romeo: And so what's been your experience as far as the pressures that developers have?
9:59
Chris Romeo: to deal with.
10:02
Dustin Lehr: Yeah, I think it's just like you described.
10:04
Dustin Lehr: I think there's expectations that you deliver something specific in a sprint.
10:08
Dustin Lehr: You know, you story pointed it, you planned it out, and you have to deliver it at that point.
10:15
Dustin Lehr: Uh, and there's really nothing else.
10:16
Dustin Lehr: Like I remember very very many occasions where even a meeting or an email is too much, right?
10:26
Dustin Lehr: Like, can you just clear my schedule?
10:29
Dustin Lehr: Do you want this feature to get done or not?
10:32
Dustin Lehr: Why do we have to have a conversation?
10:34
Dustin Lehr: You know, and I think we'll get into this in a little bit, uh, a little bit later, but, um, You know, I was very heads down.
10:41
Dustin Lehr: I think a lot of developers are very heads down to the point that some security person reaches out for any reason.
10:47
Dustin Lehr: And they're like, first of all, you broke their concentration.
10:50
Dustin Lehr: Like, I think this is a huge thing for non developers to understand.
10:54
Dustin Lehr: It takes a lot of concentration to be able to code correctly or code accurately.
11:00
Dustin Lehr: Right.
11:01
Dustin Lehr: You got to think about how your little piece of code is going to fit into all the other moving parts.
11:06
Dustin Lehr: You keep all this stuff in your head, you're in the zone.
11:09
Dustin Lehr: Any sort of interruption that comes in is annoying, breaks your train of thought, has to essentially lead you to context switch, which is a very expensive thing to do.
11:21
Dustin Lehr: Um, I also remember lots of like troubleshooting, you know, hey, there's a production issue, can you look at this?
11:28
Dustin Lehr: And then if it's a high visibility issue, Engineering leadership or even leadership from other departments are like pinging you.
11:35
Dustin Lehr: Hey, how's it coming?
11:36
Dustin Lehr: What, you know, is there any progress?
11:39
Dustin Lehr: It's like, okay, do you want me to make progress or do you want me to answer your question?
11:44
Dustin Lehr: Right.
11:45
Dustin Lehr: So, and I think that's the mindset.
11:47
Dustin Lehr: So anything that comes in sideways is like pretty immediately annoying for a developer.
11:55
Dustin Lehr: How's that for
11:56
Chris Romeo: me think of, you guys ever seen the uh, Maker vs.
11:59
Chris Romeo: Manager article?
12:01
Chris Romeo: I think it's an article, I don't even know where it originally was posted.
12:03
Chris Romeo: You should look it up, it's fascinating.
12:05
Chris Romeo: It's this, it's, it, it speaks to that exactly.
12:08
Chris Romeo: Cause like people either have a Maker schedule or they have a Manager schedule.
12:12
Chris Romeo: Manager schedules are based on filling up your calendar with meetings.
12:16
Chris Romeo: So that you can understand what's happening and you can manage better.
12:19
Chris Romeo: And maker schedules are all about freeing your calendar of all of those things and giving you blocks of time to do what you just described about getting in the zone and, and being able to put out some good, decent code that works because nobody's bothered you and caused you to, to make an error in the code because you got interrupted.
12:37
Chris Romeo: And by the time you came back, you missed something.
12:39
Chris Romeo: Oh, I was supposed to validate that input, but I just, I missed it because it was, uh, I got interrupted so many different times.
12:46
Dustin Lehr: Yeah, now that said, I think, you know, again, I spent a good part of my career in that mindset.
12:56
Dustin Lehr: But I would say that since then, I've learned the importance of communication.
13:02
Dustin Lehr: I actually have a rule.
13:03
Dustin Lehr: It used to be like 90%, you know, heads down, work, leave me alone.
13:08
Dustin Lehr: Uh, Maybe 10 percent was allocated to meetings and emails and everything else.
13:14
Dustin Lehr: Um, I think to a certain degree, you're limiting your career if you take that approach, okay?
13:21
Dustin Lehr: Like you're a builder.
13:21
Dustin Lehr: Great.
13:22
Dustin Lehr: You're getting stuff done.
13:24
Dustin Lehr: My rule of thumb now is, is 50 50.
13:26
Dustin Lehr: Even for tech people, like you need to be talking about what you're building and sharing with other people and mentoring and doing that kind of stuff.
13:35
Dustin Lehr: 50 percent of your time, uh, 50 percent coding heads down, leave me alone.
13:39
Dustin Lehr: But I do think for the first part of my career.
13:43
Dustin Lehr: I didn't, I wasn't able to go as far as I could have if I did spend more time sharing what I was building, because otherwise people, you know, people aren't going to know, like you, you just refactored this entire piece of code to make it more efficient, but you didn't tell anyone about it.
14:02
Dustin Lehr: You know, there's, there's a few problems with that.
14:04
Dustin Lehr: Like you haven't communicated that people, the next person that goes into that piece of code is going to be like, what the heck just happened?
14:10
Chris Romeo: yeah.
14:11
Dustin Lehr: You just made a quality move that needs to be communicated more broadly.
14:17
Dustin Lehr: Why did I refactor this?
14:19
Dustin Lehr: Because I found it was unmaintainable.
14:21
Dustin Lehr: It was basically tech debt.
14:23
Dustin Lehr: Well, what's tech debt?
14:24
Dustin Lehr: Well, let me help you understand that because that's important.
14:28
Dustin Lehr: And maybe we can better align our development team to focus on things more like tech debt.
14:32
Dustin Lehr: And if you, again, if you don't say anything about that, it's not going to go anywhere.
14:37
Dustin Lehr: You know, you can't just be like this silent hero working in the background and never telling anyone, you know, what you're doing.
14:46
Chris Romeo: Yeah, it's it makes me think about like the rise of the staff engineer that we've seen over the last couple of years, and I think, you know, I mean, I was at Cisco for a long time, and we had principal engineers, which were effectively what, you know, I think people are calling staff engineers now, but really the difference between a staff engineer and a senior engineer, staff engineers can code.
15:06
Chris Romeo: They can both code at a super high level, probably the same level.
15:09
Chris Romeo: A staff engineer can bring five other people with them on the journey.
15:13
Chris Romeo: So the staff engineer is the one that can be interrupted during the day when somebody says, uh, I just don't, I don't know how to design this right.
15:22
Chris Romeo: Can you help me?
15:23
Chris Romeo: And the staff engineer is the person who can look at it and say, okay, here's what I would do.
15:26
Chris Romeo: One, two, three, this would be the order how I do these things.
15:29
Chris Romeo: Hopefully it'll help you.
15:30
Chris Romeo: So they're, they're able to draw people forward and make them better.
15:33
Chris Romeo: And so I think that's a good, good point from your side.
15:35
Chris Romeo: Like the, the fact that you can definitely hinder yourself in your career.
15:40
Chris Romeo: It doesn't matter if you're a developer, if you're a security person, it doesn't matter what you do.
15:43
Chris Romeo: If you are, if you can only put your head down and say, I need 90 percent of my time to put my head down to be successful.
15:50
Chris Romeo: You're never going to move beyond, you might get to a senior engineer because you can, you can still be a senior engineer and, and not be as good at communicating, but that's going to be the end of the road for you.
16:01
Chris Romeo: There's no, there's no place higher you're going to be able to go.
16:07
Robert Hurlbut: So, Dustin, you mentioned a couple things that, um, for yourself in terms of what you saw, what, you know, quality and so forth that was important to you, but what else would you say is unique about a developer that would lead them to becoming a champion?
16:25
Dustin Lehr: Yeah, I mean, I was mentioning before, I do think that sort of bigger picture view is important.
16:31
Dustin Lehr: I think realizing that communicating and, and that you can learn from other people besides just heads down research.
16:40
Dustin Lehr: Um, Realizing that you, you don't know what you don't know to a degree.
16:46
Dustin Lehr: There's this sort of humility that I think is important as well.
16:51
Dustin Lehr: Um, you know, do I think I know everything or, or is there room for things that, you know, that I don't even know that I should know?
16:59
Dustin Lehr: Um, I think that's important.
17:00
Dustin Lehr: I also think there has to be this sort of care for, um, communication and then sort of the people side to a large degree as well.
17:11
Dustin Lehr: Like, How do I help other people learn?
17:13
Dustin Lehr: You know, I just learned some stuff that I didn't know I didn't know, you know, I think having a desire to help other people be enlightened with that stuff that you learned, I think that's important as well.
17:26
Dustin Lehr: Um, you know, so there's a bit of a care for your fellow developers, um, and, and sort of wanting to help them too.
17:33
Dustin Lehr: And I think that's, that's important.
17:35
Dustin Lehr: That's important because of how champions are typically positioned.
17:39
Dustin Lehr: And that's, they're the representation of security within their teams.
17:43
Dustin Lehr: They're learning stuff that then they're expected to turn around and share with their teams.
17:48
Dustin Lehr: Um, so I, I do think that mindset is important as well.
17:53
Chris Romeo: Can a developer become a champion with zero security knowledge?
17:58
Chris Romeo: There, you get a developer, they're like, you're on a call and they're like, OWASP top who?
18:03
Chris Romeo: Is that somebody who can become, can they become a champion in your
18:06
Dustin Lehr: Why not?
18:08
Dustin Lehr: Yeah.
18:08
Dustin Lehr: So I, I actually get this question a lot is, is it, you know, is the opportunity to become a champion only for senior level people, right?
18:18
Dustin Lehr: Only for people who understand security.
18:22
Dustin Lehr: And what I find is there are benefits to actually having less experienced people be part of it too, especially if they, uh, start to, you know, drink the Kool Aid a little bit and they get excited about it and they start to go down that rabbit hole.
18:38
Dustin Lehr: I don't, I don't know how, and maybe we can talk about your experiences with this, but when I started going down the security rabbit hole, uh, I didn't know anything at first.
18:46
Dustin Lehr: I read a book on, uh, what was it?
18:49
Dustin Lehr: Um, PHP security of all things, uh, way back in the day.
18:55
Dustin Lehr: And I was like, this is fascinating.
18:57
Dustin Lehr: And it just led me down this path towards security.
19:01
Dustin Lehr: You know, I think finding someone that's sort of just getting into it and.
19:05
Dustin Lehr: Starting to go down that rabbit hole, uh, is, is awesome.
19:09
Dustin Lehr: And I think, you know, bringing these new, fresh ideas, uh, from someone who's less experienced is a great benefit to your champion program.
19:20
Dustin Lehr: And frankly, it's a great benefit to our security industry in general.
19:24
Dustin Lehr: I think there's this archetype mindset where, you know, there's this model.
19:30
Dustin Lehr: Purple unicorn.
19:32
Dustin Lehr: person that should be a cybersecurity person, but I actually think there's a whole lot of positions, marketing and sales, and I don't know, psychology, that have a lot to give to cybersecurity, you know.
19:46
Dustin Lehr: So even if they know very little, I think, and I talk to folks about this all the time, like bring, bring your unique, Uh, background into cybersecurity.
19:58
Dustin Lehr: And we should welcome that because it's going to force us to think of things in different ways.
20:04
Dustin Lehr: So, yeah.
20:05
Dustin Lehr: So I think that answers your question.
20:08
Chris Romeo: Yeah, I was, I was smiling a little bit when you were, as you were going through there, only because I was remembering some past champions that I've worked with who were more senior security knowledge people, and they tended to be more challenging.
20:23
Chris Romeo: And the newer people who are, cause they were, they were, I mean, let's face it.
20:27
Chris Romeo: Security does make us a little, it hardens our edges a little bit, right?
20:31
Chris Romeo: We get a little cynical.
20:33
Chris Romeo: Um, you know, I mean, I've been doing this for 26 years at this point and I look at things and go, it's, it's half empty, not half full, right?
20:42
Chris Romeo: Like
20:43
Dustin Lehr: They're skeptical.
20:44
Chris Romeo: Yeah, I mean, that, that just came from, and to some degree, I think that helps you as a security person.
20:50
Chris Romeo: But when you're thinking about something programmatically, where you're trying to positively influence change, being cynical is not a, it's not a good thing because, you know, your first answer is, well, they're just gonna screw it up.
21:02
Chris Romeo: You know, like, and that's not my, my, I'm cynical when it, from an architectural perspective in security.
21:08
Chris Romeo: I'm not cynical about the people side.
21:10
Chris Romeo: But a lot of the senior people that I work with.
21:12
Chris Romeo: We're more of a challenge to get them to play nice in the pool with everybody else.
21:17
Chris Romeo: Whereas junior people were more open to, you know, having that humility saying, I know what I don't know, but also being ready to just dive head first into the pool to solve things and then bring other people with them through the process.
21:35
Robert Hurlbut: What does, um, what would a person who became a champion, where do they get out of it?
21:41
Robert Hurlbut: I know we've talked about what they offer, uh, their perspectives, and, and I agree with that, that, uh, even if they have zero to very little experience, it's, it's good for them to be interested and join, but what do they get out of being a champion?
21:57
Dustin Lehr: Yeah.
21:57
Dustin Lehr: I think there's a handful of things I think to start.
22:02
Dustin Lehr: They're enhancing.
22:05
Dustin Lehr: a portion of their knowledge that is very valuable these days, you know, being able to, and frankly, if I was an engineering leader, uh, this is what I would look for in folks that I would hire too.
22:17
Dustin Lehr: Like, great, you can write code.
22:18
Dustin Lehr: Okay.
22:19
Dustin Lehr: But can you write it?
22:21
Dustin Lehr: You know, in a way that is secure, like, are you going to think about things beyond just making it work?
22:27
Dustin Lehr: Um, and I think there's a lot of benefit becoming a champion to learn that stuff, to learn more about security.
22:33
Dustin Lehr: So essentially it's a resume builder, you know, to sort of put a pin in that one.
22:37
Dustin Lehr: Um, besides that, again, if people are interested in going down the rabbit hole, I think there's a lot of intrinsic value in just learning more about cybersecurity and, you know, Is Hackers the movie accurate?
22:50
Dustin Lehr: Is this really a thing?
22:51
Dustin Lehr: Or is it A little different than maybe they first expected, you know?
22:57
Dustin Lehr: Um, and how do they bring these concepts into their personal lives?
23:01
Dustin Lehr: I, you know, I see this a lot with champions where, you know, maybe not even just the technical ones, but the, the folks who are from sales or something, you know, they learn more about.
23:11
Dustin Lehr: general good security practices, password managers and data sharing and phishing emails and that kind of stuff.
23:18
Dustin Lehr: And they bring it to their personal lives, you know, and they help, they help grandma, uh, you know, secure her situation.
23:25
Dustin Lehr: And so I think there's a lot of benefit from that perspective as well.
23:30
Dustin Lehr: Um, and then I think when you're designing your champion program, you need to think about what are the sort of rewards and incentives that you're going to offer.
23:38
Dustin Lehr: You know, is security part of your performance review in some way?
23:42
Dustin Lehr: If you can make it that way, then there's a lot of benefit, obviously, for, for champions in terms of career growth.
23:49
Dustin Lehr: Um, and then I also think, you know, things like recognition, like being, doing the extracurricular things.
23:56
Dustin Lehr: Uh, I think is always something that is, you know, kind of held as, um, a good reason for promotion or, or just that this person goes above and beyond.
24:06
Dustin Lehr: Not only are they killing it coding, but they're also a security champion as well.
24:12
Dustin Lehr: And they represent our team from a security perspective.
24:15
Dustin Lehr: Um, it's a really easy case to make that this is clearly a leader on our team.
24:20
Dustin Lehr: So there's a lot of recognition and so forth that comes with that too.
24:25
Chris Romeo: Yeah, I can say one of the, one of the biggest rewards in my career has been.
24:32
Chris Romeo: Watching champions succeed, go from developer to joining a security team.
24:38
Chris Romeo: I had a bunch of folks that came out of the Cisco program that were developers, different levels of engineers inside of Cisco, and they either migrated to security roles at Cisco, or a lot of them went to other companies.
24:51
Chris Romeo: And LinkedIn as they're developing.
24:54
Chris Romeo: Progressing through their security careers.
24:56
Chris Romeo: And so that's, that's just, that's just one of the side benefits of, of being around these things is we all know we need to be building the next generation, but the champions program gives you the ability to participate.
25:09
Chris Romeo: And, and follow, then follow along for the years into the future while you watch these folks getting promotions and taking on new roles and stuff.
25:16
Chris Romeo: And it's like, oh, that's really, this is really cool.
25:18
Chris Romeo: I remember, I remember that person when they were just starting out.
25:23
Chris Romeo: What about gamification?
25:25
Chris Romeo: So I know that's something that you think about a lot, Dustin.
25:28
Chris Romeo: Um, but let's think about it from the developer side.
25:31
Chris Romeo: Like how, how are developers influenced and impacted by gamification?
25:37
Dustin Lehr: Yeah, it's funny you mention this because I get some skeptical questions around gamification sometimes where folks are like, do people care about games?
25:49
Dustin Lehr: Like what, why would gamifying this have any sort of effect whatsoever?
25:54
Dustin Lehr: And my response to that is, um, what do you think your developers are doing when they're done with work these days?
26:00
Dustin Lehr: What are they doing?
26:02
Dustin Lehr: Right?
26:02
Dustin Lehr: They're gaming people.
26:04
Dustin Lehr: So, you know, if we're thinking, well, do points matter?
26:07
Dustin Lehr: Does status matter?
26:08
Dustin Lehr: This sort of stuff.
26:10
Dustin Lehr: They're building their World of Warcraft character on the side.
26:14
Dustin Lehr: Like these are things that I think, and maybe it's a generational thing, but we are seeing more and more people really sort of embrace this.
26:22
Dustin Lehr: Sort of gamer mindset.
26:23
Dustin Lehr: Now, let me be clear that gamification does not mean necessarily turning something into a game.
26:29
Dustin Lehr: There's a bit of a misrepresentation of that term.
26:32
Dustin Lehr: It's not my favorite term, to be honest.
26:35
Dustin Lehr: I prefer, um, sort of more of a human focused design type of thinking, where you're appealing to our drives, like our actual human drives.
26:46
Dustin Lehr: And one of the reasons that we look to games for that is because you don't have to play games.
26:51
Dustin Lehr: Why do people do that?
26:53
Dustin Lehr: You don't get money from it.
26:55
Dustin Lehr: I mean, you could in some situations, but for the most part, yeah, right?
26:59
Dustin Lehr: Uh, but for the most part, you're playing it because it has appealed to your drives in some way, right?
27:07
Dustin Lehr: There's urgency with it.
27:08
Dustin Lehr: There's scarcity.
27:09
Dustin Lehr: There's, uh, meaning or, or even just the feeling of accomplishment, you know, that, that is appealing to people.
27:17
Dustin Lehr: So the idea behind gamification is how do we take those elements that the game industry has clearly mastered.
27:23
Dustin Lehr: and apply those to non game situations.
27:27
Dustin Lehr: Uh, and that has just been a fascinating thing.
27:30
Dustin Lehr: I've gone down this road of studying this now at this point for, for years.
27:34
Dustin Lehr: And, um, I will say that the more that I can sort of understand humans through these drives and by thinking about the techniques that games use, the more success I've had in terms of building great experiences where people keep coming back and they do crazy things like.
27:53
Dustin Lehr: We, you know, I've had non security people start book clubs and start other clubs, and it makes me think, like, how would we get there without, without this sort of, uh, motivational element?
28:06
Dustin Lehr: Could you imagine a CISO asking some regular non security person to start a book club?
28:13
Dustin Lehr: Like it just doesn't work, right?
28:15
Dustin Lehr: So they have to basically be inspired to do those things themselves.
28:19
Dustin Lehr: And you know, you know, you've succeeded when you've set up that recognition and sort of reward system that does, um, incentivize that behavior.
28:28
Dustin Lehr: So it's pretty, the whole field is just fascinating.
28:36
Robert Hurlbut: Let's talk a little bit about, uh, on the, on the security side, we talked a lot about developers, but for the team, the, the, The organization that's trying to justify creating a champion program.
28:50
Robert Hurlbut: How would you, or even developers talking to, uh, the VP of engineering, how would they help justify a need for a champion program?
29:03
Dustin Lehr: Yeah.
29:03
Dustin Lehr: I mean, I think this is sort of a culture shift that has to be.
29:10
Dustin Lehr: Um, pursued with care, right?
29:12
Dustin Lehr: So the way that I always try to talk about this stuff is what is the ultimate problem that you're trying to solve?
29:19
Dustin Lehr: I always use the term, I'm going to say this on this show, and I'm, I'm going to get a look from Chris, but this whole concept of shift left, I think makes sense if you think about the right side activities as contributing to those shift left concepts.
29:35
Dustin Lehr: So let me say it like this.
29:37
Dustin Lehr: You need to prove that there's an issue in your environment before you make a change, right?
29:43
Dustin Lehr: So I always say start right, which means do a pen test, look at Prod, look at, you know, like set up scan tools, understand that you have issues, you have vulnerabilities or risks that have been introduced in your systems in prod today, because from there you can start to make a case.
30:03
Dustin Lehr: Well, do you want to prevent those things from existing?
30:08
Dustin Lehr: Do you want to fix those A and B?
30:10
Dustin Lehr: Do you want to find ways to prevent those from happening again?
30:13
Dustin Lehr: And that's when you can start to have a conversation around.
30:16
Dustin Lehr: How do we actually shift some of our activities to earlier in the cycle, or left, um, to start to prevent those things?
30:24
Dustin Lehr: And that's kind of where training and security champions and threat modeling and that, those sorts of activities can be justified.
30:32
Dustin Lehr: Because I think something to think about as a thought experiment is, if you literally hire someone to do a pen test, you set up scan tools, you look for issues in prod, And you don't find anything, you're not going to have any luck building a business case for some of these left side activities.
30:49
Dustin Lehr: So, I do think it starts with that.
30:52
Dustin Lehr: Um, it's a pretty hard conversation to have though because, um, a lot of folks will say, Well, that can't happen to us.
30:59
Dustin Lehr: Or, is that really real risk?
31:02
Dustin Lehr: You know, is ransomware really going to happen to us?
31:07
Dustin Lehr: And I think that's pretty important.
31:08
Dustin Lehr: Yeah, you just need to, that's, that's a hard problem to solve.
31:11
Dustin Lehr: So I'd be curious to know your thoughts about that too.
31:14
Chris Romeo: Well, first I want to put my VP of engineering hat on and I want to, I want to have a little conversation.
31:19
Chris Romeo: I'm going to, I'm going to, I'm going to give you this for my take.
31:22
Chris Romeo: And I'm curious, you can now play the role of security leader, which you are in real life.
31:27
Chris Romeo: Uh, you don't have to play a different role, but let me, let me kind of, let me, let me spin it to you how I perceive it as a VP of engineering.
31:34
Chris Romeo: And I'm curious to see how you would convince me or what, what you would give me to think about.
31:38
Chris Romeo: So, uh, Dustin, I'm.
31:40
Chris Romeo: You know, I, I get you from the security team, but product is driving what we do.
31:46
Chris Romeo: Product drives, everything product hears from the customer product tells us what we got to build.
31:50
Chris Romeo: And so I've only got, I don't have enough resources already.
31:53
Chris Romeo: to keep up with all the things that they want to build.
31:55
Chris Romeo: And so now you're telling me that you want me to, to, uh, put some people in a champions program.
32:01
Chris Romeo: You want me to add these tools.
32:02
Chris Romeo: You want me to do, you want me to, you want to take a slice of my resources here, but I don't see a path forward for me to take a slice of those resources and, um, still get what the customer slash product wants.
32:15
Chris Romeo: And my fear is I won't be here in six months.
32:19
Chris Romeo: If I take your, if I, if I take a slice of my, my folks time and try to apply it to what you're, what you're doing.
32:26
Chris Romeo: So give me some justification.
32:28
Chris Romeo: Give me some rationale for what, how I can keep my job six months from now.
32:33
Dustin Lehr: Yeah, so I do think there has to be a lot of alignment in terms of leadership here, and that's where I would say I would definitely work with the product team to help this engineering leader, Chris, understand that it is actually something that we need to pursue, because for instance, we've got customers who are asking for this.
32:55
Dustin Lehr: We've got customers who are asking, what are our SLAs in terms of fixing vulnerabilities?
33:02
Dustin Lehr: Uh, what, what sorts of risks do you have?
33:05
Dustin Lehr: You know, they're, they have to fill out like SIG lights and, and that sort of stuff.
33:10
Dustin Lehr: You know, I would say the compliance folks on our team can only help so much.
33:17
Dustin Lehr: We need to turn around and actually implement best practices when it comes to this so we can keep them honest as possible, you know, um, so I'd say customers are a big reason.
33:28
Dustin Lehr: I would also say that the, uh, compliance requirements themselves has a lot to do with customers too.
33:35
Dustin Lehr: Like there are strict requirements from, you PCI, and those are even expanding a little bit more.
33:40
Dustin Lehr: I think there's a lot of dancing around that happens.
33:43
Dustin Lehr: This is a whole nother conversation.
33:45
Dustin Lehr: Probably.
33:45
Dustin Lehr: There's a lot of dancing around that happens in terms of answering these types of questions to auditors, um, without any sort of feedback back to engineering, what I would like to see is VP of engineering.
33:58
Dustin Lehr: Um, we just told the auditor this, but we don't do this consistently.
34:01
Dustin Lehr: What do you want to tell them next year?
34:06
Dustin Lehr: They gave us six months or something to correct this.
34:09
Dustin Lehr: If we don't correct this, we're losing our cert.
34:12
Dustin Lehr: You know, and I think a lot of folks are suspicious of certs as sort of being this, this checkbox.
34:18
Dustin Lehr: But I think if you use them correctly, you can make differences in your org.
34:23
Dustin Lehr: So yeah, an imminent threat.
34:26
Dustin Lehr: Sorry, one other thing.
34:27
Dustin Lehr: Imminent threat of a customer or prospect not purchasing our product.
34:32
Dustin Lehr: because of some sort of security concern.
34:34
Dustin Lehr: Like the more you can highlight those, you know, VP of engineering, we just lost this deal because they didn't like our security practices.
34:43
Chris Romeo: then you can tie revenue to, potential revenue to, once again, now we're talking about a business case because we lost money and as long as it's clear, everybody knows we lost it because we didn't have the right security.
34:56
Dustin Lehr: even entire sectors.
34:57
Dustin Lehr: And this is kind of where the product conversation comes in.
35:00
Dustin Lehr: Do we want to appeal to, you know, public sector type companies, uh, government agencies and so forth.
35:08
Dustin Lehr: If that's part of our plan this year, we need to get our stuff together when it comes to security.
35:17
Chris Romeo: Are you hearing anything different?
35:19
Chris Romeo: Um, we can leave the role playing.
35:22
Chris Romeo: We can shift back to just normal conversation here.
35:24
Chris Romeo: I'm not the VP of engineering anymore, that's what I'm saying.
35:26
Chris Romeo: But I'm just curious, like, are you hearing Anything else other than from customers other than what you just kind of described as far as I get the, I get people are asking for certs, they're asking for SOC 2, they're asking for us to fill out um, these what appears to be uh, infinity length uh, questionnaires where you answer a question and then it branches out to eight more questions.
35:50
Chris Romeo: Like no, I just answered one, how do I now have eight that I haven't finished?
35:54
Chris Romeo: That's a whole side topic as well.
35:56
Chris Romeo: Uh, but other than certs like SOC2, um, SLAs, uh, compliance driven type of stuff, anything else like our, is the modern customer, how savvy is the modern customer, or modern buyer I guess
36:12
Dustin Lehr: Much more so than we've seen.
36:15
Dustin Lehr: I think, you know, in terms of what I see at, at, uh, at Fivetran and what I've seen in other companies as well, I do think that There's just generally more awareness around this stuff.
36:27
Dustin Lehr: I think the, the press, uh, around this stuff, the, the, the government, the White House focus on a lot of this stuff has really helped increase that awareness where people are asking harder questions that they may not have asked, you know, five, five or 10 years ago.
36:43
Dustin Lehr: And that is putting a lot more pressure on, on vendors, um, to, you know, or really, you know, depending on your perspective on, on companies, you know, to make sure that they're, um, following what their prospects and customers expect.
36:58
Dustin Lehr: So that's really, I think, done.
37:02
Dustin Lehr: Uh, you know, a lot of favors for the security industry in general.
37:06
Dustin Lehr: So we need to get out there more though.
37:08
Dustin Lehr: I, you know, I think we're heading down that path, but I still think there's a pretty major lack of awareness, especially when it comes to the business side of, of a lot of companies, they just don't realize to what extent there is a risk that they need to pursue.
37:21
Dustin Lehr: Um, and that's all of us, you know, in the security industry.
37:25
Dustin Lehr: I think we need to continue to highlight, uh, concerns, you know, and, and increase awareness.
37:32
Chris Romeo: Yeah.
37:33
Chris Romeo: So, uh, Champion Success Guide.
37:36
Chris Romeo: And Katilyst.
37:38
Chris Romeo: I wanna, I wanna hear about both of these things as far as where they are right now.
37:41
Chris Romeo: I know you wrote the Champion Success Guide a couple of years ago now, so I'm curious to hear about how that's progressed.
37:47
Chris Romeo: And then I also wanna hear about this thing called Katilyst.
37:51
Chris Romeo: Which, the way, for the audience's perspective, I'm giving Dustin an invitation to speak about.
37:57
Chris Romeo: He's not speaking about, I'm, I'm, I'm asking him to.
38:00
Dustin Lehr: fantastic.
38:01
Dustin Lehr: Thanks for clarifying that.
38:02
Dustin Lehr: Um, yeah, so security champion, uh, success guide I wrote almost two years ago.
38:08
Dustin Lehr: And really it was an opportunity to just literally write down everything I know when it comes to building security champion programs so that I could help others, uh, build security champion programs.
38:20
Dustin Lehr: And I think the main reason for that is, um, I've just seen them be so successful in the past that I want other people to be able to build something that is successful as well.
38:32
Dustin Lehr: So that's why I named it the success guide.
38:35
Dustin Lehr: I think that's the exciting part about it.
38:37
Dustin Lehr: So it's all free.
38:38
Dustin Lehr: It explains what a champion program is.
38:40
Dustin Lehr: It sort of gives you a step by step procedure for building your own champion program.
38:47
Dustin Lehr: And one of the approaches that I wanted to take with it is to.
38:51
Dustin Lehr: you know, kind of nod to the fact that every culture is different and you can't necessarily just prescribe a solution.
38:58
Dustin Lehr: Hey, do these three things and it's going to be awesome.
39:01
Dustin Lehr: No, it's more make sure you methodically plan for building something unique for your culture.
39:06
Dustin Lehr: So that's kind of the unique spin that that guide provides.
39:10
Dustin Lehr: And then, um, yeah, Katilyst.
39:12
Dustin Lehr: So I started Katilyst to actually me and a co founder started Katilyst about three years ago now, and we have basically just been helping people shift culture and, uh, build security champion programs.
39:25
Dustin Lehr: Uh, we noticed that there wasn't necessarily a company that was focused on that specifically, so we decided to take that on, and it's been growing ever since, and definitely getting very exciting.
39:36
Dustin Lehr: Uh, we have a handful of services that we provide, um, including designing your program, Helping with things like training, you know, if you have brown bags and so forth, we can help bring folks in to host those.
39:49
Dustin Lehr: Uh, we help with some of the administration, which there's a lot of time, uh, that you should be putting into, uh, your champion program if you're going to make it work.
39:58
Dustin Lehr: So we help with that.
39:58
Dustin Lehr: And then we're actually building a product as well that helps to essentially gamify your program, uh, which is a pretty unique offering as well.
40:06
Dustin Lehr: So
40:07
Chris Romeo: Yeah, very cool.
40:08
Chris Romeo: I've been, I've been following along.
40:09
Chris Romeo: I've been a fan, so I'm, I'm curious to see where, how, as the product progresses, um, as somebody who's run a champions program at large, at scale, um, a lot of the things you described there are the challenges, right?
40:22
Chris Romeo: Like, it's easy to start a champions program and run it for three months.
40:25
Chris Romeo: Anybody can do that.
40:26
Chris Romeo: The question is, what do you do in year three, when you're like, who is gonna talk to these people this month?
40:32
Chris Romeo: Like, you have to build, you have to build a strong, uh, team that can drive this thing, because you really want that champions program to exist years into the future, not just be, so many champions programs are a flash in the pan.
40:45
Chris Romeo: It's like, uh, we tried it, but it just kind of fizzled out.
40:48
Chris Romeo: Like how many times have we heard that it's because they haven't done the things you're talking about here as far as having a way to get speakers, having some type of tool that helps them, you know, get the right behavior from each other.
41:00
Chris Romeo: So, um, very cool.
41:01
Chris Romeo: Definitely be following along.
41:02
Chris Romeo: I'm a big fan.
41:03
Chris Romeo: Um, but with that, we have to transition to the now, is it famous or infamous?
41:09
Chris Romeo: I can never remember the lightning round led by Mr.
41:13
Chris Romeo: Robert Hurlbut.
41:15
Robert Hurlbut: So in the lightning round, we asked three questions.
41:17
Robert Hurlbut: Uh, first is a controversial one.
41:20
Robert Hurlbut: What's your most controversial opinion on application security and why do you hold that view?
41:26
Dustin Lehr: Yeah, good one.
41:27
Dustin Lehr: I would say this, that I grew up in my career as a tech person, uh, so very focused on tech type solutions, as I mentioned before, writing code and so forth.
41:41
Dustin Lehr: But I think people are the way.
41:43
Dustin Lehr: I think that peop that Learning how to influence, learning how to, um, focus on what people need, and so forth, um, that's going to, uh, make or break your AppSec program, period.
41:58
Dustin Lehr: You could have the coolest tech, but if you can't reach people, you're in trouble.
42:05
Robert Hurlbut: All right.
42:05
Robert Hurlbut: Second question.
42:06
Robert Hurlbut: What would it say if you could display a single message on a billboard at the RSA or Black Hat conference?
42:12
Dustin Lehr: Yeah, so here's where the, uh, cynicism that we were talking about earlier comes in.
42:18
Dustin Lehr: I think I'd probably say something in line with what I was just talking about.
42:22
Dustin Lehr: Um, 80 percent of breaches are caused by human error.
42:26
Dustin Lehr: So let's just go buy some tech and basically ignore that problem, right?
42:31
Dustin Lehr: Which I've seen this a lot, like, just to expand on this, I've seen people say, That sucks.
42:36
Dustin Lehr: And then they just go buy scan tools and they just ignore the problem.
42:39
Dustin Lehr: So I'd probably hold that up at, at RSA or Black Hat.
42:44
Robert Hurlbut: Perfect.
42:46
Robert Hurlbut: And then the final question is, what's your top book recommendation and why do you find it valuable?
42:52
Dustin Lehr: So no surprise here.
42:53
Dustin Lehr: I'm going to go with the people side as well.
42:56
Dustin Lehr: And specifically around negotiation.
42:59
Dustin Lehr: I think that this book, which I just read recently, uh, has had a major impact on the way that I approach, uh, conversations and negotiations.
43:10
Dustin Lehr: Never split the difference.
43:13
Dustin Lehr: Have you heard this
43:14
Chris Romeo: I've read it.
43:15
Chris Romeo: It's good.
43:15
Dustin Lehr: you've read it already?
43:17
Dustin Lehr: Yeah, it's very good.
43:18
Dustin Lehr: It's very good that, you know, things come out of sort of left field where, as an example, the, you know, the faster you can get someone to say no is actually in your benefit.
43:30
Dustin Lehr: What, so there's sort of these mind blowing type, types of approaches that come from the book and I just really enjoyed it.
43:37
Chris Romeo: Good stuff.
43:39
Chris Romeo: Might have to add that to my list to reread.
43:41
Chris Romeo: It's been a few years for me since I read that one, but definitely a great, uh, great choice.
43:47
Chris Romeo: So, Dustin, we come to the end of our conversation.
43:49
Chris Romeo: What's a key takeaway, a call to action, something you want to send our audience away with?
43:56
Dustin Lehr: Yeah, just in line with what we were just talking about, I would say don't, don't give up on the side.
44:01
Dustin Lehr: I think that you can approach the the people challenge, uh, or opportunity with science in the same way that you might approach tech solutions.
44:13
Dustin Lehr: There are fields that are dedicated to this psychology, behavioral science, and so forth.
44:18
Dustin Lehr: There are answers out there for how to influence people, how to reach people, how to encourage people, how to motivate people, and so forth.
44:25
Dustin Lehr: And I would say, spend the time, look into that.
44:28
Dustin Lehr: It's going to bring a lot of benefit.
44:31
Chris Romeo: Excellent.
44:32
Chris Romeo: So, uh, Dustin, thanks for joining the application security podcast for another episode here.
44:38
Chris Romeo: Um, you're brilliant.
44:40
Chris Romeo: I'm a big fan and I can encourage folks to follow you on LinkedIn because you're always good for a question that gets the brain churning.
44:48
Chris Romeo: Sometimes you put them out too early on a Monday morning, but that's okay.
44:52
Chris Romeo: Um, I'm often, I often reply to those and jump into the mix and those, cause they're always very helpful.
44:57
Chris Romeo: Very good questions that give us a number of different ways to debate on a particular issue.
45:02
Chris Romeo: So I'm a big fan of those, so people should look you up on LinkedIn.
45:06
Chris Romeo: Folks, thanks for listening to the Application Security Podcast.