Latest / Full Tech Ahead / Delivering High Quality Software with AI

Transcript

Delivering High Quality Software with AI · Full Tech Ahead

• 0:19
  SPEAKER_01: Hello and welcome to Full Tech Ahead.
• 0:22
  SPEAKER_01: I am your host, Amanda Rosani, and with me today, I'm so excited to have Max Real.
• 0:27
  SPEAKER_01: He is the Vice President of Delivery for Rise 8.
• 0:30
  SPEAKER_01: How are you doing today?
• 0:31
  SPEAKER_00: Hey, Amanda.
• 0:32
  SPEAKER_00: Great to see you.
• 0:33
  SPEAKER_00: Thanks for having me on today.
• 0:35
  SPEAKER_01: Happy to have you on the show.
• 0:37
  SPEAKER_01: Well, um, can you share a little bit about Rise 8 and what services you provide?
• 0:42
  SPEAKER_00: Yeah, sure.
• 0:43
  SPEAKER_00: So at Rise 8, we're uh defense tech and GovTech focused.
• 0:47
  SPEAKER_00: So we have a lot of work with the defense sector, and um we're expanding a bunch of work that we have with the rest of the public sector as well.
• 0:55
  SPEAKER_00: We've always been steeped in the VA and now looking at other areas.
• 0:59
  SPEAKER_00: What we do is we build mission unique software for any mission, whatever your mission is, but specifically in high compliance industries where the security of that software uh is at stake.
• 1:11
  SPEAKER_00: Um, and so we we've positioned ourselves as compliance experts in the public sector, and uh we like to be able to create mission outcomes for all the users that we're working with and uh for all of our clients.
• 1:23
  SPEAKER_01: Awesome.
• 1:24
  SPEAKER_01: Thanks for sharing.
• 1:25
  SPEAKER_01: Well, our topic for today is outcome-driven software delivery leadership in high compliance environments and what accountability actually looks like when large tech programs succeed or don't.
• 1:38
  SPEAKER_01: So, with that, my first question is from your perspective, what does good software delivery actually look like today?
• 1:46
  SPEAKER_00: Yeah, that's an interesting question.
• 1:48
  SPEAKER_00: I've noticed several of your other guests that you've had on the show, on the podcast, that it's all been uh AI focused.
• 1:55
  SPEAKER_00: And we have had an AI invite within our company for over a year now.
• 2:00
  SPEAKER_00: Um, we really push ourselves to make sure that we're being as efficient as we can be at the engineering level.
• 2:06
  SPEAKER_00: And we actually push everybody in the company to become technical.
• 2:09
  SPEAKER_00: We've held um events in the past like Impact Labs, where we're upskilling everybody in our company to become AI native and make sure that they're using it to be as efficient as possible within their work sections.
• 2:19
  SPEAKER_00: That being said, uh, we've had our own kind of Mission OS Live where we've interviewed different people across the tech industry that understand that while AI is allowing us to deliver much higher quantity, um, there has to be a continued focus on quality.
• 2:35
  SPEAKER_00: To answer the question more directly, what we think really good software delivery looks like in this kind of age where AI is adopted into all the software baselines is that we're still driving that software towards mission outcomes, meaning we're working backwards from a mission impact that we're trying to affect for the users we're delivering the software for.
• 2:54
  SPEAKER_00: So whether it was all hands-on keyboard developing the code or whether it was assisted with uh agenc development, the outcome still needs to be the outcome.
• 3:02
  SPEAKER_00: Did you achieve the impact that you meant to achieve?
• 3:05
  SPEAKER_00: And did you do so with respect to the full compliance and security that's necessary to protect the mission at the end?
• 3:11
  SPEAKER_01: So, my next question is how do business leaders track that process and make sure that it doesn't derail and they do get that return on investment that they're looking for?
• 3:24
  SPEAKER_00: Yeah, that's a really great question.
• 3:27
  SPEAKER_00: We have kind of a full uh cookbook around it, to be quite honest.
• 3:31
  SPEAKER_00: Um, we devote ourselves to it, uh, not necessarily in a dogmatic way, but we're very principled in the way that we think that we should work backwards from impacts, um, kind of the Kellogg model where then you you you determine your mission outcomes, the outputs that are necessary to get to those outcomes, and then the resources and activities that are necessary for you to achieve uh those outcomes.
• 3:53
  SPEAKER_00: I'm sorry, the outputs that get you to the outcomes and the impacts.
• 3:56
  SPEAKER_00: So I know that that sounds fairly parochial, but in its most basic sense, there's a way at the team level that every single iteration or sprint that you're working through at the engineering level, you can track the work that you're planning for today, tomorrow, this week towards an outcomes-oriented roadmap that you've agreed upon with your users because it's foundationally built on the mission metrics that they need to be improving on.
• 4:19
  SPEAKER_00: Um, and so when you have commitment like that from the users of the software, that they know you're working on the right mission metrics, and then your team can check in to make sure that everything along their roadmap tracks to the outcomes that are going to improve upon those specific metrics, you've now subordinated yourself to the mission improvement that you will really want to stay true to at the team level.
• 4:40
  SPEAKER_00: And that's what we focus on on a daily and iteration basis.
• 4:45
  SPEAKER_01: Okay.
• 4:46
  SPEAKER_01: So, from your experience, where do things typically break down in large tech initiatives, especially in more complex or regulated environments?
• 4:54
  SPEAKER_00: Yeah, there's a couple of different failure modes that we've seen over and over again.
• 4:58
  SPEAKER_00: Um, I myself spent 20 years in the government, so I was operating uh large-scale systems within a pretty thick bureaucracy.
• 5:06
  SPEAKER_00: Um, and these failure modes pronounce themselves on several occasions.
• 5:11
  SPEAKER_00: One is kind of this boil the ocean big bang release theory, where you think you can create a modernization effort that's gonna replace an entire what I'll call legacy system, but it is the operating system of record, right?
• 5:23
  SPEAKER_00: This is like in the in ops today, going to you know, the fight tonight type of thing.
• 5:28
  SPEAKER_00: And if you're building off on the side and then planning to just like do a hard cutover of a large-scale legacy system, regardless of how many users are on it, into a fully modernized system, but you haven't been integrated all the along the way and making sure that you're taking the users along a journey that's going to allow them to understand the usability of the new system, that tends to fail or have massive amounts of friction right at the release point of the new system.
• 5:54
  SPEAKER_00: And sometimes uh that's even if you get to release.
• 5:58
  SPEAKER_00: Because often when we're doing these big bang type of development efforts, you wind up going running over on your costs and running over on your schedule to the point where it becomes untenable to the key stakeholders in Congress that are, you know, representing our taxpayers.
• 6:13
  SPEAKER_00: And um, for our government systems, when you start running that far over, I think it is only the sensible thing to do is to take a hard look at that, cancel those programs, and then figure out where you can bite off smaller chunks to improve the capabilities that you were intending without these big bang releases.
• 6:30
  SPEAKER_01: Absolutely.
• 6:31
  SPEAKER_01: Well, let's go back to the fact you said, of course, AI is the key topic amongst many business leaders right now.
• 6:40
  SPEAKER_01: And so, how are you seeing AI impact engineers and developers and that reliability that you mentioned?
• 6:49
  SPEAKER_01: There is still a bit of a trust issue with the reliability.
• 6:54
  SPEAKER_00: Yeah, this is a fantastic topic.
• 6:57
  SPEAKER_00: Um, one that we're researching heavily, and uh I'm working with a couple other leaders from other companies too to dive into this on a journal article right now.
• 7:05
  SPEAKER_00: In fact, what does productivity really mean?
• 7:08
  SPEAKER_00: Um, and how are you going to maintain the reliability when you're using agents across your the entirety of your code base or at least to some uh heavy extent?
• 7:18
  SPEAKER_00: And kind of what we found is that everybody can become builders with agentic assistance in your development effort, but not everybody's really great builders.
• 7:26
  SPEAKER_00: And it takes the seasoned software engineers to understand how to interact with the AI agents in the most um healthy way, I'll say.
• 7:34
  SPEAKER_00: And what I mean by that is interacting with the agents where you are uh experienced enough in software engineering that you can understand when an agent has kind of taken you into a place that seems like particularly hallucination is top of mind, but it really could be anything else.
• 7:50
  SPEAKER_00: We've seen agents start to falsify data to pass test cases.
• 7:55
  SPEAKER_00: And it takes a really experienced engineer to start to see those things that are happening within the code base.
• 8:00
  SPEAKER_00: And so we have a way of approaching that that we are kind of hypothesizing right now and experimenting with.
• 8:05
  SPEAKER_00: But within RISE 8, we still believe really firmly in extreme programming.
• 8:09
  SPEAKER_00: And as a core tenant of that, we believe in pairing.
• 8:12
  SPEAKER_00: So pairing more experienced engineers with um, you know, up-and-coming engineers or less experienced engineers uh gives us the ability for them to still pair, pair with the agents and still have the seasoned set of eyes that are helping the younger engineers understand how do I test the agent, how do I continuously prompt it to make sure that what it's doing is staying within the guardrails that we've established.
• 8:38
  SPEAKER_00: Um, and so I think there's a really strong use case here for kind of metering tokens or however else you want to incentivize it, where you keep pairing a really core tenant to the development you're doing, even when you do have agents that are ripping away at it, uh, whether it's during code review or during pure development.
• 8:57
  SPEAKER_00: I think you should always have experienced engineers pairing with maybe the less experienced engineers that are now getting to develop in much higher quantities.
• 9:04
  SPEAKER_01: Absolutely.
• 9:05
  SPEAKER_01: That process makes a lot of sense.
• 9:07
  SPEAKER_01: Do you think that there's still an issue with silos and communication?
• 9:13
  SPEAKER_00: Yeah, I don't know that we're ever going to necessarily break that down.
• 9:16
  SPEAKER_00: In fact, it might be getting deeper now that people can kind of go heads down, develop their own tooling, and then kind of pop up when something's fully developed.
• 9:24
  SPEAKER_00: And specifically, uh, we try and combat that uh within the defense sector and most government uh clients that we have by making sure that we have consistent review cadence that gets us to bi-weekly demos of working software with the key stakeholders and with uh, you know, important users uh that are made to be early and first adopters.
• 9:46
  SPEAKER_00: That forces you to show working software, not a hypothesized roadmap or not, you know, kind of a PowerPoint presentation, but working software of what are the features that we've released, what's the capabilities that exist within the software now.
• 9:59
  SPEAKER_00: And it forces the stakeholders to stay in tune to where you are on the roadmap.
• 10:03
  SPEAKER_00: Without that, I do see the the silo effect could become deeper and deeper as people kind of work on their feature set over here, they're doing security hygiene over here, they're working on tech debt over there, because it only takes one or two engineers to be working on any one of those mission threads.
• 10:18
  SPEAKER_00: But when you're having to demo your working software every other week to your primary stakeholders, you do have to pull all that together to make sure that your demo itself is coherent and make sure that it's going to stand up to peer review and stakeholder review uh on a bi-weekly cadence.
• 10:34
  SPEAKER_01: Absolutely.
• 10:35
  SPEAKER_01: Well, AI is rapidly advancing.
• 10:38
  SPEAKER_01: What do you see as the next future issue business leaders need to be ready for?
• 10:46
  SPEAKER_00: Yeah, I think um we're probably in a place where we're kind of empowering people to develop and get things out into production before they're fully enabled.
• 10:56
  SPEAKER_00: And I think that's always been the case.
• 10:58
  SPEAKER_00: That's something that we've struggled with, especially within um GovTech, where you know you have government employees that don't get as much training and experience in the tech industry as people that are purely in commercial industry.
• 11:12
  SPEAKER_00: Yet they're expected to live up to the same standards of like um Dora metrics and achieving elite Dora for deployment frequencies and things.
• 11:20
  SPEAKER_00: So those pressures force people to develop more and more.
• 11:23
  SPEAKER_00: Now, with the tools at hand, um they're gonna continue to develop more and more.
• 11:29
  SPEAKER_00: Um, and so I think though we need to get to a place where we can understand and give people the latitude that if they want to raise their hand and say, hey, I need more enablement on how to use these tools safely and how to make sure that our code base stays reliable as I'm incorporating agents into what are seemingly mundane functions.
• 11:47
  SPEAKER_00: Every business leader out there should recognize that's not a fault, that's a strength of that engineer who's kind of raising their hand and asking for more upskilling in these areas, and then go out there and look for expert practitioners to come in and help enable your workforce to be really strong at reliably using the tools that are at our disposal.
• 12:06
  SPEAKER_00: So once they start, you know, deploying with much higher frequency and much more quantity, you can make sure that they're doing so reliably as well.
• 12:15
  SPEAKER_01: Absolutely.
• 12:16
  SPEAKER_01: Well, if there was one key takeaway you could leave our audience with today, what would that be?
• 12:21
  SPEAKER_00: Yeah, I would say um kind of back to our early talking points about Rise 8, stay focused on the mission, especially here within GovTech.
• 12:29
  SPEAKER_00: We get wrapped around the axle of budget cycles and what comes out for each government organization that we're working with.
• 12:34
  SPEAKER_00: And then uh it almost forces us into this like inappropriate subordination to the budget line and to make sure that we're um getting things done so that we're invoicing on contracts fast enough to make those program managers successful at spending all their money.
• 12:49
  SPEAKER_00: Let's throw that out the window for a second.
• 12:51
  SPEAKER_00: Like take a big deep breath, pause, make sure you're consistently working towards the mission outcomes that you've agreed upon with your customer.
• 12:59
  SPEAKER_00: This is why we came into business with you.
• 13:01
  SPEAKER_00: We wanted to improve these specific mission metrics.
• 13:05
  SPEAKER_00: Let's keep checking in on those and have like the business review be kind of a secondary effect of what goes on.
• 13:12
  SPEAKER_00: I would say if there's one key big takeaway, it's please just stay focused on the mission you're trying to improve and uh let the business operations follow.
• 13:21
  SPEAKER_01: Exactly.
• 13:22
  SPEAKER_01: Well, thank you so much for coming on the show and sharing your insights with us.
• 13:26
  SPEAKER_00: Yeah, thanks, Amanda.
• 13:27
  SPEAKER_00: I really appreciate you having me on.
• 13:29
  SPEAKER_01: And thank you to our audience.
• 13:31
  SPEAKER_01: If you have any questions or comments about this, leave them in the comments below.
• 13:35
  SPEAKER_01: I'll try to reach out as soon as possible.
• 13:37
  SPEAKER_01: And until the next podcast, have a wonderful week.