Latest / AI-First Engineering Podcast / Is AI Rewriting the Rules of Agile Testing?
Transcript
- speaker-0: Welcome to AI First Engineering, the podcast where engineering and technology leaders share how AI is transforming software delivery, quality, and the future of engineering. Let's get into it. speaker-1: Hello and welcome back to the AI First Engineering Podcast. I'm John Robinson from TestGrid. Today we're gonna be talking about a discipline that's been through some significant transformations over the last few years. and guess what? AI's changing it again. is AI rewriting the rules of agile testing? agile testing kind of has changed how teams sought about quality. You know, we we've we've been doing this for almost twenty years now where that it's been a real thing. It brought testing closer to development. It emphasized faster feedback and you know it it kind of popularized the idea of shared responsibility. But with it changing how software gets built and and tested and delivered, the pace we're seeing right now we've not seen before. What happens to all those good principles and practices that we've thought about when AI becomes part of this conversation? So given the topic that we're talking about, I figured I'd go straight to what I consider one of the sources of this. my guest today is one of the more influential figures in testing over the past twenty years. She's co-authored four books on the topic and was one of the co-founders of the Agile Testing Fellowship. Lisa Crispin, it is great to have you with us. speaker-0: Well, it's an honor to be on your podcast. Thank you for inviting me. And of course these are great topics to talk about. So so for people who don't know, I'm I'm a tester by trade. the last few years I've been a more of a consultant and doing a lot of sort of volunteer work in the software community, trying to give back off after all those years of taking advantage of open source tools and things like that and all the learning. and I'm Based in northwest Vermont and where I live on a farm with with three donkeys, four cats and a husband and a lot of wildlife. speaker-1: I I get the animal side of it. We we have a quite healthy farm ourselves. So yes, totally totally get and understand that. So you've watched this discipline kind of move through a lot of different changes, I guess. You've automation, continuous delivery, DevOps, and now AI is kind of coming into the the the scene a little bit. You quite literally wrote the book on this stuff a long, long time ago. And That gives you a perspective that a lot of people I don't think have in this conversation. So let's start with that. What do you feel is fundamentally different about like how AI is compared to some of these other shifts that you've you've seen and lived through? speaker-0: You know, I think the big difference is just the the pace of it. Yeah, literally every day. You can learn something new. There's some new thing that AI is doing for us or could do for us. so I think just how fast it's going on, like the cognitive overload of that, the comprehension debt of that, it's very hard to keep up. And so it's hard, you know, when test automation became popular Well, all the corporate leaders thought that was the magic beans. Let's fire the testers. We don't need them anymore. And you know, even things like extreme programming, and I mean I work for a company that was one of the pioneers of extreme programming and quite influential as a consulting company. And they, I mean, they would actually tell people they didn't need testers anymore because they're doing test driven development and all these other good practices. So it's of course it's building quality, and which is true. If you do all those things, you're you're gonna get rid of eighty percent of your bugs. That doesn't mean you're not gonna still have serious problems in production, you know. So eventually people had to hire back their their tension. speaker-1: Is the number of up and down cycles that I have seen in that regard the last twelve speaker-0: And I think this time it might take longer for people who are just all of a sudden, AI's the thing, let's adopt it and it's gonna it's gonna do miracles for us. It's the silver bullet we've been waiting for all these years. I think it'll take longer for those companies to see enough other companies losing money or losing money themselves before they realize, we laid off all those testers and other people. and and now we're gonna have to regroup. The people I know who've been working on teams that are AI first, you know, some people have been doing this six, eight years. It's not it's not all new. they are quick to tell ya, you can't get rid of the people. You know, it's wonderful. It's really helping us scale, but we still have to be the humans driving the process and making the decisions. speaker-1: Yeah, I mean I I think that's the one thing that a lot of people are you already start to see that whiplash, right? Of people realizing, no, we got rid of too many people. but all right, so so one of the things that, you know, in in kind of the prepping for the show, one of the things you mentioned, and I I kinda wanna get a little bit more insight into this and in what what your thought process is, but you and Janet kind of started moving away from the term agile. Tell me more about that. speaker-0: Yeah, the f a few years ago, you know, we were really inspired actually back in twenty sixteen, we were really inspired by Dan Ashby's post that he called continuous testing and his model. He took the continuous DevOps loop and all the different stages around that. And he wrote, We test here, we test here, and you know, we we do testing activities throughout the whole software development loop. And that was such a good way to think about it. And you know, Jan and I always had our we love visual models and we always had that agile the agile testing quadrants and the test automation pyramid from Mike Cohn. And, you know, this generate such great conversations and really help the team build a good test strategy for all the different things that they have to build. And so we can we, I say we mostly Janet, came up with this holistic testing model, you know, greatly inspired by Dan. And we have these different stages throughout the software development lifecycle. It's an infinite loop as we know. And Not only does testing occur through the whole life cycle, but we know, and you know, Janet and I know from our own experience, but that's backed up by research from a lot of different really good scientific organizations that the whole team has to own it. Testing's not a separate phase or separate activity. It's an integral part of software development, which we're seeing more with AI. It's it's developers are realizing that they've got to think about that from the get go. And they're kind of forced to do it. And so we use the term holistic to refer to that because the term agile, you know, that term has been corrupted. So it means so many different things to different people that we just though though it was our whole brand, we've just moved away from it. speaker-1: Well but I I I think you're right though. And I think this is one of the challenges about software development in general, but really even within the testing space specifically, we do that to terms a lot. there there's just a lot of lack of standardization across the board. And so that tends us to pick up these these terms that we think we understand what they mean and we start using them for purposes that they weren't really designed for. I would argue DevOps is another one of those. Selenium is another one of those. th it's just we we have this tendency to do that. So that is not at all surprising. speaker-0: Well, we're humans, right? And we need labels. And I mean, I remember having a conversation with Kent Beck at a testing conference back in I don't know, two thousand two or three? Maybe two thousand one. It was early on. And and I said, Why'd you call this extreme programming? 'Cause it's just such an off putting, scary term for it. And he said, you know, I just hope in ten years people will won't have need a label. They just call it good software development. That didn't really happen. But that's what I I often use. Yeah. I o I often use the term modern software development or successful modern software development because it's whatever we're doing. And you know, DevOps is another interesting example. I'm quite active in the in Google's Dora community. And Dora's, which was their annual survey and all their research, stood for DevOps research and assessment. And last year they abandoned, they still kept the name Dora because people know it. But it doesn't stand for anything anymore because they realize no, it's just software development. Hello. speaker-1: Well it's it's so funny because you have these trends that every few years come along and are gonna fix the the problems or or it's the magic bullet and you know AI feels like the latest iteration of that. speaker-0: For sure. speaker-1: so let me let me ask you a little bit of a follow up then. So if most organizations that have historically called themselves agile, but we're not really doing agile. they weren't really following good best practices. They were just kind of muddying through it, if you will. What exactly is AI being layered on top of right now? speaker-0: This is an interesting question because research by Dora and Beyond Quality and some other I can't remember the other name, but some other organizations is is showing that what AI does is it amplifies. So if you're an organization using good software development practices, you're building quality and the whole team is owning quality, you're really paying attention to it. The AI tools are gonna help you work better because you already do good practices. You already work in small batches. You already drive your development with testing. You already collaborate across functions. You have diverse teams. You have close relationships with your business stakeholders. You have you understand your customer needs and get feedback from your from your users. So all those good things mean that with AI, it can do a lot of work for you and save you time so that you get to do more thought work. And and so you you scale, you do better. The teams that are floundering around and they're not they're still working in great big batches, they have a lot of risk, something goes wrong, they don't know which of the million changes they just put in production caused it. And now they've got AI, their their agent can spew out huge amounts of code. So let's give it bigger spec files and do the spec divid driven development and we'll have all this code. Well, now they've got code, nobody can validate. I you know, I know the agents can you can get agents to validate the agents and things like that. But basically you end up with code that's not high quality, that's hard to maintain. Nobody can understand it. They're they're just the cognitive overload is is immense. And it's gonna slow them down even more. The AI is is gonna do bad things for them, it's gonna amplify the bad. And so we see that over and over. And it's really interesting to me that We keep seeing this message and keep everybody is seeing is noticing it now. speaker-1: Yeah. Well it's it's funny because you you mentioned you when we were talking before that you listened to a recent episode of the show and that it sounded very waterfall ish. wh what what do you mean by that? 'cause I don't disagree with you. I'm just curious like what speaker-0: I I don't I hate giving waterfall a bad name. I actually worked for an awesome waterfall team back in the early nineties. And guess what? While we did have long phases, and you know, our customers didn't it was a database company. The customers didn't want something new more than every six months. But yeah, we had a re analysis phase, requirements phase, development phase, test phase. But guess what? Everybody participated in those phases, or at least representatives from each part of the development organization. speaker-1: You got a really bad rap. speaker-0: And we we we did continuous integration. We did a whole lot of collaboration between developers and between developers, testers, tech writers, customer support. We pulled all these people in to work together. you know, we had con we our whole continuous delivery pipe. Well we didn't have continuous delivery, but our pipelines were all automated. Our deployment pipelines were all automated. You know, we did a lot of the practices that that people got to know better when continuous slavery and and agile came along. It's the good sovereign development practices most have been around since the seventies. It's just that we tend to not learn from history. So speaker-1: What's really funny is i I would often, you know, when you put the agile stages that I would say the majority of people use as kind of their definition of what agile development meant. And if you looked at it and then you zoomed out, it kinda looked like waterfall. It wasn't really all that different. It was just speaker-0: Shorter. Yeah, they do the the mini waterfall. And you know, but the tr the trick is, and again, research has shown this over and over. The key is working in small batches, little tiny changes. And now we have these great release strategies of you can start putting things in production, hide them behind a feature flag or do graduated rollout or canary releases, whatever you want, whatever's appropriate, and you can just put one little change at a time out there with your continuous delivery or continuous deployment. And then you've r greatly lowered the risk. Everybody's focused on finishing one thing at a time. It's done. You know it works or it doesn't work. If it doesn't work, you bring it back and fix it. Yeah. And you know the problem is that p the delivery teams get the stakeholders saying, we but we need this and we need this and we need it now. We need it now and they don't know how to push back and say we can't work that way. you know speaker-1: Prioritization is yeah, prioritization is is one of speaker-0: Let's cut back. Let's do less because this because if the delivery team doesn't understand the business domain well, like if they do understand the business domain, they can s they can tell the business organization it's like, well, this little piece you wanna do is gonna take us as much time as this other eighty percent. Do you really need this twenty percent? speaker-1: Yeah, well, and sometimes that's that conversation is a lot of people are oftentimes afraid to have that conversation. Right. The the the okay, I can do this for you. I'm willing to do this for you. It's not a flat no. However, it requires you to take an equivalent amount of work out of the scope today. And that's the conversation I think a lot of people are just sometimes too scared to have because maybe those other items also went through that exact same conversation. And it is oftentimes I think on leadership sometimes to know that going into it that hey, I just asked for you to prioritize this thing yesterday. And now I'm telling you this thing's more important. I should come in there with an understanding of which of those two things is more important. Right? Is the thing yesterday priority or is the thing today priority? That's the thing that gets lost a lot of time speaker-0: Mm-hmm. So Yeah. Definitely. And and the leadership is so such an important component of it. Yeah. speaker-1: Yeah. So This this agile testing idea, holistic testing idea, it's built around collaboration. It's built around fast feedback. It's built around quality as a whole team responsibility. Here's a tricky question for you. Maybe it's not true. Does AI strengthen that or does it push teams back into silos? speaker-0: Well, that doesn't worry. And that's the other thing with, you know, with the teams that are waterfolly, even though they say they're agile. They've still got the testers in one corner and the analysts in another and the designers over here. and you know, these AI tools should enable more collaboration, especially around quality activities and testing activities. Because okay, nobody wants to automate UI tests, right? It's just it's just the most dredged work, soul crushing work you can do. Other than manual testing and or manual regression testing. And but if the agents and skills and all those are doing the grant work for you, now we can bring people in and now we get to do the fun stuff. Like, hmm, what are the risks? Well, how would we manage that risk? Or what do our customers really want the most? And and once we decide what we want, and you know the AI agents can, you know, Claude or whatever can even guide you through and ask you questions to help you understand what you want. And once you have that shared understanding, then something else can generate the code for you. and so I think that that's a better way to get collaborating because people don't have to go, I don't want to look at that UI test tool. But but I see what's happening though is people are working with their agent. They often give or with their LLM, they often give it a name. you know, hello Max or you know, whatever it is. And They feel like they're collaborating with somebody. And I mean, it's like a rubber duck. It's still gonna help you think better and it's gonna brainstorm ideas you may not have. That's great. But I feel like that's drawing people away from other humans. And I think that's a real slippery slope there. We've gotta we know that having a people, humans, with diverse backgrounds and diverse experience and diverse skill sets, that's what leads to innovation. That's what you know, we can do s and we can do these small experiments so fast now with AI tools. That's the really cool thing. It's like I wonder if this would work and and you do it in ten minutes instead of speaker-1: And and I think that enables such, you know, rapid prototyping and and the ability to like really test out theories quickly. But the thing that it does seem like gets lost is and you can argue that that the work from home model and and various other things kind of contributed to this, but so many things happened or were decided or were figured out or were discussed in those hallway conversations that weren't planned. You just walked by somebody and said, Hey, about that thing. And then 20 minutes later you guys have solved it. But you would have never had that conversation if you were just sitting at your desk and slacking all day, right? Or working with your AI tool. Right. Doesn't matter. I think we've lost a little bit of that collaboration. and that's the stuff that I miss. speaker-0: And don't we all miss the physical workflow board? I was just reading, I'm I'm reviewing a book. I can't think now. I scared my name out of of myself. But a book that was originally published about Scrum back in twenty ten or something, and he's revisiting it and and adding codas to each chapter to say, well, how here's how we think now. And one one thing it's like, well, we miss those physical boards because You know, back early on when people were trying to create online project tracking tools, people were like, How can we recreate the magic of you're having your daily stand up and you're looking and pointing at the board and physically holding cards generates? Yeah, exact well, yeah, it should be. But I've never found a virtual board that brings the magic of somebody happens to pick up a card and look at the back or whatever. speaker-1: There's no tangible like that the value that you get out of it isn't there. Right? It there there is something about getting the sticky note and moving it to the next lane that that speaker-0: And I've joined a lot of teams that that when they did go remote during the pandemic or whatever, th they didn't start using collaborative virtual whiteboards. speaker-1: Yeah. speaker-0: All this stuff virtually. And you know, you need to have some place where people can write their sticky notes or draw a flow chart or whatever. And I joined a lot of teams that weren't doing that. I'm like, hello. speaker-1: Yeah, I I I have a little whiteboard here sitting next to me for that very reason. Because I I liter I need that physical interaction to think through problems. speaker-0: I still do my mind maps on the paper to start with. Plus I'm doing with somebody else. speaker-1: I can't that's how I think through, right? Like I I can put it on a virtual thing, but it's not the same mental pathways being used as when I'm literally drawing it and thinking through as I'm drawing it. That physical action is actually part of the process. speaker-0: Mm-hmm. Yeah, that's how our brains work, right? speaker-1: It is so go ba let's go back to the Agile manifesto. you said i i you know the whole concept behind it was working software over comprehensive documentation. One of the things that you've long argued, I think, is that that message itself landed rather badly. What do you think that ultimately cost us in that regard? speaker-0: Gosh, I think it costs a lot of people affected their careers because they got burned out because they joined the team where nothing was documented. And if they wanted to know anything, they had to ask somebody. Well, that's fine. Well, maybe the person that knew the answer is gone now. So they had to figure it out. And so they're this speaker-1: Your job. speaker-0: Yeah, exactly. And so then you have to go through this like I gotta figure out what this code does, or I gotta understand our branching strategy. Or I've gotta understand which people in the business are the experts in which area. there are just so many things that you could just write down and make it searchable and make it easy for speaker-1: You don't need to write your own manifesto, right? You you you just need to get the core elements out on the page so that other people understand it, right? speaker-0: Yeah. And it is, it's a pain, you know. It's like, we got w a wiki, we'll put it all on Wiki. Well, how many times have we had to refactor wikis because they just speaker-1: How times do wikis just get abandoned after like to speaker-0: So that's not mess. speaker-1: six months? Yeah. Like well, we and then you just hit reset. They start all over. Either a new wiki page or a new platform where this thing's gonna solve that problem. speaker-0: sorry. speaker-1: No, I was just gonna say I love the idea. It's just not it it never per speaker-0: Yes. But the teams that was on that were high performing were part of that was because we put the effort in to try to figure out how will we solve this documentation problem. Maybe it would be better to use you know, one team I used w we decided it used beers better to use GitHub. and it just hap because in our context everybody was able to use they all knew how to use Markdown. Yeah. Yeah. it made it. You gotta we but that was the result of a lot of conversations and experiments and that's what teams need to invest that time and then you get the good documentation. And today, guess what? We have great AI tools that will help you with your documentation like the the LLM wiki and with Saul Williamson's Markdown Context Protocol Tool. speaker-1: But C P. speaker-0: Just works with along with you in documents as you go in small, concise Files that your AI agents can also understand so that they do a good job. It's just so it's more important than ever now. speaker-1: It it is and it's it's so funny, like just watching the pendulum swing and I I agree with you, like the the idea behind the let's get the software out, let's worry less about the documentation made sense in practice or in theory rather. But in practice there was still it didn't mean get rid of documentation. It meant don't prioritize that over shipping working software. And I think that's the part That a lot of people forgot. So so where do you actually see AI creating meaningful value at this point? Right? speaker-0: Well, I think I think it does have potential to help with to encourage collaboration and get more of the team consciously focused on how are we going to build in quality, how are we going to validate this code that the AI is generating for us, how you know how and the main thing to me, and and here's where I think t people with testing specializations, quality engineers, bring their skills to the table is How do we work with the stakeholders to make sure that they know what they want and and that the minimum value speaker-1: That is always the hard. speaker-0: that they want? And then what risks might be involved with that? And how might we manage those risks? And and also, you know, customers want everything. What's the few most valuable things they want? Because we can't focus on everything. We have to focus on just the most important things. And that's where the humans come in come in. And and you know, there are There are AI tools that can help us with all those activities, for sure. All the way around that holistic testing loop or or DevOps loop, we can use those tools and they'll help us. But we still need to have the humans engage in that and and, you know, driving driving the robots, not the other way around. speaker-1: I look at it a little bit like the like if you're gonna paint a picture, right? It really helps to know what you're trying to paint a picture of before you start. speaker-0: That's a great way to look at this. speaker-1: if if you're trying to create something like that and you just start splattering paint up there, I mean it will be a picture, but is it the picture you want it? Right? Is it actually gonna look like anything? I that's where I I really that's the part that I think we've struggled with the most in in what people have called the agile development process is that that definition has been missing all too often. And one of the challenges that I've seen is the the QA people have been asked to fill that gap in. And that makes it real th not necessarily that they can't do it, but that's not necessarily their job function or something they're trained to do. It is a skill. It is something that is a very hard problem to solve sometimes. speaker-0: You know, we can't all be great at everything. There may be a few geniuses who can be and and we need like I say, we need that diversity on the team. Again, scientifically proven that organizations with diverse teams are the most successful. They and they're the most competitive because they've got people who are able to innovate and experiment and come out with something new that customers want. And there's a lot that goes into making that possible. and the tools are at hand to help us do it. speaker-1: Given given that, the way you framed that, and you know, one of the things Jerry Weinberg, the great Jerry Weinberg has always said is it's always a people problem. Mm-hmm. Does that actually hold up still? Or is that just something we now say and it's just because it's Jerry Weinberg is we we we keep repeating. speaker-0: That's interesting, you know, people talk about things like, what about accountability? If your AI agent generated bad code, who's responsible for that? Well of course the people are still responsible for it, but that's why we wanna that's why we wanna work together. And you know, one of the things that's that has not changed is that the prerequisite to any software team success, it's probably true of any team in any industry, is psychological safety. So we have to feel free to s experiment. We have to feel free to fail. And not get in trouble for it because we learned something. We tried something. Hopefully we tried something small. Yep. And you know, we try these little bets and we are free to ask questions. We're free to point out issues. I actually one job I had the the team I was on had not had a successful build that we could test in like three days. And so every day the stand up, I said, you know, we still don't have a green build. We still can't test anything. And after the third day, the scrum master called me up and said, Lisa, stop asking questions in the stand up. I didn't stay at that job very long. but you have to have that psychological safety, you have to have that trust. Yeah. You have to be able to have discussions about what's the best way to go, what's the best way to implement things, which tool do we really want to use. You have to be free for everybody to express their opinions without feeling somebody's attacking them. Those are tricky. speaker-1: And it it's it's really funny because like I've often said the QA people they're the voice of quality in the room. They're sometimes the only voice that is going to stand up and ask those questions. And if you're quiet, that means nobody's asking that question, right? Is it good enough? Right? Is are are we really okay with this? Because it doesn't seem like it's okay. speaker-0: And and working on your process quality as well and do you know, good practice, like let's have a process retrospective every so often. Let's look at you know, who's doing what and where are the bumps. speaker-1: Yeah, it is shocking. speaker-0: U stream mapping. Yeah. speaker-1: Quality is not just the testing. Too many people, this is another term that we've unfortunately misused. Testing is not the the overarching thing. Quality is, right? Testing is just one of the tools in which you go about delivering quality. There are speaker-0: Customers don't care about the customers don't care about testing. The executives in the company don't care about testing. No. But hopefully they do care about quality. speaker-1: They will tell you if it works. speaker-0: Yeah. speaker-1: Absolutely. So let me ask you, we're we're almost out of time here, but let me ask you one final question. get your take on this. If AI can do eighty percent of the testing work, which I think a lot of executives are trying to sell these days, right? Do we still need dedicated testers on an agile team or a a small software dev team? speaker-0: Think so. I mean, I think the modern the modern tester or the modern quality engineer, and this is something that I've gotten a lot of ideas from Alan Page and Brent Jensen on this from their A B testing podcast, is our job is to help people learn how to do our job, right? It'd be nice if they didn't need us. It'd be nice if they had all the skills, but like I said, you we can't all have the deep skills. But as a tester, and this is what I've called myself a testing consultant for the last, I don't know, 10 years, because the last few teams I worked on full time, I did some hands-on testing for sure. But what I did a lot of was, we're having a lot of, you know, the the development managers realize we're having a lot of problems in production because we have great automated regression tests, but we're not having enough time, the testers don't have enough time to do all the exploratory testing needs to be done. Gosh, I guess the developers should do exploratory testing too. So let's have some workshops. Let's pair with the developers so that they learn some of these skills. And they take some of that responsibility. That's the kind of thing or or I might bring in, you know, we're having a lot of stories where we hand it, we finish it, we're really happy with it, we give it to the product owner, and the product owner said, that wasn't what I wanted at all. well, maybe we should try some new technique. And then I can suggest, well, I've just learned about example mapping from Matt Wynn, where we in our pre-iteration planning meeting, we go through the stories and look at what's the goal of a story. What are the business rules? And for each business rule, give us some examples of how the business rules should work. We can take those examples and turn them right into tests that guide our development so that we have the shared understanding up front, which is which is so important. So you know, that's the kind of things that you know we have data experts bring in, you know, data's really valuable in this AI age. So we've got the data experts to help us How can we keep that data correct? How can we keep that data stored in the right place? How can we make it accessible to whatever needs to access it? That's just each each person or each skill set needs to transfer some of that skills to everybody on the team. Yeah. But we can't all be experts. We just have to do what we need in our context. speaker-1: No, and I I I think that nails it because look, we're never gonna get rid of humans. If if we do get to the point where the the AI systems can do what a human can do from a just thought process standpoint, we're in trouble. speaker-0: Well well at the point that they actually at the point they actually do know things and they're just L L Ms that scrape things off of the speaker-1: That that again, that's a whole different question. But Lisa, this has been awesome. I could do this for a lot longer. but I wanna be respectful of your time. Tell me where people can find you. Where where can people go learn more about Lisa Crispin and the stuff you're doing today? speaker-0: Well, and you mentioned Agile Testing Fellowship and Janet, Gregory and I do a we'll try to make it monthly, a newsletter, a blog post, and we have a video chat that we post on our YouTube channel every month. And then I also have my website, LisaCrisman.com. I haven't been blogging as much lately because I've been overwhelmed with other trying to learn AI. But you know, you can look there. There's lots of resources there that that people can can find. And also Our book website is agile tester.ca. So you can learn about the four books that Janet and I have written together. And there's a lot of, like I said, there's lot of free resources on there that people might find really handy. And it's even this, these practices are even more important. To get the benefits of AI, you need to have these really solid development and testing practices. And so start there, get the basic foundation, and then you're going to be so happy with how you benefit from AI. speaker-1: It will make life much easier, I promise. I will get all of that in the show notes as well. I appreciate you. speaker-0: And LinkedIn. You can always reach out to me on LinkedIn. So speaker-1: absolutely. i if there's anything that we can do for you, please holler. But you know, I just want to thank you for coming on and we will talk again soon. speaker-0: Thank you and I look forward to your future episodes as well. So thank you. speaker-1: Awesome. Thank you guys. We'll see you next week. speaker-0: If enjoyed this episode, follow TestGrid for more conversations with engineering and technology leaders building the future of AI-driven software delivery. Thanks for listening to AI First Engineering.