Latest / AI-First Engineering Podcast / Did Playwright Win the AI Testing War?
Transcript
- speaker-0: Welcome to AI First Engineering, the podcast where engineering and technology leaders share how AI is transforming software delivery, quality, and the future of engineering. Let's get into it. speaker-1: Welcome back to AI First Engineering. This is the show where we're going to talk about how AI is actually changing the way software gets made. ⁓ I'm John Robinson. I'm your host from Test Grid, and we're going to talk today about something that's been happening for a few years now. We've got a shift that's going on in the test automation space. For a long time, Selenium's kind of been the go-to tool for you know all things test automation, whether that's Selenium or Appium. And you know, lately we've Playwright's kind of become one of the most talked about tools in the space. and kind of in conjunction with that, AI is changing how developers are are building software and how those tests are being created and just how testing's happen in itself. So here's what I want to talk to about today with my host or my guest is has Playwright actually won the AI war? Right, from a testing standpoint? And more interestingly, ⁓ along with that, where does Playwright fit so in the AI space. Like why did it why did it win? ⁓ so with me is Andrew Knight. And you guys might know him by ⁓ another name, Automation Panda, Pandy. ⁓ Andy is gonna be talking about Playwright, AI, and where software testing goes next. Andy, great to have you on, man. speaker-0: Thanks so much, John. I'm really, really glad to be here. speaker-1: So give us a you know fifty thousand foot view. Why are you and I talking today? What what's your, you know, credentials in this space? speaker-0: Sure, absolutely. So ⁓ I'm known as the Automation Panda, as you said. I have spent pretty much my entire career in the software testing, quality, and automation space. ⁓ I cut my teeth as a rank and file SDET, did that for about a decade, going from college grunt all the way up to team lead. ⁓ in in the course of that, many different companies, many different industries, ⁓ open source projects along the way, like Boa Constrictor. ⁓ I also after that. took a a left turn into developer relations where I worked with Apple Tools, which is automated visual testing and AI autonomous testing now. Yep. And during that time I was director of test automation university. So I helped not only before helped make my own courses, but then helped other people make courses. ⁓ currently right now I am senior director of product and engineering at Cycle Labs. We're building a test automation platform targeting supply chain and warehouse management systems. And so testing, testing, testing all the way through. speaker-1: All the thanks. Seems like it's a common thing for you. speaker-0: Yeah, yeah. So yep. And so John, you and I had met in person most recently at QA or the highway. speaker-1: Gonna wear my shirt, but ⁓ yeah, it does it doesn't quite ⁓ it's it's not as cool. I like that one better, actually. ⁓ so let me let me start off with a you know, let's just bury the lead here. Did Playwright win the AI testing war, right? Mm-hmm. speaker-0: Yeah, the Ohio. ⁓ yes maybe. speaker-1: Okay, so so follow on to that. Is it did it win the war or is it more that this is the current battle that's being played out? ⁓ this is the current chapter we're in. speaker-0: I would say it appears that Playwright is winning the war. It is certainly winning the battles for mind share. It is also, from my perspective, winning the battles of of ⁓ new projects. speaker-1: Right. So I wanna ask you a question on that. Why do you think that is? speaker-0: 'Cause it's a great tool. And it's AI forward. speaker-1: I so so so there's there's the real answer that I think we were we were we were probably I I I think the real thing that has stood out to me about it in comparison really to any of the other tools that are out there is how much they embraced the d the AI developer ecosystem. They had an ⁓ C P serper really, really quickly and that seemed like for a lot of the the Claude kids that are out there got the bandwagon rolling and the mind share just took off, right? Everybody that started experimenting with Claude internally, the natural extension of that was playwright because, hey, there's a tool for it. Right. Here's we got the server already ready for it. You can talk to it natural language. You can build your scripts, et cetera. So given that, like if if that's the case, you know, we could we could have arguments about why that was good or bad or, you know, the thing that's interesting though is Where does that leave the people that have been historically building Selenium based frameworks? speaker-0: Great question. ⁓ it depends. speaker-1: No, I I I I I would agree with you b by the way. ⁓ I don't I don't think there's an easy answer to this, but I'm curious your opinion. speaker-0: Sure, sure. So when I look out at web testing, the the three big tools are Selenium, Cypress, Playwright. ⁓ I will say I think each tool is is great. I think each tool was developed with care. I think each tool has a a purpose even possibly today, though things are changing and the world is changing. ⁓ so I I want to I wanna I don't wanna like dog on on the others too much of the speaker-1: No, I d I d I don't think there's a the I I to your point, I think there's value regardless. speaker-0: Mm-hmm. Mm-hmm. So in in terms of the the original question of like w w what do Selenium users do right now in in this world? ⁓ from my perspective, what I've seen and what I've heard, and even a bit of what I've felt, is is that like if you have a Selenium-based test project and it's meeting your needs, it runs reliably, you're able to maintain it, there really isn't a a a strong need To say rewrite it or convert it, if it if it's meeting your needs, right? ⁓ but if you are starting a new project for, let's say, a new web app, I think that whether AI or not is involved, which it probably should be, but like re well, I guess what I'm saying is regardless of the AI factor, when you have a a tool like Playwright, which is which which natively blends in well with standard developer workflows. Right. If you're d if you're developing a web app, you're probably using Node.js, React Frontend, or Angular something, right? So you're already full stack node, why wouldn't you have a ⁓ speaker-1: Right on the test framework, keep go right down that path. speaker-0: Right in there, right? Yep. And now with agentic AI workflows, right? ⁓ if if you are the developer in that seat and you are writing out the spec of what you want to do and then you're handing it off to AI to generate code. Well, generate product code, you can also generate the test code. Why not do that altogether? That's why it's like the new newer projects I feel like are are leaning much more towards playwright than Selenium, because if you want to do Selenium, a lot of times you gotta do it in like a different stack or different projects or speaker-1: It's it's a development project in and of itself. Yeah. speaker-0: It it is. You know, the there there's more friction to to setting up things in Selenium, which Playwright has done a really good job of of putting that focus on better developer experience. Right. You know, the fact that Playwright came with its own browser projects. You didn't need to install a little separate executable to to kick it up and go. Right. ⁓ now I I must be fair to say Selenium has tried to address some of those things. Like there's now a tool like Selenium Manager out there. Nevertheless, I feel like the playwright experience is much easier because from one command line entry you just go boom and it just wait for it to download and it's there. Selenium you still need to do a little more piecemeal stuff to get it together. speaker-1: I think there's there's something to be said for what the tools were originally built for and and how they grew up and what they're the core of what they were designed to do from day one. and and fundamentally they are very different tools. They just the way they they work and that's okay. ⁓ I think one of the the questions in in this is it's weird how How because you use a lot of the the terms that I think is really interesting when you you see what's going on with kind of the shift in how quality teams are shrinking or morphing into something different. And you you mentioned a lot of the developer experience and and and I'm curious how one of the things that's interesting to me is I mentioned the MCP survey earlier. I think that even though it is a you know, let's call it a developer centric tool. It does have a good front end for the non developer to be successful. Is what do you see from a let let's take a you know a traditional QA team that that has been responsible for developing this, you know, what is the impact on teams like that where all of a sudden they'll now they have the potential for a tool, but the lens through which it's it's viewed is as a developer tool. speaker-0: Yeah. I think that even with playwright being very developer centric, I feel like there is a very natural bridge from from let's say Selenium thinking to playwright thinking or from more traditional QA approach to using playwright versus a tool like moving from Cypress to Playwright or Selenium to Cypress. ⁓ and the reason why is because even though let's let's just keep comparing Selenium to playwright, right? ⁓ even though the the mechanisms are different, the the ⁓ the impetus behind the projects were different, the the outcomes and guiding philosophies have been different. ⁓ ultimately if you are in the role of a tester, meaning I need to pick a tool to make a browser go I need tool to make browser dance. ⁓ Selenium and Playwright at that fifty thousand foot view level actually look very similar. Right. speaker-1: Up, right? From the outside. speaker-0: Yes, because what does Selenium do? It's it's a browser automation engine at its core that makes you click buttons and do things, right? What's playwright at its core? It's a browser automation engine that clicks buttons and navigates. ⁓ and so conceptually, if you're in the tester mindset, you've had experience doing these kinds of tests before. You're like, can I just like kind of drop in playwright in in my own mental model wherever Selenium was? And the answer is like, yeah. speaker-1: Probably, yeah. For the average user, absolutely. speaker-0: Yeah. Yeah. So it's like, okay, well, you know, I I need to have my test cases and then I would have like page objects and then play rates just a little thing do it da da da da. Yeah. And so people are going like, ⁓ so it's all the same and then then they can mentally map, right? And then they find out, wait a minute, player, I don't have to do my own weighting. I don't have to do that. ⁓ yeah, ⁓ this is better. ⁓ this is faster. ⁓ this is nicer. Right. That's the mental bridge. speaker-1: You I I think you hit the one thing that most traditional automation people probably gravitate towards the quickest is the speed factor. ⁓ that that's because of the way the two tools interact with the browser. Uh-huh. It it there's just a I mean, I remember using Nightwatch JS and just the experience when when that was an attempt to start moving away, right? Many years ago and the the speed differences were insane already. And when it's purpose built for that, you you can tell difference. one of the things that I think is is probably a factor as people start to look at where playwright fits in the ecosystem. w what is it not good at, right? We know it's designed for the web, ⁓ huh. Right, but what does that mean for some of those cross platform implementations or situations where, you know, it crosses boundaries? speaker-0: Sure. That's that's a great question. So you can use playwright for web browser. Great. You can use it for API stuff. ⁓ but it's I would say like that that initial or let me put it this way, every time you start to move away from web, it's like ⁓ the API testing support in playwright is great. I personally would not use playwright exclusively for API testing, though I've had t teams ask me, can I do that? I think I want to do that. I'm like You can, but I think the power really comes from when it's with web UI plus API together. Yes. Right. Which is speaker-1: And what a lot of people should be starting to think about these days anyway, but that's a different topic in a different speaker-0: ⁓ yeah. Yeah. Yeah. Should we be doing traditional API testing? Spicy. Okay. One of the well, one of the most frequent questions I get as the next extension is, ⁓ can Playwright do mobile testing? And I say, Well, it can emulate mobile browsers. Yes. But it it cannot test on your phone and it cannot do native mobile testing. So if you have like a native app on iOS or Android, you can't do it. And people are like, ⁓ speaker-1: ⁓ Yeah, but if you So th this is I I totally agree with you and I think that's the the to me f in a large enterprise type of environment where a lot of people are gonna stumble on the playwright conversation because Selenium and Appium are so compatible with each other, right? Mm-hmm. ⁓ but if you were around for the early days of when Appium came to be, there didn't it it wasn't a straight line either. Right. There were a lot of hacks that you had to put in place to standardize across those platforms. And it wasn't just, ⁓ Appium, here we go. It was no, what particular Android version are you hitting and what iOS devices are you hitting? And like there was ⁓ you had to cobble it together from so many different pieces. And I see I think we're still in the early days of playwright from that standpoint. Where do you think it eventually goes down that road, or do you think there's another Appium two point ⁓ or w something else coming? speaker-0: I'm really not sure. So I I do want to be clear. I I don't have secret knowledge of what goes on in in the playwright world, even though I'm an ambassador, right? ⁓ the just for the listeners, you know, Playwright is an open source project developed by a team at Microsoft. So it is official Microsoft. Yes. ⁓ there's there's a core team that develops, there's a bunch of us in the community known as the speaker-1: of you that didn't know Microsoft is very heavily involved with play right. Yeah. Just just just for the record. speaker-0: Yeah. But ⁓ like there's there's a a small select group of ambassadors. I think there's like no more than a dozen of us. ⁓ we don't get secret knowledge. We don't get to see upcoming roadmap. I'm just out here, I'm literally talking playwright 'cause I love it, you know. ⁓ so everything I would say is pure speculation. ⁓ do I think that that playwright is going to try to do like any sort of native mobile stuff? I would be very doubtful. Playwright, from from everything I can gauge, they're trying to do one thing really, really well, and that's web testing. Right. ⁓ and and to that effect it's like, well, if let's say you were a big enterprise and you had all these different platforms, you know, it's like you have web and you have mobile. You know, ⁓ well playwright can't do the mobile one. Should that stop you from doing web? You're gonna have different projects for each one anyway, so what does it matter? speaker-1: And and this this is again, this goes back to philosophical strategic level decisions that I don't know that we spend enough time on. A lot of people often want to take the shortest route to success versus the most effective. ⁓ and and excuse me, I I do think the the Appium selenium pairing, if you will. speaker-0: ⁓ yeah. speaker-1: made it easy for a lot of people because you had one thing you had to learn. But I think the the MCP side of things and the ease at which you can generate and develop stuff is going to change the the the reality. speaker-0: Yeah, absolutely. Like I can't imagine developing any automated tests today without AI assistance 'cause it just makes it go so much better and faster. speaker-1: I I I think so this has been I've managed automation teams now ⁓ God going back almost twenty years and I've seen a lot of change over the years and the the thing that is that has stuck out the most n I I've done it as both an internal manager, I've done it as an external product coming in. I've done it as a consultant and without fail, I have yet to see two Selenium implementations actually look the same. There they did nobody, there is no good standard out there. And I think that's part of the challenge is everybody this is my other it's let's call it ⁓ it really irritates me. the the fact that Selenium had become There is this Selenium implementation, but there's also the the web driver that is the core part of it. And that oftentimes gets confused and gets called Selenium. Mm-hmm. And Selenium became kind of the catch-all term for test automation for a very long time. Mm-hmm. And really, really, really confused the landscape, I think. And if you go into a lot of companies that's were staffed by teams that only built internal stuff. They never worked anywhere else. They only learned they they had just enough knowledge of Java or, you know, insert language of choice. And they were put on to, hey, build test automation. You understand how to write this code. Go do this. We don't have another tester on their team that can do that. Mm-hmm. I you look at the implementations. I I went into one company and we threw seven years of scripts away. Because they were they were spending 80% of their time just keeping those scripts running. And and that's that's the worry that I have with a lot of people just with Selenium in general, is the maintenance part of it. So talk to me a little bit about what does maintenance look like on the playwright side. speaker-0: Sure, sure. ⁓ so apart from AI, just looking at the tool, the calls, everything itself, ⁓ I have found from personal experience that the maintenance burden is a bit less with playwright. ⁓ and the the the biggest thing it comes down to, I think, is in how playwright handles its interactions, and that every interaction in playwright has automatic weighting baked in. Right. With with Selenium, you had to do explicit weighting. You had to Well, ⁓ nuance, but if you were doing it right speaker-1: I I'm I'm laughing because that is the number one question Selenium teams come in and ask when they evaluate test grid and look at the platform. Does it have weights? speaker-0: Yeah. Yeah. Yeah. 'Cause 'cause like with with Selenium, if you're doing it right, ever before every single time you fire off some sort of interaction to the page, you've got to wait for some actionable condition. Which means in Selenium, like you have to be very picky about what that actionable condition is. speaker-1: Yes. And there are certain actions that will never, no matter what you do, be capturable in the browser. speaker-0: Yep. Yep. And so now, like for one given interaction, it's okay. You do the weight, you code it up as kind of ick, and then you make the the little click call. ⁓ scale this up to the size of your test suite, hundreds to thousands of tests. Like, are you doing duplicative code everywhere? Are you burying these in page objects? Did you do screenplay? Did you do some roll the dice kind of thing that you were just mentioning? Everybody's got a different implementation. speaker-1: Everybody's got their their own secret sauce of how I did I I solved this problem, guys. Cool. Yep. You and him and her and them and speaker-0: Yep. Yeah. I mean this that was a I mentioned I did open source for a little bit before. Like my I have an open source project from a previous company called Bell Constrictor. It was a dot net implementation of the screenplay pattern because I primarily I wanted a universal way for people to just when they say click, the weighting is already baked in and it's it's the right way to do it. Yeah. speaker-1: So so fun fact I did some homework. ⁓ your last ⁓ commit that I can find on Book Restrictor was actually adding playwright support. speaker-0: Amazing. It's been a while. Yeah, it's been a while. Yeah. But moving back to the playwright side of things, like when when you have that click interaction in playwright, it has the waiting automatically are already built in. And playwright, like on their doc page, you can see they have an entire actionability table to say for every single interaction type, here are the actionability traits that need to be true that we automatically check for. And it's not just for interactions with playwright, it's also verifications. And every single one of their assertions. has has the smart checking as well. speaker-1: And that's the thing that Selenium never had from the beginning. And it was not possible to retrofit that in. I think there were attempts made to add some of that in over the years. Four was definitely a big step forward in that that regard. But there were some things just fundamental at an architectural level was never gonna be possible. speaker-0: No. And so yeah, with Selenium, you ended up having to do a lot of DIY. You had to know the recipes, you had to know the design patterns, you had to put it all together yourself. And you could, and we did, right? But with Playwright, it's like one solution, everything out of the box, and it just works. speaker-1: Yeah. Well and I I I definitely think it it makes the you know, we we I've worked at a number of different test automation companies and we always had people come in and go, Hey, we we wrote Selenium, it's it's the the ubiquitous thing that's out there. Can you convert our tests automatically? No, because no two were the same. They're the the calls, everybody used the the way they used the calls were different. the custom, you know, interactions that they did were different. Like everybody's stuff was different. And that made it back to the maintenance thing. Like that made it a maintenance nightmare. Yeah. Right? Because say the person that wrote it left. Mm-hmm. Right? Then the next guy isn't going to know what what's going on. Yep. So speaker-0: So if if we can now add in the AI factor. You have AI, which is getting better and better every day. Which one is it gonna be easier to look at to maintain? Really? Like you've got this this cobbled mess. Well, I shouldn't I should be more polite. With Selenium, you have speaker-1: But it's not perfect. Absolutely. But it is. It is for for the most part custom frameworks. speaker-0: Custom frame. That I'll say you you have highly customized things that are kind of snowflake like, right? The the AI will have to spend the time figuring out what the patterns are. And even in within one project, the patterns might not be consistent. Yep. Or it comes over to playwright where there's still variability for sure, but it's it's it there's going to be less variability. There's going to be speaker-1: Right, depending on who wrote it. speaker-0: to be more predictability. There's going to be less code that needed to be written to do the same kinds of things. And it's going to be more standardized. Right. Cause even like let's say like a big division playwright would be do you use raw calls or page objects? Okay. Well, playwright gives you the standards for what the page objects should be. They're pretty straightforward. Right. So even then it's like there isn't that there isn't as much variability. Right. And so the AI is going to be a lot better at coming in and hooking in and being able to help you maintain that. Plus, it it comes with the skills in the MCP server out of the box. It's like here is the way the the particular one to you. In Selenium, it's like I've seen Selenium MCP servers, but there's a there's no official speaker-1: Different. They're all different. They're all different. And again, everybody's got their own and if you look at where did AI learn from the last twenty, thirty years of all the sources that are out there that are some are good, but a lot are not. speaker-0: Uh-huh. speaker-1: ⁓ you ask the same thing twice to an AI, you know, ask Claude twice to give you a script and it's gonna look different both times. It's just reality. ⁓ we're we're getting short on time, but I got a couple more questions I want to ask. Sure, sure. So one of the questions that that keeps coming up is this whole build versus buy idea, right? You know, w at what point does building it ourselves stop making sense and you know, buying a platform, right? 'Cause Because Playwright gives you the framework, it gives you the tools to do a lot of that. But there are still things that are also important. You got, you know, where it runs, the reporting, the orchestration, the devices, maintenance, governance, a lot of those things. Some of it's coming, but it w where does that where does that line kind of start to blur? speaker-0: Sure, sure. I would say for whatever team is considering that choice, look look to your limits. Look to where the friction becomes unmanageable. Right. ⁓ if you are a set of bona fide ests with AI superpowers, right, and you have access to everything you want in your cloud provider of choice and you know what you're doing, like you probably don't need to buy very much, right? You could probably get something like playwright. Put some scale sets out there like Docker containers, whatever, and just press your images and just bake it all into your dev process and amazing, right? You don't need to buy very much if anything at all. How many teams in the testing world are like that out there? There speaker-1: There are not many. speaker-0: There's not many. And so the typical points of friction or the the typical like limits I see people have would be things like greater test orchestration, right? Being able to to ⁓ run different suites of tests on a schedule, being able to have the the execution resources made available to you very easily, ⁓ and then being able to report out from that. Right. That's something that where it's like I can run it on my little laptop and I can even throw it up say a Jenkins pipeline. But if I really want to have like team or organization-wide orchestration with the dashboard, with all that kind of stuff, that's where it breaks down. ⁓ other points of friction, like it let's say you're not your your test team are more traditional, or say like like with my company with Psycholabs, like these people don't even identify as software people. They're just trying to automate workflows. Like, like they need the low-code to no code kind of stuff. That's okay for where they're at. Right. That's why you know you could have a visual. builder or recorder kind of tool because they're not trying to go from one to infinity. They're trying to go from zero to one. Yeah. Right. speaker-1: They're they they have a very specific thing that they need to do on a regular basis and it's they know how to do that thing, but that's it. speaker-0: Exactly. Exactly. So it's like, you know, I for i I in my more distant past I used to like look down on tools like Selenium IDEs, like, this is crap, you're not gonna go cut tests. But then I realized, like, wait a minute, not everybody has the skills I have, right? But they still have the job that I have. How can how can you enable success? How can you get them to to advance? speaker-1: And that's where I think the most interesting shift that's happening right now in the quality space is going on is is, you know, for better or worse, and in a lot of cases it has unfortunately been worse, you do see a lot of people losing QA test automation roles because of these tools that are coming along, right? But what that's doing is shifting that burden to people like the people you just described that do not have that background, do not have that technical ability, do not have the quality understanding, right? Just just what is QA testing good practices look like? ⁓ that's where I think is really interesting is how do these tools really start to enable them to to be successful? Because they're coming at it from a a space that you and I, it would be completely foreign to us, you know, maybe twenty, twenty plus years ago for me, you know, that That's a different thing. Like I I I was that person at one point, but even then I had a technical background. A lot of these people are coming from the business side of the house or they're product owners or admins, right? Like it it just it really depends. Andy, this has been great, man. Like I could talk and geek out over this for hours. ⁓ what t tell us the thing that you know you really think is a let's call it a hot take for You know, the next few years. Gimme a give me a hot take. speaker-0: Okay, I got one. Talking about QA roles and elimination and stuff. This is one that I've only I've been pondering a while. I feel it's probably gonna be true, but I've been hesitant to say until recently, 'cause now I feel more confident in it. ⁓ I suspect that in the coming years testing will be reduced from a role to a responsibility. Here's what I mean by that. When I look at speaker-1: I a hundred percent agree with you. speaker-0: So we look at the classic roles in in software dev. You have biz dev tests. Those are called the three amigos, right? Business, you think of like business analyst, product owner, product manager, right? They're trying to figure out what to build. They're the ones talking with customers, soliciting feedback, generating a roadmap, having a vision, communicating developers, writing specs, right? That kind of thing. Dev, those are the builders, those are the engineers, those are the ones who are addressing how. And then test, the ones who take both of that and try to make sure they line up. Right. And so I've been in the test role for the majority of years of my career, right? Primarily being an S Det. ⁓ but what something that's that I've that's always bothered me about the tester role is it's always third wheel. It's always the second class citizen. It's always the one left out of conversations, the one that's paid the least, the one that's always cut first in the lab. Right. And and we always bang our fists on the table being like, Give us respect. And so the the answer that at least from from speaker-1: Totem pole. speaker-0: our wing of the tech world has always given to this point has been, yes, you managers need to make sure we have a seat at the table. You need to make us equal in importance. speaker-1: And there are there are some people that are in the QA world that are very, very insistent on that point. ⁓ but I think it hits on one of my longstanding beliefs i i within the space is the other two functions that you named, the biz and the dev side, both have career paths that lead into them. There is a development of that. And the test world never did. It was always people that kind of fell out of those other roles. Right. Because whether it was either they fell out or they were using it as a stepping stone to get into them, testing was never an intentional direction for ninety-five percent of the people that are in testing. That was not their intended career for the majority of it. And so I I I I got into it accidentally. It was not where it just happened to be like I got into it. I'm like, ⁓ wow, this clicks with my brain. Yep. Great. But That's not how it is for a lot of people. And what is really interesting, yes, we have, you know, ISTQB and some of the other certification routes that you can go through, but those other roles go through years of development sometimes for that capability, right? You can go to school for, you know, computer science. You can go to school for business. You can go to school to be, you know, what all these different functions that doesn't exist for testing. At best you get a class, right? Or a mention. in a class. And I think that's really fascinating because I agree with you. The testing function is really the thing that the other people didn't want speaker-0: you do. And so now, where where does this go with AI? I see that testing function kind of splitting between dev and products. If you were more of the automation side of testing, you've proven you've got development skills, right? You can naturally step in if if you take on the mindset that you're no longer a tester but a builder, yes. You can easily migrate into the developer world. Because with with agentec workflows, it's like, You know enough of the code and you know enough of architecture and have good taste and smell, right? If you what you're doing is developing specs and the specs are going to generate product code and test code, ⁓ well wait a minute. You know, developers were doing the product code and leaving the rest of you. You were doing the test code, leaving the rest of them. You could own both. You could keep them. And then for the the side of testers who are more human oriented, exploratory driven, manually focused, right? The not the Not the automation neers, but like the the more like holistic testers. That graduates n naturally up into product management. Because they're the ones advocating for the customer. They're the ones who care speaker-1: I know the product better than anybody. Like most test teams know the product better than anybody else in the company. speaker-0: Exactly. And so they're naturally going to fit into a more product management role. And so testing devolves into a responsibility. It's a responsibility to have automated tests that automatically check to make sure your specs are good. But then it's also it becomes a holistic concern of making sure when the engineers deliver the product that the the product owners and managers say it's good or not. And so I'm like speaker-1: Yeah. Right. I what is really funny about that, because I I completely agree with you. I've been talking about this for almost two years now, that I think over the next few years the role of the tester is going away. Like it just is. I I you know, people don't like it, it is what it is. But the flip side of that is I don't think the role of the QA person or the quality advocates is going away. Because now there might not be as many of them in an organization. But somebody still has to have the understanding of what good looks like. Somebody has to be the voice and the advocate for quality in the room. Somebody still has to make sure we're testing the right things at the right times in the right places with the right data and the right scope, et cetera. Like those are still things that have to happen and you just may not need as many of them. speaker-0: Mm-hmm. Mm. Yep. I could see that. ⁓ yeah, I could become more speaker-1: Orchestrators, I think, at that point. speaker-0: like yeah, the having your your subject matter expertise is of the testing quality perspective where you would have like a champion in the org. I could see that. ⁓ I also think that like How how to say. We're we're going to see a a ⁓ a mm a wide I won't say adoption, but like the way this is going to happen is going to be ⁓ uneven in different areas. I can see large slow moving enterprises continuing to have test speaker-1: Absolutely. It's it's not gonna change for a while. Like healthcare industry, yeah, twenty years from now we'll be still dealing the same thing. Like it's just the reality of that industry. speaker-0: But I think like the the the s the small startups, the fast movers, because with AI you're able to ⁓ eliminate a lot of the human grind work, you know, that means that y peop what people were doing in their traditional roles now becomes more compressed, which enables them to take on more because they they now have the skills and the the ability with the tooling to be able to do more. So it's very natural that developers can pick up tests, or it's very natural that product managers can pick up tests, you know. And so like seeing that that people moving towards more of a builder mindset overall, they're gonna have to be able to do more things than before, but they're going to be able to do more things. Right. And so the harder separation of roles is starting to go speaker-1: And I I I keep saying we're almost done and and I have ⁓ now I have one more question. But one of the things you just said is actually really interesting to me because I do think this is test has always been a dirty word in the dev and product space, right? Like it just it it just has been. Like I don't like it, but it is it's all like you said, it's always been looked down on, it's always been seen as the the dirty work, the ground work, the the somebody else's job to do that job. Mm-hmm. Do you think there's a world then when that shift changes to it becomes more of a validation function ⁓ as opposed to n and maybe not necessarily the explicit thing changing, but maybe the terminology and the way we refer to it changes as we move away from this distinct function. speaker-0: I'm not sure if if the vernacular we use around testing will change much. ⁓ I think maybe. Let me let I can see there being more emphasis on terms like acceptance and evaluation, particularly in the AI age. Because if if we're we're we're talking about this kind of role split and migration of of functionality, if we live in a world where product managers are taking on more of that testing responsibility in the human sense of making sure that this product is acceptable. I think product managers would think more in terms of acceptance and and and meeting user needs versus functional testing. Right. I think that that speaker-1: I do think that that's one of the things that's gonna be interesting to me because I've always seen that as a split within the testing space anyway. There's functional testing. Did the devs build the thing they said they were going to build? And then there's user acceptance testing or regression testing, whichever bucket you want to put it under, which is does it still do what I need it to do or what the user expects it to do? And a lot of times those are conflated and put in the same bucket. And sure, it'd be great when they overlap, but they don't always overlap. And I think we have to recognize and I think that distinction is probably gonna get more clear now. Agreed. Tell us where we can hear more, learn more about the the panda. speaker-0: Sure. So if if y'all just search Automation Panda, you will find me. I have my blog at automationpanda.com. Also, I have a cool little extra fun website now, Autopandamotive.com, or if you want to learn about my cars that I work on. But otherwise, yeah, I'm on X. I'm more on LinkedIn these days because speaker-1: Yeah. X has become kinda useless. speaker-0: It's wild, man. It's crazy what happens on that platform. I'm still there. I lurk. I don't post much, but I lurk. speaker-1: Threads is just as bad. Just but from a diff from a different from a di like in a a different kind of bad though. Like it's it's wild. But anyway. Yeah. speaker-0: ⁓ shit. But other than that, I speak at tons of events around the world. I also have recently published a LinkedIn Learning course for Playwright with AI. So check that out. You'll see my bright shiny face. I forget the exact name, but if you like LinkedIn Learning, Playwright AI. ⁓ speaker-1: I I heard a rumor about a book somewhere too. speaker-0: I was trying to write a book, but then life happened. So eventually my speaker-1: I wish I didn't know anything about that, but we will put all of these links in the show notes, man. ⁓ this has been great and ⁓ you know, hopefully people can learn something from this. So thank y'all. We'll talk to you soon. speaker-0: Thanks everybody. If you enjoyed this episode, follow TestGrid for more conversations with engineering and technology leaders building the future of AI-driven software delivery. Thanks for listening to AI First Engineering.