Latest / Beyond Vibe Coding / #S2.07 Agentic Engineering Is Bringing XP Back - Ben Hoskins (NewStore)
Transcript
- Ben Hoskins: the somewhat funny thing is that it would be rare if I didn't have a black t shirt on. Sebastian: It it it feels like this is pretty much the standard CTO dress code, right? So wearing a black t shirt or a black hoodie. Ben Hoskins: Ha ha ha. There you go. André Neubauer: depends on the season. Sebastian: Welcome to another episode of our new season of Beyond Vibecoding, partnering with Impala Search. André Neubauer: The go to tech and executive search agency in Germany. Sebastian: In this podcast, we explore the transformational change in software engineering and knowledge work in general. I'm Sebastian Heidemeyer zu Erpen, CTO at North.io. André Neubauer: And I'm Andre, C DPO at Trusted Shops. Great to have you back. this time we are talking to Ben Hoskins, a very seasoned tech leader, so more than twenty-five years industry experience. Sebastian: Yes. it was like old pals talking about yeah, old experiences, but also adapting them to the new world. And we actually discovered many connections between the old and the new world, like agile, BDD, technical excellence, all of this stuff helps specifically when embracing agentic engineering. And he describes exactly how he's doing this at New Store, where he currently serves as a CTO. we hope you enjoy the discussion as much as we did. We definitely did. So have a listen. Enjoy. Welcome to the episode, Ben. We're excited to have you. And as usual, we'd like to ask you to introduce yourself to our listeners. Yeah, would you be so kind to do that right now? Ben Hoskins: Yeah, thank you. Yeah, Ben Hoskins, currently working for New Store, CTO, I look after product, engineering, design, etc. and I think you asked me a question earlier, Sebastian, as we're warming up, like what's a career? I haven't really had a career. I've had a a series of choices of things that sounded fun. so I've worked for the likes of eBay, Wayfair. I've also worked with different size companies, Newstoff, for example. WDS back in the day was a was a smaller company. And I just kinda go where and and I I've got a weird sense of what I think fun is, but I go where I think things are fun. That's that pretty much sums me up. Sebastian: That's an awesome answer. And I I would probably as well I would definitely say the same about myself. So there's no red line. It's just like all over the place, right? Probably also for Andre. It's I mean that that's what drives us, right? The the curiosity to always learn something new. Learning curve is super important for me personally, right? The journey for mastery. Yeah. Cool. Thanks a lot. Ben Hoskins: Mm-hmm. Yeah. André Neubauer: sure. Ben Hoskins: Yeah. Sure. André Neubauer: Nevertheless, probably the best answer you could give if someone asks you for professional career. I like Sebastian: And usually we start after the intro, pretty much, with the status quo. So how are you currently working? Be it in your coding. If you do some coding, you probably do, right? But also your other duties. you already mentioned some skills that you use for your managerial work, right? So yeah, that all of that is super interesting for us. Ben Hoskins: So I actually you're sat in my I'm sat in my office right now, which is basically it's got a ring of desks around it and they're all full of computers. And then the one that is to my left right now is an old Alienware laptop that I've got. I've subsequently s installed Ubuntu into. And I have got tail scale and T Mox on that thing. And I basically use that whenever I'm going anywhere around the planet. And I will leave it on my hundred dollar cloud license generating software. so basically I built maybe about fifteen years ago, I built a thing called card planner. And card planner is a a way of seeing a Kanban board and seeing the same board, whether you're in India or in Germany, so that when you're dragging a card, you can actually see it move in real time in both places. I extended that, I added MCP to it. So I go through a phase of doing a distillation of the specification. That gets into a series of cards. I then pull that over into Allium, which is it's kind of like behavior-driven development for LLMs. and you describe the specification, it's a little bit of ping-ponging of what the specification looks like. I then use that to Generate and I do it in big chunks. Everyone goes, test run do only small increments, doesn't really suit LLMs. so actually I do big chunks of tests both unit and end-to-end, all red to begin with, using the right types of rules that are in there. So for example, you would say apply boundary value analysis, apply If you're doing something that is multi-tenanted, like a card planner thing that I built, use Bola and it then effectively you you say use boundary value analysis and it'll triangulate all of your tests. You that I then hand that over to the LLM with a set of instructions around architecture and say, Go generate me some software. And basically If it's already got some good references of how you want your target architecture. Like hexagonal was made for this as an architecture, if you're doing something standard, because basically it's got the right level of decoupling. And weirdly enough, I've also learned how to build Flutter apps because of LLMs, because I never used it before and I just pointed it and said, make me make me the repo, make me the app. And I've learned how the app actually works. through when I'm generating it. So it'll it'll run through that mostly unprompted. There's a couple of things I don't allow it to do. I don't allow it to do all the usual bash stuff that it loves to do and and it's like a it's it's some kind of addiction that it crawls back to going to a grep every time. And I I make it use the IntelliJ tools. So basically I hand it the IntelliJ tools via MCP and I use that to do all the refactoring, et cetera, automatically. And it pops out working software at the end. And sometimes I look at the source code, I often don't. Now the that's in my own work. That's quite dangerous sometimes. I spotted about two months ago That my subsecond super fast front end for this particular app that I was building, because I I didn't use any frameworks, right? It was just straight JavaScript. It was it was taking a second, a second and a half. And when I went to go look at it, it had serialized the whole of the object model, including people's names, emails, phone numbers. And I'm like, there was no test that would have caught that. So every now and again I go and I do maybe weekly I will do an architectural review. I'll review what is actually coming out of it. But increasingly I'm not touching it. So that's my own stuff. so basically, and maybe I'll drop in at lunchtime when it's running to see what it how it's running, how it's carrying on. But fundamentally, nothing. It's let it produce software for me. Sebastian: That's super interesting. You you mentioned a BDD framework that was like adopted to to LMs. Wha what was the name again? Ben Hoskins: Yeah. So it's called Allium. So it's the name for all the different types of onions and garlic, etc. That's the family. So A-L-L-I-U-M. It's from a com Juxt G-U-X-T, the company that built it to begin with. but it's it's basically a way of describing the behavior of your application. And it's a way of like really precisely like codifying down something that would take multiple loops and pro often your L L would get wrong. Sebastian: interesting. That that's why I love these episodes because I always learn something new. Ben Hoskins: Th there's another one I haven't tried yet that is very old that describes language that one of my old old mentors from years ago is trying and I'm I'm gonna be giving that a go in the next few weeks as well to see if it does a better job. But I can't remember the life of me, the name of it. I'd need to go look at his LinkedIn to find out what it was called. Sebastian: Yeah. André Neubauer: You seem to be very happy with your approach, very relaxed, right? So you adopted to that. Yeah. Ben Hoskins: Yeah, I mean it it works, right? I mean, like one of the things that is really, really startling is that I tried vibe coding to begin with. I really just went, let's just have it generate, whatever. That was a hot mess. Like the architecture was all over the shop. It it barely did what it was meant to do. It would break frequently. Not you tried to go back and it would just break. There would be the history would be all over the place. And I I tried lots of stuff like that. And then I kinda realised It needs to be in a loop it needs to have a feedback loop. And that feedback loop is what I've been teaching people for the best part of twenty five years, which is give it some failing tests and make the test pass. So all of the agile stuff that I've learned forever is suddenly very applicable, which is kind of fun. Sebastian: Absolutely. That that's something we learn over and over again in our sessions. Like the the better your technical excellence actually yet that you have applied before, the the better the the outcome with the LLMs, right? And also the way you describe it, that you write a big chunk of tests before, which I ha haven't heard in that detail anywhere before. Super interesting. We'll we'll think about it and and try myself. It's pretty much providing the the the scaffolding that also was there when they tried to I think one-shot what have they rebuilt bun I think no no not not bun but I think a browser, right? So they some company you used a extensive testing framework of I don't know it was Chromeoth or something, right? And then yeah. Ben Hoskins: Yes. That also also did this with I can't remember which compiler it was, but they rewrote a compiler almost completely from scratch from this, yeah. Yeah, it was it? Okay, yeah. Yeah. Sebastian: The C. The C pr compiler, I think it was or something. Yeah, yeah. Same same thing. You're you're right, super extensive test suite and the L L could just or the agent could just work against the the test suite. Yeah. Ben Hoskins: And it's really interesting that things you learned 20 years ago, like everyone, if you're talking to anyone doing test run development, they'll talk about triangulation. That is just a lightweight way of saying boundary value analysis or design by contract. And all of those ideas, you're like, you give it those magic words and you say apply boundary value analysis to the test suite generation, all of a sudden you get a very, very good specification. André Neubauer: because we just talked talking about TDD have you ever tried semantic anchors? have you heard about that? Ben Hoskins: No, what's that? André Neubauer: Semantic anchors. we had a, I think, was it a German? Was it still a German episode? Ah, OK. Then we may need to record that again with Ingo. That's more or less an established term in an LLM, which activates very precise knowledge. So if you say, do TDD according to something, Sebastian: Yes. Yeah. Ben Hoskins: Yes, now now we understand, yeah. André Neubauer: then the results are pretty pretty like, well, like they are better like compared to build a TDD application following TDD approach. So I was just curious to hear if you made the same observations. Ben Hoskins: So B V A is very much one of those semantic ankles then because it switches in a completely different mode. Sebastian: Yes, absolutely. for sure. It's like T D D London School that you can avoid to fill up your context with a long description of how exactly you want to do T D D. You say T D London School and it does it. Or ADRs according to Nygaard is another example, we'll do perfect ADRs. Yeah. Yeah. so maybe we need to record I think it was Ralph, right? He's the inventor of this this term. yeah. Yeah. André Neubauer: Yep. Yep. Ben Hoskins: Yeah. Yeah, interesting. Yeah. André Neubauer: Yeah, you're right. was not Ingo, was Ralf. shit. Ben Hoskins: Because I guess Sebastian: Anyhow. Ben Hoskins: it's like, hey, that specifically means this particularly small part of the graph and then it's super contained, then like the references on it will all be like, Yeah, that makes a lot of sense. André Neubauer: Yeah, absolutely. Sebastian: Exactly. Exactly. Yeah, and maybe before we go to the the main part of the episode, you described your personal coding setup, right? Super interesting. Do you also have like some like specific tools for your managerial setup that you use or or is there also a tech stack that that you're already using? Ben Hoskins: Mm-hmm. Actually that's more a case of building a bunch of skills out of this, right? And I I was as I was mentioned earlier, like I built a couple recently which were think like Ben and Write Like Ben. And basically what I've done is I've shoveled every piece of communication I've had since I've been in New Store into it. All of my emails, all my conference docs, all my Google docs, all of the Slack. Messages that I've had. and I made it actually literally go through everything that was in there. And it distilled down the way I think and the way that I communicate. And it's not for the reasons that you think, right? It's like actually I built the think like Ben because I'm going to go on vacation in September. I'm going to go off for three weeks. And I want to give it to my CEO and say, look, here's a here's a Ben bot. Like ask it a question to see how it would a how it would answer while I'm off. And the the write like Ben I did because sometimes when you're using Claude to do architectural analysis or you're looking particularly at products, or you like the the way that you want to place this product, the way that you want to think about it and position it, it comes back with garbage sometimes, like weird long chain sentences. So I use write like Ben that so it can then come back. To me in a language that I understand, then I can instantly understand what it's trying to explain to me in terms of concepts. And it works incredibly well because there's I I guess it's you match the the cognitive like so it's it's quite a smart way of doing it. But I I guess one of the things that we're we're really in work, one of the things that we're we're really grappling with now in in order to actually get down to something that's consistent is that canonical. model across the piece, right? So it's a SaaS company that I'm in. So you've got a sales team, you've got a go to market team, you've got like customer success, you've also then got product, engineering, design, support. And like like most companies, like the the product definition was across multiple places. And one of the things that we're pulling together right now is pulling that that into one. And Of course we started in knowledge bases because that's where you start, right? So knowledge bases, Henning, great, the the guy that looks after design for for the organizations, great. He's light years ahead of most people and when it comes to LLM adoption. And he also runs content because content and design, especially in SaaS companies, are fundamentally the same thing. Because your product is often a suite of APIs and how the documentation tells you they work. And He's pulled a bunch of that stuff together into like Doc three sixty, external, internal docs, all through MCP feeds, and realized that that got him to a certain level. And now he's built rag he's built a rag system, he's built other pieces on top of it. After trialing a couple of tools that do this, he decides no, we can go build our own and he's bootstrapped building this to pull the information in from the organization. And the next thing for us is to build that canonical layer of what the product definition actually is. So we're I would say we're reasonably early on in that journey. and that more than anything is going to drive that like that next level of what we what we unlock with AI. Sebastian: And and maybe one one more question on that. What is the specific artifact that you have in mind that's the outcome? So is it pretty much information that's in this rag retrievable knowledge base? Yeah. Ben Hoskins: Yeah, which will be like, yeah, so this is what our module structure looks like. This is who the modules are you compine together to have these different product sets. And then on that, then you can base your contracts, you can base your product decisions, you can you can then test different product ideas, potentially with synthetic personas, right? Because we've been playing with this as well. like the other piece that we're which is a complete aside. Sebastian: Yep. Yeah. Yep. Yeah, yeah. Ben Hoskins: Like synthetic persona is really interesting because we're we're also trying to build well, I'm not like Tanvious exploratory testing automatically through an LLM, which I've been trying to do for the best part of twenty years, by the way. Like I tried building a framework for this fifteen years ago and it was not good enough at all. but I I yeah, so that canonical layer will effectively be something that you putting your rag that is probably gonna be Jason, we don't know yet in terms of structure, maybe a series of documents. Sebastian: Yep, yep, got it. All right. Thanks a lot. Super interesting as usual. And yeah, I guess it's time to go to the meet, which is pretty much agentic engineering at news store. So you described your own approach already, which probably differs somewhat from how the company currently is working, however, might influence this. Yeah, could you share how the company is working, what you're doing there? Ben Hoskins: Yeah, of course. Like I think what's interesting generally about adoption is especially with AI, is that a lot of the early folks that tried stuff, like they they brought up Cursor and they tried using Cursor a year and a half, two years ago, and then they kind of got bumped with it. They saw it as being a bad copy of Stack Overflow, something that's horribly copied and pasted in. And what happened was Up until literally November last year, people were summarily kind of ignoring it. because they thought, this is just not good. And they hadn't realized how much it had actually progressed to the level that it actually is now. And it took me I had to then go, I went and built a point of sale system in four days. and I went onto an all hands for the product engineering design support all hands. And I demoed it to the team and I said, look, if you you guys aren't if you're not looking at this, this is what our competition's gonna do. Like you can't sit on this, we can't have it. And then actually that just started to like raise the level a little bit. And I would say we got to le like level one, level two. And at the same time we did that, we started looking at changing the the role descriptions so that we could begin to incentivize the right people. on the right types of st on the right types of behaviors that we wanted. And it really varies per team, right? So you look at like what we've got around the point of sale and and the selling team, they they often they mob together, they often have multiple people with an LLM with them. Some other teams will pair with an LLM individually, share prompts, like like share skills, and they're not doing that inter team yet. Right. So they're doing that within each of the the the domain areas, of which there are five larger teams with multiple domains. And I would say that we're starting to get some traction in the guild that we've got. So I would s it's more along the lines of shared prompt at this point. and mostly a couple of different languages. So it's it's Python and Go. There's a few others in there. and it's partly Because of the fact that the teams historically, when I joined the company, there was 15 teams, not five, and they were all building whatever infrastructure they wanted to build. So actually there's a coming together of that before you really get the benefit of the LM. But actually, even with that, with people like sharing prompts, sharing skills within teams, they're already ahead, right? So I would say that the bottleneck has already moved into product. And when look at like historically, what is what happens in product is that when you're doing SaaS, you need to kind of get it right because then as soon as people start adopting it, that you can't go back and say, no, no, that was an experiment, sorry, and they've just spent, I don't know, 20, 30 grand with a service integrator to actually make it work on their infrastructure. So like what h historically has happened is product folks will be Like bringing in like we're doing this in some areas, like talking to six, seven different retailers, getting feedback on the feature that we're looking to build. And then looking at this commercially, looking at all the different angles before it even reaches the team. And the biggest thing that we've we're starting to change here is we're saying, okay, now let's get much, much smaller. We've still got these d the teams looking after the domains, but you've got much, much smaller teams. One product. one or two engineers and those will cycle out depending upon the mission that you're on. And you'll have something end-to-end to deliver. And actually I would say that already with this, we're getting much, much further with that. Because you you kind of you got to break that like weeks and weeks of discovery and say, okay, we'll still need to do some of that due diligence, but let's do it while we're building stuff out. So For example, right now we're doing stuff around moving us from being a system of record to a system of intelligence. And we're building a bunch of MCP tools on top of that. but what we're saying is we don't fully know how retailers are going to use that. So we wanna get it out to them as early as we possibly can. Get that feedback in. And then like and then there's two ways that you can go. You can either use it to generate reports or you can really enhance your MCP toolchain so that folks are using the likes of Chat GPT and Claude directly. And basically it's let's do that based on what the use looks like, which we would never have done before. It would have always been let's let's get the pre-testing done with wireframes, etc., and then we will release it. Right. And it was a lot more deliberate. I think it's what's really interesting is that AI as a technology has kind of released you from that because you don't need a whole team to build stuff now. You need one or two engineers. And like that's that part of the code that you you thought was going to take months suddenly takes a couple of weeks. And you're s you're you're no longer bound by saying, my god, that's going to take a month to Or three months to rewrite an API, it's going to take a couple of weeks instead. And your bigger problem is how are people actually going to use it? So we haven't yet, we've we've started to invest a little bit more on areas that are easier. Like everyone does this, right? So for example, something happens in support. André Neubauer: Hmm. Sebastian: absolutely. Yeah. Ben Hoskins: Well, then you've got all the diagnosis and all of the the tool chains, et cetera, that you need to go through to do that. Let's get MCP tooling around that. We I just last week saw something come out of hack day that I want to push out across the whole of the company. So anything that gets you your time back so that you can then invest, it's is a win in my book. So like things that you've got like just dead work, just use that. use an AI to do most of that work for you. so you get the time back so you can reinvest it. Sebastian: That's I mean that's that's always the like the most obvious part, right? Where you can invest to read these efficiency gains. And then the second thing is also no, actually it's three things, right? In one thing is how to internally use utilize the the tools and and l like change and adapt your your ways of working, right? Then second thing is how can you leverage this to to get efficiencies across the org? And then third thing is yeah actually put it into the product and see how you can enhance the product experience with the MCPs that you're building out, right? Ben Hoskins: Yeah. Yeah. But it's not only on the engineering side, you look in the product side, we've had hundreds upon hundreds of product requests come through. You can stop pulling that together and you start making personas out of it. Sebastian: Yeah. Yeah. And and you can do pattern matching automatically to see w where's the highest signal and and whatnot, right? Yeah. Absolutely. And Ben Hoskins: Yeah. Yeah, exactly. Yeah. So like at that speeds up as well. And so anything where so what I'm kind of finding is sure it's not the Pansea, but any area area that you're looking at that was previously labour intensive, you can often apply an LLM of some description into that and suddenly start speeding up. So it goes back to then back to fundamentals and like look Goldwrat's theory of constraints. So What you need to do is you need to look at your system as a whole to work out what your constraints actually are. And then don't be optimizing continually in engineering if that's if if you think that's the right thing to do. Look at everything systemically and say, actually, the bottlenecks here in product. And after you've removed all of that unnecessary stuff that people are doing, then you need to then look at how you can elevate on that bottleneck and start doing some work there to actually get your throughput up and then it may go back to engineering or it might be design or it might be something in support. But the the general gist of it is if you can optimize across the whole of your organization and not just micro optimise, it'll it'll make your teams faster. And the that's a fundamental thing that I learned as an agile coach years and years and years ago. Sebastian: Yes, absolutely. I think there's so much into what you've just described, so we need to unpack it a little bit. maybe starting from the last point, we say, okay, you need to look at the pretty much where the bottlenecks are moving following the theory of constraints. How do you measure this? Do you have KPIs or how or is it more a qualitative thing? So how do you assess which are the next bottlenecks? Ben Hoskins: I think it's what there's super easy way of doing it is what look if you're looking at a Kanban board and you see lots of stuff piling up in one part, maybe that's the first place to start. And you know, people do cumulative flow diagrams, etc. Most of your tool chains that you've got out there will give you that stuff by default. But ultimately, eventually you kind of get a nose for it. Right? Like it's it's a it's a weird thing to say, but you You can have the data and you can look at the data, but if it smells like it's slow, it's probably slow. Right. Sebastian: So so basically you're saying you're an exchange with the different parts of the organization, right? And then you you you you get the insights from customer success or wherever there's currently a bottleneck and then you you approach it. Yeah. Ben Hoskins: Yeah. Yes. Yeah. Yeah, exactly. So and that's absolutely vital that you maintain and you have those really strong relationships with the organization to work out where the bottlenecks potentially are and and sure everyone's got their queuing systems, they've got their ticketing systems, whatnot, and you can dive into that and actually an LLM can help you there as well. But genuinely the place to start looking, you probably have a nose for it. And and the other thing that you want to do is you don't want to bias yourself because you've got a nose for it. You might be proven wrong. and you wanna you wanna leave yourself open to the fact that you're probably wrong. but still then go in and have a proper look. André Neubauer: How do you approach the fact that, well not the fact, but the chance that speeding up things will lead to less quality? I think you mentioned earlier, so product is now the bottleneck, I think so too. So what could happen, right, if you now have a faster engineering organization, like you just build everything, right? So like every feature gets built. Ben Hoskins: Yeah. André Neubauer: Not sure whether this is the desired intention. would say no. But is there something to avoid, like bidding average stuff? Ben Hoskins: I think that all depends on what how you value user experience. Because I think, like for me, it's sure the architecture's important. What's much more important is how people are using the thing. Right. So I would say that yes, you can build lots of stuff of average quality. but again you need to question whether you actually want to do that or not. André Neubauer: Yeah, absolutely. you have anything in place at Newstore to avoid. You're building average stuff. So you do any kind of, I don't know, you have some quality metrics in place to ensure product features are well thought through. fits in the strategy. Ben Hoskins: It's a it's a very hard thing to describe. so I'm gonna steal a quote from the my head of platform and developer experience, and it's culture eats metrics for breakfast. Right. and it's like it's the same as like everyone obsessed three or four years ago about Dora metrics, and I'm like, I I I don't measure that. not to begin with. I what I want to do is I want to build a team that values that and then the team want to measure it themselves. And it's the same as like quality out to the end user. I want a team that cares about that. Because if you don't, it they won't. You'll get the the same thing on quality. So you can measure it. and we do. We we measure things like failure rate. We we've got what we're doing investing more and more And like production and and issues there, and and we're really, really, really focused on resiliency more than anyone else that I know that are building a product similar to ours. Like, and we're at the point where we're pushing more stuff to the device to actually make that resiliency work in a store environment where the Wi-Fi might not always be great. and we do a lot of that work based on data, but ultimately it's about that touch and that feel and that affinity for the customer. We've had teams working in stores with the retailer so that they can actually feel how that thing is being used and where the problems are. So like, yeah, we do have measurements, but we've got teams that care. It's probably the best way of describing it. And that kind of is a great counter to then just shipping whatever of like intermediate quality that you could. I think basically if that gate wasn't there, yeah, I think it wouldn't be anywhere near as good as what it is. Let's put it like that. Sorry, it's not a sexy answer, but it's Yes. Sebastian: Yeah. André Neubauer: No, no, no. I think it's also a hard answer. So I would also tend to say sometimes it's good to slow down, to not just make use of the speech which is there. I think some people call it product taste, It needs to fit in to quote Steve. It needs to fit into a cohesive structure. Ben Hoskins: Yeah. But it cohesive structure, but I I would still like it faster. André Neubauer: Yeah, sure. Right. In the moment where you realize it fits into that, then full force. just, I'm a bit worried what we will see in the next years is that just giving the speed like, like average products, every product will be able to do everything. And there is no, like there is no, I don't know. It's, it's just not, can we say sexy on this podcast, Sebastian? is it, it, it, it, yeah. Yeah. Okay. Ben Hoskins: Yes. Sebastian: Think we can, yes. Yeah. And he did, yeah. Ben Hoskins: I just did earlier, so I think the the thing that's interesting is that now that you can release faster, will people iterate, right? Will people learn? And having a learning organization that will then take those lessons and say, that didn't work, and we're deleting that. that that's gonna be very interesting because I think the ones that do are probably the ones that are gonna win. But actually There's potentially a way that you can automate that as well. It's like you you you do your metrics and if something doesn't get traffic, just delete it. Delete the software. Sebastian: I I think what we're discussing here is pretty much the initiativation issue, right? Which is now even sped up by the the use of agents. And it in the end it all comes down to I think you're right there, Ben, to understanding your customers where they are and if they are actually happy and capable to like adopt Ben Hoskins: Yeah. Sebastian: 10 new features every week or every month, or if they need something slower and maybe gradual changes that that guide them into a new direction. And and there are folks like in in both in in both pretty much extremes of the continuum, right? and and it yeah, in the end, the you can have and should have measurements, I I think to pretty much have kind of like To to to see if you're on a healthy path, to see if there's something going out of bounds that you don't see any way other, but the more important signal or channel is actually the relationships with the customers and understanding where they are and how they feel, right? Ben Hoskins: You can't yeah, you can't measure a team to be awesome. It's like you y somehow that doesn't work, right? Sebastian: you you can't measure the lines of code and then you you know that the team is awesome. The the the the old the the old idea of measuring developer productivity, right? So what is it and how do you measure it and yeah. André Neubauer: You Ben Hoskins: Ha ha. Yeah, I produced one when ten would have done. Yeah. There you go. Sebastian: Yeah, yeah, exactly. Exactly. yeah, cool. And early on I think you mentioned also that that you changed the roles a little bit in your teams. Did I understand that right? Ben Hoskins: Well what's what's interesting is that the roles change through the nature of AI, right? So when you look at basically domain knowledge used to be king and now I've got the full software on my machine and I can generate a sequence diagram like instantaneously from it and I can I can tell how the software's actually working. And what's also happening is that that those mythical full stack pie or t-shaped people that everyone was going after for years, we've kind of got them now. Right? So you don't need massive teams of specialist front end, specialist mobile, specialist back end. Because genuinely, like as long as you're a decent engineer, you can get a good like a a far across many parts of the stack that you haven't you don't actually know our own. André Neubauer: Hmm. Ben Hoskins: So what has happened is it's gone back to what happened at the beginning of XP. The the the team that I started with in 2003, there was no product, there was no design. There was a there was engineers and there was customers. But in reality, those roles existed. There were just different people in the team picked it up, right? I did a lot of the product work, I did a lot of the design work when I was in that team as well. But you look at now, it's like a lot of the engineering complexity has collapsed. André Neubauer: Hmm. Ben Hoskins: And you see the engineers that are moving over towards product thinking are getting ahead. And product folks that can move into like really good commercial business thinking are also getting ahead. So it's it's shifting the role through the nature of like that. I would say it's more like a collapse of skill sets that you don't really need to know all those esoteric things as much anymore because your LLM has got the whole history of humanity in it, and you can just ask it. Sebastian: And and did you already translate this into role description changes? Ben Hoskins: We we are adding the role description changes. I think they went out in the last the last one that we did in February. But we did change them, yeah, we had to. It you can't you couldn't keep them the way that they were. Sebastian: Yeah. So so it's now more in the direction of the product engineer, if you wish, right? So someone who pretty much Yeah. Yeah. Ben Hoskins: Starting to go that way, yeah. I think more than anything when you look at role descriptions, they they sound like boring things, but they're basically a way of distilling the culture of your organization. Right? They're a way of crystallizing what you actually what you value as a as an org. So we're starting to push to more towards that. And we've got a lot of people on the team that are already like that. Because a lot of people that do XP have got that type of thinking anyway. André Neubauer: How do you incentivize that path? Because at the end, people are doing the job, so you need to convince them. Is there any secret sauce you discovered on your journey? Ben Hoskins: You you promote the people. You promote the people that that like have the values. You you give people like you give people the right incentive to say, Yeah, you're getting ahead because of that. You will get your next role because of that. That's the best way of doing your incentivization. And just live the values yourself, right? It's like it's you it's no point in saying, Hey, we're doing this and then you do something different. André Neubauer: okay. Sebastian: And then the the follow-up question would be you mentioned that you went from fifteen teams to five teams, right? Five five bigger teams, but now it seems like actually you're going to smaller teams that are working on software increments or features, right? But these are probably temporary. Ben Hoskins: Yeah, so and that that was also that was also a progression. So we didn't just do that overnight. So we we had like teams that had we had to fix some architectural issues from having fifteen teams all doing their own thing. so having five teams means that when you've got a fixed architectural problems, you don't need to go and ask three different teams to do it. You can just do it within your team and you can plan it. but basically we'd already formed the concept of temporary squads. before like we really started adopting AI. And what AI basically did is it made those temporary squads smaller. You don't need as many people in them. Right. So we'd already had cross team teams to do these larger initiatives. Because most of the stuff now, because whenever you've done stuff within domain, like at this point the company's what, 10, 11 years old? Sebastian: Yes. Ben Hoskins: They've already got all of those in domain things pretty much done and all of the problems are left or interdomain. Sebastian: Yes, got it. okay, that's that's interesting. So then you you pretty much have these five bigger like domain orient teams, right? And then you form temporal squads that are working or temporary squads that are working on inter domain issues with representatives from from the different bigger teams, right? Ben Hoskins: Yes, that's right. André Neubauer: You have any experience or learnings? How does it work with cognitive load and stuff like that? Because smaller teams still... Ben Hoskins: It's everyone com everyone complains about it as being like, there's too much information overload. But actually when you do it, it it doesn't really the reality of it isn't that much. And actually now with LLMs it's much, much easier. And a part of code that you haven't touched for like a year or two, it'll tell you immediately what it does. We've got tests as well across everything. André Neubauer: It's more theory. Okay. Yeah, what I often hear is that you have smaller teams, or like also smaller organizations, but in brownfield environment, that this is a bit unfair, right? For new organizations, it's simple to adapt because well, like you build your environment differently. So with brownfield approaches, you need to care about like the old world still. And then you often are confronted with the fact that it's like, it's still complex, right? So a lot of cognitive load, but Ben Hoskins: no, we've got that everywhere. André Neubauer: Yeah, but in existing organizations, I think you build software maybe also differently in the past. So you followed way more also approaches to handle cognitive load. Now where AI is taking over the larger part of software engineering, you may also can skip certain architectural patterns and end up with different systems, is what I'm saying. So what I can share maybe also from our organizations we run. of hundred Lambda functions, probably something you would not do if you built that now with agentic engineering, but nevertheless this now exists, someone needs to take care of that. And then you always get the feedback, well, how do you handle that with a smaller organization? Still, the software is built in the old way. Ben Hoskins: Yeah, I mean like w we've got all of that. We've got like every possible architecture you can think of, we've got. But like basically the gist of it is the gist of it is that complex system, you still need to invest in proper levels of quality, et cetera, in order to actually make any effective change. André Neubauer: Sure. Congrats. Sebastian: thanks, Ben, for giving us this comprehensive overview of how you're working was super interesting and was great having you on the show. Thank you. Ben Hoskins: You're welcome. Thanks both. hopefully it speaks in. André Neubauer: to the recap, what stuck with us, Sebastian? Sebastian: for me really interesting was his personal setup and how he writes these big chunks of tests first before he starts actually implementing. I will also definitely have a look into the allium framework that he described. Yeah, really interesting. I definitely will take something away from this, yeah. André Neubauer: Yeah, and one of the reasons why we do this podcast, right? These small but very powerful recommendations. so my first takeaway is think like Ben, write like Ben. I need to adapt it to myself. I have also a bunch of skills helping me and also well like trying to I would not say simulate me, but well, like cover certain parts of my of my thinking. Sebastian: Yeah. André Neubauer: process, but the way he specifically uses this like very powerful so really need to think about building like think like andre, write like andre. Sebastian: Absolutely. I did something similar to this in the past, but on a on a way smaller scope where I feel like, yeah, the agent has like eighty percent of my writing style now adopted, but I think I can do better. And this is also something that I will definitely look into. André Neubauer: I I I can also share because you you shared that I did I think it's very simple to well not very simple, but compared to the thing like Ben, write like ban in that case is way more difficult. so I also tried that myself. I think you needed really a ton of of documents to really derive the writing style. so at least I I would also say I I I I landed at eighty percent and eighty percent is not good enough. Sebastian: Yes, you you always need to like do the final polishing. Yeah. Which actually for daily work I don't mind or even like it that way because this ensures that I always go over everything, check that the content is really right, right? But for some purposes it might be cooler to to be closer to the 100%. Anyway, we'll definitely also look into that. Yeah. And then what I also took away was the canonical model that he spoke about of the application. And the ideas, how he w wanted to utilize it are also quite interesting. Would would love to hear then the outcome of this. So what he described was that they put in all the information about the application or the the software basically, the the platform and For example, in order to test new feature ideas via specific personas against this canonical model, he he wants to utilize it, which is ki kinda interesting. And also the holy grail basically to automate exploratory testing, which in theory actually should be possible now. And I heard someone else do something similar in that maybe we we even had someone on on the on on one of the episodes who described that. He's also asking after every implementation the agent to use the browser and click through all the newly implemented features and basically do an exploratory test. So that's really interesting and I think that that's the holy grail and but probably we're we're getting there. André Neubauer: What I also liked looking back is the discussion on product taste. I think at the end we can summarize it there, or engitification, to put it the other way around. like using the gained, the new gain speed, right? so not just implementing everything, but accelerating testing, so to say, or like iterating. also one of the things you mentioned in the intro. so l like making use of the things we we we we always looked for, right? so continuous improvement and not just like now building every feature which a certain customer may request it. Sebastian: Absolutely, a hundred percent. Yeah. And then last thing that I took away was the temporary squats. This was something that every leader wanted for ages, I think, and in the past was super hard to do this. Also, and you made the point due to the cognitive load, right? Context switching always, which is now probably getting way easier because you have this agent. by your side who can help you get up to speed and and switch context way easier by giving you exactly the context that you need when you enter a new topic. And this reminded me of our episode with Bastian from the Getaway Group who described exactly the same approach. So also curious to see if if that works and how this works out for New Store and also for the getaway group. André Neubauer: Meh good reminder that we need to invite both maybe half a year or nine months to see how things are going. Sebastian: Yes, yes. Yes, and with that, I think we have it And thanks all for tuning in again and see you, next week. Bye-bye.