Latest / Rethink Engineering / Patrick Debois: Your Software Factory Should Be a Dream Factory

Transcript

Patrick Debois: Your Software Factory Should Be a Dream Factory · Rethink Engineering

• speaker-1: Moin moi and welcome back to another episode of Rethink Engineering. This is a very special one for me because we discuss a lot of the topics why we renamed the podcast with our guest today. I'm Benedict. I'm the co founder of Hackers and Wizards. We teach engineering teams how to work with agents and factories. And this show here is powered by us and also by Rethink. My guest today is Patrick Dubois. I'm a huge fan. He organized the first DevOps days in Ghent, I think two thousand nine. So he's been doing that for a while now. And there are a lot of similarities between DevOps and Gentec Engineering. So he has really interesting thoughts on on what we need to do and what is happening at the moment. We debate factories, organization, also how this changes our roles. And one thing that that got me specifically was that we are currently maybe focusing too much on code and we overlook the possibilities that these factories have once we focus on outcome. And yeah, that is what we are going to discuss in the episode. I really enjoyed this one. I hope you do too. Have fun. speaker-0: Mm. speaker-1: Mm. Hi, hello Patrick. super awesome to have you here. Thank you so much. I'm really honored. really looking forward to ep to the episode. speaker-0: As they say, muang muang? No, I don't know what to say. This is how speaker-1: Well speaker-0: you always greet me. So thanks for having me, Benedict, on the show. So speaker-1: Yeah, awesome. So let's I I don't know, let's just I think dig into the topic. I'm super interested on on what what's on your mind at the moment, how do you feel about engineering? I think you have a lot to share, so maybe you just start with whatever you think about at the moment the most. speaker-0: I'm trying to make sense of software factories and dark factories and given my background in DevOps in CI C D, trying to see what's the same, what is different. Kind of that is kind of maybe like my analysis, where is this going and where's the where's the adventure heading, basically. So that's kinda on my mind. speaker-1: we have a have a similar journey here, also trying to figure that out and and I'm I'm super interested in in what do you feel is the the difference though. Like what is your like is there a difference? Is it the same as C I C D? What what's the current status there? speaker-0: Well, I was looking comparing, companies that are offering factory solutions and I was a little bit sad in a way that it was very similar. They were all kind of, hey, we're producing code, we're reviewing code, we're deploying code, and and it felt like in a way it was the same. Of obviously it's the the technology is new and we can do new functionalities, like doing in the review, we could like have Different ways instead of just running and testing the code, we can have it assess code, we can have adversary code checks. and I think one maybe emergent topic that comes up more often is the one about risk. And that is especially if you take it up to a traditional CI C D, you level it up you know, step by step and add like more checks to it. And You assume that the checks are enough to do a certain level of deployment, right? But there was still a lot of human involvement. And now with the what we hope for is that there's a lot more autonomy on the system putting things into production. And that verification layer was traditionally testing, as I mentioned, but now it's it's like, you know, a whole set of verifiers you could call them. But when do you actually merge? Right? Like that decision When well, it it's it's actually even beyond that. It's not just what do you merge. If you merge, you deploy, right? Like what do you put in front of your customers and what's the risk analysis of that? And I find that fascinating that everything that we've done is kind of re re like reduced to what's the risk of putting something out there versus the benefit of putting something out there for a customer. So but yeah, and anyway, that's Interesting in a way that I draw the parallel of we automated so much with DevOps that we could, instantly get a new machine, it has a new version, very fluently. But it was always the discussion you should automate, let's say, things like de database schemas too. But not a lot of people dared to do that because the risk was outweighing like that manual step. and I feel now we're in similar conversations, but what could you do? And and I feel there's a strong correlation to and and I think that's an undersold value of the software, where you can run all the tests and it passes and you can review the code and it's in a good shape. But we learned with the deployment that it is your architecture of your application that will absorb a certain level of uncertainty. The fact that I could kill my container and it starts a new one is the way we architected microservices in a way. And I I'm I'm feeling like, hey, if you compare to make like compartments of your code and pieces, they could say, you can touch this, which is okay. Like another color might not be the end of the world. But if your payment system is goes down, you might have a bigger issue. So kind of like find these those pieces and it's just an example. There's there there's obviously per domain, per company, the risks are different, and how do you deal with that? speaker-1: Interesting. So so you say basically that the the idea the idea of a factory is that you can actually automate more of your engineering work away that you were not able to to do before, right? So you the the writing of the code or the reviewing of the code and so on. And you're saying that really hard to figure out how to lift that up from being a lot in the loop and being not so much in the loop, right? And that is this risk problem. speaker-0: Correct. Yeah. speaker-1: Yeah. Okay. And the idea that you described is that maybe not every part of the code has the same risk problem. And that's okay. And that's why someone could kind of figure out if that is something that we could also do in the factories that that we kind of compartmentalize our code and so on. Different than we did it before? Is that the thing that you were saying? speaker-0: Well, we're we're tr like it's one way of looking at it, right? I'm almost already projecting, imagine everybody's automating that same loop. Code, review. It's not gonna be that different in every factory. Okay, we'll maybe have some bells and whistles on on how that loop will go. but ultimately what the the bigger promises of the dark factory, not just, the factory as we know it, is that there's decisions made by machines. in a way that we don't have to put our attention to it. So if the factory decides on its own to do a change, will we always have to do the review yes or no? And that's kind of okay, I can put a million tests on there, but still what's the risk of it doing this without you in review? and and that's kind of the more of the advanced piece that I would look at in a factory. Like how does it help you not just like review the code, but what could it do to take away pieces of that piece. So while it's not really engineering work, like it's not building it, it's not the mechanical deployment. It is more of a business centric like decision, whether you should like do it, yes or no. and I feel that there's a correlation between the architecture, but it could also be maybe your service level agreements or Your customer size or I I don't know what, they're all factors that go into that piece. but I also recognize this is maybe more for the advanced piece in a factory where a lot of this will accumulate too, because we want it to be autonomous. And that's kind of one of the the challenges that needs to be there. speaker-1: I think that's that's quite on point. for for me, I I didn't think that much of risk at the moment. So I think that is a really really insightful. For me it was I I always talk this story about trust, right? I I trust my CI C D pipeline because it's deterministic and because I have a lot of checks in there where I know okay, it's not going to do anything stupid. Kind of the question for me was always like, Yeah, how need do I need to build my factory so that I can trust it to make the right decisions? I don't know if that changes the idea, but maybe risk is is it's a better approach because I I really like this notion what you're getting at. It's like the architecture has some correlation there. Because I mean I mean we have dependencies, for example. I don't I mean I pin the version of the dependency and I'm fine with this part of the code, and I I I feel the risk is low then, right? So for this part of the and I could build a I could rebuild this. Dependency if the interface to the outside world is the same. I I don't I mean there's maybe this is like a thing which is compartmentalized where I I don't have a lot of risk because the interface is quite clear and I could kind of do that with more of my components and then say this component is not the factory does not decide anything there because I don't trust it yet to do that or something. I don't know. speaker-0: There's certain fallacy into believing that we can test all the things. and that kind of is maybe an uneasy truth if you put things out there, because there's always a different piece of you know, okay, of course, in production is different than test, right? Because there is users, there's different things that will happen. But if we really wanna like look at all the code that the the thing is producing, there's gonna be some cases where It just slipped. And but it's it's similar to what kind of a human does. But they usually have this kind of feeling of, hey, this needs a peer review and this is like that kind of thing. So analyzing that and bringing that to surface and then mitigating that in your architecture, is probably something that makes sense. And I started to think about the the dark factory has gradations in a way that for some cases. There's no it doesn't make sense with the risk you're taking to automate certain pieces. And others will be, yes, of course. That that's the the so it's a spectrum of that. And the the way you decide on the spectrum is what is the impact and the risk and the likelihood, kind of similar to you know, things that you would put into production. And and automating that will be more instrumental to it's saying. You can go to production, yes, no. But it's maybe additional. I'm not saying dump all the tests, but it's a it's another decision that you have to see. Like, hey, I looked at the piece, you didn't touch that, you didn't touch that. Seems fine. I can do this autonomously. And and and that's maybe a step that we couldn't do, like something adding to something to a pipeline that like it decided on its own, and then kind of saying, Well. You can do it on your own. speaker-1: There's a lot of discussion about should all of these checks be deterministic, right? And what about also non-deterministic checks in there? and and I feel that that this is really a a new task here as well is to, as you said, like to figure out what we can we do non-deterministically and also still have trust or low risk in it? Or how can this even be like a benefit that we didn't have before? Now now that we have non deterministic abilities, can this even be a An a huge improvement than only the tests that were deterministic before. speaker-0: Correct. I'm I'm a big fan in turning whatever we can into deterministic checks. There's usually a benefit of speed and cost that we don't have to do it with like LLMs or turns or it takes too long to rediscover those pieces. Now it's a double edged things. It's like when you think about the test as a hardness you put around the LLM, right? Like okay, you like you can't pause if you do you can't sol solve this piece. There's a there's a fine line between microcontrolling your L LLM and kind of giving it some freedom that it it can solve pieces, right? So so there are like a cache on in one way of rules you could write in in the prompt, but you need to know almost your you need to have a a maintenance check that says like, is this still the right way? And that's kind of the factory reflecting on itself and seeing like is this still kind of correctly in place because it happened b and we added it because the there was an incident. But is that still true? So it's doubting yourself on the architecture, on the solutions as well. So speaker-1: Like you you said before that that you feel or or that that you saw that most of these loops might end up to be very similar, right? you have this default spec and and plan and so on. I I know that there's a discussion at the moment that we might not even need these like pipelines with these checks and so on, that that you can just have an orchestrator and it will figure it out all on its own. So what is your take on that? speaker-0: LLMs are great, but they're also not great at certain things. Let's say you want to have a a counter. Just just don't use an LLM for that, right? So that doesn't make sense. Or I had it now multiple times. I build a tool and I say, well, read kind of use an API to get the IDs from the database and then pass that on to something else. Now It just hallucinates IDs after a while. So the the technology inherent isn't the reliable piece all the time, or it might lose context, or like you you can't treat it as this will replace all the things. Now there's the ultimate hope that as long as you specify everything perfectly, you know, things will go right. But I I don't buy into that series. I'm not saying you shouldn't specify things. It definitely helps that to give context and make and give better results. But I I don't see any even with like exhaustive training, it can't make like understand your business decisions, it can't put those things into its own understanding. Well, it's not really understanding, it's it's more like predicting. So I'm I I don't buy into kind of it will take over. you see quite often an orchestrator starts quite loosely off Hey, pick the next task. Right? Randomly, here's a thing to do, like do something. what you see, most of the people in the end building is close to a state machine. Well, b first it's here, when you're in that piece, then you can go there and they put things in in a kind of a graph system. Why they're doing that? Because they know that it's unreliable to just have it to it on its own. Or the camp of the LLM training will say, given enough training. It will do it. But we're we're somewhere not there, right? So why hope it? And I I I think it's that fallacy of those shall not fail is kind of what almost in the last decade that we found it fought against and we say assume it's gonna fail. That's a way better approach to kind of deal with that. But that's why if you assume it's gonna fail, you gotta think about the risk. speaker-1: Also, camp camp more deterministic and more let's let's make it a state machine. I I think there's also this difference between am I sitting in front of it and looking at it all the time, or do I want it to run autonomously somewhere? so that's always how I mean if I'm or when I'm using it myself for building apps and I'm really in front of it. I tend to do it more in this orchestrator approach because I can just follow what it does and then interrupt it or figure out if it's in the in the right direction. And when I build factories for our clients, it's more like okay, we need to figure out how we can make that more trustworthy and autonomous in the cloud, or everybody will just sit in front of it all the time. Sure. so maybe this is also a notion where this it's just a different situation, right? speaker-0: There's there's nothing wrong with sitting in front of it in a way that I I ex I explained that to when you were let's say traditionally developing, you you run things and you kind of you know have them on your laptop and then eventually you'll move it to your CI C D. So there's something that you do locally, maybe with a restricted environment, like simplified, and you can kind of do that and then you put it in the bigger, kind of more testable environment. the same thing with testing, there is a phase where you say run all these tests to exploratory testing. So I don't think personally that the factory will be per se the best way of iterating fast on a like an ID. So that I would typically I I I keep that local. like when we're Tesla, we have a factory and I I kind of like hate to do it there because the latency of getting a feedback is just too too insane. And and so that's why I like the rapid innovation. But I can on the other hand, if I have some kind of toss and I say, research this and it goes off, of course. And then I I look at the results, I don't care that much about having to wait for that whole piece. so it's making a different tools for different kind of parts of your your journey on this. But one is explorative and I I hope that's somewhere more of an emerging system of the the factories that we didn't have before. we see ourselves now delegating, like, hey, you know, come back and check the code, what's kind of causing this? Okay, a very inquisitive thing, right? But it's not like I need to do coding, but it needs to do some research and digging, right? Like me digging in will take a long time and I could just say, go ahead, come back with the result, and I probably can kind of analyze that. But There's no thing stopping us by saying, Well, give me four options. it's another typical LLM agent thing. so why aren't we building those research loops in our factories? Like our factory is still so focused on like take this back and go ahead. Like and and I hope there's something more. and this morning I had a discussion and somebody said, Well, I hate the word factory because that like kills the creativity in there. And then I came up with the idea, it's like What about it's a dream factory, right? it it it should allow you to dream about IDs that you could do. And that's also how you defend, I think, a lot more of the AOI. If it's something before the factory you would never be able to get to, there's not just a multiplier of 1.x. It's it's almost to infinity. Because I couldn't have done this. Like, there's no other way. And I I think those cases will amplify the value of factory even more than just making sure that we we're getting faster at testing, deploying and those spaces as well. speaker-1: point it's it's a little bit in the direction as well of outcome versus output, right? So you could get more output at the moment or y we are there maybe yet already, but in the end what we want is more outcome and and figuring out what what really creates outcomes is the exploratory part, right? And and the the dreaming part. speaker-0: Yeah. And that's also why I think there's a closer relationship now with the product owner, being able to use the factory, and bridging the gap between them having to build a prototype what was vibe coded and then they come to the factory that needs to be redone. I think there's there's now products trying to fuse the two worlds and going there. So that's almost like the the upstream that we're improving. when I was working on DevOps, I I al always said there's the pipeline that we know, the DevSA ops pipeline, but there's a sales pipeline, there's a GTM pipeline, there's legal pipeline. Like every there's everyone has intake, triage, how like do something, how like review, okay, loop back. Right. So and that's another opportunity I think with the factories that if you be able to start linking that because the bottleneck is moving from okay, we can produce more. now we have to come up with more things. Well, no, we shouldn't. But the whole thing is actually to say no to things instead of saying yes, which we couldn't do in the past. And then when we're producing that, the sales decks, the kind of all the the documentation isn't up to date. So all those pieces are new opportunities that we couldn't keep under control with a factory. But it's not just in code test and deploy. There's so much more we could put in a factory. And that's maybe the imagination, the first iteration needs to settle on what we have and then we can start adding to it. But I'm I'm very eagerly waiting until we we start adding more to it. Yeah. speaker-1: Super inspiring. the the question that immediately pops into my mind is something like that that will probably change a lot also on on the organizational level, right? Like how do organizations actually build like teams and and departments and so on and how do you do leadership? Do you have any idea there that what what do you think about that? speaker-0: So the well there's this whole notion of agile is dead, but it's not dead because it it's the same practices of fast feedback and you go for like, you know, the the thing that matter. the cycles of a scrum though, they for example, they they would change because instead of just talking about it, you could just do it and kind of give the results. Another thing that changes is you know, in the past somebody would raise a ticket and they They would have to wait for engineering capacity. It would be on a backlog. Now they could just start the factory okay, let's have a conversation. What is it? what is it failing? Just create the PR and do almost the work, and then the the engineer is reviewing. So there there is a kind of opportunity to get like to smaller teams where maybe eight was in the past, between you know kind of that level, get closer to Four or five in a in a good consistent way. but there's still the matter of understanding the code base, I feel, although AI helps with that as well. but there's a certain level of almost like latent knowledge that somebody in the team needs to have to to understand what good looks like, what is the decision of the the impact of decisions and so on. so it's almost like we're back to We like to fuse the product and the engineering role into one now because they could do it, but not everybody's a good product owner that is an engineer and the other way around. it's like a full stack, you full stack in everything. Maybe so we're even putting more scrutiny on that. But in general, I I kind of look for overlapping. So it's similar to the the DevOps, the T-shaped people kinda they overlap, but they kind of collaborate. So somebody with more product affinity from an engineer and somebody with engineering. Obviously that makes somewhere sense but because there's a a shared language to that. I think there's still if you start doing things at more scale, there's somewhere an attention span needed on operational sides. It's not just creating sites, but there's operational, although AI will help us there as well. So you know, I I'm now like with the the dev, the PM and the ops person. And then I would say you should almost at I wouldn't call it the trainee, but the person who's the backup somewhere. Because that is still a factor that like one person, they're human, they can leave, they're not available. Yes, there's a factory, but kind of that like almost like overseer of the domain. And so if you don't put that person in and could be a junior kind you know learning along the others, or it could be just a backup person that you say you put in there. So I I I personally I wouldn't go lower than four. you could go lower, but that's kind of forgetting the sustainable base, the risk of a person going away. It kind of like that that that feels like the under limit to me. speaker-1: Yeah, I I also think that there's often also this interface to the outside world. there's a lot of people coming from outside that want something from you. So someone also needs to buffer that somehow, right? That the team can can work productively. So lower than four is actually super small, yeah. speaker-0: But I I wouldn't go lower, but there's people like I'm a solo founder, and of course you can do it everything if you have and and the more ties you cut on like having to have communication, there's a certain piece of going faster, but you know, kind of there's a a buzz factor definitely in there and there's a limiting factor. But for your solo project, that might be fine. and and you put it out there. but it comes back again to like what's the risk, right? If if you don't respond in time. What's the risk for your customers? if systems kind of go wrong, what's your response time? And and that kind of always makes me uneasy when people say, Well, I'm not looking at the code anymore. And then I I dig in. Okay, you're not look looking at the code anymore. Well, I have my own verifiers. Okay, that's great. Okay. And then and then I ask them, what's the impact? Well, I'm a solo dev, I don't care. Of course, you can do it. Go for it. nobody cares. Big company, it's different, right? Like it's money, like impact. So that's that's why I say there's gradations of people and they they usually don't tell the full story of what they are doing and how they are dealing with kind of that risk from their perspective. So speaker-1: Yeah, and I feel it's also in bigger companies or in general really hard for a person to view them from outside on themselves. Like what am I actually doing day to day? Also for the engineers, even? Because I mean this is something that you would need to do to build such a factory with low risk, right? You would need to analyze yourself what do we do? Then take that and build I I feel there's also like L maybe something like a new role emerging here or or or our current roles are changing a little bit into some engineers are able to to do that, to view on top of the so you know what I mean, right? To reflect more on the process, how they actually engineer stuff. speaker-0: It's system thinkers, right? that's that's what you're looking at. And probing the system that failed. when the pipeline of CI C D you know in the around DevOps kind of settled, there was the emerging role of the chaos engineer, which was basically probing at the system whether it was still working and kind of do hypothesis and kind of you know, almost like the fire drills. That That will likely be there as well in the future as this progresses. you know, we need to have the safety checks in order. We need to make sure that like the backups are there. so I don't see that going away. But obviously you can like say like the only thing I need to do is give a spec and it's taken care of. but that's yeah, usually people who haven't done like the maybe the longer running ops kind of like things will fail. and that kind of you have to deal with that. My opinion. speaker-1: In my experience, there's also a lot of engineering skill involved here, right? So you to build these these state machines, to build these factories, and to be able to system think on what you are doing there, you need to have all of the engineering knowledge. You can't simply just say, well, the LLM will do it somehow, right? So speaker-0: Either they're proficient in kind of more of the agentic engineering and they're heading towards factory and learning basically what like you know, that factory and the loops and and and sandboxes and you know cost and all the things that you know would typically go into a platform, or it's the other way around. Like people have been used to running a platform and they say, well, we could like bring in the agents, we can bring in kind of those pieces as well. Now there is a broader spread of knowledge, and as you mentioned about like things changing into an organization. I feel it's not that clear cut a split, but roughly I would say there's gonna be people working and making the factory work and make keep that humming. And there's people who will likely do more spec technical requirements and going in there. So there's like a like the nice thing is that the factory as such is a little bit of a escape for the more technical people that wouldn't thrive well into just writing specs. Okay. but there is also, like you say, maybe it requires some experience. It's not about age, but more about like, okay, how much of the meta pieces that you can plug into there you see the effect if I change, you know, a prompt here, what will happen there? Or what are the risks on my sandbox? Or how's the cost going up? So you you got a more holistic view on the whole thing, to think about that. So but yeah, it's it's I think there's a logical platform piece to it, but it's uncertain whether it'll be more taken from the AI engineer side or more from the platform side. I'm still, you know, the jury's out I I think in the industry there. So speaker-1: Yeah, it really reminds me of the early cloud days where it was not really clear yet where the right abstraction layer would be and and how to actually w because with these AI platforms there's it's it's more than an infrastructure, you know? It's like there there's a lot of process or or rules and so on. Also like d is the rules already part of the in of the platform or is it like something you put out from from outside and so on? I feel we d we don't really have an answer to that yet, right? So speaker-0: No, but yeah, the platform is never just the machinery, right? So it's it's always like going into what is your business process, how do you want to deploy, what's the risk, what so it's it's a z very similar discussion if you onboard new projects onto a platform. But it depends. some people just say, I built the platform, I don't care who runs on top of that. But the whole narrative of running your platform as a product is listening to your customers what they need, and that's basically going in and listening. What do you need? how do we need to adapt? What what's useful? What's important for you? and bring them onto that factory as well. So speaker-1: Yeah, interesting. I I also really like the idea of splitting these roles a little bit into people that are providing or building on the factory as as more being the platform team, maybe, right? Like some something like that. We we don't really know it yet, right? But it could could be. And then the people that are maybe more also doing the explor exploratory work and and the creative work and and more to the product side and And then there needs to be this this handover and and both have probably also a different mental model of what they are actually currently you mentioned this latent space. I don't know. Like for me, like both groups would probably have their own mental model of what they are building, which is both very valuable, you know? But I think speaker-0: There's well, you know, in platfor speak, you would say there's the feature teams and the platform team. Now I I you don't know yet whether that centralization is something that we'll end up with, or in the end it will be a very configurable Saa solution that everybody builds their own pipeline. But it's a little bit like when as a platform team you put a CI C D flow forward, it rarely stayed vanilla within a team. They all adapted it into some form or some flavor of, well, I want to run this linter. No, you wanna and so it needs to be configurable, but the machinery maybe, you know, we're not all re-implementing prompt injection kind of detection or something, right? Or not a registry or kind of like an inference gateway. Like no, nobody's but we all wanted it to be adapted to our workflow to go for that. So it's a little bit like we're The GitUps thing came in, we define it, but the platform is actually run centrally, with with those pieces as well. speaker-1: as as the as the person that would do the more explanatory stuff, I also don't wanna need to know all of that, right? I would just want to have it okay, I can use inference and it's safe. Check. Correct. And yeah. Someone else will do the all the stuff that it's safe for me to use it. speaker-0: Well there's there's no complete abstraction. I'm gonna give the example. Let's say you go to a newer model in your factory. Okay, the impact of me writing prompts and me writing pieces, so that's something that still happens. But it's the same thing as I upgrade my version of whatever versal or to a new version I I need to do upgrade. So dependencies and upgrades are still gonna happen, even in factories. but you know. They we might get a pull, hey, we did this, we run your test with this is proactively for you. but then on a major upgrade, like what do we do? Like we we all kind of like the nagging on security and we completely dislike it, right? So it's like probably the other way around, but yeah. speaker-1: Yeah, probably. I mean your your p your factory is also a product, right? So you would need to, as you say, you would need to have like a dependency updates for the the parts of the factory in there as well. And then yeah, if the model changes, this is going to be a hard change, right? Like th I think this is like part of the problem why we don't really know this abstraction layer yet. Because if the model changes, then a lot of your non deterministic stuff changes immediately because you need to ref reconfigure it. I don't know, a little bit like reverse engineering, how is the model behaving in certain scenarios and then getting rules out and new rules in and so on, depending on which model you you choose, right? It's like speaker-0: Yeah. And usually there's a migration period of hey, central team is preparing the upgrade and they take the usual use cases and they test it and they see what the pattern is. they try to extrapolate that. They turn that into some tools. Maybe, for the model, there's some evals you can run and see what the model gives. but it's still a lot of engineering, on both sides that to deal with the kind of the complexity. So the power is great, but what a lot of people forget is yeah, there's there's work still to do when when that happens. And the impact is bigger on those flows. reducing things on a prompt. it's a lot easy to add things to a memory and add things to a prompt, but reducing things like everyone's like, what what am I gonna remove? Like, well, let me check what the model already knows. Like what can it do? And so kind of those yeah, very interesting questions on and I I think I was Chatting with you a person from Anthropic who's dealing every time there's a new model, they have to go to customers and they have to say, Hey, beware. Yeah, do something with it. And it's not gonna be different in a factory, but it's a major cloud upgrade, it's the same thing. So it's kinda like speaker-1: To get that in a little bit di different direction, but to to add up to something that you mentioned earlier, the the the adding the junior into the organization, I really like the idea of of this backup person being a junior, or like or someone that would like a trainee who would is not there yet, maybe, or like because this is like a huge discussion at the moment, like how do you deal with I mean we mentioned this factory builders are probably some s system thinkers, so you need to become a system thinker somehow, right? this usually it's not age, as you said, but it's more like yeah, you need to have experience for that. I don't know. there's this huge discussion how how what will happen with this now? How do we do this? How do we do this? What's your idea about that? speaker-0: It's a common question. and goes in both directions. seniors would often tell me, well, the juniors will not have an opportunity to learn. Right? So and then the juniors tell me we can learn faster than anybody else, big thanks to AI. So it goes both ways. speaker-1: Okay. speaker-0: but it's kind of the exposure, some guidance, some lessons. So that's why I'm saying age is not per se kind of the case, but bringing them on to the journey. kinda makes sense and and it's not just that they learn about the technicalities of the system but also on the business, right? So that's why I'm saying you could be senior in engineering, but still junior in the domain or the problem that you're trying to solve there as well. So it wasn't like per se an illusion, but I I do think in general, the the jury's out. the on the one hand, people always say there's a risk of automation reducing the number of people. Well I've seen it now, cloud, DevOps. Every time initially there's that discussion. Like something gets automated and it gets reduced. And my my peril would be if you would think about photography. So there were less professional photographers on the one hand, but there was an explosion of YouTube people, all right. So where does the complexity reduce the number of people and what new kind of roles does it bring? and kind of maybe there's something in the exploratory phase or something in kind of the the engineering phase. So far the waves that I've seen is that yes, there's a reduction, but then there's a whole new world of new roles and That's maybe where the juniors then eventually drive in. and there's also a certain level of reduction of complexity. early days, CI C D things were very complex and messy and you had to know what you were doing. And then it molded into that almost like very ac accessible way of doing and kind of defining things. And it was easy to get somebody on board. on that new flow and then they experience the new flow and then they could learn about the flow. So it's a different way of of dealing with that complexity. speaker-1: Yeah, and and and and you also CI C D is a nice example, I I think. I I'm a huge fan of the GitLab CI C D. I I think it's really brought us on the next level. And if people are now starting to use it, they don't need to have the experience from the journey it took to get there to just now understand and use it, right? And I was w one of the early person to jump on Node.js. And I feel if you compare the early days of Node.js to where we are now, there's also this yeah, you can use it now without needing to know how we got there because it was super messy. And I think I mean Node is still super messy in a lot of places, right? speaker-0: There there's a privilege about witnessing a new field emerging. that you say, that was explored, that went that direction. but at a certain moment that becomes useless and not useless, right? Like useless in a way that you don't need a day to day. But when there is a certain thing, like why was the decision taking? It's like, yeah, because that. Like, you know, because of that. And and that's useful, but I'd I also understand people saying, well, we're gonna skip until this matures, and then we're gonna jump in and they'll have a different kind of a learning experience to that. I guess we're both probably more hooked on the new things and understanding the new things. So but it's a probably a personality thing or like why we're here now in the space and in this podcast. So speaker-1: Yeah, my take is also I as you say, from a personality standpoint, I really like to do that. But also I feel that that it it's also very beneficial to already do these things now than to wait, right? Because I I mean it wasn't s that clear in the past, like a little bit clearer w but like with CI C D pipelines and so on, it was still a huge amount of work to do to gain a little advantage. But now with AI I feel that even the little stuff that you can do alre now already gives you a huge advantage compared to persons that are waiting and not not doing it at all, right? speaker-0: Yeah, that makes sense. it's you put have to put in the work. It's one of those how do we get started? It's a question I often get and it's like, yeah, just pick a project, start doing it. Like, you know, there's no you can read books, but it will go p completely past you if you don't have skin in the game, which like I tell them pick a project that you care about, like something you want to improve yourself and just dig in. is it the most efficient way? There's different ways of learning if If it's well laid out, you could probably absorb a lot more information than you stumbling into the field. But there's still a lot of waste of having to watch all the videos, all the blog posts, right? Like something will settle eventually and you you kind of can forget all the rest. But it's I think it's a little bit like the craftsman, right? Like a craftsman is about having a lot of tools on his belt that kind of knows what when to pick the right one and It doesn't mean you can't get away with a few tools, if if that is the thing you do. but that's what you pick along the the journey. And I've had it many times that I go into these rabbit holes and th they don't lead to anything, but five years later it's like, whoa speaker-1: Yeah, what do you mean? speaker-0: something like that. speaker-1: I mean to to wrap this up a little bit also in the direction of of risk again, is like or like how do how how does company how do companies now actually like implement factories in their day-to-day work and the learning of I I feel that it's often not really a a technical learning thing. It's more like the organization around it needs to change and adopt like it like when we did introduce agile into these companies. This is not really about learning In the sense that I learn what Agile is, it's it's a very short book. So it's not a lot n not a lot to learn in in terms of like theory. And there's a lot to learn in terms of like real experience and real doing the this the work. So what is what is your take on that in in terms of like f how do you do that as for the factory now? How do you how should companies start this? speaker-0: So as I started seeing the difference between people working on their agent locally on their laptop and they kind of tune it, they add their own stuff and their own flavor to it. Then they started checking things like shared context and their harness into more of the repo. And then that was ext almost exposed through the factory to other people as well. So that kind of exposure was probably the biggest impact. yes, there's going to from solo to that team was useful in a way that, we now stick all to the same standards and there was a kind of alignment going on. the fact that you can ask tickets now to a factory and say, please do this, please give comments, please help me here, that kind of on the one hand opened it up so there was like became democratized in the in the in the company of doing a change. so that was definitely a change. And then the fact that it could reuse all the knowledge that was there because we're adding to the factory and making it more robust, that means there's a value of not having to reinvent things. And I recently did a talk, and I'd say there there's there's all the metrics of deployment still make sense. Okay. Robust and it needs to go out and tests and escape rates and All is valid and delivery, the speed delivery, whatever you value in there. But I think the number of human touches going down in processes is definitely one because that creates that efficiency. Like we're not having to have all the people be involved. And the the second one is how much of this is being reused by other people. So if you expose the factory as a service and everybody can use it the same knowledge. Then whatever work you did in the factory is not just getting a multiplier on your laptop of I got X better, but the whole org got better at doing this. And then I think those are like two metrics like the reduce the number of t human touch points if you can, with with the right risk and all kind of you know, the things in it. So don't you do it YOLO. But then kind of how much is it reused across the business? That's my second metric. to to think about it. And I think factory and the interface, whether that's a chat interface or it's looking at our tickets or something, that embodies those two values or metrics, I must say, to kind of get get improvements there. speaker-1: the way you describe it, it it also sounds to me like it's a very structured and deliberate process of doing that and that a factory also helps in in that. Because I I see a lot of companies that are like everybody is just using it on their laptops, super chaotic. Everybody's just, I don't know, has their own setups and I need more tokens. I need more tokens. I and and so on. But that that seems that that it needs like a like a program around it that says, Okay, we wanna build the factory, we wanna get there and what what what do you think? speaker-0: There's stories that talk about what almost like what I would call death factories. Indeed, they run on their laptops and they got better and the harnesses are, taking care of that. But when I usually would look for the factories are the ones that are being in the c not not just in the cloud, because you could put your developer thing also in the cloud, but the fact that it's being centralized and being reused across. And I I think that's what the Ubers, the Rams, so kind of all these centralized engineering groups are doing in their companies. and I think that's where the benefits get multiplied. I'm not saying a dev factory is bad or anything, but like it's it's not like it will get you up to a level, but if you're can't like your colleague can't use it, the impact is limited. So that's why I say like it's from solo to shared to multiplayer is sometime a phrase that you hear. And that multiplayer is what a factory exposed as a self servicing to the whole org could promise way more than having it solo in your your your on your environment. and I even started to think about like a lot of people talk about I can install skills and I started to think like, aren't we deploying skills? Like is shouldn't we kinda like make them centrally and then reusable and we're doing that in a way because our our our coding agents kinda like allow us to pick that up centrally. But it's almost like I I want to centralize and make it available in that sort of servicing mode that everybody could use my skill you know instantly with the right credentials, with the right like environment, with the right things that are hooked up and going from there. So speaker-1: It's interesting it's interesting to see how the the the field still emerges, right? And now we have this idea of plugins and and maybe there's going to become like more of an an installer for this and so we we haven't really figured it out yet, right? So we speaker-0: But it's look it's thinking in patterns, right? Okay, we got skills, they became plugins. Maybe the next thing is that we'll have loops that you can install in your factory, which seems to be the case. a lot of people talk about loops you installed, your code review is a loop, your your spec to code is a loop. So that kind of will become its new entity that you work on, and maybe then again it becomes like interchangeable. Maybe we'll have to think about interchangeable factory configs. You know, eventually we'll get there. compatibility one factory to the other. So you you can like it's it's extrapolating. We're not there yet, but that's co that's that's almost the cool stuff that you can start thinking about. So speaker-1: So as we as we near near the end of our time, I have I have one last question. so what would you advise people to do now? Like if they h hear the episode, I don't know, in in a few weeks, what what is the thing that that you you think should people should think about or should start with? I know it's a hard one. speaker-0: Yeah, it definitely depends on your maturity, of course. But if you're just starting, just start playing with it. you the more experienced you can do it, then do it with, like I said, with something that you care about. And if you get more experienced, think about how I would advise like please think beyond kind of just the coding loop and go into the company. And I often say that senior engineers they don't just are about like Building better code. They go into the business, they think about architecture, they think about cost, they they think about other values as well. So start exploring those and maybe in that hand is like figure out where do you wanna be? You wanna be more in the platform team, you wanna be more in the exploratory spec team and try shaping your career there. of course there's gonna be new acronyms, forward deployed engineer, AI product engineer, and we're we're gonna have a name for all of them. but choose your own path and yeah, learn more, I would say. That's the motto right now. speaker-1: Thank you so much, Patrick. I really enjoyed this episode. I think we had a lot of insights here. hope you we see you again. I I think you are at c are you at Code Talks this year or I I I'm not sure if you were speaker-0: Unfortunately I'm not. I have I am content creating another conference. speaker-1: okay, okay. speaker-0: AI DEF CON in New York speaker-1: wow. speaker-0: to four November and I'm curating the program and the theme is factories, so hey, yeah, you might as well come. speaker-1: May maybe I'm going to go there. I was never in New York, so Yeah, let's do a switch. That would be awesome. Thank you so much. and yeah, have a great time and see you see you again. speaker-0: Thanks for having me. Really appreciate it. speaker-1: Thank you so much for listening. That was another episode of Rethink Engineering. Patrick, thank you as well. I really enjoyed having you here on the show. I hope I'm getting better at this. Let's let's see what the comments say. it was a pleasure taking the factory apart. if you want to learn more about this, stay with us, follow us, or join Patrick in Brooklyn at the AIDEF CON. fourth no second to fourth of November. You will learn a lot more about software factories there. He also curates AI native DevIO, so hop over there. You will find the links in the description. And now it has finally come to the point where we are going to have an episode with DJ. I'm really glad that he's joining us. And we are going to find out what he's currently doing, if he's going to join us more often. And yeah, I think That's it. That was Rethink Engineering with Hackers and Wizards and Rethink. See you next time. Bye bye. speaker-0: Mm.