Latest / Beyond Vibe Coding / #S2.09 Automating the Grind: Freeing Up Human Attention - Pedro Gil Carvalho (Thermondo)
Transcript
- Pedro Carvalho: If you're if you have just the one unified system anyway, then you don't need any of these tools and you can be more productive. So that's the theory. And in two months, I might come to you guys and say, Great news. Or or Sebastian: Please do that. Yeah. Pedro Carvalho: I'd be saying, you know what? Forget it. I was full of shit. This didn't Sebastian: Yeah. Pedro Carvalho: work. I tried, it didn't Sebastian: Welcome to the show, Pedro. We're super happy to have you. And it's amazingly interesting also what we discussed in the pre-discussion. I mean, you're a tech company that is not just digital but also has a physical footprint and has processes around that, right? So we're probably going into this a little bit later in the show. But before we start with our usual sections, please introduce yourselves briefly to our listeners. Pedro Carvalho: Yeah, happy to be here. Thanks for having me today. I'm Pedro. I'm a VP of engineering at Termondo. I've been in the Berlin Tech scene for over 10 years now. I'm originally from Portugal. I live here with my wife and two daughters. and I worked for a few scale ups. you might be familiar with some of them. I was head of engineering at HelloFresh before the IPO. I worked at Contentful, a pretty pretty well known API. IPI first CMS and I also was director of engineering at GitHuur. so I have a little bit of experience with you know kind of scale ups that work worldwide and I also worked for another you know a couple of smaller companies, startups that maybe didn't go as far. So I got quite a bit of experience of the small and the large and I'm trying to bring that experience to Termondo. Our ambitions are to decarbonize every home and beyond. we're currently amongst the top providers in Germany, and the ambition obviously is to go European and to expand our footprint and to decarbonize more and more. I think we're in the middle of a heat wave. I think everybody can see how important that work is. yeah, that's me in a nutshell. Sebastian: Awesome. Thanks a lot. Yeah. And the good thing is also, you're mainly I think in the business of heat pumps, right? And heat pumps also help with cooling during heat wave. So as a double André Neubauer: True. True. Sebastian: double benefit. Yeah, exactly. Pedro Carvalho: Two in one. Yeah. Sebastian: Two in one. Yeah. Yeah. Awesome. thanks a lot. And like usual, we want to start also this episode with the status quo. So how are you working currently how you are utilizing AI in your maybe coding but also in your managerial Pedro Carvalho: Yeah, well, as we speak, there's a cloud code session that is keeping an eye on all my Slack, confluence, email, and a few more data sources. And once I'm done, I will have a briefing. it also sends me briefing every morning about things that I want to know more about. It does deep dives into things that happen at the company and at the industry. so that's just this passive way to kind of like be a sort of chief of staff for me. it also helps prioritize my work. The prioritization part I haven't been doing for a very long time. I just started like a few weeks ago. and I'm surprised by actually how decent it is. turns out a lot of this is not rocket science. and it really helps me to you know keep things in mind. I also use it for coding. I don't ship code as much in production at Tomondo. I find that if I start to do work myself like that, I My mind just focuses on that and I forget how to be a VP of engineering for a second. and I also kind of get in the way and I prevent others from learning. So I try not to ship production code at work, but I use it a lot to explore code bases, try to understand maybe technical issues that we're looking at, and just having a more grounding to my opinions when I talk to my heads of engineering and my staff engineers. it really helps to have informed conversations. And we all do the same thing, my management layer and me. I also use it for research, you know, like whenever I'm trying to understand something in the industry or some changes that are happening, it's really useful to just send a couple of agents out and then you know, five minutes later you have a nice document that you can consume. and yeah, I also have a few other agents that are to that I coded myself, and that's a way to learn how to, you know, code AI. And they do slightly more proactive things, like they're in my email inbox and they're you know drafting messages and and and those kinds of things. And lastly, I have a massive knowledge base. You've probably heard of this LLM wiki concept, which came out a few months ago. So it's basically that. I was doing that for a while, and when Karpathi came out with it, it helped me to improve my my own format, which was nice, which is basically a massive obsidian vault. which is you know structured and indexed a certain way and you know everything that I do everything every every conversation every brainstorm every project it just gets processed and stored there for me to have you know available it really helps with interviews with planning with with all these things so those are like the main ways in which I personally use AI every day I think the risk sometimes is that you can spend too much time just chatting with Claude. Instead of making progress, that's the trap. But when you can keep yourself focused, it's just really good. I feel like my awareness is much bigger than it was before that. Sebastian: Yeah, awesome. Thanks a lot. I'm I'm also like constantly working on my knowledge base. So I try to utilize something like an LLM wiki approach. now the latest hype is OKF by Google, right? Also wanted to look into this and but I still on the other hand have only recently managed to set up obsidian working on my like I I use it on on Cloudbot like on on the host, right? And have a headless obsidian there with self-hosted sync plugin and then have it sync with my computers. The app on the smartphone is not yet working, but I'm but I'm getting there. So it's it's also a lot of time that I need to spend there. But anyway it already helps me also in in my day-to-day work definitely. Yes. André Neubauer: Are using AI mainly for productivity or is it also experimenting? What would you say? Pedro Carvalho: Me personally both. I yeah, I I mean André Neubauer: Yes. Pedro Carvalho: I've been building I've been building small agents since like November last year to try to understand what they're good for. And these agents are like kind of custom builds that I that I that I make myself because I really want to understand how every step of the agentic loop works. And it's every time a new model comes out, I just plug that model into those agents and I see how they behave with a new model. I see how they behave with different kinds of system prompts and instructions and tool calling. And that really helps me to have a feel for what's possible and what's hype, right? And I think that it's no coincidence that I've been doing this since November, because at the end of that month is when the this kind of wave of LLMs came out, which were kind of subtly but meaningfully better. Right. The Gemini 3s, the the Opus 4.5s. that's when that's when you could have like these like long running agents that kind of kept track of what they were doing and didn't go off on wild tangents. so I do I do that a lot and I I still do that every day and and I throw out these these agents and I build new ones to to try out different SDKs and tools and different programming languages. I I find that that that really helps ground my opinions with my AI team at work and also to engage with my you know C levels in terms of you know being realistic with what's possible but also being enthusiastic about what's possible that people don't really know about. Right, so trying to try to try to navigate that from more information that's more grounded. I do that a lot. Sebastian: So you basically have have a small eval suite basically to e evaluate the the new capabilities of of new LLMs. Pedro Carvalho: Yeah. Yeah, I th I think you're being generous by calling it an eval suite. That implies way more structure than it has. It's more like a crazy scientist's desk with all kinds of bits and pops on top of it. But but yeah, I I use it to evaluate how they work, yes. I also do eval loops, of course, but it's a bit more chaotic than that. Sebastian: But but honestly it it rhymes with I just yesterday read an article by Steve Yege or an essay, like he writes these long essays, right? But always Pedro Carvalho: yeah. Yeah. Sebastian: like very very interesting stuff. And he like the latest one is called the Flat Curve Society, where he also makes the point that certain people didn't see a difference between Opus four point whatever and Fable Five, while he saw a massive difference. And the reason was he did have these like these projects in his back pocket where that he could pull out that other recent or older LLMs failed with but Fable 5 would ace basically that's how he pretty much tests the newest generations of LLMs and I have to admit that I was also guilty of that. So I I used it like I used the other models before and I didn't see a big difference. but I There d so the approach he mentioned there and also some other content is really, really nice. Yeah. Pedro Carvalho: I mean that that resonates. Yeah. Sorry, go ahead. André Neubauer: Yeah, they put... Sorry, sorry, sorry, Pedro Con. Pedro Carvalho: I was just gonna say that's kind of why in November my my my AI experimentation also took off. because I had this use case in my back pocket, which wasn't just running so well. And then when the new models came out, suddenly that that that that became possible. And then the thing that I've been exploring since then is is is actually very simple. I I fundamentally think that the whole concept of like prompts and skills is wrong in a way because it's very one-shot. It's like, hey, can this agent do this one job? And can this agent, you know, kind of loop until it gets there? but it's very limiting because it's very the way people do it tends to be really prescriptive, right? It's these like really specific do this, don't do that, when this happens, do that other thing. And what I was trying to see is. Can I give an agent more like a job description? Like, hey, your job is to onboard a customer and make sure that they, you know, do these things in the product, right? You're like a you're like a CSM for onboarding of customers in a hypothetical SaaS product. and you receive signals, you receive events that are happening in the tool. Somebody signed up, they did this, they did that. and and and you can at any point. Decide what to do. You have your mission, you have your tools, you have your signals. Go figure that out. It it just didn't work last year. And since, you know, especially since Opus four five came out, suddenly that started looking like it worked. And then it started working better and better. So I have these experimental loops at home where I have like a fictional SAS tool that I that I built just to be like the guinea pig. And you know, I will plug the like like webhooks from that thing to another piece of code that I wrote and it just events just go in there. And it just reacts to things and takes actions and it kind of looks like it's just a person doing a job, which is pretty crazy. You obviously the job has to be narrow. and it it doesn't get it right all the time. But you know who else doesn't get it right all the time? Sebastian: Us humans Pedro Carvalho: So it's yeah, so so so that's why I saw that big leap. And I can't say that for that use case I saw such a big leap when Fable came came out. I think Fable's better at other things. but but yeah, that's kind of yeah. I I think I think there's more coming from that topic, I think, as before the year is over. That's my prediction. But we can talk about that later. Sebastian: To be honest, the setup that you mentioned sounds like a pretty sophisticated Evel setup. Yes, you might not have like the typical evals that where you have a clear categorization of answers, and then you automatically would categorize if it's a pass or not. But for what you are utilizing LLMs for, this is probably the much better Evel compared to a standardized test. so it it it sounds pretty sophisticated already. But I think in the interest of time, we should move over to the main segment of our episode, right? The meat, which is talking about specifically how you utilize obviously agentic engineering at the Mondo, and then also, yeah, how you personally view the AIs helping bring different functions together. But maybe let's start with with the yeah, way you are utilizing agentic engineering at Pedro Carvalho: Yeah, happy to talk about that. I think it's pretty standard. We we have you know, it started by giving people access to tools. in the beginning, we we didn't token max, but we definitely, you know, congratulated the biggest spenders in the beginning. And that was just a way to like drive adoption. it was always gonna be unsustainable. but yeah, everybody uses cloud code. Some people use like open code with you know different models directly from the API. we also allow people to have like a secondary tool if they want. So a lot of people have code and then codecs. and the individuals use this in different ways. Like we don't mandate like a workflow. We obviously have our soft development lifecycle that everybody abides by, but you can use AI in your own way. There's no mand mandates to use it one way or another. But we do have just almost a religious schedule of knowledge sharing sessions, where every every week, unless there's something going on in the company, like a lot of vacations or you know, some big announcement or something, unless something like that happens, every week we do a knowledge sharing session. And two people come on, they demo or they present and they talk about what they discovered, what they're doing. And this is a really good way to share how we work. And it's actually gotten popular with PMs and people from different business units of the company just just wanted to come and see what the engineers are doing and what the PMs are doing. yeah, it just became way more popular than I expected. There's there there's a lot of people who that watch it every day, every week. We recorded videos, we share them later. We also build and share our skills. I think everybody does that. we recommend skills to each other. We build our own, we share them. so those are like very kind of standard ways in which we we use we use AI. one really good benefit of this has been with a design team, actually. Like the design team has been able to almost fully transition away from Figma and more towards like components that they just make available, they have skills which know how to use those components. And so an engineer can very easily or PM can very easily put together A UI that looks and feels much like what our designers would have produced anyway. And then when that prototype is ready to go to or or looks like it's actually going to production, then the the designers can come in and make it a bit more, you know, intentional and everything. but it really accelerates that you know design to engineering loop. PMs are going, you know, they're going nuts with AI. So now they can bring their ideas to life, they can make prototypes, the prototypes actually work, which is the great thing. They can actually take those prototypes to users, have them use the thing, collect information and feedback, and then bring it to engineering to you know, build in a way that's more, you know, secure and production ready and scalable and all those good things. And we try to empower the PMs with as much as possible like skills that tell them or don't tell them, tell their agents how to build it right from the get-go so that the The prototype gets as close to production as possible. That scares, I think, some people, not necessarily at Termondo, but that idea that, a PM can do my job now, a designer can do my job now. What I like to remind people is this is great. We should be doing more of this. Let's allow more people to use software. Because you know, if if a person who isn't a programmer can build a production app with AI, what can you do? You know, you who are a professional software engineer, you can do even more than that, right? So your value is always, I think, is always assured assured. So let's actually focus on empowering as many people as possible. The more software there is, the more there is for us to do as well. So these would be like some ways in which we we use AI at at Termondo as a as a as a as a tech Sebastian: that's pretty interesting. And I would like to double click off some on on some of the points that you mentioned. Number one, PMs are going nuts with AI and building prototypes and then they're bringing it to users and collect feedback. How like could you give an example? Because I assume that it's not like productive use, but rather something they use in order to see how their users react. So it's more like an experiment setup or how are you doing it? Pedro Carvalho: It's it's it's like a an experiment on steroids because you're not just bringing wireframes to to users and and have them click around and tell you what they think. you're actually y bringing things that are in some cases actually connected to production systems and and these and these things have real side effects. so somebody's actually doing a meaningful piece of work with an AI prototype. Which is I mean i i i it it is way easier to do today because now we have this kind of platform underneath everything that you can just plug into like it's an API, right? We built we we didn't have this a year ago and now we have this Tomondo API that makes it just really easy to build whatever software you want on top. And it kind of imposes rules so that you can't just go, you know, and and And and creates changes that you don't want to create. So that gives a degree of safety. that but it's still obviously not 100%. So these experiments are still, you know, controlled and they're just done with a few selected users who are maybe a little bit more technically inclined and so they can do things more safely. Obviously, for us, security is a huge concern at all times, and we want to push the envelope while not making any any any obvious mistakes. Sebastian: Yes. So the the safeguards pretty much that we've already also discussed in other episodes with specifically fintech companies help a lot. So if you have a proper setup, and in your case it's also super serious because there's actual hardware being involved, right? So you you need to have safeguards and not in in order to not do deer serious damage. Sorry, then you can experiment easily, right? Then you pretty much use it if I understand it correctly to validate ideas, right, that later on then are being productionized by the engineers to be built into product proper product features, right? Pedro Carvalho: Yeah. Correct. I think that I think that using AI does not mean that you have to be, you know, shy and it also doesn't mean that you have to accept, you know, security compromises, but it does mean that you have to move deliberately. Yeah. And with and with full awareness of what you're doing and what's happening around you. Yeah. I think that's its whole topic. We so one thing about that that I wanna say that I think might be relevant. we definitely had situations when somebody did something with AI that maybe they shouldn't have done and was never serious, but that's we take it really We take these things to heart and even though there was never like a proper security incident, there was a situation of somebody accessing some data that they shouldn't have put online and they didn't, but there was a risk that they might have done if if if if the safeguards had clicked into place. And we we take these as as learning opportunities. So we don't, you know, blame people. We just admit that whenever anything goes off track, it's Because the system didn't, you know, protect people. And we just take these as learning opportunities. And typically somebody that makes a little whoopsie with AI tends to be way more careful going forward. And you actually want to make sure that person feels safe. because that was a lesson that was potentially expensive, and you don't want to lose the person who made that lesson. So we make sure that hey, it's okay to experiment, it's okay to push the boundaries, let's just not make any like negligent. Mistakes, right? As long as everybody's doing the best that they can, that's all we can ask for, and we just constantly learn and get better at at this together. Sebastian: And you have the safeguards to avoid serious damage, right? Pedro Carvalho: Yeah, of course. That is super important. I have some pretty safety minded people in my team that keep us all honest, which is important. André Neubauer: Yeah, I think on the other side, think this having a certain failure culture, I think is super beneficial these days because it's changing a lot. What I was wondering when you explained your AI transition is, is there something like a proper project in place to transition the company to something new? Because it's like it's pretty massive or are you just following the flow? So you have a specific plan in mind to change Pedro Carvalho: Yeah. André Neubauer: towards agentic engineering, for example, for engineering or how do you approach that transition? Pedro Carvalho: Yeah, I I wish that we were just going with the flow. I think it would be a lot less stressful. Just see what happens and adapts to it. there's definitely an intention, and I think that our intention is Going towards the Gentic, of course, we we as a as a as a team, we we want to kind of save ourselves as much time as possible so that we can focus on the things that matter more. That's the mindset that we go into this with. the way to get there is where you have to be a little bit reactive because you you you can't just declare, okay, today it's a dark factory for software, right? No no one looks at any code anymore. Just tickets come in, agents build things, and we just focus on the design. so we have to kind of, you know, whenever a new model comes in, we have to see what it actually can do, how far you can push it. But I I have to say that my my my target state is Probably not going to be one where everybody just web codes everything. I think that engineers are absolutely necessary. And so our target state is not one where we don't code anymore. It's one where we trend closer and closer to wherever it is that our attention should really be. Right. and we go back André Neubauer: Okay. Pedro Carvalho: and forth on this. Yeah. We go back and forth. Like Sebastian: So Pedro Carvalho: so some sorry, just just just to say, sometimes you feel like you make huge progress and that software suddenly can become almost autonomous and then you pick up a different piece of work and it turns out AI is just not as good at that thing and you have to kind of rethink the whole the whole approach. So I think this adaptability right now is really important. Sebastian: A hundred percent. I I couldn't agree more. and a question or maybe two questions in one. how would you say just roughly percentage wise how much code is currently generated by AI in your organization and and what's the target state? So it doesn't seem to be a hundred, but it seems to be like as I'm totally nearing the one hundred or something. Pedro Carvalho: That really depends. So unattended code production, I think, is zero. we well, not zero. Unattended code production is I don't have hard numbers for this. Let's call it five percent. what it is is we have some agents that are constantly looking for like sentry alerts and they're looking for root causes by basically, you know, diffing. Changes on GitHub and finding out what is potentially the thing that broke, writing a pull request, which is not groundbreaking. A lot of people are doing that. So, so that tends that's there. It's just not the main part of it. We also have a quality culture, so we don't really have that many like incidents or or stuff that needs that that kind of agent. but code that went through AI. 80-90 percent? I think André Neubauer: Yeah. Pedro Carvalho: the difference the difference is it it's it it all gets reviewed at this point. we struggle with code reviews. We are aware of some tactics that you could use to lower your burden there. but we we've decided that for the time being, we still want a human to have a look at the code before it goes to production. That's not a dogmatic position. It's about accountability, I think, in my in my in my perspective. in terms of target state, I don't think that an engineer needs to write every line that they that they that they ship. I don't think an engineer necessarily even has to see every line that they ship. But the engineer is accountable for every line that they ship. And that's that's where I try to focus my attention. we're currently having These discussions also also the country. Like what does it mean for us to do code review? Why do we do it? Do we do it because we want the responsibility and the accountability to be shared? Or do we do it because we genuinely just want to improve? These are very different motivations. We have to have a motivation where code reviews are for improvement and not for diluting accountability. I want a situation where a person in my team target state, a person in my team can have a team of agents just build all the code for them and they're still. going to sleep at night, they're still happy. They still feel safe. They still feel like their work is under control. They know what it is and they know how to fix things if they break. I think the way to get there is going to be a hell of a lot more tooling around around code production and landing. Sebastian: That's a that's a super interesting question. Have you been looking into tooling around code reviews specifically? and how you could automate this this process? Pedro Carvalho: Yeah, for sure. we we have some experience obviously with with these kinds of you know automations. There's a bunch of tooling that I I don't think it's it's necessarily worth mentioning many, but I I would say that the more important ones are the ones that kind of automate security checks, like things that actively like try to break your code and try to break in. and also just these more like generally code quality kind of kind of tools that look for, you know. Complexity, design, those kinds of things. one keeps you safe or makes it less likely that something slips that might bite you, and the other one helps you keep the code maintainable. I think that with AI, one challenge is it it knows how to make software that works. Because all the training data and and data annotation that it does is about, you know, having software which passes deterministic tests. I don't think that this training process for for models really has a function to reward quality, which is also really difficult for us humans to judge. But you know, I think this is why when you give Claude a code base and tell it to improve it, it just goes completely bananas. that at some point you have, you know, way too much code, very large functions. Like how to make good code is just not in in in models, in the models definitions. And then Good code matters because as a human you still want to be able to understand just the flow of your data and like how things are put together, even if you don't care about the implementation necessarily, you do care about the design. Because even like paradoxically, good design that helps humans be productive also helps LLMs be productive, I find. So these things matter. And automation around these kind of lints and checks and and and and evals, I think, matters as well. Sebastian: Interesting. Yeah. We we've we've talked to people from from other companies that would probably say that the maintainability of the code for from a human perspective is less important for them than for the from an LLM perspective. And hence they're perfectly fine with automating more and more of this code review process so that the Quality bar for LLMs is is raised basically. Obviously, you have security, you have performance checks, you have the pretty much deterministic checks that the code is actually doing what it's supposed to do, right? But code quality then is more a matter of can the the future AI still work with it, refactor it and and everything. And then they would like there was an example from Paloa, I think. They have these different kinds of agents who are part of the review workflow. And then they have an orchestrator LLM on top, which then pretty much collates all the information and does a judgment. So it's the judgment. And if it crosses a certain risk bar, it would then still call out for a human review, but only above a certain risk bar. Is that something that you could see also for Thermondo, or or are you still more focused on, yeah. Well still we don't know where the future is, right? So maybe i it it André Neubauer: True. Sebastian: stays the way it is, right? But but are you more on the path, no, no, the the human needs to ensure that the the human quality bar is is met. Pedro Carvalho: yeah, a fun fact fun fact Italo is one of my best friends, so we talk a lot about this. I kinda thought you might bring it up. I want to do that. I a hundred percent want to do that. Here's why I don't yet. like I said, we didn't really have this API a year ago, right? So for us it's still pretty new. And we're at this kind of foundational stage of our new kind of platform first, API first kind of tech iteration of Termondo. And that means that It's really important for us right now to be really deliberate and really careful about the foundations that we're laying down. And so I actually want a lot of human eyeballs on the code, a lot of human eyeballs on the interfaces, and how this whole thing is put together. I think we're approaching the point where, you know, even though we kind of just launched it, I think that we're gonna have enough maturity. In terms of the system being productive and stable and people knowing how to use it enough, that I think then I'm happier to give it more slack and just you know, then really go to a system where you have agents going around and prioritizing pull requests and assessing risk. Because if people understand the tool well enough, we can, we can, we can recover if the agents make a make make a misjudgment. and I think that's true of kind of any agentic development. if you if you if you have a really well understood, well observed, well behaving system, it's way safer to give agents more autonomy. Sebastian: Yes. Pedro Carvalho: this is not new. I mean, like when I was head of engineering at HelloFresh, we we did kind of the same maneuver. We built an API that everything was plugged into. And I noticed that this is like 10 years ago, right? way before AI. So I noticed that once you had like a design and a procedure. People could just stamp out features, right? They had to think only about the business logic, but everything else about how that business logic plugs into the overall system, you almost don't have to think about because the structure's all there. and so your productivity just shoots up. and I think it's the same thing now with AI. so we're we're we're getting to that point, I think, where we can do that. I can't wait to try it personally. I I I saw what it did, I love it, I want to try it for sure. Sebastian: Got it, got it. That makes a lot of sense. And and I just want to say that this is exactly I think how every company needs to approach it. Paloa obviously running agents in production. So agents is their business. So they are masters in building agents the right way that do what they should do. So for them it's much easier to build these agents also for their internal per internal purposes, also for code reviews, right? So makes a lot of sense. couldn't agree more. And I think Andrew, you had one final question, right? And then then I think we need to slowly but surely come to the end. André Neubauer: I was just wondering if there, if you just made the statement that, agentic engineering requires certain prerequisites, if that, what, if that is going in that direction. so if I got you right, you say, given the limitations in like, in your platform, you haven't got far in the transition is that. correct or did I got you wrong? Pedro Carvalho: I think I think it's directionally correct. the the point that I was making was that to to give agents more freedom, you need to have a more solid foundation. I think when the foundation is not solid, or even if the foundation is solid, I I do believe that ours is. But the the engineers don't have, I think, enough time at the wheel with the new platform so that they can actually understand it well enough so that they can, you know, better recover, like if an agent goes amok, for example. So I think it's a it's it's a mix of just the pure solidity of the underlying platform and the the human's you know skill level with that particular platform. André Neubauer: I find that very interesting. I tried to formulate that way just to get this right because so far my hypothesis was LLM is not caring so much, especially the newer models, the newer generations, they do not care that much about certain structures, qualities, and so on and so on. I need to think about that. But actually also brings me to the point, Sebastian, we want to record a session on architecture and agentic engineering. So what is needed to make that work? Maybe we invite you for that session as well, Pedro. Pedro Carvalho: I have opinions on that. So yeah, I'd love to join. But but Sebastian: Awesome. Sounds great. Pedro Carvalho: but I think you're fundamentally right. I think it's a matter of do you have the right safeties around? Do you have strong enough harnesses and tests and interfaces? I think if you have those, then it almost doesn't matter what's going on inside the boundary, right? I think the question is what what's the boundary that I want to pay attention to? André Neubauer: True. True. Pedro Carvalho: For me, that boundary is still kind of slightly low down the stack, but I'm I'm I'm I'm trying to push it up for sure. Sebastian: Makes sense. Awesome. And with that, I think we should come to the final segments of our podcast. One thing that we're always talking about is the reality check. So what are the what the fuck moments or the wow moments that you still sometimes experience with LLMs? Pedro Carvalho: I think I think the the the it's so easy for an LLM to get confused still. if you're if you're mixing up I see this with my with my chief of staff, right? Like now it works heavily with sub agents because if I tell the same agent go check out all these different things, it very easily mixes things up and then the really the really bad Failure mode of LLMs is still that they will take their own words as fact. Like you're brainstorming something, they tell you something back, and if you and if it's wrong, that session is poisoned. Because at some point they will cite themselves back. so so my my my my chief of staff has just have a heavy, heavy use of of subagents so that I can, you know, compartmentalize the context as much as possible. A wow moment though is this. one of the biggest challenges for me as an engineering leader is that things are scattered, right? Like the same subject, same topic can be on Slack, on email, on Confluence, on GitHub. You're gonna have evidence and conversations that all pertain to the same topic happening all over. And so what I did was all right, I'm actually gonna like periodically, you know, grab. updates from all these sources and then I'm gonna have an LLM agent just pass through them actually multiple passes but basically coalesce these updates into either existing threads I call them or new threads and and this enables me to actually have way higher situational awareness because some code might have landed on GitHub that relates to some discussion which happened on Slack that has something to do with an incident or like a a higher error rate on sentry. LLMs are surprisingly good at piecing that together. so that that to me was a wow moment when I was like, hang on a second. I can actually have like, you know, telegram updates coming into my phone when the when things really matter and things that don't matter so much I just check them out later. But they're all correlated. That's pretty cool. Because it's harder to do that as a human, I think. Sebastian: absolutely. Because of the sheer volume of information that's flowing in, right? And the pattern recognition capabilities of AIs, LLMs, that that's what they're made for, right? Pedro Carvalho: Yeah. Sebastian: perfect. Thanks a lot. And as a final point, do you also have a prediction for our listeners? Pedro Carvalho: my prediction is not original, but I think that we are closer than we think to having let's call them synthetic workers. I think a lot of office work is very procedural. this involves this also includes a lot of what I do. There's a lot of stuff that I do that honestly doesn't really need a human to do. It's very procedural, very chore like. I think LMs have been good enough to do a significant amount of this work since November last year. what I think is missing is tooling and the ability to discriminate between different situations that look the same but are not. And once we crack that, which I don't think is is is that difficult, we're gonna be able to have people that have, you know. truly have like armies of agents that they control to do all kinds of office work. we've been talking about this for almost a year. I think we're way closer to that than we realize. And the key is that a lot of what we do is not that complicated. We just need better ways of giving agents the information and the tools that they need to act on it. Like if you have to pick up the phone and call somebody to do something, obviously that's still difficult, even with voice models, even with all that stuff. if you have to go through five different systems, one spreadsheet, Salesforce, App Sheets, you know, some whiteboard that somebody's keeping somewhere, that's all obviously off limits to to to agents in practical terms. But I think that if you can bring all the data to a sip to the same surface, make it accessible, agents can surprise you. In fact, at Termondo, we're kind of living in that future in a way. That's also another reason why we made the replatforming of the new API. is because a year ago we said, you know what, we can see where this is going. We want all of our data to be accessible to agents no matter what it is. and we're seeing that already. We're automating a lot of work and it th this makes people way more André Neubauer: We are at the end. I just need to ask that how do you do this? How do you make all data available to the LLMs? Is that via MCPs, you have a central platform or do we need to invite you again? Pedro Carvalho: I'm happy to come again, but currently MCPs, which annoy the hell out of me because they're way too slow. So we're moving towards more like you know, hybrid search knowledge bases. But the way that those are usually done is a little bit naive. And so the way that you should use those is way more agentic. Like there should be an agent in the knowledge base that is like interpreting your convers your ask and taking action to to answer it. But that's that's a whole nother topic. But yeah. knowledge bases, hybrid search and agents trying to decide how to get you the right information. the trouble with this permissions. But once you can search like your whole confluence in five seconds instead of a minute, that's pretty transformational. hybrid search is again not rocket science. So we actually have like two or three different kind of knowledge bases at Termondo that different people are maintaining. The thing that I'm working on tries to solve a few other problems as well. One of them is we being a German company, we have a lot of content in German, but we also have a lot of content in English and being able to search across different languages efficiently is not hard, but it is a problem. You have to think, you have to solve it. so the the new knowledge base that we're building is more unified, can work across all these different languages and domains. and also like the agentic part of it is like People often ask multiple questions in the same query, you know, like what are the different models that we serve and what's the process to install each? Now this is going to live in different parts of the knowledge base, right? So if you if you just do a naive query, you get nothing or you're getting complete information. So the agent has to take your question, actually check if it's multiple questions or just one, ask it in different languages, call us all the answers, all that kind of stuff. But what I think is really interesting, and I I did an experiment about this during my vacation because I can't stop. and Sebastian: Yeah. Pedro Carvalho: and I actually am like trying to to to to like build this as a prototype internally, is when you actually start putting hard business data in this knowledge base, right? So the knowledge base is not just pros, it's also like tabular data. And you give the agent that knowledge and you give it this dual mode of operation where it it can either or both try to get narrative answers to questions or just retrieve data. When it can do both things and it's all in the same place, so it's fast. it it is kind of crazy how you can very quickly, and by quickly I mean like under five seconds, right? Get a very complete answer to whatever it is. And once you do that. The other question that you start a asking yourself, and this is what I really want to explore, is well, if I have all this data in this system, anyway, and it's structured in some way. How much do I still need the other systems where the data comes from? What Sebastian: Yeah. Pedro Carvalho: is going on in those systems that is worth so much that I have to pay Salesforce like two million a year or something, right? Sebastian: Yeah. Yeah, yeah. Pedro Carvalho: and the answer is a lot, obviously. Like there's reasons why these things are are there. I'm Sebastian: Yeah. Pedro Carvalho: not saying SaaS is dead, but I am saying I I do think that this this is like a whole new way of building software that I think not enough people are exploring and that could actually be quite transformational because like the number one problem that I see is information being scattered. Sebastian: Yes. Pedro Carvalho: And you have entire industries that came up, like Looker is there, big like BigQuery is there to kind of paper over this problem. Sebastian: But but honestly, Pedro Carvalho: I don't know. We'll find out. Sebastian: I I really do like this approach and I I I so it resonates a lot with me. So I'm I'm always telling my people in my company, like the systems of record don't like start working on something with having in mind that you wanna get rid of the system of record, but rather making the information more accessible to agents and then more like easily accessible to to everyone, right? If in the long run you can replace certain systems of record, and I th I would say that probably probably like analytics databases that hold tabular data have a have a a purpose because what you what you wanna have in a knowledge base is probably the the the like collated information from different data points that you don't need always and every time, right? But but still like the if you do the first thing, then you can like contemplate about the the second thing later on per system of record, right? So makes a lot of sense. André Neubauer: One question, like what qualifies as knowledge in your definition? Pedro Carvalho: That's super fuzzy. I'm struggling with the same question. you have an answer, okay. André Neubauer: Okay, that is good. Sebastian: I have an answer. I yeah. I so I actually there's the so I I was in in knowledge management when I was a s working student back then and there the categorization was as follows. So you have data, which is individual data points, basically, then you have information, which is derived from individual data points, right? And then the next layer would be knowledge, which is pretty much if you wish derived knowledge from different like pieces of information. So that that's the categorization in knowledge management. Pedro Carvalho: I love the definition, right? I I I often work with that definition as well. I think the reason why I'm struggling is because I think we're we're we're calling these things knowledge bases so that we don't call them databases because that that that that term is already taken. but but I I I do think that w the way that I think of a knowledge base includes all those three categories. Sebastian: Yeah. Pedro Carvalho: And it's kind of the job of the agents to try to figure out what you need at which point. And that's where Sebastian: Mm. Pedro Carvalho: kind of the complexity lives. And it's also where the usefulness can come out of. because for example, the thing that I'm literally working on right now has a lot of like SOPs, right? Standard operating procedures for customer support, for installation, for all those things. So it's you might call it a a knowledge base, like this is derived from other things and It's it's it's it's it's end user consumable. but at the same time, I'm I I I built this this this prototype, you know, months and months ago where this is like a a as I said, I built a lot of agents for fun. And one of the agents I actually built a whole platform around them, and the platform gave the agent the ability to build its own database because you know it was working with a lot of data, right? Like just hard data, just numbers and and figures. and at some point it was kind of stupid to keep asking the target system to give you all the data at all like all the time. So I just built a a data store. Once I built a data store, I I I I remembered my contentful experience, where you know each customer gets their own like custom data content model. And I was like, Well, what if the agent could build its own data model? What if the agent could decide, hey, to do my job, I need these entities with these relationships and this kind of data? This is super, super trivial. Like an agent can do that super well. it's a very thoroughly studied kind of knowledge. And so the agents are really like the LMs are really good at that. and now that was a actual wild moment that I had was like, hey, you are a I don't know what it was. I built an agent to try to find me a doctor because you know, Berlin, some specialties are hard to to book to book with. And so this agent just like was able to send emails and listen out for replies and keep track of who had replied and who hadn't. And the agent just said, All right, I'm gonna build myself a little database of doctors and messages and replies and statuses so that I can remember this outside of my context, right? Just because if you rely on context, it can you know rot and and stuff. So the agent just saying, Hey, I'm gonna build a database and I'm gonna go off and do this until you're either booked or on a waiting list. That was a wow moment moment for me. And the Sebastian: Can't imagine. Pedro Carvalho: yeah, and this kind of like dynamic database. I looked at that and I thought, this is basically this is basically Salesforce, right? Like a lot of the complexity of Salesforce is what is your data model and what are your data flows. Sebastian: Yeah. And it's it's pretty much the database is just a bunch of text files, D files or JSON or whatever, Pedro Carvalho: Yeah. Sebastian: C S V or Pedro Carvalho: Yeah. The workflows, the behavior, those are the really tricky bits. and again, I'm not saying I'm gonna replace Salesforce with some five coded experiment, but I am saying Sebastian: Yeah. Pedro Carvalho: that these tools do boil down to very simple primitives. And the a and the and the AI is actually surprisingly good at creating and manipulating these primitives. and and I think that is where the interesting developments are gonna come from. Yeah. Sebastian: Sure. All right. I think we need to get to the end now Thanks a lot, Pedro. It was great speaking with you. And see you next time. Bye bye. Pedro Carvalho: Likewise. Thank you guys. Bye.