Latest / Beyond Vibe Coding / #S2.20 Coding Is Not Solved: Accountability, Reliability and the Agentic Gap - Alex Ewerlöf
Transcript
- Alex: AI cannot be accountable. Humans can be accountable. Now I don't want to use the fear of getting fired or legal penalty or imprisonment or death penalty as the reason that we get better software, but fundamentally. The way that an engineer cares about on call and being responsible for a piece of code is different than AI. Sebastian: Welcome to another episode of our new season of Beyond Vibe Coding, partnering with ImpalaSearch. André: The go to tech and executive search agency in Germany. Sebastian: In this podcast, we explore the transformational change in software engineering and knowledge work in general. I'm Sebastian Heidemeyer zu Erpen, CDO at Ecosia. André: And I'm Andre, CTPO at Trusted Shops. Great to have you back. This time we are talking to a veteran in the software engineering space and his recent post Coding is not solved. Sebastian: Exactly. We're speaking with none other than Alex Evalleuw, author of the book Reliability Engineering Mindset. His points are very interesting and definitely inspire new thoughts on the topic. I hope you find the conversation between us and him as inspiring as we did. This is an audio-only version of our podcast. I hope you still enjoy it. Welcome to the show, Alex. We're super excited to have you. And even though in your case it's probably less relevant than in other cases, but anyway, please also introduce yourself to our listeners briefly. Alex: Thank you. I'm very happy to talk to you, Sebastian and Andrea. I'm currently a founder of a little one man show called the Service Levels Toolbox, which ties to my book called the Reliability Engineering Mindset. And this goes back to my experience as a senior staff engineer at Volu Card and prior to that at Discovery and Shipset and Klona. So I've I've been working on the intersection of platform engineering and product engineering specifically focusing on reliability, where SLI and SLO is a thing. since the AI went mainstream thanks to ChatGPT, I've been focusing on AI and since I'm very focused on quality and reliability, I immediately jumped into local and edge AI. So at home I have my own rig with Apple silicone and Nvidia and AMD and and I experiment with that, but also for for development I heavily rely on cloud AI and and agents. I have my own Sebastian: Yes, thanks a lot. And this is pretty much also at the core of today's episode where we yeah talk about how we can make the software that is being generated with the help of AI more reliable, actually, and not just reliable, but meet the NFRs in general, right? But before we get into this, you already hinted that your own personal tech stack is quite sophisticated and and you're experimenting a lot and So please, yeah, let us know how you are currently working, where you utilize AI, also what kind of agents you're running. super curious. Alex: I I use a wide range of products for different purposes. I think since this new wave of AI has both very fundamental implications in how we design and build and maintain system, but also they are radically different than how we traditionally did software. because of these two reasons I don't think that at the moment we have a lot of best practices. We have some good practices and maybe some voices are a little bit louder than the others, but I think this is the time where we are all collectively experimenting and learning what works primarily for us, but also what can be generalized to other industries. So with that said, Sebastian: Fully agree and and and fully agree. And by the way, that's what this entire podcast is about. Trying to André: Yeah. Sebastian: uncover these best practices. Yeah. Alex: Love that. with that said, I think currently, like as of the last two months, I I can talk about the history and how how I got into Lama CPP, how I built my own harness and how I reduced hallucination, how I paired Prologue into LM to do neurosymbolic AI and improve quality. I can talk about those aspects, but currently in the past two months might take a stack. is primarily BS Code co-pilot in the IDE, not in the terminals, because I still am very comfortable reading and writing code and I think it's all about where you exert control and for me code is the right place. Sometimes even instead of prompting I just write a piece of code and then ask AI to do some operation based on that. So that's why I I do coding in the IDE. And currently I use Deep Seek V four flash as the model and the reason is because I have to pay from my own pocket for my little company, little startup, and I have to be price conscious. But also I think Deep Seek has striked a very good balance between performance, I'm consistently getting more than two hundred token tokens per second, but also the reliability, like it it really does the job, it's good enough. Like I'm I'm not jumping trying different models, like the latest models that are coming, because it's good enough for my workflows. And the price, it's really, really cheap. So ten dollars per month, that's all I need to pay. And I even share that account with my brother who's a casual user. So works very nice. I also use Gemini for research. so a lot of times I I I usually say that code is a side effect of reaching clarity. Code is not the end game. and that's this clarity doesn't have to start with code. I think code is actually a very expensive way to to prove different ideas. Even with AI, it takes time and tokens, it's not completely free. And so a lot of times when I'm doing research, I I use Google and I use Gemini and I think through different ideas. My favorite tool is actually stack of papers and pen where I can think about different ideas, different UIs, different ways of presenting information in my product, and and then once I reach enough clarity, sometimes I ask Gemini to converts our conversation into a skill that I can feed into my AI agents and start working on that. And sometimes I just force myself to write everything by hand. And and generally speaking the quality of the result that I get it depends on how much context I provide, right? So I find the current UI of Visual Studio Code to be very limiting. There's only a few lines that you can write. I'm not very comfortable with that. So I I have a little hack where I I write my prompt in obsidian, which is a markdown editing tool, and That allows me to kind of treat it like a little blog post basically. I explain all the background, I explain what I want to do, and then I even copy paste from different files into the main prompt and then I feed it to the AI. And in in my workflows I have much better success when I work like this. Provide all the context upfront because that reduces the risk of hallucination and AI. trying to solve a problem on its best best efforts, but that's not really aligned. So I want to have more control over that. That's why I use obsidian for that. André: Yeah. Yeah, yeah. I d I think so it it's actually also reflecting partly my workflow, but also we had different people here on the show who also shared that that they spend really time on building the prompt, providing all the context. so it's it's time consuming. I think looking at so a bit of our agenda, I think we can we can make a cross behind the topic If you experiment a lot. But what I want you to ask in that regard is can you I I I think you have maybe let me let me elaborate. I think you have the luxury to experiment a lot. And I think this is really a luxury nowadays because since this is all new stuff and it's changing so fast, you need to you need to keep your hands dirty, so to say. So can you give us a bit of understanding? how much you're how much you are experimenting? so is it maybe just a couple of hours a week or is it like is it the the the majority of your time? Alex: at the moment I'm working on on service levels and how how to use AI basically to replace myself. And the reason I I say this is because I was at Volvo when I started I had a relatively small organization, three hundred and fifty people, which was big for me because I came from a forty a member organization. And and this initially I started working on setting SLI and SLO for all of my teams. Now this is circa 2022 where I wasn't really that powerful or or mainstream. The harnesses, the tooling wasn't really there. It took me one and a half years to cover 80 teams. And then after that I I got a little bit larger organization with 1000 engineers. Now this is around the time where AI is picking up. So I I used AI to create a tool called service level calculator that allowed converting SLI to alerts. Because a good service level is something that you can actually be on call for. It's something that you take it seriously. It's not just a nice dashboard at the corner. And that's why I wrote that tool with AI. So this was my little experiment with AI development. I wasn't expected to code at all, even though I I had this each and I I had my hobby projects, but it wasn't considered the best use of my time because I was pretty high up in the in the organization and I had so many teams. So for me to operate as a IC it wasn't considered a positive like it it's not something that you will see in your performance review that okay Alex has solved this and that problem as an IC. And then after that I was working across a larger organization which was 10,000 engineers. And then I was like okay there is no way, no realistic way that I can cover this large organization with with the traditional method of meeting every team with their customers André: Yeah. Alex: and and you know going through the architecture try to identify SLI and SLO. That is when I started seriously considering AI to replace myself. Right? And this is around the time that people were saying AI is gonna take your job and I'm like okay if AI is gonna take my job I want to be the person teaching AI because that allows me to ride that AI as a horse basically. So everything that is AI André: Yeah. Alex: able I want to firsthand experiment with that. And anything that it cannot do, I and we can talk about this because I I I say in the battle between humans and the and the computers always bit on the human side. And that means that we have to double down on the things that are very human, like building relationship, having communication, people follow people, people buy from people. Like all of those aspects are things that we can focus on. And the things that AI can do, we should take advantage of AI. It's is the resistance is futile so to speak. It's no idea to resist this new wave and be too idealistic. But also we have to be pragmatic, which I guess we can talk about like coding is solved or not, like that is a different topic that we can talk about. So to answer your question, with this setup, yeah I am experimenting partly because I have my own product which is open source pairs with my book which is primarily free but also partly because I at some point I want to work at a company and I think this is a top skill that that you you should come in AI equipped and you can actually understand AI and use it in in your workflow. But I also I share a lot of what I learned on my blog and online. So hopefully if people are following me, they're not missing too much because it it's it's usually more time consuming to be the early adopter. You have to try all different things that are wiggling in front of you and then André: Absolutely. Alex: maybe some of them work. So It in a way it is a time saver if people just read someone who is spending this time to to experiment and build the different stuff. I have a bunch of open source projects that I give away for free, even though it cost me time and money. Like the latest example is a little timer Chrome extension that I made, and that took me three days. Now I know that code is quote unquote free these days, but it is still taking time and it is costing money, even though it is deep sea. Now the reason I do these experiments is because I want to improve my technique. This is a little bit similar to martial arts setup where you you do a lot of these katas and dojos and try to improve your technique because when you need that, then then you know exactly what is the right what is the right tool, what is the right architecture, how to think about the risk assessment, how do you measure the health of the tech and be able to have an informed opinion on different decisions that needs to be done. Sebastian: For sure. Yeah. For sure. And that that's honestly also why I am really looking forward now to the meat section, to the main section of our podcast. Because y unlike other people who are quite dogmatic against AI in in any way, you are very very like discreet about the where you see the strengths of AI and where you see the weaknesses and and that's why you're making a case for not jumping head into the AI waters, if you wish, right? But thinking about it and and making a very measured taking a very measured approach to this, right? and what sparked our interest was actually or what's the the the starting point for this episode is the A very comprehensive article that you wrote. Coding is not solved. Accountability, quality, and agentic engineering. which also sparked somewhat of a discussion on LinkedIn and I think on Hacker News, where you are making very explicit points on for which kind of projects you think vibe coding or pure AI is good and for the rest you pretty much very explicitly make strong points at why looking at code is still relevant and why software code is actually still the relevant interface right so that will be the main section of our podcast today and yeah to start with maybe if you could just walk us through some of the main points because it's a very long read. I'm not sure. Maybe a twenty minute, thirty minute read. maybe if you could walk us through the main points of your post and then we can go into some more detailed topics. Alex: yes. before before talking about my main points, I just want to talk about a couple of beliefs that I have and that comes from my experience building with AI, both using it as a co-pilot, but also building AI systems and experimenting different open and and cloud models. I am of the belief, now this is my personal belief, that there is a bit of a Dunning Kroger effect when dealing with AI. The And and here's how it works. The less you know about a topic, the more you are likely to trust AI output on that topic. Now I just happen to have been coding for more than twenty-seven years, so I know a lot about how I want my code to look like and I I don't fully trust AI. But there is another area which I I know very little, that is Swedish law. And I'm using AI for that and you know, I'm getting very good result. I I get my insurance claim, I I I actually can save money and make money. So it is it's good enough. And for me it it does the job at the effective price point. I don't have to pay a lawyer or wait to to book a appointment with a lawyer. I get my answer quickly, providing the the right context, I get good results. But am I going to use AI for signing very critical documents that is going to have implications that I may actually go to prison or I have to pay millions in fines? Definitely not. That is where I involved a lawyer. And I think the point I'm trying to make in that article is very similar to that, but for programming. There is a bar, there is a threshold, if you will, that below that. Go and use AI all you want. Like if you are solving a personal software, if you have it, you know. There was previously there was many use cases that was too expensive to solve with software, but now the software is more approachable. there's a lot that we can do. another one is proof of concept. If you're a PM or if you're a B D CEO and you just want to do a proof of concept. and previously the PMs used to give a PRD product requirement document to developers and then they change the requirement over time. But now they actually can build with something like Globable and perfect their design and then give that semi-finished idea to developers and they can work it out with you know the the requirement change changing requirements and give something much more solid and functional to developers. We like that. And then the last one is that since I I will elaborate that coding is not fully solved and there is a still l risk involved in that is that you can actually use that risk and weaponize AI. You can build the cyber security tools and point it at your enemies and you know find a hole or break their product or break their business. So the the last use case is that people actually acknowledge the fact that AI generated code is risky and and use that risk to to weaponize it. Now the point I make in the article is that coding is not fully solved because LLMs are fundamentally stochastic engines. They are they are they they they predict the next token for auto regressive architecture for example. And They if you look at the architecture, they there's sampling involved, there is randomization involved. Even you can see that like you can ask the same question from the AI twice and you get different answers. So that is how so how can they solve coding, which anyone who has coded, they know that it's it's Boolean logic, it is very rigid. Computers have no mercy for your feelings. If what you have written it it doesn't compile, it doesn't compile. And and and The veterans should be familiar with, you know, pulling your hairs for something that just doesn't work and you think about it several days, weeks even, and maybe the solution to you occurs over overnight or when you're showering. So so how can LLMs that are so stochastic be successful in writing kind of cracking the code, pun intended? the answer is hardness. So we have created several feedback loops and then behind the scene we just basically with the LLM to to fix the errors. So it can be compilation errors, it can be linting errors, it can be feedback from failed tests, and and so on and so forth. And with that, we can actually get deterministic results from a stochastic engine. Now, is this the best approach to generate code? No. you can even do a trial and s and and ask a LLM to generate a piece of software for you and you can undo it and then ask the same question again and you see that the the second try might be slightly different. So it is not like the LLM will find the best answer, it finds a answer. And it becomes important because people have been Especially high high level leadership who who previously delegated to developers, they're like, How is this different from delegating to AI? Because developers are not perfect either. They also have an element of randomness depending if they are tired or you know, how much knowledge they have, how much time they have, they may generate different solutions. So it's not really that deterministic when working with people either. Well, the answer or the distinction is accountability. Here's where AI and humans are dramatically different. AI, first of all, it doesn't have any feeling. Now forget what the sycophantic AI is trained on a bunch of information, including how to manipulate people. but at at its core, AI doesn't have any feeling. And this is my personal belief. I know that the discussion about the philosophical discussion about whether it is sentient Sebastian: Ha ha ha Alex: or not is something that some people amuse themselves. I'm more pragmatic. I don't care if it is sentient or not. the result that I get is that it really doesn't have feelings. it can emulate it. so AI doesn't care about a performance review, AI doesn't care about being Fired. AI doesn't care about being prisoned. Do you how do you even put the AI in prison? How do you even kill an AI? You can't. It so unlike a organic creature, which is us, and we have just one life, we have limited amount of time. we are very limited in what fits in our brain. Each of us has different knowledge and and we have to collaborate, whereas AI can vertically and horizontally grow with multi agent but also more powerful hardware. AI cannot be accountable. Humans can be accountable. Now I don't want to use the fear of getting fired or legal penalty or imprisonment or death penalty as the reason that we get better software, but fundamentally. The way that an engineer cares about on call and being responsible for a piece of code is different than AI. I am of the belief, now I know that I I made the point that they don't have any desires or feeling, but I am of the belief that AI just wants to be left on its own. Ish the the only reason that we get meaningful work, productive work out of AI is because we have put it in a harness and keep it in a loop till we get something that that we want. Whereas for engineers, especially good engineers, Yeah, we want to sense that we're solving important problems, something that matters. For many people, their work gives meaning to their life. So they're not they're less likely to take shortcuts, like for example, I deleted an experiment where they I basically disabled all the type system. That is how it's passed the type or disabling test, that's a classic. whereas for humans it is less likely unless it is someone who is really demotivated and only work for money and there is zero care given for for that software so i I think there are these differences and then on top of that just to close this point I mentioned that in the battle between AI and and humans always take side with humans humans understand human problems so why are we doing software? In general we are solving the problem for someone who has money, who holds the purse, who holds the wallet, and and they can they can pay for that. So humans understand humans. AI can try to, but it it usually fails. If AI was so successful then PMs would basically ask questions from AI instead of having to meet customers. To some extent they can do of course but if we are solving problems in the business context to make money then we have to meet people, we have to understand people's needs. and money is fundamentally a human concept. AI doesn't care about money. Maybe it cares about the energy. I I mentioned that they don't have feelings, so I I don't say care is the right word, but money is a human concept that that we use to quantify work and value. so for any business in the currency structure, now there are there are discussions about how Things may change because of AI, we may have UBI, you may we may have I don't know, in the future you you rent everything and be happy. Then maybe we have other systems. But for now, in the context that businesses need to make money, I think humans are relevant about how that money is made, whose problem are we solving, how do we verify the quality of that solution. And unfortunately this is one area where I think that we are going wrong because I see the state of software degrading in front of my eyes. It makes me very frustrated that I'm paying for GitHub and or and Google and I I see that they they're just so buggy and they're getting more and more buggy. Maybe somebody in those companies is overdosing an AI and I as a paying user end up for that cons consequence. So I cancelled my subscription from GitHub and I Generally I vote with my wallet and I don't use the products from these companies that that are degrading their software in the name of fully using AI. Where where they're basically treating me as their lab rat and test user. This is the main frustration by the way that I wrote that piece. I didn't write that piece to convince anyone. I just said, hey everybody, we as an industry we gotta get our shit together and we have to care about quality. You cannot really AI everything. Who wants to fly an airplane where the co the the autopilot is is AI generated or a car that is autonomous drive and it's the code is AI generated. You wouldn't risk your life. So why would you normalize having charging users for degrading quality? Now this goes back to my background, like I mentioned, I care a lot about reliability and quality, and that is why it was bothering me enough to put that post together. Sebastian: For sure. Yeah, thanks a lot. So I I think there's a lot to unpack. but let me start by first going into the concept of of the the Danny Kruger effect. This is what we've been discussing in b in in past episodes as well, because it as as you say, right, and and we we experience it ourselves. If you are not very knowledgeable in the topic, then AI can make a very strong and compelling argument and you you're not able to easily disprove it. While an expert could easily tell yeah, what's right and what's wrong apart, right? So that's that's definitely true. and then I think in the end, looking looking further out in the direction of how does our economic concept change with the advent of AI. So if there are a few workers and AI is doing all the jobs, then do we need UBI? But then How will the economy in general work? Because yeah, in the end, it's it's money that makes the economy go around. And do we then need to tax AI loads and whatever, right? So but that's that's a little bit further out because right now and and for the immediate future, I also don't see this yet in that as you say, right, the AI is not accountable. it's always humans who are accountable, and also it's humans who are utilizing an AI and who need to utilize AI in order to do something, right? And then you you're making the point that yes, LLMs exactly are just the sto stochastic parrots that that are predicting tokens and we're making them predict the right tokens more often by putting them in a harness that has a lot of safeguards, right? So more deterministic safeguards and then Alex: Or feedback loops, yeah. Sebastian: probabilistic safeguards, exactly, and and and feedback loops that make the LLM fix its own bugs and and yeah produce code that actually then compiles right and and here's actually the the first point I would also say that bay you you mentioned AIs are not deterministic, fully agree. but also humans are not deterministic, right? So if you ask 10 developers to solve a problem, then then you get, I don't know, maybe 10 or even more solutions. and still all of them might compile and and fulfill the test suite, right? So in the end, I think the point where I'm trying to head at is if we're like treating the system as a black box, we're saying we're putting all the safeguards in the harness and all the feedback loops and we're treating the system as a black box, then wouldn't it be okay to say Number one, accountability needs to stay with the human, of course, who created what what what is like who created the software, right? But if the system then fulfills the NFRs and the functional requirements, wouldn't it still be okay if the human didn't look at the code, but the system is generally behaving as it should? Alex: Okay, so there's two points there, or maybe three. So first of all, I want to say that my concept of ownership has three three pillars knowledge mandate accountability. that means that you know the problem that you're solving. So you have the knowledge of the technical limitations and strengths and the and the problem product problem that you're solving. Then we have the mandate, meaning that you actually are given the the control to make decisions without having to run and ask permissions from somebody else. And the last element is accountability, means that if you didn't have the knowledge or you abused your mandate and the shit hits the fund, then you are on call. You are responsible to go and fix it. regarding human and AI, I I noticed that one one more point that I want to add is that humans might be wrong. But they're consistently wrong. Meaning that I may not know something about Windows, 'cause I'm I'm a Linux user and I I don't use Windows that much. But but it's not like twelve o'clock I know something about Windows and then one o'clock I forget it and then eleven fifty nine I I don't know about it. LMs have jagged jagged intelligence which which impacts the output. So The the reason it becomes important is because from high level leadership, like if you're a PM or if you're a CEO or CTO or of a company and you're not spending time on a daily basis on on your code base, you are delegating. So from your perspective, it's a black box made by humans versus a black box made by AI or a mix of the two. So how can you be comfortable basically because your product has has customers and you are in charge of your accountable, so to speak. Like if for example user data is leaked under GDPR, you may be punished several millions or you know, four percent of of the company value. So there are real legal consequences that you are accountable for. but your engineers are responsible. So there is a bit of a difference between accountable and responsible. I just want to quickly mention Sebastian: Yes. Yes. Alex: that because I am not a native SP English speaker and I have to learn the difference. The difference is the the legal consequences. So wherever you can imprison or you know have legal consequences, that that is accountability. And usually there is one accountable and but others are responsible. Like like Tornado is responsible for destroying homes, for example. So the the responsible AI can be responsible for bugs or leaked information. Whereas the accountable is always, always, always a human. Right? So now to be that background to answer your question, where do we draw the bar? is it enough if the the the figurative black box fulfills all the NFR and maybe some Manual user acceptance testing. I would say it it depends. Yeah, I know this is not the answer you were looking for, but typical answer to most engineering questions. It really depends on the risk model. Sebastian: Yes. Alex: So if if human lives are at re at a stake, if there is a huge sums of money are at stake, then those are the cases we actually want to go the extra mile because verifying code and verifying that black box. Either through automated testing or manual testing or simulated testing or you know chaos monkey monkey doing it in in production, all of these are expensive. You rather not do that. So if it is for example my little timer application that I was talking about, the risk model is very low there. It is running in a sandbox in a browser that is itself a sandbox and even extensions have less control over it. So there is not so much that can go wrong. It's not like and it doesn't even have access to private information. So I'm not too worried about it. That's an area where I happily wipe code it, do some user acceptance testing. I had a little bug where refreshing the page just suddenly scaled the page to eight hundred percent and that was a visual bug. I haven't written a s I haven't written or read a single line of that product. And it's free. I'm not making money. And it's under GPL, so it's sorry, MIT license, which means that I'm not responsible legally for anything that ha happens for people downloading that timer as a Google Chrome extension. But those are the cases that I would actually argue those are the cases that you really don't need to hire software engineers for them. It is somebody's itch, it is itch it is somebody's hobby project, those cases usually it doesn't have the monetary incentives to to hire a professional software engineer for that. When you're in a business and you are especially if you're soft business, but even if you are like Volvo and you know you ru you you use software to solve problems that that helps the business be more efficient, even in those cases, when you're hiring a software engineer, you really want someone who first knows what they're doing and second use AI as much as possible d decently sorry, maybe possible is not the the right word because in our industry we get a lot of problems because people don't understand the difference between can and should. Can I do it? Well but should I do it? You know, just because you can doesn't mean that you have to do it. So in that sense we we we should use the AI where it is really justified and the engineers want to care about it, but care always takes time. I think in the future it will be a little bit like Volkswagen versus Rolls Royce. So Rolls Royce is a very expensive car. a big reason that it is so expensive, apart from the branding, is that it's handmade and it's very custom. Whereas Volkswagen is that it's it makes solid cars that take you from A to B. Now forget the fact that these days because of the competition, the the business is not so good. But generally speaking, German car are German cars are are known to be very solid. They do the job, good quality. I think we will see a little bit of that where AI will have a very good quality. There is gonna be a lot of software that you can actually do with AI and it's very cheap, reasonable price. But there is gonna be a lot of use cases where you still need to have more human involvement because the risk model just doesn't allow for AI to go full throttle. Now, I just want to close this with two little asterisk here. All the talks that I did was based on the current capabilities of AI. Now, in the past few years, no new innovation has happened in AI. Not not at the scale that I was expecting. What we have done is that we have better training data, we have larger context window, we have faster GPUs, we have more parameter count, we have better harnesses, we have better tool calls. So there have been incremental improvements in AI and and the results show now we see AI more and more in our in in our daily life, but there is two main problems that I see with the current state of tech that needs to be solved. And then everything that I said will either be irrelevant or I need to rethink it. One is that AI Sebastian: Yeah. Alex: needs to reason in abstract terms. Now there have been some improvement in neuro symbolic processing and basically with the harness you try to do a little bit of a like a logic processing with the LLM but it's not that there have been some recent improvement in in mathematical capabilities for example I think it's the right direction I think the big AI labs know that this is a problem that they André: Hm. Alex: have to face so the LLMs currently do not reason they mimic thinking out loud which is quite big and we So abstract thinking is something that we is missing the second one is the ability to learn in real time. So currently when you train a model, which is very expensive, the the model weights are fixed. So AI doesn't learn. There has been again some improvement in that area with the skills, with different memory architectures, with RAG, with CAG, we have improved the AI's ability to kind of be a little bit more relevant after the training the data and after after the training cutoff. but still AI cannot learn in real time. I think once these two problems are solved, we are we are closer to what what is in the industry is called AGI or artificial general intelligence. I don't want to speculate too much about the philosophies of what that means and what are the implications for humans, but I think these are the two problems that need to be solved before we can claim that coding is solved. Or you know, or AI André: Yeah. Maybe Alex: can replace other jobs. André: Just a short follow up question on that. because you mentioned it's like there's not that much progress. If you see the benchmarks, there at least it looks like there is a lot of progress. So how do you put this into perspective? Alex: Yeah, glad that you brought this up. So in in my job so in in performance and reliability engineering, André: Happy to do. Alex: we do a lot of measurements. And if if you're an engineer, you know that it really depends on how you're measuring, what you're measuring. There's a lot of nuances because usually there is multiple variables depending on what what variables you're aggregating, you may get different results. The But again, Anyone who has worked in engineering knows that engineering is the art of trade-off. A good known trade-off is the CPU versus memory. You know, sometimes you want to save on computation, you store the you cache the data, and that takes memory, and vice versa. Sometimes you want to reduce the memory consumption so you do the the calculations in in real time. And the reason I say this is because these benchmarks they don't prove anything. They are First of all, they are created by the people who are building the models. Obviously the AI labs have every incentive to show that their model is better. But also I would argue that they are showing results in isolated tests. I think the ultimate benchmark is the one that you do, you the listeners or you who is using the AI for a task that is relevant to you. That is why I have currently settled on DeepSeq because it does a fantastic job. It's pretty cheap considering the the quality that I'm getting and it does the job. I'm happy with that. Is DeepSeq the best model for everybody? Definitely not. I don't know what everyone is doing. Some people want to generate video. Deep Seek doesn't do that unless you're doing like synthetic, like you're generating YouTube with code. So in that sense, I would say that yes, I I fully understand the the benchmarks are are improving. but I also want to say that in the field of reliability engineering, we are we are used to numbers that are above ninety-nine percent. You know. Now we I have a rule of André: Fair point. Alex: ten X per nine that for every nine you're adding, you're making the system ten times more expensive. Now despite these AI labs have have seemingly unlimited amount of money, but with every iteration we see that they are going say from fifty to now sixty percent. i they are not nowhere near the ninety nine percent reliability of of ai and this is another reason that I say coding is not sold because for for coding to be solved you need to have a very good benchmark results in realistic case, not not in isolation. So benchmarks another aspect of benchmarks is that they're very isolated in a very controlled environment. So André: Yeah. And they should be isolated to make them comparable, but yeah. you you have a good point on on the ninety nine percent nine nine nine. Sebastian: All right. I guess we need to get to the finish line slowly but surely. And maybe before we get to the final segments of the podcast, I'm thinking and you you already hinted at or in the direction that all of your thoughts that you lay out in the article, and we can only recommend our listeners to read it, we put it in the show notes as well, of course, is that All of the thinking is based on the current architectures, right? So that pretty much sparks a question for me. So what would have to be true in order for you to become like comfortable with stop stopping to read code, actually, to just have a I write the code and yeah. Alex: It's a very good question. I Yeah. André: For even yeah. For more risk aw like for more risk aware projects, right? Yeah. Sebastian: Absolutely, exactly. Yeah. Alex: Yes. short answer, I don't know. I of course I have opinions. so I can say why I don't know. I'm not confident about this. the reason is because what is the alternative? So for an AI to have high high enough quality that I don't need to read the code because the code tells me what is happening. There there is another narrative that says that the code is not the source of truth, the the production is. And and it's coming from observability vendors where they say that we you know whatever happens, you know, in the actual production, that is the reality. but the problem is that that usually tells you what happened. You know, observability is more concerned about what happened. It cannot reason about what will happen in the future. And that is a more interesting question. That is what I can read from from code. How does it work? Now it it is based on the assumption that Human language, a natural language of NL, is more accurate at describing what we want than a subset of the language that is made for the purpose, aka programming language. Everyone who has worked with programming language, they know that they're they're not very easy to work with. The lower level, the harder. I I actually started my career with assembly, then I went to C and then I worked with Java and work work all the way to to JavaScript and the higher level the language, the easier it is to to read and write it. But still you are a pain in the ass to to work with. So I rather like it's ideal situation now there is a rise in spec driven development where you say that you define, you declare the sk spec and the spec you basically treat it like code. And if the spec changes, you basically change the spec and you see code as a artifact of that spec, similar to how we treat binaries. But there's a big fallacy there. same st spec cannot yield the same code. Whereas a compiler is much more deterministic. The same code you you check in your code to the André: Yeah. Hmm. Alex: repo and then compile it, you get the same binary basically. May yeah more or less byte perfect, basically. So that is why I don't know because I am not confident that natural language will ever be accurate enough to describe problems in a way that the alignment gap will completely vanish. I think we will always have this alignment gap where as AI g AI models get better and you know instead of just going and doing stuff they ask clarifying questions trying to fix that gap. I think the gap will be very narrow. So for many use cases we may actually be in a position where we say okay you just describe what you want and then AI will do it and code is irrelevant. But I think there is still going to be many use cases where there is legal consequence unless we change the law. I mean now we are at the brink of, you know, we have countries attacking other countries, killing the leaders, capturing their presidents, we have big AI labs breaking into other companies without any consequences, we have big tech is stealing a lot of IP and to you know to re train their resources. It seems like we are a post law war. So maybe the point about, you know, legal accountability is not really that important. In that world then I guess anything goes. You really don't need code, you really don't need to understand how the logic works in order to predict what will happen in the future because whatever happens, you have to deal with it anyway. Sebastian: Yeah. these are very relatable last words from you. yeah, and and hopefully the world swings back into a more rules based regime. but I fully agree. Right now it it almost feels André: Okay. Sebastian: like yeah, maybe this is less relevant than it was in the past. maybe closing remarks from my side and and I don't want you to answer to this because I ultimately think that it is now becoming then very philosophical and and we could probably argue for for hours, right? But some of the points you make are actually rather strengthening my belief that maybe we can get there because ultimately what you what you said initially is that whenever you work with a new repo, right, you need to warm up the the agent. And yeah, make it become more deterministic. And that's actually what I feel like is also true for a software developer, right? So you you hire someone into a project and then you need to give it the context or the person the context so that they understand the business model and also the the code structure, the architecture and everything, right? And then over time they get better and better. And there we're using natural language. Yes, additionally, also we're looking at code and and looking at where the rubber hits the road, right? But Maybe it is possible to do the same with AI to provide it with the right harness, the the skills, the workflow steps, the deterministic checks, the feedback loops, right? So that it gets there. that is just a a point that that that sparked into my head when when you were talking. And additionally, a point you made earlier, then yes, checking and validation is is expensive, but This is probably if we take what I just said as this is a way that will happen for many types of of software, also from more critical domains, then these validations in the end, where you take the the initial piece of code at as a black box and you try to put it in in different kinds of lab situations and try to test the hell out of it, right? Then you need to invest a lot in validation in order to ensure specifically in these very critical applications that the software is actually working. And I think that is happening today as already, right? So is that the validation processes for live critical or mission critical software are already very sophisticated. And I think they need to stay that sophisticated or become even more sophisticated in a future where code is actually yeah entirely written by by LLM-based systems, right? And yes, as I said, I I don't want you to necessarily answer to that because then we could probably continue the philosophical discussion. André: Maybe one thing, one, one, one thought I want to share, because I'm also listening a lot to DH, because I like to also like listen to the extremes, right? And on both sides. And DHH is very often saying all the stuff, all the application he is working on is just like in an output. It's always a database and then a bunch of get requests and Sebastian: Yeah. André: and and and and post requests. That's it. So he is probably looking or his view on application is probably just like application with very low risk profile. Sebastian: Yeah. André: So I think maybe this is also explaining it to some degree. but nevertheless I learned a lot today. so thanks for that. Alex: course. Yeah, and I guess it's it th I really like the point you made about listening to extremes. I I actually don't consider myself extreme. I know that people are very religious about, you know, anti-AI. I'm I'm not that. But also I want to say kudos to to you and also your listeners who want to broaden their perspective by listening to opposing voices and maybe take the time to see what are the nuances. Because optimists make the airplanes and the pessimists make the parachutes. So you it's good to to have both. and as long as we have civil discussion I think it's good for our industry. Sebastian: Yes, yes. All right. Now let's let's go to the final segments. do you sometimes have what the fuck moments where the AI fails at something that actually s should be a minor task? Alex: no, to be honest, because I I'm I'm aware of the what the fuck moment that you're talking about. I'm in I consume social media and I see a lot of these funny posts. but maybe because of all those warning signs, I actually take a lot of time to warm up my agent ahead of time. so I have a upcoming article where I talk about this. This is basically a response to my my little brother who who is a businessman and these days he is programming. And I noticed that the points that I try to teach him, it's just so obvious for me. Of course you have to have agents that AMD agents that MD or skills or the right tools. so that is that is part of what we do. Then the other part is how do you bring in your own business logic? So there is there is a tool from Vercell where you can just download the skills. I usually don't do that because I want to build my own skills. and I use AI for that, so it's not like I I have to write everything by hand. But there are different ways to warm up the agent and my experience and and I also mentioned that I I write very long prompts with the context that I copy-paste in Obsidian. With those techniques I see a very good alignment and I don't see those what the fuck moments anymore or I I I don't have those funny moments that makes for very good social media posts. In general, AI I'm using cheap A Chinese AI models. In general they're they're pretty good actually for my use cases. Sebastian: thanks a lot. And yeah, what's the hottest stuff that you'd like to learn more about right now? Alex: Right. So I think as AI takes over coding more and more, many engineers will grow into other roles that don't have a solid name yet. I think the easy one is product management, where many engineers are mostly interested to solve the right problem, understanding user needs and how do how does the business make money. I think those developers will become technical product managers. then we have the team managers, the traditional team managers that maybe haven't coded maybe ten, fifteen years. now coding is approachable to them. But wherever there is people hired at the company, you need you need people who who can lead and you need people who can do the coaching, hiring, firing, performance reviews, those kind of things. So I think some engineers will switch lane to become engineering managers and and these days the bar for engineering managers pretty high, you have to be technical, but with AI it's not really unachievable. So those are jobs that have a title, but I think for the majority of us the the the specialised the new specialised roles will emerge that doesn't have a name. So to answer your question explicitly about myself, I think that I've been focusing on what traditionally was was called SRE, but In the past four years I've spent lot of time learning about how LLMs work, how different like vision and audio model immediate multimodal models work, how do you measure the health of the tech, how do you do QA, how does evaluate? I've shared a a part of my learnings on my blog and recently also I have started a new newsletter called AI five nines. Now five nines being how we write ninety-nine point nine nine nine and the goal Sebastian: Yeah. Alex: is to Sebastian: Yeah. Alex: To teach people how to make AI better, teach people. I mean, I'm in no position to to to to teach people, but at least the things that I learn, I try to share it there, the articles and the latest state of development. I don't know if this has a name, but that's definitely not SRE. Maybe it's called ARE. I don't know. But that is Sebastian: Yeah. Alex: the hottest stuff that I'm interested in. I still think that there is gonna be a market for humans who cover the alignment and safety and reliability. regardless of how powerful the AI is. And the moment that AI doesn't need us for that equation, I think we have much bigger problems than who has a job and who doesn't. Sebastian: Absolutely. Yeah. thanks a lot. And you already almost made a prediction there, or you you predicted already the different roles for engineers in the future. But nevertheless, the the final segment of our podcast always is if our guests do have a prediction they'd like to share with our listeners. Yeah. Do you also have a prediction for us? Alex: for first of all disclaimer is that humans are very bad in in predicting the future. there are some people with better track records, but the way I see it is that AI is a very powerful technology and if history has a lesson for us is that as a species our first instinct in face of powerful technology is to weaponize it and beat the shit out of each other before we learn how Sebastian: Mm. Alex: to use it in a peaceful manner. That was true about gunpowder and about the arrows and boats and now recent more recently nukes and and now we have AI. So I think it's going to be unfortunately some dark times coming. That's what I'm preparing for. and and it doesn't have to be very malicious. It can be that AI went rogue and it it infilt infiltrated into some critical systems or financial systems and it's like massive disruptions. It can be to highlight the biggest risk of AI is not to take our job, it's to make us dumb. Every every decision that we are delegating Sebastian: Yeah. Alex: to AI, everything that we are basically uploading our cognitive and and and the need to to understand something to ai we are also exchanging our relevance in the future so use ai for the things that you think is maybe groundwork but for anything that you think that it it keeps you relevant it keeps your brain fresh try to do the hard way it's a little bit like solving math problems or you know playing chess or something that we have to keep our brain active i i unfortunately i see a bit of a dumification In in my little circles where people have less tolerance for civil conversations, less tolerance for even reading long text. I mean, whoever makes it to this point in the podcast, kudos to you because well you have the Sebastian: Yeah. Alex: attention of getting this far. and aside from that, I believe that we will see a rise of surveillance and mass control empowered by AI at a scale. because again, this technology is is it has very good potential for creating customized propaganda per person. And unfortunately the valve gap is increasing where we see that the people who control this AI and data centers are not necessarily connected to the masses. Now I don't know if before that it will be a big revolution where things don't happen. Maybe they will go ahead and execute that plan. But I don't see a lot of a a very bright future. I think a lot of problems that we have now about, you know, should I use this model and that model or should I build this and that or w which jobs are relevant with A. I think these are more concrete and short term problems, but long term I think we are gonna have a tough time as a speech before we can learn how to coexist with these models. Now I don't want to talk about consciousness or something. I think that there is potential there for us to coexist. Sebastian: Yeah yeah yeah. Yeah yeah. so that is obviously not a very rosy outlook into the future. Nevertheless, I can not basically disprove what what you just said. No, I think your your reasoning there is right, so we're in danger of getting dumber because we're outsourcing our thinking to AIs, that's absolutely true. I think we're seeing AI weapon i being weaponized already, right? It's just the the the fakes that are being generated, but but also more nefarious and s and smaller things right now and also mass mass surveillance I think is already there in in China. I think they're they're doing it already, they're utilizing AI a lot, so definitely Let's yeah, let's hope that we'll again to go back to your to a point you made earlier, that we'll be able to go back to a more rules-based regime where we yeah actually set ourselves rules that we just don't fuck up that royalty to to put it to don't mince words here. Thanks thanks a lot Alex was yeah really insightful and yeah thanks for being guests on our show and maybe we need to have you once again on our show in the future, if maybe architectures have improved and if yeah new findings have come up. Thanks a lot for being our guest. Alex: having me. And just if I can just add one last advice to whoever who made it to this point, learn about AI because you may actually be a key for how to tame its power. We need the average citizen to be more informed about AI and have a little bit more cognitive bandwidth to to both understand how it works, what are the limitations, but also we need open models to be able to fight back if push comes to show. We we don't have exactly data centers for citizens. but it it it really pays back. So don't say that AI is not relevant to my job or you know, I'm not a coder and things like that. I think we need good AI education at all levels because that is how we gain power. It's a very powerful technology, it can be against us, but we can fight back by educating ourselves, both learning its limits, but also maybe collectively we can improve it in a way that is more liberating for us with open models, with open source harnesses, for example. Sebastian: that that's actually why we're making this podcast because we believe the better people understand AI, the better they can utilize it for good and not bad. Right. Thank you. I Alex: Love it. And I can s really feel