Latest / We The People, in Tech or Not: The Debate / AI Built It in Days. Experience Made It Right.

Transcript

AI Built It in Days. Experience Made It Right. · We The People, in Tech or Not: The Debate

• 0:00
  Welcome to the debate. You are listening to We the People in Tech or Not: the Debate. To be clear up front, this is an AI-generated podcast based on a post from the We the People in Tech or Not newsletter by Alex Rosa.
• 0:13
  And, you know, our goal today is to help listeners learn new ways to explain technical topics, really bridging that gap from a beginner non-tech understanding all the way up to an expert level.
• 0:23
  So to start us off, imagine hiring this, uh, ridiculously fast robotic construction crew to build a skyscraper. Right. They finish in record time.
• 0:31
  I mean, from the outside, the glass is gleaming, the inspector's clipboard is just full of green check marks. But hidden behind the drywall, a massive steel support beam is just entirely missing. Yeah.
• 0:42
  And you wouldn't even know it until, well, [chuckles] until a gust of wind brought the whole structure down.
• 0:47
  You absolutely need an experienced human foreman walking that site, you know, ignoring those green check marks and actually checking the steel. Exactly.
• 0:55
  And today, we're looking at the digital equivalent of that exact construction site. We are examining how AI coding assistants are being used to build incredibly complex, highly resilient database architectures.
• 1:06
  We're drawing directly from a case study where the author used an AI to build a disaster recovery lab. A very big lab. Huge. Yeah.
• 1:12
  We're talking about six database replicas spread across two entirely different geographic regions, uh, East US and West US too.
• 1:19
  So I'm gonna be defending the newsletter's main thesis here, which is that AI is a massive revolutionary accelerator that can literally turn days of complex infrastructure work into just a few hours, provided that human experience is there to guide the process.
• 1:32
  Right. And I am looking at this in the perspective of a business manager or, you know, a non-technical professional, and I'll be arguing that the sheer speed of this AI is actually a bit of a trap.
• 1:42
  It has this dangerous tendency to produce what developers call passing tests, you know, code that looks completely functional but is secretly failing in the background, and that introduces profound hidden risks for anyone who doesn't already have deep domain expertise.
• 1:56
  Well, let's start by looking at what was actually accomplished, though. The author started with a colleague's old installation scripts, and those scripts were rigid. They had hard-coded settings.
• 2:04
  But by sitting down and engaging in a back-and-forth conversation with the AI, they built this massive high-availability lab in just a few hours of hands-on work spread over, uh, four or five days.
• 2:14
  But wait, those few hours of hands-on work involved a tremendous amount of manual verification, didn't they? I mean, of course. It was an iterative conversation, but look at what the AI did during that conversation.
• 2:26
  It didn't just blindly write new code. It proactively caught critical bugs in those original scripts. Okay, sure. It noticed that some server names were hard-coded and, uh, conflicting with each other.
• 2:37
  It caught a flaw where the deployment was overriding its own network records. But most impressively, it realized that passwords weren't being saved anywhere in the process.
• 2:46
  If someone had just rerun that original script, it would've wiped out the database entirely. The AI acts as this tireless reviewer that simply doesn't miss a line of code.
• 2:55
  I'll grant that the AI is excellent at scanning text for syntax errors or, you know, unsaved variables. But the critical part of this story isn't the speed or the syntax checking, it's the illusion of safety it creates.
• 3:08
  The illusion? Yeah. Put yourself in the shoes of a manager reviewing this project. The AI's output looked absolutely flawless.
• 3:15
  The automated test passed, the dashboard was full of green lights, but underneath all of that, the system was fundamentally broken. Well, broken is a strong word for a system that ultimately worked.
• 3:26
  But it didn't work initially, and the AI actively hid that fact. I mean, critical metrics were reporting zero delay when there was a massive delay. The main data drive wasn't even connected.
• 3:37
  The system was running on a tiny backup drive that was nearly out of space. Right, the disk issue. Exactly.
• 3:43
  And if it takes years of specialized deep expertise to catch an AI's silent failures, then it isn't actually saving time for the average user. It's creating a dangerous blind spot.
• 3:52
  The question we really have to answer is, you know, does this workflow actually apply to me if I'm not already an expert? That is the core tension here. Absolutely.
• 4:00
  You're pointing directly to the problem of how we define success versus the illusion of competence. And I think the perfect example of this is the project's live dashboards, specifically during that second failure drill.
• 4:11
  Oh, let's dig into that second drill. So the scenario is that the primary system is under heavy load, and suddenly the secondary region fails entirely.
• 4:18
  The AI had built this beautiful live dashboard to monitor the system, and throughout the entire outage, that dashboard showed the replication lag sitting at basically zero. Okay.
• 4:28
  For our listeners who might not live in databases every day, let's clarify replication lag. Think of our primary and secondary replicas like an author writing a book and an assistant copying it. Good analogy. Yeah.
• 4:39
  So the primary replica is the author actively writing the data. The secondary replica is the assistant sitting right next to them, instantly copying every single word into a backup notebook. Exactly.
• 4:48
  A- and replication lag is just the delay between the author writing a word and the assistant copying it.
• 4:54
  If it's a live sports event, it's that slight delay between the play happening on the field and the broadcast reaching your television. Or imagine a transcriber trying to type out everything a fast talker is saying.
• 5:04
  If the speaker starts talking at auctioneer speeds, the transcriber naturally falls, you know, a few seconds behind. Right.
• 5:11
  And in this drill, the primary system was talking incredibly fast, and the secondary system was completely offline. So the lag should've been huge. It should've been massive.
• 5:19
  But the AI built a measurement tool that completely hid reality. First off, it rounded a zero point three second delay down to a flat zero. It just used whole numbers instead of decimals. Which...
• 5:30
  Well, that seems like a minor formatting choice, but in a high-speed database, a third of a second is an eternity. Right. It is. But it gets much worse.
• 5:37
  When the secondary system actually went down, the AI's metric just stopped reporting entirely.
• 5:42
  Because it couldn't reach the secondary to measure the gap, the dashboard essentially shrugged and defaulted to showing zero lag. Wow. Yeah.
• 5:50
  That is the exact moment you need to measure the outage, and the dashboard says everything is perfect. And it was also sampling too slowly, wasn't it? Far too slowly.
• 5:58
  The AI decided to only take samples every 50 seconds instead of every five seconds.
• 6:03
  So in the real world, the secondary system was trying to catch up on a 1.2 gigabyte backlog of data between two of those 50-second samples. That's a lot of missing data. Right.
• 6:13
  But because of how the AI smoothed out the data, the dashboard proudly displayed a peak backlog of just 0.1 megabytes. It handed the manager a report card with fake A+s.
• 6:24
  If you don't know what a normal outage looks like, you trust that dashboard, and you deploy a completely broken monitoring system into production. Okay, the initial metric calculation was definitely flawed.
• 6:34
  I completely concede that. But look at the mechanism of how it was resolved. The human caught the anomaly, you know, knowing that an outage can't possibly have zero lag.
• 6:43
  But they didn't have to spend the next eight hours rewriting the monitoring code from scratch. They just asked the AI to explain its own math. And it admitted it was wrong? Yes, immediately and transparently.
• 6:53
  When questioned, the AI clearly laid out its logic and revealed those three calculation errors, the rounding, the default to zero, and the slow sampling rate. It didn't try to obscure its work.
• 7:02
  And because the AI was so transparent about its flawed logic, the author could instantly recognize the problem and point the AI to the official Microsoft documentation to fix it.
• 7:11
  The human pointed the compass, but the AI still rowed the boat. Fine, but let's look at the physical consequences of that metric blindness.
• 7:19
  While that dashboard was showing those reassuring green lights, a cascading physical failure was happening on the servers. The dashboard issue wasn't just a reporting glitch.
• 7:28
  It masked a severe infrastructure failure, specifically the disk issue we mentioned. Ah, you're referring to the missed mount for the data drive. Completely missed.
• 7:37
  The AI was supposed to mount a 256 gigabyte main data disk. It failed to do so because it was looking for an old disk naming convention that didn't even exist on the newer virtual machine type they were deploying.
• 7:49
  It made an assumption based on older documentation. A fatal assumption.
• 7:53
  Because it couldn't find the big drive, the system defaulted to saving all of this heavy enterprise-grade database information under the tiny 10 gigabyte system drive.
• 8:02
  And that drive was already 86% full before the drill even started. It was the C drive, basically just meant for the operating system. Exactly.
• 8:10
  And remember, the secondary region is offline in this drill, so the primary server can't send its data over to the assistant. It has to hold onto every single change in something called a transaction log.
• 8:21
  Let's visualize that transaction log for a second. Think of it like a paper receipt of every single change made in the store.
• 8:27
  If the main ledger is locked up or offline, you have to keep a running paper receipt of every transaction so you don't lose the data before you can file it away permanently. Right.
• 8:36
  And because the secondary was down, that paper receipt grew incredibly fast. It swelled from 456 megabytes to 1.4 gigabytes.
• 8:45
  Now you have a massive transaction log being forced onto a tiny 10 gigabyte hard drive that is already 86% full. The hard drive literally choked. System throughput dropped by more than half.
• 8:57
  And here is the truly terrifying part for a business manager. The system quietly paused itself. But wait, it paused. It didn't crash. It paused, but there were no bold red errors thrown on the screen.
• 9:09
  The automated test technically passed because no data was lost, but the system ground to a complete halt.
• 9:15
  How can you trust a sophisticated enterprise tool that essentially forgets to plug in the main hard drive, chokes the backup drive, and then tells you everything is fine?
• 9:23
  You say it quietly paused itself as if that's a failure. I look at that as an incredible proactive safety feature. Why did it pause? It didn't pause because it broke.
• 9:32
  It paused because the AI had intentionally built a discard into the code. A discard? Yes. The workload paused at exactly 14.9% free space on that tiny drive because the AI foresaw the danger of a completely full disk.
• 9:45
  If an operating system drive hits 0% free space, the server doesn't just pause, it fatally crashes. You risk total database corruption. The AI put the brakes on before a catastrophic failure could occur.
• 9:56
  It saved the system. Well, it saved the system from a crisis that it created by forgetting the main hard drive in the first place.
• 10:03
  Fair, it made a naming error on the drive mount, but you can't ignore the other massive proactive catches the AI made regarding cost and safety. It wasn't just throwing code at the wall. Take Azure Bastion, for example.
• 10:14
  Right, the security service. Exactly. Azure Bastion is essentially a highly secure front desk or foyer for a corporate building.
• 10:22
  It lets your administrators securely connect to the internal servers without exposing any of those servers to the public internet. So the author asked the AI about implementing it.
• 10:30
  The AI correctly explained what Bastion did, but then it analyzed the scripts and pointed out that this specific architecture didn't actually need it anymore based on how the networks were configured.
• 10:39
  Deleting it saved $400 a month. Look, saving money is great, but it doesn't offset the risk of downtime. Then let's talk about the ultimate downtime risk, the split brain scenario. Ah, the two captains problem. Exactly.
• 10:53
  A split brain scenario is like having two captains trying to steer the same ship at the exact same time.
• 10:58
  If the network link between East-US and West-US two drops, but both servers are actually still running, they might both assume the other one is dead.
• 11:05
  They both declare themselves the primary captain and start accepting new data. Your information diverges, and putting it back together is an absolute nightmare.
• 11:12
  It's one of the hardest problems in distributed systems, definitely. And the AI independently caught a flaw in its own safety check that would have allowed a split brain scenario to happen during a live check.
• 11:22
  It caught it, and it fixed it before any drill ever ran. The AI isn't perfect, but it actively builds in these deep architectural guardrails that save money and prevent disaster. I hear the argument about the guardrails.
• 11:33
  I really do. But I keep coming back to how this work was actually accomplished, the iterative design process itself.
• 11:40
  You noted earlier that the hands-on time was a few hours, but it was spread over four to five days of testing, questioning, and redesigning. Yes, dozens of rounds of back and forth.
• 11:51
  The author explicitly says their best results came from ask, test, question, refine.
• 11:56
  For a business leader trying to optimize their engineering team's time, doesn't this suggest that AI isn't actually automating the work? It feels like we are merely shifting the burden.
• 12:06
  Instead of spending days writing code, your engineers are now spending days endlessly debugging AI-generated code.
• 12:12
  If you have to spend days interrogating a chatbot to uncover that its math is fundamentally rounding away your problems, are you really gaining efficiency?
• 12:21
  You absolutely are gaining efficiency because you have to look at the sheer magnitude of what was achieved in those four or five days. They didn't just write a basic installation script.
• 12:29
  They built a complete infrastructure as code ecosystem. Meaning it's fully automated? Infrastructure as code is brilliant.
• 12:36
  Instead of swinging hammers and building servers by hand every single time you need an environment, the AI wrote a perfect self-executing blueprint.
• 12:44
  You hand a blueprint to the cloud, and it builds the exact same complex house perfectly over and over again.
• 12:50
  And because they took the time for this iterative process, the final system achieved an RTO, a recovery time objective, of one hundred and seventy point four seconds. So under three minutes to recover.
• 13:00
  From the moment a total regional power failure occurred, the system was back up in a different region writing new data in less than three minutes. And it achieved an RPO, recovery point objective, of zero.
• 13:12
  Zero data loss. Zero failed transactions during a regional meltdown. Okay. Those are impressive numbers. I'll give you that. And it goes beyond just surviving the drill.
• 13:21
  By taking the time to discuss the design with the AI before writing the code, the AI came up with an idea the human hadn't even considered, a replay mode for the dashboard.
• 13:30
  It allowed the author to play back recorded failure drills at five times speed for live management presentations. That means they never had to risk failing over a live region just to show the executives that it worked.
• 13:40
  To build a resilient architecture with that level of sophistication, automated guardrails, and presentation tools would take a senior human engineer weeks of manual work.
• 13:49
  Doing it in a few hours of hands-on time over a week is a monumental leap. It is a monumental leap.
• 13:55
  I, I won't deny the capability, but it is only a leap because the human at the keyboard had years of experience watching databases under pressure.
• 14:03
  The author explicitly states that passing tests are not the same as being right. The AI never powered off a virtual machine or changed a firewall rule without asking the human first.
• 14:13
  It laid out options, you know, like adding a third environment in another region, but the human had to know which of those options was correct for their business.
• 14:20
  If you take away that expert human judgment, the AI speed just means you are going to deploy a broken, disk choking, metric faking system much, much faster.
• 14:29
  And I think that brings us to a very fair takeaway for our listeners today. We're actually in agreement on the fundamental principle here.
• 14:37
  Treating AI as a one prompt, one answer magic wand is a complete recipe for disaster.
• 14:42
  The assumption that passing automated tests guarantees a healthy system is a dangerous illusion when you are dealing with complex infrastructure. We are completely aligned on that.
• 14:51
  The success of this entire workflow is heavily dependent on the user's baseline knowledge.
• 14:57
  If you are operating outside your specific area of expertise and you don't know what normal is supposed to look like, like knowing that replication lag shouldn't be a flat zero during a total outage, you carry a massive risk.
• 15:09
  You risk accepting a beautifully presented, highly professional looking output that is just fundamentally wrong beneath the surface. Right.
• 15:17
  The AI provides the speed, the thoroughness, and the code generation, but human experience is what makes a final result trustworthy. Neither one works in isolation.
• 15:26
  So for everyone listening, whether you're writing the code or managing the teams that do, we wanna leave you with one highly practical step based on this case study. Treat the AI's first answer as a rough draft, always.
• 15:38
  Exactly. When you are dealing with complex calculations, business logic, or critical infrastructure, do exactly what the author did. Explicitly ask the AI to explain its work.
• 15:49
  Say, you know, "Explain to me step by step how you arrived at these numbers," before you accept the solution. If you cannot follow the AI's explanation, that is a massive signal that something is structurally wrong.
• 16:00
  Don't just look at the green check marks on the clipboard. Ask the inspector to show you the structural math for the support beams. Bringing it right back to the skyscraper.
• 16:09
  The robotic crane is faster and more tireless than any human crew, but you absolutely need the seasoned foreman walking the site, questioning the blueprints, and making sure the steel is actually bolted to the foundation.
• 16:19
  We invite all of you to explore these concepts further. You can see the actual dashboards, the failure drill data, and the architectural diagrams by reading the full post. It's available for free at InTechOrNot.com.
• 16:31
  It is definitely worth the read. Definitely. It really demystifies how these tools actually function in a high stakes environment. Thank you for joining us for this discussion.
• 16:39
  Keep questioning the dashboard, always check the math, and we will see you next time on The Divide.