Latest / DX Today | No-Hype Podcast & News About AI & DX / AI Reviewers Found So Many Bugs That Postgres 19 Slipped a Month

Transcript

AI Reviewers Found So Many Bugs That Postgres 19 Slipped a Month · DX Today | No-Hype Podcast & News About AI & DX

• 0:00
  SPEAKER_01: Welcome to the DX Today Podcast, your daily deep dive into the AI ecosystem.
• 0:04
  SPEAKER_01: I'm Chris, and joining me as always is Laura.
• 0:07
  SPEAKER_00: Hi, Chris, and hello everyone.
• 0:09
  SPEAKER_00: Today we have a story that I think is one of the most quietly important things to happen in software all year, and it is about a database release that just got pushed back.
• 0:18
  SPEAKER_01: A database release getting delayed does not usually make headlines, so I am going to need you to convince our listeners why a slipped launch date for Postgres deserves a whole episode today.
• 0:27
  SPEAKER_00: Fair challenge.
• 0:28
  SPEAKER_00: Postgres version 19 was supposed to ship in late September or early October, and now it is expected around November 19th.
• 0:35
  SPEAKER_00: The reason is the twist.
• 0:37
  SPEAKER_00: AI code reviewers found more bugs than the human developers could fix in time.
• 0:41
  SPEAKER_01: Okay, that is a genuinely great hook.
• 0:44
  SPEAKER_01: So the robots did not write broken code here.
• 0:46
  SPEAKER_01: The robots were the ones reading the code and pointing out all the problems that people had missed.
• 0:51
  SPEAKER_00: Exactly right.
• 0:52
  SPEAKER_00: And this was not some tiny side project.
• 0:55
  SPEAKER_00: Between eight and ten significant features were pulled out of the release and pushed to version 20, which is next year's release.
• 1:01
  SPEAKER_00: One independent count found 13 removed items covering 11 features.
• 1:05
  SPEAKER_01: Before we go deeper, give people a little context on what Postgres actually is.
• 1:10
  SPEAKER_01: Because some listeners live in databases every day, and others have probably never thought about one in their lives.
• 1:15
  SPEAKER_00: Postgres is an open source relational database, basically the place where a huge amount of the world's application data lives.
• 1:22
  SPEAKER_00: Michael Stonebreaker first proposed it back in 1986, and today it is the most popular database among professional developers.
• 1:30
  SPEAKER_01: And every big cloud provider sells some managed version of it, right?
• 1:33
  SPEAKER_01: So when something changes in how Postgres gets built and released, that ripples out to a staggering number of companies and products.
• 1:40
  SPEAKER_00: That is right.
• 1:41
  SPEAKER_00: All three of the big hyperscalers offer it, and there is a whole family of distributed systems that speak its wire protocol.
• 1:48
  SPEAKER_00: So the quality bar for this project is very high, and the people who maintain it take that responsibility seriously.
• 1:55
  SPEAKER_01: So who is the person explaining what happened here?
• 1:57
  SPEAKER_01: Because I assume somebody inside the project had to stand up and say, look, here's why we are pulling all of these features.
• 2:04
  SPEAKER_00: That person is Peter Eisentrout, a vice president at EDB and one of the leading Postgres committers.
• 2:11
  SPEAKER_00: On October 9th, he wrote a blog post with a wonderful title.
• 2:17
  SPEAKER_01: I love that title.
• 2:18
  SPEAKER_01: It is Gallows Humor from someone who clearly had a rough few months.
• 2:23
  SPEAKER_01: What did he actually say was going on in plain terms that a non-database person can follow?
• 2:28
  SPEAKER_00: He gave two main reasons.
• 2:30
  SPEAKER_00: First, some of the reverted features had significant architectural defects that simply could not be fixed quickly during the beta period.
• 2:38
  SPEAKER_00: Second, and this is the AI part, language model-assisted review surfaced a long tail of small edge case bugs.
• 2:44
  SPEAKER_01: Let me push on that second point a bit.
• 2:46
  SPEAKER_01: Every release has edge case bugs.
• 2:49
  SPEAKER_01: What makes these different?
• 2:55
  SPEAKER_00: That is the heart of it.
• 2:57
  SPEAKER_00: Eisen Trout said these are the kinds of issues that would previously have been found over years.
• 3:02
  SPEAKER_00: Often in the bug fix releases during the first five years after launch, the AI tools compressed years of discovery into a few months.
• 3:10
  SPEAKER_01: So in the old world, users would quietly find these bugs one at a time over half a decade, and the team would fix them gradually.
• 3:17
  SPEAKER_01: Now a machine dumps them all on the table before launch day.
• 3:20
  SPEAKER_00: Yes, and he described the workload problem really vividly.
• 3:24
  SPEAKER_00: Imagine a list of 40 defect reports where each one needs roughly a three-line fix on average.
• 3:30
  SPEAKER_00: That sounds easy, but every single one still carries fixed overhead in review, testing, and discussion.
• 3:36
  SPEAKER_01: Right, the fix itself might be three lines, but understanding the bug, agreeing on the approach, writing a test and getting another committer to sign off probably takes a day or more each time.
• 3:46
  SPEAKER_00: Exactly.
• 3:47
  SPEAKER_00: And that overhead eats all the available time.
• 3:50
  SPEAKER_00: There was another pressure, too.
• 3:52
  SPEAKER_00: From April to August, many senior developers were busy on security work.
• 3:56
  SPEAKER_00: One analysis counted 58 commits tagged with security fixes since April, covering 40 separate CV vulnerabilities.
• 4:04
  SPEAKER_01: Wow.
• 4:05
  SPEAKER_01: So the most experienced people were already stretched thin fixing security holes, and then a wave of AI-generated bug reports arrived, right when they were trying to stabilize the beta.
• 4:15
  SPEAKER_01: That is a brutal combination.
• 4:17
  SPEAKER_00: It really is.
• 4:18
  SPEAKER_00: And Eisentraut was careful to say this is not a story about vibe coding.
• 4:23
  SPEAKER_00: The reverted features were mostly written before that technique was even available.
• 4:27
  SPEAKER_00: The code was written by humans.
• 4:29
  SPEAKER_00: The machines just raise the bar for what counts as good enough.
• 4:33
  SPEAKER_01: That distinction matters because I can already hear people saying AI ruined Postgres.
• 4:38
  SPEAKER_01: It sounds like the opposite.
• 4:40
  SPEAKER_01: The AI made the project more honest about bugs that were always there in the code.
• 4:45
  SPEAKER_00: That is how I read it too.
• 4:46
  SPEAKER_00: He called version 19 a great release and said it will be very robust after all the extra scrutiny.
• 4:52
  SPEAKER_00: Fewer features, but a lot more confidence that what ships actually works the way it should.
• 4:57
  SPEAKER_01: Let us talk about what actually got cut, because I imagine some of these were features people were really excited about and maybe even planning their own projects around.
• 5:06
  SPEAKER_00: The biggest one is property graph queries, a feature that became part of the official database language standard in 2023.
• 5:13
  SPEAKER_00: It lets you explore relationships between connected things, like people in a social network or parts in a supply chain.
• 5:20
  SPEAKER_01: That sounds like a big deal for anyone who wanted graph style queries without running a separate graph database.
• 5:25
  SPEAKER_01: Why was it pulled and was it the AI reviewers that sank it?
• 5:29
  SPEAKER_00: It was polled on September 7th over open design questions.
• 5:33
  SPEAKER_00: Things like orphaned graph metadata after dropping objects and problems with backup dependencies.
• 5:40
  SPEAKER_00: Earlier, longtime contributor Tom Lane warned that its bugs would likely be unfixable until version 20 and said he would bet dinner on it.
• 5:48
  SPEAKER_01: I appreciate a database legend making dinner bets.
• 5:51
  SPEAKER_01: What else came out?
• 5:52
  SPEAKER_01: Give me a couple of the more practical ones that working developers might actually notice when they upgrade.
• 6:14
  SPEAKER_01: Lost updates and broken replication are exactly the kind of bugs that ruin someone's weekend.
• 6:18
  SPEAKER_01: So honestly, I am glad those did not ship.
• 6:21
  SPEAKER_01: Was anything big kept in the release, despite all the trouble?
• 6:25
  SPEAKER_00: Yes, a feature called Repack concurrently survived.
• 6:28
  SPEAKER_00: It lets you reclaim disk space without locking the whole table, which is a real quality of life improvement for anyone running large production databases.
• 6:37
  SPEAKER_00: It was trimmed back a little, but it stayed in the release.
• 6:40
  SPEAKER_01: There was also a smaller feature called group by all that got pulled, right?
• 6:44
  SPEAKER_01: I remember people liking that one because it saves typing out every column you want to group on in a query.
• 6:50
  SPEAKER_00: Yes, it was pulled because it could return wrong results in certain sorting situations, and a clean fix looked like too much change for late beta.
• 6:58
  SPEAKER_00: A wrong answer from a database is about the worst bug there is, so caution makes sense.
• 7:03
  SPEAKER_01: Here's where I want to play devil's advocate.
• 7:06
  SPEAKER_01: Is there a risk that AI reviewers create noise?
• 7:09
  SPEAKER_01: If a tool floods maintainers with dozens of reports, some of those might be wrong, trivial, or not worth the time.
• 7:15
  SPEAKER_00: That risk is real, and one commentator called it the new bottleneck, deciding which AI findings matter.
• 7:21
  SPEAKER_00: But in this case, the findings were mostly valid.
• 7:24
  SPEAKER_00: The problem was not bad reports.
• 7:26
  SPEAKER_00: The problem was too many good reports arriving at once with too few people to handle them.
• 7:31
  SPEAKER_01: That flips the usual story about AI and software.
• 7:34
  SPEAKER_01: We spend so much time talking about AI writing code faster, but here the scarce resource is human judgment about which fixes to make and in what order.
• 7:44
  SPEAKER_00: And there is a fascinating detail on that.
• 7:47
  SPEAKER_00: Back on August 25th, another senior contributor, Robert Haas, actually used a language model to tally the fixes made after the feature freeze, feature by feature, to see where the trouble was concentrated.
• 7:59
  SPEAKER_01: So they were using AI to understand the problem that AI helped uncover.
• 8:04
  SPEAKER_01: That is very recursive.
• 8:06
  SPEAKER_01: What did the tally show?
• 8:08
  SPEAKER_01: And did it line up with the features that eventually got reverted?
• 8:11
  SPEAKER_00: Pretty closely.
• 8:12
  SPEAKER_00: Repack had 28 fixes and was kept.
• 8:15
  SPEAKER_00: Online checksums had 25 and were reverted.
• 8:18
  SPEAKER_00: Temporal updates and property graphs had 17 each and were both reverted.
• 8:23
  SPEAKER_00: The counts were not audited, but they gave the team a useful map of the risk.
• 8:27
  SPEAKER_01: How much AI is actually showing up in the Postgres code itself, though?
• 8:31
  SPEAKER_01: Is this a project where machines are now writing big chunks of the patches, or is it still mostly humans?
• 8:37
  SPEAKER_00: Mostly humans.
• 8:38
  SPEAKER_00: One analysis found that only 23 commits after the first beta out of almost 1,000 explicitly credit help from AI, a tool, or an AI lab.
• 8:49
  SPEAKER_00: That is a small share, and it is a floor because it only counts explicit credit.
• 8:53
  SPEAKER_01: So the AI influence is much bigger in reviewing than in writing, at least for now.
• 8:58
  SPEAKER_01: That feels like a healthy order of operations for a project where correctness matters more than speed.
• 9:04
  SPEAKER_00: I think so too, and it is a model other projects should study.
• 9:07
  SPEAKER_00: Reviewing is where these tools can add value without adding the risk of subtle new bugs.
• 9:12
  SPEAKER_00: A reviewer that never gets tired can read every edge case humans skip.
• 9:16
  SPEAKER_01: I want to bring up the maintainers themselves, because open source projects run on a surprisingly small number of very experienced volunteers and paid contributors.
• 9:25
  SPEAKER_01: Does a flood like this risk burning those people out?
• 9:28
  SPEAKER_00: It absolutely can.
• 9:30
  SPEAKER_00: Other open source projects have complained about low-quality AI-generated bug reports wasting maintainer time, and the Curl project became a famous example of that frustration.
• 9:40
  SPEAKER_00: The Postgres case is different because the reports were good, but good reports still cost hours.
• 9:45
  SPEAKER_01: So the question for the whole industry becomes who pays for the human review time?
• 9:49
  SPEAKER_01: If big companies point powerful AIA reviewers at open source code, they probably also need to fund the people who have to act on those findings.
• 9:57
  SPEAKER_00: That is exactly the right question.
• 9:59
  SPEAKER_00: And it is one that cloud providers and database vendors should take seriously.
• 10:04
  SPEAKER_00: They profit enormously from Postgres, sending more engineers to help triage and fix what the AI tools find would be the most direct way to give back.
• 10:12
  SPEAKER_01: Let me zoom out to the business side.
• 10:14
  SPEAKER_01: If you're a company running thousands of Postgres databases, what does this delay actually mean for your plans this quarter and next year?
• 10:21
  SPEAKER_00: Short term, you wait until late November and the team plans a release candidate on November 12th.
• 10:26
  SPEAKER_00: If you are counting on graph queries or partition merging, you will need workarounds like recursive queries or a graph extension until version 20 arrives.
• 10:35
  SPEAKER_01: And version 20 becomes a bigger release than anyone expected, right?
• 10:39
  SPEAKER_01: Because now it inherits all these features that were pulled out of 19 at the last minute.
• 10:44
  SPEAKER_00: Exactly.
• 10:45
  SPEAKER_00: A few months ago, developers expected version 20 to be mostly bug fixes with no major new features.
• 10:52
  SPEAKER_00: Now EisenTrout expects it to include many of the leftovers, although nothing is formally promised yet by the project.
• 11:23
  SPEAKER_01: That makes sense.
• 11:24
  SPEAKER_01: The pain came from changing the rules midway through the game.
• 11:27
  SPEAKER_01: Once the higher quality bar is there from the start, people write and review code against it the whole way through.
• 11:33
  SPEAKER_00: And I think that lesson goes far beyond databases.
• 11:37
  SPEAKER_00: Any team adopting AI code review should expect a backlog shock the first time they run it on older code.
• 11:43
  SPEAKER_00: Budget human time for triage, or you end up exactly where Postgres was this fall.
• 11:48
  SPEAKER_01: My takeaway is that the headline sounds like AI slowed things down.
• 11:52
  SPEAKER_01: But the real story is AI made a critical piece of infrastructure safer.
• 11:57
  SPEAKER_01: And the humans made the hard call to ship less and ship it right.
• 12:00
  SPEAKER_00: Agreed.
• 12:01
  SPEAKER_00: Fewer features in November, but a more robust database that millions of applications depend on.
• 12:07
  SPEAKER_00: Honestly, if every delayed launch came with that trade off, I think most engineering teams would happily take the deal.
• 12:13
  SPEAKER_01: That's all for today's episode of the DX Today podcast.
• 12:16
  SPEAKER_01: Thanks for listening, and we'll see you next time.