Latest / Future of Work Tech with Fexingo: Remote Tools, AI Productivity, and Workplace Software / How AI Is Automating Your Code Review Process
Transcript
- Lucas: Luna, I have to ask — are you still doing code reviews the old-fashioned way? Line by line, human eyeballs only? Luna: Guilty as charged. I mean, we use linters, but the actual review is still me staring at diffs. Why, is that outdated? Lucas: Not outdated, but there's a growing body of evidence that ai assisted code review tools are catching bugs that humans miss. A study from earlier this year — I think it was from researchers at Google and MIT — found that teams using AI review assistants caught about 30 percent more critical bugs compared to human-only reviews. Luna: Thirty percent is significant. But I've heard complaints about false positives — the AI flagging things that aren't actually problems. Lucas: That's the trade-off. The same study found a false positive rate around 15 to 20 percent. So developers have to sift through some noise. But the argument is that the cost of a false positive — a few seconds to dismiss it — is far lower than the cost of a missed critical bug that makes it to production. Luna: Which tools are we talking about specifically? I know GitHub Copilot has a code review feature now. Lucas: Copilot's code review launched in preview last year, and it's pretty good at catching logic errors and suggesting improvements. Amazon CodeGuru has been around longer — it focuses on security vulnerabilities and performance issues. There's also SonarQube with ai enhanced rules, and newer entrants like CodeRabbit and PullRequest that use large language models to generate review comments. Luna: So the AI is writing the review comments? That feels like a shift from just flagging issues to actually explaining them. Lucas: Exactly. Instead of just a red line, you get a comment that says 'This loop could be optimized by moving the invariant calculation outside,' or 'This SQL query is vulnerable to injection — consider using parameterized statements.' The AI explains the 'why,' which is hugely valuable for junior developers. Luna: That's a big deal for onboarding. New engineers often don't know the team's conventions or the security pitfalls. Lucas: Right. And it scales. A senior developer can only review so many pull requests in a day. An AI tool can review every single one, at least as a first pass, and then the human reviewer focuses on the high-risk or nuanced changes. I've seen teams cut their review cycle time by almost half. Luna: But isn't there a risk that developers become less careful if they know the AI will catch mistakes? Kind of like over-reliance on spellcheck. Lucas: That's a real concern, and it's been studied. The term is 'automation bias' — when people trust the AI too much and stop thinking critically. Some teams mitigate this by using AI review as a suggestion engine, not a gatekeeper. The developer still has to approve every change. Luna: I've also heard that AI review tools can introduce their own biases — like favoring certain coding styles or ignoring edge cases in less common languages. Lucas: Absolutely. Most models are trained on open-source code, which is overwhelmingly in Python, JavaScript, and Java. If you're writing in, say, Rust or Go, the quality of suggestions drops. And the training data itself has biases — it might over-represent certain design patterns or under-represent secure coding practices from older codebases. Luna: So it's not a silver bullet. But if used smartly, it's a force multiplier. Lucas: That's the framing I'd use. One of the more interesting implementations I've seen is at a fintech company called Mercury — they built their own AI review bot that not only checks code but also cross-references it with regulatory compliance rules. So it catches a potential PCI violation before the code even gets to a human. Luna: That's smart. In regulated industries, that kind of automated compliance check could save millions in fines. Lucas: On the flip side, if the AI misses something and the human assumes the AI checked it, you have a blind spot. So the team at Mercury still requires a human sign-off on every compliance-related review. Luna: How do you see this evolving over the next couple of years? Will we get to a point where AI does most of the review and humans only step in for the trickiest cases? Lucas: I think we'll get close, but not fully. The really nuanced decisions — architecture trade-offs, whether a design pattern is appropriate for future scalability — those require human judgment. But the mechanical stuff, like checking for null pointers, off-by-one errors, or known vulnerability patterns, AI will handle nearly all of it. Luna: And that frees up senior engineers to do more mentoring and architecture work, which is probably a better use of their time anyway. Lucas: Exactly. One senior engineer I talked to said he used to spend two to three hours a day on code reviews. Now he spends about forty-five minutes, and the rest goes into pair programming with juniors or designing the next big feature. Luna: That's a compelling argument for adoption. But I'm sure there are teams where the culture hasn't caught up — managers who see AI review as a replacement rather than a tool. Lucas: That's the risk. If a manager says, 'We have AI review, so we don't need as many senior devs,' they're missing the point. The AI can't replace the institutional knowledge and judgment of an experienced engineer. It can only augment it. Luna: So the human element remains crucial. But the tools are getting better fast. I read that OpenAI's latest model, GPT-5, scores in the 95th percentile on coding challenges. That's going to impact code review too. Lucas: It already is. Some teams are using GPT-5 to generate unit tests based on the code changes, and then reviewing those tests. That's another layer of automation. The review process isn't just about the code itself — it's about the surrounding artifacts. Luna: Have you personally tried any of these tools on your own projects? You code a bit on the side, right? Lucas: I do, and I've been using Copilot's code review on a small side project. Honestly, it caught a couple of things I would have missed — a potential race condition in a multi-threaded section, and a deprecated API call I didn't realize was deprecated. The explanations were clear enough that I learned something. Luna: That's the ideal outcome — not just catching bugs, but teaching the developer. If AI review can do that at scale, it's a net positive for the industry. Lucas: And you know, that kind of value is exactly why we keep digging into these tools on the show. Speaking of which — quick honest thing. A handful of listeners chip in monthly through buy me a coffee dot com slash fexingo, and that's literally what funds making this many episodes. No ads, no sponsors — just listener support. Luna: Yeah, it's a small group, but they're the reason we can stay independent and focus on what's actually useful, not what an advertiser wants us to talk about. Lucas: If today's conversation gave you something usable, consider joining them. It's a small way to keep the show ad-free. But back to code reviews — I think the key takeaway is that AI is not replacing the reviewer, it's giving the reviewer superpowers. Luna: Superpowers with a side of false positives. But I'll take that trade-off. Lucas: Same. So next time you open a pull request, maybe let an AI take a first pass. You might be surprised at what it catches. Luna: I'll give it a try. Just don't tell my team I'm outsourcing my reviews to a bot. Lucas: Your secret's safe with us.