Latest / Mind Cast / Code is the Key — Building Verifiably Correct AI

Transcript

Code is the Key — Building Verifiably Correct AI · Mind Cast

• 0:00
  SPEAKER_00: Here's a number that stopped me cold.
• 0:02
  SPEAKER_00: 35.9%.
• 0:05
  SPEAKER_00: That's the score an AI model got on a standard math reasoning benchmark, a test designed to measure whether a machine can actually think through a problem step by step.
• 0:15
  SPEAKER_00: Not great, right?
• 0:16
  SPEAKER_00: Now here's the second number: 88.8%.
• 0:20
  SPEAKER_00: Same model, same benchmark, same underlying architecture.
• 0:24
  SPEAKER_00: The only thing that changed, how that model was trained to distinguish good reasoning from bad.
• 0:30
  SPEAKER_00: That is a fundamental rethinking of what it means to teach an AI to think without adding a single new neuron to the network.
• 0:39
  SPEAKER_00: If that doesn't make you ask, wait, how is that even possible?
• 0:43
  SPEAKER_00: Stick around because that is exactly what we're going to unpack today.
• 0:50
  SPEAKER_00: Welcome to Mindcast, the show where we take the most complex ideas shaping our world and make them genuinely understandable without dumbing them down.
• 1:00
  SPEAKER_00: I'm Will, and today's episode is one I've been genuinely excited to record.
• 1:06
  SPEAKER_00: We're diving deep into a research report that came across my desk, and I have to be up front, this report is not publicly available.
• 1:14
  SPEAKER_00: It's exclusive academic research.
• 1:17
  SPEAKER_00: What you're getting today is a synthesis and my own interpretation of cutting-edge thinking you won't find anywhere else.
• 1:25
  SPEAKER_00: The report is titled, Brace Yourself, Epistemological Verification in Code Intelligence: a Comparative Analysis of Pre-training curation, preference boundaries, and automated theorem proving.
• 1:41
  SPEAKER_00: That's a mouthful, but here's what it's really about.
• 1:44
  SPEAKER_00: How do we build AI systems that don't just generate plausible sounding answers, but answers we can actually verify are correct?
• 1:53
  SPEAKER_00: And the researchers answer?
• 1:55
  SPEAKER_00: Code.
• 1:56
  SPEAKER_00: Code is the key.
• 1:58
  SPEAKER_00: By the end of this episode, you'll understand three things.
• 2:01
  SPEAKER_00: Why programming languages have a unique epistemological advantage over human language, how researchers are solving the garbage in, garbage out problem with borderline astonishing results, and how AI is beginning to mathematically prove that code is correct.
• 2:18
  SPEAKER_00: Let's go.
• 2:21
  SPEAKER_00: Key insight 1.
• 2:23
  SPEAKER_00: The epistemological advantage of code.
• 2:26
  SPEAKER_00: Let's start with the foundational idea, the philosophical core everything else in this report is built on.
• 2:33
  SPEAKER_00: When you train an AI on human language, scraped from the internet, books, social media, you are training it on text that is at best approximately true.
• 2:44
  SPEAKER_00: A Wikipedia article gets a fact slightly wrong.
• 2:47
  SPEAKER_00: A Stack Overflow answer is outdated.
• 2:49
  SPEAKER_00: A blog post confidently states something incorrect.
• 2:53
  SPEAKER_00: The AI doesn't know the difference.
• 2:55
  SPEAKER_00: It learns the patterns.
• 2:57
  SPEAKER_00: And this is the root cause of hallucinations.
• 2:59
  SPEAKER_00: When an AI confidently tells you something completely made up, it literally cannot tell the difference between a true statement and a false one.
• 3:08
  SPEAKER_00: Here's where code enters the picture.
• 3:11
  SPEAKER_00: Code exists in what researchers call a closed world system.
• 3:16
  SPEAKER_00: There is a final objective arbiter of truth.
• 3:19
  SPEAKER_00: You run the code.
• 3:21
  SPEAKER_00: Either it compiles or it doesn't.
• 3:23
  SPEAKER_00: Either the tests pass or they fail.
• 3:26
  SPEAKER_00: There is no ambiguity.
• 3:29
  SPEAKER_00: That is an extraordinary property for training in AI.
• 3:33
  SPEAKER_00: Every training example can be verified.
• 3:36
  SPEAKER_00: The universe of code has a built-in ground truth mechanism, and that changes everything.
• 3:43
  SPEAKER_00: Now, Dr.
• 3:44
  SPEAKER_00: Venkitz Subermanium, a respected software educator, calls AI in its current form an accelerated inference engine, not a thinking machine.
• 3:55
  SPEAKER_00: An inference engine, extraordinarily good at pattern matching at superhuman speed, but not what we'd call thinking.
• 4:04
  SPEAKER_00: His historical analogy, when the automobile was invented, people attached them to horse-drawn carriage bodies.
• 4:10
  SPEAKER_00: It took decades before people designed cars as cars.
• 4:14
  SPEAKER_00: We are in the horseless carriage phase of AI.
• 4:17
  SPEAKER_00: From this, he offers four core guidelines for developers.
• 4:22
  SPEAKER_00: Ideation over commitment.
• 4:24
  SPEAKER_00: Use AI for brainstorming, but don't commit to its first answer.
• 4:29
  SPEAKER_00: Verify.
• 4:30
  SPEAKER_00: Guideline two, amplification of ignorance.
• 4:35
  SPEAKER_00: AI makes a novice feel like an expert, and that's dangerous.
• 4:40
  SPEAKER_00: If you can't recognize bad code, AI will generate it confidently and you won't catch it.
• 4:47
  SPEAKER_00: Guideline 3.
• 4:49
  SPEAKER_00: Tooling over agency.
• 4:51
  SPEAKER_00: Use AI as a tool, not an autonomous agent.
• 4:55
  SPEAKER_00: Keep humans in the decision loop.
• 4:58
  SPEAKER_00: Guideline 4.
• 5:00
  SPEAKER_00: Evolution of the compiler.
• 5:02
  SPEAKER_00: Future compilers won't just check syntax, they'll check intent, security, and logic.
• 5:09
  SPEAKER_00: AI will be part of that evolution.
• 5:13
  SPEAKER_00: One final point.
• 5:14
  SPEAKER_00: The report uses the allegory of who moved my cheese for programmers.
• 5:19
  SPEAKER_00: The cheese, the economic value of syntax knowledge, memorizing APIs, knowing which function to call, has moved.
• 5:28
  SPEAKER_00: AI is very good at the syntax.
• 5:30
  SPEAKER_00: If your value as a developer lives primarily there, that value is under pressure.
• 5:36
  SPEAKER_00: The developers who thrive will be the ones who move toward higher order thinking, system design, and problem framing.
• 5:45
  SPEAKER_00: Key insight 2.
• 5:47
  SPEAKER_00: Solving garbage in, garbage out.
• 5:50
  SPEAKER_00: We've established that code has a beautiful closed world property.
• 5:54
  SPEAKER_00: You can verify it.
• 5:55
  SPEAKER_00: But here's the catch.
• 5:57
  SPEAKER_00: The internet is swimming in bad code.
• 6:00
  SPEAKER_00: Think about every buggy script ever posted to a forum, every Stack Overflow answer from 2009 using deprecated libraries, every tutorial written by someone who half understood what they were teaching.
• 6:13
  SPEAKER_00: Train your AI on that indiscriminately, and you get an AI that writes confidently broken code because that's what most of the training data looks like.
• 6:23
  SPEAKER_00: This is the garbage-in-garbage out problem, and the researchers address it with a three-tier curation hierarchy.
• 6:30
  SPEAKER_00: Layer 1.
• 6:31
  SPEAKER_00: Human compliance filters.
• 6:34
  SPEAKER_00: Humans define broad rules, filter out problematic licenses, remove obvious spam, exclude non-executable files.
• 6:44
  SPEAKER_00: A coarse sieve that removes the worst garbage but can't scale to billions of lines of code.
• 6:51
  SPEAKER_00: Layer 2.
• 6:52
  SPEAKER_00: Rule-based heuristics.
• 6:55
  SPEAKER_00: Programmatic rules, flag code with excessive comments, filter files below a certain size, exclude code that fails a basic syntax check.
• 7:05
  SPEAKER_00: Fast and cheap at scale, but still blunt instruments.
• 7:10
  SPEAKER_00: A rule can tell you a file is syntactically valid.
• 7:13
  SPEAKER_00: It cannot tell you whether the logic is elegant or genuinely useful.
• 7:18
  SPEAKER_00: Layer 3.
• 7:21
  SPEAKER_00: Researchers train a small, specialized quality scoring model whose only job is to evaluate whether code is high quality.
• 7:29
  SPEAKER_00: They use a more powerful model, in some cases GPT-4, to generate quality labels on a curated sample, then train the smaller score on those labels.
• 7:40
  SPEAKER_00: You're using AI to teach AI what good code looks like.
• 7:45
  SPEAKER_00: With clean data in hand, the report introduces direct preference optimization, or DPO.
• 7:52
  SPEAKER_00: Standard training shows the model good examples and have it learn to replicate them.
• 7:58
  SPEAKER_00: DPO goes further, show it pairs of outputs, one good and one bad, and train it to prefer the good one while explicitly penalizing the pattern of the bad one.
• 8:09
  SPEAKER_00: I think of it as subtractive optimization, not just a pull toward the correct answer, but a push away from the wrong answer.
• 8:17
  SPEAKER_00: The researchers extended this with zero-shot contrastive prompting, prompting the AI to generate both a high-quality and a deliberately bad version of the same solution, using those pairs as preference training data.
• 8:32
  SPEAKER_00: Zero human labeling required.
• 8:35
  SPEAKER_00: And the results?
• 8:36
  SPEAKER_00: On the GSM 8K benchmark, the model score jumped from 35.9 to 88.8%, a 52.9 percentage point improvement.
• 8:48
  SPEAKER_00: Same model, just trained differently on better curated data with smarter preference learning.
• 8:54
  SPEAKER_00: Human Eval, which measures direct coding ability, improved by 5.5 percentage points.
• 9:01
  SPEAKER_00: This completely upends the conventional wisdom that more data equals better AI.
• 9:06
  SPEAKER_00: It's not just about more, it's about better, teaching the model the shape of correctness and the shape of failure simultaneously.
• 9:16
  SPEAKER_00: Now we reach the most forward-looking section, and I genuinely think we are watching history being made in real time.
• 9:24
  SPEAKER_00: What if, instead of just training an AI to write code, you put that AI inside a closed feedback loop with the very machine that can tell it whether the code works?
• 9:35
  SPEAKER_00: Here's what that looks like in practice.
• 9:38
  SPEAKER_00: The AI generates some code, you compile it, if it fails, you capture the error message and feed it back to the AI.
• 9:46
  SPEAKER_00: The AI tries again.
• 9:48
  SPEAKER_00: You run the tests, if they fail, you feed back the output.
• 9:53
  SPEAKER_00: The AI refines.
• 9:54
  SPEAKER_00: Again and again.
• 9:56
  SPEAKER_00: This is called a sandboxed execution loop, implemented in frameworks like Reposit, Cycle, and commercial systems like Supercoder 2.0.
• 10:07
  SPEAKER_00: The numbers are stunning.
• 10:09
  SPEAKER_00: Deep Seat Coder V2 starts with a fixed success rate of 9 to 20%, correctly fixing broken code less than 1 in 5 times on the first try.
• 10:20
  SPEAKER_00: After 10 cycles in this refinement loop, that rate rises to over 99%, not through any change to the model itself, just through iterative self-correction within a verifiable environment.
• 10:34
  SPEAKER_00: The closed world property of code makes this possible.
• 10:37
  SPEAKER_00: Nothing analogous exists for natural language.
• 10:40
  SPEAKER_00: You can't compile an essay.
• 10:44
  SPEAKER_00: But the researchers don't stop there, and this is where we enter genuinely science fiction territory, except it's happening right now.
• 10:53
  SPEAKER_00: What if AI could not just write code that works, but mathematically prove that it works?
• 11:00
  SPEAKER_00: Not test it, prove it, with the same certainty that 2 plus 2 equals 4.
• 11:06
  SPEAKER_00: Formal verification languages like Lean, Daphne, and FSTAR allow you to write not just code, but a formal specification of what the code should do, alongside a mathematical proof that the code satisfies it.
• 11:22
  SPEAKER_00: The marriage of generative AI with these formal proof systems creates what the report calls a zero-trust self-verifying loop.
• 11:32
  SPEAKER_00: You start with a specification, a precise mathematical description of what your software must do.
• 11:39
  SPEAKER_00: The AI generates two things simultaneously, the code itself and a formal proof that the code satisfies the specification.
• 11:48
  SPEAKER_00: An independent proof checker with no hallucinations, no off days, certifies the proof.
• 11:54
  SPEAKER_00: A verified compiler, proven to translate code correctly, completes the chain.
• 12:00
  SPEAKER_00: Every link mathematically guaranteed.
• 12:03
  SPEAKER_00: Here's the concrete moment that makes this real.
• 12:06
  SPEAKER_00: Clawed code was recently applied to a formal verification task, and in a matter of days, generated over 7,800 lines of machine-checked formal proof, work that would have previously taken a team of formal methods experts weeks, potentially months.
• 12:25
  SPEAKER_00: And the report points to what's coming next: energy-based reasoning models, or EBRMs.
• 12:32
  SPEAKER_00: Instead of directly generating an output, the model learns an energy function, a scoring system, over the space of possible outputs, searching for the solution most consistent with all known constraints.
• 12:47
  SPEAKER_00: This is better suited for tasks requiring global consistency, where every piece of the solution must fit together perfectly, exactly what formal proof demands.
• 12:58
  SPEAKER_00: We are watching, in real time, the emergence of AI systems that can verify their own work at a level of rigor that humans simply cannot match consistently.
• 13:12
  SPEAKER_00: Okay, let me pull all the threads together and give you three concrete takeaways, things you can actually do with this information starting now.
• 13:23
  SPEAKER_00: Takeaway number one, your job is changing.
• 13:27
  SPEAKER_00: Don't fight it, redirect.
• 13:29
  SPEAKER_00: The report is unambiguous.
• 13:31
  SPEAKER_00: The parts of software development that involve memorizing syntax, knowing which library function to call, translating a well-understood requirement into code, these are being automated.
• 13:43
  SPEAKER_00: Not tomorrow, now.
• 13:45
  SPEAKER_00: If your entire value proposition is I know how to write the code, you are in the same position as a typist who defined their value as I type fast when word processors arrived.
• 13:58
  SPEAKER_00: The developers who will thrive are the ones who can do what AI still does poorly, understand the context of a problem, challenge the assumptions behind a requirement, design systems at a level of abstraction above the code itself, and critically evaluate AI output for subtle logical flaws that pass the syntax checker but fail in production.
• 14:20
  SPEAKER_00: Double down on those skills.
• 14:23
  SPEAKER_00: Takeaway number two.
• 14:39
  SPEAKER_00: Most developers think in terms of write the code that does the thing rather than formally define what the thing must do.
• 14:47
  SPEAKER_00: But the ability to write precise, unambiguous, machine-checkable requirements and to validate that AI output actually satisfies them is going to be an extraordinarily rare and valuable skill.
• 15:00
  SPEAKER_00: Explore languages like Daphne, even at a surface level.
• 15:04
  SPEAKER_00: Start thinking in terms of preconditions, post-conditions, and invariance.
• 15:09
  SPEAKER_00: You don't have to become a formal methods expert overnight, but you need to start.
• 15:15
  SPEAKER_00: Takeaway number three.
• 15:17
  SPEAKER_00: AI plus formal verification equals the future of trustworthy software, and you need to understand why that matters.
• 15:25
  SPEAKER_00: Think about the domains where software failure is catastrophic.
• 15:30
  SPEAKER_00: Medical devices, financial systems executing trades worth hundreds of millions in microseconds, smart contracts on a blockchain where a bug causes an irreversible transfer that can never be undone, autonomous vehicles, air traffic control.
• 15:46
  SPEAKER_00: In all of these domains, we tested it and it seemed fine is not acceptable.
• 15:51
  SPEAKER_00: And yet that has been essentially the standard.
• 15:54
  SPEAKER_00: The marriage of AI and formal verification changes that equation.
• 16:00
  SPEAKER_00: For the first time, it is becoming feasible to build software in critical domains with mathematical certainty that it behaves as intended.
• 16:10
  SPEAKER_00: Not probabilistic confidence, mathematical certainty.
• 16:15
  SPEAKER_00: That is a paradigm shift.
• 16:17
  SPEAKER_00: And the people who can bridge the gap between the AI tools and the mathematical frameworks will be essential.
• 16:26
  SPEAKER_00: Let's bring it home.
• 16:28
  SPEAKER_00: Today on Minecast, we took a deep dive into one of the most consequential questions in AI research.
• 16:34
  SPEAKER_00: How do we build AI systems that don't just sound right but are verifiably correct?
• 16:40
  SPEAKER_00: We explored that question through the lens of code intelligence, the ideal laboratory for this work, precisely because code lives in a closed world where truth is testable.
• 16:50
  SPEAKER_00: We started with the fundamental insight that code's binary nature gives AI a verifiable ground truth that natural language simply doesn't.
• 16:59
  SPEAKER_00: We explored Dr.
• 17:00
  SPEAKER_00: Van Kat's Subermanium's warning that AI is an accelerated inference engine, not a replacement for human judgment, and his four guidelines for navigating this moment as a developer.
• 17:12
  SPEAKER_00: We unpacked the three-tier data curation hierarchy and the elegant logic of direct preference optimization, teaching a model not just what good looks like, but what bad looks like and how to avoid it, producing that extraordinary jump from 35.9 to 88.8%.
• 17:30
  SPEAKER_00: And we looked at the frontier, sandboxed execution loops that take a 9% fixed success rate to 99% through iterative self-correction, and the emerging marriage of generative AI with formal mathematical proof, where systems like Claude Code are generating thousands of lines of machine-checked proof in days.
• 17:52
  SPEAKER_00: One important note before I let you go the research we drew on today, epistemological verification in code intelligence, is not publicly available.
• 18:02
  SPEAKER_00: It is exclusive research, and what you heard represents my synthesis and analysis.
• 18:08
  SPEAKER_00: You won't find this breakdown anywhere else.
• 18:11
  SPEAKER_00: If this episode sparked something in you, please do three things.
• 18:15
  SPEAKER_00: First, subscribe to Mindcast wherever you listen.
• 18:19
  SPEAKER_00: We do this every episode.
• 18:21
  SPEAKER_00: Take the hard stuff, make it real, and give you something you can actually use.
• 18:26
  SPEAKER_00: Second, leave us a review.
• 18:28
  SPEAKER_00: It takes 60 seconds and it means the world.
• 18:31
  SPEAKER_00: It's how other curious people find this show.
• 18:34
  SPEAKER_00: And third, think of one developer, one technologist, one curious person in your life who would benefit from hearing this.
• 18:42
  SPEAKER_00: Send them this episode.
• 18:44
  SPEAKER_00: The ideas we talked about today, the changing nature of programming work, the rise of specification engineering, the path to formally verified software.
• 18:54
  SPEAKER_00: These are happening right now.
• 18:57
  SPEAKER_00: The people in your life who work in tech need to be thinking about them.
• 19:06
  SPEAKER_00: And I'll see you next time.