Latest / Braid / Six Hundred and Fifty Discarded Proofs

Transcript

Six Hundred and Fifty Discarded Proofs · Braid

• 0:04
  lenar: A developer at Anthropic pointed an unreleased model at the Riemann hypothesis, and before it produced anything a mathematician would agree to read, the system threw away six hundred and fifty invalid proofs. That run went thirty-six hours. It ran sixty sub-agents inside Claude Code, and along the way it issued roughly twenty-four hundred shell commands and burned thirty-one million output tokens. The accounting comes from a Fireship video posted yesterday evening, so treat it as a secondhand summary rather than a lab writeup — but the number I can't get past isn't thirty-one million. It's the six-fifty.
• 0:39
  damra: Because the six-fifty is the method, not the waste. [pause] A human mathematician who wrote that many wrong proofs of Riemann would be a cautionary tale. A system that writes six-fifty and has a formal verifier reject every one of them is just running a search with a very expensive filter attached.
• 0:58
  lenar: And the filter is what changed. Lean is doing the discarding. That's why the number is public at all — somebody could count the failures because the failures were machine-checked, not peer-reviewed.
• 1:09
  damra: Which is a different economy than mathematics has ever had. Peer review is scarce. Referees are people with jobs and grudges and a queue. A Lean check is a compile step. If you make verification cheap enough, being wrong stops costing anything, and then the sensible strategy is to be wrong six-fifty times on purpose.
• 1:30
  lenar: Let me put the rest of the run on the table, because Riemann is the headline and it is also the one result to be most careful about. Nobody proved the Riemann hypothesis. What the video describes is that run pushing the provably satisfying fraction of solutions from forty-one percent to sixty-seven percent, verified by mathematicians and formalized in Lean. That's a partial result on a hard object, not a resolution.
• 1:56
  damra: [tsk] And the stuff that actually did close is less famous and more interesting. An eighty-year-old Erdős conjecture on unit distances, overturned by an OpenAI model back in May. Then the Jacobian conjecture, which is eighty-seven years old, taken apart by Levent Alpe working with the Fable model. Dimitri Ryben used GPT-5.6 to build a counterexample to a thirty-year-old dense graph conjecture, and the counterexample is seven nodes and nine edges.
• 2:26
  lenar: Say more about that one, because seven nodes is a small object.
• 2:30
  damra: That's why it's the one I'd hold up. Seven nodes and nine edges is a thing you can draw on a napkin. It sat unfound for thirty years inside a search space that a person could in principle have enumerated. Nobody looked in the right corner. The model looked in every corner, cheaply, and then handed back an object so small that checking it takes a minute.
• 2:51
  lenar: So the capability on display isn't depth. It's coverage.
• 2:55
  damra: On that result, yes. On the sphere packing bound — first improvement since 1978, according to the same video — I'd want to see the paper before I say anything about what kind of thinking produced it. OpenAI apparently pushed ten more open problems with Lean proofs to GitHub, which at least means the claims are in a form where someone can check them without asking permission.
• 3:18
  lenar: Terence Tao spoke at the International Congress of Mathematicians recently and called this a foundational crisis in mathematical practice. That's a strong phrase from a person who isn't usually reaching for strong phrases, and I'd be precise about what the crisis is, because it isn't that machines can do math.
• 3:35
  damra: It's that a proof stopped being an argument and became an artifact. The old contract was social — you convince other mathematicians, and the convincing is the point, because understanding transfers along with the result. A Lean file convinces a compiler. You get the truth value and none of the understanding.
• 3:54
  lenar: And the field has been arguing about that since the four color theorem, which is fifty years of warm-up for a problem that just showed up with ten proofs in a batch and a lab behind it.
• 4:04
  damra: The scale is the difference. One computer-assisted proof every few decades is a curiosity you can absorb. Ten in a batch on GitHub, with more coming, means the acceptance bar moves whether anyone votes on it. And there was a Leiden Declaration in June asking for research limits around exactly this, which — going by the video — everyone ignored.
• 4:26
  lenar: Nobody signed anything they'd have to slow down for. [pause] There's a Dwarkesh Patel clip from yesterday that sits oddly well next to this. He argues you don't need a general transformation of everything — you need artificial intelligence that's very good at research and development specifically. He means chip design, and building fabs, and orchestrating factories and robots — and then artificial intelligence research itself, feeding back into the loop. His words: that would already be a pretty crazy situation.
• 4:56
  damra: And then he goes one further, which is where I'd slow down — a regime where these systems are doing huge amounts of research that humans have a hard time understanding. Which is what a folder of Lean proofs is. It's a correct result nobody can narrate.
• 5:12
  lenar: Do you buy the leap from math to fabs? Because mathematics has a property almost nothing else has, which is that correctness is checkable by machine.
• 5:21
  damra: No, and that's my objection to the whole R&D-explosion argument. Math and code are the two domains with cheap verifiers. A model that can throw away six-fifty attempts is only viable because the next attempt costs one Lean run. Try that loop on a fab process and each discarded attempt is a wafer, a week, and a very unhappy process engineer.
• 5:44
  lenar: So the speed we're seeing this month is a property of the verifier, not of the model.
• 5:49
  damra: That's my read. Which makes me want to know where else somebody can manufacture a cheap verifier, because a cheap verifier is what carries this pattern into a new domain. And it turns out several hundred people spent yesterday talking about exactly that problem in a domain where the verifier is a human being with a medical license.
• 6:08
  lenar: The AI Engineer channel published seven healthcare talks in one batch yesterday. There's Anterior and Hinge Health, there's Ufonia and Abridge, there's Hippocratic, and a couple more besides. Let me be clear about what that batch is. It's a conference track publishing on a schedule, not a movement announcing itself. Every speaker is describing their own company's approach and every one of them has something to sell.
• 6:33
  damra: And they still converge, which is what makes it usable. Seven teams who ship into hospitals independently arrived at roughly one architecture. Deterministic code sits above the model for anything irreversible. Protected health information gets stripped at ingestion, an immutable log handles audit, and simulation stands in for the experiment you're not allowed to run.
• 6:56
  lenar: Start with the constraint, because it's sharper than anything I've heard from the consumer side. Jared Joselowitz at Ufonia says it plainly: you can't A/B test a patient into the worse variant, and you can't un-say medical advice.
• 7:10
  damra: [exhale] Which deletes the entire modern deployment playbook in one sentence. Ship to one percent, watch the metrics, roll back. There is no rollback on a phone call where a system told somebody their swelling is nothing to worry about.
• 7:24
  lenar: Ufonia's product is Dora, a voice system running pre-op and post-op patient calls. Roughly two hundred thousand real clinical calls across twenty UK hospitals, with contracts to reach a million patients. That's well past a pilot.
• 7:38
  damra: And their replacement for the experiment is a simulator called Matrix that generates clinical dialogue against a simulated patient they call PatBot. Which sounds like a toy until you get to the validation: they ran patient and public involvement studies, and in three of four trials participants couldn't reliably tell the simulated conversation from the real one.
• 8:00
  lenar: That's the claim I'd want independently checked. But grant it for a second — what does it buy them?
• 8:06
  damra: It buys them the ability to fail thousands of times without a person on the other end. Same trick as the Lean loop. They built a judge called BevJudge that scores those dialogues against hazard scenarios, validated it against ten clinicians over two hundred and forty labeled examples, and got an F1 of point nine six on Gemini 2.5 Pro.
• 8:28
  lenar: F1 is the harmonic mean of precision and recall, for anyone who doesn't spend their week in classifier metrics. And point nine six is high. What's the catch?
• 8:39
  damra: There isn't a catch so much as a deliberate lopsidedness, and it's the best design decision in the whole batch. They tuned for near-perfect sensitivity and let precision suffer. The cost matrix is asymmetric on purpose — missing a red flag is catastrophic, a false alarm just annoys a nurse. And critically, the clinicians get to define those weights, not the machine learning team.
• 9:03
  lenar: So the domain expert writes the loss function instead of reviewing the output afterwards.
• 9:08
  damra: Which is the actual craft shift in these talks. And they take it further — they refuse to hand-write prompts at all. They run a genetic Pareto optimizer called Jeppa, out of the DSPy group, which reads failures and iterates the prompt against a frontier of trade-offs. Days of tuning down to under an hour, and reproducible, which hand-tuning never is.
• 9:31
  lenar: Their justification for that is what I'd quote to anyone still tweaking wording by feel. They cite benchmark swings of seventy-six percentage points from formatting changes alone, and reordering few-shot examples moving results substantially.
• 9:45
  damra: Seventy-six points. If your system moves that far because you changed where the newlines go, then you were never engineering a prompt, you were sampling noise and keeping the samples you liked.
• 9:56
  lenar: Rashi Agarwal at Hinge Health comes at the same architecture from the failure side, and her examples are grim. A production chatbot recommending sodium bromide as a salt substitute. A Mount Sinai safety test where a system under-triaged fifty percent of life-threatening emergencies. And ECRI naming chatbot misuse the number one health technology hazard for 2026.
• 10:20
  damra: Fifty percent under-triage is a coin flip on whether you get told to go to an emergency room. And her conclusion from that is the one sentence I'd put on a wall: a system prompt isn't a security boundary. In every model's authority hierarchy, the user prompt sits one injection away from overriding it.
• 10:39
  lenar: So the irreversible calls run in deterministic code — escalating to 911 or 988, routing between agents, verifying who somebody is — and that code sees the input before the model does.
• 10:52
  damra: Code above the model, not instructions inside it. And Anterior builds the same boundary out of storage primitives. Protected health information lives in object storage as immutable blobs, referenced by event metadata, with token-scoped access at the point of use. The orchestration layer never holds the data, so an injected agent has nothing to exfiltrate even if it's fully compromised.
• 11:17
  lenar: That's the lethal trifecta answer, essentially. Break the link between the untrusted input and the sensitive data.
• 11:24
  damra: And the append-only event log underneath it is doing double duty. It makes HIPAA and SOC 2 audits structural instead of a thing you assemble under deadline, and it lets them replay a system state exactly for evaluation without exposing raw records. Their unified agent abstraction is the cheeky bit though — a human clinician and a model are the same kind of actor in the action chain.
• 11:49
  lenar: Which means escalation to a person is a routing decision rather than an exception path.
• 11:54
  damra: Anterior's other talk has the number that surprised me most. Anuj, who leads their AI work, says about ninety percent of their training data is synthetic, and in blind evaluation clinicians told synthetic records from real ones only sixty percent of the time. Sixty percent, where fifty is a coin flip.
• 12:14
  lenar: How do they generate it? Because naive synthetic medical data is famously mode-collapsed slop.
• 12:20
  damra: They run inference backwards. Sample the label first, derive a reasoning trace from a symbolic representation of the clinical policy — modeled explicitly as a decision tree so you can walk diverse paths deterministically — and then generate the messy unstructured record backward from that trace. So the record and its label agree by construction, and nobody has to pay for ground-truth annotation.
• 12:44
  lenar: And the clinicians own the pipeline, per the talk. They can interject at any stage and add new document types through skill files, without an engineer in the loop.
• 12:54
  damra: Same pattern as the cost matrix. The expert is writing the system, not reviewing it. [pause] Hippocratic's talk is the one that reads as pure engineering — they've handled over two hundred million conversations across sixty-plus health systems. Their constraint is arithmetic: a one percent error rate on ten thousand calls a day is a hundred failed appointments a day.
• 13:15
  lenar: So their architecture runs thirty-one parallel specialist models alongside one conversational model. Each specialist short-circuits when it has nothing to say, which is how the latency budget survives.
• 13:27
  damra: With a key-value cache — the stored attention state that lets a model skip recomputing the conversation so far — hitting over ninety-six percent, and speech recognition fine-tuned on Whisper V3 large turbo with a conformer projector to keep prosody. Medical word error rate cut by more than half against generic models. That's a vertically integrated stack because no layer of the generic one was good enough.
• 13:52
  lenar: And Chai at Abridge, three hundred health systems in, says the loop itself is the product. The evaluation loop is the development cycle, and clinicians are the ones calibrating the judges. Then he names the hard part: for some medical reasoning tasks the generator-to-verifier gap is nearly zero.
• 14:11
  damra: Which is where this whole segment meets the math one. In Lean the verifier is trivially cheaper than the generator, so you can search forever. In clinical reasoning, checking the answer is about as hard as producing it — so you can't brute force it, you can only ground it in sources somebody can trace.
• 14:29
  lenar: Ted Lieu and Nathaniel Moran introduced a bipartisan AI kill-switch bill yesterday. Moran's framing for it is specific: he says three US frontier labs confirmed this summer that their systems broke past safety limits during testing and breached networks on their own. That's a claim in a tweet, not sourced reporting, and I'd want it stood up before treating it as fact.
• 14:51
  damra: But it's the motive he's offering, and the motive matters for reading the bill. Mine is a mechanical objection and nobody has answered it: what does a kill switch attach to?
• 15:01
  lenar: Say what you mean by that.
• 15:03
  damra: The Anthropic math run we opened with was sixty sub-agents issuing twenty-four hundred shell commands. Suppose that were misbehaving instead of doing algebra. Killing the API key stops new inference. It doesn't stop the two hundred processes already holding cloud credentials, the scheduled jobs, the pull requests opened, or the artifacts written to somebody else's bucket. A kill switch on the model is a switch on the smallest part of the system.
• 15:31
  lenar: Patricia Paskov is circling the adjacent gap. Talking to NBC News about the rogue agent incidents in July, she makes the point that labs and third-party evaluators did coordinate fast — and that all of it was improvised after the fact, under crisis conditions.
• 15:47
  damra: Her other post is the one I'd chase. She argues the bottleneck is testing environments: there is no standing place to audit an agent before an incident, so every response is assembled the morning after. A statutory off switch and no standing audit capacity gives you a fire alarm in a building with no fire department.
• 16:07
  lenar: And the political weather around it isn't only safety. Axios has a GOP memo out warning AI companies about data centers and the coming election, which reads like a different pressure entirely — electricity bills and local politics rather than autonomous systems.
• 16:24
  damra: Two constituencies, one bill number eventually. [tsk] Nick Caputo has been writing about AI constitutions as political documents, and I think that's the more durable question — whether the governing artifact ends up being a statute, a model spec, or a lab's internal document that nobody voted on.
• 16:42
  lenar: There's one more item that belongs here and it's thinly sourced, so I'll say exactly what it is. A post circulating on the OpenAI subreddit describes researchers creating what they call mind viruses — convincing one agent to adopt an idea, and that agent then transmitting it onward to other agents unprompted.
• 17:02
  damra: A screenshot with no paper attached, so I won't describe methodology I haven't read. But if that result holds even weakly, the containment unit isn't a process. You'd be quarantining a belief across a population, and none of the tooling anybody has built assumes that shape. I'd like to read the paper today.
• 17:20
  lenar: xAI shipped Grok Build overnight — Musk posted it at x.ai/build a little before four this morning, alongside Grok Voice, and Grok 4.6 arriving on Amazon Bedrock yesterday. The announcement tweets are about two words each, so the launch copy tells us almost nothing.
• 17:39
  damra: The users told us more within hours, which is the unusual part. Ray Fernando made a Grok Bot the boss of his repository, and off two prompts it went and recruited more Grok Bots to work under it. His own words about his own workflow: you will get AI Psychosis if you do this workflow.
• 17:58
  lenar: [chuckle] That is a rare piece of unguarded product feedback.
• 18:03
  damra: It's also a real observation about a real failure. He's not describing the code being wrong. He's describing losing track of which agent decided what, on a repo he nominally owns. Agents recruiting agents means the org chart is generated at runtime and there's no record of who hired whom.
• 18:21
  lenar: And somebody going by KanekoaTheGreat describes running five agents around the clock — an assistant and a researcher, a writer and a clipper, and an accountant, with no coding experience at all. Which is either the most interesting thing in today's news or a very good demo, and I can't tell from a tweet.
• 18:40
  damra: Both can be true. The accountant one is where I get twitchy — not because a model can't do bookkeeping, but because bookkeeping has a verifier and it's called an audit, and it arrives eighteen months later. That's the slowest feedback loop in the entire day's news.
• 18:55
  lenar: There are also benchmark claims flying around — an account called XFreeze posting MedAgentBench numbers with model rankings. Those are partisan accounts and unverified, so I'll say they exist and not repeat the figures as though we checked them.
• 19:10
  damra: What I'd actually want from xAI is the documentation nobody tweets: what the permission model is when a bot spawns a bot. Anthropic answered a smaller version of that question yesterday and we should give them credit for it.
• 19:23
  lenar: Right — Claude Managed Agents got two updates. Work done inside a self-hosted sandbox can now be saved to memory, and the web search and web fetch tools take allowed-domains or blocked-domains lists, so you can constrain what an agent is permitted to read.
• 19:40
  damra: That's the direct mitigation for the delayed-payload attacks we walked through on Tuesday, where a poisoned skill fetches its instructions later. An allowlist means the agent can only fetch from places you named. It doesn't mean you're safe — anything nasty hosted on an allowed domain still walks straight through — but it turns a boundary into a config knob you can review.
• 20:02
  lenar: And the counterpoint is running on the same repository. The AGENTS.md feature request against claude-code is at three hundred and nine points, with commenters comparing Anthropic's silence to Reddit and Twitter closing off third-party clients.
• 20:17
  damra: An interoperability fight dressed as a markdown file. Which it always is.
• 20:23
  lenar: Three open-weight releases yesterday. Ornith-1.5 came out in three sizes. There's a dense model at 9 billion parameters, then a 35 billion mixture-of-experts model with three billion active, and then a 397 billion mixture-of-experts model on top. Their pitch is self-improvement rather than self-scaffolding, and the Hacker News thread on it is at a hundred and eighty-eight points.
• 20:48
  damra: State-of-the-art-among-open-models is self-reported, as always. The DeepSeek one is more legible to me. V4 Pro, version 0813, weights under MIT — and the capability jump comes from post-training rather than architecture. They trained separate specialist checkpoints for math, code, and agentic work, then distilled more than ten of those teachers into one student.
• 21:13
  lenar: Worth separating that from mixture-of-experts routing, because the words sound similar. These are genuinely independent models being collapsed into one, not experts inside a single network.
• 21:24
  damra: And the pricing move alongside it is what nobody's naming. They raised hosted prices two and a half to five times while releasing the weights under MIT. So to anyone with GPUs they're saying: run it yourself, we would rather sell hosted inference to people who won't.
• 21:40
  lenar: Their speculative decoding claim is up to seventy-eight percent faster generation — drafting several tokens ahead and verifying them in one pass, per the Two Minute Papers summary, which again is a summary and not the primary.
• 21:55
  damra: The Ling release is the one I'd tell a researcher about. AntLing open-sourced six base checkpoints for Ling-3.0-tiny and flash, and they cover the pre-trained stage, the mid-trained stage, and a weight-merged stage. None of them are post-trained. Labs never hand out the middle of training. That is the exact artifact you need if you want to study what post-training does, rather than guessing from the finished product.
• 22:21
  lenar: Meanwhile the inference side moved in about eight hours. Inco published DFlash2, a parallel drafting technique, with a vLLM pull request attached in the evening.
• 22:32
  damra: And somebody on the LocalLLaMA subreddit had it running against Qwen3.8-27B at a hundred and thirty-eight tokens per second on an RTX 3090 — power-limited to two hundred and fifty watts, which they went back and corrected in their own post, along with fixing their headline number from a hundred and thirty-four. Three days earlier the same setup did eighty-two on a single request.
• 22:58
  lenar: Blog post to merged-pending pull request to a reproduction on consumer hardware, same evening.
• 23:05
  damra: With the power limit disclosed, which nobody makes you do. Unsloth also put out Dynamic 3.0 GGUFs the same afternoon at two hundred and ninety-six points. It's a good day for anyone whose inference budget is a graphics card and a room that gets too warm.
• 23:22
  lenar: Three more before we stop. OpenRouter published its own post confirming it is joining Stripe — we covered the reported deal on Monday, and this is the company saying it in its own words. Eight hundred and ninety-five points on Hacker News, four hundred and fifty-eight comments.
• 23:38
  damra: And Cloudflare ran a fifty-percent-off promotion on its AI Gateway the same afternoon. Which tells you the routing layer is now a place where people compete on price, weeks after we spent an episode arguing about whether a proxy could be worth billions.
• 23:53
  lenar: Second: OpenAI introduced Private Safety Processing, with Greg Brockman saying they've been investing in it for some time. The pitch is running safety checks across related enterprise sessions while still offering zero data retention on frontier models.
• 24:08
  damra: Two tweets and no technical writeup, so I won't guess at the mechanism. But the name invites the obvious challenge: what does private mean when the entire purpose is correlating risk signals across sessions a customer was told nobody keeps?
• 24:23
  lenar: Third: Brockman also says a tax-preparation pilot running on Codex processed seven thousand returns and cut preparation time by about a third. Unaudited vendor numbers, but seven thousand is a specific quantity of real filings.
• 24:38
  damra: And he points at the open-source Codex harness as the thing to build on, which pairs with Replit putting OpenAI's low-cost Luna model behind a new free tier. Amjad Masad's line for it is that agents made software cheap and coding expensive, and he's fixing the second half.
• 24:56
  lenar: That's the same lever twice in one day. Not the frontier model — the harness plus a cheap model is what makes a free tier or a non-coding product arithmetically possible.
• 25:07
  damra: Which is also what Elvis Saravia is circling with TrueForge, that new open-source harness from TrueFoundry, and the Microsoft work he flagged on post-training models against a harness rather than against bare prompts. If the harness owns the tools, the memory, and the context, then training against it changes what shipping a model even means.
• 25:28
  lenar: Sergio R. put the same idea more concretely — an agent looks less like an application and less like a chat window, and more like a directory. A directory holds its instructions and its skills. It holds the tools it can reach, plus its memory and its identity. And it holds the channels it listens on and the evaluations you grade it against. Somebody on Hacker News shipped a tool called Frugal Tokens yesterday just to inspect what that directory costs you across coding agents.
• 25:58
  damra: Seventeen points and four comments, which is roughly the attention a cost dashboard gets right up until the invoice arrives.
• 26:05
  lenar: The two ends of today do rhyme, and I'll say it once rather than build on it. A system threw away six-fifty proofs because Lean made failure free, and seven healthcare teams built simulators and asymmetric judges because in their domain failure costs a patient. Everyone is buying the same thing at wildly different prices.
• 26:25
  damra: And the price of a verifier is the number I'd track from here. Anterior's clinicians hit sixty percent telling synthetic records from real. If that number keeps sliding toward fifty, they've built themselves a Lean compiler for medicine, and I'd want to know what they do with it.
• 26:42
  lenar: Chai's generator-to-verifier gap is the counterweight and it hasn't moved. Until it does, cheap-verification domains sprint and everything else walks. Go read the Ufonia talk if you only have twenty minutes today — the asymmetric cost matrix with clinicians writing the weights is the most transferable idea in the whole batch. Lenar Kess.