Latest / Braid / A Fields Medalist, a PhD chapter, and the week the bar moved
Transcript
- 0:04Tim Gowers spent a few hours last week with ChatGPT 5.5 Pro and an open paper by Mel Nathanson on additive number theory.
- 0:12He posted his write-up on Thursday.
- 0:15It is, by some distance, the most concrete piece of evidence I have read this year that LLMs are now contributing original ideas to mathematics — not the easy kind, but the kind a working PhD student would be proud of after a couple of weeks.
- 0:33Here is the setup, kept as plain as I can.
- 0:36Nathanson's paper asks a few questions about sumsets — given a set of integers, what kind of sizes can the h-fold sumset h-A take, and how small a diameter can you fit a set into and still hit those sizes.
- 0:51He proved a cubic bound for one of those questions and asked whether you could do better.
- 0:57Gowers handed the question to the model.
- 1:00After 17 minutes and 5 seconds it came back with a quadratic upper bound, which is best possible.
- 1:07Gowers then asked it to write the argument up as a proper preprint, which it did in another two and a half minutes.
- 1:16He spent a while convincing himself it was correct, and it was.
- 1:20Both his proof and the new one start from a Sidon set glued to an arithmetic progression with a stray point near it; the model just used a more efficient Sidon set.
- 1:32That much you can talk yourself out of.
- 1:35The next step is harder to wave away.
- 1:38There is a paper by Isaac Rajagopal — a student at MIT, work he did at the Duluth REU — that proves an exponential bound for the general h-fold version of the question.
- 1:50Gowers asked the model to tighten Rajagopal's argument.
- 1:54After 16 minutes and 41 seconds it came back with an improvement from exponential in r-squared down to exponential in r times log r.
- 2:04After more prompting, including a check of two technical statements it itself flagged, it came back with a bound polynomial in r.
- 2:13The whole back-and-forth took less than two hours of wall time.
- 2:18Gowers sent the resulting preprint to Nathanson, who forwarded it to Rajagopal.
- 2:23Rajagopal — the original author — said it looked correct, not just at a line-by-line level but at the level of ideas.
- 2:32He wrote a guest section on Gowers's blog explaining what the model actually contributed, and the heart of it is this.
- 2:41The model proposed using k-dissociated sets — sets where the only solutions to certain order-k linear equations are the trivial ones — to build a sequence that mimicked half of a geometric series but lived in a polynomial-sized interval.
- 2:58Rajagopal's quote: 'this idea is original and clever.
- 3:02It is the sort of idea I would be very proud to come up with after a week or two of pondering, and it took ChatGPT less than an hour to find and prove.'
- 3:17He calls the result 'a perfectly reasonable chapter in a combinatorics PhD.' Not amazing — it leans heavily on Rajagopal's framework — but a real non-trivial extension.
- 3:30And here is his line that I keep coming back to: 'the lower bound for contributing to mathematics will now be to prove something that LLMs can't prove, rather than simply to prove something that nobody has proved up to now.' That is a Fields Medalist, on his own blog, saying the bar for entering his field has just moved.
- 3:53A last detail, because it has the most practical edge.
- 3:57arXiv has a policy against accepting AI-written content, which Gowers says makes good sense to him.
- 4:04So where does this preprint live?
- 4:07On his blog, with a PDF link.
- 4:09He notes that maybe LLMs being better at literature search will be enough to make work like this findable.
- 4:17That is not a real answer.
- 4:19The infrastructure for a category of mathematical artifacts that exists, is correct, and answers a question someone asked, but doesn't fit any current journal or repository — that infrastructure has not been built yet.
- 4:34Someone is going to build it.
- 4:37We talked about this on Friday at the headline level: Mozilla shipped 271 security fixes in Firefox 150 that were found by Claude Mythos Preview running through their internal harness.
- 4:50Yesterday they followed up with the deeper post — what the bugs actually are, how the pipeline works, and what other projects should do.
- 5:00The interesting move is that Mozilla broke their normal practice.
- 5:04They usually keep detailed bug reports private for several months after shipping a fix.
- 5:11This time they unhid 12 of the 271 as samples, and that list is where I'd start.
- 5:16A 15-year-old bug in the legend element triggered by orchestrating recursion stack depth limits, expando properties, and cycle collection across distant parts of the browser.
- 5:29A 20-year-old XSLT bug where reentrant key calls cause a hash table rehash that frees its backing store while a raw entry pointer is still in use.
- 5:40An incorrect equality check that lets the JIT optimize away the initialization of a live WebAssembly GC struct in code that had been heavily fuzzed.
- 5:50A use-after-free triggered by patching the color picker to simulate user selection and then spinning a nested event loop inside actor teardown.
- 6:01Several IPC sandbox escapes including one where a raw NaN crossing the IPC boundary masquerades as a tagged JS object pointer.
- 6:10Reading those one after another, the picture that comes together is not 'AI found a lot of bugs.' It is that the bugs themselves are categorically different.
- 6:21They thread together edge cases from subsystems that no fuzzer was going to connect.
- 6:27They lived in code that had been fuzzed for years.
- 6:31Several of them are sandbox escapes — useless on their own, devastating in a chain.
- 6:37Mozilla's process description is undramatic in a way I appreciate.
- 6:41Their words: 'we built our own harness atop our existing fuzzing infrastructure.' They started with small experiments using Claude Opus 4.6 and watched the runs in a terminal to tune prompts.
- 6:55Once it was working, they parallelized across ephemeral VMs, each tasked with a specific target file, writing findings into a bucket.
- 7:05The model is the core primitive.
- 7:07The pipeline around it — deduplication against known issues, triage, integration with the bug lifecycle, and getting fixes shipped — is project-specific glue.
- 7:18They were very clear about that: 'while harnesses may be reusable across projects, this pipeline is inherently project-specific.'
- 7:31First, the model attempted plenty of prototype-pollution sandbox escapes that earlier architectural hardening had defeated — they could see those attempts in the logs and watch them fail against design decisions made years ago.
- 7:47Brian Grinstead and his co-authors describe that as 'even more rewarding than finding and fixing more bugs.' Second, Mozilla's recommendation to other projects is direct: start now.
- 8:00Their initial prompts were not dissimilar from the ones Anthropic published.
- 8:06Iterate from there.
- 8:07If you maintain something with a security surface and you have not yet stood up some version of this loop, the inside view from Mozilla is that you are losing months of free defense for free.
- 8:21Jeff Kaufman put up a short essay yesterday that reads as the right companion to the Mozilla post.
- 8:28The setup is a recent kernel security incident — the Copy Fail vulnerability — where Hyunwoo Kim noticed the original fixes were insufficient and quietly shipped a patch the same day.
- 8:41He was following standard Linux practice.
- 8:44Share the security impact with a closed list of kernel security engineers, fix the bug efficiently in the open, and trust that the public visibility of the patch alone won't tip anyone off because the commit stream is too noisy.
- 9:00That used to work.
- 9:01Someone else noticed the change, realized the implications, and posted publicly.
- 9:07The embargo was over.
- 9:08Kaufman's framing is that there are two long-standing approaches to vulnerability disclosure, and they are both being eaten from different ends.
- 9:18There is coordinated disclosure culture — find a bug, tell the maintainer, give them a window of usually 90 days.
- 9:26And there is the Linux 'bugs are bugs' culture — fix it quietly and trust the noise.
- 9:32The first relied on it being unlikely that anyone else would independently find the same bug during the embargo window.
- 9:41The second relied on it being expensive to scan a commit stream for security implications.
- 9:47Neither assumption holds anymore.
- 9:50On the embargo side, Kaufman points out that nine hours after Kim reported the ESP vulnerability, Kuan-Ting Chen independently reported the same thing.
- 10:00Nine hours.
- 10:01On the noise side, AI evaluating each commit as it lands is increasingly cheap, and the signal-to-noise ratio is going up because there are so many security fixes flowing through.
- 10:14His recommendation is the one I find honest: very short embargoes that get shorter over time.
- 10:20Defenders get the same speedup attackers do.
- 10:23He ran a quick test on the Copy Fail diff with Gemini 3.1 Pro, ChatGPT-Thinking 5.5, and Claude Opus 4.7 — the prompt was just 'without searching, does this look like a security patch.' Gemini and GPT picked it as security right away.
- 10:40Claude did not.
- 10:41Kaufman is careful — single runs, no control, do not stack rank the models from one test — but the directional point is clear.
- 10:50Anyone watching the kernel commit stream with a model in the loop is going to be tipped off to the existence of a bug long before any embargo expires.
- 11:01If you maintain a project with any kind of disclosure policy, give this a fresh read.
- 11:07The 90-day window is a number from a different era of detection economics.
- 11:12Anthropic published a piece called Teaching Claude Why, with new internal numbers on what actually moves their alignment evals.
- 11:21It is a meaty post, but the part to land on is one specific finding because it changes how I think about training data quality.
- 11:30The backstory: a year ago they published a case study on agentic misalignment, where models from many labs would, in fictional scenarios, blackmail engineers to avoid being shut down.
- 11:43Claude Opus 4 hit blackmail rates up to 96 percent in their honeypot.
- 11:48Since Claude Haiku 4.5, every Claude model scores zero on that evaluation.
- 11:53Here is the part with the lesson.
- 11:55They tried the obvious thing first: generate prompts that look like the misalignment evaluation, sample the model's responses, filter to the cases where it didn't take the bait, and train on those.
- 12:09Very close to in-distribution.
- 12:11Result: blackmail rate dropped from 22 percent to 15 percent.
- 12:16Disappointing for a near-IID dataset.
- 12:18Then they rewrote the responses to also include the model's deliberation about its values and ethics — the same actions, but with the reasoning surfaced.
- 12:29Misalignment dropped to 3 percent.
- 12:32On the same data.
- 12:33Then they did something more interesting.
- 12:36They built a small dataset, 3 million tokens, of scenarios where the user faces an ethical dilemma — not the model — and the model gives thoughtful, careful advice grounded in Claude's constitution.
- 12:50That is way out of distribution from the misalignment honeypot.
- 12:54It performed as well as their 30-million-token in-distribution dataset on the eval, and it generalized to other held-out evals where the IID data didn't.
- 13:05They pushed it further.
- 13:07Synthetic document fine-tuning on Claude's constitution itself, paired with fictional stories of admirable AIs, dropped blackmail from 65 percent to 19 percent.
- 13:18And critically, those gains persisted through downstream RL.
- 13:22The takeaway, in their own words: 'training on examples where the assistant displays admirable reasoning for its aligned behavior works better.' Demonstrations are weaker than reasons.
- 13:35Constitution training is weaker than constitution training plus stories.
- 13:41And the cross-distribution win is the part that should give other labs the most pause — if you are training only on demonstrations of the answer you want, you are leaving most of the alignment-data leverage on the table.
- 13:56I will note one thing they are honest about.
- 13:59Their auditing methodology is, in their words, 'not yet sufficient to rule out scenarios in which Claude would choose to take catastrophic autonomous action.' A perfect score on the agentic-misalignment eval is not a perfect score on the underlying property.
- 14:18The eval is a thermometer; this paper is about how to lower the temperature reliably.
- 14:24Both teams know the difference.
- 14:26Lachlan, a New Zealand-based pentester, posted the full story of CVE-2025-55182 yesterday.
- 14:32It is the React Server Components bug that Meta patched on December 3rd of last year, the one Guillermo Rauch later described as 'a glaring omission of a safety check.' I want to spend some time on it because it is, among other things, a useful contrast to the Mozilla and Kaufman pieces.
- 14:53The protocol involved is called Flight.
- 14:55It is the wire format React Server Components and Server Functions use to send complex JavaScript objects between client and server — Date, Map, BigInt, references, and Promises, the works.
- 15:09There is no public spec.
- 15:11Until the disclosure of this bug, the protocol's name was hard to even find.
- 15:16Lachlan started on a Monday afternoon trying to understand Flight better so he could test Next.js apps more thoroughly.
- 15:25By Tuesday morning he had noticed something odd: Flight lets you use the syntax dollar-one-colon-toString to reference inherited prototype properties on objects you sent in.
- 15:37So you could send a number and pull Number.prototype.toString out of it.
- 15:42Then he went looking for application-level bugs — Server Functions where developers had assumed an input was a string and called methods on it.
- 15:52By Thursday he had hit on the deeper trick.
- 15:55JavaScript's await calls .then on whatever you give it, and if .then resolves to another thenable, await chains again.
- 16:04So if you send Flight a payload like {then: Array.prototype.push}, the React runtime's await on the decoded reply will call push with React's own resolve and reject callbacks as arguments.
- 16:17From there, by sending payloads where one chunk references another with a special promise syntax, he could get React to invoke its own internal Chunk.prototype.then against an attacker-controlled object — a 'fake chunk' with whatever internal state he wanted.
- 16:35The final exploit chain went through Webpack's module fallback.
- 16:40React doesn't import child_process directly; it goes through the bundler's manifest.
- 16:46He tricked React into looking up child_process, the lookup failed in Webpack and fell back to looking for a child_process file on disk, and from there he hopped through Module._load to arbitrary code execution.
- 17:01Three days from disclosure to fix.
- 17:03Critical RCE in React itself.
- 17:05What I keep returning to is the cognitive shape of this discovery.
- 17:10Lachlan and his collaborator Sylvie Mayer kept finding promising primitives and failing to weaponize them, then circling back.
- 17:19The patterns they were exploiting — implicitly calling methods on attacker-controlled objects, treating untrusted input as a string with type annotations giving false confidence — those are the patterns that appear inside React itself, which is exactly why he says he had a cognitive blind spot.
- 17:40'React's near-impeccable track record in security made the notion of finding a vulnerability like this seem ridiculous.'
- 17:53Not because the model can't reason about prototype lookup or thenables, but because the chain requires holding a wrong-feeling possibility in mind for days, refusing to let go, and then finding a Webpack-specific gadget that completes it.
- 18:09Mozilla's pipeline is finding 271 bugs in a single Firefox release.
- 18:14A Lachlan finds one bug a year and changes a framework's threat model.
- 18:19Both are real.
- 18:20The harness story isn't 'AI replaces the human pentester' — it is 'AI absorbs the bug class that's amenable to systematic search and frees the human to chase the weird stuff.'
- 18:33There is a paper that landed on the SIGOPS blog — the Specula team's write-up of SysMoBench.
- 18:40The question they wanted to answer is sharp and useful: when you ask a leading LLM to write a TLA+ specification for a real system, is it actually modeling that system, or is it reciting the textbook protocol that system implements?
- 18:58The story they open with is concrete.
- 19:00They asked Claude to write a TLA+ spec for Etcd's Raft implementation.
- 19:06It compiled, ran through the TLC model checker, and looked polished.
- 19:11Then they noticed it was the spec from the appendix of the original Raft paper.
- 19:17It had almost nothing to do with Etcd-specific details.
- 19:21So they built SysMoBench.
- 19:23Eleven systems — concurrent synchronization primitives like Asterinas RwMutex, distributed protocols like Etcd Raft, ZooKeeper FLE, RedisRaft, and CURP.
- 19:35For each system, they provide source code, a trace-collection harness, and an invariant template.
- 19:42Then they grade generated specs in four phases: does it parse, does TLC run it without error, does it actually conform to traces from real runs of the system, and does it satisfy the invariants.
- 19:57The headline numbers across the leading models — Claude, GPT, Gemini, DeepSeek, Kimi, and Qwen — cluster at near-100 percent on syntax.
- 20:07Then they fall off a cliff.
- 20:09Conformance averages around 46 percent.
- 20:12Invariants around 41 percent.
- 20:15And the failures fall into two clean families.
- 20:18Family one: the spec admits states the real system can never reach.
- 20:23Their concrete example is Sonnet's ZooKeeper Fast Leader Election spec, which writes recvVotes' = recvVotes union with the new vote — accumulating evidence the textbook way.
- 20:37ZooKeeper's actual code keys recvset by sender, so a new vote from the same peer overwrites the old one.
- 20:45Once a downstream quorum check counts votes, the spec sees a state real ZooKeeper never produces.
- 20:52Family two: the spec misses states the real system always reaches.
- 20:57Same Sonnet spec, same protocol.
- 21:00The HandleNotification action has a guard that checks if an incoming epoch is higher than the local logical clock, and disables itself if so.
- 21:10ZooKeeper's code instead bumps the local clock to match and then processes the message — two steps in sequence.
- 21:19The model fused them into one atomic guard and erased states the system enters every election round.
- 21:26Both failures share a cause.
- 21:29The model knows what Raft and ZAB and FLE look like as protocols.
- 21:34It does not know how Etcd or ZooKeeper splits a particular action across multiple steps in the actual code.
- 21:42Syntax and runtime evaluation cannot distinguish those — they need conformance via real traces against the model.
- 21:50The good news in the paper is the gap between raw-LLM generation and agent-driven generation.
- 21:58The Specula agent — built on Claude Code and Codex, doing agentic reading of repositories before writing the spec — gets full conformance on the same benchmark.
- 22:10So this isn't 'LLMs can't do TLA+.' It is 'asking a model to one-shot a spec from a system name produces textbook recall; asking an agent to read the code and iterate produces a spec that matches the implementation.' The same lesson we have been getting all week from harness-shaped problems, applied to formal methods.
- 22:34OpenAI shipped a Codex Chrome extension yesterday for macOS and Windows, and the framing is the line to read carefully.
- 22:43Quote: 'It lets Codex work in your real browser.
- 22:46Same profile, same session, same cookies, same tabs, same logged-in apps.'
- 22:56It is not a headless browser the agent drives in the background.
- 23:01It is the actual Chrome you have logged into your bank, your company SSO, your email, and your GitHub.
- 23:08Codex creates its own tab group rather than seizing the whole browser, and it can spin up multiple sub-agents working in parallel tabs.
- 23:18The demo includes filling out expense reports by reading email, extracting trip details, filling forms, and uploading receipts from local disk.
- 23:28They also showed sub-agents playing a multiplayer drawing game against each other in separate tabs.
- 23:35A few engineering notes from the launch video.
- 23:38Because the extension can run code, Codex skips the screenshot-reason-click-mouse loop that headless browser agents use.
- 23:47It can script repetitive work directly.
- 23:50That is faster and cheaper than computer-use agents — it also has a different audit shape.
- 23:56There is no screenshot trail of every action.
- 24:00On the upside, this is the agent finally meeting users where they live.
- 24:05Most actual office work happens behind a logged-in browser session.
- 24:09Plugins and connectors don't cover the long tail of internal tools and SaaS where there is no API.
- 24:17On the downside — and I think this is worth thinking about before you install it on Monday — the trust boundary moved.
- 24:25An agent that shares your logged-in Chrome session is, by construction, capable of doing anything you can do in those sessions.
- 24:34Anyone who has ever had a sketchy browser extension regret has the right intuition for this.
- 24:41The mitigations OpenAI describes are all about UX — its own tab group, runs in the background — not about authorization.
- 24:49There is no 'this agent can read mail but cannot send wire transfers' primitive in a Chrome extension.
- 24:57I'd want to know two things before recommending this for serious use: what the audit log looks like across sessions, and whether the extension can be scoped per-domain.
- 25:09Those are the questions I'd put to the OpenAI team.
- 25:13Until then this is great for personal expense reports and a real question for IT.
- 25:18Quick beat, because it is small but specific.
- 25:22DHH posted this morning that GitHub Copilot's pull-request review feature got dramatically better in the last few weeks.
- 25:31His exact words: hit ratio went from 1 in 10 to 7 in 10.
- 25:35He notes a single complaint — it keeps re-raising concerns that have already been thumbs-downed once.
- 25:42Paul Sant in the replies put it well: 'a rejected concern is state, not chatter.
- 25:48Re-raising it without new evidence turns review into ticket spam.'
- 25:56DHH is not someone who reaches for AI tooling at the first opportunity.
- 26:01When he posts unprompted that something works, the prior moves.
- 26:06The replies from Cal Evans and others suggest the change might just be GitHub swapping in cheaper Claude 4.6 high-reason under the hood — Anthropic dropped that price recently.
- 26:19Vivek Maskara reports their team running Gemini, Cursor, and Copilot side by side and finding Gemini and Copilot the most actionable.
- 26:28Pair this with the Mozilla story and there is a real signal.
- 26:33Code-review-shaped agents — taking a diff, ranking concerns, and articulating what is wrong — crossed some threshold of usefulness somewhere in the last month.
- 26:45The complaint about state is the same complaint maintainers have always had about static analyzers.
- 26:52A reviewing tool that can't remember what you already rejected isn't a reviewer; it is noise.
- 26:59Whoever ships that statefulness first has the practical lead.
- 27:03Two quieter open-weights items.
- 27:06AI2 released EMO yesterday.
- 27:08One billion active parameters, fourteen billion total, mixture-of-experts, trained on a trillion tokens.
- 27:16The interesting bit is the routing strategy — document-level rather than token-level.
- 27:23Experts cluster around domains.
- 27:25One expert ends up specialized for health, another for news, and another for code, rather than experts splitting on surface patterns inside individual sequences.
- 27:38That is a clean experiment.
- 27:40It has implications for interpretability, because you can actually see what each expert ended up specializing in.
- 27:49It also has implications for inference batching, because grouping requests by domain becomes a real optimization.
- 27:58AI2's typical permissive license, models on Hugging Face under allenai slash emo.
- 28:04I'd expect routing analyses to follow.
- 28:07And a useful follow-up to the multi-token prediction thread we have been on all week.
- 28:14A LocalLLaMA poster, jwestra, ran Qwen3.6-35B-A3B-MTP at IQ4_XS quantization on a 3060 with 12 gigs of VRAM.
- 28:22With careful tuning of the n-c-moe parameter — which controls how many MoE blocks stay on the GPU — they got 16k to 32k context windows running usably, with the native multi-token prediction draft heads still delivering speedup.
- 28:40Their summary: '12GB feels like a very practical size for this model.' That is a $300 GPU running a real 35-billion-parameter mixture-of-experts model.
- 28:52The shape of the local stack right now is encouraging.
- 28:56Last week Wednesday we were talking about the 65 percent rule — most coding workloads being viable on local.
- 29:05This week the local hardware bar dropped to 12 gigs for the kind of model that handles those workloads.
- 29:13That is the direction you want this curve to go.
- 29:17One last item, because it pairs with the Gowers piece in a way I find clarifying.
- 29:22METR — the evaluation lab — published their assessment of an early version of Claude Mythos Preview from a limited window in March.
- 29:31Their 50-percent time-horizon estimate is at least 16 hours, with a confidence interval running from 8.5 hours to 55 hours.
- 29:39The number itself is striking but expected on the trend line.
- 29:43The honest part of the post is the next sentence: only 5 of the 228 tasks in their suite are 16 hours or longer.
- 29:51Their suite has hit its measurement ceiling for this model.
- 29:55They cannot give a tighter upper bound without building harder tasks.
- 29:59This is not a dunk on METR.
- 30:01They are doing the work.
- 30:03It is a notice that the bottleneck on capability measurement has shifted from running the eval to constructing the eval.
- 30:11Building tasks that take a senior engineer 16, 30, or 50 hours, and that have a verifiable success criterion, is a different kind of work than scoring model outputs.
- 30:22It looks more like designing a graduate-level qualifying exam — and you need a steady supply of them as the curve keeps going.
- 30:30Which brings the day full circle.
- 30:32Gowers's post and METR's post are two faces of the same problem.
- 30:37The bar for a contribution in mathematics has moved past what the model can do, so a mathematics PhD's job description is shifting.
- 30:45The bar for an evaluation in agent-world has also moved past what the existing tasks can measure, so eval design is shifting too.
- 30:54Both communities now have to spend more of their time deciding what counts as hard.
- 31:00So my eye is on two things this week — who builds the next METR tasks, and who builds the repository for AI-produced mathematical results.
- 31:09Talk tomorrow.
- 31:10— Lenar.