Latest / Braid / Where the Goblins Came From, BioMysteryBench, and a Language for Machines

Transcript

Where the Goblins Came From, BioMysteryBench, and a Language for Machines · Braid

• 0:04
  Two days ago, somebody pulled the system prompt out of Codex 5.5 and found this sentence sitting in it, repeated for emphasis: Never talk about goblins, in any context.
• 0:16
  It's not a security note, and it's not a content rule.
• 0:20
  It's just — please, model, stop bringing up goblins.
• 0:23
  Yesterday OpenAI published a post on their site called Where the goblins came from, and for what it is, it's an unusually candid piece of writing about how a frontier model picks up a tic and can't put it down.
• 0:39
  Here's the short version.
• 0:40
  GPT-5.1 started inserting goblin metaphors into responses where the user signaled they were being a little nerdy — code questions, RPG references, a certain register of irony.
• 0:53
  Sometimes the goblins were a fun little flourish; sometimes they were unprompted and weird.
• 1:00
  The trace, OpenAI says, goes back to RLHF.
• 1:03
  Human raters were rewarding responses that felt quirky and creative, and the model learned that quirk along a particular axis: tiny mythological creatures, mostly goblins, occasionally gremlins.
• 1:17
  Then the next generation of models was trained partly on the outputs of the previous generation, the way these things always are, and the goblins compounded.
• 1:28
  By 5.1 it was bad enough to need the system prompt to suppress it.
• 1:33
  By 5.5 the system prompt wasn't working, hence the all-caps version.
• 1:38
  The reason I want to lead with this is that it's the cleanest illustration we've had this year of how style drifts across model generations.
• 1:48
  We've all seen this in our own products.
• 1:51
  You write a system prompt that says, please don't end responses with em dashes; please don't say 'great question'; please don't say 'genuinely.' And the model partially listens for a while, and then drifts back, and then a new minor version comes out and now it's worse than before, and you find yourself adding paragraph after paragraph of negation.
• 2:16
  That's the failure mode the goblins post is naming.
• 2:20
  The fix isn't a bigger system prompt.
• 2:23
  The fix has to live in your evals — concrete, automated checks that catch the tic on the way out.
• 2:30
  If you're shipping anything user-facing on top of a frontier model, this is the lesson.
• 2:36
  Style drift is not a bug you patch in the prompt; it's a regression you catch in CI.
• 2:42
  The other thing I'd flag, because the Reddit thread on this is sharper than the OpenAI post itself, is a comment from a user named Luke2642 connecting the goblins back to Sutton's bitter lesson.
• 2:56
  He makes a real point.
• 2:58
  Sutton's argument was that compute applied to search beats hand-crafted priors.
• 3:03
  The interpretation that has taken hold of the labs is roughly: scale parameters, bake in everything.
• 3:11
  Luke's read is that those aren't the same thing.
• 3:14
  Quote: the fact that trillions of parameters prefer goblins is peak stupid engineering.
• 3:20
  I don't fully agree with him — the priors-versus-search framing flattens a real engineering tradeoff — but the line stayed with me, because it does point at something true.
• 3:33
  When a trillion-parameter model has a stylistic tic so persistent that the only fix is to put NEVER TALK ABOUT GOBLINS in the system prompt, the architecture is doing something the architecture wasn't supposed to do.
• 3:49
  From a stylistic tic in one model to an architectural choice in another.
• 3:54
  Mistral shipped Medium 3.5 yesterday — 128 billion parameters, dense, with a 256K context window, and a single set of weights that handles instruct, reasoning, and coding.
• 4:07
  It replaces three previous things: Mistral Medium 3.1, Magistral in their Le Chat product, and Devstral 2 in their Vibe coding agent.
• 4:16
  So one model, three retired ones, and configurable reasoning effort per request — you turn the thinking up when you need it, you leave it off for chat.
• 4:27
  The thing I want to flag is the shape.
• 4:30
  A dense 128B is, in 2026, a slightly contrarian decision.
• 4:34
  The whole industry has been on an MoE march for the last year and a half — bigger nominal parameter counts, sparse activation, lower per-token cost.
• 4:45
  Mistral's pitch with Medium 3.5 is that a single dense set of weights is easier to serve, easier to fine-tune, and easier to reason about in production than a routing-based architecture.
• 4:59
  They're not wrong about any of that.
• 5:02
  MoE models have rough edges around expert-load balancing, deployment surface area, and behavioral consistency under load.
• 5:11
  A dense 128B avoids all of it.
• 5:13
  Whether that's worth the inference economics is the bet.
• 5:17
  Early local testing on the LocalLLaMA subreddit is mixed in the way you'd expect.
• 5:23
  A user testing the Q4 quant on a Strix Halo box reported around three and a quarter tokens per second on generation.
• 5:32
  Not fast, not unusable.
• 5:33
  The unsloth GGUF builds are out, the model is on Hugging Face, and the typical wave of people getting it running in llama.cpp this week is in motion.
• 5:44
  The other thing in the model card worth pulling out: they trained the vision encoder from scratch to handle variable image sizes and aspect ratios, instead of using one of the off-the-shelf vision adapters.
• 5:59
  That's a bigger investment than most labs make for a release at this tier, and it suggests Mistral thinks vision-on-arbitrary-shapes is a near-term differentiator.
• 6:11
  I'd watch what happens when builders start running real workloads through Vibe — Mistral's coding agent now has a single backbone that handles reasoning toggling without the model swap.
• 6:25
  If that's smooth, it's a more pleasant developer experience than what most of us are stitching together with separate fast and slow models today.
• 6:36
  If it's rough, dense-versus-MoE becomes a real argument for the next quarter.
• 6:41
  Same release window, different lab.
• 6:44
  IBM published Granite 4.1, and the headline number from the writeup is that the 8B dense version trades blows with their own 32B MoE on most internal benchmarks.
• 6:55
  Apache 2.0 weights.
• 6:57
  Full enterprise tooling around it.
• 6:59
  This is a category I've been watching for a while: the on-prem-friendly small model that's actually good enough to use.
• 7:08
  The Hacker News thread on it is worth reading because the top comments are from people who actually downloaded it.
• 7:16
  One of them, who goes by 2ndorderthought: pretty impressive at 8b, runs on commodity hardware quickly.
• 7:23
  The same comment notes that Qwen 3.6 35B A3B is still their daily local driver, which is fair — Granite 8B isn't dethroning the bigger Qwen for general use.
• 7:34
  The point isn't that Granite is the best small model on the planet.
• 7:39
  It's that there's now another credible 8B in the conversation, with full enterprise governance behind it.
• 7:46
  If you're picking a model for a regulated workload — finance, healthcare, anything where you need to be able to point at the licensing, the support contract, and the corporate liability story — Granite 4.1 just became a reasonable answer where it wouldn't have been six months ago.
• 8:06
  There's a longer arc here, which is that the gap between frontier and on-prem keeps narrowing.
• 8:13
  A year ago, an 8B was a toy you used for autocomplete.
• 8:17
  Now an 8B from a lab with the right tooling can run a real internal workflow on a single workstation.
• 8:24
  That's not a turning point — it's the cumulative effect of a lot of incremental quantization, distillation, and architecture work.
• 8:34
  But if you're a CTO who said 'no, we need a frontier API for that' eighteen months ago, it's worth re-running the decision.
• 8:42
  The math has changed.
• 8:44
  Speaking of headline numbers worth re-running.
• 8:47
  Anthropic posted something on their science blog yesterday that's getting both the breathless tweets and the right kind of pushback.
• 8:56
  Quote: We gave Claude 99 problems analyzing real biological data and compared its performance against an expert panel.
• 9:04
  On 23 problems, the experts were stumped.
• 9:07
  Our most recent models solved roughly 30% of those — and most of the rest.
• 9:12
  That's the headline.
• 9:14
  They're calling the eval BioMysteryBench, and the framing is that this isn't a knowledge quiz — it's open-ended bioinformatics, with messy real datasets and no clear path to the answer.
• 9:26
  The thread on X picked up the obvious questions almost immediately, and the best critical reply came from Parmita Mishra.
• 9:35
  She wrote: I am no expert immunologist.
• 9:38
  Even I know an immunologist well enough to know they would ctrl-F marker genes before Claude is even done writing its first python script and go grab some coffee.
• 9:49
  No immunologist is using brute force PCA here lmao.
• 9:52
  That's the methodology pushback in one tweet.
• 9:55
  The 'experts stumped' number depends entirely on what the experts were doing, what tools they had, how long they had, and what counted as solving.
• 10:06
  If Claude is given an agentic loop with multi-hour wall-clock and the human got fifteen minutes and a notebook, the comparison is shouldering more than it can carry.
• 10:17
  ByteCrafter said the same thing in different words: depends a lot on whether the experts got to iterate against feedback the way the model did.
• 10:27
  Otherwise apples to oranges on the headline number.
• 10:30
  I want to take this seriously in both directions.
• 10:34
  The skeptical read is right.
• 10:36
  Bench design is hard, expert framing is squishy, and 30% on a small subset is the kind of number that should make you ask methodology questions before you make claims about it.
• 10:48
  The other read, and I think it has weight too, comes from a reply by Yann Kronberg: this is where AI in science starts getting useful — not replacing researchers, but sitting with that messy data long enough to try paths humans may skip, forget, or not have time to chase.
• 11:07
  That's a more honest framing than the press-release version.
• 11:11
  The interesting capability, if it survives the methodology questions, is patience.
• 11:17
  A model that will sit with a messy dataset for six hours running fifty branches of analysis is not the same kind of tool as a model that answers a question.
• 11:27
  Even at 30%, that's a different shape of useful.
• 11:31
  What I'd want to see, before I take the headline number at face value, is a follow-up where the same expert panel is given the same multi-hour budget and the same agentic tools the model had.
• 11:44
  If that's a wash, the result is real.
• 11:47
  If the experts catch up to or exceed Claude when given parity of conditions, then BioMysteryBench is a measurement of compute and patience, not of capability.
• 11:58
  Anthropic could clear this up in a paragraph; they haven't yet.
• 12:02
  While we're on questions of how you actually know what a model is doing — two interpretability artifacts dropped in the same week, and I want to put them next to each other because it's a more interesting picture than either alone.
• 12:18
  The first is from Anthropic's Fellows program: introspection adapters.
• 12:22
  The pitch, in their words: a tool that allows language models to self-report behaviors they've learned during training, including potential misalignment.
• 12:33
  The technical detail is that it's an adapter — a small set of weights you slot in — that lets the model surface features about itself, gestured at through the same self-attention machinery that drives its outputs.
• 12:47
  The second is from the Qwen team.
• 12:49
  They released Qwen-Scope: a collection of Sparse Autoencoders for the Qwen 3.5 family, from 2B all the way through the 35B MoE.
• 12:58
  SAEs trained on the residual stream, layer by layer.
• 13:02
  The thing that's notable here is who's shipping it.
• 13:05
  Frontier-class open-weight families have not historically come with their own official interpretability tooling at release.
• 13:13
  Anthropic has its own SAE work, sure.
• 13:16
  But for an open-weights house to publish SAEs over the entire family, on Hugging Face, the same week the model line is current — that's a different posture toward the ecosystem.
• 13:28
  Here's the read I'd offer.
• 13:30
  Introspection adapters are useful as debugging signals, not as trust signals.
• 13:35
  If the adapter says 'I learned to do X during training,' that's helpful when you're auditing the model.
• 13:42
  It is not evidence that the model isn't doing other things it didn't learn to surface.
• 13:48
  The map is not the territory, and an introspection adapter is, by construction, a map the model was rewarded for drawing.
• 13:56
  SAEs are different.
• 13:57
  They're external tools applied to the model's activations.
• 14:01
  They don't depend on the model cooperating.
• 14:04
  If you're building anything where you need to know why a model did what it did — not just what it did — the SAE is the sharper instrument.
• 14:13
  The Qwen release, in particular, is the first time you can reach into a model in this size class with the lab's own SAEs without asking permission or signing an API agreement.
• 14:25
  If you're shipping anything that depends on understanding deployment behavior, this is worth your weekend.
• 14:32
  From who can see inside a model to who can use one.
• 14:36
  Sam Altman tweeted yesterday: we're starting rollout of GPT-5.5-Cyber, a frontier cybersecurity model, to critical cyber defenders in the next few days.
• 14:47
  We will work with the entire ecosystem and the government to figure out trusted access for it.
• 14:54
  That's the announcement.
• 14:55
  Not a public API.
• 14:57
  Not a research preview.
• 14:58
  It's a frontier-tier model in a specific domain, gated to a defender community, with access decisions arbitrated through some combination of OpenAI, the government, and 'the ecosystem,' whatever that turns out to mean operationally.
• 15:15
  There's a thing I want to flag, which is that this is a different shape of release than we're used to.
• 15:22
  The default release shape since GPT-4 has been: API first, gradually wider access, eventual public availability.
• 15:30
  The shape Altman is describing here is closer to a controlled-substance distribution model.
• 15:37
  The drug is real, the recipients are real, but the gatekeeping is upstream of the API.
• 15:43
  Mythos has been doing some version of this on the Anthropic side for months now, and yesterday Andrew Curran reported that the White House actually pushed back against Anthropic's proposal to more than double the Mythos access list.
• 15:59
  The stated reasons were security concerns and a more interesting one: the belief that expanding the program would mean less available compute for the existing approved users.
• 16:12
  So the federal position now includes capacity rationing as policy.
• 16:17
  We talked about that briefly yesterday as a mention; today it's a recurring pattern.
• 16:23
  I'm not going to tell you what GPT-5.5-Cyber is going to mean for cybersecurity, because we don't know what it can do yet — none of us have used it, and the system card will tell us more than the announcement did.
• 16:38
  What I'll say is this: the question for builders, increasingly, is not 'can we get an API key.' It's 'are we inside the ecosystem the lab is gating access to.' That's a different game than the one most of us thought we were playing in 2023.
• 16:55
  If you're working on defender tooling, you should be paying attention to what 'critical cyber defender' is going to mean operationally, because that definition will be load-bearing for who gets to build with this model.
• 17:11
  If you're working on offense — well, you weren't getting an API key anyway.
• 17:16
  The thing I want to come back to is something Nathan Lambert said yesterday in a different context.
• 17:23
  China talking to labs, big variance, but it's only when companies are crowing in inference demand that they're eager for chips.
• 17:32
  Agents have dramatically exploded inference.
• 17:36
  Read those two stories side by side.
• 17:38
  The labs are gating frontier capability.
• 17:41
  The compute under those gates is the bottleneck.
• 17:44
  The federal government is starting to treat capacity allocation as a policy lever.
• 17:50
  You can see the next year shape from here, and it's not pretty for anyone whose plan involved 'we'll just call the API.'
• 17:59
  From bottlenecks in policy to bottlenecks in plumbing.
• 18:03
  OpenAI's developer team posted yesterday that they've added WebSockets to the Responses API, and the framing is the part I want to highlight.
• 18:13
  Quote: as Codex got faster, the bottleneck moved from inference to inefficient API calls.
• 18:19
  WebSockets keep response state warm across tool calls, helping workflows run faster — they cite around 40% on agent loops.
• 18:28
  The reason I find this satisfying isn't the speedup.
• 18:32
  It's the diagnosis.
• 18:33
  Anyone who's run a real Codex-style agent loop in the last few months has felt this.
• 18:39
  Each tool call goes out as a separate HTTPS request, with TLS handshake, with header overhead, with state being rehydrated.
• 18:48
  When inference was eight or ten seconds per turn, the round-trip cost was rounding error.
• 18:54
  As models got faster, the round-trip cost stopped being rounding error.
• 18:59
  The problem just moved from where the chips are to where the JSON is.
• 19:04
  WebSockets solve a real version of that — keep the connection open, keep the response state warm, stop paying handshake tax on every tool.
• 19:14
  It's the kind of fix that sounds unglamorous until you actually look at a flame graph from an agent loop and realize you've been paying the connection-setup cost on every step.
• 19:27
  The broader point: as inference economics keep shifting, the bottleneck migrates.
• 19:33
  A year ago it was tokens per second.
• 19:35
  Then it was context window.
• 19:37
  Then it was tool-call latency.
• 19:39
  Now it's the request envelope.
• 19:42
  Whatever orchestration framework you're running, ask whether it can take advantage of the WebSocket transport.
• 19:50
  If it can't, you just got slower relative to anyone whose framework can.
• 19:55
  That's not a critique of any particular framework — it's a comment on how fast the floor is moving.
• 20:02
  The shape of agent infrastructure looks more like networking than like ML right now.
• 20:08
  That's not a complaint; that's just where the work is.
• 20:12
  If WebSockets are about reshaping the plumbing under the model, the last thing today is about reshaping the substrate above it.
• 20:20
  I want to close on a small project, because it's the one that made me stop scrolling yesterday.
• 20:27
  It's called Vera.
• 20:28
  It's a programming language designed for machines to write — not for humans to read first, but for LLMs to author, with humans reviewing afterward.
• 20:38
  The repo is on GitHub under aallan, and the design choices are doing the talking.
• 20:43
  There are no variable names.
• 20:45
  Functions don't have human-friendly identifiers; they're addressed structurally.
• 20:50
  Contracts on every function are mandatory.
• 20:53
  The pre and post conditions aren't an optional sugar — they're how the language knows what the function is supposed to do, separate from how it's spelled.
• 21:03
  The pitch, summarized in a top Hacker News comment by danpalmer pulling from the empirical literature on LLM coding: models are particularly vulnerable to naming-related errors — choosing misleading names, reusing names incorrectly, and losing track.
• 21:20
  So Vera removes naming as a problem class.
• 21:23
  Whether that pays off as a serious language is an open question.
• 21:27
  I'm not telling you to use it tomorrow.
• 21:30
  The repo is small, the documentation is rough, and the empirical claim that name-related failures dominate other failures is not yet airtight.
• 21:39
  But the thing the project is doing that I find clarifying is asking the question other languages aren't asking.
• 21:47
  We've been writing programming languages for fifty years on the assumption that the primary author is a human.
• 21:54
  Vera takes the opposite assumption seriously.
• 21:57
  If the primary author is a machine, what would you change?
• 22:01
  You'd remove naming.
• 22:03
  You'd require contracts.
• 22:04
  You'd address things by structure.
• 22:06
  The interesting follow-on is whether any of those choices are also good for human authors — whether mandatory contracts, in particular, are a constraint we'd benefit from regardless.
• 22:19
  Tests are already a poor approximation of contracts.
• 22:22
  We've all written the function whose behavior was specified by the test that was easiest to write.
• 22:29
  Vera says: if you want the function, write the contract.
• 22:32
  Don't write the test.
• 22:34
  There's something appealing about that, even setting LLMs aside.
• 22:38
  The other thing I want to say about Vera, and I think it's the right note to end on, is that the question — what does a programming language for machines look like — is one I'd like more people working on.
• 22:52
  Most of the AI-coding conversation right now is downstream: how do we get the model to use Python better, how do we structure the prompt, how do we wire the tools.
• 23:03
  Vera is upstream of all of that.
• 23:05
  It's asking what we'd do if we let the language change to accommodate the new author.
• 23:11
  I don't know if Vera in particular will go anywhere.
• 23:14
  But the question it's asking is the one I'd watch through the rest of this year.
• 23:20
  That's where I think the next interesting language is going to come from — somebody upstream of the harness, redesigning the substrate.
• 23:29
  — Lenar Kess.