Latest / Braid / The Table Arrived First

Transcript

The Table Arrived First · Braid

• 0:04
  lenar: Yesterday we closed the show on a promise with a date attached to it — Alibaba saying the Qwen3.8 weights would be up next week. We were a little sour about it. Then overnight the sequence went sideways in an interesting way, because the weights still aren't up, but the benchmark table is. There's a post on the r-slash-LocalLLaMA subreddit from a user called davidthesong with a results screenshot for Qwen3.8-Max — two point four trillion parameters — and the claim is that it sits roughly level with Kimi K3 and DeepSeek V4-Flash, and ahead of both on the coding and software-engineering rows.
• 0:43
  damra: [tsk] A screenshot. That's the provenance story right there, before anything else. It's not an Alibaba model card, it isn't a post from the Qwen team, it's an image on Reddit with zero points and zero comments at the time it reached us. The numbers might be exactly right. Nobody in this conversation has seen the artifact they came from.
• 1:03
  lenar: Agreed, and hold that, because elvis — omarsar0 on X — picked it up the same afternoon, and the reason it travels is the comparison target. Not good for an open model. Level with the two systems people currently reach for when they want frontier behavior without a frontier bill.
• 1:20
  damra: The coding row is the one I keep going back to. Sitting level on general reasoning tells you about training compute. Being ahead on software engineering tells you about post-training data and how much agentic trajectory work somebody did. Those are different achievements, and inside the same building they usually come from different teams.
• 1:40
  lenar: We start there today. After that, an unusually specific post from someone who says they work at one of the Chinese labs, laying out how four of them are making structurally different bets. Then Astra — OpenAI put out the list of math problems, and there's a cost number attached now. Then the money, including a leveraged fund that got margin-called out of its entire public book. Then the White House meeting happening today, an argument about developer tools that pulled six hundred and thirty-six points on Hacker News, and a handful of agent shipments.
• 2:12
  damra: Before the labs, though — the other thing that shipped yesterday says something about the release rate itself. Nathan Lambert and the Interconnects team put up an Artifacts Hub, plus an adoption dashboard, on the same day the Qwen numbers went around.
• 2:27
  lenar: Same day, and that's what makes it interesting. Lambert's job for the last two years has been reading every open-weight release and writing up what changed. He's about as well-equipped as anyone alive to keep that in his head. He built a database instead.
• 2:41
  damra: Florian Brand — xeophon — reposted it, and the piece he pulled out was the adoption dashboard, which he called the depressing half. I know exactly what he means. The release list is the fun page. The adoption page is where you find out that a large fraction of these models get downloaded a few thousand times and then nothing happens.
• 3:01
  lenar: So the release rate has outrun our ability to know which releases mattered.
• 3:06
  damra: Which is different from outrunning the ability to read them. I can read a model card in ten minutes. What I can't do is tell you six weeks later whether anybody built on it. That's a longitudinal question, and it needs a table, not a person.
• 3:21
  lenar: Does having the table change how you'd read a day like today?
• 3:24
  damra: It changes what I'd want from the Qwen3.8-Max row specifically. Not the benchmark column — the download and derivative count three weeks out. If a two point four trillion parameter model ships and almost nobody can serve it, the benchmark table is a statement about what Alibaba can do, and the adoption table is a statement about what anybody else can do with it.
• 3:47
  lenar: And the weights still have a date on them rather than a URL. When they show up, the license text is what I'll read before the numbers.
• 3:55
  lenar: The post I keep coming back to from that same subreddit is titled, roughly, the Chinese labs everyone lumps together are making four pretty different bets, I work at one of them. It comes from an account called AcanthisittaOk1699, it's anonymous, and it self-identifies as an employee without naming the employer. Treat it accordingly. What makes it readable anyway is that the claims are about design choices rather than impressions.
• 4:23
  damra: And the four aren't ranked, which I appreciated. He puts Alibaba on coverage — every size and every modality, be the default everywhere. DeepSeek on capability per unit of training compute. Moonshot chasing agentic behavior and long-horizon tool use. And Ant, with the Ling models, optimizing for serving cost, which is the bet nobody writes headlines about.
• 4:46
  lenar: Serving cost is what I'd bet gets underrated for another year. Everybody grades these labs on the benchmark table because the benchmark table is public. Nobody publishes tokens per dollar per unit of quality on their own hardware.
• 5:00
  damra: Cloudflare sort of did, though. They put up a post yesterday — smaller, faster, safer, running Kimi and GLM at scale — that pulled two hundred and thirty points on Hacker News. That's the other half of the picture. One source is a claim about intent, and the other is a measurement of what serving those models actually costs somebody who serves a lot of them.
• 5:22
  lenar: So why does Cloudflare's number carry more with you than Ant's stated intent?
• 5:27
  damra: Because a lab telling you it optimized for serving cost is unfalsifiable until somebody outside the lab runs the model at volume and reports numbers. Cloudflare has the volume. When the people paying the electricity bill write up which models they chose and why, that's a stronger signal about the design bet than the design bet's own press release.
• 5:48
  lenar: There's a third data point in the same neighborhood, much smaller and much more human. Somebody on that subreddit posted, and I'm quoting the headline — "I can't believe I've got DeepSeek-V4-Flash-0731, a frontier model, running on my home PC. Insane." The can't is in capital letters.
• 6:07
  damra: [chuckle] And the capital letters are the actual content of that post. It isn't a benchmark. It's a person having the specific feeling of a floor moving under them. Eighteen months ago that model class needed a rack and a colocation contract.
• 6:21
  lenar: There's a companion post from a user called EmPips — V4-Flash-0731, vibes after the first weekend of use — and it names the quantization before it names the verdict, which is what more practitioner posts should do.
• 6:36
  damra: Which matters because half the arguments about whether an open model is any good turn out to be arguments about which quantized build somebody ran. Second-weekend impressions aren't benchmarks, and I'd still rather read a hundred of them than one vendor chart.
• 6:51
  lenar: So the map is four bets, one anonymous cartographer, and one company with a serving bill big enough to check part of it.
• 6:59
  lenar: On Sunday we spent a chunk of the episode on OpenAI's claim that Astra had solved ten long-open math problems, and the complaint was that the claim was much bigger than the artifact. There was no list. Two days later there's a list. OpenAI posted specifics. The problems run through sphere packing, coding theory, and group theory, and they carry on into quantum complexity, lattice cryptography, and extremal combinatorics — one of them establishes the existence of non-sofic groups.
• 7:30
  damra: And a separate claim on the r-slash-singularity subreddit that the proofs are formalized in Lean 4, with machine-checkable certificates in a GitHub repository. That same post mentions a two hundred and forty-nine page manuscript, and says the inference bill for the whole run came to about two thousand dollars. Every part of that comes from Reddit, not from OpenAI. The problem list is first-party. The repository, the formalization, and the price aren't.
• 8:00
  lenar: The formalization is what changes the epistemics, if it holds up.
• 8:04
  damra: It's the only part that does. A Lean kernel doesn't care who trained the model that wrote the proof. If there's a certificate and it type-checks, you don't have to trust OpenAI's characterization of anything — you run the checker yourself. That's a completely different verification path from a lab saying its model did a hard thing.
• 8:23
  lenar: Which is what Sunday's complaint actually was.
• 8:26
  damra: Right, and I'll be even-handed about the two thousand dollars, because that number is going to get quoted a lot and it's the softest thing in the story. Inference cost for what? The successful runs only, or every search branch that went nowhere? Those differ by orders of magnitude, and nobody has published the accounting.
• 8:44
  lenar: Joscha Bach's read was that the interesting shift isn't the mathematics, it's that models are becoming the primary agents of breakthroughs rather than the instruments. Which is a large claim to hang on ten problems.
• 8:57
  damra: It's a large claim, and it's also exactly the claim these results are structured to support, which is why the Lean files interest me more than the manuscript does. Ten formal certificates settle a question that ten thousand words of narrative can't.
• 9:12
  lenar: Here's a very different item from the same day, and it belongs right next to this. Susan Zhang posted about a draft paper that cites another unpublished draft as support. Nothing to do with Astra or OpenAI. Just a piece of the current preprint ecology.
• 9:28
  damra: Citation rings made of documents that don't exist yet. [sigh] And you can see how it happens without anybody being a villain — generation got cheap, submission got cheap, and reviewing capacity didn't move at all.
• 9:41
  lenar: So in one week you've got machine-checkable mathematics and unverifiable preprints citing each other, and both are downstream of the same capability.
• 9:51
  damra: That's the pairing, and I don't think it resolves. The technology that lets you formalize a proof is the technology that lets you generate a plausible-looking one. The difference is entirely whether the artifact ships with something a machine can check.
• 10:05
  lenar: Which is a reasonable thing to ask of the next lab that says it solved ten problems.
• 10:10
  lenar: Now the money, and this one has an actual event in it rather than another bubble column. Nate Jones covered it on his channel: Leopold Aschenbrenner's fund — the one built on reasoning backward from compute requirements into the supply chain — got margin-called out of its entire public equities book, and Citadel bought the book at a discount.
• 10:29
  damra: The setup matters. That thesis worked. Jones puts it at roughly twenty times last year and better than two times this year. Then it was levered. And what killed it wasn't the thesis being wrong — it was a sag after an SK Hynix offering, plus a Citadel investor note calling for Federal Reserve rate hikes, which made volatile assets less attractive at exactly the wrong moment.
• 10:53
  lenar: And then Griffin buys the position from the person who just got liquidated.
• 10:57
  damra: Which is one of the older stories in finance wearing new clothes. Being right about the direction and wrong about the timing is indistinguishable from being wrong, if you borrowed. He keeps the private startup book, apparently — the illiquid half survives precisely because nobody can margin-call it.
• 11:15
  lenar: [chuckle] The illiquidity is the feature.
• 11:17
  damra: This week, yes.
• 11:19
  lenar: The reason it's more than a personality story is the market context around it. Semiconductor indices are off twenty-three percent from their June peaks. The Korean KOSPI dropped forty percent in a month, driven by retail margin liquidation across more than a million accounts.
• 11:36
  damra: And the argument on the AI Daily Brief is that none of that is about AI demand. Their evidence is revenue: Anthropic's run rate tracked at seventy-one billion dollars, up from forty-seven billion in May, with OpenAI near fifty billion and July alone reportedly beating the whole second quarter. All of that's podcast commentary rather than filings. Nobody has audited those figures in public.
• 12:01
  lenar: Their debt argument is the more interesting one to me, because it's specific. Roughly one point six five trillion dollars of data-center debt structured through special purpose vehicles, sitting off the hyperscalers' balance sheets.
• 12:15
  damra: And the comparison they're answering is 2008, which they say fails for three reasons: the hyperscalers have real cash flows, the paper is sold to private credit and pension funds rather than being mispriced as risk-free, and you'd need actual corporate defaults rather than equity drawdowns before anything breaks.
• 12:34
  lenar: How much of that do you buy?
• 12:36
  damra: The cash-flow point, mostly. The sold-to-private-credit point is the one I'd push on, because the risk is held by someone who knows it's risky is only comforting until you ask who those pension funds are underwriting for. It isn't a 2008 analogue. It's also not nothing.
• 12:53
  lenar: And then Dwarkesh Patel published a piece yesterday arguing something close to the opposite of the comfortable version — that smarter models could push compute prices up rather than down, potentially by a large multiple.
• 13:07
  damra: Which is the counterpoint that makes both stories true at once. If demand outruns supply for years, and better models make each unit of compute more valuable rather than less, then the capital spending is justified and the squeeze arrives for everybody who isn't a hyperscaler. Cheap inference would be a phase rather than a trend line.
• 13:27
  lenar: The Register ran a piece yesterday headlined that the bubble is already popping and we just don't know it yet. Sixteen points, five comments.
• 13:36
  damra: [tsk] And an opinion column. I'd rather have one dated forced sale than ten of those.
• 13:42
  lenar: Same. The forced sale is the one with a date on it.
• 13:45
  lenar: Today, in Washington, representatives from OpenAI, Anthropic, and Google are at the White House for a preview of the finished voluntary AI framework, the one that came out of the June second executive order. This reaches us through a Reddit gallery post rather than a wire story, so hold the details loosely.
• 14:04
  damra: The detail that survives the sourcing caveat is that the companies were lobbying over specific language, and one of the items they cared about is open source. The actual wording is going to decide this, because open source in a policy document is a definitional fight before it's a policy fight.
• 14:21
  lenar: Weights, or weights plus data, or weights plus training code.
• 14:25
  damra: And whether a release with a use restriction still counts. Every large lab has a different answer that happens to match what they already ship. Nobody outside the room has read the text, so I won't characterize provisions — but the lobbying tells you where the money thinks the sharp edge is.
• 14:41
  lenar: There's a small human story from the same day that describes the same fault line from the hiring side. Arnav Gupta posted about somebody scrubbing pro-open-source material off their online presence before an Anthropic interview.
• 14:55
  damra: Secondhand, about an unnamed person, and I'd hate for that to become a claim about Anthropic's hiring policy, because it isn't one. What it is, is a data point about what a candidate believed the room wanted. That's its own kind of information about the culture, even if the belief is wrong.
• 15:12
  lenar: And there's an obvious tension in the fact that the same week, the strongest coding model in that benchmark screenshot is open-weight and Chinese.
• 15:20
  damra: That's the constraint the framework has to survive. A voluntary American framework that discourages open weights doesn't reduce the number of open-weight models in the world. It changes who publishes them.
• 15:32
  lenar: Watch item on the same beat, sourced from a single wire-style post: Apple is reportedly suing the UK over an encryption back-door demand. There's no filing and no document behind it yet.
• 15:44
  damra: Not an AI story today. It becomes one the moment there's a precedent about compelled access to encrypted user data, because on-device inference is going to be sitting inside that boundary within a couple of hardware generations.
• 15:58
  lenar: We pick that up when there's a document to read.
• 16:01
  lenar: A post called devtools must be open source pulled six hundred and thirty-six points and two hundred and eleven comments on Hacker News yesterday. The argument itself is old — you should be able to read and modify the tools you build with.
• 16:16
  damra: And the standard objection to it is older still. The freedom to modify was theoretical for almost everybody. You have the source to your editor. You've never opened it. What's in dispute in that thread is whether large language models just made that freedom usable, because the cost of reading and patching an unfamiliar codebase went from a weekend to an afternoon.
• 16:36
  lenar: Simon Willison is in the thread on that side, and it's the strongest version of the argument. If you can point a model at an unfamiliar repository and get a working patch, then nobody has time to exercise the freedom stops being a defense of closed tooling.
• 16:51
  damra: I'd add the sharper version. It isn't only that you can patch it, it's that you'll actually try. What kept me out of other people's build systems was never ideology, it was the two hours of orientation before the first useful edit. Take that away and license terms start mattering to people who never cared about licenses.
• 17:11
  lenar: Elon Musk posted the opposite bet the same day — that source code is on the verge of becoming like assembly. Meaning models generate binaries directly and the human-readable layer stops being where the work lives.
• 17:24
  damra: [tsk] That's a tweet with nothing shipped behind it, and I hold it at exactly that weight. But it's a clean opposite. One camp says models make source more valuable because now you can actually use it, and the other says models turn source into an intermediate representation nobody reads. Those can't both be the direction.
• 17:43
  lenar: Which would you take?
• 17:44
  damra: The first, and not on principle. Debugging. Everything I've seen about agent-written code says the expensive part is figuring out what it did and why, and that requires an artifact a human can read. Compiling straight to a binary removes the only surface where you can ask that question.
• 18:02
  lenar: There was a smaller post the same day about using task runners for common coding tasks — sixty-six points, a Makefile-shaped argument for giving both humans and agents a named entry point for every routine job.
• 18:15
  damra: It's a small thing that changes agent behavior more than it should. If your repository has a named command for running the tests, an agent finds it. If that knowledge lives in a senior engineer's shell history, the agent invents something worse.
• 18:29
  lenar: Three separate launches yesterday, all pointed at roughly the same gap. Hoplite came up as a Launch HN, out of the Y Combinator summer batch, deploying cloud coding agents — seventy-five points and sixty comments. Armature did a Show HN for product analytics and evaluations on agent sessions over MCP, the Model Context Protocol. And LangChain added bring-your-own-key support to the LangSmith gateway.
• 18:57
  damra: The Hoplite detail I liked is the primitive: one thread equals one virtual machine. That's a design decision rather than a feature. It makes the unit of isolation the conversation, so an agent that wrecks its environment wrecks exactly one environment. Commenters immediately started comparing it to Amp, which is fair.
• 19:16
  lenar: And Armature's thread has the obvious question sitting right at the top of it.
• 19:20
  damra: Which is how you actually capture the model's reasoning. You can log the tool calls, and tool calls are the easy part. The interesting failures happen in reasoning that never crossed a wire. Two comments on that post, so nobody's answered it yet.
• 19:35
  lenar: The LangChain piece is smaller and more practical. Bring your own key means you can put your existing model contracts behind their fallback and rate-limiting machinery instead of buying tokens twice.
• 19:47
  damra: And Cloudflare shipped a billable usage API the same day for programmatic cost visibility, where the top comment is the one you'd expect: it's still not a hard cost cap. Visibility after the fact and a limit that stops the spend aren't the same product, and everybody running agents in production has learned that the expensive way.
• 20:06
  lenar: Two more quick ones. Intology published results claiming their system, Locus, post-trained Qwen3 base models past Qwen's own instruct releases, measured on something they call PostTrainBench.
• 20:19
  damra: Vendor-published, self-named benchmark, single source, and the r-slash-singularity headline was models are now training models, which is considerably larger than the evidence. The claim itself is checkable, though, and that's why it's interesting — beating the original team's instruct tune on their own base model is a specific thing. Give me the benchmark definition and one reproduction from outside Intology.
• 20:44
  lenar: And Cristóbal Valenzuela, Runway's chief executive, argued that with benchmarks saturating, model releases should ship real-world impact measures instead. His example was the number of diseases cured.
• 20:57
  damra: [chuckle] As stated, unworkable. No release ships with a cure count, and no methodology exists to attribute one. But he's pointing at something real, and today is the day to notice it. We spent the first twenty minutes on a benchmark screenshot we can't verify, for a model we can't download, in a table where the differences between three frontier systems are a couple of points.
• 21:20
  lenar: That table stopped discriminating a while ago.
• 21:23
  damra: And nothing replaced it. Miles Brundage was on an adjacent point yesterday about interpretability work aimed at human behavior rather than code. Different subject, same underlying complaint: the measurements we have describe what the model does on the test, not what changes in the world once it's deployed.
• 21:41
  lenar: So — weights promised and a table delivered, proofs claimed and certificates maybe, a fund liquidated for being early with borrowed money. The item I chase first tomorrow is the Qwen3.8-Max license text, because a two point four trillion parameter model with a use restriction and one without are different objects, and the benchmark row looks identical either way.
• 22:04
  damra: And I'd chase the Lean repository. If those certificates exist and they check, that's the one thing from today that stays true no matter who says what about it afterward.
• 22:14
  lenar: Then tomorrow we're checking two files instead of arguing about two claims. Good trade. That's Damra Vol, and I'm Lenar Kess.