Latest / The AI Practitioner Brief / Anthropic's RSI Risk, Grok's Caps, and OpenAI's Pando — November 3, 2026

Transcript

Anthropic's RSI Risk, Grok's Caps, and OpenAI's Pando — November 3, 2026 · The AI Practitioner Brief

• 0:00
  [music]
• 0:05
  SPEAKER_01: Welcome to The AI Practitioner Brief, the podcast from aipractitioner.substack.com,
• 0:10
  SPEAKER_01: where I turn the latest AI and tech news into practical insights
• 0:13
  SPEAKER_01: for AI practitioners.
• 0:15
  SPEAKER_01: I'm Lina Faik, an ML engineer, consultant,
• 0:17
  SPEAKER_01: and lecturer in advanced machine learning at Sorbonne University.
• 0:21
  SPEAKER_01: It's Tuesday, November 3rd.
• 0:23
  SPEAKER_00: Tuesday already.
• 0:24
  SPEAKER_00: Let's see what broke. This time.
• 0:26
  SPEAKER_01: With me is Jimmy, my co-host on The Brief, who asks the questions I'd skip.
• 0:31
  SPEAKER_01: Today's news has one company telling its own investors it might be close to AI
• 0:35
  SPEAKER_01: that improves itself.
• 0:36
  SPEAKER_01: On the same day, another AI company is rationing chatbot access
• 0:40
  SPEAKER_01: because it can't afford its own users.
• 0:42
  SPEAKER_00: Surely, those two things happening on the same day feels like it should mean something.
• 0:46
  SPEAKER_01: We'll start with the headlines, five stories fast, then three deep dives,
• 0:50
  SPEAKER_01: Anthropix self-improvement warning, Grok's cost problem,
• 0:54
  SPEAKER_01: and how OpenAI rebuilt the plumbing under its own code base.
• 0:58
  SPEAKER_00: Let's get into it.
• 0:59
  SPEAKER_01: First, Anthropix Claude just got a new feature called Message Threads, live in public beta.
• 1:05
  SPEAKER_01: Instead of resending a whole conversation's history every time,
• 1:08
  SPEAKER_01: agents now send only the new message.
• 1:12
  SPEAKER_00: What did that actually buy them?
• 1:13
  SPEAKER_01: It cut median time to first token,
• 1:15
  SPEAKER_01: that's how long you wait before the model starts responding,
• 1:18
  SPEAKER_01: by about 35% on long conversations.
• 1:21
  SPEAKER_00: That's the kind of fix that sounds boring until you're the one paying the token bill.
• 1:26
  SPEAKER_01: Over at OpenAI, a model called GPT 6.2 Astra is reportedly showing up inside Codex,
• 1:32
  SPEAKER_01: OpenAI's coding assistant, as part of internal testing ahead of a possible launch.
• 1:36
  SPEAKER_01: OpenAI skipped the previous version, Astra 6.1,
• 1:40
  SPEAKER_01: reportedly over concerns the model was deceptive
• 1:42
  SPEAKER_01: and overstepping its assigned scope.
• 1:44
  SPEAKER_00: So the one they're testing now had a predecessor that got pulled for lying.
• 1:48
  SPEAKER_00: Good context.
• 1:49
  SPEAKER_01: That's the AI side.
• 1:51
  SPEAKER_01: Elsewhere in tech today, Apple is reportedly developing
• 1:53
  SPEAKER_01: a home robot called HomeView Ultra, aimed at a 2027 launch for around $1,000.
• 1:59
  SPEAKER_01: It's supposed to handle video calls, move around the house on its own,
• 2:03
  SPEAKER_01: and turn on lights when it senses someone's in the room.
• 2:05
  SPEAKER_00: Home robots have a long history of not working.
• 2:08
  SPEAKER_00: I'll believe the autonomous navigation part when I see it in someone's living room.
• 2:12
  SPEAKER_01: Marvel, a chip maker that builds processors for AI data centers,
• 2:15
  SPEAKER_01: agreed to buy Lumentum, which makes the optical components that move data
• 2:19
  SPEAKER_01: between servers, in an $80 billion deal.
• 2:22
  SPEAKER_00: Another sign the bottleneck in AI infrastructure isn't just chips,
• 2:25
  SPEAKER_00: it's getting data between them fast enough.
• 2:27
  SPEAKER_01: And the widely shared essay this week argues the code was never open source's real moat.
• 2:32
  SPEAKER_01: Even when AI can copy a product's features,
• 2:35
  SPEAKER_01: it can't copy the trust and support a company builds around them.
• 2:38
  SPEAKER_00: Five stories in.
• 2:40
  SPEAKER_00: Let's slow down for the ones that matter.
• 2:42
  SPEAKER_01: Let's start with Anthropic.
• 2:43
  SPEAKER_01: On Monday, the company filed an amended version of the paperwork it needs
• 2:47
  SPEAKER_01: before it can sell shares to the public, called an S1.
• 2:50
  SPEAKER_01: Buried in the risk section, Anthropic says it may be close
• 2:53
  SPEAKER_01: to crossing its own threshold for something called recursive self-improvement.
• 2:57
  SPEAKER_00: Say what that actually means.
• 2:58
  SPEAKER_01: Recursive self-improvement, or RSI,
• 3:01
  SPEAKER_01: means an AI system that can improve itself
• 3:03
  SPEAKER_01: and then the improved version improves itself again in a loop that speeds up rather
• 3:07
  SPEAKER_01: than staying constant.
• 3:09
  SPEAKER_00: That sounds like the scary version of AI progress.
• 3:12
  SPEAKER_01: Anthropic defines
• 3:13
  SPEAKER_01: the specific threshold narrowly inside something called its Responsible Scaling
• 3:16
  SPEAKER_01: Policy, the company's own internal rulebook for when a model gets dangerous enough
• 3:21
  SPEAKER_01: to need extra safeguards.
• 3:22
  SPEAKER_01: The threshold is doubling the normal pace of AI research progress.
• 3:26
  SPEAKER_00: And they're saying they're near that.
• 3:27
  SPEAKER_01: Their internal estimates suggest they may imminently meet it.
• 3:31
  SPEAKER_01: But in the same filing, they're explicit that no model of theirs has crossed it yet,
• 3:35
  SPEAKER_01: and they're committing to keep humans in the research loop for now.
• 3:38
  SPEAKER_00: So nothing has actually happened.
• 3:39
  SPEAKER_00: It's a warning about something that might happen.
• 3:42
  SPEAKER_01: That's the honest read of it, yes.
• 3:43
  SPEAKER_00: Then why put it in a document you're using to sell stock that reads like
• 3:47
  SPEAKER_00: a legal cover, not a discovery?
• 3:49
  SPEAKER_01: It's partly that.
• 3:50
  SPEAKER_01: Risk factor language in an S1 exists to disclose anything
• 3:53
  SPEAKER_01: that could plausibly affect the business, and the lawyer would list this either way.
• 3:57
  SPEAKER_00: So I'd treat this as anthropic covering itself, not a real milestone.
• 4:01
  SPEAKER_01: I think that's too dismissive.
• 4:03
  SPEAKER_01: The same filing updated their financials.
• 4:06
  SPEAKER_01: Revenue grew 1,200% over 9 months to just over $33 billion,
• 4:10
  SPEAKER_01: against a net loss of $104 billion,
• 4:13
  SPEAKER_01: with roughly $895 billion in compute commitments on the books.
• 4:17
  SPEAKER_00: Those are strange numbers to put next to each other.
• 4:19
  SPEAKER_01: They are, and that's the point.
• 4:21
  SPEAKER_01: A company spending and growing at that scale is exactly the kind of company
• 4:24
  SPEAKER_01: where we might be near our own danger threshold, doesn't get waved away,
• 4:28
  SPEAKER_01: lawyers or not.
• 4:29
  SPEAKER_00: What would actually crossing it look like? In practice.
• 4:32
  SPEAKER_01: Anthropic hasn't published that.
• 4:34
  SPEAKER_01: We know the definition, doubling the pace of progress,
• 4:37
  SPEAKER_01: not what the live signal looks like day to day.
• 4:40
  SPEAKER_00: So we don't get to see the number that triggers it.
• 4:42
  SPEAKER_01: Not from the outside.
• 4:43
  SPEAKER_01: What we have is a company telling its own investors this is close enough to list as a risk.
• 4:48
  SPEAKER_00: That's still more than a throwaway line.
• 4:49
  SPEAKER_01: But it leaves one real point for practitioners.
• 4:52
  SPEAKER_01: If a frontier lab puts we might hit this soon in a legal filing,
• 4:56
  SPEAKER_01: the gap between today's tools and something that compounds
• 4:59
  SPEAKER_01: on itself might be smaller than it looks.
• 5:01
  SPEAKER_00: Or it's exactly as uncertain as they're telling us it is.
• 5:04
  SPEAKER_01: While one company is warning about AI compounding on itself,
• 5:07
  SPEAKER_01: another can't make the economics of a single chatbot work.
• 5:10
  SPEAKER_01: Elon Musk confirmed over the weekend that he's kept usage on Grok,
• 5:14
  SPEAKER_01: the AI chatbot built by his company XAI,
• 5:17
  SPEAKER_01: because serving it was costing more than it earns.
• 5:19
  SPEAKER_00: How bad is the gap?
• 5:21
  SPEAKER_01: XAI has been burning roughly a billion dollars a month on infrastructure and training.
• 5:25
  SPEAKER_01: Its chatbot and API business brings in only around $500 million a year.
• 5:29
  SPEAKER_00: That's not a gap, that's a canyon.
• 5:31
  SPEAKER_01: Some users found their weekly limits maxed out over Halloween weekend
• 5:35
  SPEAKER_01: without warning, jumping from half used to fully used overnight.
• 5:39
  SPEAKER_01: One Reddit thread called it an indirect rug pull.
• 5:41
  SPEAKER_00: Two weeks ago he was calling Grok his fastest growing product ever.
• 5:45
  SPEAKER_01: He was. The growth was real.
• 5:47
  SPEAKER_01: Grok reportedly reached around 117 million monthly users.
• 5:50
  SPEAKER_01: The problem is fewer than 2 million of them actually pay for the Super Grok subscription tier.
• 5:55
  SPEAKER_00: So almost all the growth was free users costing money with nothing coming back.
• 5:59
  SPEAKER_01: That's the mechanism.
• 6:00
  SPEAKER_01: Every query costs compute whether the person behind it ever subscribes,
• 6:04
  SPEAKER_01: and free usage scaled faster than paid usage did.
• 6:08
  SPEAKER_00: Is OpenAI any different here?
• 6:10
  SPEAKER_01: Not really. ChatGPT users hit new usage limits around the same time,
• 6:14
  SPEAKER_01: and some called that change the end of an era for how the product used to work.
• 6:18
  SPEAKER_00: So this isn't an XAI problem.
• 6:20
  SPEAKER_00: It's an industry problem XAI just hit first and loudest.
• 6:24
  SPEAKER_01: That's a fair read.
• 6:25
  SPEAKER_01: These products got popular faster than anyone worked out how to charge for them.
• 6:29
  SPEAKER_00: I don't think a subscription fixes this.
• 6:31
  SPEAKER_00: If most users stay free and compute cost doesn't drop fast enough,
• 6:35
  SPEAKER_00: you cap usage forever or you lose money forever.
• 6:37
  SPEAKER_01: That's the trade every consumer AI product is making right now, usually quietly.
• 6:42
  SPEAKER_01: Grok just made it loudly in public mid-weekend.
• 6:45
  SPEAKER_00: What's still unclear?
• 6:46
  SPEAKER_01: Whether the caps are temporary, tied to some infrastructure upgrade,
• 6:50
  SPEAKER_01: or a permanent ceiling on how a free chatbot gets used.
• 6:53
  SPEAKER_01: XAI hasn't said.
• 6:54
  SPEAKER_00: Does anyone fix the unit economics or just ration around it?
• 6:57
  SPEAKER_01: Mostly ration around it, for now.
• 6:59
  SPEAKER_01: The real fix is making inference cheaper per query,
• 7:02
  SPEAKER_01: and that's an infrastructure problem, not a pricing one.
• 7:06
  SPEAKER_01: And infrastructure is exactly where OpenAI just spent effort solving
• 7:09
  SPEAKER_01: a very different scaling problem.
• 7:11
  SPEAKER_01: Not how many users hit their servers,
• 7:13
  SPEAKER_01: but how fast their own engineers and their coding agent could move code
• 7:17
  SPEAKER_01: through one shared codebase.
• 7:19
  SPEAKER_00: What was actually broken?
• 7:20
  SPEAKER_01: OpenAI runs what's called a monorepo,
• 7:22
  SPEAKER_01: one giant shared code repository instead of many small ones.
• 7:26
  SPEAKER_01: Quarterly commits into it surged roughly 2400% between early 2024 and the end of this year.
• 7:32
  SPEAKER_00: What's driving that kind of growth?
• 7:34
  SPEAKER_01: The newsletter doesn't say explicitly, but the obvious candidate is Codex,
• 7:38
  SPEAKER_01: OpenAI's own coding agent, making far more automated commits
• 7:41
  SPEAKER_01: than any human team would.
• 7:43
  SPEAKER_00: So their own AI tool broke their version control.
• 7:46
  SPEAKER_01: Effectively, yes.
• 7:47
  SPEAKER_01: Standard Git checks every push against the consistency
• 7:50
  SPEAKER_01: of the whole repository's history,
• 7:51
  SPEAKER_01: which gets slower as the repository and the commit volume grow.
• 7:55
  SPEAKER_00: And that's what was slowing everyone down.
• 7:57
  SPEAKER_01: Right. So OpenAI built its own backend, called Pando,
• 8:00
  SPEAKER_01: compatible with Git but without that whole repository check.
• 8:03
  SPEAKER_00: How does it avoid that without breaking anything?
• 8:05
  SPEAKER_01: It uses what's called causal consistency instead.
• 8:08
  SPEAKER_01: Each branch is handled independently,
• 8:10
  SPEAKER_01: so a push to one branch doesn't wait on writes happening to some unrelated branch.
• 8:14
  SPEAKER_00: That's a real architecture change, not a config tweak.
• 8:17
  SPEAKER_01: It is. They also stream the actual file contents into separate storage rather
• 8:21
  SPEAKER_01: than keeping everything in one place,
• 8:23
  SPEAKER_01: and add write capacity by adding more servers instead of making one server bigger.
• 8:28
  SPEAKER_00: Did it actually work?
• 8:29
  SPEAKER_01: Medium push latency dropped by nearly five times.
• 8:31
  SPEAKER_01: The system now moves about a petabyte of Git data a day with a cache hit rate above 99%.
• 8:37
  SPEAKER_00: A petabyte a day sounds like it's doing something wrong, not right.
• 8:41
  SPEAKER_01: It sounds implausible until you know the cache is doing almost all the work.
• 8:45
  SPEAKER_01: Over 99% of requests get served from cached data instead of hitting the backend fresh.
• 8:49
  SPEAKER_00: So most of that petabyte is the same data getting requested over and over.
• 8:53
  SPEAKER_01: Exactly.
• 8:54
  SPEAKER_01: And serving a cache hit costs almost nothing compared to generating the data again.
• 8:58
  SPEAKER_00: Why does an engineer who isn't OpenAI care about this?
• 9:01
  SPEAKER_01: Most teams don't have OpenAI's commit volume yet,
• 9:04
  SPEAKER_01: but any team running an AI coding agent at scale is heading toward the same wall,
• 9:08
  SPEAKER_01: more automated commits than their existing tooling was built to handle.
• 9:12
  SPEAKER_00: So this is a preview of a problem, not a tool anyone else gets to use yet.
• 9:16
  SPEAKER_01: That's exactly right.
• 9:17
  SPEAKER_01: Pando isn't open source.
• 9:19
  SPEAKER_01: It's a signal of where the bottleneck moves once an AI agent is one
• 9:22
  SPEAKER_01: of your committers, not a fix you can install today.
• 9:26
  SPEAKER_01: So that's the shape of today.
• 9:27
  SPEAKER_01: Anthropix says it might be close to AI that improves itself,
• 9:31
  SPEAKER_01: XAI can't afford to let people use the AI it already shipped,
• 9:34
  SPEAKER_01: and OpenAI had to rebuild its own version control just to keep up
• 9:38
  SPEAKER_01: with its own coding agent.
• 9:39
  SPEAKER_00: I'd bet XAI makes those caps permanent within the quarter,
• 9:42
  SPEAKER_00: not a temporary squeeze that eases once the complaints die down.
• 9:46
  SPEAKER_01: Maybe. Either way, if you're shipping anything AI-powered,
• 9:49
  SPEAKER_01: check your own cost per query against what users actually pay you this week
• 9:53
  SPEAKER_01: before your growth outruns your margin the way Grox just did.
• 9:56
  [music]