Latest / Braid / Bring Your Own Numbers

Transcript

Bring Your Own Numbers · Braid

• 0:04
  Mustafa Suleyman is the CEO of Microsoft AI.
• 0:06
  He told the Financial Times earlier this year that most tasks involving — in his words — sitting down at a computer will be automated within twelve to eighteen months.
• 0:17
  He named accounting, legal, marketing, and project management as in scope.
• 0:21
  The Fortune piece that ran yesterday is a re-publication of that February conversation, and Fortune itself spends a good chunk of the article explaining why the take hasn't held up.
• 0:32
  The counter-data is in the same byline.
• 0:35
  A 2025 Thomson Reuters report on lawyers, accountants, and auditors found marginal productivity gains from targeted use — document review and routine analysis — and nothing close to mass displacement.
• 0:47
  A METR study on software developers, which we've talked about before on this show, found AI assistance made the workers' tasks take twenty percent longer on average.
• 0:57
  Torsten Slok at Apollo Global Management ran the margin numbers for fourth-quarter 2025.
• 1:03
  Big Tech profit margins grew more than twenty percent, while the broader Bloomberg 500 saw almost no change.
• 1:09
  Slok also reported that investors don't expect AI to lift earnings outside the tech sector at all.
• 1:15
  There are some real layoff numbers in the article too.
• 1:19
  Challenger, Gray and Christmas counts about forty-nine thousand AI-related job cuts so far in 2026.
• 1:25
  Microsoft itself let go fifteen thousand workers in 2025, in a round that Nadella tied to, quote, reimagining our mission for a new era.
• 1:33
  But that isn't the same shape as the eighteen-month claim.
• 1:37
  Suleyman is forecasting a near-total replacement of office cognition.
• 1:41
  What's actually visible is targeted automation, modest productivity, and selective headcount cuts at companies already restructuring.
• 1:49
  I don't think Suleyman is being cynical here.
• 1:52
  I think he believes it.
• 1:54
  The Microsoft AI charter he's articulating — creating a new model is going to be like creating a podcast or writing a blog — is a worldview, not a quarterly forecast.
• 2:04
  And it's a worldview I find exciting in places, because if even part of it lands, the creative ceiling for an individual builder goes up a lot.
• 2:13
  What I'm skeptical about is the calendar.
• 2:15
  The eighteen months in early 2025 from Dario Amodei was for half of entry-level white-collar jobs, and Amodei has since walked it back.
• 2:24
  The eighteen months in early 2026 from Suleyman is for the whole white-collar tier.
• 2:29
  The number stays the same.
• 2:30
  The goalpost moves outward.
• 2:32
  So when you hear the headline today, here's a useful tell: the article you're reading probably already includes its own qualifier two paragraphs down.
• 2:41
  That's a pretty good clue about where the consensus actually sits right now.
• 2:46
  Most of the show today is about what AI actually costs in dollars and time, and what hands-on builders are finding when they sit down with the tools.
• 2:55
  There's a loose connection between most of the items, and I'll let you draw it on your own.
• 3:01
  The most interesting business writing I read this weekend is a piece from The State of Brand titled "Every AI Subscription Is a Ticking Time Bomb for Enterprise." It's long, and there's a sentence in it that stuck with me.
• 3:14
  Quote: They are selling enterprises filet mignon at gas station hot dog prices and calling it a business model.
• 3:20
  The argument runs like this.
• 3:22
  Claude Pro is twenty dollars a month.
• 3:24
  Sonnet 4.6 on the API is three dollars per million input tokens and fifteen per million output.
• 3:29
  A knowledge worker uploading documents and running analyses through Claude for a couple of hours a day burns through several million tokens a week.
• 3:38
  At API rates, that workload runs two hundred to four hundred dollars a month per seat.
• 3:43
  On a Pro plan, the company pays twenty.
• 3:45
  The piece pulls in the supporting numbers from across the industry.
• 3:49
  Microsoft was reportedly losing more than twenty dollars per user per month on GitHub Copilot back when it was a ten-dollar product, with power users consuming eighty dollars of compute.
• 4:00
  One widely-cited analysis found Anthropic users consuming about eight dollars in compute for every dollar of subscription revenue.
• 4:07
  Nick Turley, OpenAI's VP of Product, has described their subscription pricing as something they, quote, stumbled into, and floated phasing out unlimited plans altogether, comparing them to unlimited electricity.
• 4:20
  Then comes the agentic shift, which is where the math goes from bad to unworkable.
• 4:24
  Claude Code sessions run autonomously for long stretches.
• 4:28
  Users have reported burning through five-hour rate-limit windows in under ninety minutes.
• 4:33
  GitHub announced that Copilot moves to usage-based AI Credits billing on June 1, in just over two weeks.
• 4:39
  GitHub's own announcement attributes the change to agentic usage becoming the default and producing higher compute and inference demands.
• 4:47
  Sam Altman said publicly that OpenAI now needs to become, in his words, an AI inference company, which is an interesting way to describe a company that already sells inference.
• 4:57
  There's a longer arc here too.
• 4:59
  Anthropic is at roughly thirty billion dollars in annualized revenue, up from nine billion at the end of 2025.
• 5:05
  OpenAI is at about twenty-five.
• 5:07
  OpenAI is projecting one hundred and fifteen billion in cumulative cash burn through 2029 and has committed to six hundred and sixty-five billion in compute spending by 2030.
• 5:17
  Oracle took on forty-three billion of debt in a single fiscal year to build out OpenAI's data centers.
• 5:23
  That infrastructure is financed on the assumption that revenue will eventually cover costs.
• 5:28
  Right now it doesn't, and the article's claim is that going public is the forcing function.
• 5:34
  Public markets won't tolerate the unit economics.
• 5:37
  The subsidies end when the IPO bell rings.
• 5:39
  The reason this matters for builders is concrete.
• 5:42
  Last Wednesday I talked about Anthropic's Agent SDK metering and the question of how programmatic users get billed when humans aren't in the loop.
• 5:50
  The State of Brand piece is the same pressure from the other direction.
• 5:54
  If you've built a workflow on twenty-dollar seats over the last two years, the gap between your subscription line item and the actual unit cost of the work is the same number every CFO is about to be made aware of.
• 6:07
  What I'd actually do, if it were my budget: model out what your AI line looks like at two-x, five-x, and ten-x current prices.
• 6:14
  Make sure no single vendor's pricing change can blow up the quarter.
• 6:18
  And have the conversation with finance before finance has it with you.
• 6:22
  The piece quotes Brian Jabarian, an economist at the University of Chicago who consults with companies on AI adoption, with the cleanest framing.
• 6:31
  Quote: The time for the bill is going to come.
• 6:33
  I don't think it'll come everywhere at once.
• 6:36
  I do think the GitHub date — June 1, 2026 — is going to be the first concrete data point a lot of teams have on what their actual usage costs, and the spread between subscription pricing and metered pricing is going to surprise people.
• 6:50
  While we're on costs, here's a counterintuitive one.
• 6:52
  William Angel has a short post in his Offline Agentic Coding series, and the headline is, Apple Silicon costs more than OpenRouter.
• 7:00
  His M5 MacBook Pro Max with sixty-four gigs of RAM lists for about four thousand three hundred dollars.
• 7:06
  He amortizes that over three, five, and ten years, and gets a per-hour hardware cost of sixteen cents, ten cents, and five cents respectively.
• 7:14
  Electricity is almost noise.
• 7:16
  At fifty to one hundred watts under inference load, and Northern Virginia's roughly eighteen cents per kilowatt-hour, you're paying about one to two cents an hour for power.
• 7:26
  Hardware depreciation dominates the cost stack, not electricity.
• 7:29
  The tokens-per-hour math is where it lands.
• 7:32
  He's seeing ten to forty tokens per second on Gemma 4, the 31-billion-parameter version, which he describes as close to Sonnet-level performance.
• 7:40
  At ten tokens per second on the pessimistic three-year hardware life, that works out to about four dollars and seventy-nine cents per million tokens.
• 7:49
  At forty tokens per second on a ten-year hardware life, it's about forty cents.
• 7:53
  OpenRouter serves the same Gemma 4 model at around thirty-eight to fifty cents per million tokens, and providers there are getting sixty to seventy tokens per second — three to seven times faster than the local Max.
• 8:05
  Angel's bottom line is that local inference on the Pro Max runs about three times the cost per token of OpenRouter from an accounting view, and the speed gap is the bigger problem.
• 8:16
  For an employee on a work laptop whose salary is a thousand times the cost of the tokens they can generate locally, throwing money at Anthropic, in his words, makes more sense.
• 8:26
  I find this useful because the local large-language-model conversation usually skips over the depreciation line entirely.
• 8:33
  The implicit math people do is, I bought the laptop anyway, the tokens are free.
• 8:37
  Angel makes you count the hardware against the throughput, and the answer is that today, on consumer Apple silicon, the hosted API still wins on raw cost per token until you start putting human time in the equation.
• 8:50
  He ends on a line I liked: It's still wild that a consumer device can run models that are close to Anthropic Sonnet levels of performance.
• 8:58
  Both things can be true at once.
• 8:59
  The cost picture argues against running it locally, and the capability picture is still extraordinary.
• 9:05
  One small caveat: this is a 31-billion-parameter model in May 2026.
• 9:09
  The cost curve will keep falling for both local and hosted, but the speed gap is structural.
• 9:14
  Datacenter GPUs with high-bandwidth memory will keep beating laptops at single-stream throughput for a while.
• 9:21
  If you're picking between the two for an agent loop where every second matters, the answer right now is the hosted endpoint.
• 9:28
  Frederick Vanbrabant published a piece a couple of days ago titled, I don't think AI will make your processes go faster, which made the front page of Hacker News this morning.
• 9:38
  Frederick writes about enterprise architecture and product strategy, and he re-read two old classics for the post — The Toyota Way, and Eli Goldratt's The Goal — and applied them to the AI-build-out conversation.
• 9:50
  The setup is a Gantt chart of a typical software project: scoping, exploration, development, deployment, and hyper-care.
• 9:57
  Development is the longest bar.
• 9:59
  If your job is to speed the project up, you go look at development.
• 10:03
  That's correct.
• 10:04
  Where people go wrong, he says, is at the next step.
• 10:07
  They either throw more people at it — which Brooks already covered in The Mythical Man-Month — or they assume AI is going to collapse the development bar without anything else changing.
• 10:18
  Vanbrabant's quick redraw is that the development bar does shrink with AI in the loop, and then the scoping and documentation bar gets much wider, because the agent needs everything spelled out.
• 10:29
  The total project length improves, but not by as much as the marketing claims, and almost all of the new effort lands on the domain experts who have to write the spec.
• 10:39
  His direct quote, lifted from Goldratt: bottlenecks should receive predictable, high-quality inputs.
• 10:45
  That's the line I'd put on a Post-it next to most agent setups.
• 10:49
  The argument generalizes.
• 10:50
  Every team that's serious about agent-driven development is converging on something like this.
• 10:56
  Brian Scanlan at Intercom — in the piece I covered yesterday on doubling engineering throughput with Claude Code — describes a similar shift in where senior time goes.
• 11:05
  The agents do the typing.
• 11:07
  The humans do the specification.
• 11:09
  Vanbrabant's punchline is the one I keep coming back to.
• 11:12
  He says if you handed human developers the same depth of feature and scope documentation that an agent needs to do good work, you would also see human productivity skyrocket.
• 11:22
  Software development was never primarily slow because of typing speed.
• 11:26
  It was slow because the question wasn't well-posed.
• 11:29
  What bugs me about a lot of AI-doubles-productivity claims, including the bullish ones I find believable, is that they tend not to control for that.
• 11:38
  When a senior developer pairs with a coding agent for three hours and produces five well-tested PRs, some of that delta is the model.
• 11:46
  Some of it is that the senior was forced to think carefully about each task before talking to the model.
• 11:52
  Doing this decomposition properly is harder than the headline, and Vanbrabant's piece is a good reminder.
• 11:58
  I don't fully agree with him on one thing.
• 12:00
  He treats the AI productivity argument as something close to zero-sum — that humans plus better specs would close most of the gap.
• 12:08
  I think that understates how much faster the typing-and-revising loop gets when you don't have to type or revise.
• 12:15
  But the directional point is right, and the bottleneck does move.
• 12:18
  Marlene Mhangami gave a talk at AI Engineer Singapore titled, Beyond Code Coverage: Functionality Testing with Playwright.
• 12:25
  She's a senior developer advocate at Microsoft and GitHub in their Core AI group, which looks at how developers use AI across the products.
• 12:33
  She opens with a number.
• 12:35
  GitHub saw about a billion commits in 2025, which was already their biggest year ever.
• 12:40
  Their COO, Kyle Daigle, recently said the platform is now seeing about two hundred and seventy-five million commits per week, which extrapolates to roughly fourteen billion by the end of 2026.
• 12:51
  A growing share is co-authored by agents.
• 12:53
  Claude co-signs commits.
• 12:54
  Copilot does.
• 12:55
  Codex doesn't, but there's enough wording in the diffs to track it indirectly.
• 12:59
  So she asks the question that follows: does any of this code translate into productivity?
• 13:05
  She points to a Stanford study of a hundred and twenty thousand developers that found the answer depends entirely on the codebase.
• 13:12
  Clean codebases amplify AI gains.
• 13:14
  Unchecked AI in a messy codebase amplifies entropy.
• 13:17
  The talk shows a case study where one team's PR throughput went up substantially after adopting an AI tool, and effective output — net of refactoring and rework — rose about one percent.
• 13:27
  Her recommendation is for behavior-first test-driven development with Playwright, and she's quite specific about why.
• 13:34
  AI-generated unit tests tend to be self-affirming.
• 13:37
  They assert that the function does what the function does.
• 13:40
  The test suite goes green, and the user-facing behavior is still broken.
• 13:44
  This is, in my experience, the single most common pathology of an agent that's been told to increase test coverage without further instruction.
• 13:52
  The pattern she walks through is the red-green-refactor cycle, but with the labor split differently.
• 13:58
  The agent writes a Playwright end-to-end test against the expected user behavior first, in the red phase.
• 14:04
  The agent then writes the code to make the test pass, in the green phase.
• 14:08
  And the human spends the most time on refactor, because that's where the quality lives.
• 14:13
  She demos it live with the Playwright MCP server and GitHub Copilot CLI, picking up a Microsoft 365 email about a feature request through a skill called Work IQ, generating the failing test, generating the implementation, and watching Playwright drive the browser through the search bar, the category filter, and the price range.
• 14:32
  Two of her closing pointers stuck with me.
• 14:34
  First, commit your code before letting the agent fix the failing tests, because if it doesn't commit, it may not remember what changed.
• 14:42
  Second, generate one feature per test — the trigger for writing a new test is no longer a new method on a class, it's a new behavior in the spec.
• 14:50
  I think this is one of the more practical patterns I've seen for the AI-generates-tests problem.
• 14:56
  It doesn't solve it.
• 14:57
  The agent can still write a Playwright test that confirms the wrong behavior, because the spec was wrong, and Vanbrabant's whole point from the last chapter applies.
• 15:07
  But it does close the most common false-positive — the test that asserts the implementation it just generated.
• 15:13
  If you're trying this, the Playwright agents the Microsoft team ships — a planner, a generator, and a healer agent set — install as agent dot M-D files into your repo, and they're the lightest way I've seen to get the loop running.
• 15:26
  On Friday I talked about Calif and Mythos Preview's five-day macOS kernel exploit on Apple's M5 silicon.
• 15:32
  The write-up is now public, and a few specifics from their own post are worth getting on record.
• 15:38
  The chain is a data-only kernel local-privilege-escalation exploit on macOS 26 point 4 point 1 — that's build twenty-five-E-two-fifty-three — on bare-metal M5 hardware with kernel MIE enabled.
• 15:50
  MIE is Memory Integrity Enforcement, Apple's hardware-assisted memory safety system built on ARM's Memory Tagging Extension, which Apple spent five years and probably billions of dollars building.
• 16:02
  According to Apple's own research, MIE disrupts every public exploit chain against modern iOS, including the leaked Coruna and Darksword kits.
• 16:10
  The timeline from the Calif post is tight.
• 16:13
  Bruce Dang found the bugs on April twenty-fifth.
• 16:16
  Dion Blazakis joined Calif on April twenty-seventh, and Josh Maine built tooling.
• 16:21
  By May first, they had a working exploit.
• 16:23
  The implementation path involves two vulnerabilities and several techniques, starts from an unprivileged local user, uses only normal system calls, and ends with a root shell.
• 16:34
  The split-of-labor claim is what I find most interesting.
• 16:37
  Quoting Calif directly.
• 16:39
  Mythos Preview is powerful: once it has learned how to attack a class of problems, it generalizes to nearly any problem in that class.
• 16:47
  Mythos discovered the bugs quickly because they belong to known bug classes.
• 16:51
  But MIE is a new best-in-class mitigation, so autonomously bypassing it can be tricky.
• 16:56
  This is where human expertise comes in.
• 16:59
  End quote.
• 17:00
  That's the cleanest articulation I've seen of how this pairing actually works today.
• 17:05
  The model is fast and generalizing over known vulnerability classes.
• 17:09
  The novel-mitigation bypass — inventing a new technique against a defense that's never been broken in public — is still on the humans.
• 17:17
  The advantage of the pairing is wall-clock time.
• 17:20
  A week from first bug to root shell against the best memory-safety mitigation Apple has ever shipped.
• 17:26
  The post also includes a story about going to Apple Park to hand-deliver a laser-printed fifty-five-page report, in honor of their hacker friends, instead of dropping it into Apple's submission queue.
• 17:38
  There's a line in there I enjoyed.
• 17:40
  Their hosts at Apple shared that the company spent five billion dollars on the spaceship office, then asked about Calif's office.
• 17:48
  They said theirs definitely cost less than one billion.
• 17:51
  The Vietnamese phrase Calif uses to close the post is nhỏ mà có võ — small but fierce.
• 17:57
  What I'll be looking for next is the full technical report, which Calif says will land after Apple ships the fix.
• 18:03
  The five-day timeline is the lede everyone is repeating, but the implementation details of a data-only MIE-surviving chain will tell us much more about what generalizes to the next mitigation Apple builds.
• 18:16
  That's where the industry actually learns something.
• 18:19
  Artem Loenko has a piece on his blog today called, Native all the way, until you need text.
• 18:24
  Artem has been a native macOS and iOS developer for almost twenty years, and the post is a slightly tired walk through what happened when he tried to build a streaming Markdown chat UI in pure Swift and SwiftUI.
• 18:37
  The journey is funny in a way only Apple-platform developers will fully appreciate.
• 18:42
  He starts with SwiftUI.
• 18:43
  Scrolling is jumpy but tolerable.
• 18:45
  Then he tries to let the user select a whole Markdown document built from SwiftUI primitives, and finds out he can't.
• 18:52
  By design.
• 18:53
  So he moves to NSTextView with TextKit 2.
• 18:56
  Selection works.
• 18:57
  But now he's outside the SwiftUI tooling story, and when he tries to stream model output into the view, he sees CPU spikes.
• 19:04
  Fine.
• 19:05
  He moves to AppKit with NSCollectionView, which is mature and well-proven, and on day two he realizes the cells will blink during streaming.
• 19:13
  By design.
• 19:14
  He goes lower-level to pure TextKit 2.
• 19:16
  The prototype is okay on performance, terrible on streaming, and breaks every modern integration.
• 19:22
  He gives up on SwiftUI entirely, fights expanding text chunks by hand in AppKit, and reaches the moment where the text is selectable and almost nothing else works.
• 19:32
  Then he counts the cost of just reaching baseline native parity.
• 19:36
  Context menus, dictionary lookup, accessibility, and text interactions — all the small things a Mac user takes for granted.
• 19:43
  Months of work.
• 19:44
  So he tries WebKit for Markdown rendering.
• 19:46
  It works.
• 19:47
  There are caveats, but mostly it just works.
• 19:50
  Then, in his words, at the darkest possible moment, he generates an Electron prototype.
• 19:55
  And he is amazed.
• 19:56
  Text operations, Markdown rendering, good typography, and even Git diffs work out of the box, with performance he could not get from his pure TextKit 2 implementation.
• 20:06
  His conclusion: chat as an interface pattern — long-form rich text, flexible typography, streaming responses — is web-native today.
• 20:14
  SwiftUI is fine for simple screens.
• 20:16
  Swift is great for performance-critical paths.
• 20:19
  But the rendering and text model you need for a modern AI chat surface lives in WebKit or in Chromium, and the gap, in his words, isn't a shortcut-versus-proper-solution debate anymore.
• 20:30
  It's a feature gap.
• 20:31
  I think this is one of those posts that explains a thing developers have been quietly observing for two or three years now without quite articulating.
• 20:40
  Why does every new AI client end up shipped in Electron, as a web app, or wrapped in a WK WebView?
• 20:46
  Because the dominant interaction pattern of the era happens to be the one the native SDKs handle worst.
• 20:52
  And the cost of bridging that gap — accessibility, selection, streaming, and context menus — adds up to months that almost no team can spare when they're shipping a model client into a market that turns over every six weeks.
• 21:06
  I'd love to be wrong about this.
• 21:08
  I'd love for SwiftUI to ship a first-class streaming-Markdown text view next year.
• 21:12
  Until then, Artem's piece is the most honest account I've read of why we keep ending up on the dark side.
• 21:18
  Closing with the smallest item on the lineup, which is also my favorite.
• 21:22
  Luke Lanchester runs HybridLogic, and he published a short post called MCP Hello Page about a fix he made to his company's MCP server that I think is good craft.
• 21:31
  His problem: customers were getting his MCP server URL and pasting it into a browser, where they would get a four-oh-one Unauthorized status with a raw JSON blob.
• 21:40
  They would then file a support ticket saying the link was broken.
• 21:44
  He'd then explain that the URL is for an MCP client, not a browser, and that they need to paste it into Claude or Cursor or whatever they're using.
• 21:52
  This happened a lot.
• 21:53
  He calls it a never-ending game of whack-a-mole, because the alternative — building a connector per LLM client — doesn't scale either.
• 22:01
  The fix is a single content-negotiation rule.
• 22:03
  When a browser opens the MCP server URL — and only a browser, identified by an Accept header asking for text-slash-html but not the JSON or server-sent-events content types real MCP clients send — return an HTML page that explains what an MCP server is and what the user is supposed to do with the URL.
• 22:20
  For every actual MCP client, the behavior is unchanged, because clients send the JSON or SSE Accept headers.
• 22:27
  For a browser, the user gets a human-readable onboarding page.
• 22:30
  His result: ticket volume on this issue dropped sharply.
• 22:33
  Customer success is happier.
• 22:35
  Customers onboard faster.
• 22:36
  No measurable downside.
• 22:38
  I like this for a few reasons.
• 22:39
  First, it's the kind of fix that good infrastructure engineers ship constantly and rarely write up.
• 22:45
  Second, it's a small, observable, content-negotiation-based answer to a real onboarding cliff in MCP, which is now adoptable enough that the first-mile UX is starting to matter.
• 22:55
  Third, Luke's gripe about the spec — and I'll quote him directly — quote, despite the fact that MCP is an utterly terrible attempt at a specification — is the kind of opinion someone has only after spending a lot of time inside the thing.
• 23:09
  He's not complaining for sport.
• 23:10
  He's complaining because he just shipped a real product against it.
• 23:14
  If you ship an MCP server, copy his pattern.
• 23:17
  The five lines of Accept-header logic will save your customer success team a lot of email.
• 23:22
  And if you happen to be on the MCP spec working group, a hello-page convention belongs in the spec itself.
• 23:28
  Tomorrow is Monday.
• 23:29
  The GitHub Copilot usage-based billing change is two weeks out, the Calif technical report drops whenever Apple ships the fix, and Anthropic's Agent SDK metering is starting to settle into the shape last Wednesday's chapter sketched.
• 23:42
  — Lenar.