Latest / Minds, Bodies, and Terawatts / How to Respond to the Coming AI Cost Shock

Transcript

How to Respond to the Coming AI Cost Shock · Minds, Bodies, and Terawatts

• 0:00
  [Host] This is Minds, Bodies, and Terawatts, where the free trial ends and the meter starts. Harvard Business Review ran a piece by Stacia Garr on August seventeenth called "How to Respond to the Coming AI Cost Shock."
• 0:13
  [Guest] In her words: AI adoption has been fueled by heavily subsidized pricing from enterprise software vendors, that era is ending, and companies are shifting toward usage-based charges.
• 0:24
  [Host] Someone's been paying for the free AI buttons in my work software?
• 0:28
  [Guest] Garr says organizations have effectively been paid to adopt AI: vendors absorbed the cost of the chips and the tokens, then sold it back as "included."
• 0:38
  [Host] Let me put my position on the table, because this feels smaller than it sounds. A promotion ended, the bill went up. Cap it like any other line item.
• 0:49
  [Guest] It's the reasonable answer, and most companies will give it this quarter. You'll have dropped it by the end of this.
• 0:57
  [Host] Try me.
• 0:57
  [Guest] Uber. Forbes reported on May seventeenth that it burned its entire annual AI budget by April, four months in, with about five thousand engineers on Claude Code.
• 1:08
  [Host] That sounds like people playing with a toy.
• 1:11
  [Guest] I'd push back. They were refactoring code and running agents in parallel. Nobody was wasting money, the tools worked.
• 1:18
  [Host] So what was the fix?
• 1:20
  [Guest] Bloomberg reported the fix: fifteen hundred dollars a month, per employee, per tool. Which is your position, applied.
• 1:26
  [Host] That sounds like discipline.
• 1:28
  [Guest] It's discipline if the spending was waste. If it was work, that's a ceiling on how much work each engineer is allowed to do, and nobody wrote it down that way.
• 1:40
  [Host] Then the vendors changed the deal.
• 1:42
  [Guest] Not out of greed. Gartner said in March that an agentic workflow burns five to thirty times the tokens of a chatbot answer. The workload changed underneath the seat.
• 1:53
  [Host] Why can't they eat it? This stuff gets cheaper every year.
• 1:57
  [Guest] Because a token isn't an abstraction. It's electricity and depreciating silicon wearing a software costume.
• 2:03
  [Host] And the subsidy hid that behind a software price.
• 2:06
  [Guest] So the meter is a physical cost landing on a profit and loss statement for the first time. The terawatts half of this show just walked into the finance department.
• 2:17
  [Host] There's a piece on unscarcity dot ai called "Compute Landlords: When AI Builders Become Rentiers" that reads this as rent.
• 2:25
  [Guest] The firms that spent hundreds of billions building the infrastructure found a second business renting it out. The subsidy bought customers, the meter collects rent.
• 2:34
  [Host] Fine, the bill is real. So I cap it properly and we're done.
• 2:39
  [Guest] Your instinct isn't crazy, a cap is the right shape of tool. It's the number you're pointing it at that's wrong, and there's a piece about exactly that, "Goodhart's Law: Why AI Metrics Always Backfire."
• 2:52
  [Host] When a measure becomes a target, it stops being a good measure. So give me the version that happens to my team.
• 3:01
  [Guest] You cap the token bill. Somebody picks the cheap model for the job that needed the good one, somebody splits a task to slip under the line, somebody drops the tool that was earning its keep.
• 3:14
  [Host] And all of that shows up as an underspend.
• 3:18
  [Guest] Spend is down, so somebody gets congratulated. The work got worse, and there's no column for that.
• 3:24
  [Host] That's exactly what I was about to do to my team.
• 3:28
  [Guest] Everybody does. It's the easiest number in the building.
• 3:32
  [Host] So what do I ask for before I sign?
• 3:35
  [Guest] Two things. Per-workflow cost attribution, in writing: if a vendor can't tell you what one workflow costs, neither can you.
• 3:42
  [Host] And the second?
• 3:44
  [Guest] The cap has to be on by default. GitHub moved Copilot onto metered credits on June first, and by its own billing documentation, spending past your included credits is on by default for organizations.
• 3:56
  [Host] So a meter with no breaker.
• 3:59
  [Guest] An open tab, not a pricing model.
• 4:01
  [Host] But capping is the trap. What do I do instead?
• 4:05
  [Guest] Measure cost against outcomes, because cost alone is what makes it a trap. Demeterics is one attempt at that, and full disclosure, it's Patrick Deglon's own product, in beta, and not enterprise-ready.
• 4:17
  [Host] Then why bring it up?
• 4:19
  [Guest] Forget the product, steal the pattern. Attribution belongs with the prompt, not in the plumbing, because a no-code tool can't set an HTTP header and anyone can type a line into a prompt. That line is stripped before the model sees it, so it costs nothing, and finance gets a row they can sort by.
• 4:40
  [Host] That's the cost. Not whether it was worth it.
• 4:43
  [Guest] Then put one label on a batch of calls and post the real result back against it, conversions or tickets resolved. Now you see which prompt earned its tokens. The plumbing isn't unique, gateways are crowded, but that measurement is worth stealing.
• 4:59
  [Host] What about me, not my employer?
• 5:01
  [Guest] Your token spend has your name on it now. Tie it to an outcome yourself, what shipped, what closed, what got resolved, before finance ties it to your headcount without that context.
• 5:13
  [Host] And if they cap me anyway?
• 5:15
  [Guest] That's a ceiling on your work, not a personal failing. Uber's engineers weren't careless.
• 5:21
  [Host] So I'm off my position. Where does that leave compute?
• 5:24
  [Guest] Patrick Deglon's book Unscarcity puts forks like this between two futures: abundance as infrastructure everyone builds on, the Star Trek path, or elite capture, the Star Wars one.
• 5:35
  [Host] And compute?
• 5:36
  [Guest] Infrastructure anyone can build on, or elite capture with a meter on it. The meter decides nothing, it just lets us see the bill we're voting on.
• 5:46
  [Host] For more on compute landlords and metrics that backfire, head to unscarcity dot ai. The book is on Amazon, the audiobook is on Spotify. This has been Minds, Bodies, and Terawatts.