Latest / MBA Training Data / AI agents querying your warehouse: measure before rebuilding
Transcript
- 0:00Host: This is Leaders Insights.
- 0:07Host: On the table.
- 0:08Host: If agents are the new primary consumer of your data, is your infrastructure built for
- 0:12Host: the wrong audience?
- 0:13Expert: The mistake I keep seeing this month is CDOs rebuilding their entire stack because a keynote
- 0:18Expert: told them agents are the new users.
- 0:21Host: Panic-driven architecture.
- 0:22Host: And the keynote was loud.
- 0:25Expert: At DBT Summit last week, DBT Labs and 5tran rolled out a cluster of products all pointed
- 0:31Expert: at the same idea.
- 0:32Expert: That the thing querying your data warehouse is no longer a human analyst but an AI agent
- 0:38Expert: asking questions on its own.
- 0:40Host: Translate that.
- 0:41Host: What actually changes when the consumer is a bot instead of a person?
- 0:44Expert: A human analyst runs maybe 40 queries a day and gets bored.
- 0:49Expert: An agent will fire off thousands, chain them together, and never sanity check a nonsense
- 0:54Expert: result, so the failure mode flips.
- 0:56Expert: A person notices when revenue comes back as negative.
- 1:00Expert: An agent just passes it downstream to the next agent.
- 1:03Host: So this is real, not just a slide.
- 1:05Expert: The shift is real.
- 1:06Expert: The urgency is being sold to you.
- 1:08Expert: Those are different things.
- 1:10Host: Give me a number that tells me which is which.
- 1:12Expert: DBT Labs put out a figure that something like 30% of the queries hitting their customers'
- 1:17Expert: warehouses are now machine-generated rather than human.
- 1:21Expert: Worth flagging, they sell the tooling that makes that number go up.
- 1:25Expert: So treat it as directional, not gospel.
- 1:28Expert: Cross-check against your own query logs before you believe your stack is under siege.
- 1:33Host: And if I check my logs and it's 3% not 30?
- 1:36Expert: Then you don't touch your architecture this quarter.
- 1:38Expert: You watch the trend line.
- 1:40Expert: The independent read matters here.
- 1:42Expert: O'Reilly's radar surveys through 2026 show agent adoption climbing fast in engineering
- 1:48Expert: but still thin in the data layer specifically.
- 1:51Expert: Plenty of companies are experimenting.
- 1:53Expert: Far fewer are in production.
- 1:55Host: Here's the uncomfortable one.
- 1:56Host: Vendors always say the audience changed so you need to buy the new thing.
- 2:00Host: Why is this any different from the last five waves?
- 2:03Expert: It mostly isn't, and that's the honest answer.
- 2:06Expert: But there's one piece that isn't marketing.
- 2:08Expert: The metadata problem.
- 2:10Expert: Metadata is just the labels on your data.
- 2:13Expert: What this column means.
- 2:14Expert: Where it came from.
- 2:15Expert: Whether it's trustworthy.
- 2:17Expert: A human analyst fills those gaps from memory.
- 2:20Expert: They know the revenue table excludes refunds because they've been burned before.
- 2:25Expert: An agent has no memory and no scar tissue.
- 2:29Expert: So if your definitions live in someone's head instead of written down, the agent guesses
- 2:33Expert: and it guesses confidently.
- 2:35Host: So the real work isn't buying an agent-ready warehouse.
- 2:38Host: It's documentation.
- 2:39Expert: It's the semantic layer.
- 2:41Expert: The written down business definitions that say active customer means logged in within
- 2:45Expert: 30 days, not ever.
- 2:48Expert: MIT Sloan had a piece this year making exactly this point.
- 2:52Expert: The organizations getting value from agents aren't the ones with the fanciest models.
- 2:56Expert: They're the ones who did the boring work of defining their terms first.
- 3:01Expert: That's not a product you purchase.
- 3:03Expert: It's a project.
- 3:04Host: Second number.
- 3:05Host: What's the cost angle because thousands of automated queries can't be free.
- 3:09Expert: That's the one nobody put on a slide.
- 3:11Expert: If agents run 10 to 100 times the query volume of humans, your cloud compute bill
- 3:15Expert: scales with it.
- 3:17Expert: These are walking into pilots with no cap on how much an agent can spend and getting a
- 3:22Expert: five-figure surprise.
- 3:24Expert: So before you let an agent loose on production data, you set a query budget the same way
- 3:28Expert: you'd set a spending limit on a corporate card.
- 3:31Host: You've now told people not to rebuild, not to trust the vendor number, and to write
- 3:36Host: documentation.
- 3:37Host: That's a lot of not doing.
- 3:39Expert: Because the expensive mistake is motion.
- 3:42Expert: Reacting to a keynote by re-platforming cost you a year and a budget.
- 3:46Expert: The cheap, useful move is small.
- 3:49Host: Then land it.
- 3:50Host: One thing my listener does Monday morning.
- 3:52Expert: Pull last month's query logs and calculate what share came from a service account instead
- 3:57Expert: of a named human.
- 3:58Expert: That's your real agent load, not dbt's 30% yours.
- 4:02Expert: If it's small, you've got time to write your definitions properly.
- 4:06Expert: If it's already large and undocumented, you've found your fire.
- 4:09Expert: And it isn't the one the vendors were pointing at.
- 4:12Expert: Measure your own audience before you rebuild for someone else's.
- 4:16Host: That'll do it.
- 4:17Host: What we read for this one?
- 4:18Expert: The new stack, dbt labs, vendor, data tooling, O'Reilly radar, MIT Sloan Management Review.
- 4:25Host: End of episode.
- 4:26Host: The CDO calculators are running at MBA-training.com.