Latest / MBA Training Data / When the AI model is wrong: what CDOs must own in 2026
Transcript
- 0:00Welcome to the Leaders Insights podcast.
- 0:07Today's episode, When the AI Model is Wrong, What CDOs Must Own in 2026.
- 0:13Let me start with something blunt.
- 0:15When an AI model gives a customer the wrong answer, whose fault is that?
- 0:19Nine times out of ten, not the models.
- 0:22And that's the lie the whole industry keeps telling itself in 2026.
- 0:26That sounds like you're letting the data scientists off the hook.
- 0:29The opposite.
- 0:30I'm saying they were never the problem.
- 0:33Look at the Zillow collapse.
- 0:36The house-flipping arm that lost roughly half a billion dollars.
- 0:40Everyone blamed the algorithm.
- 0:42The algorithm did exactly what it was told.
- 0:45Nobody owned the question of what happens when it drifts.
- 0:48When the market shifts and the predictions quietly rot.
- 0:52Drift meaning the model's accuracy decays as reality changes underneath it.
- 0:56Right.
- 0:57Shift is a governance question, not a math question.
- 1:00The math was fine on the day it shipped.
- 1:03What was missing was somebody whose job was to notice it stopped being fine.
- 1:07So define the line for me.
- 1:09Where does a model failure end and a governance failure begin?
- 1:12A model failure is, the thing was built badly.
- 1:16Bad training data, wrong objective.
- 1:18That's maybe 15% of what blows up in production.
- 1:21The other 85% is, the model works.
- 1:25And no human decided who reviews it.
- 1:27When against what threshold, and with what authority to shut it off.
- 1:31That's all governance.
- 1:32That's all mine.
- 1:33You say mine, but plenty of chief data officers would say the model owner should carry that
- 1:38risk.
- 1:39And that's exactly the dodge that gets people fired.
- 1:41If you're the CDO, the chief data officer, the person accountable for how data and models
- 1:46get used, you can't outsource the failure mode to the person who wrote the code.
- 1:51The coder optimizes accuracy.
- 1:54You own consequence.
- 1:55Different jobs.
- 1:56Give me a number that proves this isn't just philosophy.
- 1:59McKinsey's 2025 State of AI survey.
- 2:02And yes, they sell consulting on exactly this.
- 2:06So read it with one eyebrow up.
- 2:08Found only about a quarter of companies had any formal process to monitor models after
- 2:13deployment.
- 2:14So, three out of four organizations are flying a plane with no instruments and calling turbulence
- 2:20a model problem.
- 2:21What does good actually look like then?
- 2:24Because governance is the word people use when they want a meeting instead of a decision.
- 2:28Fair hit.
- 2:30Concretely, every model in production gets an owner with a name, not a department.
- 2:35It gets a kill switch and a person authorized to pull it without a committee.
- 2:39It gets a defined threshold.
- 2:41Below this confidence level, a human sees it before the customer does.
- 2:45And it gets a review date on a calendar, like a smoke alarm battery.
- 2:49The smoke alarm nobody tests until the house is already warm.
- 2:53Exactly.
- 2:54And here's the dry truth.
- 2:56Most AI governance documents are like gym memberships.
- 3:00Everyone signs up in January, feels virtuous, and never goes back.
- 3:04Let me push on the kill switch.
- 3:05In practice, does anyone ever pull it?
- 3:08Or is it decoration?
- 3:09Rarely, and that's the tell.
- 3:11Air Canada's chatbot in 2024 invented a refund policy that didn't exist.
- 3:17And a tribunal made them honor it.
- 3:19There was no threshold, no human review on high-stakes answers, and no owner watching.
- 3:25The model wasn't broken.
- 3:26The governance was absent.
- 3:28A working kill switch is one you've actually rehearsed pulling.
- 3:32So you'd run fire drills for AI?
- 3:34Quarterly.
- 3:35Take a model, simulate a bad output, and see how long until a human catches it.
- 3:39And who has the nerve to stop it?
- 3:41If the answer is, we'd find out from a customer on social media.
- 3:46You don't have governance.
- 3:47You have hope.
- 3:48We're near time.
- 3:50One thing a listener can do tomorrow morning.
- 3:52Pick your single highest-stakes model in production.
- 3:55Ask one question.
- 3:56Who is the named human that can shut this off today?
- 3:59And when did they last look at it?
- 4:01If you can't answer both halves in 30 seconds, you don't have a model risk.
- 4:05You have an ownership hole.
- 4:07Close that before you touch the algorithm.
- 4:09Name and a date.
- 4:10That's the whole job.
- 4:12That's the whole job.
- 4:13The math was never the hard part.
- 4:15Done.
- 4:16Want to know where you actually stand?
- 4:18Take the CDO self-assessment at MBA-training.com.