Latest / MBA Training AI / Evaluating AI apps before you trust them: a practical playbook

Transcript

Evaluating AI apps before you trust them: a practical playbook · MBA Training AI

• 0:00
  Welcome to the Leaders Insights podcast.
• 0:07
  Today's episode, evaluating AI apps before you trust them.
• 0:11
  A practical playbook.
• 0:13
  Every vendor demo I've ever seen works perfectly.
• 0:16
  Then the thing hits production and falls apart.
• 0:19
  What's the trick they're pulling?
• 0:20
  The trick is, they built the demo on the 10 questions they know it answers well.
• 0:25
  It's like test driving a car on a track the dealer paved themselves.
• 0:29
  The moment you hand it your real data, your real edge cases, the polish comes off.
• 0:34
  So how does a smart exec who doesn't code test this before signing a contract?
• 0:38
  You start with a golden set.
• 0:40
  That's just 50 to 100 real questions from your business, with answers you already know
• 0:44
  are correct.
• 0:46
  Not vendor questions, oh, yours.
• 0:48
  You run the tool against them and count how often it's right.
• 0:51
  50 questions sounds trivial.
• 0:54
  Why not thousands?
• 0:55
  Because 50 real ones beat a thousand you'll never read.
• 0:58
  And here's the discipline.
• 1:00
  Pick the questions that hurt.
• 1:02
  The ambiguous ones, the ones where two policies contradict each other.
• 1:06
  The ones a new hire would get wrong.
• 1:08
  If it only nails the easy stuff, you've bought an expensive parrot.
• 1:12
  Most of these enterprise tools are rag-based, right?
• 1:16
  Explain that before we go further.
• 1:17
  Rag.
• 1:18
  Retrieval augmented generation.
• 1:20
  Where the AI looks up your documents first and then writes an answer from what it found.
• 1:26
  The idea is, it quotes your handbook instead of making things up.
• 1:31
  In theory, it's grounded.
• 1:33
  In practice, it depends entirely on whether the retrieval step actually fetched the right
• 1:38
  document.
• 1:39
  And if it fetches the wrong one?
• 1:40
  Then it writes a confident, beautifully worded answer based on garbage.
• 1:44
  That's the failure mode nobody demos.
• 1:47
  So the second test is, for each answer, does it show you the source?
• 1:51
  A good tool cites the exact passage.
• 1:54
  If it can't tell you where the answer came from, you're trusting vibes.
• 1:58
  Okay, retrieval and sourcing.
• 2:00
  What's the third thing people skip?
• 2:02
  Hallucination under pressure.
• 2:04
  You deliberately ask it things your documents don't cover.
• 2:07
  The right answer is, I don't know.
• 2:09
  The wrong answer is a fluent paragraph of invention.
• 2:12
  I've watched a legal AI tool from a well-known vendor cite a statute that doesn't exist
• 2:18
  with a fake case number.
• 2:20
  Their own benchmark claimed 95% accuracy, which tells you their test set was softer
• 2:25
  than a hotel pillow.
• 2:26
  That's a vendor number, though.
• 2:28
  How much should I trust any accuracy figure they hand me?
• 2:31
  Treat every vendor accuracy claim as marketing until you reproduce it.
• 2:35
  Ask one question.
• 2:37
  What was in your test set?
• 2:39
  If they won't share it, the number is theater.
• 2:41
  The 2026 releases love throwing around enterprise grade.
• 2:45
  That phrase means nothing you can measure.
• 2:47
  Say the tool passes accuracy and sourcing.
• 2:50
  Am I done?
• 2:51
  No.
• 2:52
  Now you test the boring stuff that kills you later.
• 2:55
  Latency.
• 2:56
  How long an answer takes.
• 2:58
  Cost per query at your real volume, not the pilots.
• 3:01
  And consistency.
• 3:03
  Ask the same question five times.
• 3:05
  See if you get five different answers.
• 3:07
  Generative AI is non-deterministic, meaning it can reword or even change its answer run
• 3:12
  to run.
• 3:14
  For a marketing blurb, fine.
• 3:16
  For a compliance policy, that's a lawsuit waiting.
• 3:19
  How do you test costs realistically before you've deployed it?
• 3:22
  You take your golden set, multiply by your expected daily volume, and make the vendor
• 3:27
  quote you against that.
• 3:29
  A tool that's cheap at a thousand queries a day can be brutal at a hundred thousand.
• 3:35
  Companies discover this when the first month's bill lands and someone's face goes white.
• 3:39
  What about the tool getting worse over time?
• 3:42
  Does that happen?
• 3:43
  Constantly, the vendor updates the underlying model and behavior shifts overnight.
• 3:48
  So keep your golden set and rerun it monthly after you deploy.
• 3:52
  It's your smoke detector.
• 3:54
  The teams that skip this find out from angry customers instead.
• 3:58
  Always the more expensive way to learn.
• 4:00
  If someone does one thing after this episode, what is it?
• 4:03
  Build the golden set this week.
• 4:04
  50 real questions, known answers, the nasty ones included.
• 4:09
  Before you watch a single demo.
• 4:11
  Because if you walk in with your own test, the vendor can't steer you to their paved
• 4:14
  track.
• 4:15
  You control the ground.
• 4:17
  And suddenly, the honest tools look very different from the loud ones.
• 4:21
  That's it.
• 4:22
  The AI Decision Tools are live at MBA-training.com.