Latest / MBA Training AI / How Klarna turned customer service triage into a durable AI workflow

Transcript

How Klarna turned customer service triage into a durable AI workflow · MBA Training AI

• 0:00
  Welcome to the Leaders Insights podcast.
• 0:07
  Today's episode, how Klarna turned customer service triage into a durable AI workflow.
• 0:13
  Everyone keeps telling me Klarna fired 700 people because of an AI agent.
• 0:18
  That's the story.
• 0:19
  Is it true?
• 0:20
  It's the story people want to be true, which is exactly why you should distrust it.
• 0:25
  The layoffs made the headlines in early 2024.
• 0:28
  The durable part came later, and it wasn't about headcount at all.
• 0:32
  The interesting decision Klarna made was refusing to bolt AI onto their existing customer service
• 0:38
  process.
• 0:39
  What does bolt-on actually mean, though?
• 0:41
  That sounds like a slogan.
• 0:43
  Fair.
• 0:44
  Bolting on means you keep your old workflow, not one, the ticket comes in, gets categorized,
• 0:50
  routed to a queue, then a human answers, and you just drop a chatbot in front of it.
• 0:56
  Klarna's first instinct was that, and it failed.
• 0:59
  The bot became a very expensive speed bump before you reached a real person.
• 1:04
  So what did they change?
• 1:05
  They rebuilt the triage.
• 1:07
  Triage being the sorting step, deciding what a request even is before anyone touches it.
• 1:12
  Around the AI agent, instead of around the human org chart.
• 1:16
  The old categories existed because humans needed them.
• 1:19
  Refund, dispute, where's my stuff?
• 1:23
  The model didn't need those buckets, so they threw them out and let it reason from the
• 1:27
  raw message.
• 1:28
  That sounds like a fancy way of saying they remove structure.
• 1:31
  Isn't structure the thing that keeps these systems from hallucinating?
• 1:35
  Inventing answers that sound right and aren't?
• 1:38
  That's the contrarian bit, and it's where most teams get it backwards.
• 1:42
  They think guardrails mean more rules, more decision trees, more if the customer
• 1:46
  says X do Y.
• 1:48
  Klarna found the opposite.
• 1:50
  The more rigid the script, the more the model would twist a genuine request to fit a box.
• 1:55
  Give it fewer categories and one clear escalation path to a human.
• 2:00
  And the hallucination rate dropped.
• 2:02
  Structure in the wrong place is what breaks these things.
• 2:05
  Give me a number or I don't believe you.
• 2:07
  Klarna's own figures, and this is the vendor talking about itself, so hold it loosely,
• 2:12
  claim the agent handled roughly two-thirds of chats and cut average resolution time from
• 2:17
  11 minutes to under two.
• 2:20
  The number I actually trust more is the repeat contact rate.
• 2:26
  How often the same customer comes back because the first answer was garbage.
• 2:30
  That fell, and that one's hard to fake because it costs you real money when it's high.
• 2:35
  What did they get wrong first?
• 2:36
  You keep hinting.
• 2:38
  The big early mistake was measuring the agent on deflection, the share of tickets that
• 2:42
  never reach a human.
• 2:44
  Deflection is the worst metric in this entire field.
• 2:47
  You can hit 90% deflection by making it impossible to reach a person, and your customers will
• 2:52
  quietly hate you and leave.
• 2:55
  Klarna optimized for it.
• 2:57
  Watch satisfaction sag and switch to measuring resolved and didn't come back.
• 3:02
  Different incentive, completely different behavior.
• 3:05
  So the prompt engineering, the way you write instructions for the model, that's the secret
• 3:09
  sauce?
• 3:10
  No, and I'll die on this hill.
• 3:13
  Prompt engineering is the plumbing, not the architecture.
• 3:15
  Klarna's win came from redesigning what the job was, then writing prompts to serve that.
• 3:21
  Teams obsess over clever prompt tricks, the way people rearrange furniture in a house with
• 3:26
  no foundation.
• 3:27
  A beautifully worded prompt pointed at a broken process gives you a very polite failure.
• 3:33
  That's a little cocky.
• 3:34
  What's the evidence a normal team can copy any of this?
• 3:37
  Because the transferable part is boring and free.
• 3:40
  One, kill your legacy categories and let the model read the raw request.
• 3:45
  Two, pick a metric that punishes bad answers, not one that rewards dodging humans.
• 3:51
  Three, make the escalation path to a person one click, not a maze.
• 3:57
  None of that requires Klarna's budget.
• 3:59
  The expensive part was learning it.
• 4:01
  You get it for the price of this podcast.
• 4:03
  And the funny thing nobody admits?
• 4:06
  Half these AI agents work brilliantly in the demo and fall apart in production for the
• 4:10
  same reason as Souffle does, the moment you open the door and let reality in.
• 4:15
  So the one thing I do Monday morning?
• 4:17
  Find the metric your support team is judged on.
• 4:20
  If it rewards avoiding customers rather than resolving them, fix that before you write
• 4:25
  a single prompt.
• 4:27
  Everything else is downstream of that number.
• 4:29
  That's your five.
• 4:30
  Full AI track, structured, no fluff, is at MBA-training.com.