Latest / MBA Training AI / Bloomberg's bet on fine-tuning: what it teaches every enterprise about the RAG-vs-fine-tune decision

Transcript

Bloomberg's bet on fine-tuning: what it teaches every enterprise about the RAG-vs-fine-tune decision · MBA Training AI

• 0:00
  Welcome to the Leaders Insights Podcast.
• 0:08
  Today's episode, Bloomberg's bet on fine tuning, what it teaches every enterprise about
• 0:12
  the rag versus fine tune decision.
• 0:15
  Everyone's been screaming, just use rag for three years, and then Bloomberg goes and spends
• 0:20
  a fortune training a model from scratch.
• 0:22
  So who's wrong?
• 0:24
  Nobody's wrong yet.
• 0:25
  But a lot of people copied Bloomberg for the wrong reasons.
• 0:29
  Let's be precise, rag means retrieval augmented generation, where you bolt a search system
• 0:34
  onto a generic model, so it pulls in your documents at question time.
• 0:40
  Bloomberg didn't do that.
• 0:41
  They built Bloomberg GPT, a 50 billion parameter model, trained on roughly 700 billion tokens,
• 0:48
  a token being a chunk of text, about three quarters of a word, and more than half of
• 0:53
  that was their own financial data.
• 0:55
  That sounds like a flex, not a strategy.
• 1:00
  It was a flex that made sense in early 2023, when they announced it.
• 1:05
  Back then, the open models were weak, and Bloomberg had something almost nobody else
• 1:09
  has.
• 1:10
  40 years of proprietary financial language, sitting in-house, already clean and labeled.
• 1:16
  That's the real asset, not the compute.
• 1:18
  Juan does.
• 1:19
  The data mode.
• 1:20
  But here's the uncomfortable part.
• 1:22
  It's 2026.
• 1:24
  GPT class models have moved twice since then.
• 1:27
  Is Bloomberg GPT already a museum piece?
• 1:30
  Partly, yes.
• 1:31
  That's exactly the lesson.
• 1:33
  The moment you fine-tune or train from scratch, you freeze a snapshot.
• 1:37
  The base model keeps improving without you, and you're stuck maintaining your fork.
• 1:42
  Bloomberg could afford that treadmill.
• 1:44
  Most enterprises absolutely cannot.
• 1:47
  So when someone in a boardroom says, we should build our own model like Bloomberg,
• 1:52
  what do you tell them?
• 1:53
  I ask them one question.
• 1:55
  Does your knowledge change daily, or does your language change slowly?
• 1:59
  If your problem is the model doesn't know today's contract terms, that's a knowledge
• 2:04
  problem.
• 2:05
  You want retrieval, RAG, because you can update the documents without touching the
• 2:08
  model.
• 2:09
  If your problem is the model literally doesn't speak our dialect, think derivatives desks,
• 2:15
  radiology, legal citations.
• 2:17
  That's a behavior problem, and that's where fine-tuning earns its keep.
• 2:21
  Give me the failure case.
• 2:23
  Where does copying Bloomberg blow up?
• 2:25
  A mid-sized bank I talked to spent about eight months in a seven-figure budget fine-tuning
• 2:30
  a model on their internal reports.
• 2:32
  By the time they finished, an off-the-shelf model with a decent retrieval layer beat it.
• 2:38
  Cheaper.
• 2:39
  And it actually stayed current.
• 2:40
  They'd solved a knowledge problem with a behavior tool.
• 2:44
  Classic.
• 2:45
  They confused the model sounds off with, the model is missing facts.
• 2:49
  How do you even tell those two apart before you burn the eight months?
• 2:52
  Keep test, take 50 hard questions, paste the relevant documents directly into the prompt,
• 2:59
  just stuff the context in manually.
• 3:01
  If the model suddenly gets them right, you have a knowledge gap, and RAG fixes that for
• 3:05
  a fraction of the cost.
• 3:07
  If it still fumbles even with the answer sitting right in front of it, now you've
• 3:11
  got a genuine behavior gap, and fine-tuning is on the table.
• 3:15
  That's suspiciously simple.
• 3:16
  What's the catch?
• 3:18
  The catch is people don't want the cheap answer.
• 3:21
  Plumbing sounds like ownership, like a defensible asset.
• 3:24
  RAG sounds like plumbing.
• 3:26
  But plumbing is what most of them need.
• 3:29
  And there's a hybrid nobody talks about enough.
• 3:31
  Retrieval for the facts, a light fine-tune, only to fix tone and format.
• 3:36
  Bloomberg's own follow-up work leaned harder on retrieval too, because even they couldn't
• 3:41
  retrain fast enough to keep facts fresh.
• 3:44
  So even Bloomberg quietly admitted that from scratch, BET had limits.
• 3:47
  They didn't renounce it.
• 3:49
  The model still does its job on financial language tasks, like sentiment, and named
• 3:53
  entity recognition, which is pulling out companies and tickers from messy text.
• 3:58
  But yes, for anything time-sensitive, they wrap retrieval around it.
• 4:03
  The from scratch build was a foundation, not the whole house.
• 4:10
  Run the paste-the-document test before you approve a single dollar of fine-tuning.
• 4:14
  If stuffing the answer into the prompt fixes your problem, you have a retrieval problem
• 4:19
  and anyone pitching you a custom model is either lazy or selling compute.
• 4:24
  Fine-tune only when the model fails with the answer in front of its face.
• 4:28
  That one experiment saves most companies a year and a couple million dollars.
• 4:32
  Cheaper than a data science team learning it the hard way.
• 4:36
  That's the show.
• 4:37
  That's a wrap.
• 4:38
  Fresh AI briefings drop daily at MBA-training.com.