Latest / Big Iron Radio / Homelab Brief - eGPUs on Linux: Thunderbolt, OCuLink, and the Bandwidth Tax

Transcript

Homelab Brief - eGPUs on Linux: Thunderbolt, OCuLink, and the Bandwidth Tax · Big Iron Radio

• 0:02
  Welcome to the Homelab Brief from Big Iron.
• 0:06
  Today, running an external GPU on Linux and the bandwidth tax everyone warns you about.
• 0:13
  Quick note, this is an AI voice drawn from a Big Iron guide.
• 0:18
  Links below.
• 0:19
  The idea is great.
• 0:21
  An eGPU attaches a full desktop GPU to a small host a mini-PC or a laptop over an external cable.
• 0:29
  A tiny, low-power machine gets to run a big RAM card.
• 0:34
  The catch is the bandwidth tax.
• 0:36
  That external link is far narrower than a proper PCIe X16 slot.
• 0:41
  But here's the part that changes the whole decision.
• 0:44
  The tax only matters as much as your workload actually uses the PCIe link.
• 0:50
  That link carries data between system RAM and the GPU.
• 0:54
  A workload that loads a model once and then computes barely touches it.
• 0:59
  A workload that streams data constantly gets throttled hard.
• 1:03
  So the two connection types.
• 1:05
  Thunderbolt or USB 4 is hotplugable, widely available, enclosure-based, and effectively PCIe 3.
• 1:13
  Zero by four speeds once you count tunnel overhead, at a price premium.
• 1:18
  Convenient, but you pay for it in dollars and bandwidth.
• 1:22
  Okulink is the enthusiast's pick.
• 1:23
  A direct PCIe 4, zero by four cable roughly double the bandwidth, lower latency, and cheaper.
• 1:31
  The trade-offs are that it's not hotplugable, and it's more of a do-it-yourself job, usually with a separate power supply for the card.
• 1:39
  And here's why homelabbers get away with it.
• 1:41
  LLM inference is a load-once, compute-heavy workload.
• 1:46
  You pull the model into VRAM once, then generate so the bandwidth tax barely registers.
• 1:52
  For running local models, an eGPU is far more viable than the scary benchmarks suggest.
• 1:58
  The full breakdown's in the show notes.
• 2:00
  That's your brief.
• 2:01
  The full breakdown's in the show notes.
• 2:02
  The full breakdown's in the show notes.
• 2:03
  The full breakdown's in the show notes.
• 2:04
  The full breakdown's in the show notes.
• 2:05
  The full breakdown's in the show notes.
• 2:06
  The full breakdown's in the show notes.
• 2:07
  The full breakdown's in the show notes.
• 2:08
  The full breakdown's in the show notes.
• 2:09
  The full breakdown's in the show notes.
• 2:10
  The full breakdown's in the show notes.
• 2:11
  The full breakdown's in the show notes.
• 2:12
  The full breakdown's in the show notes.
• 2:13
  The full breakdown's in the show notes.
• 2:14
  The full breakdown's in the show notes.
• 2:15
  The full breakdown's in the show notes.
• 2:16
  The full breakdown's in the show notes.
• 2:17
  The full breakdown's in the show notes.
• 2:18
  The full breakdown's in the show notes.
• 2:19
  The full breakdown's in the show notes.
• 2:20
  The full breakdown's in the show notes.