Latest / Big Iron Radio / The 2026 Homelab Workstation That Also Runs LLMs

Transcript

The 2026 Homelab Workstation That Also Runs LLMs · Big Iron Radio

• 0:02
  There's an appealing idea. One machine that's your daily driver workstation and your local LLM
• 0:09
  box at the same time. One set of parts, one power bill, one thing to maintain. And it genuinely
• 0:15
  works. But the compromise between workstation and LLM rig doesn't live in the parts most people
• 0:22
  obsess over it lives in three physical things they ignore until the machine is thermal throttling
• 0:28
  at hour two. Quick note before we start. This is an AI generated voice and everything here is adapted
• 0:36
  from a written guide over at Big Iron. The full build pattern with three price tiers is linked in
• 0:42
  the show notes. The core tension is this. A workstation and an inference rig pull in different
• 0:48
  directions and the conflict shows up in the chassis, the power supply, and the airflow. You can pick
• 0:55
  perfect compute parts and still cook the machine if you get those three wrong. So we'll do the
• 1:00
  compute quickly and spend real time on the physical traps. CPU first and one specific piece of advice.
• 1:08
  Pick a chip with an integrated GPU. Not for compute for display fallback. When your discrete card is
• 1:15
  pinned at 100% generating tokens, the iGPU keeps your desktop smooth and responsive. It's a small thing
• 1:23
  that makes the also a workstation half actually livable. Memory. DDR5, 5,600 to 6,000 megatransferers,
• 1:33
  and just too dim that's the AM5 sweet spot. Two sticks rather than four, because that's how you hold
• 1:40
  those high speeds cleanly. 64 gigabytes is the floor for serious work. 96 if the budget allows. LLM workflows
• 1:49
  each system, RAM even before the model reaches the GPU. Now the number that decides everything else.
• 1:57
  GPU VRAM, because it is the hard ceiling on model size. Let me give you the latter, because this is the
• 2:04
  one you build the whole machine around. 16 gigabytes of RAM runs a 13 billion model comfortably.
• 2:11
  24 gigabytes gets you a 30B. 32 gigabytes handles a 70B quantized. And 48 or more runs,
• 2:20
  a 70B unquantized. So pick your target model class first. Buy the RAM tier that holds it,
• 2:26
  and let every other decision follow from that. Don't buy a card in hope. Buy the card your chosen model
• 2:33
  needs. Physical trap number one. The power supply. Size it at one and a half times your load,
• 2:41
  and get an ATX 3, 0 or 3, 1 unit specifically. Why the spec matters modern GPUs pull violent,
• 2:49
  spiky power transients, and a 3, 0 or 3. One supply is designed to absorb those spikes without tripping.
• 2:57
  An undersized or older supply browns out mid-generation, and you get a crash that looks
• 3:02
  like a software bug, but is really your PSU flinching. Physical trap number two. Airflow and
• 3:10
  this is the one that quietly kills sustained performance. You want a mesh front, top exhaust,
• 3:16
  and a slight positive pressure. Workstation-style cooling, not gaming case looks. Because a card
• 3:22
  pinned for an hour needs to dump heat continuously, and a pretty case with poor airflow will throttle
• 3:28
  it right when you need the speed. And it's worth understanding why LLM load is different from
• 3:34
  gaming load here. Gaming is bursty, the GPU gets little breaks between frames. LLM inference is
• 3:42
  sustained the card is flat out for the entire length of a generation, no breaks. Thermals that
• 3:48
  are perfectly fine for a game session will throttle under a long inference run. Designed for sustained,
• 3:54
  not bursty. Physical consideration three. Less dramatic but worth getting right. Storage tiers.
• 4:02
  Put the model you're actively loading and reloading on NVMe, and keep your larger model library on SADA
• 4:09
  or spinning disk. Models are big, you swap them, and you simply don't need the entire collection sitting
• 4:15
  on fast storage, just the one you're running. Let's talk money, because the guide is refreshingly
• 4:21
  concrete. Three tiers. About $1,500 builds a solid 7-13B machine. Around $2,500 steps up to comfortable
• 4:32
  30B work. And $4,000 plus is the 70B end up serious workstation. Match the tier to the model class you'll
• 4:40
  actually run day, today not the one you daydream about, but would use twice. Now the strategic question
• 4:47
  the guide is honest enough to raise. When should you not build the hybrid at all, and split into a
• 4:52
  dedicated headless box instead? Two clear triggers. Trigger one. Your LLM use climbs past roughly four
• 5:01
  hours a day. At that point, the machine is really an inference server that happens to show a desktop,
• 5:07
  and a dedicated headless box serves it better and lets your workstation just be a workstation.
• 5:13
  Trigger two. You need 48 plus gigabytes of RAM. That's dedicated hardware territory, often multiple
• 5:21
  cards, and it doesn't want to share a case with your daily driver. And a cost most part-time inference
• 5:28
  users completely forget. Idle power. A workstation sits idle the large majority of the day, and that
• 5:35
  idle hour dominates your electricity bill more than the busy hour does. A big machine idling 20 hours a day
• 5:42
  is a standing cost even on the days you don't generate a single token. Factor the idle draw,
• 5:49
  not just the peak. On the vendor question briefly. In 2026, NVIDIA is still the default,
• 5:57
  and it's about the software stack more than the silicon CUDA, the libraries, the community all
• 6:02
  just work. AMD's Rock'em is genuinely viable now if you'd rather. And Apple Silicon is the low-fuss
• 6:09
  alternative for inference, if you're not training, and you value quiet and power efficiency.
• 6:16
  So the through line. Pick your model class, buy the RAM tier that holds it, and then respect the
• 6:22
  three physical traps power supply, airflow, chassis, because that's where a workstation LLM hybrid
• 6:28
  actually succeeds or quietly cooks itself. And if you're past four hours a day or reaching for 48
• 6:35
  plus gigs, build the dedicated box instead. The full build patterns and parts tiers are in the show
• 6:41
  notes. Thanks for listening, I'll catch you on the next one.