Latest / Big Iron Radio / Homelab Brief - One GPU for Immich, Frigate, and Whisper

Transcript

Homelab Brief - One GPU for Immich, Frigate, and Whisper · Big Iron Radio

• 0:02
  Welcome to the Homelab Brief from Big Iron.
• 0:06
  Today, fitting three hungry AI workloads Imitch, Frigate, and Whisper onto one GPU.
• 0:13
  Quick note, this is an AI voice, drawn from a Big Iron guide.
• 0:17
  Links below.
• 0:19
  First, know each tenant's appetite, because they're wildly different.
• 0:23
  Frigate is small under a gigabyte, but permanent.
• 0:26
  It's always on.
• 0:27
  Imitch's machine learning container idles, then bursts to a few gigabytes when it indexes.
• 0:34
  And Whisper is the elastic glutton.
• 0:37
  Four to six gigabytes at large V3 with int8, and climbing if you add diarization or a higher quality quant.
• 0:44
  Here's the trick that solves most of the problem for free.
• 0:48
  Daypart scheduling.
• 0:50
  Run Whisper as an overnight batch job, when Frigate and Imitch are quiet.
• 0:54
  You resolve the majority of contention, before you touch any GPU-sharing technology at all.
• 1:01
  Time slicing by the clock beats, time slicing by the driver.
• 1:05
  When you genuinely need concurrency, say live transcription, running alongside the other two reach, for CUDA MPS.
• 1:12
  It gives real per-client VRAM caps, and works on consumer cards.
• 1:17
  But it's A when you need it tool, not a default.
• 1:21
  A blunt but valid option.
• 1:23
  Just tolerate the occasional out-of-memory kill, as long as your Whisper pipeline is idempotent, so it retries cleanly.
• 1:30
  And Frigate isn't the thing getting killed.
• 1:33
  Letting the batch job die and rerun is fine.
• 1:36
  Letting your cameras go blind is not.
• 1:39
  And the cleanest fix of all.
• 1:41
  Peel a workload off the GPU entirely.
• 1:44
  A Coral Accelerator, or an Intel iGPU through OpenVINO, can take Frigate's detection off your main card.
• 1:51
  Leaving all that VRAM for the LLM and Whisper work.
• 1:55
  The full budgeting maths in the show notes.
• 1:58
  That's your brief.
• 1:59
  A Coral Accelerator, or an Intel iGPU, or an Intel iGPU, or an Intel iGPU, or an Intel iGPU, or an Intel iGPU, or an Intel iGPU, or an Intel iGPU, or an Intel iGPU, or an Intel iGPU, or an Intel iGPU, or an Intel iGPU, or an Intel iGPU, or an Intel iGPU, or an Intel iGPU, or an Intel iGPU, or an Intel iGPU, or an Intel iGPU, or an Intel iGPU, or an Intel iGPU, or an Intel iGPU, or an Intel iGPU, or an Intel iGPU, or an Intel iGPU, or an Intel iGPU, or an Intel iGPU, or an Intel iGPU, or an Intel iGPU, or an Intel iGPU, or an Intel iGPU, or an Intel iGPU, or an Intel i