Latest / Big Iron Radio / Building a Fully Local Voice Assistant: Whisper, Piper, and Wyoming

Transcript

Building a Fully Local Voice Assistant: Whisper, Piper, and Wyoming · Big Iron Radio

• 0:02
  There's a specific dream a lot of us have.
• 0:06
  A voice assistant that answers when you talk to it,
• 0:08
  controls your house, and never once ships your voice off
• 0:11
  to a company's servers.
• 0:13
  In 2026, that's genuinely buildable,
• 0:17
  entirely on your own hardware.
• 0:19
  But there's one decision inside it
• 0:21
  that quietly determines whether the whole thing
• 0:23
  feels like magic or like a broken toy,
• 0:26
  and most first-time builders spend their energy
• 0:29
  everywhere except there.
• 0:30
  Quick note before we start.
• 0:33
  This is an AI-generated voice,
• 0:36
  and everything here is adapted from a written guide
• 0:38
  over at Big Iron.
• 0:40
  The full walkthrough with every install step
• 0:43
  is linked in the show notes.
• 0:45
  First, the honest question the guide opens with.
• 0:48
  Do you actually need this?
• 0:50
  This is a real project, not an afternoon.
• 0:54
  If your goal is just a timer and a weather report,
• 0:57
  a commercial speaker is less work.
• 0:59
  Build this because you want the privacy and the control,
• 1:02
  because you want your voice to stay in the house.
• 1:05
  That's the reason that carries you through the fiddly parts.
• 1:09
  Okay, the architecture in 30 seconds.
• 1:12
  There are four independent pieces.
• 1:15
  Awake word listener, waiting for its name.
• 1:18
  Speech to text, turning your words into text.
• 1:21
  The intent engine that's home assistant itself,
• 1:23
  deciding what you meant.
• 1:25
  And text to speech, speaking the answer back.
• 1:28
  Four stages, four jobs.
• 1:31
  The glue is a thing called Wyoming, and here's the key insight.
• 1:35
  Wyoming is a protocol, not an assistant.
• 1:38
  It lets the wake word, the speech to text, and the text to speech each run as independent services,
• 1:44
  on whatever hardware makes sense for that particular stage.
• 1:48
  Your wake word can run on a tiny satellite, your transcription on a GPU box across the house.
• 1:54
  That modularity is the whole design.
• 1:57
  Concretely, the four things you're installing are open wake word for the wake word,
• 2:02
  faster whisper for speech to text, home assistant's built-in intent engine,
• 2:07
  and piper for the voice that answers.
• 2:09
  Keep those four roles straight, and the config stops being mysterious.
• 2:13
  Now the decision that makes or breaks the project,
• 2:16
  and it's the speech to text sizing.
• 2:18
  Get this right, and the assistant feels instant.
• 2:22
  Get it wrong, and it feels broken, no matter how nice everything else is.
• 2:26
  So spend your attention here.
• 2:28
  Here are the real numbers.
• 2:32
  The small whisper model on an N100 class CPU is the realistic floor for a responsive experience.
• 2:39
  A Raspberry Pi 4 running whisper on CPU is not it'll feel sluggish, and you'll stop using it.
• 2:46
  The ceiling you're budgeting against is 1 to 2 seconds of transcription time.
• 2:51
  Past that, it stops feeling like a conversation.
• 2:54
  And this is where a GPU pays off,
• 2:57
  even a modest one you're already sharing with a Mitch or Frigate.
• 3:01
  Put whisper on that card, and you jump up to medium or large V3 turbo accuracy,
• 3:06
  at sub-second latency.
• 3:08
  If you've got a spare GPU anywhere on the network,
• 3:11
  that's where your voice assistant wants to live.
• 3:14
  Compare that to the other two stages, which are comparatively cheap.
• 3:18
  Piper, the text-to-speech, is the easy one it sounds good essentially out of the box,
• 3:24
  and you should not be spending tuning time there.
• 3:27
  And Open Wake Word is the stage nobody thinks about until it misfires either ignoring you,
• 3:33
  or triggering on a word from the TV.
• 3:35
  It's also cheap to run.
• 3:37
  The mistake is tuning the Wake Word and the voice before you've fixed transcription.
• 3:42
  The rule, fix speech-to-text sizing first.
• 3:47
  Piper and Open Wake Word are cheap.
• 3:49
  Don't burn a weekend polishing them, while your real bottleneck is the transcription.
• 3:55
  Now a hardware truth that surprises people.
• 3:57
  The satellite, the little mic and speaker, device you actually talk to matters more than you'd expect.
• 4:04
  Specifically the microphone array and the echo cancellation.
• 4:08
  That's often the difference between magic and constant frustration.
• 4:11
  Here's the concrete gotcha.
• 4:14
  A bare ESP32, dev board with no acoustic echo cancellation, will fight its own text-to-speech playback.
• 4:22
  It hears itself speaking the answer, thinks that's you, and gets confused.
• 4:27
  Echo cancellation is what lets the device talk and listen at the same time.
• 4:31
  So satellite hardware with real AEC is worth the money there's a spectrum of options from raw DIY boards
• 4:39
  up to purpose-built satellites, and you pick based on how much fiddling you'll tolerate.
• 4:45
  Let's talk latency as a whole, because that's what feels responsive, actually means.
• 4:50
  The target is roughly one and a half seconds, end to end.
• 4:54
  And crucially, that budget is additive it's the sum of Wake Word detection,
• 4:58
  plus network hops, plus speech-to-text, plus intent parsing, plus text-to-speech.
• 5:03
  Every stage adds its slice.
• 5:06
  So when the thing feels slow, don't guess-measure each stage separately.
• 5:11
  Usually the culprit is speech-to-text, which is why we keep coming back to it,
• 5:15
  but confirm before you optimize.
• 5:17
  Optimizing the wrong stage is how people waste a weekend.
• 5:22
  One reassurance.
• 5:24
  This is all just a set of services, so rollback is easy.
• 5:28
  If it's not working out, you turn it off and go back to whatever you had.
• 5:32
  It's a low-risk thing to try you're not rebuilding your house, you're adding some containers.
• 5:38
  And set your expectations honestly.
• 5:40
  If your only goal is to beat a commercial assistant's raw transcription accuracy,
• 5:45
  you might be a little disappointed at the small end.
• 5:48
  But that was never the real prize.
• 5:51
  The prize is a voice assistant that's private, that's yours,
• 5:54
  and that plugs directly into the home assistant automations a commercial speaker will never touch.
• 6:01
  So the one thing to carry out of this.
• 6:04
  Get your speech-to-text sizing right first small on a decent CPU as the floor.
• 6:08
  A GPU if you have one and everything else falls into place around it.
• 6:12
  The full walkthrough, satellite options, and latency budget are all in the show notes.
• 6:18
  Thanks for listening, I'll catch you next time.