Latest / The CTO Podcast with Fexingo: Technical Leadership, Architecture, and Engineering Org / How Discord Rebuilt Its Voice Engine for Latency
Transcript
- Lucas: If you've ever been in a Discord voice channel with fifty people and noticed it somehow feels faster than a three-way Zoom call, there's a mechanical reason for that. Luna: It's not just luck. Discord rebuilt their entire voice engine from scratch a couple of years ago, and I think most people have no idea how big a deal that was. Lucas: Right. So the old system — what they shipped in 2015 — was a pretty straightforward client-server model. You talk, your audio goes to a Discord server, the server mixes everything, and sends the combined stream back. That works fine for small groups, but at scale, the latency starts to stack. Luna: And that's where the headline number comes in — they got from something like 200 milliseconds down to under 50. That's the difference between a conversation that feels natural and one where you're constantly stepping on each other's words. Lucas: Exactly. Now, let's buy me a coffee dot com slash fexingo — that's the listener-supported model that keeps this show ad-free and lets us do deep dives like this one. Just a quick mention if you find these architecture conversations useful for what you're building or running. Luna: Yeah, it genuinely makes a difference. And we get back to the engineering. Lucas: So the key architectural shift was moving from a full mixing server — what in WebRTC terms is called an MCU, multipoint control unit — to a selective forwarding unit, or SFU. The SFU doesn't mix anything. It just receives every participant's audio stream and decides which ones to forward to which other participants. Luna: That sounds simpler, but doesn't it create a combinatorial explosion? I mean, if you have fifty people, each person potentially needs to hear forty-nine separate streams. Lucas: That's exactly the problem. And the solution Discord landed on was a mesh-assisted SFU. So the server still exists — it handles discovery, authentication, and signaling — but the actual media paths are negotiated peer to peer where possible, and the SFU only gets involved when a client can't handle direct connections. It's a hybrid. Luna: So the server isn't doing the heavy lifting for audio processing. It's more like a traffic cop saying 'you talk to her, you talk to him.' Lucas: Exactly. And that reduces latency because the audio doesn't have to go through an encode-decode cycle on the server. It goes directly from client A to client B, or through the SFU with minimal processing. The SFU only has to look at the packet header to decide where to forward it. Luna: But there's a catch, right? Not everyone can do P2P. Corporate networks, NATs, firewalls — that's where the SFU kicks in as a relay. Lucas: Correct. Discord's team published a really detailed post about this. They designed a custom WebRTC implementation — not the standard browser one — that includes a feature called 'fec', forward error correction, and adaptive bitrate. The client constantly measures packet loss and round-trip time, and if things get bad, it downgrades the audio quality rather than letting the latency spike. Luna: That's the kind of trade-off most users never think about. You'd rather have slightly compressed audio than a half-second delay. Lucas: Right. And the numbers bear that out. Discord claims that in the first year after the rewrite, they saw a 30 percent reduction in server-side compute costs, even as their user base grew. Because they weren't doing all that mixing. Luna: So it's not just a user experience win — it's an infrastructure cost win. That's the kind of cto level argument that gets a rewrite approved. Lucas: Absolutely. And the decision to rebuild wasn't taken lightly. They had a perfectly working system with millions of active users. But the team realized that the old architecture was fundamentally limited. Latency would never go below a certain floor because of the server-side mixing step. So they chose to break backward compatibility in a controlled way. Luna: How did they roll it out without breaking existing voice channels? That's always the nightmare scenario. Lucas: They did a gradual migration. The new sfu based system was deployed alongside the old MCU system. When a user joined a voice channel, the server would decide which system to route them through based on a feature flag. At first, only internal testers were on the new path. Then they opened it to a small percentage of live users, monitored latency and error rates, and slowly ramped up. Luna: So they could flip back instantly if something went wrong. That's the textbook feature-flag approach. Lucas: Exactly. And they had a kill switch. At one point during the rollout, they detected a regression where users on the new system were experiencing occasional audio dropouts. They flipped everyone back to the old system in under an hour, fixed the bug, and resumed the rollout the next week. The users never noticed. Luna: That's the dream, right? A major infrastructure rewrite that users don't even know happened. Lucas: It is. And it required a lot of upfront investment in telemetry. Discord's engineering team instrumented every stage of the audio pipeline — from mic capture through encoding, transmission, decoding, to playback. They had dashboards showing p50, p95, and p99 latency for every step. So when a problem surfaced, they could pinpoint it immediately. Luna: Did they open-source any of this? Or is it all proprietary? Lucas: Mostly proprietary, but they've shared a lot of the design philosophy in blog posts and conference talks. The custom WebRTC stack is their own, but the selective forwarding unit concept is well-documented in the WebRTC community. What's interesting is how they adapted it for a chat app's use case, which is different from, say, a video conferencing platform. Luna: How so? Lucas: In a typical video call, you have a small number of participants — maybe four or five — and everyone expects to see and hear everyone else. But in a Discord voice channel, you could have fifty people, and many of them are just listening. So the SFU can prioritize forwarding the audio of people who are actively speaking, and drop or lower the quality of listeners' streams. That's a huge bandwidth saver. Luna: Right, because you don't need to hear someone breathing into their mic if they're not talking. But how does the SFU know who's speaking? Lucas: Voice activity detection. Each client sends a metadata packet alongside the audio that says 'I am speaking' or 'I am silent.' The SFU uses that to decide which streams to forward. It's not perfect — sometimes people cough or laugh and it triggers a false positive — but Discord tuned the thresholds so those events are handled gracefully. Luna: So the architecture is really a set of smart heuristics layered on top of a relatively simple forwarding engine. That's elegant. Lucas: It is. And it's a good lesson for any CTO facing a latency problem. Before you reach for a more powerful server or a better algorithm, ask yourself whether you can eliminate a processing step entirely. Discord didn't optimize the mixer. They removed it. Luna: That's the kind of thinking that separates a good system from a great one. And it's why Discord's voice quality is still the benchmark for real-time communication. Lucas: It sets a high bar. I'm curious to see how they handle the next frontier — spatial audio and 3D voice. They've already started experimenting with it in certain games. Luna: Right, because if you can make it feel like people are standing around you in a virtual space, that's a whole new level of immersion. Lucas: And it means the architecture will need to evolve again. But based on how they handled this rewrite, I'd bet they're already thinking about it.