Latest / The CTO Podcast with Fexingo: Technical Leadership, Architecture, and Engineering Org / How Discord Rebuilt Its Voice Engine for 150 Million Users
Transcript
- Lucas: In 2022, Discord's voice engineering team was staring at a graph that every operations person dreads: median latency was climbing, and the tail was getting worse. For users on slower networks, delay had doubled from about 50 milliseconds to over 120, and the team knew they couldn't throw more servers at it. Luna: Because the bottleneck wasn't compute—it was the audio pipeline itself, right? Lucas: Exactly. The original voice stack was built in 2015 when Discord was a much smaller app. It used a single-threaded audio processing model: encode, transmit, decode, play. Every step was sequential. And as the user base grew to 150 million monthly actives, the network conditions they had to handle became far more diverse. Luna: So they had to rewrite the entire pipeline. What was the first thing they changed? Lucas: They started with the encoder. Discord had been using the standard Opus codec, which is great for general voice but not optimized for high packet loss. So they built a hybrid codec: for the first 20 percent of packets, they used a low-latency linear predictive coding path, and for the rest, a more robust version of Opus. This cut encoding time by about 30 percent in bad network conditions. Luna: That's a smart trade-off—taking the worst case and optimizing it. But changing a codec means you have to support both the old and new for backward compatibility. Lucas: Right. They ran both codecs in parallel for six months, letting the client choose based on network quality. The new codec had a fallback to Opus if the receiver couldn't decode it. That's how they avoided breaking every call overnight. Lucas: Next, they tackled the jitter buffer. The original buffer was static—it added a fixed 80 milliseconds of delay to smooth out network jitter. But for users with good connections, that extra delay was unnecessary. So they built an adaptive jitter buffer that could shrink to 30 milliseconds on fast networks and expand to 150 on slow ones, but only after detecting actual jitter spikes. Luna: And that alone probably cut the baseline latency by half for the majority of users. Lucas: Exactly. For the top 75 percent of connections, latency dropped from 80 to about 45 milliseconds. But they still had the server-side mixing issue. The original design used a single mixer thread per voice channel, which became a bottleneck when a channel had 30 or 40 people talking. So they made the mixer multithreaded, splitting the audio streams by region and then mixing them per region. Luna: That's a non-trivial change because mixing audio requires tight synchronization. How did they handle the consistency problem? Lucas: They used a lock-free ring buffer for each stream, and the mixer threads would consume from multiple buffers in a round-robin. The key insight was that small timing mismatches are inaudible as long as they're under a millisecond. So they relaxed the synchronization requirements and let the jitter buffer on the client side smooth out any micro-variations. Lucas: They also parallelized the entire audio processing path using SIMD instructions. The encoding step, which used to run on one core, now processes four audio frames at once on CPUs that support SSE4.1 or AVX2. Luna: And that brought encoding latency down from about 8 milliseconds to under 2 for most frames. That's huge for real-time feel. Lucas: It is. By the end of 2023, Discord had cut end to end latency back to a median of 55 milliseconds, with the 95th percentile under 100. They published a blog post detailing most of these changes, and they open-sourced the new audio processing library called 'Sonic'. That's been picked up by a couple of smaller game chat platforms. Luna: It sounds like the biggest lesson here is about measuring the right thing. They didn't just optimize for throughput—they optimized for the user experience of real-time conversation. Lucas: Absolutely. And that meant making hard choices. For example, they consciously decided not to implement full forward error correction because the bandwidth overhead would increase latency for everyone. Instead, they focused on making the codec resilient to individual packet loss and let the jitter buffer handle bursts. Lucas: There's another aspect: they rebuilt the voice engine without changing the signaling layer—the part that sets up the call. That meant the entire rewrite was invisible to developers using the Discord API. No client SDK changes required. Luna: Which is a great example of good architectural boundaries. They kept the interface stable while completely swapping out the internals. Lucas: Exactly. And that's the kind of engineering leadership that often goes unnoticed but is critical for maintaining trust with the developer ecosystem. If you're building a platform that others depend on, you can't afford a version that requires a migration. Luna: On a related note, these conversations about engineering decisions—we have them every day in our own work. And they're one reason why shows like this exist. If you find value in these deep dives, a small contribution helps keep the podcast ad-free and independent. A couple of dollars a month at buy me a coffee dot com slash fexingo genuinely makes a difference—it covers hosting and the time we put into research. Lucas: Yeah, it's listener support that lets us spend the time to trace through a rebuild like Discord's without skipping the technical details. So if you've gotten something out of these episodes, that's the place. Lucas: Getting back to the technical story: one more piece I want to highlight is their testing strategy. They built a simulation framework that could replay 100,000 real call sessions with varying network conditions. That let them validate every change before rolling out to production. Luna: So they weren't just relying on synthetic benchmarks. They used actual user traffic patterns. Lucas: Exactly. They'd take a 24-hour trace from their production system, replay it through the new pipeline, and compare the latency distribution. That's how they caught a regression where the adaptive jitter buffer would oscillate on certain 4G networks. Luna: And that level of investment in testing is what separates a rebuild that works from one that causes a major outage. Lucas: Agreed. It's easy to talk about the big architectural changes, but the discipline of verification is what gives you confidence to ship. Discord's approach of simulating real conditions at scale is something any team tackling a critical middleware rewrite can learn from. Lucas: So, to wrap up, the Discord voice engine rebuild is a case study in focusing on the user's perception of latency, not just the average. They measured in milliseconds and made every component accountable for its contribution to the total. Luna: And they did it without breaking the experience for 150 million users. That's the kind of engineering that's invisible when it works and very visible when it doesn't. Lucas: Exactly. And the next time you're on a call with a friend and it feels like they're in the same room, you'll know some of what went into that.