Latest / The CTO Podcast with Fexingo: Technical Leadership, Architecture, and Engineering Org / How Discord Rebuilt Its Audio Engine for 200 Million Daily Voice Users
Transcript
- Lucas: So back in 2023, Discord was processing roughly 4 million simultaneous voice connections at peak. By early 2026, that number had jumped to over 12 million. And the old audio engine — which had served them well since 2015 — started showing real strain. Luna: Strain how? Were users complaining about dropouts, or was it more about the team not being able to ship new features fast enough? Lucas: Both, actually. The original engine was a single monolithic C++ library that handled encoding, decoding, jitter buffering, and network transport all in one process. Every time they wanted to tweak the noise suppression or add a new codec, they had to rebuild and redeploy the entire client. And on the server side, the media nodes were stateful — each one held the full state of a voice session. If a node went down, every call on that node dropped. Luna: Right, so high blast radius. That's exactly the kind of thing that works fine at 1 million concurrent users but becomes terrifying at 10 million. Lucas: Exactly. So around mid-2024, the engineering leadership made a call: rebuild the audio stack from scratch. But not a big-bang rewrite — they did it incrementally, component by component. Luna: We deliberately don't run ads on these episodes. If these kinds of deep-dive architectural stories are useful for what you're building or running, and you want to support keeping them ad-free, the link is buy me a coffee dot com slash fexingo. Lucas: Yeah, it's a small way to keep the show independent. So back to Discord — the first component they tackled was the jitter buffer. Luna: The jitter buffer — that's the part that smooths out network packet arrival times, right? It holds packets in a queue and plays them out at a steady rate. Lucas: Exactly. The old buffer was fixed-size: 40 milliseconds. That meant if network jitter spiked above 40 milliseconds, you'd hear gaps. If jitter was low, you were adding unnecessary delay. The new buffer is adaptive — it measures the inter-arrival time of packets in real time and dynamically adjusts the depth. They published a blog post showing that for 95 percent of calls, the buffer now sits at 20 milliseconds or less, and for the worst cases, it can expand to 80 milliseconds without the user noticing. Luna: So they effectively halved the baseline latency for most users. That's a huge win for something that's purely a software change. Lucas: It was. But the bigger architectural shift was moving from stateful media nodes to a stateless, serverless proxy mesh. Instead of each voice session being pinned to one server, the new design uses a fleet of lightweight proxy nodes that forward encrypted audio chunks to a distributed processing layer. The proxies are completely stateless — they don't care which call they're handling. You can add or remove them instantly. Luna: That sounds like a complete inversion of the old model. How did they handle the stateful parts — like mixing multiple speakers into one output stream? You can't really do that in a stateless way without some shared state. Lucas: Great question. They introduced a separate 'mixer' layer — a pool of nodes that are stateful but ephemeral. Each mixer handles a single call, but if it fails, a new mixer can pick up the call state from a lightweight checkpoint stored in a Redis cluster. The checkpoint only contains the last 200 milliseconds of audio per speaker, so it's tiny. Failover happens in under 500 milliseconds. Luna: So the failure domain went from an entire region to a single call. That's a massive reliability improvement. Lucas: And it lets them do something they couldn't before: regional redundancy. In the old system, if the US East data center had issues, all US East calls suffered. Now, each call is spread across proxies in three regions, and the mixer can be in a completely different region from the participants. The proxies handle routing based on latency — each user connects to the closest proxy, but the mixing can happen wherever capacity exists. Luna: That must have required a major rework of the client-side logic too, right? The client needs to know which proxies to connect to, and how to handle reconnection if a proxy goes down. Lucas: Yeah, they built a new signaling protocol for that — a thin layer on top of WebRTC that carries metadata about available proxies and their health. The client pings this signaling service every 30 seconds and gets back a list of three proxy endpoints. If the primary proxy fails, the client automatically switches to the next one in the list without user intervention. They tested this by deliberately killing proxies in production and measuring disruption time. Median disruption: 120 milliseconds. That's less than the jitter buffer, so users don't hear a gap. Luna: That's impressive. But I wonder about the audio quality side. With different codecs and bitrates in use, how do they ensure that someone on a bad connection doesn't ruin the experience for everyone else on the call? Lucas: They introduced per-user adaptive bitrate. Each client constantly reports its estimated bandwidth to the mixer, and the mixer assigns a codec and bitrate per user. If one user's connection drops to 20 kilobits per second, that user gets a low-bitrate Opus stream, but everyone else still gets 128 kilobit per second. The mixer also does something clever: it adjusts the mixing gain based on network quality. A user with high packet loss gets their volume boosted slightly so they're still audible. Luna: That's fascinating — treating audio quality as a per-user resource allocation problem rather than a one-size-fits-all setting. Lucas: Exactly. And they didn't just build it and flip a switch. They ran a canary deployment over six weeks. The first week, 1 percent of calls were routed through the new engine. They immediately found two issues: the adaptive jitter buffer had a bug where it would oscillate between two depths, causing a wobble effect. And the failover logic had a race condition — if two proxies failed simultaneously, the client would try to connect to a dead proxy and timeout. Luna: So they rolled back? Lucas: Twice. First rollback after week one to fix the jitter buffer. Second rollback after week three when they found the race condition. Both times, they fixed the issue and started over from 1 percent. By week six, they were at 100 percent of calls on the new engine. The final metric: overall user-reported audio issues dropped by 40 percent. And the engineering team could now deploy a new codec — like the upcoming Opus 2.0 — without changing the client. Luna: That's a great example of how to do a risky infrastructure migration right: incremental, reversible, data-driven. Lucas: And they documented the entire process publicly. Their engineering blog is one of the best in the industry for real-time comms architecture. It's worth a read if you're working on anything latency-sensitive. Luna: Absolutely. So what's next for their audio engine? Are they looking at spatial audio or something more exotic? Lucas: They've hinted at two things: first, integrating machine learning for predictive jitter smoothing — basically using a model trained on global network patterns to anticipate jitter spikes before they happen. Second, a move toward fully encrypted mixing — where the mixer can combine audio streams without ever decrypting them. That would be a first in the industry. Luna: Encrypted mixing sounds like a nightmare from a computational standpoint. But if anyone can pull it off, Discord's audio team has the track record. Lucas: They do. And given that they're now handling over 200 million daily voice users, the incentives are certainly there to keep pushing.