Latest / The CTO Podcast with Fexingo: Technical Leadership, Architecture, and Engineering Org / How Discord Rebuilt Its Voice Engine for Sub-50ms Latency
Transcript
- Lucas: So Discord ships roughly 2.5 billion voice minutes a day. That's not a typo — billion with a B. And the really fascinating part is that most of that traffic happens with less than 50 milliseconds of end to end latency. Today I want to look at how they actually built that. Luna: Fifty milliseconds is basically imperceptible to the human ear. How did they pull that off when even FaceTime and WhatsApp hover around 100 to 150 milliseconds in good conditions? Lucas: The short answer is they rebuilt almost everything from scratch. The long answer starts with a specific decision they made back in 2021. Discord had been using the Opus codec — which is excellent, it's the gold standard for VoIP — but they realized that for their use case, they could trade some compression efficiency for lower latency. Luna: What's the trade-off exactly? Lucas: Opus operates on 20-millisecond frames by default. That's great for quality, but it adds 40 milliseconds of algorithmic delay just from buffering and encoding. Discord's engineers wrote a custom codec called Siren that uses 5-millisecond frames. That alone shaved off 30 milliseconds from the encode-decode pipeline. They also rewrote the entire audio processing stack in Rust — no more C++ — to get tighter control over memory allocation and thread scheduling. Luna: Thirty milliseconds is huge. But I'm guessing the codec wasn't the only bottleneck? Lucas: Not even close. The network transport was a bigger one. Discord originally built on top of WebRTC, which is the standard for browser-based real-time communication. But WebRTC's implementation is opinionated — it bundles jitter buffers, echo cancellation, and bandwidth estimation into a black box. Discord wanted to control each piece independently. Luna: So they ditched WebRTC entirely? Lucas: They replaced the transport layer with their own protocol called Farseer. That's the name used internally. Farseer is a lightweight udp based protocol that does three things differently. First, it uses forward error correction — so if a packet is lost, the receiver can reconstruct it from redundant data without waiting for a retransmit. Second, it has a dynamic jitter buffer that adjusts in sub-millisecond increments based on network conditions. Luna: Third? Lucas: Third, it prioritises the most recent audio frames over older ones. In a gaming scenario, if the network jams for 100 milliseconds, WebRTC might try to recover all the lost packets — which creates a backlog. Farseer says 'skip the old stuff, give me the latest frame ASAP'. That's what keeps latency under 50 milliseconds even on flaky connections. Luna: That makes me wonder — did they have to do anything on the infrastructure side too? Because codec and protocol changes only get you so far if the physical distance between users is too large. Lucas: Absolutely. Discord deployed voice relay servers in over 300 locations worldwide, all running their custom software stack. But the clever part is how they route calls. Most VoIP services use a fixed relay — you connect to one server and that's it. Discord dynamically selects a relay based on real-time measurements of latency to each user, and it can switch relays mid-call without the user noticing. Luna: A handoff without glitching — that's the hard part. How do they do that? Lucas: They use a technique called session migration. The client maintains two simultaneous connections for about a second during the switch. The old connection is still active while the new one is established. The Farseer protocol includes a sequence number in every packet, so the receiver can seamlessly merge frames from both streams and discard duplicates. The whole process takes about 20 milliseconds of overlap. Luna: Right, so the user never hears a gap. That's elegant. But I'm curious about the measurement side — how do they know which relay to pick in the first place? Lucas: Every client periodically sends probe packets to a set of candidate relays. The server side aggregates those measurements into a latency map. The interesting metric they optimise for is not the average — it's the 99th percentile of one-way voice latency. They want that number under 50 milliseconds. If a relay's P99 creeps above that, the routing algorithm automatically shifts traffic to a better relay. Luna: So they're constantly tuning routing in real time. That's a lot of moving parts. Lucas: It is, and it's all built on top of their internal service mesh. The routing logic runs in a service called 'the gong' — I'm not kidding, that's what they call it internally. The gong collects latency data from every client every 30 seconds and recomputes the optimal relay assignments. Luna: And how many engineers are on the voice team? Because this sounds like a massive investment. Lucas: Discord has about 15 engineers dedicated to the voice and video team. That's a small team for the scale they operate at. The key is that they own the entire stack — from the Rust codec to the relay hardware to the routing algorithm. They don't outsource any of it. And they open-sourced some of the components, like the Siren codec library. Luna: That's actually a good point — open-sourcing parts of the stack helps the community, but it also helps Discord by getting external scrutiny. And speaking of community support, I know the listeners of this show really value deep dives like this. A couple of dollars a month is genuinely what keeps these going — buy me a coffee dot com slash fexingo, if you've gotten something out of them. Lucas: Yeah, it makes a real difference. And it's what keeps the show ad-free so we can cover topics like this without fluff. So back to Discord — one last detail I think is worth mentioning. The entire voice pipeline runs with a fixed CPU budget per user. They cap encoding at 2 percent of a single core on the client side. If the codec goes over that, they downgrade quality rather than letting latency spike. Luna: Interesting — so latency is always the priority, even over audio quality. That makes sense for a gaming chat where you need to react in real time. Lucas: Exactly. And it's why Discord's voice latency is consistently lower than competitors like TeamSpeak or in-game voice. For CTOs building real-time features, the lesson is that you have to be willing to rewrite the bottom of the stack, not just configure existing tools. The payoff is a measurable competitive advantage. Luna: Well said. Thanks Lucas. That's all for this episode of The CTO Podcast.