Latest / The CTO Podcast with Fexingo: Technical Leadership, Architecture, and Engineering Org / How Loom Rebuilt Its Video Engine for Async Communication
Transcript
- Lucas: So Loom now processes over 200 million videos a year, and the interesting thing is that the average video length has actually gone down — from about 4 minutes in 2020 to under 2 minutes today. Luna: That shorter length must change the engineering constraints pretty significantly. Less time to buffer, less tolerance for latency. Lucas: Exactly. And it forced them to rethink the entire encoding pipeline. Originally, they used a monolithic FFmpeg-based encoder running on EC2 instances. But as volumes grew, the cost and latency became unsustainable. Uploads could take 30 seconds even for a 2-minute video. Luna: 30 seconds for a two-minute video — that's brutal for an async communication tool where people expect instant feedback. Lucas: Right. So about two years ago, the engineering team started a major rebuild. The key insight was that most users never watch the full video — they scrub through or watch the first few seconds. So they optimized for fast start of playback rather than perfect full-resolution encoding. Luna: They essentially prioritized the first frame over the last frame. Lucas: Yeah, exactly. They moved to a tiered encoding strategy: a quick low-res proxy renders within seconds, and then the high-res version finishes in the background. But to make that work, they had to decouple the encoder from the server entirely. Luna: How do you decouple encoding from the server? Isn't that what FFmpeg does on the backend? Lucas: Traditionally yes. But they experimented with client-side transcoding using WebAssembly. The recorder itself — running in the browser or the desktop app — now does a lightweight first-pass encode using a custom Wasm module. That module handles the H.264 baseline profile, which is enough for the preview. Luna: So the client does the heavy lifting for the first pass, and then the server only has to do the final polish. Lucas: Exactly. And the server-side pipeline moved from a single FFmpeg process to a chain of microservices: one for audio normalization, one for resolution upscaling, one for compositing overlays like the circular video of the speaker. Each service is written in Rust, which they chose specifically for its performance and memory safety. Luna: Rust — that's a big shift from Python or Go that most video pipelines use. Lucas: It was a bet. But they found that Rust gave them about 60 percent less CPU usage per encoding job compared to the Go version they prototyped. And because they control the memory allocation tightly, they could run more concurrent encodes per instance without hitting out-of-memory errors. Luna: Sixty percent less CPU — that's a massive efficiency gain. Did it come at the cost of development speed? Lucas: The team lead mentioned that the first version took about 4 months to build, versus maybe 2 months in Go. But the operational savings paid that back in the first quarter. They also built a comprehensive test harness using synthetic video fixtures, so they caught regressions early. Luna: What about storage? 200 million videos a year adds up fast. Lucas: That's the other piece. They moved from storing every video in three resolutions to an adaptive bitrate model. The raw upload is stored once, and then they generate on-the-fly segments. For popular videos — say, ones viewed more than 10 times — they cache the high-bitrate version in a CDN. For long-tail videos, they only keep the low-bitrate proxy and generate higher resolutions on demand. Luna: So storage becomes a function of actual consumption, not just creation. Lucas: Right. And that reduced their total storage costs by about 40 percent. The trade-off is that if a long-tail video suddenly goes viral, there's a brief spike in encoding latency — maybe 5 to 10 seconds — while the higher bitrate versions are generated. But they decided that's acceptable because viral events are rare, and the cost savings are permanent. Luna: I imagine the playback experience also had to change. With adaptive bitrate and on-the-fly generation, how do they handle scrubbing — jumping to a random point in the video? Lucas: They built a custom segment index that maps time offsets to byte ranges. When you scrub, the player requests a small I-frame segment — just a few kilobytes — so you get a thumbnail preview almost instantly. The actual video stream then catches up. This is similar to how YouTube does it, but Loom optimized for the fact that their videos are shorter, so they pre-generate the index at upload time. Luna: That makes sense. Pre-generating a small index is cheap compared to re-encoding the whole video. Lucas: Exactly. And they store that index alongside the raw upload in the same object store — currently using S3 with Infrequent Access tiering for older videos. They also added a feature where the player can request a lower resolution if the network is slow, without re-buffering the entire video. Luna: So the architecture is really built around the async use case — fast start, quick scrubbing, graceful degradation. Lucas: Yeah, and it's worth noting that this rebuild wasn't just about performance. It also improved reliability. The old monolithic encoder had a single point of failure — if the FFmpeg process crashed, the whole upload failed. Now, each microservice can fail independently, and the orchestrator retries only the failed step. Luna: Did they see a drop in upload failure rates after the migration? Lucas: They went from about a 2.5 percent failure rate to under 0.3 percent. And the average time to playback — from hitting record to seeing the first frame — dropped from 12 seconds to under 1 second for the preview. That's a huge user experience improvement. Luna: That kind of change probably impacts user retention directly. Nobody wants to wait for a video to process just to say 'hey, look at this bug.' Lucas: Exactly. And that's the core of the async communication value proposition — it should be as instant as a text message, but with the richness of video. If today's deep dive was useful to you, especially if you're building or scaling a media platform, these conversations stay ad-free thanks to listener support. You can help keep it going at buy me a coffee dot com slash fexingo. Luna: Yeah, it's a small way to make sure we keep bringing you these detailed technical stories. Lucas: Alright, back to the architecture. One more detail I found fascinating: they built a custom multiplexer that combines the audio and video streams at the edge, using Cloudflare Workers. That way, the first segment doesn't need to travel all the way to the origin server — it gets assembled at the nearest pop. Luna: So they're using edge compute not just for caching, but for actual stream assembly. That's clever. Lucas: Yeah, it shaves off another 200 to 300 milliseconds of latency. And because the multiplexer is stateless, it scales horizontally without any coordination. They're serving about 15 million requests a day through that edge pipeline now. Luna: I'm curious about the team size. How many engineers does it take to maintain this kind of architecture? Lucas: The core video team is about 12 engineers — 5 focusing on the encoding pipeline, 4 on playback and CDN, and 3 on the client-side recorder. But they also rely heavily on platform teams for infrastructure and data pipelines. The key is that each microservice is owned by a single team, so there's clear ownership. Luna: That seems lean for the scale they're at. Twelve engineers for 200 million videos a year. Lucas: It's lean, but they've invested heavily in automation. For example, they have a canary deployment system that automatically rolls back if encoding latency spikes more than 10 percent. And they use feature flags to enable new encoders gradually — starting with 1 percent of users, then scaling up. Luna: So the risk of a bad deployment is contained by both automation and gradual rollout. Lucas: Exactly. And they've built a robust monitoring system around three key metrics: time to playback, encoding success rate, and cost per encoded minute. Those three numbers drive most of their engineering decisions. Luna: It sounds like a model for any startup building media infrastructure — start with a monolith, but plan your migration to microservices around specific user-facing metrics. Lucas: Right. And the lesson for CTOs is to be ruthless about which metrics matter. Loom didn't optimize for perfect video quality — they optimized for speed and reliability, because that's what async communication demands. Luna: That's a good note to end on. Thanks, Lucas. Lucas: Thanks, Luna. See you next time.