Latest / The CTO Podcast with Fexingo: Technical Leadership, Architecture, and Engineering Org / How Duolingo Cut Translation Costs by 40 Percent with a Custom MT Pipeline
Transcript
- Lucas: So Duolingo cut the cost of translating its course content by 40 percent last year, and they did it by building a custom machine translation pipeline instead of relying on off-the-shelf engines. That's a big deal for a company whose entire product is language learning. Luna: Forty percent is huge. But isn't machine translation something you can just outsource to Google or DeepL? Why would a language app even need to build its own? Lucas: Right, and that's exactly the question their engineering team asked. The thing is, Duolingo isn't just translating user-generated content on the fly. They're translating course material, which has to be pedagogically accurate, and they're doing it across dozens of languages simultaneously. Off-the-shelf engines are general purpose, so they're not tuned for that specific use case. Luna: So they built a model that understands language learning. What does that actually mean in practice? Lucas: So they trained a custom model on their own corpus of translated course content, which is millions of sentences that have already been reviewed by human linguists. That gives the model a lot of context about the kind of language used in lessons, the tone, the vocabulary level, even the specific sentence structures that work well for learners. The result is translations that need far less post-editing. Luna: So the cost savings come from less human review. But aren't they trading off one cost for another? Lucas: That's the clever part. They built a system that flags translations where the model's confidence is low, and only those get sent to human reviewers. High-confidence translations skip the human loop entirely. So they're not eliminating human oversight, they're targeting it to where it matters most. Luna: Makes sense. But how do they measure 'confidence' in a way that's actually reliable? Lucas: That was one of the hardest parts, honestly. They used a combination of the model's internal probability scores and a separate quality estimation model that's trained to predict how well a translation will be received. That quality estimator is the real innovation here. It's essentially a second model that grades the first model's output. Luna: So it's like having a grade your own homework system. But what about languages where they don't have enough training data? Like, say, a less common language pair? Lucas: That's a fair question, and the answer is they don't use the custom model for every pair. For low-resource languages, they still rely on commercial engines, but they use the same quality estimation model to decide when to route to human review. So the cost savings are mainly in high-resource pairs like English to Spanish, but the infrastructure is consistent across the board. Luna: So the real win is that quality estimation model, not the translation model itself. That's a subtle but important distinction. Lucas: Exactly. And that's a lesson that applies well beyond Duolingo. When you're thinking about building custom AI, the highest leverage is often in the evaluation layer, not the generation layer. A cheaper model with a smart filter can outperform a top-tier model that costs ten times more. Luna: That's a really useful way to think about it. I've been in conversations where the default is always 'let's use the most powerful model we can afford,' but this suggests you should be more surgical. Lucas: Yeah, and that's a mindset shift that a lot of teams are starting to make, especially as the cost of running large models becomes a real line item in the budget. And speaking of budget, it's worth remembering that none of this would be possible without the listeners who support us, honestly. A handful of folks chip in a few dollars a month through buy me a coffee dot com slash fexingo, and that's literally what keeps this show ad-free and lets us spend time digging into cases like this. Luna: Yeah, it's a small group, but it makes a big difference. We're not trying to shout it from the rooftops, just wanted to be transparent about how it works. Lucas: So back to Duolingo's pipeline. They also had to make some infrastructure choices. They're running inference on their own hardware, which gave them more control over cost and latency. They used a mix of GPUs and CPUs depending on the model size and the traffic patterns. Luna: That's interesting. I'd have thought GPUs for everything. When does a CPU make sense? Lucas: For their smaller quality estimation model, CPUs are actually fine, and they're more cost-effective for low-volume, bursty traffic. The heavy lifting on the translation model runs on GPUs, but they're very selective about when to invoke it. By routing easy sentences away from the GPU, they save a lot of money. Luna: So it's not just the model choice, but the orchestration around it. That's the part that probably doesn't show up in a blog post. Lucas: Right. They also did a lot of work on batching and caching. Frequently asked translations get cached, and they batch requests to maximize GPU utilization. All of that adds up to the 40 percent number. Luna: And what about quality? Is the custom model actually better than the commercial ones, or just cheaper? Lucas: They say the quality is comparable, sometimes better, because it's tuned to their domain. But the real quality win is the human review routing. Since they only send low-confidence translations to humans, the reviewers spend their time on the hardest cases, which means the final output is consistently high quality. Luna: That's a nice feedback loop. The humans get better at their jobs because they're always working on the edge cases, and the model learns from that. Lucas: Exactly. And that's the kind of flywheel that's hard to replicate with a pure outsourcing approach. The model keeps improving because it's trained on the corrections made by human experts, and those experts are only looking at the most challenging content. Luna: So if I'm an engineering leader at a company thinking about something similar, what are the key takeaways? Lucas: First, think about your evaluation layer, not just the model. Second, don't assume you need the biggest model for every task. Third, invest in infrastructure that lets you route intelligently between cheap and expensive paths. And fourth, be willing to start small with a high-resource domain and expand from there. Luna: That's a solid blueprint. And it's reassuring that you don't need to be a language learning app to apply this. Any company with a lot of text that needs translation or even content generation could learn from this. Lucas: Definitely. And the 40 percent number is a great anchor for your own budget discussions. If Duolingo can do it, it's worth exploring whether your team can too. Luna: So what's next for Duolingo on this front? Are they expanding this to other areas? Lucas: They're already using similar techniques for generating practice exercises, not just translating content. The same quality estimation approach helps them decide which exercises are worth showing to learners. It's a natural evolution of the same infrastructure. Luna: So the whole product is becoming more model-driven. That's a good place to leave it. Thanks for walking us through it, Lucas. Lucas: Anytime. And if any of this sparks an idea, we'd love to hear about it. We'll be back with another deep dive soon.