Latest / The CTO Podcast with Fexingo: Technical Leadership, Architecture, and Engineering Org / How MongoDB Rebuilt Its Query Engine for Real-Time Analytics
Transcript
- Lucas: So MongoDB just shipped a pretty significant rewrite of its query engine — and I think the most interesting part is that it's designed to handle real-time analytics on the same cluster that's serving your transactional workloads. Luna: That's the holy grail for a lot of teams, right? One database for both OLTP and OLAP without having to ETL everything into a separate warehouse. Lucas: Exactly. And the challenge is that those two workloads have fundamentally different access patterns. Your typical transactional query touches a handful of documents by key — it's point lookups, small writes. Analytics wants to scan millions of documents, aggregate over time ranges, compute percentiles. Those don't play nice on the same storage engine. Luna: So how did MongoDB approach it? Did they add a separate columnar store, or did they rework the existing B-tree? Lucas: They did both, actually. The core insight was that they needed to introduce a columnar compression layer on top of their existing document storage. So now, when you create a time-series collection — which they introduced a couple of years ago — the data is automatically bucketed into time intervals and stored in a columnar format behind the scenes. The query engine then decides at runtime whether to use the row-oriented path or the columnar path. Luna: So it's a hybrid transactional/analytical processing — HTAP — approach, but without forcing the user to manage two separate systems. Lucas: Right. And the really clever part is the optimizer. They rewrote the query planner to understand time-series semantics. So if you write an aggregation pipeline that filters on a timestamp range and then does a group-by on a sensor ID, the optimizer recognizes that pattern and pushes the computation down into the columnar layer. It can skip entire buckets of data that fall outside the time window. Luna: What kind of performance numbers are they seeing? I remember reading that some early testers were getting sub-50-millisecond query latencies for analytics that used to take seconds. Lucas: Yeah, the benchmark they published is impressive. They demonstrated a cluster handling 10 million writes per second — that's the transactional side — while simultaneously serving analytical queries with a p99 latency under 50 milliseconds. That's a two-order-of-magnitude improvement over the previous version for those analytical workloads. Luna: Ten million writes per second is insane. How are they even sustaining that throughput without the analytics queries starving the writes? Lucas: That's where the disaggregated storage model comes in. They separated the compute layer from the storage layer a few years ago — it's called the DocumentDB architecture internally. Writes go to a primary node in a replica set, but the columnar data is stored in a separate set of nodes that can be scaled independently. So the analytical queries read from those columnar replicas, and they never touch the primary write path. Luna: So it's not quite a Lambda architecture where you have a speed layer and a batch layer, but it's similar in spirit. Lucas: It's closer to a Kappa architecture, actually — single streaming pipeline, but with multiple materialized views. The data flows in once, gets written to the row store, and then asynchronously gets converted into the columnar format. The columnar view is eventually consistent, but for most analytics use cases, that's fine. You're usually looking at trends, not the last second. Luna: That makes sense. And it avoids the operational complexity of running both Kafka and Spark just to keep two databases in sync. Lucas: Exactly. And that's the real value proposition for a CTO — you reduce your infrastructure footprint. Instead of having MongoDB for your app, plus a separate ClickHouse or Druid cluster for analytics, plus an ETL pipeline between them, you just have MongoDB. The schema is the same — you define the time-series collection, and the engine handles the rest. Luna: But there's got to be trade-offs. What about schema flexibility? One of MongoDB's big selling points has always been that you can have different fields in different documents. Does the columnar format force you into a rigid schema? Lucas: That's a great question. They handled it by introducing a concept called 'implicit schema' for time-series collections. You define a set of measurement fields — like temperature, humidity, whatever — and those get stored in columns. But you can still have metadata fields that are stored as documents inside each bucket. So you get the flexibility of a document model for the metadata, but the efficiency of a columnar model for the numeric measurements. Luna: Interesting. So the metadata fields aren't compressed as well, but they're rarely the thing you're aggregating over anyway. Lucas: Right. And the metadata is typically small — a device ID, a location string. You don't need columnar compression for that. The big wins come from the numeric measurements that repeat across millions of documents. Luna: I wonder how this affects their engineering org. Rewriting the query engine is a multi-year project. How do you even staff that without breaking everything else? Lucas: Yeah, this wasn't a small team. They had a dedicated squad of about 30 engineers working on it for roughly two years. And they did it incrementally — they shipped the time-series collections first, then the columnar storage backend, then the optimizer rewrite. Each piece was backward compatible, so existing users could adopt it at their own pace. Luna: That's the smart way to do a big architectural change. Don't try to boil the ocean. Lucas: Absolutely. And I think the lesson for other CTOs is: if you have a hybrid workload, you don't necessarily need to throw another database at it. The database vendors are catching up. MongoDB, PostgreSQL with its columnar extensions, even MySQL with HeatWave — they're all moving in this direction. Luna: So if someone's listening and they're considering this for their stack, what's the threshold where it makes sense? Lucas: I'd say if you're doing more than, say, 100,000 writes per second and you need sub-second analytics on that data — especially time-series data — then it's worth evaluating. Below that threshold, a simple replica set with a separate read-only node is probably fine. But once you start hitting those scale numbers, the overhead of maintaining a separate analytics pipeline becomes real. Luna: And you mentioned the Kappa architecture — are there any gotchas with eventual consistency in the columnar view? Lucas: The main gotcha is that your analytics queries will lag behind real-time writes by a few seconds — typically under five seconds. So if you need truly real-time dashboards that update every second, you might need to fall back to the row store for the most recent data. But for most monitoring and reporting use cases, a five-second lag is perfectly acceptable. Luna: That's a reasonable trade-off. And it's still much better than the hours of lag you'd get with a traditional ETL pipeline. Lucas: Right. And speaking of things that are worth it — if this kind of deep dive into database architecture is useful for what you're building or running, that's exactly why we keep this show ad-free. If today's episode was worth a coffee to you, the link is buy me a coffee dot com slash fexingo. Seriously, that's all it takes to keep these conversations going. Luna: It's a small thing that makes a big difference for us. And we really appreciate everyone who's chipped in. Lucas: Alright — back to the architecture. One thing I didn't mention is how they handled the query optimizer for mixed workloads. In the old engine, the optimizer was purely cost-based, and it assumed all data was in the row store. Now it has to decide whether to use the columnar index or the row index for each stage of the aggregation pipeline. Luna: That sounds like a hard problem. How do they estimate the cost of scanning a columnar bucket versus a row-based index? Lucas: They built a new statistics collection system that samples the data distribution in each bucket. The optimizer uses those statistics to estimate how many documents will match a filter, and then it picks the access path with the lowest estimated cost. For time-series queries, the columnar path almost always wins because of the compression and the ability to skip entire buckets. Luna: So the optimizer is essentially doing what a good database administrator would do manually — choosing the right index for the query. Lucas: Exactly. And they exposed some of that decision-making through the explain plan, so developers can see why the optimizer chose one path over another. That transparency is critical for debugging performance issues. Luna: One last question — what about data durability and backups? With a separate columnar store, do you need to back up both copies? Lucas: That's a good operational concern. In their architecture, the columnar store is derived from the row store. So you only need to back up the row store — the columnar data can be rebuilt from it if needed. But in practice, most users enable point-in-time recovery on both stores because rebuilding petabytes of columnar data would take too long. Luna: So it's a trade-off between storage cost and recovery time. Makes sense. Lucas: Yeah. And I think that's the theme of this whole episode: MongoDB made a series of deliberate trade-offs to give you one database that does two things well, rather than forcing you to choose between simplicity and performance.