Latest / The CTO Podcast with Fexingo: Technical Leadership, Architecture, and Engineering Org / How Postgres Tamed the Cloud Database Chaos
Transcript
- Lucas: So, Luna, quick question. How many databases does the average tech company run today? Luna: If you'd asked me five years ago, I'd have said three or four. Now I'm guessing closer to six or seven. Lucas: Right. And that sprawl is exactly what a lot of CTOs have been quietly trying to reverse. Because every database you add is another thing to patch, scale, secure, and hire for. And here's the twist: for more and more companies, the answer isn't a new database at all. It's Postgres. Luna: Postgres, the open-source database that's been around since the mid-nineties. The one that everyone kind of knew was solid but nobody got excited about. Lucas: Exactly. But in the last three years, it's become the default for new workloads. I was looking at a survey from last quarter — over forty percent of new database deployments in the cloud are Postgres. That's more than MySQL and MongoDB combined. Luna: Forty percent. That's huge. What's driving it? Lucas: A few things. First, the cloud providers themselves. Amazon Aurora — which is compatible with both MySQL and Postgres — actually started as a MySQL service. But for the last few years, they've been pushing the Postgres version hard, because customers are asking for it. And Aurora Postgres is now the default recommendation for anyone building on AWS. Luna: So even Amazon is betting on Postgres. But what about the companies that need something more than a relational database? I'm thinking high write throughput, or flexible schemas. Lucas: That's where the extension ecosystem comes in. Postgres has over three hundred extensions. You want a document store? There's one for that — it gives you JSONB with a lot of the same flexibility as MongoDB. You want time-series? There's TimescaleDB. Full-text search? There's one for that too. You can get most of the capabilities of a specialised database without actually spinning up another service. Luna: But isn't that a double-edged sword? If you're building on extensions, aren't you just trading one vendor lock-in for another? Lucas: That's the important nuance. Some extensions are maintained by third-party companies, and if you rely on them, you're dependent on that company's release cycle. But the core of Postgres is open source, and the extension APIs are stable. So you're not locked into a cloud provider's proprietary implementation. You could migrate from Amazon to Google to a self-hosted cluster without rewriting your application. Luna: That's a huge difference from, say, DynamoDB or Cosmos DB, where you're really locked into the provider. Lucas: Right. And that portability is a big reason why companies are consolidating. I spoke with a CTO last month who ran five different databases — a document store, a time-series DB, a search engine, a key-value store, and a relational database. They moved the time-series and document workloads into Postgres with extensions, and now they're down to two. Luna: And what was the payoff? Besides fewer things to patch. Lucas: Operational simplicity is the big one. But there's also a hiring angle. Postgres is the most widely known database in the world. Every backend engineer has used it. So you're not hiring specialists for each database. And because it's SQL, it's easier for analysts and data scientists to self-serve. That's a real productivity win. Luna: But Postgres has historically had a reputation for not scaling horizontally. How do you handle that when you get big? Lucas: That's changing. The old advice was 'shard it yourself' or 'use a NoSQL database.' But in the last few years, you've got projects like Citus — which is now an extension — that gives you distributed Postgres. And then you've got the serverless wave. Neon is a good example. It's a serverless Postgres that separates storage from compute, so you can scale to zero when you're not using it. That's a game-changer for dev and test environments. Luna: Scale to zero, that's the dream. You're not paying for a database that's just sitting there. Lucas: Exactly. And Neon also does branch cloning — you can create a branch of your database for every pull request, run your tests, and throw it away. That's a workflow that used to require a whole team of database reliability engineers. Now it's a button. Luna: So Postgres is absorbing the feature sets of other databases. At what point does it just become a monolith that does everything but nothing well? Lucas: That's the fair question. And the honest answer is that extension quality varies. Some are production-grade, some are experimental. The Postgres community is pretty good about marking maturity. But you're right that not everything should be forced into Postgres. If you have a truly massive graph workload, or a search index with billions of documents, you might still want Neo4j or Elasticsearch. But for the vast majority of workloads — ninety percent of what people actually need — Postgres is enough. Luna: And I think that's the key insight. Most applications don't need the exotic stuff. They need ACID transactions, a solid query language, and the ability to handle a few million rows without breaking a sweat. Lucas: Right. And Postgres has been doing that for thirty years. But let's be honest — there was a period in the mid-2010s where everyone thought SQL was dead. MongoDB was the hot thing. But a lot of that was because scaling relational databases was genuinely hard. Now that the cloud providers have made Postgres just as easy to scale as NoSQL, the pendulum is swinging back. Luna: And that's the thing — the cloud made infrastructure easier, but it also fragmented the database landscape. Now Postgres is the consolidation point. Lucas: Yeah. And the numbers back it up. The db engines ranking has Postgres at number four, but it's been climbing steadily. And in the latest Stack Overflow survey, Postgres overtook MySQL as the most popular database among developers. That's a symbolic shift. Luna: I remember when that survey came out and everyone was like, 'Finally.' Lucas: Exactly. So what does this mean for a CTO making a decision today? My take is this: when you're evaluating a new database, ask yourself if Postgres can do the job first. Nine times out of ten, the answer is yes. And even when it's not, you'll get a clearer picture of what you actually need — because you'll have to identify the specific gap that Postgres can't fill. Luna: And that process of elimination is a lot better than just picking the newest database on the block because it has a cool name. Lucas: Right. 'We're using CockroachDB because it sounds like a party.' Luna: Ha. But honestly, there's a lesson here for how we approach tech decisions generally. The boring, mature choice often wins. And I think that's a theme of this show — we look at how builders make pragmatic choices, not just flashy ones. Lucas: Yeah. And speaking of the show — we've been doing this for a while now, and one thing we've never done is run ads. We like it that way. It keeps the conversations honest. But it means we rely on listeners who find value in what we do. If that's you, and you want to help keep this thing ad-free, the link is buy me a coffee dot com slash fexingo. Luna: It's a small gesture that makes a big difference for us. And honestly, it's the only way we fund the show — no corporate sponsors, no fancy ads. Lucas: So if you're a regular listener and you've gotten something out of these conversations, that's where you can make a difference. But no pressure — the episodes keep coming either way. Luna: Right. So back to the consolidation story — what about the teams that are already deep in a multi-database world? Is it too late for them to migrate? Lucas: That's the hard part. Migration is never cheap. But the good news is that Postgres has really good compatibility with a lot of legacy databases. There are tools that can convert MySQL schemas almost automatically. And for NoSQL workloads, JSONB takes a lot of the pain out of moving documents over. I've seen teams do it in a few weeks, not months. Luna: But is there a risk that you're just moving your data into a dumpster fire because you didn't redesign your queries? Lucas: That's a real concern. If you just lift and shift, you might not get the performance you expect. The key is to spend time on the query optimization. Postgres has an excellent query planner, but it needs the right indexes and the right schema. You can't just copy a MongoDB collection and expect it to fly. But that's true of any migration. Luna: So what's your one concrete takeaway for someone who's listening and thinking about their own stack? Lucas: Take a hard look at what you're using and ask, 'Do I really need all these databases?' You might find that you can consolidate two or three into Postgres with extensions, cut your infrastructure cost, and make your team's life easier. And if you're starting something new, just start with Postgres. It's the default for a reason. Luna: And that reason is that it's battle-tested, flexible, and everywhere. It's the boring choice that turns out to be the exciting one. Lucas: Exactly. Because in a world of constant change, having one less thing to worry about is a big deal. So, will Postgres eventually eat everything? Probably not everything. But it's going to keep taking a bigger and bigger bite. And we'll be here to watch it happen.