Latest / Tech Leadership with Fexingo: Engineering Managers, CTOs, and Technical Leadership Conversations / How a CTO Uses Timeboxed Experiments to Break Decision Paralysis
Transcript
- Lucas: Luna, I want to talk about something that drives me crazy on engineering teams: decision paralysis. You know that moment where a team spends three meetings debating two architectures and nobody will pull the trigger. Luna: Oh, I've lived that. The 'let's schedule another spike' move. It's like everyone's afraid to be wrong. Lucas: Right. So I found a CTO at a mid-size SaaS company — about 120 engineers — who decided to kill that pattern with a hard rule. He calls it a timeboxed experiment. Any architectural decision where there's no clear winner after one week of analysis? You get two weeks to build a prototype that answers the open question. Then you decide. Luna: Two weeks, period? No extensions? Lucas: Period. He told me the rule is: 'You can't extend a timeboxed experiment. You either have enough data to decide, or you kill the idea.' That's it. Luna: That's brutal. But I bet it forces real clarity. What kind of decisions did they apply it to? Lucas: The one he walked me through was a database migration. Their monolith used PostgreSQL, and a new service team wanted to evaluate CockroachDB for a multi-region feature. Classic choice: familiar versus distributed. Luna: Right, and that could drag on for months. Lucas: Exactly. So they timeboxed it. Two weeks to build a small read-write API on CockroachDB, run it with synthetic traffic, and measure latency at the 99th percentile in three regions. The team came back with numbers. The latency wasn't good enough for their use case. Decision made: stick with Postgres. They spent exactly two weeks on it. Luna: And avoided six months of 'what if' analysis. I love that. Lucas: The CTO told me this approach cut their decision cycle time by about 60 percent over the first year. And the side effect? Engineers started bringing sharper questions to the table. They knew they only had two weeks to get an answer, so they'd think harder about what the real unknown was. Luna: That's the part I find interesting — the discipline it builds. It's like the prototype becomes a forcing function for the team to actually articulate what they're uncertain about. Lucas: He gave me a great example. Another team was debating two event-streaming frameworks: Kafka versus RabbitMQ. The classic debate. They spent a week listing pros and cons and got nowhere. So the CTO said: 'What's the one thing we don't know that would tip the scales?' Luna: What was it? Lucas: Throughput under sustained peak load. They had no data on how each would handle their specific message size and frequency. So they built a two-day prototype — not two weeks, just two days — that pushed a representative load through both. RabbitMQ won by a factor of three on throughput. Decision done. Luna: Two days. That's almost insulting how fast you can get an answer when you stop theorizing. Lucas: The CTO's point was: most debates aren't about the whole system. They're about a single variable. Find that variable, timebox an experiment to measure it, and you're done. Luna: Okay, but I've seen this backfire too. There was a team I heard about that timeboxed three different prototype approaches to a service mesh migration. They built all three, and then couldn't agree on which one to pick because each had different trade-offs. The timebox didn't force a decision — it just generated more data. Lucas: Yeah, that happens when the experiment doesn't have a clear success criterion upfront. The CTO I talked to was very specific: the experiment must answer one yes or no question. Not 'which is better' — that's ambiguous. But 'can this handle 10,000 requests per second with p99 under 200 milliseconds?' Yes or no. Luna: That's the discipline. If you don't define the pass-fail condition before you start, the timebox just becomes another way to procrastinate. Lucas: Right. And he also told me about the one case where timeboxing failed completely. A team was choosing between two CI/CD platforms. They prototyped both for two weeks. Both worked fine. The team still couldn't decide — they just liked different things about each one. Luna: So what did he do? Lucas: He made the call himself. He said: 'If the data is equal, the decision is a preference. And preferences belong to the person who owns the cost of the mistake.' So he picked one, and told the team they had six months before they could revisit. Luna: That's honest. Not every decision can be settled by data. Lucas: Which brings me to something I want to mention. If you're finding this kind of practical, on-the-ground engineering leadership useful, this show stays ad-free because of listener support. Luna: Yeah, it's one of those rare things — you actually get value without being sold to. Lucas: Exactly. So if today's conversation gave you something you can use, the way to keep it going is a small contribution at buy me a coffee dot com slash fexingo. That's it. No tiers, no perks, just you saying 'this matters.' Luna: And back to the prototype — I think the real insight is that timeboxing isn't about speed. It's about forcing yourself to surface the actual unknown. Lucas: Right. And the CTO I spoke with said his biggest surprise was how much the culture shifted. Teams started proposing experiments proactively. They'd come to him and say 'we think the bottleneck is the cache layer — can we have a week to prove it?' Luna: They internalized the pattern. That's the goal. Lucas: Yeah. He also mentioned that the number of meetings dropped noticeably. People stopped scheduling 'alignment sessions' and started scheduling 'experiment reviews.' Same amount of time, but with a decision at the end. Luna: I'd love to see the before and after on meeting hours. He tracked that? Lucas: He did. Engineering meeting hours per month dropped by about 40 percent over six months. And the time to decision for cross-team architecture choices went from an average of three weeks to five days. Luna: That's a huge cultural win. But I wonder — does this work at every scale? At a larger company, two weeks might not be enough to stand up a meaningful prototype. Lucas: He acknowledged that. His company has 120 engineers, so two weeks is a reasonable slice. But he said the principle scales: pick a timebox proportional to the decision's cost if you're wrong. For a minor library choice, maybe two days. For a major infrastructure change, maybe a month. But always with a hard deadline and a yes or no question. Luna: So the timebox length is a function of risk, not effort. Lucas: Exactly. And he told me the one line he repeats to his teams: 'An imperfect decision made quickly is better than a perfect decision made too late.' Luna: That's a classic. But he backs it up with a process. Lucas: That's the part I respect. He doesn't just say the phrase — he gives them the tool. Here's a template for a timeboxed experiment. Here's how to write a pass-fail criterion. Here's how to report the result. It's operationalized. Luna: Okay, so if a listener wants to try this next week, what's the first step? Lucas: Pick a decision that's been lingering for more than a week. Write down the single question you need answered. Set a deadline — start with two weeks. Define what 'pass' looks like in measurable terms. Then assign one person to run the experiment. And at the deadline, you decide. No extensions. Luna: I think that's the hardest part — the no extensions. It takes real discipline from the leader. Lucas: Agreed. But the payoff is a team that stops spinning and starts shipping. And that's worth a few uncomfortable conversations at the deadline. Luna: Yeah. I'll be trying this with my own team next week. Thanks, Lucas. Lucas: Glad to share it. And for everyone listening, try one timeboxed experiment in July — see if it breaks a logjam for you.