Latest / The CTO Podcast with Fexingo: Technical Leadership, Architecture, and Engineering Org / How Ethereum Switched to Proof of Stake Without Breaking
Transcript
- Lucas: If these conversations are useful for what you're building or running, the simplest way to keep them going is throwing a couple of dollars a month at the show. Buy me a coffee dot com slash fexingo — it genuinely covers hosting and keeps these ad-free. Luna: Yeah, and it makes a real difference. We're not backed by a big network, it's just listener support. Lucas: Exactly. So, September 15, 2022 — 2:42 UTC — Ethereum switches off proof of work and turns on proof of stake. No downtime. No chain split. The entire network, 500,000 validators, billions in value, transitions in one block. Luna: And people had been calling it impossible for years. 'You can't change the consensus of a live blockchain with that much value without something catastrophic.' Lucas: Right. So how did they actually pull it off? I want to focus on the engineering architecture that made the Merge possible — because it's not a single code change. It's a coordinated upgrade across two entirely separate software layers that ran in parallel for over a year. Luna: The Beacon Chain. That's the proof of stake consensus layer that went live in December 2020, almost two years before the Merge. Lucas: Exactly. The Beacon Chain was running alongside the existing proof of work chain — processing attestations, managing validators, building a finality gadget — but it wasn't executing transactions. It was essentially the consensus engine idling, waiting for the execution layer to plug in. Luna: So it's like building a new transmission while the car is still driving on the old one, and then swapping it out at highway speed. Lucas: That's the analogy. The execution layer — what was the Ethereum mainnet — kept processing blocks using proof of work. Meanwhile, Beacon Chain was building its own canonical chain of consensus data. The Merge is the moment where the execution layer stops using miners and starts using the Beacon Chain's validators to determine which blocks are canonical. Luna: But that requires the execution layer to know about the Beacon Chain's finality. How do you bridge that without a hard fork that resets everything? Lucas: Two coordinated upgrades: Bellatrix and Paris. Bellatrix was the consensus layer upgrade, which happened on September 6. It prepared the Beacon Chain for the Merge by adding the logic to accept execution payloads from the execution layer. Paris was the execution layer upgrade, triggered by a terminal total difficulty — a specific difficulty value on the proof of work chain. Luna: So Paris didn't have a fixed block number. It was triggered when the difficulty reached a certain threshold. Lucas: Exactly. The Ethereum mainnet had a difficulty bomb — code that progressively increased the difficulty of mining, making it unsustainable. The Merge was timed so that the difficulty bomb would make mining impossible around the target date. So when the execution layer clients saw a block with that terminal total difficulty, they knew to stop mining and start following the Beacon Chain's head. Luna: And that happened on block 15537393. I remember watching the countdown. Lucas: That block was mined by a proof of work miner. The next block, 15537394, was proposed by a validator on the Beacon Chain. And it was a valid block with transactions. The transition was seamless from the user perspective — your wallet, your dApp, your smart contract, nothing changed. Luna: Except the energy consumption dropped by 99.95 percent. That's not nothing. Lucas: Right. Ethereum went from consuming about 0.2 percent of global electricity — roughly the same as the Netherlands — to about 0.002 percent. But that's a consequence, not the engineering challenge. The real challenge was client diversity and coordination. Luna: Because you've got multiple execution layer clients — Geth, Nethermind, Besu, Erigon — and multiple consensus layer clients — Prysm, Lighthouse, Teku, Nimbus, Lodestar. All of them need to agree on the transition logic. Lucas: Exactly. The Ethereum Foundation ran a series of shadow forks — testnet-like environments that mirrored mainnet state — to simulate the Merge months in advance. They did at least six shadow forks, each one catching edge cases. For example, one shadow fork revealed that the execution layer clients had a different interpretation of how to handle the terminal total difficulty if the difficulty bomb caused an uneven block time. Luna: And that's the kind of bug that could have caused a chain split. One client finalizes on one fork, another client on another. Lucas: Exactly. So they standardized the terminal total difficulty calculation across all clients. They also ran a 'Merge readiness' checklist for node operators — you had to run both an execution client and a consensus client, and they had to be configured to talk to each other via the Engine API, which is a json rpc based interface. Luna: And that's where the actual integration happens. The execution layer sends execution payloads to the consensus layer, and the consensus layer sends fork choice updates back. Lucas: Right. The Engine API was defined in the Ethereum Improvement Proposal 3675. It specifies exactly how the two layers communicate. And it's designed to be client-agnostic — any execution client can work with any consensus client, as long as they both implement the spec. Luna: But in practice, most people ran Geth and Prysm. At one point before the Merge, Geth had over 80 percent market share on the execution layer. That's a centralization risk. Lucas: It is. And the Ethereum Foundation actively encouraged diversity. They funded development of minority clients like Nethermind and Besu. They also ran a client incentive program — small bounties for validators who switched to a minority client. By the time of the Merge, Geth was still dominant, but Nethermind and Besu had meaningful shares. Luna: And on the consensus layer, Prysm was also dominant — around 60 percent. But Lighthouse and Teku were significant. Lucas: The key thing is that no single client failure could take down the network. If Geth had a bug, as long as other clients were following the canonical chain, the network would survive. And that's exactly what happened a few weeks after the Merge — Geth had a bug that caused a chain split on the execution layer, but because other clients didn't have the same bug, the network recovered quickly without a loss of finality. Luna: So the Merge wasn't just a code change. It was a social coordination problem — getting thousands of node operators to upgrade their software, run two clients, and agree on a specific block height. Lucas: Exactly. And they did it with a remarkably low failure rate. According to the Ethereum Foundation, over 99 percent of validators successfully transitioned on the first try. Only a handful of operators who hadn't updated their clients or had misconfigured their stacks ended up on the wrong chain. Luna: And those were quickly resolved by the community. There was even a coordinated 'Merge watch party' where developers monitored the terminal total difficulty in real time. Lucas: I think the key takeaway for engineering leaders is how they managed the dual-layer architecture. They didn't try to upgrade the entire system at once. They built the new consensus layer alongside the old one, let it run for 18 months, and then flipped a switch. That's a pattern you see in other complex migrations — like how Stripe migrated from a monolith to microservices, or how GitHub moved to a new storage layer. Luna: But Ethereum's case is unique in that the old system was still actively processing billions of dollars in transactions. The stakes were higher. Lucas: Absolutely. And the engineering rigor paid off. The proof of stake system has been running for almost four years now, processing over a million transactions per day on the base layer, with a 12-second block time and 99.9 percent uptime. Validators earn about 3 to 5 percent annualized in ETH rewards — depending on the total amount staked. Luna: And the total value staked is around 30 million ETH. That's over $50 billion at current prices. So the security budget is enormous. Lucas: It is. And it's a testament to the architecture that the system has handled that value with no major incidents. There have been some issues — like the finality stall in May 2023 caused by a bug in Prysm — but they were contained. Luna: So what's the next big architectural challenge for Ethereum now that the Merge is behind them? Lucas: The next major upgrade is sharding — specifically, proto-danksharding, which is part of the Ethereum 2.0 roadmap. It's about scaling the data availability layer so that rollups can post cheaper data. That's a whole other architectural challenge. But the Merge proved that Ethereum can execute a fundamental protocol change without breaking the network. That's a huge vote of confidence for the entire ecosystem. Luna: And for any engineering team facing a similar 'impossible' migration, there's a lot to learn from how Ethereum did it. Lucas: Yeah. Build the new system in parallel, test it relentlessly with shadow forks, coordinate across client teams, and design a clean API between layers. It's not magic — it's just really disciplined engineering.