Latest / The Financial Freedom Podcast with Fexingo: Quitting Your Job, Living Off Investments, Independence / Managing a Mortgage in Early Retirement
Transcript
- Lucas: You've hit your FIRE number. You're ready to hand in your notice. But there's this mortgage payment that keeps showing up every month. Should you pay off the house before you quit, or carry that debt into early retirement? Luna: It's one of those decisions where the right answer depends a lot on your personal situation. And the math isn't always intuitive. Lucas: Right. So let's make it concrete. Imagine a couple, let's call them Jen and Carlos. They're 45, they've got a $1.2 million portfolio, and a $200,000 mortgage at 3.5% with 20 years left. Their monthly payment is about $1,160. They're aiming for a $48,000 annual withdrawal — that's 4% of their portfolio. Luna: So the mortgage payment alone eats up about 29% of that withdrawal. That's a big chunk. Lucas: Exactly. And it creates what we call a fixed expense that doesn't go away. If the market drops in the first few years — sequence of returns risk — that mortgage payment still has to be made. So one option is to pay off the mortgage before retiring, using $200,000 from the portfolio. Luna: But then their portfolio drops to $1 million, and their 4% withdrawal becomes $40,000. With no mortgage, they only need to cover other expenses. If those other expenses are, say, $36,000 a year, they're fine. But if they're $45,000, now they have a shortfall. Lucas: That's the trade-off. Paying off the mortgage reduces your portfolio, but also reduces your required withdrawal. In Jen and Carlos's case, their non-mortgage expenses are $34,000. So with the mortgage, they need $34,000 plus $14,000 in mortgage payments — $48,000 total. Without the mortgage, they only need $34,000. So a $1 million portfolio at 4% gives $40,000. That covers $34,000 with a $6,000 buffer. Luna: So mathematically, paying off the mortgage actually improves their margin. They go from a $48,000 need on $1.2 million to a $34,000 need on $1 million. The withdrawal rate drops from 4% to 3.4%. Lucas: And that's significant. A lower withdrawal rate drastically reduces sequence risk. But there's a catch: they lose liquidity. That $200,000 is tied up in home equity. If an emergency comes up — a big medical bill, a new roof — they might have to tap the portfolio at a bad time or take out a home equity loan. Luna: Plus, mortgage interest rates matter. At 3.5%, that's cheap debt. Historically, the stock market has returned more than that. So from a pure return perspective, investing the $200,000 and keeping the mortgage could come out ahead. Lucas: Yes, but that's an expected return argument. The market doesn't deliver 10% every year. In early retirement, you're more vulnerable to volatility. So the question becomes: do you prioritize minimizing sequence risk or maximizing expected growth? Luna: And there's also the tax angle. Mortgage interest is deductible if you itemize, but with the higher standard deduction, many retirees don't benefit. For Jen and Carlos, their mortgage interest is about $7,000 a year. If they take the standard deduction of $30,000 for a married couple, that interest doesn't help. Lucas: Right, so the tax benefit is minimal. On the other hand, if they keep the mortgage, they might have more room for Roth conversions in early retirement. Because they need more cash to cover the payment, they might withdraw more from taxable accounts, which can be managed to stay in a low bracket. Luna: That's a good point. Let's say they keep the mortgage and withdraw $48,000. If $20,000 of that comes from dividends and capital gains, and $28,000 from selling appreciated shares, their taxable income might be around $30,000 after deductions. That's well within the 12% bracket. They could even do a small Roth conversion. Lucas: But if they pay off the mortgage, they only need $34,000. That's even less income, so even more room for conversions. Actually, paying off the mortgage lowers their income needs, which is good for Roth conversion space. So the tax argument isn't clear-cut. Luna: I think the emotional side matters too. Some people sleep better knowing they own their home free and clear. That peace of mind has value, even if the numbers don't favor it. Lucas: Absolutely. And I think the right framework is: look at your withdrawal rate both with and without the mortgage. If carrying the mortgage pushes your withdrawal rate above 4.5% or so, you might want to pay it off. If it stays under 4%, you can probably keep it. Luna: And consider the interest rate. If your mortgage rate is above 5%, paying it off becomes more attractive. Below 3%, it's cheap money. Lucas: Another strategy: use a bond tent or a cash reserve to cover the mortgage payments for the first five to ten years. That way you don't have to sell stocks in a downturn. You essentially pre-fund the payments. Luna: So you set aside, say, $70,000 in cash or short-term bonds to cover five years of mortgage payments. That reduces sequence risk without fully paying off the loan. Lucas: Exactly. And that cash reserve comes from the portfolio, so your total withdrawal rate calculation changes. But it gives you flexibility. You can always decide later to pay off the mortgage if rates spike or if you want to simplify. Luna: What about refinancing? Some early retirees consider a cash-out refinance to invest the proceeds, but that's risky. Lucas: I'd generally advise against that in early retirement. You're increasing your debt and your payment, which adds sequence risk. Better to either pay it down or keep it as is. Luna: So for Jen and Carlos, what's the verdict? Lucas: In their specific case — 3.5% rate, 20 years left, $1.2 million portfolio — I think paying it off makes sense. It drops their withdrawal rate to 3.4%, which is very safe. They lose some liquidity, but they can rebuild that by diverting the freed-up cash flow into savings for the first few years. Luna: They could also consider a partial paydown. Maybe put $100,000 toward the mortgage and recast it to lower the payment, keeping $1.1 million invested. Lucas: That's a great middle ground. A recast reduces the monthly payment but keeps the loan term. That could lower their required withdrawal without sacrificing all liquidity. Luna: And if they have a high risk tolerance, they might just keep the mortgage and invest the $200,000. Historically that would win, but it's not guaranteed. Lucas: I think the key takeaway is to run your own numbers. Look at your withdrawal rate in both scenarios. Factor in taxes, liquidity, and your personal comfort. There's no one-size-fits-all answer. Luna: And if you're not sure, you can always do a dry run. Live on your expected retirement budget for six months while still working. That includes the mortgage payment. See how it feels. Lucas: Great advice. And speaking of running numbers, if these conversations about withdrawal strategies and mortgage decisions have helped you think about your own plan, consider supporting the show. It keeps us ad-free and independent. Luna: Yeah, listener support really makes a difference for a show like ours. And it's easy. Lucas: You can find us at buy me a coffee dot com slash fexingo. No pressure, but if you find value here, it's a great way to give back. Luna: Alright, so back to mortgages. One last thing: if you're considering paying off your mortgage, don't forget about potential prepayment penalties. And make sure you have an emergency fund left. Lucas: Solid point. The decision is personal, but the math gives you a clear starting point. For Jen and Carlos, paying off the mortgage gives a safer withdrawal rate. For others with a lower rate or larger portfolio, keeping it might be fine. Luna: And as always, talk to a fee-only financial planner who understands early retirement. They can model the scenarios for you. Lucas: Good note to end on. Thanks, Luna. And thanks for listening, everyone. We'll see you next time.