Latest / The Financial Freedom Podcast with Fexingo: Quitting Your Job, Living Off Investments, Independence / The Portfolio Tax Audit That Saves Early Retirees Thousands
Transcript
- Lucas: Let me start with a number: eighteen thousand dollars. That is roughly how much a typical early retiree with a one-point-two-million-dollar portfolio is giving away to the IRS every single year — not because of high income, but because of something called portfolio tax drag. Luna: Tax drag — that is the silent leak most people don't even realize they have. We obsess over expense ratios and withdrawal rates, but where we hold our assets can cost us more than both combined. Lucas: Exactly. And today I want to walk through a real scenario. Imagine you are forty-five, you left your job three years ago, and your portfolio is split roughly equally between a taxable brokerage account, a traditional IRA, and a Roth IRA. Your bonds — say twenty percent of the portfolio — are sitting in the taxable account because that is where you had room when you first built it. Luna: That is incredibly common. And the problem is, bonds throw off interest and dividends that are taxed as ordinary income — at your marginal rate. In early retirement, that rate might be twelve percent or twenty-two percent, but it is still a drag you could avoid. Lucas: Right. So let's run the math. On two hundred and forty thousand dollars in bonds yielding four percent, that is nine thousand six hundred dollars in interest each year. At a twenty-two percent marginal rate, that is over two thousand dollars in taxes — just on the bonds. If those bonds were in the traditional IRA, they would grow tax-deferred, and you would only pay income tax when you withdraw. But the real killer is the interaction with your stock dividends. Luna: Because stocks in the taxable account also throw off dividends — qualified dividends that get the lower capital gains rate. But if your bond interest pushes your total income above the threshold for the zero percent capital gains bracket, those qualified dividends suddenly get taxed. Lucas: Bingo. In 2026, the zero percent long-term capital gains bracket goes up to about forty-seven thousand dollars for a single filer. With nine thousand six hundred in bond interest, plus maybe fifteen thousand in qualified dividends, you are already past that threshold. So now you are paying fifteen percent on those dividends — another two thousand two hundred fifty dollars. Add it up, and the tax drag from just that bond-location mistake is over four thousand dollars a year. Luna: And that is before we even talk about the lost compounding. Over twenty years, at a six percent return, that four thousand dollars a year not invested could grow to nearly a hundred and fifty thousand dollars. That is real money. Lucas: So the fix is straightforward: swap the location. Sell the bonds in the taxable account, buy stocks there instead. And in the traditional IRA, sell an equivalent amount of stocks and buy bonds. No net change in your overall allocation — but now the bond interest is tax-deferred, and your taxable account holds stocks that generate mostly qualified dividends. Luna: But you have to watch out for capital gains taxes when you sell those bonds in the taxable account. If the bonds have appreciated, you could trigger a tax bill. Lucas: That is a real concern. However, in the current rate environment — June 2026 — many bonds are actually trading below par because rates have risen. So you might realize a capital loss, which you can use to offset other gains or even deduct up to three thousand dollars against ordinary income. That is a feature, not a bug. Luna: And if you do have gains, you can spread the swap over multiple years, staying within the zero percent capital gains bracket. It is a multi-year project, but the payoff is huge. Lucas: Exactly. But there is another layer here that gets overlooked: the pro-rata rule for Roth conversions. A lot of early retirees do Roth conversions to fill up the lower tax brackets. But if you have a traditional IRA with pre-tax money, any conversion is taxed proportionally based on your total IRA balance — including any after-tax money you might have. That can mess up your plans. Luna: Right. So if you have a mix of pre-tax and after-tax dollars in your IRAs, the IRS treats every conversion as a blend. You cannot just convert the after-tax portion. That is the pro-rata rule. Lucas: And this is where the tax-drag story gets even more interesting. Many early retirees think they can do a 'backdoor Roth' every year, but if they have a large traditional IRA balance from a previous 401 rollover, the pro-rata rule makes that conversion mostly taxable. The solution? Roll the pre-tax IRA into a current employer's 401 if allowed, or just accept the tax hit on a small conversion. But the key is knowing the rule before you act. Luna: And that brings us to the bigger point: tax efficiency in early retirement is not just about minimizing current taxes — it is about maximizing the amount of money that stays invested and compounds over decades. A one-time portfolio audit can save you more than any hot stock tip. Lucas: And speaking of audits — I have to say, one thing that keeps our show going and ad-free is the support from listeners. If these money conversations have ever helped you think about a decision differently, a couple of dollars a month genuinely makes a difference. You can find us at buy me a coffee dot com slash fexingo. Luna: Yeah, it is such a small thing, but it means we can keep digging into these topics without sponsors or ads. We really appreciate everyone who chips in. Lucas: So back to the portfolio audit — the other big area is tax-loss harvesting. In a taxable account, if you have positions that are down, you can sell them, realize the loss, and use it to offset gains or income. But there is a wash-sale rule: you cannot buy a substantially identical security within thirty days before or after the sale. Luna: Right. So if you sell an S&P 500 ETF at a loss, you cannot buy the same ETF back within thirty days. But you can buy a different one — say a total market ETF or a large-cap value ETF — to stay invested while harvesting the loss. Lucas: Exactly. And in early retirement, when your income is low, those losses can be especially valuable because they offset gains that would otherwise be taxed at the zero percent rate — but any unused losses carry forward indefinitely. So you are building a tax asset for future years. Luna: It is like having a rainy-day fund for taxes. And it works best when markets are volatile, which — let's be honest — is often. Lucas: Alright, let me wrap with one concrete action item. If you are retired or planning to retire early, spend one hour this month mapping out where every asset is held. Ask yourself: are my bonds in tax-advantaged accounts? Are my international stocks in taxable to claim the foreign tax credit? Am I doing Roth conversions up to the top of the zero percent bracket? That one hour could be the highest-return activity you do all year. Luna: And if you want to go deeper, the IRS publication 550 on investment income and expenses is actually readable — surprisingly. It covers wash sales, capital gains, dividends, everything we talked about. Lucas: Great suggestion. For next time, I want to dig into the hidden costs of early retirement healthcare — not the premiums, but the tax interplay with health savings accounts and premium tax credits. That is a whole other layer. Luna: I am already curious about that one. See you next episode.