Want To Reach FI Sooner? Join more than 18,000 others and get new tips and strategies from Can I Retire Yet? every week. Subscription is free. Unsubscribe anytime:

Are taxes going to be your single biggest financial concern in retirement? A ravenous velociraptor stalking your financial security? Many pundits say so. Or, are taxes just another expense, and by no means your largest one?

Despite their scary reputation, I don’t worry about taxes much. I laid out the reasons why — my political and financial analysis of tax rates for middle-income retirees — in an article last year. That post was a bit controversial. Seems it’s not politically correct to be ambivalent about taxes!

Politics notwithstanding, if you’re able to live on less than about $6,000/month or $72,000/year in retirement — as more than 70% of my readers expect to do — then income taxes are unlikely to be a deciding factor in your finances. The U.S. Census Bureau reports, as of 2012, that the median household income was a bit over $51,000 — taxes will be even less of a factor at that level. Below I’ll show why, with examples from our own tax return.

(If you are wealthier, or have unique financial circumstances, then you may pay more in taxes and may need some tax planning to reach your objectives. But I’m going to focus on the common cases for this post.)

Do-It-Yourself Tax Prep

A few weeks ago, in late March, I completed my tax returns for the 2013 tax year. I’m usually early out of the gate in most financial matters, but I’ve tended to leave taxes until late March/early April in recent years. That’s because of the ever-present possibility of receiving an updated 1099 for my investments that would require filing a revised return. Completing tax returns is one of my least favorite annual rites — so I take care not to have to repeat the process!

As has been the case for many years, I used TurboTax Home and Business to prepare my federal and state returns. Every year I reluctantly plod through the somewhat arduous “interview” where TurboTax asks me questions about my tax situation. And I’m once again brought face-to-face with our grotesquely complex tax code and its 1,001 favors for every imaginable interest group. You would think that, as an engineer, I’d prefer to work through the efficient “forms” view, focusing only on the data that I need to enter. But, though I always resolve to try that, most years I cave in and perform the interview, fearful that I’ll miss some new wrinkle in the tax code otherwise.

The root of the problem is that, doing taxes only once a year, I never quite feel confident enough in my mastery of tax law to go it alone with the forms. That’s despite having done my own taxes all but a couple of my approximately 30+ adult earning years. I even keep extensive notes on how to do our taxes, and I update those every year. When mid-April arrives and my taxes are complete, I’m usually feeling highly confident about my ability to navigate the tax code. But by the time the next year rolls around, the keen edge has worn off, and I again gratefully accept TurboTax’s handholding. So why not pay a pro? Because every time I’ve used a tax preparer in the past, I’ve wound up doing just as much work — collecting all the information so they can punch it into their computer, then reviewing the results.

I spend as little time on taxes as humanly possible, though it still feels like too much. Our tax situation has always been relatively simple, and I work to keep it that way. When the choice is between complexity or paying a bit more in taxes, I’ll pay. Shocking to some, I’ll forego modest deductions because the paperwork just isn’t worth my time. For example, I generally never took the home office deduction — even back when the rules were much looser, because the paperwork implications, including depreciation recapture when you sell, were grim. I preferred to invest those hours I would spend understanding and applying tax law to building my business instead.

I always e-file my federal and state returns, using electronic funds transfer for refunds/payments. Even with all that automation and my laser focus on simplicity, it takes me at least a solid 8 hours of work, sometimes more, spread out over several days, to prepare my taxes.

Ugh! But you have your own horror stories. Let’s focus on the bottom line next….

Taxes on Your First $70K

If your lifestyle requires under about $70K in income, and you don’t have any special financial considerations, then here is how your federal income taxes will play out:

For this past tax year — 2013 — the standard deduction for a married couple filing jointly was $12,200, and the personal exemption was $3,900 each. So that makes exactly $20,000 of income, for a couple, which is not taxed at all. You get $20K tax free. That’s enough for baseline food and shelter right there. You can live a frugal RV lifestyle in the U.S. without paying taxes. Though, understandably, most of us want more comfort than that.

Factoring in that first $20K of tax-free income and the two lowest tax brackets of 10% and 15%, retirees with annual incomes of $72,000 — covering a plush $6,000/month in expenses — might expect to pay a maximum of about $575/month in federal taxes, or an effective tax rate of about 9.6%, worst case.

But there are a number of other factors such as itemized deductions, favorable capital gains treatment, and withdrawing from after-tax accounts for some of your income that are likely to reduce that tax bite further, possibly much further.

Consider that any qualified dividends or long-term capital gains that fall into the 15% tax bracket, or below, will not even be taxed! That’s right, there is a 0% long term capital gain tax rate for the first two tax brackets.

An even bigger factor for many early retirees, and anybody with a Roth account, is that you may have substantial after-tax assets to live on. These are savings where you’ve already paid the taxes. So withdrawals, other than new gains in taxable accounts, will be tax free. (Because our own investments are split almost evenly between pre- and post-tax accounts, we could conceivably go for years declaring very little taxable income.)

Unexpected Bonuses

For better or worse, the underlying structure of the U.S. tax code — standard deductions, personal exemptions, and tax brackets — favors those who operate around the median income. And, there are many other goodies in the tax code for middle and lower income earners, some of which I’m still discovering.

With a generous software engineering salary during my working years, I grew accustomed to paying handily into government coffers without much expectation of benefits in return. That high salary moved us well out of range of the grab bag of tax deductions and credits created to benefit those with lower income or higher expenses.

Now, suddenly, having rejoined the middle class, I’m reaping surprise benefits. This year, in particular, we received a couple of unexpected bonuses from the tax code:

The first gift we received this year was thanks to the modest part-time income from this blog. I’m “working” again, but not making too much. So, for the first time in many years, I was able to make a small deductible IRA contribution, that further reduced our taxes. Not a big deal, but a nice benefit for any retiree working part time: You can tax shelter some income, if you wish.

That first benefit led to another surprise: We qualified for the Retirement Savings Contribution Credit. We’re already financially independent, but thanks in part to our modest income in retirement, and the fact that we aren’t needing to take distributions from our retirement plans yet, the government wants to reward us for saving more towards our retirement.

Bottom Line: Our Effective Tax Rate

Opinions will differ about whether we’re all getting our money’s worth from the taxes we pay, but the fact is, in our middle-income range, taxes do not take a large percentage of our income:

When all is said and done, our effective federal gross tax rate (total tax divided by adjusted gross income) for my first two full years of retirement has been just 4.1% each year. When I add in our state taxes, that rate rises to around 6%. Not trivial, but not our largest expense, by a long shot. We spend more than that every month on transportation, food, health care, shelter, or travel, for example.

I’m OK with that effective rate. And I expect it could go even lower in the coming years. That’s because Caroline retired only last year, so we won’t have her taxable income going forward. And I expect us to get savvier about tax planning for living off our mixture of taxable and tax-sheltered investments, plus part-time income, as the years pass.

And, if I’m wrong, and our effective tax rate does go up? If that happens, the most likely cause will be that we are doing better than expected — earning more income. And we can deal with that.

Our Retirement Lifestyle

My conclusion, reached not through political machination but via real-world experience, is this: “If you have to worry about taxes, it’s probably because you’re doing well.”

That’s right. It’s one of those problems that most of the human beings in the world would love to have.

Meanwhile, we enjoy our middle-income retirement lifestyle. We live in a modest two bedroom rental, and drive modest, older vehicles. Other than that, we live like kings. We keep our own schedule. We eat excellent food. We travel when and where we want. We own nice things: With our recent change in location, we splurged on new clothes, bikes, and outdoor gear, and we can’t even see the impact on our net worth.

So that is what a retirement lifestyle in the lower tax brackets can look like. Given quite a bit of flexibility to control our income from various taxable and tax-free sources, I expect the story to be the same or better in future years. And if, instead, our taxes or tax rates go up, it will most likely be because we are even more financially comfortable.

Life is good. I have no complaints. Even with that velociraptor on the loose….