I spent a lot of the first decade of my professional life complaining about income taxes. But I never took time to understand how we are taxed, how much influence we have over our tax burden, and how friendly the tax code is to retirees, particularly early retirees.
One of the first posts I read on this site was Why I Don’t Fret About Taxes. It was eye opening to see how tax friendly early retirement is. I was further encouraged when Darrow shared how low his income taxes were in early retirement in 2014, 2016, and 2018 with minimal effort to optimize them.
My wife and I are taking a phased approach to early/semi-retirement. She cut back to working part time in 2012, five years prior to my retirement. She continues to work in that same role since I retired in 2017.
Making changes gradually gave us the confidence and courage to begin making major life transitions sooner than we otherwise would have. How does it impact our tax situation compared to full early retirement? (Spoiler alert: the answer is in the title.) Let’s dive into the numbers.
The Challenge of Comparison
It was initially my intention to share numbers from our actual tax returns over several periods: a household with two full-time incomes, one full-time and one part-time income, and then one part-time income only.
My goal was to provide an apples-to-apples comparison of taxation in these different scenarios. It quickly became apparent how challenging that would be.
My last year of work was 2017. In 2018, the Tax Cuts and Jobs Act significantly changed the tax code. This law lowered tax rates, increased the standard deduction and eliminated itemized deductions for us and many others.
Our personal situation changed as well. My wife’s income has steadily increased each year. I began working on this blog in 2018.
My writing provides some self-employed income. As such, I am now responsible for paying both the employer and employee portions of Social Security and Medicare taxes that I hadn’t as an employee.
We switched to a high-deductible health plan in 2017 and opened a Health Savings Account (HSA) that offers new tax advantages. Prior to that we’d never been eligible for a HSA.
We rented out our current residence during the second half of 2017 and first half of 2018, financed by a home equity loan with a variable interest rate. After breaking even in 2017, we sustained a substantial loss in 2018 when the interest rate increased. The loss decreased our taxable income that year.
Finally, we moved from Pennsylvania to Utah in July, 2018. Our state income tax rate increased from 3.07% to 4.95%.
Apples to Apples
It’s easier to demonstrate the tax implications of semi-retirement using a scenario based on our numbers while controlling key variables. This allows an apples to apples comparison to demonstrate key principles.
The assumptions about this couple are as follows:
- Filing taxes as married filing jointly with no dependents
- Each individual earns a full-time equivalent of $90,000/year
- Taxable investments create $12,000/year of qualified dividends
- Taxable savings create $1,200/year interest
- Each spouse when working may contribute to a 401(k)
- One of the spouse’s employers provide health insurance as a benefit
- The couple is not eligible to contribute to an HSA
- Taxes are based on the 2020 tax code
We’ll address the following tax implications in a separate discussion below:
- Filing taxes as a single person or someone with dependents
- Using Traditional and Roth IRAs
- Social Security and Medicare taxes
- State income taxes
- Tax implications of purchasing medical insurance in early/semi-retirement
I double checked tax calculations against the Federal Income Tax Calculator from Go Curry Cracker!.
Federal Taxes for a Two Income Household
In our first scenario, both spouses are working full-time. Total earned income is $180,000. Total taxable income with investments is $193,200.
The maximum contribution to a 401(k) account is $19,500 per person in 2020. Since both partners are working, the couple’s total contribution is $39,000 which lowers adjusted gross income (AGI) to $154,200.
You would subtract the standard deduction of $24,800 to arrive at a taxable income of $129,400. Subtracting the $12,000 of qualified dividends gives $117,400 of income which is taxable at ordinary income tax rates.
The first $19,750 is taxed at a rate of 10% for $1,975. The next $60,500 is taxed at 12% for $7,260. The final $37,150 is taxed at a rate of 22% for $8,173. The total is $17,408.
Qualified dividends are taxed at a rate of 15% for married couples earning between $80,000 and $496,600. The dividends generate an additional $1,800 of taxes.
The total federal income tax paid by this couple is $19,208 on $193,200 total income. Their effective tax rate is 9.9%.
The Impact of Cutting Back Work
The second scenario reflects the impact of one earner cutting back hours. After the birth of our daughter, my wife cut back to 30 hours/week.
Cutting back one spouse’s earnings by 25% reduces the household pre-tax income by 12.5%. But that income would have been taxed at the highest marginal tax rate, so what does this do to after tax income?
The couple’s total earned income in this scenario is reduced to $157,500. Total income with investments is $170,700. Subtracting $39,000 for 401(k) contributions lowers AGI to $131,700.
Subtracting the standard deduction of $24,800 leaves $106,900 of taxable income. Subtract the $12,000 qualified dividends. This leaves $94,900 taxed as ordinary income.
The first $19,750 is again taxed at 10% for $1,975. Then next $60,500 is taxed at 12% for $7,260. Only $14,650 is now taxed at their marginal rate of 22% for $3,223. The total is $12,458.
The qualified dividends are again taxed at 15% for an additional $1,800 of taxes owed.
Their total federal income tax owed is $14,258 on $170,700. Their effective tax rate drops to 8.3%.
Working a little less allows this couple to keep about two more cents out of every dollar they earn.
The Impact of One Spouse Retiring
The third scenario reflects the impact of the full-time spouse retiring fully. This is what I did when I left my career in 2017. When married filing jointly, you can obtain a similar outcome if each partner worked 15 hours/week.
The couple reduced their earned income by 62.5% from the first scenario. They pay tax on the remaining income at the most favorable rates.
The total earned income in this scenario is $67,500. Adding in investment income brings total income to $80,700. Since only one spouse is working, they can contribute $19,500 to a 401(k), lowering AGI to $61,200.
Subtracting the standard deduction of $24,800 leaves $36,400 taxable income. Subtract the $12,000 qualified dividends. This leaves $24,400 taxed as ordinary income.
The first $19,750 is again taxed at 10% for $1,975. Only $4650 is taxed at the lower marginal rate of 12% for $558. The total is $2,533.
Qualified dividends are taxed at a rate of 0% for couples married filing jointly if taxable income falls below $80,000. Thus no tax is owed on this $12,000 of investment income.
The total federal income tax in this semi-retired scenario is $2,533 on $80,700 of total income. The effective tax rate is 3.1%.
The Tax Code is Much Friendlier to Low Earners
Semi-retirement allows you to earn a substantial amount of income and pay little in federal income taxes. You keep about seven cents more of every dollar you earn when compared to working full-time, everything else equal.
Semi-retirement also provides more time for your investments to grow. During this time, you eliminate tax drag on qualified dividends and long term capital gains.
The 0% long-term capital gains rate provides the opportunity to harvest capital gains, meaning more of your future income could be tax free with good planning. It also gives the opportunity for creative planning, using taxable investment accounts to produce tax free income to meet current spending needs while directing new earned income into tax advantaged accounts.
Getting Into The Weeds
Creating this apples to apples scenario is great for comparison and demonstration. But life can sometimes be messier.
Let’s address some challenges as well as missed planning opportunities that were overlooked in this oversimplified scenario.
What About Singles and Kids?
When I wrote a post with similar tax planning scenarios in the past, a reader asked why I don’t include info for single folks as well.
Taxes are complicated. I try to simplify things as much as possible by creating a single scenario in an attempt to avoid confusion.
The principles I outlined are universal. If you want to create your own scenario, here are the tax rates and income cut offs for 2020 summarized concisely. And here is a link to the free and simple income tax calculator I used to double check my numbers.
The other reason I tend to do these scenarios with a married couple filing jointly is… frankly because I’m lazy. As previously noted, this is based loosely on our own planning so it provides me familiarity.
However, I did make the hypothetical couple childless. That’s because kids don’t affect the calculations. They simply provide a $2,000 per child tax credit.
If you have a dependent child or children, just multiply $2,000 by the number of children you have. Then subtract that number from the tax you calculated to get your final tax cost.
What About IRAs and Roth IRAs?
I didn’t include IRAs and Roth IRAs because they also complicate the scenario. The ability to contribute to Roth IRA accounts and the ability to deduct traditional IRA accounts are dependent on your income in a given year. I couldn’t keep these variables constant for a fair apples to apples comparison.
Also, tax deductions are always more valuable when in the highest marginal tax rates. Roth IRAs are always more valuable for those with low enough income that they don’t pay federal income tax. In between, there is a gray area that is highly dependent on your personal situation.
Working and earning less might provide more options to utilize these accounts to your benefit. Deductible contributions can be tremendously valuable for someone looking to lower taxable income to optimize for ACA subsidies (more on this below).
A benefit we experienced was being able to fully contribute to Roth IRA accounts when decreasing our income as in the second scenario. Previously we had too much income to contribute.
Someone who could not deduct an IRA when working full-time may benefit by being able to make a deductible contribution by earning less in semi-retirement.
Another planning opportunity for a married couple is to contribute to each spouse’s IRA, even if only one has earned income. This could allow the couple to defer more taxation with a traditional IRA or have more tax free investments by contributing to a Roth depending on which made more sense for their situation.
What About Social Security and Medicare Taxes?
Federal income taxes are not the only tax that we pay. Federal Income Contributions Act (FICA) taxes are 15.3%. Employers withhold them from our paychecks.
The social security portion is 12.4%. Employees pay half (6.2%) and employers pay the other half. Self-employed individuals pay both portions, the full 12.4%.
Medicare tax makes up the other 2.9%. Similarly this is split evenly between employees and employers. The self-employed pay both portions.
Unlike payroll tax which is progressive (as you earn more, you pay a higher rate of taxation), social security is regressive. You pay a flat tax on every dollar of income up to an income cap of $132,900 in 2020. Medicare tax is a flat tax, applied from your first dollar of income with no income cap.
This tax picture may not seem so rosy. However, unlike income taxes, which are a straight expense, Social Security Taxes provide a direct return to you in the form of benefits paid in retirement.
According to ssa.gov, “Social Security calculates your average indexed monthly earning during the 35 years in which you earned the most.” You benefit by paying more into Social Security.
Consider the following scenarios for someone contemplating early retirement with only 20 years paid into Social Security.
- Scenario 1: Work two more years earning $90,000/year, then retire 4 years sooner.
- Scenario 2: Cut down to ⅓ time work now. Earn $30,000/year in semi-retirement for 6 years.
In either scenario, you would have paid Social Security taxes on $180,000 of income over six years, for an average of $30,000/year. However, in the second scenario you would have earned the $180,000 much more tax efficiently from the perspective of federal income taxes while potentially achieving a better life balance years sooner.
What About State Taxes?
I refrained from writing about state income taxes, again because of my desire for simplicity and clarity in these scenarios. State income taxes vary considerably. Seven states don’t tax income, eleven more (including the two where we’ve lived) have flat income tax rates, and the remainder have progressive tax structures.
Within that framework, rates vary widely as does policy on what type of income is taxed. It is best to first learn how your state taxes income. Then decide whether it’s even worth your time to develop a strategy for state income taxes. If so, develop one based on your circumstances.
States with progressive taxes may provide an opportunity to leverage the federal income tax strategies. States with a flat tax would be similar to thinking about FICA taxes.
If thinking about relocating in retirement, you could consider opportunities for domestic geoarbitrage. However, tax planning is only one consideration and in my opinion far from the most important.
What About Healthcare?
Before I dove into the numbers on Affordable Care Act (ACA) Subsidies, I thought the biggest drawback to semi-retirement was that having earned income would make health insurance unaffordable.
For this final point of discussion, I’ll share actual numbers from our 2019 tax return to challenge that assumption. Our MAGI last year was $73,000.
Plugging that number into the Kaiser Family Foundation Health Insurance Marketplace Calculator with our specific geographic location and family demographics (two adults and one child) reveals that we would receive a subsidy of $725/month and pay $595/month for a silver plan through the ACA.
This is more than we pay to purchase insurance through my wife’s employer. But it is still reasonable.
Because we weren’t buying insurance through the ACA, we did nothing to optimize our MAGI in 2019. I’ve outlined our current tax planning strategy. We could have contributed about $15,000 more to my wife’s 401(k) and contributed $12,000 to traditional rather than Roth IRAs.
Lowering our MAGI by $27,000, from $73,000 to $46,000, would drastically increase our subsidy and lower our premium costs. Running the numbers with the lower MAGI shows we would pay only $270/month while receiving $1,050/month in subsidies.
***A reader, Mister DS, pointed out an error in the comments. Deductions for traditional IRA are added back to AGI when calculating MAGI. Thus the correct calculation would be based on a MAGI of $58,000. Insurance premiums would cost $425/month with a subsidy of $890/month.
We could then meet our spending needs by selling off investments with long-term capital gains which would be taxed at 0%. The capital gains do count toward MAGI.
For this reason, we’re working to harvest capital gains to increase tax basis now while our income is relatively low, but we aren’t yet relying on subsidies to keep insurance affordable. When we sell shares later to fund lifestyle they will be mostly cost basis, which is not taxable income, and less capital gains.
When choosing your own health insurance, you also have the ability to choose a HSA compliant high deductible plan and contribute to an HSA. Choosing to do so gives more control over your MAGI and thus your insurance premiums.
Semi-Retirement Is Very Tax Efficient
Making the jump from full-time work and a high savings rate to full retirement and drawing down assets is intimidating for many. We were among those afraid to plunge fully into early retirement.
Semi-retirement can feel a lot less risky and allow you to dip your toes into the water sooner than you may otherwise feel comfortable. I’m happy to share that this path offers many of the same tax benefits as full early retirement.
* * *
* * *
[Chris Mamula used principles of traditional retirement planning, combined with creative lifestyle design, to retire from a career as a physical therapist at age 41. After poor experiences with the financial industry early in his professional life, he educated himself on investing and tax planning. Now he draws on his experience to write about wealth building, DIY investing, financial planning, early retirement, and lifestyle design at Can I Retire Yet? Chris has been featured on MarketWatch, Morningstar, U.S. News & World Report, and Business Insider. He is also the primary author of the book Choose FI: Your Blueprint to Financial Independence. You can reach him at firstname.lastname@example.org.]
* * *
Disclosure: Can I Retire Yet? has partnered with CardRatings for our coverage of credit card products. Can I Retire Yet? and CardRatings may receive a commission from card issuers. Other links on this site, like the Amazon, NewRetirement, Pralana, and Personal Capital links are also affiliate links. As an affiliate we earn from qualifying purchases. If you click on one of these links and buy from the affiliated company, then we receive some compensation. The income helps to keep this blog going. Affiliate links do not increase your cost, and we only use them for products or services that we're familiar with and that we feel may deliver value to you. By contrast, we have limited control over most of the display ads on this site. Though we do attempt to block objectionable content. Buyer beware.