The Ghosts of Investments Past

Want To Reach FI Sooner? Join more than 18,000 others and get new tips and strategies from Can I Retire Yet? every week. Subscription is free. Unsubscribe anytime:

“I told you these were shadows of the things that have been.That they are what they are, do not blame me!”  — The Ghost of Christmas Past from Charles Dickens’ “A Christmas Carol”.

Ghost of past investment mistakesI frequently hear from readers haunted by past investments. Many of you are trying to build an investment portfolio that meets your current goals, needs and ideals while trying to figure out what to do with these remnants of the past. 

After reading our blog and others like it, you realize that a simple, low-cost, tax-efficient approach to investing is the best approach for most people. Choosing a few index funds, or maybe a single target date fund, combined with an appropriate savings rate will get most readers to the secure retirement numbers they desire.

Unfortunately, for many that realization comes after years or even decades of investing. Many of us have a hodge-podge of investments accumulated over time. We were sold products by financial advisors, tried our hands at stock picking schemes, or threw darts at the myriad selections in our 401(k).

So what do we do with these ghosts of investments past? I dealt with this problem when unwinding my past investment mistakes before I could move forward. Trying to deal with these mistakes can feel overwhelming.

Specific actions will vary depending on your circumstances. But, there is a universal process that we can all follow to make the best decisions going forward. It will enable action and help avoid expensive errors as we build the portfolio we desire while unwinding past mistakes.

Step 1: Stop Investing

My goal when writing is to simplify complex topics and inspire action. In this case, your actions have created the situation you’re in. 

So the first step is to stop investing. There are several reasons for this if you are planning to change your investment approach.

The first reason is risk. If you are investing in a volatile asset that you may want to sell in the short term, there is substantial risk that your investment could drop in value in the interim.

I made this mistake when I put the cart before the horse and impulsively started investing large amounts into my 401(k) before developing a comprehensive investing plan, only to then reallocate the money into different investments a few months later.

The second reason to stop investment contributions applies if you are investing in taxable accounts. Short-term capital gains are taxed at a less favorable rate than long-term capital gains

It is therefore generally wise to hold investments with substantial gains until they receive more favorable tax treatment. Not adding to these investments stops the cycle of creating new short-term gains that complicate the process of selling old investments.

Make sure you are not unintentionally adding more new money to these investments. It is generally good advice to automatically reinvest your dividends and interest. In this case, it is wise to shut off this feature so you are not adding to investments you no longer want. Contribute only to investments you plan to keep long term.

Step 1A: But Keep Saving!

To be clear, you should continue contributing to your investment accounts. This is particularly true if stopping contributions means you would miss out on free money from employer matches to retirement or health savings accounts. And be careful not to miss the opportunity to contribute to tax advantaged accounts for a particular year.

You can automate contributions into high-yield savings accounts, money market or short term bond funds where your money can earn interest for the short time until you determine where you want to direct future contributions for the long term. 

This allows you to maintain the habit of automating savings while avoiding short-term market risk.

Step 2: Develop a Plan

Once you have stopped throwing good money after bad, it’s time to establish a plan to guide you moving forward. I wrote out an investment policy statement. Taking this action was incredibly helpful to start and stay the course when managing our investments.

It’s easy to get caught in paralysis by analysis. Set a deadline to finish your plan and start implementing it. This should be a month to several months, not years!

I’ve reviewed the four resources that have had the most impact on my investment philosophy:

After developing a basic plan for controlling fees, taxes and behavior, the most important decision to make is determining an asset allocation appropriate for you needs and risk tolerance. Once you know where you want to be going forward, you will need to determine how old investments fit into this portfolio.

My wife and I started with a sizable portfolio of actively managed mutual funds split between taxable, tax-deferred and Roth accounts as well as inside of a variable annuity. We’d have liked to have gotten rid of all of those funds and start over, but considerations that I’ll outline below meant that it made sense to make this transition slowly.

We looked up each of the funds we held and saw where they fit in the Morningstar Style Box. We then considered the old funds as part of our target allocation in the most closely correlated asset class. Gradually, we sold them and bought the actual funds we wanted in that asset class.

Step 3: Start Making New Contributions

Once you see where you are and where you want to be, start buying new investments in alignment with your investment plans. You can start by buying assets that are short of your desired allocation.

Any cash you’ve saved while figuring out your investment plan can now be allocated. As new money becomes available, you can dollar cost average into the asset classes.

Dealing With The Ghosts

Emotions can run high when you consider your past investment mistakes. I experienced anger at the poor advice I received and regret over the opportunity costs of my mistakes. Learning from those mistakes is part of the price you pay to invest. Accept it and move on.

Emotions and investing can be a lethal combination. It is crucial to separate emotion from your decisions to avoid acting rashly and making more mistakes. If this is difficult or impossible for you, hiring a financial advisor may be beneficial.

We decided to manage our own investments. We developed a plan to address our past mistakes in a way that allowed us to move on systematically without incurring unnecessary fees or taxes that would worsen our situation.

Step 4: Address Tax Advantaged Accounts

Start rebuilding your investment portfolio in tax advantaged accounts. You avoid taxation in the year you make a contribution and pay the taxes when the funds are withdrawn with tax deferred accounts. With Roth accounts, you pay income taxes up front, and you never pay taxes again.

In either case, you don’t pay taxes on capital gains on a year to year basis. This means you can sell any investments held inside these accounts without worrying about tax consequences.

You do want to check that your specific account doesn’t have rules that limit trading. Check that the investments inside the accounts are not held within a structure like an annuity that may come with surrender charges.

You can roll over retirement accounts from former employers to an IRA at a brokerage of your choice. If you’re still working, you may also have the option to roll them into your current retirement plan. Either provide options to control fees and simplify your portfolio by consolidating it.

Assuming you hold simple investments with no onerous trading or surrender fees, you can sell everything inside a tax advantaged account at once without worrying about tax consequences. You can then immediately re-invest the money according to your desired asset allocation and move on with your life.

Step 5: Address Taxable Accounts

Switching investments in taxable accounts is more complex. When you sell investments in a taxable account, it creates a taxable event.

Step 5A: Understand Capital Gains Taxes

You will owe taxes on any increase in value over the initial investment amount. For example, assume you invest $100 and your investment is now worth $150. You will owe taxes on the the $50 capital gain if you sell the investment. You owe no tax on the $100 you invested, known as the cost basis.

It is generally unwise to make investment decisions solely based on tax consequences. You want to make money on investments, which results in taxation of those gains. However, you do want to be aware of the tax consequences of your investment decisions.

If you haven’t kept great records of your investments, the brokerage where you hold your investments should provide information as to how much gain or loss your investments have accumulated. They should also be able to tell whether these are short or long-term gains or losses. See the screenshot below as an example from one particular fund held at Vanguard.

Example of capital gains displayed in a brokerage account

Step 5B: Understand the Impacts of Creating Capital Gains

There are a few things to consider before selling investments in a taxable account:

  1. Understand the difference between the taxation of short-term and long-term capital gains. In many cases it is wise to hold an investment for at least a year to avoid paying taxes at the higher short-term rate.
  2. Understand that selling taxable investments may create taxable income. This income may affect other areas of your tax planning. A few examples:
  3. The long-term capital gains rate is 0% for those in the lowest tax brackets. The income produced by selling off investments could push your long-term capital gains rate to 15%. For those nearing retirement, it may make sense to hold onto investments you don’t want long term until you can gradually sell them without a tax hit once your income is lower.

Consider whether it makes sense to sell off your taxable investments after looking at your situation holistically.

Step 5C: Selling Taxable Investments

If it makes sense to proceed with selling off taxable investments, you need to develop a plan. Plans will vary depending on individual circumstances.

When dealing with my own past investments, the strategy was to start selling old actively managed mutual funds. I had competing goals. My first goal was to eliminate the high fees and unnecessary and unpredictable taxable income the actively managed funds produced, and to do so as quickly as possible. My second goal was to avoid creating too much income by selling off the old investments so that my wife and I could fully contribute to our Roth IRAs each year.

I started by looking for holdings that had gone down in value and sold those first. This allowed me to harvest those losses. I then sold off the largest holdings with the smallest gains until my gains matched my losses. This allowed me to eliminate a substantial portion of my old taxable investments without any tax consequences.

Going forward, at the end of each year I considered our household income and then sold off as much of our old investments as possible without the capital gains pushing us above the Roth IRA income threshold. 

It took three years  to completely sell off the taxable portion of our old investment portfolio. I’ve documented the details here.

Special Circumstances

What I’ve written to this point is true for simple investments including individual stocks, bonds or mutual funds. Unfortunately, many of us have to deal with even scarier ghosts in our investment pasts. There is great incentive for the financial industry to push complex and expensive financial products including whole life insurance, equity indexed annuities and variable annuities. 

I unfortunately became somewhat of an expert on variable annuities. My wife and I realized that about a quarter of our portfolio was tied up in one. So we spent a lot of time determining the best way to get out of it. I published our step by step process when I was writing for Doughroller.

Thankfully, I have no direct expertise with other products. Several lessons from my experience with the variable annuity can be applied to these other investment products you may not want to keep.

  • These products are complicated. If you don’t have the time or stomach to deal with them, it may be worth your while to pay a fee-only, fiduciary financial advisor to help you figure out your options and avoid getting stuck.
  • Before paying an advisor, go in with realistic expectations. 
    • Insurance companies produce these products with teams of lawyers. “Advisors” operating under suitability rather than fiduciary standards sell the products. Don’t expect to find a loophole that will allow you to escape the contract without penalty or sue the person who sold it to you.
    • Teams of actuaries design these products to assure the insurance companies will not lose money. To some degree you’re going to have to take your medicine. You’ll likely either pay high fees if you keep the product or a surrender fee if you sell it.

Take Action

Dealing with past investment mistakes can cause emotional distress. It also can be technically complicated for those unfamiliar with the rules that govern different investment accounts and products.

All of this can feel overwhelming. I understand. I’ve been there. Recognize that you can’t change the past, but you can take control over your future. 

Start today. Leave your investing ghosts in the past where they belong.

* * *

Valuable Resources

  • The Best Retirement Calculators can help you perform detailed retirement simulations including modeling withdrawal strategies, federal and state income taxes, healthcare expenses, and more. Can I Retire Yet? partners with two of the best.
  • Free Travel or Cash Back with credit card rewards and sign up bonuses.
  • Monitor Your Investment Portfolio
    • Sign up for a free Empower account to gain access to track your asset allocation, investment performance, individual account balances, net worth, cash flow, and investment expenses.
  • Our Books

* * *

[Chris Mamula used principles of traditional retirement planning, combined with creative lifestyle design, to retire from a career as a physical therapist at age 41. After poor experiences with the financial industry early in his professional life, he educated himself on investing and tax planning. After achieving financial independence, Chris began writing about wealth building, DIY investing, financial planning, early retirement, and lifestyle design at Can I Retire Yet? He is also the primary author of the book Choose FI: Your Blueprint to Financial Independence. Chris also does financial planning with individuals and couples at Abundo Wealth, a low-cost, advice-only financial planning firm with the mission of making quality financial advice available to populations for whom it was previously inaccessible. Chris has been featured on MarketWatch, Morningstar, U.S. News & World Report, and Business Insider. He has spoken at events including the Bogleheads and the American Institute of Certified Public Accountants annual conferences. Blog inquiries can be sent to Financial planning inquiries can be sent to]

* * *

Disclosure: Can I Retire Yet? has partnered with CardRatings for our coverage of credit card products. Can I Retire Yet? and CardRatings may receive a commission from card issuers. Some or all of the card offers that appear on the website are from advertisers. Compensation may impact on how and where card products appear on the site. The site does not include all card companies or all available card offers. Other links on this site, like the Amazon, NewRetirement, Pralana, and Personal Capital links are also affiliate links. As an affiliate we earn from qualifying purchases. If you click on one of these links and buy from the affiliated company, then we receive some compensation. The income helps to keep this blog going. Affiliate links do not increase your cost, and we only use them for products or services that we're familiar with and that we feel may deliver value to you. By contrast, we have limited control over most of the display ads on this site. Though we do attempt to block objectionable content. Buyer beware.


  1. One thing I did last year to take control was to start doing my own taxes. When I had a business and two multi-family rentals, one out of state, I used an accountant and didnt bother trying to learn the tax code. Now with just one property and a better understanding of the tax code I feel completely in control doing my own. With tax software such as TurboTax I can now run scenarios throughout the year and make more informed decisions. In my situation, I need to put my Income where it needs to be to avoid 15% capital gains and qualify for an ACA subsidy at the same time ; I do that by doing improvements at the rental property, creating deductions and by careful selling of stocks, minimizing capital gains as much as possible.

    1. I agree Mitch. Last year we used an accountant b/c we felt overwhelmed b/t using our home as a rental for half the year, living half the year in two different states, and having some income from my writing for the first time rather than having a regular salary from a job. We ended up having to figure out things and catch a lot of mistakes anyway after the fact, and we missed out on planning opportunities b/c we were detached from things assuming we could hand it off to the accountant.

  2. For those old enough, there is another income cliff in addition to ACA, Roth and Social Security: Medicare. There is a surcharge on your Medicare premium for those with a high income. Basically they look back 2 years at your income tax return to see if you should pay more for Medicare and there are VERY few exceptions. A one-time event, such as converting a tax-deferred IRA to a ROTH or capital-gains on property sale, may push your income for that year above a threshold that will result in the surcharge in a later year. For us being married, $1 over the limit may result in about a $2000 surcharge for the year. As Mitch said above, it is critically important to do you own taxes to understand this stuff; then use a CPA if you want to actually file them.

  3. Chris, a thought for seniors who enjoy this forum and hold overly large taxable fund(s). If you can, hold them as a legacy. Your kids or other will inherit at the stepped up basis at date of death, thus avoiding significant capital gains should they decide to sell. I plopped a tidy sum in an index fund 25 years ago and with my vegetative style of investing (plop and grow) it has grown to a sum too large to sell without invoking punishing taxes. Of course it is there should I ever need to sell some shares.

    1. Betsy,

      You make a good point on the tax law. However, as noted in the article, I would make the distinction that you shouldn’t hold onto an investment you don’t otherwise want JUST for tax reasons. Sometimes it makes sense to hold onto something b/c the taxes are more punishing than the fees or performance of switching to a different investment. Other times it is better to pay the taxes and move on. This is something we each have to determine on a case by case basis.

  4. It’s a chore dealing with taxable accounts while keeping in mind these multiple issues, and sometimes piecemeal turns out to make the most sense.
    For 2019 I took the opportunity to get out of a high ER actively managed fund in which I had $5K LT cap gains. This amount of tax gain harvesting will not endanger my ACA subsidies and it will be in the 0% tax range, so I believe my only hit will be paying some state taxes.

    1. That’s was my strategy as well. That’s why we found having a framework to guide us was so helpful, particularly with the taxable investments which required ongoing decisions until we got rid of all of the old funds.

      Thanks for sharing Lisa.

  5. Just read the Doughroller post about how to get out of a variable annuity. My question: are there any variable annuities that are decent, especially ones that were sold 10 or so years ago? How can one judge whether rolling it to an IRA or to another variable annuity thru Vanguard is the right move or not? Might one be giving up something worth keeping, especially once the surrender charges are past? Thanks.

    1. East Coast,
      When I wrote that I became an expert on VAs, it was a bit facetious. I do not know the whole marketplace. I’m sure some are better than others. There are some rare circumstance where they can be useful products. The most common is for people who max out all other tax-deferred savings, want more tax-deferral, and for whom the fees outweigh the tax benefits.

      I would recommend you start by asking yourself a few questions. Why do you own the VA? Since you already own it, is there any place for it in your portfolio or overall financial plan? If yes, does that reason justify the fees and restrictions that come with the VA?

      For us, the decision was easy. We were in our late 30’s when we figured out what the annuity was. We had no plans to annuitize any portion of our portfolio for at least 30-40 years and even then we may not want to. In the interim, we would be paying 2+% fees on the portfolio every year. We would have never bought this product if we understood it, and it didn’t fit our objectives once we did understand it. So it was a no brainer.

      Hope that helps. Your situation may be different.

  6. Annuities! what a sham! My wife (H.S. Teacher) got sucked into this ridiculous investment because the commission driven investment advisers prey on teachers…actually the whole system is setup as a real cash cow for advisers as schools systems actually allow these advisers to sit in the schools and market to teachers who know little to nothing about investments. When we got married I took a close look at my wife’s 403b and the INSANE FEES she was getting charged on her annuity…8% commissions then fees on top of that, with high redemption fees. I really laid into the adviser when we sat down with them. He tried to tell me the investments would more than make up for the fees and commissions. Needless to say we fired the adviser moved her 403b to a different company dropped all the annuities paid the redemption fees…it’s been almost 20 years…and she’s still paying the price. Her 403b has grown significantly less than my 401k…because of the annuity mess when she was younger. With time you can really see the impact of fees and high commissions.

    1. Wade,

      I agree 100% with you about the overuse and high fees of annuities. It’s funny b/c I’ve been writing for over 5 years and while readers will disagree with me, I rarely get flat our negative comments and never nasty ones. The exception is when writing about these products. The people that sell them want to protect this sacred cow that make them a lot of money.

      I also agree that this is an extremely common situation among teachers, as I have two in my family who are/were in similar situations. It amazes me that it is so common, b/c teachers have relatively strong unions who should be able to negotiate on their behalf for better benefits and this is a big one where they are being taken for a ride.

      One bit of nuance is that we can’t lump all annuities together, just as you can’t lump all mutual funds together as good or bad. There are certainly examples of both. A single premium immediate annuity is a useful product that is relatively simple to understand, can be fairly priced, and protects against longevity risk.

      That caveat out of the way, I’m fairly certain that’s not the product your wife was sold and it’s not the product we were in either.


  7. I made a few mistakes early on but the stock series you referenced got me back on track before I could do any real damage. I used capital loss and capital loss carryovers to help clean up some of the small issues in my brokerage accounts (ghosts indeed). This time of year isn’t a bad time of year to look at everything.

    We lived on the cusp of the 0% capital gains bracket for several years and could adjust our retirement contributions accordingly to stay there. It is a great place to be.

    Have a good holiday.


    1. Max,

      Great point that this is a great time of year to know where you are with income, which allows you to know how much you can take in capital gains without crossing over any of those thresholds.

      Happy holidays to you as well!

  8. I’m a natural skeptic of complicated financial investments. I don’t have a lawyer or actuary team to protect me from these giant financial companies whom may not be working in my best interest.

    I went to a State Farm “free” seminar for retirees sponsored by wife’s hospital employer. The ranking retired military officer was very informative and stimulating. He led everyone to consider stable monthly income as the corner piece of care free retirement. His parents enjoyed life with no worries since knowing their teacher pension would always be in the mailbox. My wife had a friend that wanted my evaluation. Of course she was only was looking to confirm the wonderful life with annuity income.

    I had an experience with a highly successful vacation company in Florida. Very impressive and they worked on me for a couple years. I’ve listen to relatives who were all sold on the timeshare method of vacation. After being handed off to more and more VIP type people the last being a lady with an accent and adorned with expensive jewelry. She was the wealthy and wise deal closer. Just at the end she mentioned the only way to invest is through their finance system and yes you can refinance with other company’s if you wish at a later time. I remembered one family member mentioned that they got stuck with the original financing as the penalties were to refinance were great. The terms were a joke, so I laughed and told the lady I wanted in on this and would be happy to finance anyone with similar terms. She folded up her folder and walked out. Later I read the company main source of revenue was this complicated financing scheme they just mentioned at the end of long presentation.

    All of my local friends have moved assets to an impressive personal financial advisor. They wish not to talk of the specifics, but what they talk of is very complicated investments for minimising tax burden and high load funds, insurance packages and annuities. This guy needs to control all of your finances as his team and coordination works the best in your behalf to make life enjoyable and carefree. He is a typical confidence man. I have experienced this marketing phenomena on some high force sales promotions. The salesmen have a very high force presentation of losers and winners. They won’t tolerate losers and those that don’t get the wisdom of their brilliance. In fact just attending and listening to them is very telling of the quality of your wisdom. They inform the winners to not talk to those whom are attempting to steal your wisdom and this knowledge for their own on the cheap. We are in this together for security and wealth.

    1. Forrest,

      I think we should all be skeptics. If something sounds too good to be true, a good assumption to start with is that it probably is.


  9. It’s a very good point that it may be worthwhile to wait for a low income year to sell “mistakes” held in taxable accounts. I recently sold off a few such holdings. Unfortunately this is my highest income year ever. Next year (I’m retiring) my income will be the lowest in decades. I should’ve waited. At least I had an equal balance of gains vs losses.
    Does anyone know if Muni Fund income (Federal tax free) counts as far as ACA subsidies?

Comments are closed.