Most of us would like to consider ourselves savvy shoppers: We research purchases, we compare prices, and some of us even negotiate, especially on big ticket items like cars or houses. Yet many Americans don’t pay attention to something we pay for that can have a drastic effect on our entire lives – the fees on our retirement and brokerage accounts.

A study by the FINRA Investor Education Foundation showed 63% of investors either don’t think they pay fees or don’t know how much they pay. Another study from State Street Global Advisors showed almost half of investors mistakenly think the cost of investments like mutual funds and exchange-traded funds are included in the fee they pay to their adviser.

Over time, ongoing fees affect your investment portfolio in two ways: Not only is your investment balance reduced by the fee, but you also lose any return you would have earned on that amount. This is how even a small fee can end up having a big impact on your investment portfolio.

A percentage point here or there doesn’t sound like much, but over a lifetime of investing, it can really add up to tens of thousands of dollars. Use this calculator to find out how much of a difference these fees can make.

[RELATED: More Financial Calculators From MOAA]

There are all different types of fees, and some of them can be charged simultaneously. For example, you could have an investment adviser who charges an ongoing fee of 1% (a very common rate in the industry). That means for every $100,000 you invest, you pay $1,000 per year in advisory fees.

But many firms also charge transaction fees, and your mutual fund could have internal expenses it passes along to you. These costs are known as an expense ratio and are expressed in units knowns as basis points, with one basis point equal to .01%, or 0.001 of the amount you have invested. You can do a deeper dive on some of these fees using MOAA’s Mutual Fund Expense Calculator.

Expense ratio fees are not subtracted from your account directly, but they reduce the returns on your account. For example, if your mutual fund’s investments deliver an annual return of 9% and the fund has an expense ratio of 1%, your actual return is 8%.

Yet another type of fee is a commission charged on certain mutual funds when they are bought or sold. If the commission is charged when the fund is purchased, it is called a “front end load” and if it is charged upon the sale of the fund, it is called a “back-end load.”

[RELATED: More Finance News From MOAA]

These aren’t the only fees out there, but they are some of the more common ones. You can avoid some of these fees by choosing broad index mutual funds and Exchange Traded Funds (ETFs); these funds have low fees because they mirror a particular market index rather than paying a fund manager to actively choose the funds.

Many passively managed funds, like the above-mentioned index mutual funds and ETFs, charge fees of between 0.2% and 0.5%. In order to outperform these funds, actively managed funds must often beat the market by 3% or more to overcome fund costs, and this is very difficult to do.

You can look up a fund’s fees on its prospectus, a formal document that must be filed with the Securities and Exchange Commission. FINRA, the Financial Industry Regulatory Authority, also has a fund analyzer. Some other types of fees may be spelled out clearly on your account statements, but others may not be as obvious – it’s always a good idea to ask.

Not all fees are bad: I have previously shared my thoughts on the benefits you can reap if you pay a financial professional to give you advice. But if you’re going to pay a fee, make sure you know about it and understand it and the implications it can have on your portfolio.