The Paradox of Index Funds (And the Joy of Shopping)

Mark Perry makes the (updated) case for index funds.  I need no convincing, as most** of my savings (such that they are) are in Vanguard index funds of various sorts.

But as I was reading his article, I couldn't help thinking that there is a flaw with the "everyone should be in index funds" advice -- if everyone actually was 100% in index funds, they would not work.  Index funds are premised on the idea that stock prices are pretty well reflective of the information out there in the marketplace -- the company's future prospects, the strength of its market position, the direction of external factors such as economic growth and interest rates, etc.  But this is only going to be true if there are investors out there trying to pick stocks and beat the market -- ie if everyone is not in index funds.

It sort of reminds me of the old economics joke where a man is walking down the street with an economist, and the economist walks right past a $20 bill lying on the ground.  The man says to the economist -- "do you realize you just walked past a $20 bill?" and the economist answered, "It couldn't really be there -- in an efficient market, someone would have already picked it up."

In some ways, the stock pricing paradox here is just an example of a larger phenomena which for the lack of a better name I call "the joy of shopping."  People make fun of shopping all the time, but economically shopping is really a miracle.  All the things we attribute to prices and efficient markets and competition and the accountability of markets depend on shopping.  Individuals have to be out there making price-value trade-offs between products, or between buying something and not buying something.   For example, at least half of everything wrong with health care economics can be explained by lack of shopping.

The interesting thing is that only a small percentage of consumers in any particular market have to be hard-core shoppers (meaning they do tons of research and compare prices across multiple sellers) for all of us to benefit.  I seldom look at a price in Wal-Mart because I know other people who care a lot have enforced a discipline on Wal-Mart.  Just as with my Vanguard mutual funds, I depend on that core of folks who walks the aisles of Wal-Mart checking every price against Amazon and Target.


** I do enjoy picking stocks, and have a particular affinity for shorting things too early.  I never, ever let this portfolio grow to more than 5% of my total savings, and treat it explicitly as a sandbox to play in rather than real investments on which my future well-being depends.


  1. Pinebluff:

    Pension funds which are a form of index funds are a big player in the stock markets so it is unlikely that everybody eventually will be in index funds.

  2. Dan Wendlick:

    If enough people were in index funds, the individual stocks in the index would lose their individuality. The index itself might move with respect to other asset types, including stocks not part of the index, but the individual shares should move in lockstep.

  3. slocum:

    Yep. Many years ago, I was arguing with my uncle who was convinced that managed funds would beat index funds in the long term. After a while, I gave up arguing and told him that I was actually happy that I couldn't convince him since for index funds to keeps working, there needed to be active investors out there paying for all the research and establishing the prices.

  4. Todd:

    This was a topic earlier this week in Tyler Cowen's interview of Cliff Asness. Interesting throughout, as Tyler might say. The "everyone indexes" issue is addressed in the audience Q&A towards the end. Apparently there is active debate and no consensus on how the market would behave in such an environment.

  5. Arrian:

    If everyone invested in index funds, wouldn't that violate the assumptions of the Efficient Market Hypothesis? Index funds use no information, so if everyone is investing in them, _nobody_ is using any information on the market. But, then, that would mean that anyone who had _any_ information would be able to profit, and therefore should defect and use that knowledge.

    Looking at it that way, the EMH is basically the result of unravelling the "every invests in index funds" hypothetical: If nobody is using any information, then any information you have is unique to you and you can profit from it. Then the next investor who has information beyond the information can profit because that information is unique and thus valuable. Iterate until the marginal investor has no unique information and now you've got a situation where there's no value for information to the marginal investor because it's already been used and is this reflected in prices. The only difference is that the EMH says we're already doing this all the time, and nobody has unique information, so there's no value to information instead of moving to this situation from a base point of no information being used.

    I think it's important to remember that models are like maps, not pictures: The describe the world in an accurate, but abstracted way. If you look at your house using Google Maps, then look out your window, you'll see two completely different things. And, while there's a lot of stuff out your window that isn't shown by Google Maps, the web site is more useful for telling a friend how to get to your house than a photo of the same area precisely because it abstracts to clarify the important information. It doesn't matter that your street is paved in asphalt, has a stop sign and the sign is on the south side of the intersection. It does matter that the street you're on is named Broadway and it intersects Main.

  6. kevinsdick:

    My memory of financial economics is that this is not at all controversial. There will always be a few people with a modest analytic or information advantage in evaluating stocks. They will comprise the active traders and earn rents in the form of slightly better returns. There's an equilibrium in the number of active traders due to the opportunity costs of such traders' next best career option. As the potential active trading rents fluctuate, active traders enter and exit the market as their other prospects become relatively worse or better.

    Higher potential rents induce more active traders into the market, which drives down rents, maintaining an equilibrium. The very best stay in the active market for a long time until some technology change displaces them (just like every other market). And there's an equilibrium in the size of funds any one shop can manage due to scarcity of arbitrage opportunities relative to how much they move the market with their trades.

    So this does not violate EMH in any way whatsoever. There's a meta-market in trading skill that efficiently compensates those with a competitive advantage in trading for doing the work that makes the securities market efficient.

  7. morganovich:

    even relatively small proportions of index investing reduce overall market returns.

    market returns are a function of earnings growth. earnings growth is a function of economic growth. economic growth is a function of how well capital is used. thus, every time anyone indexes as opposed to actively invests, market returns drop. less efficient use of capital leads to less growth and a higher cost of capital for those not in an index. it floods index companies with capital, but starves others. as index firms tend to be the most mature, they tend to need it least. this leads to far less growth and less new corporate formation.

    this reduces overall economic growth. that reduces long run return on equities because their earnings grow less rapidly.

    a few indexers might have little effect, but they are still having some effect just as if you toss a rock in to the sea, it does raise sea level. it may not raise it enough to notice or even measure, but we know it did cause a rise.

    the more folks index, the more long term returns drop. this can be masked by capital flows in the short run, but in the long run, you might be better off under-performing the market that does 8% than mimicking the one that does 4.

    clearly, we'll never hit the point where everyone indexes. the return to being one of the last few active investor would be outlandish. it would equilibrate out long before then. but we are currently deep into the zone where indexing is suppressing returns and distorting the price signal for capital to the detriment of overall returns.

  8. morganovich:

    it's quite a paradox.

    EMT requires active investing to work because that is how information is discounted and disseminated, but the prescription of EMT is to index. thus, if we follow EMT, it breaks EMT.

    only an academic could dream that up.

  9. morganovich:

    actually, it depends a lot on the funds.

    ironically, vanguard, champion of the index fund, has beaten its own index funds with its active funds over the last 3, 5, and 15 years.

    over each full bull/bear cycle, the hfri hedge fund index has beaten the S+P handily. since 2000 (2 bears, 2 bulls) it is up 119% vs 85% for the S+P if you have full, tax free dividend reinvestment.

    index funds tend to outperform during bulls. at the top of the cycle, they always look great because the 5 year comp is all bull market and the 10 year is generally 2 bulls and one bear. they under-perform during bears and, historically, over the full bull bear cycle.

    capital preservation is a bigger deal in making money than sprinting ahead in a bull. index beat hedge funds in 1995-98 in all 4 years. people were making the same arguments then that are popular now. then, in 1999-2002, hedge funds outperformed them by 60 percentage points. in the 2000-2 bear, they were up 8.6% vs down 40.

    from 2003-7, the spx was up 82.45%, beating hedge funds which were up 72.36. then, the 2008 bear hit. this dropped the spx (with div) return for 2003-8 to 14.55% while the HFRI was now 37.06%, more than twice the return over the full cycle.

    so, you have to be very careful about how you make a comparison. at the top of a bull, indexing is always looking great. at the bottom of a bear, it generally looks AWFUL compared to active, especially compared to hedged.

    over a full cycle, it actually tends to lose.

  10. Dan Wendlick:

    I think the index would trade as though it were a conglomerate of all of the companies included in the index. Remember, when you buy into an index, you are not buying the whole market, you're only buying the constituent companies. There are still plenty of companies that exist that are even outside the S&P 500 that the value could float against, not to mention asset types other than stocks: commodities, bonds, real estate, etc. Ultimately, this should lead to individual companies not wanting to be included in the index, since their share price would be tied to the performance of all the other companies in the index, and the whole principle behind founding a firm is the assumption that the firm will provide a higher long-term rate of return than a lower or zero-risk alternative.

  11. kevinsdick:

    This is not at all the case. The number of 4-sigma+ smart people and petaflops of computing power currently devoted to trying to arbitrage prices in securities related to public equities is quite large. They are viciously competing with enormous resources to find the efficient market prices. Anyone who thinks they can do better is free to get in the game and see if they can beat these pros.

  12. Michael Stack:

    I always thought of index funds as belonging to a class of problems: "Things that are true because a lot of people think they aren't". The only other example that comes to mind is voting ("voting is a waste of time" is true because most people believe it isn't).