Confirmation Bias and the Morningstar Story in the WSJ
Like many folks, including Mark Perry, I would tend to agree with the following statements
- Past performance is not a very good indicator of likely future performance
- One should generally eschew managed funds in favor of low-cost index funds
However, I think a lot of folks who believe these same things are applying confirmation bias when looking at the data in a recent WSJ story on Morningstar. Morningstar analyzes mutual funds and rates them based on their past 1/3/5/10-years performance in relation to other funds of the same type. Funds in the upper quintile of past performance get 5 stars, the next quintile gets 4 stars, etc.
Morningstar is sort of coy about whether the ratings are supposed to have predictive value. They will say that of course they only measure past performance, but there would be no way to sell these ratings to folks for millions of dollars (as they do) without there being some implication the ratings were at least partially indicative of future performance.
So the WSJ did something interesting -- they went back 10 years and took all the 5 star funds and looked at how they have done since (as measured by Morningstar itself with its star ratings). So how many 5-star funds ten years ago actually had 5-star performance over the subsequent years, and so on. And it turns out that a lot of the 5-star funds have not performed very well. This is a good reminder to us all.
BUT. Look at their own data:
Yes, the 5-star funds from 10 years ago only average 3 today. Everything regresses towards the mean, as we random walk folks might expect.
But the 5-star funds did better than the 4, which did better than the 3, which did better than the 2, which did better than the 1. This actually understates the difference, because many of the lowest performing funds in the lower star categories closed in this 10 year period, so are not in the final metrics, which likely raises the scores of some of the lower buckets because they dropped out (59% of the 1-star funds closed or merged in this period while only 22% of the 5-star funds did so).
This is actually -- to someone who doesn't really buy into the whole stock-picking thing -- a pretty impressive achievement. I challenge you to take stocks or bonds or mutual funds of roughly the same type and divide them into 5 buckets, rank the buckets by expected performance, and actually have this ranking hold for 10 years.
C078342:
I wouldn't necessarily say the performances converge __ 5* funds can only go down and 1* funds up. The important metric would be return. That being said, Vanguard index funds are the way to go, INHO>
October 26, 2017, 3:53 pmJames:
Actually 5-star and 1-star could also just stay the same.
October 26, 2017, 7:39 pmC078342:
and what do you mean
October 26, 2017, 7:46 pmMike:
It looks like most of the regression towards the mean happened in the first three years. After that, skill won out.
October 26, 2017, 8:08 pmjohnmoore:
I wonder if they picked a 10 year period where their predictions just happened to look better than random.
October 26, 2017, 8:31 pmJames:
Five star ratings could go down or continue to be five star. You said they could only go down. Similar for one star.
October 26, 2017, 9:31 pmkidmugsy:
In a rising market riskier funds will do better than less risky irrespective of the skill with which individual holdings are selected. Did they allow for that?
October 27, 2017, 4:13 amPaul Ralley:
Does the average rating after 10 years not also include performance in year 1 (for at least the 10 year rating)? I.e. a poor year 1 (i.e. a 1 star on the left hand side) would (even with all other things being equal) lead to an average lower performance for the 'average of ratings' on the right hand side?
October 27, 2017, 5:19 amDan Wendlick:
As I interpret it, the average rating in year 10 of funds rated 5-star in year one is 3. It is not an average of the ratings of the funds over the 10-year period.
October 27, 2017, 7:17 amcc:
If you go back 10 years, this was near the peak of the stock market. Everything went down in late 08, stocks, bonds, housing, art, gold. So maybe a bad comparison. Try going back 5 years instead.
October 27, 2017, 7:44 amstan:
Using active advisors, we beat the S&P 500 every single year for 12 years (1999-2011, got out of the biz to pursue other interests). And this was after paying all fees. Cumulative beat was over 85%. Keys: 1) asset allocation (overweight value and small/midcap), 2) use advisors who specialize in their niche without style drift (e.g. small cap growth or large cap value, etc), 3) use advisors who stress cash flow and real earnings with sound sell disciplines. Choose advisors on the basis of long term outperformance measured against appropriate benchmark.
October 27, 2017, 10:52 ammarque2:
You beat the market by having a higher risk portfio than the SP500.
October 27, 2017, 9:05 pmprogenitive:
I would tend to agree with your criticism, but stan says he beat S&P500 every single year. If the portfolio were indeed more volatile (I guess what you mean by 'risk'), then it should have had larger losses than the S&P some times, no?
Or, stan got lucky... one person beating an index 12 years in a row doesn't really prove anything. We'd need to see gazillions of stans beating the index at equal or lower volatility, consistently, over a longer time period.
October 28, 2017, 1:11 pmmarque2:
During times when the market goes up, the high risk portfolios tend to go up more. When the market crashes, they all crash at once, but the higher risk ones crash harder.
You are
October 28, 2017, 4:10 pm1: assuming an independence between small, midcap, and large stocks.
2: Assuming this guy is telling the absolute truth, or if he believes he is truthful, might not be aware of statistical techniques used to mask a loss.
3: There is luck involved. If you have 8196 stock advisors in year one, and half get it wrong every year, after 12 years you have about one person who looks like a market picking genius, when it could have just been random luck.
4: The guy said his average was mostly small midcap. But did he cash out in 2007 before the crash? Seems likely. There are several people who did, and they are considered geniuses as well, but seem to not be able to predict much else.
marque2:
This data set is too small to evaluate MorningStar. You need to do an analysis of the level of stocks over many years. calculate the 10 year average for "predictions" from 1997 to now to get a proper analysis of whether MorningStar has predictive value. This chart is basically saying they had one bad year. How about the rest?
October 28, 2017, 4:11 pmslocum:
This is good. Index funds need other people to continue to believe in active investing, pay for all the research, and establish market values. If indexing take over too much of the market, the strategy will stop working. So we index investors need all the arguments in favor of active investing we can get. Thanks, Warren.
October 29, 2017, 4:10 amMercury:
"This is actually -- to someone who doesn't really buy into the whole stock-picking thing -- a pretty impressive achievement."
October 29, 2017, 5:56 am---------------------------------------------------
Agreed. But let's see these same funds over 10 years, beginning with each "star" group and tracked over time by actual performance, not "star" rating.
C078342:
"and what do you mean" Yes, 5* funds and 1* funds can remain unchanged -- but if they change there is only one way for them to move. Agreed???? Are you familiar with the statistical term "regress to the mean?" I'm not willing to put mega $$$$ into an actively managed fund in the long run>
October 30, 2017, 4:29 pmBob Beeman:
Good article. You undoubtedly familiar with the book, A Random Walk Down Wall Street. It sounds good but there are still fund managers that beat there measuring indices, but I don't think they do it each and every year. As far as predictive value of a fund's past performance, the past is certainly not the best predictor, but nevertheless is a predictor and must be taken into account. Would you buy a managed fund with miserable performance for the last several years? Investing blindly in index funds can be exactly the wrong thing because of the tendency to overload a portfolio with more risky indices, resulting in a portfolio that is more turbulent during difficult times, which is pretty much all the time.
October 31, 2017, 12:32 amJames:
I understand what you’re saying, I was just pointing out that not changing at all is a possibility.
October 31, 2017, 10:23 pmTO BLACKS:
I never paid attention to stars at Morningstar because I invest in index funds only.
November 10, 2017, 4:53 pmTO BLACKS:
I agree.
November 10, 2017, 4:54 pm