Mon, July 06, 2009
Taleb On Distinguishing Between Investing Skill and Random Chance
Several weeks ago I found time to read Nassim Nicholas Taleb’s Fooled By Randomness. This book, entertaining and eminently readable, is something between a memoir and a treatise on the pervasive human tendency to discern patterns in places where they don't actually exist. Among many examples, he shows how easy it is to mistake a lucky idiot for a skilled investment manager.
Here are some of my favorite quotes from Fooled By Randomness:
As a practitioner of uncertainty I have seen more than my share of snake-oil salesmen dressed in the garb of scientists, particularly those operating in economics.
I do not dispute that arguments should be simplified to their maximum potential; but people often confuse complex ideas that cannot be simplified into a media-friendly statement as symptomatic of a confused mind.
Beware the confusion between correctness and intelligibility…. Any reading of the history of science would show that almost all the smart things that have been proven by science appeared like lunacies at the time they were first discovered.
Taleb is better known for his 2007 book, The Black Swan, but Fooled by Randomness is also well worth reading. Taleb tells several stories about professional traders who, having begun well, seem to suddenly lose their edge after several years. One of his goals in the book is to explain the wrong-headedness of the popular notion that an investment manager with a great track record can be counted on to perform well in the future.
Midway through the book, he offers a thought experiment: Suppose you could take an infinite number of monkeys and give them all typewriters (OK, word processors). Sooner or later, by sheer luck, one of them will type out Homer’s Illiad. He then asks, if you could identify that monkey, what are the chances that you can get it to write the Odyssey?
His choice of thought experiment probably reflects his feelings about a great many investment managers. But Taleb goes on to offer a more quantitative illustration of how hard it can be to tell the difference between dumb luck and investing skill.
Consider a group of 10,000 investment managers whose investment success is comparable to that of a fair coin toss. Each manager has a 50-50 chance of making or losing 10% in any year. At the end of each year the losing managers are thrown out of the game.
After the first year 5,000 managers have made money and the rest have bailed out. After seven years there will still be 78 managers who have made money continuously, purely on the basis of random chance. Their portfolios are up a total of almost 95% after 7 years. If these folks then go off and start their own hedge funds, there’s no way to know that their apparent genius was merely luck, and they’ll probably attract a lot of investors.
Now suppose that instead of giving the managers an even chance of making or losing money, we start with a pool of 10,000 relatively incompetent managers. These individuals are more likely to lose money than to make it: they each have a 45% chance of making a profit and a 55% chance of losing money. At the end there are still 37 fund managers who have made 10% a year for seven years. Even though this group is more likely to lose money than to make it, a few will still run up impressive gains and develop reputations as hot managers.
This demonstrates that if there are lots of fund managers, a certain number of them can look like geniuses when in fact they are merely “lucky,” even if, in reality, they’re moderately incompetent! As Taleb notes, knowing that a manager has an extraordinary investment track records doesn’t tell you anything: “without knowing how many managers out there have tried and failed, we will not be able to assess the validity of the track record.” If the manager population were small, a good track record would be impressive. But in fact, if you count all the money managers in the U.S. alone, 10,000 underestimates the reality.
Jeffrey Ptak, Chief Investment Officer for Morningstar, recently wrote a blog post in which he similarly demonstrated that one’s chances of selecting a mutual fund that will outperform its index significantly are poor. This can be difficult to discern due to “survivorship bias:” if you look at a fund’s five-year track record and compare it to the record of existing funds, you’ll underestimate the number of competitors. Why? Because there will be funds that existed five years ago that have gone out of business or have been merged with other funds. This latter practice is a favorite trick of fund companies: if a fund has a lousy record, merging it with a better fund obliterates its past failures.
Ptak notes that if you look at all the active large-cap mutual funds that existed over the ten-year period ending May 31, 2009, you’ll find that 878 funds beat the Vanguard 500 Index Fund (a passively-managed index fund) and 425 beat its total return by at least 2 percentage points. Looking at the 1,478 large-cap funds in Morningstar’s database, you might conclude that ten years ago you had a 59% chance of picking an active fund that could outperform the S&P500 index and a 29% chance of finding one that could significantly outperform it.
But 10 years ago, there were not 1,478 large-cap funds, there were 2,757! Almost 1,300 have disappeared. So if you were shopping for a large-cap fund then you’d have had a 31% chance of beating the index and only a 15% chance of beating its performance by a significant margin.
To be sure, 15% is more than you’d expect from a cohort that was randomly lucky. But you’d have had to pick the right funds ten years ago and stick with them to obtain that performance. Ptak shows that most investors don’t do this. Morningstar has a methodology that allows it to take into account investors’ tendency to pull out of funds when the market is down and pile in when the market rises. This behavior reduces the actual returns achieved. When fund inflows and outflows are taken into account, only investors in 358 large-cap funds – 13% of the total – managed to beat the index at all.
This doesn’t prove that passive mutual funds are always superior to active ones, by any means. But it does show that most investors, lacking the means to correctly predict which fund managers will outperform over the long term, will usually be better off buying index funds.
This post has been included in the 213th Edition of the Carnival of Personal Finance.