There’s a simple and elegant test of whether there is skill in an activity: ask whether you can lose on purpose. If you can’t lose on purpose, or if it’s really hard, luck likely dominates that activity. If it’s easy to lose on purpose, skill is more important.
In this report, we will discuss why unraveling skill and luck is so important, provide a framework for thinking about the contribution of skill and luck, offer some methods to help sort skill and luck in various domains, and define the key features of skill in the investment business.
Reversion to the mean
Your initial reaction may be that more luck means less predictable outcomes—which is true. But there is also an important insight that follows from knowing the relative contribution of skill and luck: the ratio of skill to luck shapes the rate of reversion to the mean. Specifically, the outcomes of activities laden with luck revert to the mean faster than activities with little luck. Reversion to the mean is present in all activities that have even a dash of luck, but the rapidity of the process hinges largely on how important a role luck plays. In fact, you can infer the relationship between skill and luck by analyzing past patterns of reversion to the mean….
You can think of skill as a drag on the reversion process for the activities that combine skill and luck. An above-average shooter in basketball, for example, may go through good or bad stretches of shooting. But she will not revert back to the average over time because of her skill. The same would be true of a below-average player. Luck may help or hinder in the short term, but reversion is limited because of the level of skill.
Humans, as natural pattern seekers, have a very difficult time dealing with reversion to the mean. The main challenge with the concept is that change within the system occurs at the same time as no change to the system. Change and no change operate side-by-side, causing a lot of confusion. The change part is reversion to the mean. As we will see, the results of the groups that have done really well or poorly in one period tend to move toward the average in future periods. Most people find it hard to internalize reversion to the mean because it is more natural to extrapolate the performance of the recent past. So if a stock, or an asset class, has done well the natural inclination is to assume it will continue to do well and to act accordingly….
Insensitivity in sample size
About 40 years ago, Amos Tversky and Daniel Kahneman identified a common decision-making bias they called the “belief in the law of small numbers.”
The idea is that we tend to view a relatively small sample of outcomes of a population as representative of the broad population of outcomes. The magnitude of this mistake grows larger as the luck-to-skill ratio rises….There are a number of factors that shift activities toward the luck side of the continuum. One, naturally, is simply sample size. A small number of observations make it very difficult to sort skill and luck….People often assume that building sample size is a matter of time. Within an activity that is true, but what really matters is the number of trials. Some activities pack a lot of trials into a short time, and others reflect a few trials over a long time. You can evaluate a high-frequency trading strategy a lot quicker than a buy-and-hold investment approach.
You don’t make money by being smarter than the bettor next to you or by knowing which horse has the best odds of winning. You make money when the collective misprices the odds. The important idea is that you are competing not against other individuals, but against the wisdom of the crowd.
The illusion of control also comes into play here. This illusion says that when we perceive ourselves to be in control of a situation, we deem our probabilities of success to be higher than what chance dictates. Saying it differently, when we are in control we think our ratio of skill to luck is higher than it really is. Remarkably, this illusion even holds for activities that are all chance. For example, some people throw dice hard when they want a high number, and gently when they seek a small one. Like the belief in small numbers, this illusion is not a problem in high skill, low luck activities but becomes more problematic as the contribution of luck grows. Here again, our minds are poor at differentiating between activities, so what works in one setting fails miserably in another.
The best way to ensure satisfactory long-term results is to constantly improve skill, which often means enhancing a process. Gaining skill requires deliberate practice, which has a very specific meaning: it includes actions designed to improve performance, has repeatable tasks, incorporates high-quality feedback, and is not much fun.
Transitivity holds when there’s a clear pecking order of skill and the better competitor always wins. In reality, matchups between individuals, teams, competitors, or strategies generally yield a lack of transitivity. What allows you to win in one environment may not work in another. As a general rule, transitivity tends to decrease as the complexity of the interaction increases. It is hard to characterize the degree of transitivity thinking only of skill and luck, but the idea remains very useful for decision makers…
The two main ways to assess skill and luck are through an analysis of persistence of performance (with streaks being a particularly useful subset of this approach) and its alter ego, reversion to the mean. The research shows evidence for persistence of performance in sports, business, and investing, although the evidence is strongest in sports. Studies of business and investing point to skill in both domains, although the percentage of companies or investors with skill is small.Reversion to the mean is also clear in each realm.
The central insight is that the more the outcomes of an activity rely on luck (or randomness), the more powerful reversion to the mean will be. As important, it is clear that many decision makers do not behave as if they understand reversion to the mean, and predictably make decisions that are, as a consequence, harmful to their long-term outcomes. This is particularly pronounced in the investment industry…