Fans of the National Hockey League (NHL) were excited to see the season open this past weekend after a four-month lockout. Similar to the National Basketball Association (NBA), which went through a lockout last year, the NHL will play a reduced schedule. Last year, NBA teams played 66 games during the shortened season and the NHL will play 48 games this year (a full season is 82 games for both leagues). A closer examination of the truncated seasons offers some useful lessons for executives.
These situations seem similar on the surface. The leagues play an identical number of games during the same time of year and the sports have a like number of players in the game at a time. Basketball has more scoring, but hockey games are longer. Playing a slate of games that is 60% or more of the full season seems to be a comparable solution.
But if the objective of the regular season is to determine which teams are best, there is a huge difference between the NBA and the NHL. The key is the relative contribution of skill and luck in determining results for a season. Of the professional sports leagues — which also includes include the National Football League, the Premier League (soccer), and Major League Baseball — skill plays the largest role in the NBA (PDF) and the smallest role in the NHL.
Here's the way to think about it. When a sport has lots of skill and little luck, you don't need a large sample size to determine which team or individual is more skillful. Less than 1/6th of a minute of running against Usain Bolt, for example, would do the job completely. But when there's lots of luck, you need a large sample size to ensure that luck evens out and skill shines through. Baseball has a lot of luck, but you can be reasonably confident that the better teams make the playoffs because the regular season includes 162 games (over 27,500 minutes).
To make this concrete, we can calculate the number of games you need in the NBA and the NHL to get an equivalent contribution of skill and luck. Because basketball has lots of skill, you only need 10-15 games, or about 15% of the season, to obtain the same signal that you get from 70-75 hockey games, or nearly 90% of the season. Sixty-six games for the NBA is plenty to allow the top teams to climb to the surface. But 48 games in the NHL is way too few. The NHL season is simply too short to be confident that the more skillful teams will rise to the top. If there's a year for a mediocre team to do unexpectedly well, this is it.
What does all of this have to do with business? Employees within a company carry out a variety of tasks. Some are like basketball — almost all skill and very little luck. One example is six sigma, where a company seeks to have no more than 3.4 defects per million products it manufactures. Others are similar to hockey, where there's lots of luck. Strategy development is a case in point. A company can devise a wonderful strategy that fails as the result of bad luck. Michael Raynor, a consultant at Deloitte, shares the example of Sony's strategy for the MiniDisc. He argues that Sony's strategy was brilliant but that the product failed because of bad luck.
As a business leader you need to do two things:
1. Make a judgment about where a business activity lies on a continuum from all-luck and no-skill on one end to no-luck and all-skill on the other.
Answering three questions can get you on your way.
First, ask if you can easily assign a cause to the effect you see. When cause and effect are clear, skill tends to be prominent.
Next, what is the rate of reversion to the mean? If reversion to the mean is slow, then skill is shaping results.
Finally, ask: Where can we predict well? Where predictions are accurate, skill is present. If predictions are little better than random, then luck is carrying the results.
Once you know where an activity lies on the continuum you can appropriately tailor feedback, remuneration, and incentives.
Where skill is dominant, outcomes and skill are highly correlated and feedback is straightforward. Where luck plays a large role, feedback based on outcomes is insufficient. In these cases, you need to focus on the process. It's hard to know a blackjack player's skill based on her short-term outcomes but you can evaluate how she plays her cards and gain a sense of her ability.
The principle behind thoughtful remuneration is easy enough: Pay for skill but not for luck. Companies have violated this principle in recent years, creating a sense of unfairness. One example is executive pay through stock options that are not indexed. In bull markets, CEOs haul in large sums as the beneficiaries of luck. In bear markets, superior CEOs fare poorly as a consequence of bad luck.
2. Finally, align incentives to match the activity. On the skill end, incentives can be tied to output because skill and results are closely aligned. On the luck end, incentives should reflect a proper process, and you should consider only long-term results.
The main lesson from the shortened season is events that may appear similar on the surface can be quite different once you analyze them. In your company, some activities are more like basketball and others are more like hockey. Understanding those differences and tailoring feedback, remuneration, and incentives to match the task is the ultimate goal.