Farnam Street helps you make better decisions, innovate, and avoid stupidity.

With over 350,000 monthly readers and more than 87,000 subscribers to our popular weekly digest, we've become an online intellectual hub.

What Matters More in Decisions: Analysis or Process?

Think of the last major decision your company made.

Maybe it was an acquisition, a large purchase, or perhaps it was whether to launch a new product.

Odds are three things went into that decision: (1) It probably relied on the insights of a few key executives; (2) it involved some sort of fact gathering and analysis; and (3) it was likely enveloped in some sort of decision process—whether formal or informal—that translated the analysis into a decision.

Now how would you rate the quality of your organization’s strategic decisions?

If you’re like most executives, the answer wouldn’t be positive:

In a recent McKinsey Quarterly survey of 2,207 executives, only 28 percent said that the quality of strategic decisions in their companies was generally good, 60 percent thought that bad decisions were about as frequent as good ones, and the remaining 12 percent thought good decisions were altogether infrequent.

How could it be otherwise?

Product launches are frequently behind schedule and over budget. Strategic plans often ignore even the anticipated response of competitors. Mergers routinely fail to live up to the promises made in press releases. The persistence of these problems across time and organizations, both large and small, would indicate that we can make better decisions.

Looking at how organizations make decisions is a good place to start if we’re trying to improve the quality of decisions and remove cognitive biases.

While we often make decisions with our gut, these decisions leave us susceptible to biases. To counter the gut decision a lot of organizations gather data and analyze decisions. The widespread belief is that analysis reduces biases.

But is putting your faith in analysis any better than using your gut? What does the evidence say? Is there a better way?

Decisions: Analysis or Process

Dan Lovallo and Olivier Sibony set to find out. Lovallo is a professor at the University of Sydney and Olivier is a director at McKinsey & Company. Together they studied 1,048 “major” business decisions over five years.

What they discovered will surprise you.

Most of the decisions were not made based on gut calls but rather rigorous analysis. In short, most people did the all the leg work we think we’re supposed to do: they delivered large quantities of detailed analysis. Yet this wasn’t enough. “Our research indicates that, contrary to what one might assume, good analysis in the hands of managers who have good judgment won’t naturally yield good decisions.”

These two quotes by Warren Buffett and Charlie Munger explain how analysis can easily go astray.

I have no use whatsoever for projections or forecasts. They create an illusion of apparent precision. The more meticulous they are, the more concerned you should be. We never look at projections … — Warren Buffett

[Projections] are put together by people who have an interest in a particular outcome, have a subconscious bias, and its apparent precision makes it fallacious. They remind me of Mark Twain’s saying, ‘A mine is a hole in the ground owned by a liar.’ Projections in America are often a lie, although not an intentional one, but the worst kind because the forecaster often believes them himself. — Charlie Munger

But Lovallo and Sibony didn’t only look at analysis, they also asked executives about the process. Did they, for example, “explicitly explore and discuss major uncertainties or discuss viewpoints that contradicted the senior leader’s?”

So what matters more, process or analysis? After comparing the results they determined that “process mattered more than analysis—by a factor of six.

This finding does not mean that analysis is unimportant, as a closer look at the data reveals: almost no decisions in our sample made through a very strong process were backed by very poor analysis. Why? Because one of the things an unbiased decision-making process will do is ferret out poor analysis. The reverse is not true; superb analysis is useless unless the decision process gives it a fair hearing.

To illustrate the weakness of how most organizations make decisions, Sibony used an interesting analogy: the legal system.

Imagine walking into a courtroom where the trial consists of a prosecutor presenting PowerPoint slides. In 20 pretty compelling charts, he demonstrates why the defendant is guilty. The judge then challenges some of the facts of the presentation, but the prosecutor has a good answer to every objection. So the judge decides, and the accused man is sentenced.

That wouldn’t be due process, right? So if you would find this process shocking in a courtroom, why is it acceptable when you make an investment decision? Now of course, this is an oversimplification, but this process is essentially the one most companies follow to make a decision. They have a team arguing only one side of the case. The team has a choice of what points it wants to make and what way it wants to make them. And it falls to the final decision maker to be both the challenger and the ultimate judge. Building a good decision-making process is largely ensuring that these flaws don’t happen.

Understanding biases doesn’t make you immune to them. A disciplined decision process is the best place to improve the quality of decisions and guard against common decision-making biases.

Still curious? Read this next: A process to make better decisions.

The inspiration for this post comes from Chip and Dan Heath in Decisive.