In a SPIEGEL interview, Nobel Prize-winning psychologist Daniel Kahneman discusses the innate weakness of human thought, deceptive memories and the misleading power of intuition.
SPIEGEL: By studying human intuition, or System 1, you seem to have learned to distrust this intuition…
Kahneman: I wouldn't put it that way. Our intuition works very well for the most part. But it's interesting to examine where it fails.
SPIEGEL: Experts, for example, have gathered a lot of experience in their respective fields and, for this reason, are convinced that they have very good intuition about their particular field. Shouldn't we be able to rely on that?
Kahneman: It depends on the field. In the stock market, for example, the predictions of experts are practically worthless. Anyone who wants to invest money is better off choosing index funds, which simply follow a certain stock index without any intervention of gifted stock pickers. Year after year, they perform better than 80 percent of the investment funds managed by highly paid specialists. Nevertheless, intuitively, we want to invest our money with somebody who appears to understand, even though the statistical evidence is plain that they are very unlikely to do so. Of course, there are fields in which expertise exists. This depends on two things: whether the domain is inherently predictable, and whether the expert has had sufficient experience to learn the regularities. The world of stock is inherently unpredictable.
SPIEGEL: So, all the experts' complex analyses and calculations are worthless and no better than simply betting on the index?
Kahneman: The experts are even worse because they're expensive.
SPIEGEL: So it's all about selling snake oil?
Kahneman: It's more complicated because the person who sells snake oil knows that there is no magic, whereas many people on Wall Street seem to believe that they understand. That's the illusion of validity …
SPIEGEL: … which earns them millions in bonuses.
Kahneman: There is no need to be cynical. You may be cynical about the whole banking system, but not about the individuals. Many believe they are building real value.
SPIEGEL: Do we generally put too much faith in experts?
Kahneman: I'm not claiming that the predictions of experts are fundamentally worthless. … Take doctors. They're often excellent when it comes to short-term predictions. But they're often quite poor in predicting how a patient will be doing in five or 10 years. And they don't know the difference. That's the key.
SPIEGEL: How can you tell whether a prediction is any good?
Kahneman: In the first place, be suspicious if a prediction is presented with great confidence. That says nothing about its accuracy. You should ask whether the environment is sufficiently regular and predictable, and whether the individual has had enough experience to learn this environment.
SPIEGEL: According to your most recent book “Thinking, Fast and Slow,” when in doubt, it's better to trust a computer algorithm.
Kahneman: When it comes to predictions, algorithms often just happen to be better.
SPIEGEL: Why should that be the case?
Kahneman: Well, the results are unequivocal. Hundreds of studies have shown that wherever we have sufficient information to build a model, it will perform better than most people.
SPIEGEL: How can a simple procedure be superior to human reasoning?
Kahneman: Well, even models are sometimes useless. A computer will be just as unreliable at predicting stock prices as a human being. And the political situation in 20 years is probably completely unpredictable; the world is simply too complex. However, computer models are good where things are relatively regular. Human judgment is easily influenced by circumstances and moods: Give a radiologist the same X-ray twice, and he'll often interpret it differently the second time. But with an algorithm, if you give it the same information twice, it will turn out the same answer twice.
SPIEGEL: IBM has developed a supercomputer called “Watson” that is supposed to quickly supply medical diagnoses by analyzing the description of symptoms and the patient's history. Is this the medicine of the future?
Kahneman: I think so. There's no magic involved.