Philip Tetlock on Expert Prediction

Great talk by Philip Tetlock, author of Expert Political Judgment: How Good Is It? How Can We Know?, on the Long Now Foundation’s Seminars About Long-Term Thinking series:

In a nutshell, “How you think matters more than what you think.”

It’s a matter of judgement style, first expressed by the ancient Greek warrior poet Archilochus: “The fox knows many things; the hedgehog one great thing.” The idea was later expanded by essayist Isaiah Berlin. In Tetlock’s interpretation, Hedgehogs have one grand theory (Marxist, Libertarian, whatever) which they are happy to extend into many domains, relishing its parsimony, and expressing their views with great confidence. Foxes, on the other hand are skeptical about grand theories, diffident in their forecasts, and ready to adjust their ideas based on actual events.

The aggregate success rate of Foxes is significantly greater, Tetlock found, especially in short-term forecasts. And Hedgehogs routinely fare worse than Foxes, especially in long-term forecasts. They even fare worse than normal attention-paying dilletantes — apparently blinded by their extensive expertise and beautiful theory. Furthermore, Foxes win not only in the accuracy of their predictions but also the accuracy of the likelihood they assign to their predictions— in this they are closer to the admirable discipline of weather forecasters.

The value of Hedgehogs is that they occasionally get right the farthest-out predictions— civil war in Yugoslavia, Saddam’s invasion of Kuwait, the collapse of the Internet Bubble. But that comes at the cost of a great many wrong far-out predictions— Dow 36,000, global depression, nuclear attack by developing nations.

Hedgehogs annoy only their political opposition, while Foxes annoy across the political spectrum, in part because the smartest Foxes cherry-pick idea fragments from the whole array of Hedgehogs.

Bottom line… The political expert who bores you with an cloud of “howevers” is probably right about what’s going to happen. The charismatic expert who exudes confidence and has a great story to tell is probably wrong.

Still Curious?
What’s Wrong With Expert Predictions
Generalists vs. Specialists (And the Specialist’s Dilemma)
Future Babble: Why expert predictions fail and why we believe them anyway
The real reason it’s so hard to predict bubbles
The (In)Accuracy of Long-Range Forecasting
The Future of Prediction

H/T Steve Hsu