A great edge.org conversation with Philip Tetlock on how to win at forecasting.
An introduction by Daniel Kahenman:
Philip Tetlock’s 2005 book Expert Political Judgment: How Good Is It? How Can We Know? demonstrated that accurate long-term political forecasting is, to a good approximation, impossible. The work was a landmark in social science, and its importance was quickly recognized and rewarded in two academic disciplines—political science and psychology. …
Tetlock’s current message is far more positive than his earlier dismantling of long-term political forecasting. …
How To Win At Forecasting
There's a question that I've been asking myself for nearly three decades now and trying to get a research handle on, and that is why is the quality of public debate so low and why is it that the quality often seems to deteriorate the more important the stakes get?
About 30 years ago I started my work on expert political judgment. It was the height of the Cold War. There was a ferocious debate about how to deal with the Soviet Union. There was a liberal view; there was a conservative view. Each position led to certain predictions about how the Soviets would be likely to react to various policy initiatives.
One thing that became very clear, especially after Gorbachev came to power and confounded the predictions of both liberals and conservatives, was that even though nobody predicted the direction that Gorbachev was taking the Soviet Union, virtually everybody after the fact had a compelling explanation for it. We seemed to be working in what one psychologist called an “outcome irrelevant learning situation.” People drew whatever lessons they wanted from history.
The truth is somewhere in between
It's fascinating to me that there is a steady public appetite for books that highlight the feasibility of prediction like Nate Silver (The Signal and The Noise) and there's a deep public appetite for books like Nassim Taleb's The Black Swan, which highlight the apparent unpredictability of our universe.
We're unwilling to get better
I recall Daniel Kahneman having said on a number of occasions that when he's talking to people in large organizations, private or public sector, he challenges the seriousness of their commitment to improving judgment and choice. The challenge takes the following form–would you be willing to devote one percent of your annual budget to efforts to improve judgment and choice? And to the best of my knowledge, I don't think he's had any takers yet