An interesting excerpt from Tim Harford's review of The Signal and the Noise on Bayes’ theorem and improving prediction:
Thomas Bayes, an 18th-century minister and mathematician, nonconformist in both roles. Bayes’ theorem, published posthumously, tells us how to combine our pre-existing view of the world with new information in a rational way.
Silver explains Bayes’ theorem with a dark example: the attacks on the World Trade Center. When the first plane hit the tower, horrified observers instinctively updated the possibility of a terrorist attack that day from “barely thinkable” to “distinctly possible”, although at that stage an accident could not be discounted. Bayes’ theorem shows that when the second plane hit, the chance of terrorism could be updated again, from “distinctly possible” to “all but certain”.
There is no need for a mathematical analysis to tell us that, but Silver argues convincingly that Bayes’ theorem is an important reality check on our efforts to forecast the future. How, for instance, should we reconcile a large body of theory and evidence predicting global warming with the fact that there has been no warming trend over the last decade or so? Sceptics react with glee, while true believers dismiss the new information.
A better response is to use Bayes’ theorem: the lack of recent warming is evidence against recent global warming predictions, but it is weak evidence. This is because there is enough variability in global temperatures to make such an outcome unsurprising. The new information should reduce our confidence in our models of global warming – but only a little.
The same approach can be used in anything from an economic forecast to a hand of poker, and while Bayes’ theorem can be a formal affair, Bayesian reasoning also works as a rule of thumb. We tend to either dismiss new evidence, or embrace it as though nothing else matters. Bayesians try to weigh both the old hypothesis and the new evidence in a sensible way. …