Farnam Street helps you make better decisions, innovate, and avoid stupidity.

With over 350,000 monthly readers and more than 88,000 subscribers to our popular weekly digest, we've become an online intellectual hub.

The Limits of Science

There can be no cast-iron guarantee that the cutting-edge science of today will not represent the discredited alchemy of tomorrow. More Intelligent Life asks whether we should be skeptical of science itself and finds many reasons why we should: Journal articles are rarely checked for accuracy, discredited research is regularly cited in support of other research (even after it has been discredited), and, wary of yielding ground to those who think differently scientists are often unwilling to openly report anything that is not 100% in favor of their views. Our own bias to over-rely on authority leads us to be less critical than we should be when presented with anything. 

After publishing, authors who claim something have a huge psychological resistance to change their mind even in the face of new or changing evidence – There is still a Flat Earth Society. Scientists are as likely as anyone to knowingly or unknowingly fall victim to confirmation bias and once they’ve publicly declared something as the “truth” they are as reluctant as anyone to change their mind. (When is the last time you changed your mind?)

From More Intelligent Life:
No group of believers has more reason to be sure of its own good sense than today’s professional scientists. There is, or should be, no mystery about why it is always more rational to believe in science than in anything else, because this is true merely by definition. What makes a method of enquiry count as scientific is not that it employs microscopes, rats, computers or people in stained white coats, but that it seeks to test itself at every turn. If a method is as rigorous and cautious as it can be, it counts as good science; if it isn’t, it doesn’t. Yet this fact sets a puzzle. If science is careful scepticism writ large, shouldn’t a scientific cast of mind require one to be sceptical of science itself? There is no full-blown logical paradox here. If a claim is ambitious, people should indeed tread warily around it, even if it comes from scientists; it does not follow that they should be sceptical of the scientific method itself. But there is an awkward public-relations challenge for any champion of hard-nosed science. When scientists confront the deniers of evolution, or the devotees of homeopathic medicine, or people who believe that childhood vaccinations cause autism—all of whom are as demonstrably mistaken as anyone can be—they understandably fight shy of revealing just how riddled with error and misleading information the everyday business of science actually is. When you paint yourself as a defender of the truth, it helps to keep quiet about how often you are wrong. That fact partly explains why some influential climate scientists today, and the UN’s Intergovernmental Panel on Climate Change, are having a hard time. Wary of yielding any ground to those who think that global warming is some sort of hoax, they have sometimes been mightily unwilling to be open about exaggerations, mistakes and confusions in influential reports about climate change—such as the flawed “Hockey Stick” paper, published in Nature in 1998, which estimated global temperatures over the past 600 years, and has become one of the most cited publications on the topic. This defensiveness has backfired, and the credibility of climatologists has suffered. At the end of her book “Science: A Four Thousand Year History” (2009), Patricia Fara of Cambridge University wrote that “there can be no cast-iron guarantee that the cutting-edge science of today will not represent the discredited alchemy of tomorrow”. This is surely an understatement. If the past is any guide—and what else could be?—plenty of today’s science will be discredited in future. …

Most laymen probably assume that the 350-year-old institution of “peer review”, which acts as a gatekeeper to publication in scientific journals, involves some attempt to check the articles that see the light of day. In fact they are rarely checked for accuracy, and, as a study for the Fraser Institute, a Canadian think-tank, reported last year, “the data and computational methods are so seldom disclosed that post-publication verification is equally rare.” …

In a recent book, “Wrong: Why Experts Keep Failing Us—And How to Know When Not to Trust Them”, David Freedman, an American business and science journalist, does a sobering job of reviewing dozens of studies of ignorance, bias, error and outright fraud in recent academic science. He notes that discredited research is regularly cited in support of other research, even after it has been discredited. Trials of the safety and efficacy of drugs, which are often paid for by pharmaceutical companies, seem to be especially liable to errors of various sorts. That helps to explain why medicines that can do unexpected harm—such as Thalidomide, the sedative which was withdrawn in 1961 after causing deformities in babies, and Vioxx, a painkiller that had been used by 84m people before it was pulled in 2004—make it to the market.

It is perhaps the biases of science reporting in the popular press that produce the most misinformation, especially in medicine. The faintest whiff of a breakthrough treatment for a common disease is news, yet the fact that yesterday’s breakthrough didn’t pan out—which ought to be equally interesting to a seeker after truth—rarely is. When a drug is tested on animals and seems promising, it makes headlines, even though the majority of drugs that pass animal trials never become usable for people. And barely a day goes by without the media exploiting an almost universal misunderstanding of statistics and reporting something that has no relevance to anything. When researchers are said to have found that an effect occurs to a statistically significant degree, this means that it probably isn’t caused by a fluke, not that it is large or definite enough to be useful. …

Happily, there is another way out of the impasse between fallible science and even-more-fallible non-science. The contest is not a zero-sum game: the shortcomings of science do not make it rational to believe cranks instead. It’s a fair bet that many of today’s scientific beliefs are wrong, but only your grandchildren will know which ones, and in the meantime, science is the only game in town.

Continue Reading @ The Economist

This seems like a neat book Wrong: Why Experts Keep Failing Us—And How to Know When Not to Trust Them.