That's the title of a new book by Jim Manzi. Manzi argues that we can take what makes science good at producing reliable knowledge and apply the same principles and methods to business and social policy. By using a randomized controlled trial or, as Manzi calls it, the “randomized field trial,” governments and companies can gain critical insights. Randomized trials, after all, are nothing new. The medical field uses them constantly to judge the effectiveness of drugs and other treatments.
Companies such as Google, eBay, and Amazon are relentless experimenters. These companies, through testing experiments in real-time on millions of consumers, can effectively determine what works and what doesn't.
If you've read In The Plex, you know that Google uses data, not opinion, to guide changes. Google alone, says Mr. Manzi, “ran about 12,000 randomized experiments in 2009, with about 10% leading to business changes.” And It turns out that data is the great cultural equalizer — it makes everyone in the organization, regardless of rank, equal. At Google, you don't have to have an internal debate about where to place the button to maximize use, you just run an experiment and let the data tell you.
Manzi proposes that this approach should be applied to business and social policy. We think we know what works but we don't really know. The catch? You need to insist that every experiment gives you useful information from the results.