Tag: Experience

Garrett Hardin: The Other Side of Expertise

From Garrett Hardin's mind-blowingly awesome Filters Against Folly.

In our highly technological society we cannot do without experts. We accept this fact of life, but not without anxiety. There is much truth in the definition of the specialist as someone who “knows more and more about less and less.” But there is another side to the coin of expertise. A really great idea in science often has its birth as apparently no more than a particular answer to a narrow question; it is only later that it turns out that the ramifications of the answer reach out into the most surprising corners. What begins as knowledge about very little turns out to be wisdom about a great deal.

So it was with the development of the theory of probability. It all began in the seventeenth century, when one of the minor French nobility asked the philosopher-scientist Blaise Pascal to devise a fair way to divide the stakes in an interrupted gambling game. Pascal consulted with lawyer-mathematician friend Pierre de Fermat, and the two of them quickly laid the foundation of probability theory. Out of a trivial question about gambling came profound insights that later bore splendid fruit in physics and biology, in the verification of the causes of disease, the calculation of fair insurance premiums, and the achievement of quality control in manufacturing processes. And much more.

The service of experts is indispensable even if we are poor at ascertaining under which circumstances they add value, when they add noise, and when they are harmful. Hardin cautions that each new expertise introduces “new possibilities of error.”

“It is unfortunately true that experts are generally better at seeing their particular kinds of trees than the forest of all life.”

— Garrett Hardin

Thoughtful laymen — that's us — can, however, “become very good at seeing the forest, particularly if they lose their timidity about challenging the experts. … In the universal role of laymen we all have to learn to filter the essential meaning out of the too verbose, too aggressively technical statements of the experts. Fortunately this is not as difficult a task as some experts would have us believe.”

Filters Against Folly is Hardin's attempt “to show there …. (are) some rather simple methods of checking the validity of the statements of experts.”

Elon Musk on How to Tell if People Are Lying


Great tidbit from Elon Musk at the Ignition Conference on how having job applicants explain their thinking at multiple levels helps him figure out if they really worked on the problem.

If you just talk to the people on your team you can learn a tremendous amount. As you iterate through problems … when you struggle with a problem that's when you understand it. Once you've done that for (years), then you have a pretty good grasp of it. In fact, that's one of the ways, when I interview someone … is to ask them to tell me about the problems they worked on and how they solved them. And if someone was really the person that solved it, they will be able to answer at multiple levels — they will be able to go down to the brass tacks. And if they weren't, they'll get stuck. And then you can say, “oh this person was not really the person who solved it because anyone who struggles hard with a problem never forgets it.”

This connects for me a bit with the false record effect:

A group of managers of identical (moderate) ability will show considerable variation in their performance records in the short run. Some will be found at one end of the distribution and will be viewed as outstanding; others will be at the other end and will be viewed as ineffective. The longer a manager stays in a job, the less the probable difference between the observed record of performance and actual ability. Time on the job increased the expected sample of observations, reduced expected sampling error, and thus reduced the chance that the manager (of moderate ability) will either be promoted or exit.

It's also a good trick for the batesian mimicry problem — that is, separating those who know from those who act like they know.