The Difference Between Prodigals and Misers

Gustave Flaubert, the keen observer of human nature, looked at the hidden motives that lead people to act in accordance with their nature.

This passage from The Letters of Gustave Flaubert is worth reflecting on.

From the idiot who wouldn’t give a sou to redeem the human race, to the man who dives beneath the ice to rescue a stranger, do we not all seek, according to our various instincts, to satisfy our natures? Saint Vincent de Paul obeyed an appetite for charity, Caligula an appetite for cruelty. Everyone takes his enjoyment in his own way and for himself alone. Some direct all activity toward themselves, making themselves the cause, the center, the end of everything; others invite the whole world to the banquet of their souls. That is the difference between prodigals and misers: the first take their pleasure in giving, the second in keeping.

Commenting on this passage in Happiness: A Philosopher’s Guide, Frederic Lenoir believes it “describes the core of egotism that underlies the pursuit of our aspirations and the realization of our actions.”

Mental Model: Regression to the Mean

regression to the mean

It is important to minimize instances of bad judgment and address the weak spots in our reasoning. Learning about regression to the mean can help us.

Nobel prize-winning psychologist Daniel Kahneman wrote a book about biases that cloud our reasoning and distort our perception of reality. It turns out there is a whole set of logical errors that we commit, because our intuition and brains do not deal well with simple statistics. One of the errors that he examines in Thinking Fast and Slow is the infamous regression to the mean.

The notion of regression to the mean was first worked out by Sir Francis Galton. The rule goes that, in any series with complex phenomena that are dependent on many variables, where chance is involved, extreme outcomes tend to be followed by more moderate ones.

In Seeking Wisdom, Peter Bevelin offers the example of John, who was dissatisfied with the performance of new employees so he put them into a skill-enhancing program where he measured the employees’ skill:

Their scores are now higher than they were on the first test. John’s conclusion: “The skill-enhancing program caused the improvement in skill.” This isn’t necessarily true. Their higher scores could be the result of regression to the mean. Since these individuals were measured as being on the low end of the scale of skill, they would have shown an improvement even if they hadn’t taken the skill-enhancing program. And there could be many reasons for their earlier performance — stress, fatigue, sickness, distraction, etc. Their true ability perhaps hasn’t changed.

Our performance always varies around some average true performance. Extreme performance tends to get less extreme the next time. Why? Testing measurements can never be exact. All measurements are made up of one true part and one random error part. When the measurements are extreme, they are likely to be partly caused by chance. Chance is likely to contribute less on the second time we measure performance.

If we switch from one way of doing something to another merely because we are unsuccessful, it’s very likely that we do better the next time even if the new way of doing something is equal or worse.

Peter Bevelin Regression to the Mean

This is one of the reasons it’s dangerous to extrapolate from small sample sizes, as the data might not be representative of the distribution. It’s also why James March argues that the longer someone stays in their job, “the less the probable difference between the observed record of performance and actual ability.” Anything can happen in the short run, especially in any effort that involves a combination of skill and luck. (The ratio of skill to luck also impacts regression to the mean.)

 

***

Regression to the Mean

The effects of regression to the mean can frequently be observed in sports, where the effect causes plenty of unjustified speculations.

In Thinking Fast and Slow, Kahneman recalls watching men’s ski jump, a discipline where the final score is a combination of two separate jumps. Aware of the regression to the mean, Kahneman was startled to hear the commentator’s predictions about the second jump. He writes:

Norway had a great first jump; he will be tense, hoping to protect his lead and will probably do worse” or “Sweden had a bad first jump and now he knows he has nothing to lose and will be relaxed, which should help him do better.

Kahneman points out that the commentator had noticed the regression to the mean and come up with a story for which there was no causal evidence (see narrative fallacy). This is not to say that his story could not be true. Maybe, if we measured the heart rates before each jump, we would see that they are more relaxed if the first jump was bad. However, that’s not the point. The point is, regression to the mean happens when luck plays a role, as it did in the outcome of the first jump.

The lesson from sports applies to any activity where chance plays a role. We often attach explanations of our influence over a particular process to the progress or lack of it.

In reality the science of performance is complex, situation dependent and often much of what we think is within our control is truly random.

In the case of ski jumps, a strong wind against the jumper will lead to even the best athlete showing mediocre results. Similarly, a strong wind and ski conditions in favor of a mediocre jumper may lead to a considerable, but temporary bump in his results. These effects, however, will disappear once the conditions change and the results will regress back to normal.

This can have serious implications for coaching and performance tracking. The rules of regression suggest that when evaluating performance or hiring, we must rely on track records more than outcomes of specific situations. Otherwise we are prone to be disappointed.

When Kahneman was giving a lecture to Israeli Air Force about psychology of effective training, one of the officers shared his experience that extending praise to his subordinates led to worse performance, whereas scolding led to an improvement in subsequent efforts. As a consequence he had grown to be generous with negative feedback and had become rather wary of giving too much praise.

Kahneman immediately spotted that it was regression to the mean at work. He illustrated the misconception by a simple exercise you may want to try yourself. He drew a circle on a blackboard and then asked the officers one by one to throw a piece of chalk at the center of the circle with their backs facing the blackboard. He then repeated the experiment and recorded each officer’s performance in the first and second trial.

Naturally, those that did incredibly well in the first try tended to do worse in their second try and vice versa. The fallacy immediately became clear: the change in performance occurs naturally. That again is not to say that feedback does not matter at all – maybe it does, but the officer had no evidence to conclude it did.

***

The Imperfect Correlation and Chance

At this point you might be wondering why the regression to the mean happens and how we can make sure we are aware of it when it occurs.

In order to understand regression to the mean we must first understand correlation.

The correlation coefficient between two measures which varies between -1 and 1, is a measure of the relative weight of the factors they share. For example, two phenomena with few factors shared, such as bottled water consumption versus suicide rate, should have a correlation coefficient of close to 0. That is to say, if we looked at all countries in the world and plotted suicide rates of specific year against per capita consumption of bottled water, the plot would show no pattern at all.

no correlation
No Correlation

On the contrary, there are measures which are solely dependent on the same factor. A good example of this is temperature. The only factor determining temperature – velocity of molecules — is shared by all scales, hence each degree in Celsius will have exactly one corresponding value in Fahrenheit. Therefore temperature in Celsius and Fahrenheit will have a correlation coefficient of 1 and the plot will be a straight line.

Perfect Correlation
Perfect Correlation

There are few if any phenomena in human sciences that have a correlation coefficient of 1. There are, however, plenty where the association is weak to moderate and there is some explanatory power between the two phenomena. Consider the correlation between height and weight, which would land somewhere between 0 and 1. While virtually every 3 year old will be lighter and shorter than every grown man, not all grown men or three year olds of the same height will weigh the same.

Weak Correlation
Weak to Moderate Correlation

This variation and the corresponding lower degree of correlation implies that, while height is generally speaking a good predictor, there clearly are factors other than height at play. When the correlation of two measures is less than perfect, we must watch out for the effects of regression to the mean.

Kahneman observed a general rule: Whenever the correlation between two scores is imperfect, there will be regression to the mean.

This at first might seem confusing and not very intuitive, but the degree of regression to the mean is directly related to the degree of correlation of the variables. This effect can be illustrated with a simple example.

Assume you are at a party and ask why it is that highly intelligent women tend to marry men who are less intelligent than they are. Most people, even those with some training in statistics, will quickly jump in with a variety of causal explanations ranging from avoidance of competition to the fears of loneliness that these females face. A topic of such controversy is likely to stir up a great debate.

Now, what if we asked why the correlation between the intelligence scores of spouses is less than perfect? This question is hardly as interesting and there is little to guess – we all know this to be true. The paradox lies in the fact that the two questions happen to be algebraically equivalent. Kahneman explains:

[…] If the correlation between the intelligence of spouses is less than perfect (and if men and women on average do not differ in intelligence), then it is a mathematical inevitability that highly intelligent women will be married to husbands who are on average less intelligent than they are (and vice versa, of course). The observed regression to the mean cannot be more interesting or more explainable than the imperfect correlation.

Assuming that correlation is imperfect, the chances of two partners representing the top 1% in terms of any characteristic is far smaller than one partner representing the top 1% and the other – the bottom 99%.

***

The Cause, Effect and Treatment

We should be especially wary of the regression to the mean phenomenon when trying to establish causality between two factors. Whenever correlation is imperfect, the best will always appear to get worse and the worst will appear to get better over time, regardless of any additional treatment. This is something that the general media and sometimes even trained scientists fail to recognize.

Consider the example Kahneman gives:

Depressed children treated with an energy drink improve significantly over a three-month period. I made up this newspaper headline, but the fact it reports is true: if you treated a group of depressed children for some time with an energy drink, they would show a clinically significant improvement. It is also the case that depressed children who spend some time standing on their head or hug a cat for twenty minutes a day will also show improvement.

Whenever coming across such headlines it is very tempting to jump to the conclusion that energy drinks, standing on the head or hugging cats are all perfectly viable cures for depression. These cases, however, once again embody the regression to the mean:

Depressed children are an extreme group, they are more depressed than most other children—and extreme groups regress to the mean over time. The correlation between depression scores on successive occasions of testing is less than perfect, so there will be regression to the mean: depressed children will get somewhat better over time even if they hug no cats and drink no Red Bull.

We often mistakenly attribute a specific policy or treatment as the cause of an effect, when the change in the extreme groups would have happened anyway. This presents a fundamental problem: how can we know if the effects are real or simply due to variability?

Luckily there is a way to tell between a real improvement and regression to the mean. That is the introduction of the so-called control group, which is expected to improve by regression alone. The aim of the research is to determine whether the treated group improve more than regression can explain.

In real life situations with the performance of specific individuals or teams, where the only real benchmark is the past performance and no control group can be introduced, the effects of regression can be difficult if not impossible to disentangle. We can compare against industry average, peers in the cohort group or historical rates of improvement, but none of these are perfect measures.

***

Luckily awareness of the regression to the mean phenomenon itself is already a great first step towards a more careful approach to understanding luck and performance.

If there is anything to be learned from the regression to the mean it is the importance of track records rather than relying on one-time success stories. I hope that the next time you come across an extreme quality in part governed by chance you will realize that the effects are likely to regress over time and will adjust your expectations accordingly.

Regression to the Mean is a Farnam Street Mental Model.

The Books That Influenced John Kenneth Galbraith

PX 82-5: JKG 1961 Portrait

John Kenneth “Ken” Galbraith, was a long-time Harvard faculty member. By long time, I mean he was a professor of economics for over half a century. A prolific author, with about 4 dozen books to his name, he also published more than a thousand articles and essays. Among his most famous works was the trilogy on economics: American Capitalism (1952), The Affluent Society (1958), and The New Industrial State (1967).

In The Harvard Guide to Influential Books: 113 Distinguished Harvard Professors Discuss the Books That Have Helped to Shape Their Thinking, we can find the books that influenced him.

In the preface to his brief response, he writes:

I do not urge economics; others will do that. Instead I urge the enjoyments and enlightenment to which the well-seasoned economist and citizen of the future are entitled and which have brought both pleasure and reward to me in the past.

Then he goes on to name the books that gave him enjoyments and enlightenment in point form.

By Anthony Trollope:
Barchester Towers
The Last Chronicles of Barset
The Warden

Decline and Fall by Evelyn Waugh

By W. Somerset Maugham:
Of Human Bondage
Christmas Holiday

Gullible’s Travels by Ring Lardner

A Farewell to Arms by Ernest Hemingway

The Naked and the Dead by Norman Mailer

By Paul Scott:
The Jewel in the Crown
The Day of the Scorpion
The Towers of Silence
The Division of the Spoils

The Deptford Trilogy by Robertson Davies

Follow your curiosity, for more in this series check out the books that influenced E. O. Wilson, B. F. Skinner, Thomas C. Shelling, Michael J. Sandel, Jerome Kagan, and Stephen Jay Gould.

Ten Pairs of Opposite Traits That Creative People Exhibit

Traits of creativity

This beautiful excerpt from Mihaly Csikszentmihalyi’s Creativity: Flow and the Psychology of Discovery and Invention beautifully illustrates why it’s so hard to pin down creativity and creative people. His book passes the Lindy test — it was written many years ago, which is incredible in today’s world of pop psychology.

Are there no traits that distinguish creative people? If I had to express in one word what makes their personalities different from others, it would be complexity. They show tendencies of thought and action that in most people are segregated. They contain contradictory extremes – instead of being an ‘individual’, each of them is a ‘multitude’. These qualities are present in all of us, but usually we are trained to develop only one pole of the dialectic. We might grow up cultivating the aggressive, competitive side of our nature, and disdain or repress the nurturant, cooperative side. A creative individual is more likely to be both aggressive and cooperative, either at the same time or at different times, depending on the situation. Having a complex personality means being able to express the full range of traits that are potentially present in the human repertoire.

  1. Creative individuals have a great deal of physical energy, but they are also often quiet and at rest.
  2. Creative individuals tend to be smart, yet also naive at the same time.
  3. A third paradoxical trait refers to the related combination of playfulness and discipline, or responsibility and irresponsibility.
  4. Creative individuals alternate between imagination and fantasy at one end, and a rooted sense of reality at the other.
  5. Creative people seem to harbor opposite tendencies on the continuum between extroversion and introversion.
  6. Creative individuals are also remarkably humble and proud at the same time.
  7. Creative individuals to a certain extent escape this rigid gender role stereotyping [of ‘masculine’ and ‘feminine’].
  8. Creative people are both traditional and conservative and at the same time rebellious and iconoclastic.
  9. Creative persons are very passionate about their work, yet they can be extremely objective about it as well.
  10. The openness and sensitivity of creative individuals often exposes them to suffering and pain yet also a great deal of enjoyment.

An Extraordinary Birthday Present (Plus a Free Re:Think Innovation Ticket)

So I turn 36 this weekend.

Ask anyone that knows me and they’ll tell you I’m difficult — or even impossible — to buy presents for. Which isn’t true. I love wine, books, and memberships. But most of all, I love it when people do something nice for someone they’ll never meet. 

In the past, we’ve given to the Ottawa food bank and purchased over $10,000 in books for schools that need them. Together we’ve done some amazing things.

Now I want to do something I’ve never done on my birthday.

In lieu of gifts, my birthday wish this year is to raise $1,000 supporting something near and dear to my heart: Education.

Why? Because the best way out of poverty is literacy.

I want to give back to the most in need, most impoverished schools. And I want your help.

What if it were your kids going to these schools? 

I’ll sweeten the pot.

Here’s how to get it:

  • Spread the word however you can (Twitter, Facebook, Megaphone, …). Send people to this post or the Donorschoose.org page.
  • Leave a comment below telling me how you spread the word  (Measurement of any type gets huge bonus points. Comment must be put up no later than 11:59pm EST on Sunday, July 12th, 2015.)
  • Lastly, answer one question at the very top of your comment: “What does literacy mean to you?” Put “#LiteracyMeans” at the very top, followed by your answer. This is an IQ test in following directions, as I’ll skip entries without #LiteracyMeans at the top.

I’ll pick the winner on Monday July 13th. You must be over 18, void where prohibited, no aliens, etc.

The best reason of all is that you’ll feel awesome. Even if it’s $1 it would mean the world to me.

Again, here is where you can donate $36, $1, $1,000, or whatever you can.

(This unusual birthday present isn’t my idea, I stole it from Tim Ferriss. When people have better ideas, don’t fight it. Just adopt them.)

Simple Rules: How to Thrive in a Complex World

Simple Rules

“Simple rules are shortcut strategies that save time and effort by focusing our attention and simplifying the way we process information. The rules aren’t universal— they’re tailored to the particular situation and the person using them.”

We use simple rules to guide decision making every day. In fact, without them, we’d be paralyzed by the sheer mental brainpower required to sift through the complicated messiness of our world. You can think of them as heuristics. Like heuristics, most of the time they work yet some of the time they don’t.

Simple Rules: How to Thrive in a Complex World, a book by Donald Sull and Kathleen Eisenhardt, explores the understated power that comes from using simple rules. As they define them, simple rules refer to “a handful of guidelines tailored to the user and the task at hand, which balance concrete guidance with the freedom to exercise judgment.” These rules “provide a powerful weapon against the complexity that threatens to overwhelm individuals, organizations, and society as a whole. Complexity arises whenever a system— technical, social, or natural— has multiple interdependent parts.”

They work, the authors argue, because they do three things well.

First, they confer the flexibility to pursue new opportunities while maintaining some consistency. Second, they can produce better decisions. When information is limited and time is short, simple rules make it fast and easy for people, organizations, and governments to make sound choices. They can even outperform complicated decision-making approaches in some situations. Finally, simple rules allow the members of a community to synchronize their activities with one another on the fly.

Effective simple rules share four common traits …

First, they are limited to a handful. Capping the number of rules makes them easy to remember and maintains a focus on what matters most. Second, simple rules are tailored to the person or organization using them. College athletes and middle-aged dieters may both rely on simple rules to decide what to eat, but their rules will be very different. Third, simple rules apply to a well-defined activity or decision, such as prioritizing injured soldiers for medical care. Rules that cover multiple activities or choices end up as vague platitudes, such as “Do your best” and “Focus on customers.” Finally, simple rules provide clear guidance while conferring the latitude to exercise discretion.

***
Simple Rules for a Complex World

People often attempt to address complex problems with complex solutions. For example, governments tend to manage complexity by trying to anticipate every possible scenario that might arise, and then promulgate regulations to cover every case.

Consider how central bankers responded to increased complexity in the global banking system. In 1988 bankers from around the world met in Basel, Switzerland, to agree on international banking regulations, and published a 30-page agreement (known as Basel I). Sixteen years later, the Basel II accord was an order of magnitude larger, at 347 pages, and Basel III was twice as long as its predecessor. When it comes to the sheer volume of regulations generated, the U.S. Congress makes the central bankers look like amateurs. The Glass-Steagall Act, a law passed during the Great Depression, which guided U.S. banking regulation for seven decades, totaled 37 pages. Its successor, Dodd-Frank, is expected to weigh in at over 30,000 pages when all supporting legislation is complete.

Meeting complexity with complexity can create more confusion than it resolves. The policies governing U.S. income taxes totaled 3.8 million words as of 2010. Imagine a book that is seven times as long as War and Peace, but without any characters, plot points, or insight into the human condition. That book is the U.S. tax code.

[…]

Applying complicated solutions to complex problems is an understandable approach, but flawed. The parts of a complex system can interact with one another in many different ways, which quickly overwhelms our ability to envision all possible outcomes.

[…]

Complicated solutions can overwhelm people, thereby increasing the odds that they will stop following the rules. A study of personal income tax compliance in forty-five countries found that the complexity of the tax code was the single best predictor of whether citizens would dodge or pay their taxes. The complexity of the regulations mattered more than the highest marginal tax rate, average levels of education or income, how fair the tax system was perceived to be, and the level of government scrutiny of tax returns.

***
Overfitting

Simple rules do not trump complicated ones all the time but they work more often than we think. Gerd Gigerenzer is a key contributor in this space. He thinks that simple rules can allow for better decision making.

Why can simpler models outperform more complex ones? When underlying cause-and-effect relationships are poorly understood, decision makers often look for patterns in historical data under the assumption that past events are a good indicator of future trends. The obvious problem with this approach is that the future may be genuinely different from the past. But a second problem is subtler. Historical data includes not only useful signal, but also noise— happenstance correlations between variables that do not reveal an enduring cause-and-effect relationship. Fitting a model too closely to historical data hardwires error into the model, which is known as overfitting. The result is a precise prediction of the past that may tell us little about what the future holds.

Simple rules focus on the critical variables that govern a situation and help you ignore the peripheral ones. Of course, in order to identify the key variables, you need to be operating in your circle of competence. When we pay too much attention to irrelevant or otherwise unimportant information, we fail to grasp the power of the most important ones and give them the weighting they deserve. Simple rules also make it more likely people will act on them. This is something Napoleon intuitively understood.

When instructing his troops, Napoleon realized that complicated instructions were difficult to understand, explain, and execute. So, rather than complicated strategies he passed along simple ones, such as: Attack.

***
Making Better Decisions

The book mentions three types of rules that “improve decision making by structuring choices and centering on what to do (and what not to do): boundary, prioritizing, and stopping rules.

Boundary Rules cover what to do …

Boundary rules guide the choice of what to do (and not do) without requiring a lot of time, analysis, or information. Boundary rules work well for categorical choices, like a judge’s yes-or-no decision on a defendant’s bail, and decisions requiring many potential opportunities to be screened quickly. These rules also come in handy when time, convenience, and cost matter.

Prioritizing rules rank options to help decide which of multiple paths to pursue.

Prioritizing rules can help you rank a group of alternatives competing for scarce money, time, or attention. … They are especially powerful when applied to a bottleneck, an activity or decision that keeps individuals or organizations from reaching their objectives. Bottlenecks represent pinch-points in companies, where the number of opportunities swamps available resources, and prioritizing rules can ensure that these resources are deployed where they can have the greatest impact. In business settings, prioritizing rules can be used to assign engineers to new-product-development projects, focus sales representatives on the most promising customers, and allocate advertising expenditure across multiple products, to name only a few possibilities.

Stopping rules help you learn when to reverse a decision. Nobel Prize-winning economist Herbert Simon argued that we lack the information, time, and mental engine to determine the single best path when faced with a slew of options. Instead we rely on a heuristic to help us stop searching when we find something that’s good enough. Simon called this satisficing. If you think that’s hard, it’s even hard to stop doing something we’re already doing. Yet when it comes to our key investments of time, money, and energy we have to know when to pull the plug.

Sometimes we pursue goals at all costs and ignore our self-imposed stopping rule. This goal induced blindness can be deadly.

A cross-continental team of researchers matched 145 Chicagoans with demographically similar Parisians. Both the Chicagoans and Parisians used stopping rules to decide when to finish eating, but the rules themselves were very different. The Parisians employed rules like “Stop eating when I start feeling full,” linking their decision to internal cues about satiation. The Chicagoans, in contrast, were more likely to follow rules linked to external factors, such as “Stop eating when I run out of a beverage,” or “Stop eating when the TV show I’m watching is over.” Stopping rules that rely on internal cues— like when the food stops tasting good or you feel full— decrease the odds that people eat more than their body needs or even wants.

Stopping rules are particularly critical in situations when people tend to double down on a losing hand.

These three decision rules—boundary, prioritizing, and stopping—help provide guidelines on what to do—”what is acceptable to do, what is more important to do, and what to stop doing.”

***
Doing Things Better

Process rules, in contrast to boundary rules, focus on how to do things better.

Process rules work because they steer a middle path between the chaos of too few rules that can result in confusion and mistakes, and the rigidity of so many rules that there is little ability to adapt to the unexpected or take advantage of new opportunities. Simply put, process rules are useful whenever flexibility trumps consistency.

The most widely used process rule is the how-to rule. How-to rules guide the basics of executing tasks, from playing golf to designing new products. The other process rules, coordination and timing, are special cases of how-to rules that apply in particular situations. Coordination rules center on getting something done when multiple actors— people, organizations, or nations— have to work together. These rules orchestrate the behaviors of, for example, schooling fish, Zipcar members, and content contributors at Wikipedia. In contrast, timing rules center on getting things done in situations where temporal factors such as rhythms, sequences, and deadlines are relevant. These rules set the timing of, for example, when to get up every day and when dragonflies migrate.

While I was skeptical, the book is well worth reading. I suggest you check it out.