Over 500,000 people visited Farnam Street last month to learn how to make better decisions, create new ideas, and avoid stupid errors. With more than 100,000 subscribers to our popular weekly digest, we've become an online intellectual hub. To learn more about what we do, start here.

The Power of Your Subconscious Mind

We think that we're in control. We believe that our conscious mind directs our thoughts and somehow controls our subconscious mind. We're wrong.

In Richard Restak's The Brain Has a Mind of Its Own:

At the moment of decision we all feel we are acting freely, selecting at will from an infinity of choices. Yet research suggests this sense of freedom may be merely an illusory by-product of the way the human brain operates.

Restak gives the example of reading this essay. You scan the title and a few sentences here and there and eventually make a decision to stop reading or read on. You might then go back to the beginning and start reading, or you might start reading wherever it was in the article when you decided to stop skimming.

“The internal sequence,” Restak writes, “was always thought to be: 1. you make a conscious decision to read; 2. that decision triggers your brain into action; 3. your brain then signals the hands to stop turning pages, focuses the eyes on the paragraph, and so on.”

But this isn't what happens at all. “An inexplicable but plainly measurable burst of activity occurs in your brain prior to your conscious desire to act.”

The subconscious mind controls a lot of what we think and the connections we make. And, of course, our thoughts influence what we do.

In The Thinker's Toolkit, Morgan Jones recalls the story found in David Kahn's The Codebreakers.

Breaking codes in World War II was perhaps the largest big data project ever to happen in the world up until that point. The conscious mind could only do so much. One German cryptanalyst recalled, “You must concentrate almost in a nervous trace when working on a code. It is not often done by conscious effort. The solution often seems to crop up from the subconscious.”

Believing that the conscious mind calls the shots prevents us from understanding ourselves, others, and how to make better decisions to name but a few things.

In Plain Talk, Ken Iverson offers some insight on how to turn these thoughts into practical utility.

“Every manager,” he writes “should be something of a psychologist—what makes people tick, what they want, what they need. And much of what people want and need resides in the subconscious. The job of a manager is to help the people accomplish extraordinary things. And that means shaping a work environment that stimulates people to explore their own potential.”

We place too much emphasis on the conscious mind and not enough on the subconscious one.

Unless you manage your environment, it will manage you. The old question ‘would you rather be the poorest in a wealthy neighborhood or the richest in a poor neighborhood?' is based on how the environment controls our subconscious and our subconscious controls our happiness.

Do Algorithms Beat Us at Complex Decision Making?

Algorithms are all the rage these days. AI researchers are taking more and more ground from humans in areas like rules-based games, visual recognition, and medical diagnosis. However, the idea that algorithms make better predictive decisions than humans in many fields is a very old one.

In 1954, the psychologist Paul Meehl published a controversial book with a boring sounding name: Clinical vs. Statistical Prediction: A Theoretical Analysis and a Review of the Evidence.

The controversy? After reviewing the data, Meehl claimed that mechanical, data-driven algorithms could better predict human behavior than trained clinical psychologists — and with much simpler criteria. He was right.

The passing of time has not been friendly to humans in this game: Studies continue to show that the algorithms do a better job than experts in a range of fields. In Daniel Kahneman's Thinking Fast and Slow, he details a selection of fields which have demonstrated inferior human judgment compared to algorithms:

The range of predicted outcomes has expanded to cover medical variables such as the longevity of cancer patients, the length of hospital stays, the diagnosis of cardiac disease, and the susceptibility of babies to sudden infant death syndrome; economic measures such as the prospects of success for new businesses, the evaluation of credit risks by banks, and the future career satisfaction of workers; questions of interest to government agencies, including assessments of the suitability of foster parents, the odds of recidivism among juvenile offenders, and the likelihood of other forms of violent behavior; and miscellaneous outcomes such as the evaluation of scientific presentations, the winners of football games, and the future prices of Bordeaux wine.

The connection between them? Says Kahneman: “Each of these domains entails a significant degree of uncertainty and unpredictability.” He called them “low-validity environments”, and in those environments, simple algorithms matched or outplayed humans and their “complex” decision making criteria, essentially every time.

***

A typical case is described in Michael Lewis' book on the relationship between Daniel Kahneman and Amos Tversky, The Undoing Project. He writes of work done at the Oregon Research Institute on radiologists and their x-ray diagnoses:

The Oregon researchers began by creating, as a starting point, a very simple algorithm, in which the likelihood that an ulcer was malignant depended on the seven factors doctors had mentioned, equally weighted. The researchers then asked the doctors to judge the probability of cancer in ninety-six different individual stomach ulcers, on a seven-point scale from “definitely malignant” to “definitely benign.” Without telling the doctors what they were up to, they showed them each ulcer twice, mixing up the duplicates randomly in the pile so the doctors wouldn't notice they were being asked to diagnose the exact same ulcer they had already diagnosed. […] The researchers' goal was to see if they could create an algorithm that would mimic the decision making of doctors.

This simple first attempt, [Lewis] Goldberg assumed, was just a starting point. The algorithm would need to become more complex; it would require more advanced mathematics. It would need to account for the subtleties of the doctors' thinking about the cues. For instance, if an ulcer was particularly big, it might lead them to reconsider the meaning of the other six cues.

But then UCLA sent back the analyzed data, and the story became unsettling. (Goldberg described the results as “generally terrifying”.) In the first place, the simple model that the researchers had created as their starting point for understanding how doctors rendered their diagnoses proved to be extremely good at predicting the doctors' diagnoses. The doctors might want to believe that their thought processes were subtle and complicated, but a simple model captured these perfectly well. That did not mean that their thinking was necessarily simple, only that it could be captured by a simple model.

More surprisingly, the doctors' diagnoses were all over the map: The experts didn't agree with each other. Even more surprisingly, when presented with duplicates of the same ulcer, every doctor had contradicted himself and rendered more than one diagnosis: These doctors apparently could not even agree with themselves.

[…]

If you wanted to know whether you had cancer or not, you were better off using the algorithm that the researchers had created than you were asking the radiologist to study the X-ray. The simple algorithm had outperformed not merely the group of doctors; it had outperformed even the single best doctor.

The fact that doctors (and psychiatrists, and wine experts, and so forth) cannot even agree with themselves is a problem called decision making “noise”: Given the same set of data twice, we make two different decisions. Noise. Internal contradiction.

Algorithms win, at least partly, because they don't do this: The same inputs generate the same outputs every single time. They don't get distracted, they don't get bored, they don't get mad, they don't get annoyed. Basically, they don't have off days. And they don't fall prey to the litany of biases that humans do, like the representativeness heuristic.

The algorithm doesn't even have to be a complex one. As demonstrated above with radiology, simple rules work just as well as complex ones. Kahneman himself addresses this in Thinking, Fast and Slow when discussing Robyn Dawes's research on the superiority of simple algorithms using a few equally-weighted predictive variables:

The surprising success of equal-weighting schemes has an important practical implication: it is possible to develop useful algorithms without prior statistical research. Simple equally weight formulas based on existing statistics or on common sense are often very good predictors of significant outcomes. In a memorable example, Dawes showed that marital stability is well predicted by a formula: Frequency of lovemaking minus frequency of quarrels.

You don't want your result to be a negative number.

The important conclusion from this research is that an algorithm that is constructed on the back of an envelope is often good enough to compete with an optimally weighted formula, and certainly good enough to outdo expert judgment. This logic can be applied in many domains, ranging from the selection of stocks by portfolio managers to the choices of medical treatments by doctors or patients.

Stock selection, certainly a “low validity environment”, is an excellent example of the phenomenon.

As John Bogle pointed out to the world in the 1970's, a point which has only strengthened with time, the vast majority of human stock-pickers cannot outperform a simple S&P 500 index fund, an investment fund that operates on strict algorithmic rules about which companies to buy and sell and in what quantities. The rules of the index aren't complex, and many people have tried to improve on them with less success than might be imagined.

***

Another interesting area where this holds is interviewing and hiring, a notoriously difficult “low-validity” environment. Even elite firms often don't do it that well, as has been well documented.

Fortunately, if we take heed of the advice of the psychologists, operating in a low-validity environment has rules that can work very well. In Thinking Fast and Slow, Kahneman recommends fixing your hiring process by doing the following (or some close variant), in order to replicate the success of the algorithms:

Suppose you need to hire a sales representative for your firm. If you are serious about hiring the best possible person for the job, this is what you should do. First, select a few traits that are prerequisites for success in this position (technical proficiency, engaging personality, reliability, and so on). Don't overdo it — six dimensions is a good number. The traits you choose should be as independent as possible from each other, and you should feel that you can assess them reliably by asking a few factual questions. Next, make a list of questions for each trait and think about how you will score it, say on a 1-5 scale. You should have an idea of what you will call “very weak” or “very strong.”

These preparations should take you half an hour or so, a small investment that can make a significant difference in the quality of the people you hire. To avoid halo effects, you must collect the information one at a time, scoring each before you move on to the next one. Do not skip around. To evaluate each candidate, add up the six scores. […] Firmly resolve that you will hire the candidate whose final score is the highest, even if there is another one whom you like better–try to resit your wish to invent broken legs to change the ranking. A vast amount of research offers a promise: you are much more likely to find the best candidate if you use this procedure than if you do what people normally do in such situations, which is to go into the interview unprepared and to make choices by an overall intuitive judgment such as “I looked into his eyes and liked what I saw.”

In the battle of man vs algorithm, unfortunately, man often loses. The promise of Artificial Intelligence is just that. So if we're going to be smart humans, we must learn to be humble in situations where our intuitive judgment simply is not as good as a set of simple rules.

Rich Thinking Versus Poor Thinking

“Thought is the original source of all wealth, all success, all material gain,
all great discoveries and inventions, and of all achievement.”
—Claude M. Bristol

***

One of the most controversial chapters in Brian Tracy’s book, Get Smart!, is “Rich Thinking versus Poor Thinking.”

In that chapter, he shares a series of simple ideas you can learn and apply. While I fundamentally disagree with much of the gross over-simplification, there are veins of excellence that we can use to add to our mental toolkit.

(Pause for a second before we continue. Just to be clear, this isn’t an article about going from zero to a million in a lifetime. No clickbait here. No, this article is about giving you tools you can add to your mental toolbox.)

The Role of Mindset

Best-selling author Og Mandino says:

There are no secrets of success. There are simply timeless truths and universal principles that have been discovered and rediscovered throughout human history. All you have to do is to learn and practice them to enjoy all the success that you could desire.

Sounds a lot like what we’re trying to discover.

Fearing Failure

A lot of us do things not to succeed but to avoid failure. This is what Elon Musk calls the fundamental problem with regulators. Tracy writes:

Because of destructive criticism in early childhood and mistakes they have made as adults, they are paralyzed by the fear of making a mistake, of losing their time or money. Even if they are presented with an opportunity, they go into a form of paralysis.

Their fear of failure causes them to create all kinds of reasons not to take action. They don’t have the time. They can’t make the minimum investment. They don’t have the necessary knowledge and skills. Like a deer caught in the headlights, they are paralyzed by the idea of failure, which causes them to never take any action at all.

As it happens, most fortunes in America were started by the sale of personal services. The people had no money, but they had the ability to work hard, to upgrade their skills, and to become more and more valuable. As a result, more and more doors of opportunity opened up for them.

Fearing Disapproval and Criticism

This relates to our fear of criticism and disapproval, which results in approval-seeking behavior. And when we’re seeking approval and acceptance, we’re more likely to think conventionally. And when we think conventionally, we're unlikely to get above-average results.

We don’t want to look different. As a result, we stop learning and growing.

***

“I will study and prepare myself and someday my chance will come.”
— Abraham Lincoln

Tracy writes:

To achieve something you’ve never achieved before, you must learn and practice something that you’ve never done before.

If you’re learning something universal you’ll always have an opportunity to practice what you learn.

Putting all of this together becomes tricky.

Often we have the courage to think and act differently, we mentally prepare ourselves for the critical feedback and then we dip our toe in the water only to find it’s not to our liking.

This is where persistence comes in.

Most of us are simply unwilling to sacrifice in order to succeed. We want our cake and we want to eat it too. Most of the people I know that are incredibly successful have suffered some setback that they had to overcome. A lot of people would have given up. Only they persisted. (Of course, there are plenty of people that persist and fail too.) I’m generalizing a bit here but the people who look for the nearest exit when things get tough are usually the ones with the average results.

Something-For-Something

There is only one type of relationship that is sustainable over a long period of time and that's one where everyone wins. Tracy writes:

Rich people are always looking for ways to create value, to develop and produce products and services that enrich and enhance the lives and work of other people.

They are always willing to put in before they take out. They do not believe in easy money or something for nothing. Rich people believe that you have to justly earn and pay for, in terms of toil and treasure, any rewards and riches that you desire.

Poor people lack this fundamental understanding, the direct relationship between what you put in and what you get out. They are always seeking to get something for nothing or for as little as possible. They want success without achievement, riches without labor, money without effort, and fame without talent.

Poor people gamble, buy lottery tickets, come to work at the last possible moment, waste time while they are there, and then leave work at the first possible minute. They line up by the hundreds and thousands to audition for programs like American Idol, thinking that they can become rich and famous without ever having paid the price necessary to develop the level of talent and ability that enables them to rise above their competitors.

One of the great secrets of becoming wealthy is to always do more than you are paid for. If you do, you will always be paid more than you’re getting today. And there is no other way.

Go the extra mile. Be willing to put in far more than you are taking out. There are never any traffic jams on the extra mile.

Fear can often keep us mediocre. We don’t risk being wrong.

Getting rich isn't as simple as changing your mindset. However changing your mindset can go a long way to changing the way you see the world. And when you see the world differently you can behave and respond differently to the stimuli around you. When you do that, you have the potential to outperform.

Competition, Cooperation, and the Selfish Gene

Richard Dawkins has one of the best-selling books of all time for a serious piece of scientific writing.

Often labeled “pop science”, The Selfish Gene pulls together the “gene-centered” view of evolution: It is not really individuals being selected for in the competition for life, but their genes. The individual bodies (phenotypes) are simply carrying out the instructions of the genes. This leads most people to a very “competition focused” view of life. But is that all?

***

More than 100 years before The Selfish Gene, Charles Darwin had famously outlined his Theory of Natural Selection in The Origin of Species.

We’re all hopefully familiar with this concept: Species evolve over long periods time through a process of heredity, variation, competition, and differential survival.

The mechanism of heredity was invisible to Darwin, but a series of scientists, not without a little argument, had figured it out by the 1970’s: Strands of the protein DNA (“genes”) encoded instructions for the building of physical structures. These genes were passed on to offspring in a particular way – the process of heredity. Advantageous genes were propagated in greater numbers. Disadvantageous genes, vice versa.

The Selfish Gene makes a particular kind of case: Specific gene variants grow in proportion to a gene pool by, on average, creating advantaged physical bodies and brains. The genes do their work through “phenotypes” – the physical representation of their information. As Helena Cronin would put in her book The Ant and the Peacock, “It is the net selective value of a gene's phenotypic effect that determines the fate of the gene.”

This take of the evolutionary process became influential because of the range of hard-to-explain behavior that it illuminated.

Why do we see altruistic behavior? Because copies of genes are present throughout a population, not just in single individuals, and altruism can cause great advantages in those gene variants surviving and thriving. (In other words, genes that cause individuals to sacrifice themselves for other copies of those same genes will tend to thrive.)

Why do we see more altruistic behavior among family members? Because they are closely related, and share more genes!

Many problems seemed to be solved here, and the Selfish Gene model became one for all-time, worth having in your head.

However, buried in the logic of the gene-centered view of evolution is a statistical argument. Gene variants rapidly grow in proportion to the rest of the gene pool because they provide survival advantages in the average environment that the gene will experience over its existence. Thus, advantageous genes “selfishly” dominate their environment before long. It's all about gene competition.

This has led many people, some biologists especially, to view evolution solely through the lens of competition. Unsurprisingly, this also led to some false paradigms about a strictly “dog eat dog” world where unrestricted and ruthless individual competition is deemed “natural”.

But what about cooperation?

***

The complex systems researcher Yaneer Bar-Yam argues that not only is the Selfish Gene a limiting concept biologically and possibly wrong mathematically (too complex to address here, but if you want to read about it, check out these pieces), but that there are more nuanced ways to understand the way competition and cooperation comfortably coexist. Not only that, but Bar-Yam argues that this has implications for optimal team formation.

In his book Making Things Work, Bar-Yam lays out a basic message: Even in the biological world, competition is a limited lens through which to see evolution. There’s always a counterbalance of cooperation.

Counter to the traditional perspective, the basic message of this and the following chapter is that competition and cooperation always coexist. People see them as opposing and incompatible forces. I think that this is a result of an outdated and one-sided understanding of evolution…This is extremely useful in describing nature and society; the basic insight that “what works, works” still holds. It turns out, however, that what works is a combination of competition and cooperation.

Bar-Yam uses the analogy of a sports team which exists in context of a sports league – let’s say the NBA. Through this lens we can see why players, teams, and leagues compete and cooperate. (The obvious analogy is that genes, individuals, and groups compete and cooperate in the biological world.)

In general, when we think about the conflict between cooperation and completion in team sports, we tend to think about the relationships between the players on a team. We care deeply about their willingness to cooperate and we distinguish cooperative “team players” from selfish non-team players, complaining about the latter even when their individual skill is formidable.

The reason we want players to cooperate is so that they can compete better as a team. Cooperation at the level of the individual enables effective competition at the level of the group, and conversely, the competition between teams motivates cooperation between players. There is a constructive relationship between cooperation and competition when they operate at different levels of organization.

The interplay between levels is a kind of evolutionary process where competition at the team level improves the cooperation between players. Just as in biological evolution, in organized team sports there is a process of selection of winners through competition of teams. Over time, the teams will change how they behave; the less successful teams will emulate strategies of teams that are doing well.

At every level then, there is an interplay between cooperation and competition. Players compete for playing time, and yet must be intensively cooperative on the court to compete with other teams. At the next level up, teams compete with each other for victories, and yet must cooperate intensively to sustain a league at all.

They create agreed upon rules, schedule times to play, negotiate television contracts, and so on. This allows the league itself to compete with other leagues for scarce attention from sports fans. And so on, up and down the ladder.

Competition among players, teams, and leagues is certainly a crucial dynamic. But it isn’t all that’s going on: They’re cooperating intensely at every level, because a group of selfish individuals loses to a group of cooperative ones.

And it is the same among biological species. Genes are competing with each other, as are individuals, tribes, and species. Yet at every level, they are also cooperating. The success of the human species is clearly due to its ability to cooperate in large numbers; and yet any student of war can attest to its deadly competitive nature. Similar dynamics are at play with ants, rats, and chimpanzees, among other species of insect and animal. It’s a yin and yang world.

Bar-Yam thinks this has great implications for how to build successful teams.

Teams will improve naturally – in any organization – when they are involved in a competition that is structured to select those teams that are better at cooperation. Winners of a competition become successful models of behavior for less successful teams, who emulate their success by learning their strategies and by selecting and trading team members.

For a business, a society, or any other complex system made up of many individuals, this means that improvement will come when the system’s structure involves a completion that rewards successful groups. The idea here is not a cutthroat competition of teams (or individuals) but a competition with rules that incorporate some cooperative activity with a mutual goal.

The dictum that “politics is the art of marshaling hatreds” would seem to reflect this notion: A non-violent way for competition of cooperative groups for dominance. As would the incentive systems of majorly successful corporations like Nucor and the best hospital systems, like the Mayo Clinic. Even modern business books are picking up on it.

Individual competition is important and drives excellence. Yet, as Bar-Yam points out, it’s ultimately not a complete formula. Having teams compete is more effective: You need to harness competition and cooperation at every level. You want groups pulling together, creating emerging effects where the whole is greater than the sum of the parts (a recurrent theme throughout nature).

You should read his book for more details on both this idea and the concept of complex systems in general. Bar-Yam also elaborated on his sports analogy in a white-paper here. If you're interested in complex systems, check out this post on frozen accidents. Also, for more on creating better groups, check out how Steve Jobs did it.

Focusing is an Art, Not a Science

Productivity is all the rage. People want to get more done in less time. Productivity systems abound: Getting Things Done, Pomodoro, the Seinfeld thing, etc. There’s certainly something to be said for each of them.

But have you thought about something a little simpler and more basic: How to focus? Like, really how to focus your mind on one hard, long project until it’s done?

Productivity systems are great in that they keep you accountable for getting lots of task-oriented work completed. But they don’t answer the larger question, which is: What do you do that creates value in your career? And more than that, what are you doing that’s going to have a cumulative effect, that’s really going to matter years down the road?

I see these two concepts as intertwined and incredibly important, and ignored by overly task-oriented productivity methods.

The first is figuring out where you’re going to create a massive amount of value in your career, The second is figuring out how you’re going to carve out the time and energy to focus deeply on the first.

The thing is, that type of work — whether it’s building a new product, writing a book, learning a hard subject, building a keynote speech, writing a complicated piece of software, whatever — doesn’t happen by saying “I’ll get to it”, and then allocating 15 minutes here or there in between checking your email and going to meetings.

It happens by stringing together sessions of deep, focused effort. Hours at a time, over and over. The intense kind where you sort of lose yourself and wake up later with a lot of awesome work done.

Learning how to do that kind of work, I think, is something of an art.

I say “art” for a reason. I see a lot of people out there promoting their “science-based” system for getting a lot done. Let me tell you something: The word science is being used to fool you and trick you. To make you salivate, Pavlov-style. “Science” is not some monolith that tells you how to create really meaningful work. There’s no “science” of success. There’s no “science” of productivity. That’s pure charlatanism.

Doing great work is an art. A group of researchers can’t answer the complex question of how to live and work correctly; the real world is too varied. We don’t live in a controlled experiment and we’re not lab rats, or worse, college students in psych labs.

Some scientific research papers can certainly give you hints on how the mind works, sure. They might even tell you a few things about information retention and task-based memory. I can see how that might be useful.

But that’s a long way away from creating a career you care about, where you regularly do focused, meaningful work that feels satisfying. Your life is not the one measured in the labs: You’re not trying to memorize flashcards or strings of numbers; what I’m talking about cannot be boiled down to rigorous science. (And anyone who reads Farnam Street knows the deep respect I have for real science.)

No — it’s art! Or more properly, artisanship. And the essence of being an artisan is that it’s deeply personal: It has to speak to you. You must be willing to put your soul into the game. This means everyone will go about the Art of Focus in their own way. It takes experimentation, dedication, and an understanding that no one can do it for you.

I even called a course I put together The Art of Focus, for this very reason. I don’t claim to have all the answers, or to “scientifically” solve your problems or fix your brain, like you’re a mouse in a lab. I just wanted to give people all of the tips and tricks I knew about doing focused, meaningful work, so they could build a system themselves.

Because the truth of the matter is that, however you go about it, you do need to build your capacity for hard, focused work. That is vital in an age of complexity, where we need to carve out a niche. Most of us aren’t making widgets anymore, and much of that work is being replaced by machines anyways.

And if you’ll let me be controversial for a second, I think that’s a good thing for humanity. Humans aren’t meant to live on a factory assembly line (or the white-collar equivalent – spreadsheets and Powerpoint). We’re meant to lose ourselves in valuable and satisfying work that smacks of originality and humanity.

I know a lot of finance people who want to switch into some related craftsmanship, or writing, or software-building, but not the other way around. Do you know any woodworkers who want to switch into finance? Do you know any writers who want to switch into corporate accounting? Me neither.

But in order to build an awesome career doing hard but satisfying long-term work, you need to build your ability to focus for hours at a time. You need to learn hard skills. You need to let go of multitasking, distraction, and the temptation to be “busy.”

I built the Art of Focus to get people started on that path, but I recommend doing it any way you feel comfortable. With apologies to Phil Knight, just do it.

Scientific Concepts We All Ought To Know

John Brockman's online scientific roundtable Edge.org does something fantastic every year: It asks all of its contributors (hundreds of them) to answer one meaningful question. Questions like What Have You Changed Your Mind About? and What is Your Dangerous Idea?

This year's was particularly awesome for our purposesWhat Scientific Term or Concept Ought To Be More Known?

The answers give us a window into over 200 brilliant minds, with the simple filtering mechanism that there's something they know that we should probably know, too. We wanted to highlight a few of our favorites for you.

***

From Steven Pinker, a very interesting thought on The Second Law of Thermodynamics (Entropy). This reminded me of the central thesis of The Origin of Wealth by Eric Beinhocker. (Which we'll cover in more depth in the future: We referenced his work in the past.)


The Second Law of Thermodynamics states that in an isolated system (one that is not taking in energy), entropy never decreases. (The First Law is that energy is conserved; the Third, that a temperature of absolute zero is unreachable.) Closed systems inexorably become less structured, less organized, less able to accomplish interesting and useful outcomes, until they slide into an equilibrium of gray, tepid, homogeneous monotony and stay there.

In its original formulation the Second Law referred to the process in which usable energy in the form of a difference in temperature between two bodies is dissipated as heat flows from the warmer to the cooler body. Once it was appreciated that heat is not an invisible fluid but the motion of molecules, a more general, statistical version of the Second Law took shape. Now order could be characterized in terms of the set of all microscopically distinct states of a system: Of all these states, the ones that we find useful make up a tiny sliver of the possibilities, while the disorderly or useless states make up the vast majority. It follows that any perturbation of the system, whether it is a random jiggling of its parts or a whack from the outside, will, by the laws of probability, nudge the system toward disorder or uselessness. If you walk away from a sand castle, it won’t be there tomorrow, because as the wind, waves, seagulls, and small children push the grains of sand around, they’re more likely to arrange them into one of the vast number of configurations that don’t look like a castle than into the tiny few that do.

The Second Law of Thermodynamics is acknowledged in everyday life, in sayings such as “Ashes to ashes,” “Things fall apart,” “Rust never sleeps,” “Shit happens,” You can’t unscramble an egg,” “What can go wrong will go wrong,” and (from the Texas lawmaker Sam Rayburn), “Any jackass can kick down a barn, but it takes a carpenter to build one.”

Scientists appreciate that the Second Law is far more than an explanation for everyday nuisances; it is a foundation of our understanding of the universe and our place in it. In 1915 the physicist Arthur Eddington wrote:

[…]

Why the awe for the Second Law? The Second Law defines the ultimate purpose of life, mind, and human striving: to deploy energy and information to fight back the tide of entropy and carve out refuges of beneficial order. An underappreciation of the inherent tendency toward disorder, and a failure to appreciate the precious niches of order we carve out, are a major source of human folly.

To start with, the Second Law implies that misfortune may be no one’s fault. The biggest breakthrough of the scientific revolution was to nullify the intuition that the universe is saturated with purpose: that everything happens for a reason. In this primitive understanding, when bad things happen—accidents, disease, famine—someone or something must have wanted them to happen. This in turn impels people to find a defendant, demon, scapegoat, or witch to punish. Galileo and Newton replaced this cosmic morality play with a clockwork universe in which events are caused by conditions in the present, not goals for the future. The Second Law deepens that discovery: Not only does the universe not care about our desires, but in the natural course of events it will appear to thwart them, because there are so many more ways for things to go wrong than to go right. Houses burn down, ships sink, battles are lost for the want of a horseshoe nail.

Poverty, too, needs no explanation. In a world governed by entropy and evolution, it is the default state of humankind. Matter does not just arrange itself into shelter or clothing, and living things do everything they can not to become our food. What needs to be explained is wealth. Yet most discussions of poverty consist of arguments about whom to blame for it.

More generally, an underappreciation of the Second Law lures people into seeing every unsolved social problem as a sign that their country is being driven off a cliff. It’s in the very nature of the universe that life has problems. But it’s better to figure out how to solve them—to apply information and energy to expand our refuge of beneficial order—than to start a conflagration and hope for the best.

Richard Nisbett (a social psychologist) has a great one — a concept we've hit on before but is totally underappreciated by most people: The Fundamental Attribution Error.

Modern scientific psychology insists that explanation of the behavior of humans always requires reference to the situation the person is in. The failure to do so sufficiently is known as the Fundamental Attribution Error. In Milgram’s famous obedience experiment, two-thirds of his subjects proved willing to deliver a great deal of electric shock to a pleasant-faced middle-aged man, well beyond the point where he became silent after begging them to stop on account of his heart condition. When I teach about this experiment to undergraduates, I’m quite sure I‘ve never convinced a single one that their best friend might have delivered that amount of shock to the kindly gentleman, let alone that they themselves might have done so. They are protected by their armor of virtue from such wicked behavior. No amount of explanation about the power of the unique situation into which Milgram’s subject was placed is sufficient to convince them that their armor could have been breached.

My students, and everyone else in Western society, are confident that people behave honestly because they have the virtue of honesty, conscientiously because they have the virtue of conscientiousness. (In general, non-Westerners are less susceptible to the fundamental attribution error, lacking as they do sufficient knowledge of Aristotle!) People are believed to behave in an open and friendly way because they have the trait of extroversion, in an aggressive way because they have the trait of hostility. When they observe a single instance of honest or extroverted behavior they are confident that, in a different situation, the person would behave in a similarly honest or extroverted way.

In actual fact, when large numbers of people are observed in a wide range of situations, the correlation for trait-related behavior runs about .20 or less. People think the correlation is around .80. In reality, seeing Carlos behave more honestly than Bill in a given situation increases the likelihood that he will behave more honestly in another situation from the chance level of 50 percent to the vicinity of 55-57. People think that if Carlos behaves more honestly than Bill in one situation the likelihood that he will behave more honestly than Bill in another situation is 80 percent!

How could we be so hopelessly miscalibrated? There are many reasons, but one of the most important is that we don’t normally get trait-related information in a form that facilitates comparison and calculation. I observe Carlos in one situation when he might display honesty or the lack of it, and then not in another for perhaps a few weeks or months. I observe Bill in a different situation tapping honesty and then not another for many months.

This implies that if people received behavioral data in such a form that many people are observed over the same time course in a given fixed situation, our calibration might be better. And indeed it is. People are quite well calibrated for abilities of various kinds, especially sports. The likelihood that Bill will score more points than Carlos in one basketball game given that he did in another is about 67 percent—and people think it’s about 67 percent.

Our susceptibility to the fundamental attribution error—overestimating the role of traits and underestimating the importance of situations—has implications for everything from how to select employees to how to teach moral behavior.

Cesar Hidalgo, author of what looks like an awesome book, Why Information Grows, wrote about Criticality, which is a very important and central concept to understanding complex systems:

In physics we say a system is in a critical state when it is ripe for a phase transition. Consider water turning into ice, or a cloud that is pregnant with rain. Both of these are examples of physical systems in a critical state.

The dynamics of criticality, however, are not very intuitive. Consider the abruptness of freezing water. For an outside observer, there is no difference between cold water and water that is just about to freeze. This is because water that is just about to freeze is still liquid. Yet, microscopically, cold water and water that is about to freeze are not the same.

When close to freezing, water is populated by gazillions of tiny ice crystals, crystals that are so small that water remains liquid. But this is water in a critical state, a state in which any additional freezing will result in these crystals touching each other, generating the solid mesh we know as ice. Yet, the ice crystals that formed during the transition are infinitesimal. They are just the last straw. So, freezing cannot be considered the result of these last crystals. They only represent the instability needed to trigger the transition; the real cause of the transition is the criticality of the state.

But why should anyone outside statistical physics care about criticality?

The reason is that history is full of individual narratives that maybe should be interpreted in terms of critical phenomena.

Did Rosa Parks start the civil rights movement? Or was the movement already running in the minds of those who had been promised equality and were instead handed discrimination? Was the collapse of Lehman Brothers an essential trigger for the Great Recession? Or was the financial system so critical that any disturbance could have made the trick?

As humans, we love individual narratives. We evolved to learn from stories and communicate almost exclusively in terms of them. But as Richard Feynman said repeatedly: The imagination of nature is often larger than that of man. So, maybe our obsession with individual narratives is nothing but a reflection of our limited imagination. Going forward we need to remember that systems often make individuals irrelevant. Just like none of your cells can claim to control your body, society also works in systemic ways.

So, the next time the house of cards collapses, remember to focus on why we were building a house of cards in the first place, instead of focusing on whether the last card was the queen of diamonds or a two of clubs.

The psychologist Adam Alter has another good one on a concept we all naturally miss from time to time, due to the structure of our mind. The Law of Small Numbers.

In 1832, a Prussian military analyst named Carl von Clausewitz explained that “three quarters of the factors on which action in war is based are wrapped in a fog of . . . uncertainty.” The best military commanders seemed to see through this “fog of war,” predicting how their opponents would behave on the basis of limited information. Sometimes, though, even the wisest generals made mistakes, divining a signal through the fog when no such signal existed. Often, their mistake was endorsing the law of small numbers—too readily concluding that the patterns they saw in a small sample of information would also hold for a much larger sample.

Both the Allies and Axis powers fell prey to the law of small numbers during World War II. In June 1944, Germany flew several raids on London. War experts plotted the position of each bomb as it fell, and noticed one cluster near Regent’s Park, and another along the banks of the Thames. This clustering concerned them, because it implied that the German military had designed a new bomb that was more accurate than any existing bomb. In fact, the Luftwaffe was dropping bombs randomly, aiming generally at the heart of London but not at any particular location over others. What the experts had seen were clusters that occur naturally through random processes—misleading noise masquerading as a useful signal.

That same month, German commanders made a similar mistake. Anticipating the raid later known as D-Day, they assumed the Allies would attack—but they weren’t sure precisely when. Combing old military records, a weather expert named Karl Sonntag noticed that the Allies had never launched a major attack when there was even a small chance of bad weather. Late May and much of June were forecast to be cloudy and rainy, which “acted like a tranquilizer all along the chain of German command,” according to Irish journalist Cornelius Ryan. “The various headquarters were quite confident that there would be no attack in the immediate future. . . . In each case conditions had varied, but meteorologists had noted that the Allies had never attempted a landing unless the prospects of favorable weather were almost certain.” The German command was mistaken, and on Tuesday, June 6, the Allied forces launched a devastating attack amidst strong winds and rain.

The British and German forces erred because they had taken a small sample of data too seriously: The British forces had mistaken the natural clustering that comes from relatively small samples of random data for a useful signal, while the German forces had mistaken an illusory pattern from a limited set of data for evidence of an ongoing, stable military policy. To illustrate their error, imagine a fair coin tossed three times. You’ll have a one-in-four chance of turning up a string of three heads or tails, which, if you make too much of that small sample, might lead you to conclude that the coin is biased to reveal one particular outcome all or almost all of the time. If you continue to toss the fair coin, say, a thousand times, you’re far more likely to turn up a distribution that approaches five hundred heads and five hundred tails. As the sample grows, your chance of turning up an unbroken string shrinks rapidly (to roughly one-in-sixteen after five tosses; one-in-five-hundred after ten tosses; and one-in-five-hundred-thousand after twenty tosses). A string is far better evidence of bias after twenty tosses than it is after three tosses—but if you succumb to the law of small numbers, you might draw sweeping conclusions from even tiny samples of data, just as the British and Germans did about their opponents’ tactics in World War II.

Of course, the law of small numbers applies to more than military tactics. It explains the rise of stereotypes (concluding that all people with a particular trait behave the same way); the dangers of relying on a single interview when deciding among job or college applicants (concluding that interview performance is a reliable guide to job or college performance at large); and the tendency to see short-term patterns in financial stock charts when in fact short-term stock movements almost never follow predictable patterns. The solution is to pay attention not just to the pattern of data, but also to how much data you have. Small samples aren’t just limited in value; they can be counterproductive because the stories they tell are often misleading.

There are many, many more worth reading. Here's a great chance to build your multidisciplinary skill-set.