By Elizabeth Kolbert
In 1975, researchers at Stanford invited a group of undergraduates to take part in a study about suicide. They were presented with pairs of suicide notes. In each pair, one note had been composed by a random individual, the other by a person who had subsequently taken his own life. The students were then asked to distinguish between the genuine notes and the fake ones.
Some
students discovered that they had a genius for the task. Out of
twenty-five pairs of notes, they correctly identified the real one
twenty-four times. Others discovered that they were hopeless. They
identified the real note in only ten instances.
As
is often the case with psychological studies, the whole setup was a
put-on. Though half the notes were indeed genuine—they’d been obtained
from the Los Angeles County coroner’s office—the scores were fictitious.
The students who’d been told they were almost always right were, on
average, no more discerning than those who had been told they were
mostly wrong.
In the second phase
of the study, the deception was revealed. The students were told that
the real point of the experiment was to gauge their responses to thinking
they were right or wrong. (This, it turned out, was also a deception.)
Finally, the students were asked to estimate how many suicide notes they
had actually categorized correctly, and how many they thought an
average student would get right. At this point, something curious
happened. The students in the high-score group said that they thought
they had, in fact, done quite well—significantly better than the average
student—even though, as they’d just been told, they had zero grounds
for believing this. Conversely, those who’d been assigned to the
low-score group said that they thought they had done significantly worse
than the average student—a conclusion that was equally unfounded.
“Once formed,” the researchers observed dryly, “impressions are remarkably perseverant.”
A
few years later, a new set of Stanford students was recruited for a
related study. The students were handed packets of information about a
pair of firefighters, Frank K. and George H. Frank’s bio noted that,
among other things, he had a baby daughter and he liked to scuba dive.
George had a small son and played golf. The packets also included the
men’s responses on what the researchers called the Risky-Conservative
Choice Test. According to one version of the packet, Frank was a
successful firefighter who, on the test, almost always went with the
safest option. In the other version, Frank also chose the safest option,
but he was a lousy firefighter who’d been put “on report” by his
supervisors several times. Once again, midway through the study, the
students were informed that they’d been misled, and that the information
they’d received was entirely fictitious. The students were then asked
to describe their own beliefs. What sort of attitude toward risk did
they think a successful firefighter would have? The students who’d
received the first packet thought that he would avoid it. The students
in the second group thought he’d embrace it.
Even
after the evidence “for their beliefs has been totally refuted, people
fail to make appropriate revisions in those beliefs,” the researchers
noted. In this case, the failure was “particularly impressive,” since
two data points would never have been enough information to generalize
from.
The Stanford studies became
famous. Coming from a group of academics in the nineteen-seventies, the
contention that people can’t think straight was shocking. It isn’t any
longer. Thousands of subsequent experiments have confirmed (and
elaborated on) this finding. As everyone who’s followed the research—or
even occasionally picked up a copy of Psychology Today—knows,
any graduate student with a clipboard can demonstrate that
reasonable-seeming people are often totally irrational. Rarely has this
insight seemed more relevant than it does right now. Still, an essential
puzzle remains: How did we come to be this way?
In
a new book, “The Enigma of Reason” (Harvard), the cognitive scientists
Hugo Mercier and Dan Sperber take a stab at answering this question.
Mercier, who works at a French research institute in Lyon, and Sperber,
now based at the Central European University, in Budapest, point out
that reason is an evolved trait, like bipedalism or three-color vision.
It emerged on the savannas of Africa, and has to be understood in that
context.
Stripped of a lot of what
might be called cognitive-science-ese, Mercier and Sperber’s argument
runs, more or less, as follows: Humans’ biggest advantage over other
species is our ability to coöperate. Coöperation is difficult to
establish and almost as difficult to sustain. For any individual,
freeloading is always the best course of action. Reason developed not to
enable us to solve abstract, logical problems or even to help us draw
conclusions from unfamiliar data; rather, it developed to resolve the
problems posed by living in collaborative groups.
“Reason
is an adaptation to the hypersocial niche humans have evolved for
themselves,” Mercier and Sperber write. Habits of mind that seem weird
or goofy or just plain dumb from an “intellectualist” point of view
prove shrewd when seen from a social “interactionist” perspective.
Consider
what’s become known as “confirmation bias,” the tendency people have to
embrace information that supports their beliefs and reject information
that contradicts them. Of the many forms of faulty thinking that have
been identified, confirmation bias is among the best catalogued; it’s
the subject of entire textbooks’ worth of experiments. One of the most
famous of these was conducted, again, at Stanford. For this experiment,
researchers rounded up a group of students who had opposing opinions
about capital punishment. Half the students were in favor of it and
thought that it deterred crime; the other half were against it and
thought that it had no effect on crime.
The
students were asked to respond to two studies. One provided data in
support of the deterrence argument, and the other provided data that
called it into question. Both studies—you guessed it—were made up, and
had been designed to present what were, objectively speaking, equally
compelling statistics. The students who had originally supported capital
punishment rated the pro-deterrence data highly credible and the
anti-deterrence data unconvincing; the students who’d originally opposed
capital punishment did the reverse. At the end of the experiment, the
students were asked once again about their views. Those who’d started
out pro-capital punishment were now even more in favor of it; those
who’d opposed it were even more hostile.
If
reason is designed to generate sound judgments, then it’s hard to
conceive of a more serious design flaw than confirmation bias. Imagine,
Mercier and Sperber suggest, a mouse that thinks the way we do. Such a
mouse, “bent on confirming its belief that there are no cats around,”
would soon be dinner. To the extent that confirmation bias leads people
to dismiss evidence of new or underappreciated threats—the human
equivalent of the cat around the corner—it’s a trait that should have
been selected against. The fact that both we and it survive, Mercier and
Sperber argue, proves that it must have some adaptive function, and
that function, they maintain, is related to our “hypersociability.”
Mercier
and Sperber prefer the term “myside bias.” Humans, they point out,
aren’t randomly credulous. Presented with someone else’s argument, we’re
quite adept at spotting the weaknesses. Almost invariably, the
positions we’re blind about are our own.
A
recent experiment performed by Mercier and some European colleagues
neatly demonstrates this asymmetry. Participants were asked to answer a
series of simple reasoning problems. They were then asked to explain
their responses, and were given a chance to modify them if they
identified mistakes. The majority were satisfied with their original
choices; fewer than fifteen per cent changed their minds in step two.
In
step three, participants were shown one of the same problems, along
with their answer and the answer of another participant, who’d come to a
different conclusion. Once again, they were given the chance to change
their responses. But a trick had been played: the answers presented to
them as someone else’s were actually their own, and vice versa. About
half the participants realized what was going on. Among the other half,
suddenly people became a lot more critical. Nearly sixty per cent now
rejected the responses that they’d earlier been satisfied with.
This
lopsidedness, according to Mercier and Sperber, reflects the task that
reason evolved to perform, which is to prevent us from getting screwed
by the other members of our group. Living in small bands of
hunter-gatherers, our ancestors were primarily concerned with their
social standing, and with making sure that they weren’t the ones risking
their lives on the hunt while others loafed around in the cave. There
was little advantage in reasoning clearly, while much was to be gained
from winning arguments.
Among the
many, many issues our forebears didn’t worry about were the deterrent
effects of capital punishment and the ideal attributes of a firefighter.
Nor did they have to contend with fabricated studies, or fake news, or
Twitter. It’s no wonder, then, that today reason often seems to fail us.
As Mercier and Sperber write, “This is one of many cases in which the
environment changed too quickly for natural selection to catch up.”
Steven
Sloman, a professor at Brown, and Philip Fernbach, a professor at the
University of Colorado, are also cognitive scientists. They, too,
believe sociability is the key to how the human mind functions or,
perhaps more pertinently, malfunctions. They begin their book, “The
Knowledge Illusion: Why We Never Think Alone” (Riverhead), with a look
at toilets.
Virtually
everyone in the United States, and indeed throughout the developed
world, is familiar with toilets. A typical flush toilet has a ceramic
bowl filled with water. When the handle is depressed, or the button
pushed, the water—and everything that’s been deposited in it—gets sucked
into a pipe and from there into the sewage system. But how does this
actually happen?
In a study
conducted at Yale, graduate students were asked to rate their
understanding of everyday devices, including toilets, zippers, and
cylinder locks. They were then asked to write detailed, step-by-step
explanations of how the devices work, and to rate their understanding
again. Apparently, the effort revealed to the students their own
ignorance, because their self-assessments dropped. (Toilets, it turns
out, are more complicated than they appear.)
Sloman
and Fernbach see this effect, which they call the “illusion of
explanatory depth,” just about everywhere. People believe that they know
way more than they actually do. What allows us to persist in this
belief is other people. In the case of my toilet, someone else designed
it so that I can operate it easily. This is something humans are very
good at. We’ve been relying on one another’s expertise ever since we
figured out how to hunt together, which was probably a key development
in our evolutionary history. So well do we collaborate, Sloman and
Fernbach argue, that we can hardly tell where our own understanding ends
and others’ begins.
“One
implication of the naturalness with which we divide cognitive labor,”
they write, is that there’s “no sharp boundary between one person’s
ideas and knowledge” and “those of other members” of the group.
This
borderlessness, or, if you prefer, confusion, is also crucial to what
we consider progress. As people invented new tools for new ways of
living, they simultaneously created new realms of ignorance; if everyone
had insisted on, say, mastering the principles of metalworking before
picking up a knife, the Bronze Age wouldn’t have amounted to much. When
it comes to new technologies, incomplete understanding is empowering.
Where
it gets us into trouble, according to Sloman and Fernbach, is in the
political domain. It’s one thing for me to flush a toilet without
knowing how it operates, and another for me to favor (or oppose) an
immigration ban without knowing what I’m talking about. Sloman and
Fernbach cite a survey conducted in 2014, not long after Russia annexed
the Ukrainian territory of Crimea. Respondents were asked how they
thought the U.S. should react, and also whether they could identify
Ukraine on a map. The farther off base they were about the geography,
the more likely they were to favor military intervention. (Respondents
were so unsure of Ukraine’s location that the median guess was wrong by
eighteen hundred miles, roughly the distance from Kiev to Madrid.)
Surveys
on many other issues have yielded similarly dismaying results. “As a
rule, strong feelings about issues do not emerge from deep
understanding,” Sloman and Fernbach write. And here our dependence on
other minds reinforces the problem. If your position on, say, the
Affordable Care Act is baseless and I rely on it, then my opinion is
also baseless. When I talk to Tom and he decides he agrees with me, his
opinion is also baseless, but now that the three of us concur we feel
that much more smug about our views. If we all now dismiss as
unconvincing any information that contradicts our opinion, you get,
well, the Trump Administration.
“This
is how a community of knowledge can become dangerous,” Sloman and
Fernbach observe. The two have performed their own version of the toilet
experiment, substituting public policy for household gadgets. In a
study conducted in 2012, they asked people for their stance on questions
like: Should there be a single-payer health-care system? Or merit-based
pay for teachers? Participants were asked to rate their positions
depending on how strongly they agreed or disagreed with the proposals.
Next, they were instructed to explain, in as much detail as they could,
the impacts of implementing each one. Most people at this point ran into
trouble. Asked once again to rate their views, they ratcheted down the
intensity, so that they either agreed or disagreed less vehemently.
Sloman
and Fernbach see in this result a little candle for a dark world. If
we—or our friends or the pundits on CNN—spent less time pontificating
and more trying to work through the implications of policy proposals,
we’d realize how clueless we are and moderate our views. This, they
write, “may be the only form of thinking that will shatter the illusion
of explanatory depth and change people’s attitudes.”
One
way to look at science is as a system that corrects for people’s
natural inclinations. In a well-run laboratory, there’s no room for
myside bias; the results have to be reproducible in other laboratories,
by researchers who have no motive to confirm them. And this, it could be
argued, is why the system has proved so successful. At any given
moment, a field may be dominated by squabbles, but, in the end, the
methodology prevails. Science moves forward, even as we remain stuck in
place.
In “Denying to the Grave:
Why We Ignore the Facts That Will Save Us” (Oxford), Jack Gorman, a
psychiatrist, and his daughter, Sara Gorman, a public-health specialist,
probe the gap between what science tells us and what we tell ourselves.
Their concern is with those persistent beliefs which are not just
demonstrably false but also potentially deadly, like the conviction that
vaccines are hazardous. Of course, what’s hazardous is not
being vaccinated; that’s why vaccines were created in the first place.
“Immunization is one of the triumphs of modern medicine,” the Gormans
note. But no matter how many scientific studies conclude that vaccines
are safe, and that there’s no link between immunizations and autism,
anti-vaxxers remain unmoved. (They can now count on their side—sort
of—Donald Trump, who has said that, although he and his wife had their
son, Barron, vaccinated, they refused to do so on the timetable
recommended by pediatricians.)
The
Gormans, too, argue that ways of thinking that now seem self-destructive
must at some point have been adaptive. And they, too, dedicate many
pages to confirmation bias, which, they claim, has a physiological
component. They cite research suggesting that people experience genuine
pleasure—a rush of dopamine—when processing information that supports
their beliefs. “It feels good to ‘stick to our guns’ even if we are
wrong,” they observe.
The Gormans
don’t just want to catalogue the ways we go wrong; they want to correct
for them. There must be some way, they maintain, to convince people that
vaccines are good for kids, and handguns are dangerous. (Another
widespread but statistically insupportable belief they’d like to
discredit is that owning a gun makes you safer.) But here they encounter
the very problems they have enumerated. Providing people with accurate
information doesn’t seem to help; they simply discount it. Appealing to
their emotions may work better, but doing so is obviously antithetical
to the goal of promoting sound science. “The challenge that remains,”
they write toward the end of their book, “is to figure out how to
address the tendencies that lead to false scientific belief.”
“The
Enigma of Reason,” “The Knowledge Illusion,” and “Denying to the Grave”
were all written before the November election. And yet they anticipate
Kellyanne Conway and the rise of “alternative facts.” These days, it can
feel as if the entire country has been given over to a vast
psychological experiment being run either by no one or by Steve Bannon.
Rational agents would be able to think their way to a solution. But, on
this matter, the literature is not reassuring. ♦
