Data-Driven Difficulties

As regular readers are all too well aware, I am committed to data-driven analysis and investing.  We’re suckers for stories, of course, and are ideological through-and-through, but the goal is to make sure that our investment decisions are based on real, quantitative evidence (at least to the extent possible). 

That’s easier said than done, of course.  We are prone to all sorts of cognitive and behavioral biases — perhaps most prominently confirmation bias — all of which threaten our analysis.  We are also highly susceptible to bias blindness, the inability to see our own biases even when others’ are crystal clear.  And now comes further evidence that our reasoning abilities are even worse than we thought.

A new paper from Yale’s Dan Kahan and colleagues finds that our political views  undermine our most basic analytical skills. More specifically, the study finds that people who are otherwise very good at math often totally screw up a problem that they would otherwise probably be able to solve simply because the correct answer is not politically palatable. Even worse, both liberals and conservatives with especially strong math skills were even more susceptible to letting politics skew their reasoning than were those with less mathematical ability.

So much for the proclaimed information deficit model — the idea that if people had more knowledge or more reasoning ability (a better education) they would be better able to come to agreement on all kinds of difficult or vexing questions relating to the analysis of data. This study is pretty clear evidence that our political biases skew our reasoning abilities and that the problem is even worse for people with advanced capacities. “If the people who have the greatest capacities are the ones most prone to this, that’s reason to believe that the problem isn’t some kind of deficit in comprehension,” Kahan explained in a recent interview.

In essence, Kahan thinks this is confirmation bias working at a deeper level. The study was designed such that a subject’s first instinct is to jump to the wrong conclusion. When that easy (but wrong) answer confirms what they already think — game over.  But when it contradicts a subject’s ideology — and particularly when the subject has high ability — s/he is both equipped and motivated to check the work and find an error.

More than half a century ago, describing a famous case study, Stanford psychologist Leon Festinger described the issue pretty clearly in the opening lines of his book.  “A man with a conviction is a hard man to change. Tell him you disagree and he turns away. Show him facts or figures and he questions your sources. Appeal to logic and he fails to see your point.” Sherlock Holmes put the issue to Watson in The Adventure of Wisteria Lodge thusly: “it is an error to argue in front of your data. You find yourself insensibly twisting them round to fit your theories.” Sadly, it now appears that we can find it remarkably hard to come up with good data in the first place.