We Suck at Probabilities

I have often noted (for example, here) that we generally suck at math, to our great detriment.  I have also noted that we are especially poor at dealing with probabilities.  If a weather forecaster says that there is an 80 percent chance of rain and it remains sunny, instead of waiting to see if it rains 80 out of 100 times when his or her forecast called for an 80 percent chance of rain, we race to conclude — perhaps based upon that single instance — that the forecaster isn’t any good. Data trumps our lyin’ eyes, but we don’t routinely see it.

These concepts were addressed earlier this month during a discussion at the New York Public Library (which can be viewed here) between two of the world’s leading thinkers about decision-making under uncertainty, Nassim Taleb and Daniel Kahneman. Taleb is a retired derivatives trader, but has since become famous for his books, The Black Swan: The Impact of the Highly Improbable, Fooled by Randomness: The Hidden Role of Chance in Life and in the Markets, and Antifragile: Things That Gain from Disorder. He is currently Distinguished Professor of Risk Engineering at New York University’s Polytechnic Institute. Kahneman was awarded the 2002 Nobel Memorial Prize in Economic Sciences for his pioneering work in behavioral economics.  Kahneman, together with his late collaborator Amos Tversky, was largely responsible for the growth and development of the study of heuristics and biases, developing prospect theory, demonstrating the impact of framing, and much more.  He is the Eugene Higgins Professor of Psychology and Professor of Public Affairs Emeritus at Princeton University.  His Thinking Fast and Slow is a wonderful retrospective and summary of his life and work.

Throughout the discussion, Kahneman evaluates Taleb’s ideas through the lens of psychology and often highlights the contrasts. When someone in the audience asked why it is so difficult for people generally to compute and deal with probabilities, Kahneman offers an interesting answer. He did not point to innumeracy. Instead, he said, “to compute probabilities you need to keep several possibilities in your mind at once. It’s difficult for most people. Typically, we have a single story with a theme. People have a sense of propensity, that the system is more likely to do one thing than the other, but it’s quite different from the probabilities where you have to think of two possibilities and weigh their relative chances of happening.”

This analysis may well explain much of why models — which simplify, often greatly, of necessity — can be so inadequate, especially with respect to markets, where there are infinite possibilities.  It may also explain, at least in part, why it is so difficult for us to deal with our cognitive and behavioral biases.  Finally, and perhaps most practically for the purposes of this blog’s usual readers, it helps to explain why success in the markets is so hard to achieve.

We prefer to think linearly, manufacturing a storyline, in effect, with a beginning, middle and end.  That’s why we are so susceptible to the “narrative fallacy.”  We inherently prefer stories to data.  Contingencies and (perhaps random) consequences don’t correspond to the way we like to see the world.  We are — pretty much all the time — either looking backward and creating a pattern to fit events and constructing a story that explains what happened along with what caused it to happen, fitting what we see or assume we see into a preconceived narrative, or both.

Dealing effectively with probabilities and the markets requires that we recognize the power of the random and contingent.  No matter how good a story we have concocted with respect to what we expect to happen, no matter how careful our analysis, stuff happens that can and often does mess up and with our hopes, dreams and schemes. The poets sometimes understand these concepts better than those of us who focus on analytics. Per Robert Burns:

The best-laid schemes of Mice and Men
oft go awry,
And leave us nothing but grief and pain,
For promised joy! 

I encourage all of you to take advantage of this excellent broadcast.  I don’t promise you joy, but you won’t suffer grief and pain either.

Advertisement