I have repeatedly raged against our human failings with respect to all things mathematical and probabilistic (examples are here, here, here, here, here and here). Therefore, I was delighted to see Danica McKellar (best known for playing Winnie Cooper on The Wonder Years, but also featured on favorite shows such as The West Wing and The Big Bang Theory as well as being a math whiz from UCLA) at this past week-end’s Los Angeles Times Festival of Books to promote various methods to help us (and particularly young women) to improve at math. She was encouraging, engaging and even frequently insightful. Danica (or at least Winnie Cooper) is also featured in The New Yorker today. You can check out her books here. I encourage you to do so, especially if you are a young woman or have any young women in your life. The odds are very good that you will be glad you did.
Jack is looking at Anne but Anne is looking at George. Jack is married but George is not. Is a married person looking at an unmarried person?
(a) Yes.
(b) No.
(c) Cannot be determined.
To see the answer and an explanation for it, right-click over the text between the asterisk marks below (I tried to match that text color with the background as closely as I could; the text will “show up” when you do that).
*The answer is (a) but more than 80 percent of people choose (c). You can reach the correct answer only by considering all the possibilities. If Anne is unmarried, Jack is looking at her and if she is married, she is looking at George. This is a very helpful exercise for investing and for life in general. Consider all the possibilities. *
Hurricane/Superstorm Sandy rocked the eastern seaboard last week to devastating effect. In a significant instance of good planning, markets and schools were closed, states of emergency declared and mandatory evacuations begun well before the storm made landfall. Yet nearly until the storm reached land in New Jersey last Monday, I heard lots of grousing about alleged hysteria and overreaction with respect to the precautions and preparations being undertaken to mitigate potential damage (see below for a prominent example).
Some went so far as to defy evacuation orders, and some people paid for doing so with their lives. Once the storm actually hit and caused serious damage – albeit no longer officially as a hurricane, but as a “superstorm” – the complaining stopped. Fortunately, the governmental disaster preparedness organization seems to have performed well overall. You can read about these events in many venues, including here, here and here.
The pre-crisis grousing and the refusal of so many to evacuate are worth thinking about because of what is thereby revealed about us as humans and the cognitive biases that beset us. I offer three “take-away” thoughts that are broadly applicable as well as specifically applicable to the investment world.
1. We don’t deal well with probabilities. When a weather forecast says that there is a 70 percent chance of sun, we tend to think that the forecaster screwed up if it rains. But that’s not how we should evaluate probabilities. Instead, we should consider how often it rains when the forecast calls for a 70 percent chance of sun. When the forecast is spot-on perfect, it will rain 30 percent of the time when it calls for a 70 percent chance of sun. The odds favor sun, but because complex systems like the weather (and financial markets) encompass so many variables, nothing approaching certainty is possible. We don’t handle that kind of thinking very well (a very current and interesting example in a political context is examined here).
To illustrate the level of complexity I’m talking about, consider that we can construct a linear, one-dimensional chain with 10 different links in 3,628,800 different ways. For 100 different links, the possibilities total 10158. If those are the possibilities for making a simple chain, imagine the possibilities when we’re talking about complex systems where wild randomness rules.
Perhaps the key argument of Nobel laureate Daniel Kahneman’s brilliant book, Thinking Fast and Slow, is that without careful and intentional deliberation (and often even then), we suffer from probabilistic irrationality. Remember back in 2009 when New England Patriots coach (and my former New Jersey neighbor) Bill Belichick famously decided to go for a first down on fourth-and-two in Patriots territory rather than punt while up six points late against Peyton Manning and the Indianapolis Colts? When Wes Welker was stopped just short of the first down and the Colts went on to score the winning touchdown, the criticism was overwhelming even though Belichick’s decision gave the Pats a better chance of winning. Those withering attacks simply demonstrate our difficulties with probabilities. Doing what offers the best chance of success in no way guarantees success. As analyst Bill Barnwell, who was agnostic on whether Belichick was right or wrong, wrote: “you can’t judge Belichick’s decision by the fact that it didn’t work” (bold and italics in the original). We can (and should) hope for the best while preparing for the worst.
The world is wildly random. With so many variables, even the best process (when we are able to overcome our probabilistic irrationality) can be undermined at many points, a significant number of which are utterly out of anyone’s control. As Nate Silver reports in his fine new book, The Signal and the Noise, the National Weather Service is extremely good at weather forecasting in a probabilistic sense. When the NWS says there is a 70 percent chance of sun, it’s sunny just about 70 percent of the time. Because we don’t think probabilistically (and crave certainty too), we tend to assume that the forecasts on the days it rains – 30 percent of the time – are wrong. Accordingly, when a probabilistic forecast of a dangerous hurricane is generally inconsistent with our experience (“I didn’t have a problem last time”) and isn’t what we want to hear (think confirmation bias), we can readily focus on the times we remember weather forecasts being “wrong” and discount the threat. As mathematician John Allen Paulos tweeted regarding the trouble that so many seem to have election probabilities:
Many people’s notion of probability is so impoverished that it admits of only two values: 50-50 and 99%, tossup or essentially certain.
In a fascinating research study, economists Emre Soyer and Robin Hogarth showed the results of a regression analysis to a test population of economics professors. When they presented the results in the way most commonly done in economics journals (as a single number accompanied by some error measures), the economists — whose careers are largely predicated upon doing just this sort of analysis! — did an embarrassingly poor job of answering a set of questions about the probabilities of various outcomes. When they presented the results as a scatter graph, the economists got most of the questions right. Yet when they presented the results both ways, the economists got most of the questions wrong again. As Justin Fox emphasizes, there seems to be something about a single-number probability assessment that lures our primitive brains in and leads them astray.
Due to complexity and the wild randomness it entails, the investment world — like weather forecasting — offers nothing like certainty. As every black jack player recognizes, making the “right” play (probabilistically) in does not ensure success. The very best we can hope for is favorable odds and that over a long enough period those odds will play out (and even then only after careful research to establish the odds). That we don’t deal well with probabilities makes a difficult situation far, far worse.
2. We’re prone to recency bias too. We are all prone to recency bias, meaning that we tend to extrapolate recent events into the future indefinitely. Since the recent experience of residents of the eastern seaboard (Hurricane Irene) wasn’t nearly as bad as expected (despite doing significant damage), that experience was extrapolated to the present by many. When confirmation bias (we tend to see what we want to see) and optimism bias are added to the mix, it’s no wonder so many didn’t evaluate storm risk (and don’t evaluate investment risk) very well.
3. We don’t deal well with low probability, high impact events. In the aggregate, hurricanes are low-frequency but high impact events. As I have explained before, when people calculate the risk of hurricane damage and make decisions about hurricane insurance, they consistently misread their prior experience. This conclusion comes from a paper by Wharton Professor Robert Meyer that describes and reports on a research simulation in which participants were instructed that they were owners of properties in a hurricane-prone coastal area and were given monetary incentives to make smart choices about (a) when and whether to buy insurance against hurricane losses and (b) how much insurance to buy.
Over the course of the study (three simulated hurricane “seasons”), participants would periodically watch a map that showed whether a hurricane was building as well as its strength and course. Until virtually the last second before the storm was shown to reach landfall, the participants could purchase partial insurance ($100 per 10 percent of protection, up to 50 percent) or full coverage ($2,500) on the $50,000 home they were said to own. Participants were advised how much damage each storm was likely to cause and, afterward, the financial consequences of their choices. They had an unlimited budget to buy insurance. Those who made the soundest financial decisions were eligible for a prize.
The focus of the research was to determine whether there are “inherent limits to our ability to learn from experience about the value of protection against low-probability, high-consequence events.” In other words — whether experience can help us deal with tail risk. Sadly, we don’t deal with this type of risk management very well. Moreover, as Nassim Taleb has shown, such risks — while still not anything like frequent — happen much more often than we tend to think (which explains why the 2008-09 financial crisis was deemed so highly unlikely by the vast majority of experts and their models).
The bottom line here is that participants seriously under-protected their homes. The first year, they sustained losses almost three times higher than if they had bought protection rationally. The key problem was a consistent failure to buy protection or enough protection even when serious and imminent risk was obvious (sounds like people refusing to evacuate, doesn’t it?). Moreover, most people reduced the amount of protection they bought whenever they endured no damage in the previous round, even if that lack of damage was specifically the result of having bought insurance.
Experience helped a little. Participants got better at the game as season one progressed, but they slipped back into old habits when season two began. By season three, these simulated homeowners were still suffering about twice as much damage as they should have. As Meyer’s paper reports, these research results are consistent with patterns seen in actual practice. For example, the year after Hurricane Katrina there was a 53% increase in new flood-insurance policies issued nationally. But within two years, cancellations had brought the coverage level down to pre-Katrina levels.
We simply don’t do a very good job dealing with low-probability, high-impact events, even when we have experience with them. Since those in the northeast have so little experience with hurricanes, their discounting of hurricane risk is (again) even more understandable. Given what happened to the vast majority of investment portfolios in 2008-09, the alleged market “professionals” often don’t manage tail risk very well either. That said, when a low-frequency event is treated as a certainty or near-certainty as a matter of policy, that overreaction can be disastrous and the costs too high to bear, as a Navy SEAL Commander here in San Diego once took great pains to explain to me in the context of fighting terrorism.
Taleb goes so far as to assert that we should “ban the use of probability.” I disagree, but we ought to use probabilities with care and be particularly careful about how we convey probability assessments. For example, a potential range of outcomes is better than a single number (as with the scatter graphs noted above). Similarly, an outlook that shows the weighing of probabilities together with costs and potential outcomes will also help (this discussion makes a start in that direction). Despite the risks of being perceived as “crying wolf,” we intuitively understand that when and as the potential negative outcomes are greater, lower likelihood events should generally be treated more seriously and that the progression is typically non-linear.
In virtually every endeavor, our cognitive biases are a consistent problem and provide a constant challenge. In terms of investing, they can and often do rock us like a hurricane – or at least a superstorm. As Cullen Roche points out, consistent with the research noted above, we can and should learn from our investment errors, cognitive or otherwise. Sadly, we do so far less often than we ought, as last week’s events amply demonstrate.
For investors, the lessons to be gained here relate to diversification, a carefully delineated and bounded process, clear execution rules, and stick-to-itiveness over the long haul. This doesn’t mean that quants should control everything. Old school analysis and judgment still matter, perhaps more than ever since the pile of available data has gotten so large. But it does mean that our conclusions need to be consistent with and supported by the data, no matter how bizarre the numbers or how long the streak.
Tom Stoppard’s Rosencrantz and Guildenstern are Dead presents Shakespeare’s Hamlet from the bewildered point of view of two of the Bard’s bit players, the comically indistinguishable nobodies who become headliners in Stoppard’s play. The play opens before our heroes have even joined the action in Shakespeare’s epic. They have been “sent for” and are marking time by flipping coins and getting heads each time (the opening clip from the movie version is shown above). Guildenstern keeps tossing coins and Rosencrantz keeps pocketing them. Significantly, Guildenstern is less concerned with his losses than in puzzling out what the defiance of the odds says about chance and fate. “A weaker man might be moved to re-examine his faith, if in nothing else at least in the law of probability.”
The coin tossing streak depicted provides us with a chance to consider these probabilities. Guildenstern offers among other explanations the one mathematicians and investors should favor —“a spectacular vindication of the principle that each individual coin spin individually is as likely to come down heads as tails and therefore should cause no surprise each individual time it does.” In other words, past performance is not indicative of future results.
Even so, how unlikely is a streak of this length?
The probability that a fair coin, when flipped, will turn up heads is 50 percent (the probability of any two independent sequential events both happening is the product of the probability of both). Thus the odds of it turning up twice in a row is 25 percent (½ x ½), the odds of it turning up three times in a row is 12.5 percent (½ x ½ x ½) and so on. Accordingly, if we flip a coin 10 times (one “set” of ten), we would only expect to have a set end up with 10 heads in a row once every 1024 sets {(½)10 = 1/1024}.
Rosencrantz and Guildenstern got heads more than 100 consecutive times. The chances of that happening are: (½)100 = 1/7.9 x 1031. In words, we could expect it to happen once in 79 million million million million million (that’s 79 with 30 zeros after it) sets. By comparison, the universe is about 13.9 billion years old, in which time only about 1017 seconds (1 with 17 zeros after it) have elapsed. Looked at another way, if every person who ever lived (around 110 billion) had flipped a 100-coin set simultaneously every second since the beginning of the universe (again, about 13.9 billion years ago), we could expect all of the 100 coins to have come up heads two times.
If anything like that had happened to you (especially in a bet), you’d agree with Nassim Taleb that the probabilities favor a loaded coin. But then again, while 100 straights heads is less probable than 99, which is less probable than 98, and so on, any exact order of tosses is as likely (actually, unlikely) as 100 heads in a row: (½)100. We notice the unlikelihood of 100 in a row because of the pattern and we are pattern-seeking creatures. More “normal” combinations look random and thus expected. We don’t see them as noteworthy. Looked at another way, if there will be one “winner” selected from a stadium of 100,000 people, each person has a 1 in 100,000 chance of winning. But we aren’t surprised when someone does win, even though the individual winner is shocked.
The point here is that the highly improbable happens all the time. In fact, much of what happens is highly improbable. This math explains why we shouldn’t be surprised when the market remains “irrational” far longer than seems possible. But we are.
Much of that difficulty arises because we neglect the limits of induction. Science never fully proves anything. It analyzes the available data and, when the force of the data is strong enough, it makes tentative conclusions. But these conclusions are always subject to modification or even outright rejection based upon further evidence gathering. Instead, we crave and claim certainty, even when we have no basis for it.
In his brilliant book, On Being Certain, neurologist Robert Burton systematically and convincingly shows that certainty is a mental state, a feeling like anger or pride that can help guide us, but that doesn’t dependably reflect anything like objective truth. One disconcerting finding he describes is that, from a neurocognitive point of view, our feelings of certainty about things we’re right about is largely indistinguishable from our feelings of certainty about things we’re wrong about (think “narrative fallacy” and “confirmation bias”).
As Columbia’s Rama Cont points out, “[w]hen I first became interested in economics, I was surprised by the deductive, rather than inductive, approach of many economists.” In the hard sciences, researchers tend to observe empirical data and then build a theory to explain their observations, while “many economic studies typically start with a theory and eventually attempt to fit the data to their model.” As noted by Emanuel Derman:
In physics it’s fairly easy to tell the crackpots from the experts by the content of their writings, without having to know their academic pedigrees. In finance it’s not easy at all. Sometimes it looks as though anything goes.
I suspect that these leaps of ideological fancy are a natural result of our constant search for meaning in an environment where noise is everywhere and signal vanishingly difficult to detect. Randomness is difficult for us to deal with. We are meaning-makers at every level and in nearly every situation. Yet, as I have noted before, information is cheap and meaning is expensive. Therefore, we tend to short-circuit good process to get to the end result – typically and not so coincidentally the result we wanted all along.
As noted above, science progresses not via verification (which can only be inferred) but by falsification (which, if established and itself verified, provides relative certainty only as to what is not true). Thank you, Karl Popper. In our business, as in science generally, we need to build our investment processes from the ground up, with hypotheses offered only after a careful analysis of all relevant facts and tentatively held only to the extent the facts and data allow. Yet the markets demand action. There is nothing tentative about them. That’s the conundrum we face.
Even after 100 heads in a row, the odds of the next toss being heads remains one-in-two (the “gambler’s fallacy” is committed when one assumes that a departure from what occurs on average or in the long-term will be corrected in the short-term). We look for patterns (“shiny objects”) to convince ourselves that we have found a “secret sauce” that justifies our making big bets on less likely outcomes. In this regard, we are dumber than rats – literally.
In numerous studies (most prominently those by Edwards and Estes, as reported by Philip Tetlock in Expert Political Judgment), the stated task was predicting which side of a “T-maze” holds food for the subject rat. Unbeknownst both to observers and the rat, the maze was rigged such that the food was randomly placed (no pattern), but 60 percent of the time on one side and 40 percent of the time on the other.
The rat quickly “gets it” and waits at the “60 percent side” every time and is thus correct 60 percent of the time. Human observers kept looking for patterns and chose sides in rough proportion to recent results. As a consequence, the humans were right only 52 percent of the time – they (we!) were much dumber than rats. Overall, we insist on rejecting probabilistic strategies that accept the inevitability of randomness and error.
As I described yesterday, the great gambler Billy Walters uses a probabilistic sports betting model that is correct roughly 57 percent of time. He expects and plans for being wrong 43 percent of the time. Since he can’t predict the timing of his successes and failures, he has to be prepared for long losing streaks (although he obviously hopes that none are long as Guildenstern’s). Common gambling practice had been (and often still is) to make fewer bets – to bet only on those games one is most sure of. But that approach is not thinking probabilistically. Walters makes as many bets as he can within the confines of his model (when he thinks the point spread is off by at least one-and-one-half points).
For investors, the lessons to be gained here relate to diversification, a carefully delineated and bounded process, clear execution rules, and stick-to-itiveness over the long haul. This doesn’t mean that quants should control everything. Old school analysis and judgment still matter, perhaps more than ever since the pile of available data has gotten so large. But it does mean that our conclusions need to be consistent with and supported by the data, no matter how bizarre the numbers or how long the streak.
Suppose that you are a contestant on Let’s Make a Deal. You are shown three doors and are told that behind one of them is a car. The other two conceal goats. You choose one of the three doors. Without revealing what is behind the door you chose, Monty Hall has Carol Merrill (I know she left the show a long time ago, but work with me here — she was on the show when I was a kid) open one of the other two doors, revealing one of the goats. Our host now gives you the option of either sticking with your door or of switching to the other unopened door. Which option should you choose to maximize your chances of winning the car?
This issue is called the “Monty Hall problem” (from the title of an analysis of the problem in the journal American Statistician in 1976), and it fools just about everyone. The intuitively obvious answer – that it makes no difference whether you stick or switch – is incorrect. There is actually a big advantage to be gained from switching doors when permitted to do so. The answer is thus a “veridical paradox” because the result appears odd but is demonstrably true nonetheless. You can test it out by playing the game yourself here.
Why should you always switch to the other door? If the car is initially equally likely to be behind each door, a player who picks Door #1 and doesn’t switch has a 1 in 3 chance of winning the car while a player who picks Door #1 but then switches has a 2 in 3 chance. Monty removed an incorrect option from the unchosen doors, so contestants who switch double their chances of winning the car. Put another way, when you stick with your original choice, you’ll win only if your original choice was correct, which happens only 1 in 3 times. If you switch, you’ll win whenever your original choice was wrong, which happens 2 out of 3 times.
Even with this explanation, many refuse to believe that switching is beneficial. After the Monty Hall problem was described and the answer explained in Marilyn vos Savant’s “Ask Marilyn” column in Parade magazine in 1990, approximately 10,000 readers wrote to the magazine claiming that vos Savant was wrong. It’s a common response. Even when given explanations, simulations, and formal mathematical proofs, many people (including many mathematicians and scientists) still do not accept that switching is the best strategy (you can read several funny and arrogant responses from academics to vos Savant here).
The problem can get murkier because Mr. Hall is in control of the game and can play with contestants’ minds. Implicit in the problem’s formulation is the idea that Monty knows where the car is and will always have Carol open a door with a goat behind it after your choice. But he doesn’t really have to. He can even exert psychological pressure to influence your decision.
Suppose your original choice (Door #1) is correct but, understanding the probabilities and not knowing you’re right, you decide to switch doors (to #2) when asked. But further suppose Mr. Hall then offers you money not to switch – perhaps a lot of money. Odds are you’ll still switch even though a bird in the hand should be worth two in the bush. As Monty explains:
“Now do you see what happened there? The higher I got [the more money offered not to switch], the more you thought the car was behind Door 2. I wanted to con you into switching there, because I knew the car was behind 1. That’s the kind of thing I can do when I’m in control of the game. You may think you have probability going for you when you follow the answer in her column, but there’s the psychological factor to consider.
“…If the host is required to open a door all the time and offer you a switch, then you should take the switch,” he said. “But if he has the choice whether to allow a switch or not, beware. Caveat emptor. It all depends on his mood.
“My only advice is, if you can get me to offer you $5,000 not to open the door, take the money and go home.”
As with the markets, our psychological make-up tends to push us to “buy high” and “sell low” — to act against our own interests.
As I noted yesterday, we simply aren’t very good at analyzing probabilities. As Persi Diaconis, a former professional magician who is now a Harvard University professor specializing in probability and statistics puts it, “Our brains are just not wired to do probability problems very well….” This issue is of particular import to investing since it always involves analyzing probabilities in uncertainty.
Bottom line: we need to rely on the data, even when it’s both counterintuitive and difficult.
It is not a smart play to hit on 19 in Blackjack or to draw to an inside straight in Poker because the odds are so heavily against winning. But people do it anyway and sometimes – very occasionally – they succeed.
We all like to think that we wouldn’t do that, and in extreme cases like those I mentioned above we rarely do. Unfortunately, all of us go against the odds far too often – in investing and in life. We all suffer from what is often called neglect of probability bias when making decisions under uncertainty. Note that we don’t misuse probabilities; we disregard them entirely.
The problem, as so often happens, lies in ourselves and with our cognitive biases. For example:
Football coaches and teams “go for it” on fourth down far too seldom.
Lots of people are afraid to fly while few are afraid to ride in a car even though flying is much safer.
Sadly, knowing the facts doesn’t necessarily help (as anyone who read Moneyballor saw the movie realizes). The typical response to the facts isn’t a refutation, it’s a story or anecdote about someone who defied the odds and won. For these people, a single, vivid example completely overrides statistics and probability.
Our brains do many things well. But calculating probabilities isn’t one of them. That flaw colors nearly every debate, argument and policy consideration.
In this experiment involving giving electric shocks to subjects, scientists found people were willing to pay up to $20 to avoid a 99 percent chance of a painful electric shock. On its face, that seems reasonable. However, those same subjects would also be willing to pay up to $7 to avoid a mere 1 percent chance of the same shock. It turned out that the subjects had only the vaguest concept of what the math means and represents. They were pretty much only thinking about the shock.
On the other hand, it is not irrational or unreasonable to protect against unlikely events when the consequences of the unlikely happenings are sufficiently disastrous. That’s the premise behind Nassim Taleb’s terrific book, The Black Swan. In those instances, our math skills aren’t any better, but we under-value the risk.
The bottom line here is that we tend to be lousy at analyzing probabilities. Therefore, since investing is so dependent upon probabilities and likelihoods under uncertainty (there are no sure-things), we’d be well advised to look at the math carefully and be sure we understand it fully before we make decisions and to make sure that our work is checked. Otherwise, chances are we’ll make mistakes and keep making mistakes. If we do, the probabilities say that our chances of ultimate success are greatly (and gravely) diminished.
I have repeatedly raged against our human failings with respect to all things mathematical and probabilistic (examples are here, here, here, here, here and here). Therefore, I was delighted to see Danica McKellar (best known for playing Winnie Cooper on The Wonder Years, but also featured on favorite shows such as The West Wing and The Big Bang Theory as well as being a math whiz from UCLA) at this past week-end’s Los Angeles Times Festival of Books to promote various methods to help us (and particularly young women) to improve at math. She was encouraging, engaging and even frequently insightful. Danica (or at least Winnie Cooper) is also featured in The New Yorker today. You can check out her books here. I encourage you to do so, especially if you are a young woman or have any young women in your life. The odds are very good that you will be glad you did.
