I occasionally write for outside publications. When those publications are digital iterations of traditional media outlets, their editors tend to be especially vigilant about keeping posts to 400-600 words (as compared with my 4,000 word monstrosity at The Big Picture today). That’s because their research says that few readers will go beyond that length, no matter how good the content (which is a real problem for me since I tend to favor and write long and fairly involved pieces). Accordingly, information can readily be provided. But meaning, which takes careful sifting and evaluation of evidence, must remain rare indeed. Continue reading
The “Semmelweis Reflex” is a metaphor for our reflex-like tendency to reject new evidence or new knowledge because it contradicts our established norms, beliefs or paradigms. It is named for Ignaz Semmelweis, a Hungarian obstetrician who found lasting scientific fame, but only posthumously.
Semmelweis discovered that the often-fatal puerperal fever (“childbed fever”), common among new mothers in hospitals, could essentially be eliminated if doctors simply washed their hands before assisting with childbirth. After observing that a particular obstetrical ward suffered unusually high instances of the disease and that doctors there often worked in the morgue right before aiding in childbirth but had not washed their hands in between, Semmelweis speculated that “cadaverous material” could be passed from doctors’ hands to patients, causing the disease. He thereupon initiated a strict regimen at his hospital whereby all who would assist in a birthing must first wash their hands with a chlorinated solution. As a consequence, death rates plummeted.
Semmelweis expected a revolution in hospital hygiene as a consequence of his findings. But it didn’t come.
In a recent post I made the obvious point that randomness has a very significant impact on investment performance. Of course, we’re all too ready to acknowledge bad luck when things go wrong, but when we succeed we want all the credit. In any event, my friend Cullen Roche also published the post at Pragmatic Capitalism, where it got a comment that I find interesting and deserving of a bit more examination. Continue reading
The Wall Street Journal is promoting what it purports to be a “hot tip” for the upcoming NCAA basketball tournament. According to the Journal’s Jim Chairusmi, second-ranked Duke (full disclosure: I’m a Duke alum) won’t win the national championship this season. Neither will No. 5 Georgetown, No. 6 Michigan or No. 7 Kansas. That’s because all four of these teams had one game this season where they got demolished, and that rarely happens to championship teams. As the Journal points out, only one team since 1994, the 2001-02 Maryland Terps, lost a game by more than 20 points and went on to win the title. Moreover, three of the five most recent NCAA champions went through the season without even losing a game by double digits — third-ranked Indiana and No. 4 Louisville fit that mold this season. If this holds true again, that’s bad news for the Blue Devils, Hoyas, Wolverines and Jayhawks and potentially good news for the Hoosiers or the Cardinals.
Maybe you should fill out your NCAA bracket next week accordingly. But then again, maybe not. Continue reading
As reported by The New York Times, film critic Pauline Kael expressed shock at Richard Nixon’s landslide victory over George McGovern in 1972. “I live in a rather special world. I only know one person who voted for Nixon. Where they are I don’t know. They’re outside my ken. But sometimes when I’m in a theater I can feel them.” Even after the votes were in and counted, Kael wanted to believe her lyin’ eyes. This year, it’s Republicans who have fallen prey to confirmation bias and rejected the data in favor of preconceived ideological commitments and intuition.
Stats wizard Nate Silver has been at the center of a controversy this election season as his data-driven presidential election analysis, outlined at his FiveThirtyEight blog, contradicted the desires of Republicans and pundits who did not want a clear victory for President Obama (albeit for different reasons). Silver created a forecasting model that was uncannily accurate in 2008 (49 of 50 states) and which consistently predicted that President Obama was a clear favorite over Mitt Romney, angering conservatives in the process. When the President won a clear victory last night (the extent of which is still being determined as I write this), Silver’s method and approach were vindicated.
Silver critics such as Politico’s Dylan Byers (“Nate Silver could be a one-term celebrity”), David Brooks of The New York Times (“The pollsters tell us what’s happening now. When they start projecting, they’re getting into silly land”), Morning Joe‘s Joe Scarborough (“Nate Silver says this is a 73.6 percent chance that the president is going to win? Nobody in that campaign thinks they have a 73 percent chance — they think they have a 50.1 percent chance of winning”), The Washington Post’s Michael Gerson (“Silver’s prediction is not an innovation; it is trend taken to its absurd extreme”) and Politico’s Josh Gerstein (“Isn’t the basic problem with the Nate Silver prediction in question, and the critique, that it puts a percentage on a one-off event?”) have all demonstrated that, consistent with my warnings, we simply do not deal with probability very well. More fundamentally, their data-deficient “analysis” has been weighed and found wanting.
With respect to probability, as Silver warned Byers, one shouldn’t confuse prediction with prophecy. As Zeynep Tufekci proclaimed at Wired in his careful defense of Silver, this “isn’t wizardry,” but “the sound science of complex systems.” Accordingly, “[u]ncertainty is an integral part of it. But that uncertainty shouldn’t suggest that we don’t know anything, that we’re completely in the dark, that everything’s a toss-up.” Here’s the key:
What his model says is that currently, given what we know, if we run a gabazillion modeled elections, Obama wins 80 percent of the time…Since we’ll only have one election on Nov. 6, it’s possible that Obama can lose. But Nate Silver’s (and others’) statistical models remain robust and worth keeping and expanding — regardless of the outcome this Tuesday.
Wa-Bam. The probabilities were clear. Governor Romney could have won, but it was unlikely.
With respect to data, Ezra Klein ‘s Wonkblog at the Washington Post offers a detailed defense of quantitative analysis as well as Silver (more here). Had Silver’s model been wrong, it would have been because the underlying polls — lots of them — were wrong. Silver’s model is a sophisticated form of poll valuation and aggregation together with demographic and voting trend analysis.
As my Above the Market masthead proclaims, I believe in and strive to focus on “data-driven analysis.” Because Silver’s work is quintessentially that, it was easy for me to rely upon it in making my prediction of 303 electoral votes for the President (Silver predicted 313). The pundits, however, were all over the map. Data must override ideology, punditry and feelings whether we’re talking about elections, markets or anything else. Data wins. If you want to oppose what the data suggests, it can only be done via better data or better analysis of the data.
To be clear, my prediction (like Silver’s) could have been dramatically wrong. Again, it was based upon data and probabilities rather than certainties. The electorate could have defied the odds (in much the same way that a longshot can win the Super Bowl). Silver, in his fine new book The Signal and the Noise, urges us to “stop and smell the data — slow down, and consider the imperfections in your thinking.” Those of us who work in the markets should do exactly as he suggests.
There is no denying the Republicans have the passion now, the enthusiasm. The Democrats do not. Independents are breaking for Romney. And there’s the thing about the yard signs. In Florida a few weeks ago I saw Romney signs, not Obama ones. From Ohio I hear the same. From tony Northwest Washington, D.C., I hear the same.
Is it possible this whole thing is playing out before our eyes and we’re not really noticing because we’re too busy looking at data on paper instead of what’s in front of us? Maybe that’s the real distortion of the polls this year: They left us discounting the world around us.
Her writing is still lovely but her lyin’ eyes were wrong and that form of punditry (and market analysis) is d-e-a-d.
All of us who work in the financial markets have seen narratives like it thousands of times and said something like it ourselves almost as often. From Alan Abelson in the current Barron’s:
“THE STOCK MARKET TOOK A wicked hit in last week’s early sessions on revived fears that the negotiations might fall through to keep Greece among the living, or at least still a member, however green around the gills, of the European Union. Instead, holders of its bonds and other creditors blinked and Greece seemingly managed to survive sufficiently to warrant another bailout despite its incredible shrinking GDP. Markets the globe over, including our own, natch, breathed a huge sigh of relief.
“While the reprieve afforded by the announcement of the latest last-minute resolution enabled equities to regain a good chunk of the ground lost, the rally was notably lacking in the kind of combustible conviction that generates big trading volume. For one thing, Greece, shmeece, Europe is enmeshed in a recession, hopefully a modest one, but who knows?”
That’s a compelling narrative, but is it actually true? I’ve sat on enough trading desks to know that there are always many reasons why people trade and that those reasons are often unknown and concealed. Obfuscation is common — in all directions. Even flow trading desks never have a great handle on what’s driving whom.
A big trade after a big number might be in response to the number. But it might also have been planned before and delayed to get past the risk of the number. Or it might simply reflect redemptions due to poor performance, or something else.
Back in the days when I was routinely asked about flow and called upon to interpret what’s driving whom on a day-to-day and sometimes moment-to-moment basis, I was careful to talk to traders, read the latest news and collect whatever other information I could before opining. But, at best, the information I offered had to be treated very tentatively.
In essence, that is because information is cheap while meaning is expensive.
The traders I talked with might not have been seeing all the flows. Or they might have been longer (or shorter) than they would have liked and were biasing their commentary accordingly. Or maybe they were distracted about something. Or perhaps they were angry with me for not pushing what they had to sell hard enough and freezing me out from the best intel they had. Maybe they just didn’t care. Everybody has an ax to grind. As a consequence, every interpretive conclusion is extremely tentative – of necessity.
Even so, I still created partial and even overarching narratives routinely, if cautiously. The sales process is largely characterized by organizing the available facts into plausible narratives. Facts don’t exist in a vacuum and they must be interpreted to be actionable. That is the crucial (a-hem) fact behind the power of narrative.
We love stories. They help us to explain, understand and interpret the world around us. They also give us a frame of reference we can use to remember the concepts we take them to represent. Perhaps most significantly, we inherently prefer narrative to data — often to the detriment of our understanding because, unfortunately, our stories are also often steeped in error.
In the context of the markets, as elsewhere, we all like to think that we carefully gather and evaluate facts and data before coming to our conclusions and telling our stories. But we don’t.
Instead, we tend to suffer from confirmation bias and thus reach a conclusion first. Only thereafter do we gather facts, but even so we tend to do so to support our pre-conceived conclusions. We then take our selected “facts” (or thereafter examine any alleged new facts) and cram them into our desired narratives, because narratives are so crucial to how we make sense of reality. Keeping one’s analysis and interpretation of the facts reasonably objective – since, again, analysis and interpretation are required for data to be actionable – is really, really hard even in the best of circumstances.
That difficulty is exacerbated because we simply aren’t very good with data. In this experiment involving giving electric shocks to subjects, scientists found people were willing to pay up to $20 to avoid a 99 percent chance of a painful electric shock. On its face, that seems reasonable. However, those same subjects would also be willing to pay up to $7 to avoid a mere 1 percent chance of the same shock. It turned out that the subjects had only the vaguest concept of what the math means and represents. They were pretty much only thinking about the shock.
Nassim Taleb calls our tendency to create false and/or unsupported stories in an effort to legitimize our pre-conceived notions the “narrative fallacy.” That fallacy threatens our analysis and judgment constantly. Therefore, while we may enjoy the stories and even be aided by them, we should put our faith in the actual data, especially because they are so often in conflict. Our interpretations of the data need to be constantly reevaluated. As John Allen Paulos noted in The New York Times last fall: “There is a tension between stories and statistics, and one under-appreciated contrast between them is simply the mindset with which we approach them. In listening to stories we tend to suspend disbelief in order to be entertained, whereas in evaluating statistics we generally have an opposite inclination to suspend belief in order not to be beguiled.”
How badly are we beguiled? Let’s take a look at some of the data.
We all live in an overconfident, Lake Wobegon world (“where all the women are strong, all the men are good-looking, and all the children are above average”). We are only correct about 80% of the time when we are “99% sure.” Despite the experiences of anyone who has gone to college, fully 94% of college professors believe they have above-average teaching skills. Since 80% of drivers say that their driving skills are above average, I guess none of them are on the freeway when I am. While 70% of high school students claim to have above-average leadership skills, only 2% say they are below average, no doubt taught by above average math teachers. In a finding that pretty well sums things up, 85-90% of people think that the future will be more pleasant and less painful for them than for the average person.
Our overconfident tendencies are well-known of course and obvious in others if not to ourselves (there’s that bias blind spot thing again). Even (especially?!) experts get it wrong far too often.
Milton Friedman called Irving Fisher “the greatest economist the United States has ever produced.” However, Fisher made a statement in 1929 that all but destroyed his credibility for good. Three days before the famous Wall Street crash he claimed that “stocks have reached what looks like a permanently high plateau.”
Sadly, mistakes like that are anything but uncommon. Y2K anyone? Or how about the book by James Glassman and Kevin Hassett forecasting Dow 36,000? Philip Tetlock’s excellent Expert Political Judgment: How Good Is It? How Can We Know? examines why experts are so often wrong in sometimes excruciating detail. Even worse, when wrong, experts are rarely held accountable and they rarely admit it. They insist that they were just off on timing, or blindsided by an impossible-to-predict event, or almost right, or wrong for the right reasons. Tetlock even goes so far as to claim that the better known and more frequently quoted experts are, the less reliable their guesses about the future are likely to be, largely due to overconfidence.
The more we repeat and reiterate one of our explanatory narratives, the harder it is to recognize evidence that ought to cause us to re-evaluate our previous conclusions. By making it a careful habit skeptically to re-think our prior interpretations and conclusions, we at least give ourselves a fighting chance to correct the mistakes that we will inevitably make.
More specific market predictions do not generally fare any better than the general ones. Back in 2000, Fortune magazine picked a group of ten stocks designed to last the then-forthcoming decade and promoted them as a “buy and forget” portfolio of their best ideas. Unfortunately, one who purchased that portfolio would want to forget it. A $100 investment in an equally weighted portfolio of these stocks back then would have been worth about $30 ten years later. There are many similar – even worse – examples. In December of 2005, Fortune (again!) was pitching “10 sturdy stocks” that it claimed were “built to last.” Citigroup at $50 and Washington Mutual at $42 featured prominently. Within two years, both of these stocks had gotten totally crushed. Similar examples are legion (and often embarrassing).
As reported by Bespoke, Bloomberg surveys market strategists on a weekly basis, and along with asking them for their year-end S&P 500 price targets, Bloomberg also asks for their recommended portfolio weightings of stocks, bonds and cash. As of last week, the consensus recommended stock weighting stood at 57%, down significantly from the start of the year. However, these alleged experts are generally wrong. The peak recommended stock weighting came just after the peak of the Internet bubble in early 2001 while the lowest recommended weighting came just after the lows of the financial crisis. Can anyone say “recency bias”? As John Kenneth Galbraith famously pointed out, we have two classes of forecasters: those who don’t know and those who don’t know they don’t know.
My general conclusions:
- Be skeptical about the data you collect, the patterns you detect, your interpretations thereof and the conclusions you draw.
- Keep looking for more data — especially data that questions or contradicts your assumptions, hypotheses and conclusions.
- Be especially skeptical of the story you think the data tells.
- Rinse and repeat #1, #2 and #3 as appropriate (pretty much all the time).
Bear Stearns won a famous 2002 litigation involving former Fed Governor and Bear Chief Economist Wayne Angell over advice he and the firm gave to a Bear Sterns client named Count Henryk de Kwiatowski after the Count lost hundreds of millions of dollars following that advice (backstory here). The jury awarded a huge verdict to the plaintiff but the appellate court reversed. That Court held that brokers cannot be liable for honest opinions that turn out to be wrong when providing advice on non-discretionary accounts.
What is significant for our purposes was a line of testimony offered at trial by then-Bear CEO Jimmy Cayne. Cayne apparently thought that Bear could be in trouble so he took a creative and disarmingly honest position given how aggressive Bear was in promoting Angell’s alleged expertise. Cayne brazenly asserted that Angell was merely an “entertainer” whose advice should never give rise to liability. Economists are right only 35 percent to 40 percent of the time, Cayne testified. “They don’t really have a good record as far as predicting the future,” he said. “I think that it is entertainment, but he probably doesn’t think it is” (and I doubt that the Count was much amused). Cayne even noted that Angell did not have a real job description at Bear. “I don’t know how he spends most of his time,” testified Cayne. “He travels a lot and visits people and has lunches and dinners and he is an entertainer.”
In an odd sense, Cayne was precisely if hypocritically correct. There is nothing wrong with using or being assisted by a good story. But stories aren’t facts and should never be treated and relied upon as such, entertaining as they are.
Elsewhere: The Value of Data Visualization: When Will People Finally Start Getting This Stuff? (Core77); Infographics: Here’s How to Make Them Work Best (The PR Coach); The Anatomy of an Infographic: 5 Steps To Create A Powerful Visual (SpyreStudios); The Dos and Don’ts of Infographic Design (Smashing Magazine).