Kobayashi Maru and the Forecasting Follies

In the Star Trek universe, the Kobayashi Maru is a Starfleet Academy training exercise for future officers in the command track. It takes place on a replica of a starship bridge with the test-taker as captain. In the exercise, the cadet and crew receive a distress signal advising that the freighter Kobayashi Maru has stranded in the Klingon Neutral Zone and is rapidly losing power, hull integrity and life support.

The cadet is seemingly faced with a decision (a) to attempt to rescue the freighter’s crew and passengers, which involves violating the Neutral Zone and potentially provoking the Klingons into an all-out war; or (b) to abandon the ship, potentially preventing war but leaving the freighter’s crew and passengers to die. As the simulation is played out, both possibilities are set up to end badly. Either both the starship and the freighter are destroyed by the Klingons or the starship is forced to wait and watch as everyone on the Kobayashi Maru dies an agonizing death.

The objective of the test is not for the cadet to outsmart or outfight the Klingons but rather to examine the cadet’s reaction to a no-win situation. It is ultimately designed as a test of discipline and character under stress.

However, before his third attempt at the test while a student, James T. Kirk surreptitiously reprograms the simulator so that it was possible to rescue the freighter. When questioned later about his ploy, Kirk asserts that he doesn’t believe in no-win scenarios. And he doesn’t like to lose. So he changed the game. Thus for Trekkies, the test’s name is used to describe a no-win scenario as well as a solution that requires that one change the game in order to jerry-rig a solution to the proffered problem.

For would-be market experts, their Kobayashi Maru is a public market target, most often included in an annual market preview publication. It’s an expected part of the gig. Similarly, when a Wall Street strategist, economist or even a run-of-the-mill investment manager or analyst gets a crack at financial television, he or she is routinely asked, often as almost an afterthought, to give a specific target forecast for the market. Instead of thinking like Captain Kirk and wisely objecting to the premise of the question, the poor schlemiel answers and, once matters play out, is shown to have been less than prescient. Indeed, as I often say, one forecast that is almost certain to be correct is that market forecasts are almost certain to be wrong.

Every December I take a look at these predictions for the year that’s ending and they are almost always uniformly lousy. Moreover, when somebody does get one right or almost right, that performance quality is not repeated in subsequent years. That’s because, at best, complex systems – from the weather to the markets – allow only for probabilistic forecasts with very significant margins for error and often seemingly outlandish and hugely divergent potential outcomes. Chaos theory establishes as much. Traditional market analysis has generally failed to grasp the inherent complexity and dynamic nature of the financial markets, which chaotic reality goes a long way towards explaining highly remarkable and volatile outcomes that seem inevitable in retrospect but were predicted by almost nobody.

This year hasn’t been any different. As year-end approaches, let’s once again take a look at how badly various Wall Street market forecasts missed it with their prognostications for the S&P 500 in 2014. What follows is a table of my survey of the 2014 year-end target forecasts for the S&P 500 from 50 investment strategists and money managers. It tracks their “achievements,” using forecasts from the beginning of 2014.

Strategist (Firm)                                          Year-End 2014 S&P 500 Target

Brad McMillan (Commonwealth)                                           1,800
David Joy (Ameriprise)                                                          1,845
David Bianco (Deutsche Bank):                                             1,850
Gina Martin Adams (Wells Fargo)                                         1,850
Barry Bannister (Stifel Nicolaus)                                           1,850
Craig Callahan (ICON)                                                          1,850
Gary Thayer (Tradition Capital Management)                       1,875
Brian Belski (BMO)                                                                1,900
Barry Knapp (Barclays)                                                         1,900
Tobias Levkovich (Citigroup)                                                 1,900
David Kostil (Goldman Sachs)                                               1,900
Kim Forest (Fort Pitt)                                                              1,900
Jack Ablin (BMO)                                                                   1,915
Mark Luschini (Janney Montgomery Scott)                            1,920
Michael Kurtz (Nomura)                                                         1,925
Matt King (Bell)                                                                      1,925
Peter Cardillo (Rockwell)                                                       1,935
Derek Hoyt (KDV)                                                                  1,940
Sean Darby (Jefferies)                                                           1,950
Jonathan Golub (RBC)                                                           1,950
Julian Emanuel (UBS)                                                            1,950
Jeff Weniger (BMO)                                                                1,950
Fred Dickson (D.A. Davidson)                                                1,950
Ben Halliburton (Tradition Capital Management)                    1,950
Cam Albright (Wilmington Trust)                                             1,950
Bob Doll (Nuveen)                                                                   1,950
Andrew Garthwaite (Credit Suisse)                                         1,960
Jim Kee (South Texas Money Management)                          1,970
Joe Tatusko (Westport Resources)                                         1,975
Mike McGarr (Becker)                                                             1,980
Donald Selkin (National Securities)                                         1,980
Dan Greenhaus (BTIG)                                                           1,980
Frank Fantozzi (Planned Financial Services)                          1,995
Savita Subramanian (Bank of America)                                  2,000
Craig Johnson (Piper Jaffray)                                                 2,000
Rob Stein (Astor)                                                                    2,000
John Burke (Raymond James)                                               2,000
Peter Tuz (Chase)                                                                  2,000
Thomas Nyheim (Christiana)                                                  2,000
Oliver Pursche (Gary Goldberg Financial)                              2,000
Brian Jacrobsen (Wells Fargo)                                               2,000
Adam Parker (Morgan Stanley)                                              2,014
John Stoltzfus (Oppenheimer)                                                2,014
Doug Cote (ING)                                                                     2.020
Steven Goldman (Goldman Mgmt)                                         2,025
Brian Peery (Hennessey Funds)                                             2,050
Tom Lee (JP Morgan)                                                             2,075
Philip Orlando (Federated Investors)                                      2,100
Ryan Detrick (Schaeffer’s Investment Research)                   2,100
Byron Wein (Blackstone)                                                        2,200

Median Forecast                                                1,950 (up 6.44 percent)

S&P 500 Actual (through November, 2014)     2067.56 (up 11.86 percent)

Sources: Morningstar & Above the Market

The bottom line is that other than a very few notable exceptions (luck abounds), all the alleged experts missed and missed by a lot (short of a major market move between this writing and the end of the year). Indeed, the S&P 500 (through the end of November) returned 84 percent more than the median expert prediction. Astonishingly, that’s a lot better than last year though still dreadful. For what it’s worth, according to MSN Money, individual investors did even worse. Its annual survey of Main Street retail investors — as opposed to Wall Street pros — came up with an average 2014 forecast that U.S. stocks would decline “at least” 10 percent for the year.

These dreadful results correspond to the “performance” generated by active money managers as a class.* As Jason Zweig reports for The Wall Street Journal, among all mutual funds that invest in large-cap U.S. stocks, only 9.3 percent are beating the index through September 30, according to Denys Glushkov, a senior researcher at Wharton Research Data Services at the University of Pennsylvania. Although the year isn’t over yet, that is well under the previous annual low of 12.9 percent in 1995 and the average of 38.6 percent over the past quarter-century. Data through October 31 from Lipper shows results nearly as dismal. Not surprisingly, Nuveen’s Bob Doll expected active managers to outperform index funds in 2014. Jeff Kleintop (then of LPL) agreed.

Oh well.

It’s no wonder then that Nassim Taleb tells a sardonic story about forecasting. As the story goes, a trader listened to the firm’s chief economist provide a forecast about the markets and then lost a bundle acting on it, getting him fired. The trader angrily asked his boss why he was fired rather than the economist, as the economist’s poor forecast led to the poor trade. The boss replied, “You idiot, I’m not firing you for losing money; I’m firing you for listening to the economist.” Interestingly, there is very good evidence that Wall Street firms really do think that such forecasting is essentially worthless (they keep those ideas to themselves, of course).

To be fair, these dreadful results aren’t at all unusual. Market forecasting has a long and ignominious history. Irving Fisher was a noted 20th century economist. No less an authority than Milton Friedman called him “the greatest economist the United States has ever produced.” But three days before the famous 1929 Wall Street crash he claimed that “stocks have reached what looks like a permanently high plateau.”

Let’s stipulate that these alleged experts are highly educated, vastly experienced, and examine the vagaries of the markets pretty much all day, every day. But it remains a virtual certainty that they will be wrong and often spectacularly wrong.  One can look at many other high-profile stock pickers and find numerous tales of woe (more here). If you think you can predict the future in the markets, think again. Your crystal ball does not work any better than anyone else’s.

Since 1990, the Federal Reserve Bank of Philadelphia has conducted a quarterly Survey of Professional Forecasters, continuing research conducted from 1968-1989 by the American Statistical Association and the National Bureau of Economic Research. The survey asks various economic experts their views of the probabilities of recession for each of the following four quarters and comes up with an “Anxious Index” reflecting those asserted probabilities.

A CXO study of that data determined that the forecasted probability of recession for a quarter explained absolutely none of the stock market’s returns for that quarter. In fact, the data suggests that the forecasts were a mildly (if not materially) contrarian indicator of future U.S. stock market behavior. The survey reads like a primer on recency bias in that bear markets lead to bearish market forecasts and vice versa while the forecasts have no predictive power whatsoever. The lousy track records highlighted by CXO are entirely consistent with a long line of academic research (most prominently from Philip Tetlock) establishing the lack of value provided by so-called “expert” forecasters.

The market predictions offered by experts (and others) and the thought processes underlying them can be very entertaining indeed. They are even the engine that drives much of what pretends to be financial and business news. But none of us should take them seriously. Doing so would be very dangerous indeed. Based upon the historical record (and the 2014 record!), it’s a good thing that so few bother to hold people accountable for their forecasts. If accountability were the norm, who could stand?

_____

* To be clear, I support active management in certain situations, most notably as a means to mitigate risk and to “tilt” a portfolio.  But there is precious little evidence for the idea that active management can generate big alpha on a consistent or persistent basis or that we can somehow predict in advance who those successful alpha-generating managers might be — now or at any other time. Indeed, if I had a dollar for every time some alleged expert claimed that it was “a stock pickers’ market” without the least bit of support before or evidence since, I would have as much money as a big-time hedge fund manager.