Underestimating the Density of the Fog

The story has essentially attained the level of holy writ, at least to those committed to data and evidence, such that it now seems almost too good to be true. The quick-and-dirty version of the tale is that stats geeks with computers, like those former player and broadcaster Tim McCarver called “drooling eggheads,” outsmarted and outmaneuvered the stupid yet arrogant “traditional baseball men” who ran our most traditional sport at the professional level and who thought they knew all there was to know about the game. Thus, it is said, everything the old-time baseball men thought they knew about evaluating players and teams has been found wanting, not that those whose day has passed, committed to wizardry and witchcraft as they are, have recognized it.

This revolution – as shocking as it has been comprehensive – is said to have brought about the ultimate revenge of the nerds. The geeks now run the baseball show, having moved the level of analytical precision involved in running teams and evaluating players from zero-to-sixty in a flash. The new breed of “baseball men” aren’t grizzled scouts looking for “five tool guys” but, rather, Ivy League educated experts in computer modelling and statistical analysis who use those skills to determine who to scout, who to sign, who to play and how to play. The prevailing narrative describes this new contingent as dominating professional baseball at every level, down to the depths of the independent minor leagues.

Is the analytics overhaul of baseball proper as complete and comprehensive as the telling claims? No. The real story is much more interesting and enlightening than that.

Baseball is particularly amenable to the use of statistical analysis because it offers large sample sizes, discrete individual-performance measures (such as plate appearances, pitches, and the like), and ease of identifying positive results (such as winning, home runs, and the like). However, when humans are involved – and baseball is as human as can be – interpretation of the underlying data is highly complicated.

Great interpretation of difficult data sets, especially those involving human behavior, involves more sculpting than tracing. It requires great skill, imagination and even a bit of whimsy as well as collaboration as to whether the various interpretive choices are acceptable (not to say the right) ones. That’s why we understand reality better with respect to the natural sciences than in the social sciences. As ever, information is cheap but meaning is expensive.

To lead off, let’s recall that if it seems too good to be true, it usually is. To see what I mean by that in the context of our story will require some in-depth analysis of its own, starting with more than a bit of background information and history. 

Advertisement

Mean Reversion, Small Sample Size and the Mets

Nobody saw it coming before the season and even well into the season (see, e.g., here, here and here). But, as of tonight, the New York Mets will be playing in the World Series for the first time in 15 years. Sports is so entertaining in large part because the combination of luck, skill and variation involved creates an unpredictable stew of drama, intrigue, greatness and disappointment. Today, that stew smells a lot like the Mets and, so far, it smells pretty tasty.

While their pitching has been sublime this postseason and is the team’s overall key to success, Met second baseman Daniel Murphy’s bat is the big story.

Prior to this postseason, Murphy was a good-but-not-great player whose career highlight was being a reserve All-Star selection in 2014. But this postseason has been one for the ages and the record books to this point thanks to his having homered in six straight games and having hit .421. We’re talking about a player the owner of the American League champion Kansas City Royals recently called “the reincarnation of Babe Ruth” but who never previously hit five home runs in a month.

Smart Investing is Reality-Based

Barry Ritholz has published one of my pieces at his terrific blog, The Big Picture, today. Here’s a taste.

When I was a kid in the 1960s, obsessing over baseball players and stats, plenty of people were telling me to question everything, but the implicit (and erroneous) suggestion was that I reject everything. Instead, I suggest honoring the past without being bound by it. Sabermetrics doesn’t eliminate the need for old-fashioned scouting. Consistent with Robert Hagstrom’s idea that investing is the last liberal art, we should always explore and learn, combine thoughts from multiple sources and disciplines, try to think nimbly because the need for new approaches is ongoing, and we should test and re-test our ideas. I think that idea applies to baseball too.

If we are going to succeed, we’re going to have to ask questions and keep asking questions. Data won’t give us all the answers, but all of our good, objective answers — upon which we should build our investment (and baseball) beliefs — will be consistent with the data. Thus our processes should be data-driven at every point. Smart investing is reality-based. As James Thurber (and later Casey Stengel) would have it, “You could look it up.”

I hope you’ll read it all (and it’s a lot).

Smart Investing is Reality-Based

Bringing the Stupid

Earlier this week I wrote about the growth of the sports analytics movement, particularly in baseball, as an inevitable outgrowth of trying to make one’s beliefs data-driven and thus reality-based. “Arguments and beliefs that are not reality-based are bound to fail, and to fail sooner rather than later.” I also noted that the investment world needs similar growth and development.

But despite the exponential growth in the use of analytics (earlier this year the Phillies became the last Major League Baseball team to hire a full-blown stats guy), not everyone in baseball (as with investing) has gotten the message. For example, The Book goes into great detail about the percentages and when it makes sense to execute a sacrifice bunt and finds — conclusively — that sacrifice bunts are grossly overused and rarely make sense. Simply going back over previous games so as (a) to calculate how many runs each team scored when it had a runner on first and nobody out, and (b) to compare those results to when teams had a runner on second and one out, makes the general point pretty well. In fact, Baseball Prospectus has a report that shows that sacrifice bunting reduced a team’s run expectancy for innings that played out that way from .83 runs to .64 runs in 2013. The same is generally true in previous years.

Note too that we’re not talking here about difficult concepts or high-level math. It’s just a simple calculation — more teams scored and scored more with runners on first and nobody out by not-bunting than by bunting. Easy.

But not to Texas Rangers manager Ron Washington. Washington likes to sacrifice bunt. In response to a question about the dubious nature of that strategy based upon the math, Washington took offense.

“I think if they try to do that, they’re going to be telling me how to [bleep] manage,” Washington said. “That’s the way I answer that [bleep] question. They can take the analytics on that and shove it up their [bleep][bleep].”

Wow. He even went third person.

“I do it when I feel it’s necessary, not when the analytics feel it’s necessary, not when you guys feel it’s necessary, and not when somebody else feels it’s necessary. It’s when Ron Washington feels it’s necessary. Bottom line.”

Double wow. No matter one’s chosen ideology, if the data doesn’t support it, the risks of continuing on that path are enormous. It’s not the percentage play (by definition). And it’s not the smart play.

Insisting upon a course of action that is shown to be wrong is, quite simply, a recipe for disaster. A good investment strategy, like good baseball strategy, will be – must be – supported by the data. Reality must rule. When it doesn’t, we’re simply bringing the stupid.

Get Real

Investment Belief #2: Smart Investing is Reality-Based

Anytime is a good time to talk baseball. I’ve done it pretty much my whole life. If you’re watching a game, its pace is perfectly conducive to discussing (arguing about) players, managers, strategy, tactics, the standings, the pennant races, the quality of ballpark peanuts, and pretty much anything else. In the off-season, the “hot stove league” allows for myriad possible conversations (arguments) about how to make one’s favorite team better. And now that spring training camps have opened, baseball talk about the upcoming season and its prospects has officially begun again in earnest.  The coming of Spring means the return of hope — maybe this will finally be the year (Go Padres!) — which of course means talking (arguing) about it.

Our neighborhood quarrels about the National Pastime when I was a kid were incessant and invigorating, and didn’t have to include the vagaries of team revenues and revenue-sharing, player contracts, free agency and the luxury tax, as they do now. We could focus on more important stuff. Who should be the new catcher? Who should we trade for? Do we have any hot phenoms? Who’s the best player? The best pitcher? The best hitter? The best third baseman? Who belongs in the Hall of Fame? Which team will win at all this year? How do the new baseball cards look? Is the new Strat-O-Matic edition out yet?

Early on, my arguments were rudimentary and, truth be told, plenty stupid. They were ideological (the players on my team were always better than those on your team), authority-laden (“The guy in the paper says…”), narrative-driven (“Remember that time…”), overly influenced by the recent (“Did you see what Jim Northrup did last night?”) and loaded with confirmation bias.

Quickly I came to realize that it’s really hard to change an entrenched opinion, and not just because I was arguing with dopes. Slowly it became clear that if I wanted to have at least a chance of winning my arguments, I needed to argue for a position that was reality-based. I needed to bring facts, data and just-plain solid evidence to the table if I wanted to make a reasonable claim to being right, much less of convincing anyone. Arguments and beliefs that are not reality-based are bound to fail, and to fail sooner rather than later.

Worth Reading

He was my favorite player. He was the greatest right-handed pitcher of the modern baseball era: twelve-time All Star; three-time Cy Young Award winner; 311-game winner; 2.86 career ERA; more than 3,000 strikeouts; recipient of the highest-ever percentage of votes, 98.84, of any Hall of Fame member. I proudly own many of his baseball cards (I used to own his rookie card — thanks, Mom), including one that is autographed. Take some time and read this wonderful and insightful article on Tom Seaver by Pat Jordan. A few morsels are offered below.

“When Tom was pitching for the White Sox, as he was approaching his 300th win, he was warming up in the bullpen before a game. Dave Duncan, the pitching coach, watched him throw. He shook his head and said, ‘Tom, you don’t have shit.’ Tom said, ‘Yeah, so what?’ Tom pointed to the other team’s dugout and said, ‘They don’t know that! So what’s the problem? By the time they find out, it’ll be too late.'”

I especially love this one.

“Big Willie McCovey, waiting at the plate, the bat on his shoulder. Tom said, ‘This is my all-time favorite moment in baseball. I managed to get 3-2 on Willie, and all of a sudden, it came to me. I’m in my stretch, and I keep checking the runner on first. Now, everyone knows the runner can’t go anywhere, the fucking bases are loaded, so what is Seaver doing? I keep checking him, refusing to make eye contact with Willie, throwing him a little birdseed, getting him to think, “What the fuck is Seaver doing?” I wanted him to be anxious, confused.’

“He stopped talking. I blurted out, ‘So what happened!’

“‘I struck him out on a changeup. Twenty years later, we’re at the Hall of Fame, and Willie says to me, “Tom, why the hell did you throw me a changeup in that game?'”

“‘Why did you?’ I said. ‘You broke your own rule.’

“‘That’s the point, big boy. Everyone knew how methodical I was. How this pitch had to be a fastball. I was Tom Seaver. So this time, I went on instinct.’ He looked at me with a grin. ‘Even Tom Seaver has to acknowledge mystery in life.'”

Don’t miss it. 

The Constant Gardiner

Squadouche

“Expert” forecasting is generally terrible. Beyond terrible, actually. As if it were needed, here is some more evidence to add to the huge, steaming pile that already exists.

ESPN employed 43 alleged baseball experts who predicted the 2013 play-off teams, division winners, league champions and World Series champions. These are all people who are well paid to follow baseball closely — all day, every day — in order to report on it. You’ll recognize many of the names. 

How did they do? Dreadfully. Of course.

Of the 43, eight thought the Cardinals would win their division and 10 more thought that the Cards would get to the one-game play-in as a wildcard team. But none had St. Louis in the World Series. Not a single one. But that’s a really, really good record compared to what they thought about the Red Sox.

No “expert” thought that Boston would win the American League East. Four of them thought the Sox would squeak into the post-season as a wildcard team. Not a single one thought that the Red Sox would even make the Fall Classic, much less win it. None.

Think about that the next time you consider putting Series serious money to work based upon a market or economic forecast. Not one of 43 full-time, highly paid “experts” called for either the Cardinals or the Red Sox to be in the World Series.

Zero. Zip. Nada. Squadouche.

“Just your basic 8-2-5 double play”

The Red Sox dispatched the Cardinals last night to win the World Series. It was also Tim McCarver’s swan song as a big-league broadcaster — a career that spanned three decades. You can see his touching farewell here. McCarver has gotten more than his share of (often deserved) criticism in recent years, but I remember his early work, particularly as a local Mets broadcaster on Channel 9 in the 1980s, quite fondly.

My favorite McCarver memory is from August 27, 1986. For some reason I was up (very) late to see a West Coast Mets match-up with the San Diego Padres. Ironic now, I know.

The Mets built a 5-0 lead for Dwight Gooden before the bullpen blew it and the game went into extra innings. Tony Gwynn had three hits, threw out 3 baserunners, stole two bases and made an unbelievable diving catch at the wall in right. Darryl Strawberry hit his 100th career home run. Goose Gossage pitched for the Padres and took the loss. Besides Hall-of-Famers Gwynn and Gossage, Steve Garvey was also in the San Diego line-up that night. The 1986 Mets were, of course, to go on and win 108 regular season games, the National League pennant in an epic NLCS versus the Houston Astros and the World Series in a famous 7-game stand-off with the Red Sox.

In the 11th inning, Gossage — in his third inning of work — gave up two singles and then a sacrifice fly to Keith Hernandez, giving the Mets a lead. Garry Templeton led off the bottom of the 11th with a double for the Friars. Pitcher Craig Lefferts hit for Gossage and struck out. But Tim Flannery came up and ripped a base hit to center. Enjoy McCarver’s call of the game’s very unusual ending.

“Just your routine double play.”

The full bottom of the 11th follows.

“Just your basic 8-2-5 double play.”

Enjoy your retirement, Tim.

Game 6

Game 6 of the World Series is tonight in Boston.  I was in the stands down the third base line for this World Series Game 6 at Shea Stadium in Flushing, Queens, New York on October 25,1986 as the Mets held off elimination by the Red Sox. It’s a wonderful memory and was a birthday gift from my lovely bride. Play ball!

Overselling Competitive Balance

The New York Times has a piece out trumpeting competitive balance in baseball and highlighting the fact that “the 2013 baseball playoffs include more teams from the bottom 10 in payrolls than the top 10.” Big spenders include the Dodgers, Red Sox and Tigers while the Pirates, Indians, Rays and Athletics are play-off teams despite being relative penny-pinchers. The Yankees, perennial spending leaders with a payroll in the range of $230 million this season, are watching at home while the Rays (spending very roughly a quarter of that amount) are still playing. The conclusion from the Times: “Market size matters, but not as much as shrewd management.” That analysis isn’t quite wrong, but it is more than a bit incomplete with respect to the data and leaves out a crucial element to the story: luck.