Smart Guy but Dumb Conclusion

I grew up near Buffalo and the Bills were my team back in the day.  Even though I haven’t lived in western New York for more than 30 years, I still pay attention to what happens there and have been more than a bit disappointed at the failings of the franchise since the Bills went to four straight Super Bowls in the 1980s (and lost them all).  I was thus intrigued that new Bills president Russ Brandon announced Tuesday that the Bills will create a football analytics operation to help guide football operations there.

It should be noted that Brandon comes from baseball, where such analytical analysis is now pretty routine.  And if you were to think that that sounds just a bit like Moneyball, you’d be right.

For those of you just coming in from a long spell on the north forty — and as I have written before — Moneyball (a book by former Salomon Brothers bond salesman Michael Lewis and subsequently a movie) focuses on the 2002 season of the Oakland Athletics, a team with one of the smallest budgets in Major League Baseball.  At the time, the A’s had lost three of their star players to free agency because they could not afford to keep them.  A’s General Manager Billy Beane, armed with reams of performance and other statistical data, his interpretation of which was rejected by “traditional baseball men” (and also armed with three terrific young starting pitchers), assembled a team of seemingly undesirable players on the cheap that proceeded to win 103 games and the divisional title.

Unfortunately, much of the analysis of Moneyball has focused upon the idea of looking for cheap assets and outwitting the opposition in trading for those assets.  Instead, the crucial lesson of Moneyball relates to finding value via underappreciated assets (some assets are cheap for good reason) by way of a disciplined, data-driven process.  According to Beane, the key to the process is “identifying and using undervalued assets to create and sustain a competitive edge.”  Instead of looking for players based upon subjective factors (a “five-tool guy,” for example) and who perform well according to traditional statistical measures (like RBIs and batting average, as opposed to on-base percentage and OPS, for example), Beane sought out players that he could obtain cheaply because their actual (statistically verifiable) value was greater than their generally perceived value.

The Bills appear to be ready to try to do the same thing.  “We are going to create and establish a very robust football analytics operation that we layer into our entire operation moving forward,” Brandon says. “That’s something that’s very important to me and the future of the franchise.”

Like the A’s, small market teams like the Bills need to find an edge to help them compete with more revenue-rich teams, even though revenue sharing and the NFL salary cap mean that the money disparity is both less marked and less important in football than in baseball.  And to be sure, football is also less conducive to pure analytical analysis of players because it is far less an individual undertaking than baseball.  But all of this is not to say that the sound analysis and application of data can’t help football teams improve.

Just don’t tell that to Bill Polian.  To be sure, Polian is a smart and accomplished guy.  He assembled the four Bills Super Bowl teams and later won an NFL title as president of the Indianapolis Colts.  He was also the guy who ended up picking Peyton Manning over Ryan Leaf (in a surprisingly close call).

“As a practical tool, Moneyball does not work in the NFL,” Polian claims, “because there are very few undervalued players and no middle class because of our salary cap.”

Sadly, Polian utterly misunderstands the lessons of Moneyball and their potential applicability to any and every field of endeavor, including the NFL.  And he isn’t alone.

To begin with, Polian seems to misunderstand what being undervalued means in this context.  Polian seems to think that since there is no real “middle class” in the NFL because the pay levels are so high, no players are undervalued.  Instead, as in Moneyball, good analytical analysis can identify players that cost less than their intrinsic value because the common metrics used to evaluate their performance don’t suggest what they should really be worth.

While it remains to be seen how effective analytics can be in a football context, there is very good reason to believe that remaining skeptical of conventional football wisdom and trying to look at things differently can indeed work.

For example, Tom Brady was a mere sixth round draft choice (199th pick) and then only a clipboard-carrying back-up who became a multiple Super Bowl MVP that married a supermodel, but only after Drew Bledsoe was injured and he got a chance to play.  And as we in San Diego know all too well, for every Peyton Manning there is at least one Ryan Leaf.

Indeed, there has been a surprising amount of analysis done on NFL Draft economics (see here, for example), but this chart from Yale’s Cade Massey tells us pretty much all we need to know for these purposes.


The chart plots where the top six most awarded (via Pro Bowls) and rewarded (with cash salary) players from each draft were selected (it starts with 2005 and goes back to allow player success to shake itself out). According to Massey, “Draft order explains only 31 percent of variation in players’ career starts, 22 percent of free-agent [money], and 9 percent of pro bowls.” It’s “largely a lottery.” As Derek Thompson pointed out for The Atlantic, Massey’s metrics could be better (he doesn’t normalize the numbers to reflect the difference in pay at various positions, for example), but the big picture should be clear.

In every year except 2005, at least three of the top six players were drafted in the first round and in most years, at least four of the top six landed in the first round. But that also means that nearly half of the best players (again, using Massey’s metrics) were not drafted in the first round.  It is thus reasonably clear that NFL GMs do a pretty good job of evaluating talent in the aggregate but make lots of individual mistakes, no doubt for a wide range of reasons.  It also suggests that there is a lot more to be learned about what makes a good football player and what makes a football player good on a particular team and in particular situations.  More and better analytical analysis may be a way to do that. Every sports team — indeed, every business endeavor — no matter how constrained by financial wherewithal or salary caps, should still seek to use its resources as efficiently as possible.

Secondly, Polian’s rejection of analytics out-of-hand reeks of ideology.  It’s entirely possible that analytics won’t be all that helpful in football or that it will turn out to be exceedingly difficult to implement.  It’s even possible (if highly unlikely) that good analytics will confer no useful advantage.  But to reject the concept without examining all of its permutations and ramifications is an ideological commitment rather than an evidence-based decision.  In football — as in the markets — evidence needs to trump ideology to maximize opportunity.

This sort of “old school” rejection of newfangled data analysis reeks of how market-making traders I knew — proud of their instincts and seat-of-the-pants judgments — reacted to and actively resisted the analytical revolution in the markets that began in the 1980s and really ramped up in the 1990s.  It should be no surprise that some of those who benefitted most from the analytical revolution in the markets have gone on to buy sports teams and use similar approaches there (John Henry of the Red Sox and Liverpool FC, for example).

Guys like Bill Polian may have to be dragged — kicking and screaming — into the 21st Century.  Even so, that he still exists and that there are others like him means that Russ Brandon and the Bills have a real shot to succeed.  It won’t be easy, but the opportunity is real.

And that’s a very good thing indeed.  Go Bills.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s