Optimism Bias at Work

forecastESPN has 32 team bloggers, and each has predicted the 2014 season record for the team he or she (they aren’t all guys) covers. They are remarkably positive on the chances for each team next season, forecasting that, in the aggregate, NFL teams will win far more games than they lose, quite impossibly going 290-222 (as reported by The Big Lead). Again. Last year, also as reported by The Big Lead, they collectively predicted that NFL teams would go 283-229.

It’s easy to assume that we’re witnessing the collective math or probability suckage that so easily beset us. But that would be an erroneous conclusion since ESPN is careful to inform us that each team’s blogger made his or her prediction independently and only made a guess as to his or her covered team. Instead, what we’re seeing is an obvious example of how bad we are at forecasting and how susceptible we are to optimism bias. In general, it’s a lot more fun — and you get a lot more readers — when covering a winning rather than a losing team).

Remarkably, only five teams are forecast as having a losing record this year, with Washington (at 7-9) the only sub-.500 team in the entire NFC. Three teams in the powerful NFC West are expected to go 12-4. My Chargers are even expected to go 10-6 (don’t we wish). But at least the blogger for the Cowboys seems to have (finally?!) learned his lesson as what was once America’s Team is predicted to go 8-8 (yet again). And, happily, the hated Raiders are said to be a 5-11 team. There’s at least some reality-based thinking going on at ESPN!

Fans are well-known to be excessively optimistic in general and especially so before the season begins. It seems clear, yet again, that the alleged “experts” are too.

Russell’s Revenge

In the investing world as elsewhere, we face the all-too-human tendency to jump to immediate conclusions, to accept conventional wisdom too eagerly and to fall prey to hyperbolic discounting – valuing now too highly and not yet not enough. But as I often say, hope is not a strategy and lunch is not a long-range plan. This problem was illustrated in a fantastically funny way recently by Super Bowl winning quarterback Russell Wilson of the Seattle Seahawks on Twitter.Russell's Revenge

Continue reading

The Probability Problem

ProbabilityInvesting is a probabilistic enterprise. Since certainty is even rarer than high risk-free returns, we’re left trying to make the best decisions we can based upon the knowledge we have. If we do that extremely well, we might be right most of the time, but still a long ways away from all of the time. The improbable — the highly unlikely even – happens and happens surprisingly often.

Take yesterday’s NFL action, for example. More specifically, consider the astonishing Vikings v. Ravens game in snowy Baltimore. Continue reading

Outcome Bias

BiasedWhile I was concluding a difficult cross-country trip last evening, my Chargers were wrapping up a surprising win against the Indianapolis Colts. To clinch the win, Nick Novak hit a 50-yard field goal, his fourth of the night, with 2 minutes left to give the Chargers a 19-9 lead. ESPN’s highlights are shown below.

There should be at least three clear takeaways from these highlights and from the game itself:

  1. The Bolts’ “powders” are the best uniforms in sports;
  2. Rookie receiver Keenan Allen was a steal in the 3rd round of the draft (full disclosure: he was a teammate of my younger son at Cal); and
  3. Kicking that last field goal was a mistake.

My focus here is #3. Continue reading

Jake Locker, Randomness and Outcomes

I often write about the relative importance of luck and skill in various endeavors, including sports and investing (here, for example) and how the outcomes in such things — heavily influenced by luck — can cause us to miss important aspects of the process involved, which is much more important in the long run (for example, here). This past week’s loss by my San Diego Chargers to the Tennessee Titans provides a terrific example of how these things work.

Titan quarterback Jake Locker is 2-1 after three games and has thrown zero interceptions so far this season. He also led the Titans on a 94-yard drive for a touchdown to beat the Bolts as time expired (against a very soft zone defense — arrrggggg!). However, Locker’s overall statistics this season are virtually identical to last year’s mediocre numbers when the Titans had a blah 6-10 record (as Grantland’s Bill Barnwell has carefully pointed out). Is he much improved or not?

It’s too early to tell for sure, but the following play (courtesy of Bolts from the Blue) offers one good data point and a helpful jumping off point toward my still quite tentative view that the overall statistics may be a better gauge of where he is than Locker’s won-loss record and lack of picks so far this season.

Gilchrist

Wow.

Marcus Gilchrist of the Chargers flat-out drops an interception with just seconds left in the game that would have secured the win for my guys. It isn’t on the level of the late-game Marlon McCree post-interception fumble that cost the Chargers a 2007 play-off game to New England (I was in the stands for that one), but it’s still pretty bad.  Obviously, the play isn’t Locker’s fault in that he hit Delanie Walker in stride and Walker tipped the football straight to Gilchrist. But think for a bit what this play demonstrates.

If Marcus makes the pick, the outcome (Titans loss) could cause us to conclude that Locker isn’t really progressing.  We’d look at his losing record and think that he could only score 13 points at home against the Chargers and couldn’t get it done in the two-minute drill. But since Gilchrist dropped the ball and the Titans went on the win, we may now forget that Locker badly missed a wide open Damian Williams in the end zone just before the game-winning play, didn’t make a great throw on the final play (although it was pretty good) and that Locker was just 2-for-11 on throws that traveled 15 yards or more in the air for the game.

These events provide great examples of how outcomes can disguise crucial elements of the process that — together with a significant amount of randomness — dictates those outcomes.  For example, the Gilchrist drop shows how and why players who outperform for a given stretch tend to regress toward the mean. That’s also why, despite the sample size being much too small to be sure, a lot of talent and, as a very young quarterback, a much better chance of significant improvement than more seasoned pros, it seems more probable that Locker is the player we thought he was last year than a budding star, despite some very good outcomes to this point in the season.

Smart Guy but Dumb Conclusion

I grew up near Buffalo and the Bills were my team back in the day.  Even though I haven’t lived in western New York for more than 30 years, I still pay attention to what happens there and have been more than a bit disappointed at the failings of the franchise since the Bills went to four straight Super Bowls in the 1980s (and lost them all).  I was thus intrigued that new Bills president Russ Brandon announced Tuesday that the Bills will create a football analytics operation to help guide football operations there.

It should be noted that Brandon comes from baseball, where such analytical analysis is now pretty routine.  And if you were to think that that sounds just a bit like Moneyball, you’d be right.

For those of you just coming in from a long spell on the north forty — and as I have written before – Moneyball (a book by former Salomon Brothers bond salesman Michael Lewis and subsequently a movie) focuses on the 2002 season of the Oakland Athletics, a team with one of the smallest budgets in Major League Baseball.  At the time, the A’s had lost three of their star players to free agency because they could not afford to keep them.  A’s General Manager Billy Beane, armed with reams of performance and other statistical data, his interpretation of which was rejected by “traditional baseball men” (and also armed with three terrific young starting pitchers), assembled a team of seemingly undesirable players on the cheap that proceeded to win 103 games and the divisional title.

Unfortunately, much of the analysis of Moneyball has focused upon the idea of looking for cheap assets and outwitting the opposition in trading for those assets.  Instead, the crucial lesson of Moneyball relates to finding value via underappreciated assets (some assets are cheap for good reason) by way of a disciplined, data-driven process.  According to Beane, the key to the process is “identifying and using undervalued assets to create and sustain a competitive edge.”  Instead of looking for players based upon subjective factors (a “five-tool guy,” for example) and who perform well according to traditional statistical measures (like RBIs and batting average, as opposed to on-base percentage and OPS, for example), Beane sought out players that he could obtain cheaply because their actual (statistically verifiable) value was greater than their generally perceived value.

The Bills appear to be ready to try to do the same thing.  “We are going to create and establish a very robust football analytics operation that we layer into our entire operation moving forward,” Brandon says. “That’s something that’s very important to me and the future of the franchise.”

Like the A’s, small market teams like the Bills need to find an edge to help them compete with more revenue-rich teams, even though revenue sharing and the NFL salary cap mean that the money disparity is both less marked and less important in football than in baseball.  And to be sure, football is also less conducive to pure analytical analysis of players because it is far less an individual undertaking than baseball.  But all of this is not to say that the sound analysis and application of data can’t help football teams improve.

Just don’t tell that to Bill Polian.  To be sure, Polian is a smart and accomplished guy.  He assembled the four Bills Super Bowl teams and later won an NFL title as president of the Indianapolis Colts.  He was also the guy who ended up picking Peyton Manning over Ryan Leaf (in a surprisingly close call).

“As a practical tool, Moneyball does not work in the NFL,” Polian claims, “because there are very few undervalued players and no middle class because of our salary cap.”

Sadly, Polian utterly misunderstands the lessons of Moneyball and their potential applicability to any and every field of endeavor, including the NFL.  And he isn’t alone.

To begin with, Polian seems to misunderstand what being undervalued means in this context.  Polian seems to think that since there is no real “middle class” in the NFL because the pay levels are so high, no players are undervalued.  Instead, as in Moneyball, good analytical analysis can identify players that cost less than their intrinsic value because the common metrics used to evaluate their performance don’t suggest what they should really be worth.

While it remains to be seen how effective analytics can be in a football context, there is very good reason to believe that remaining skeptical of conventional football wisdom and trying to look at things differently can indeed work.

For example, Tom Brady was a mere sixth round draft choice (199th pick) and then only a clipboard-carrying back-up who became a multiple Super Bowl MVP that married a supermodel, but only after Drew Bledsoe was injured and he got a chance to play.  And as we in San Diego know all too well, for every Peyton Manning there is at least one Ryan Leaf.

Indeed, there has been a surprising amount of analysis done on NFL Draft economics (see here, for example), but this chart from Yale’s Cade Massey tells us pretty much all we need to know for these purposes.

lucky2

The chart plots where the top six most awarded (via Pro Bowls) and rewarded (with cash salary) players from each draft were selected (it starts with 2005 and goes back to allow player success to shake itself out). According to Massey, “Draft order explains only 31 percent of variation in players’ career starts, 22 percent of free-agent [money], and 9 percent of pro bowls.” It’s “largely a lottery.” As Derek Thompson pointed out for The Atlantic, Massey’s metrics could be better (he doesn’t normalize the numbers to reflect the difference in pay at various positions, for example), but the big picture should be clear.

In every year except 2005, at least three of the top six players were drafted in the first round and in most years, at least four of the top six landed in the first round. But that also means that nearly half of the best players (again, using Massey’s metrics) were not drafted in the first round.  It is thus reasonably clear that NFL GMs do a pretty good job of evaluating talent in the aggregate but make lots of individual mistakes, no doubt for a wide range of reasons.  It also suggests that there is a lot more to be learned about what makes a good football player and what makes a football player good on a particular team and in particular situations.  More and better analytical analysis may be a way to do that. Every sports team — indeed, every business endeavor – no matter how constrained by financial wherewithal or salary caps, should still seek to use its resources as efficiently as possible.

Secondly, Polian’s rejection of analytics out-of-hand reeks of ideology.  It’s entirely possible that analytics won’t be all that helpful in football or that it will turn out to be exceedingly difficult to implement.  It’s even possible (if highly unlikely) that good analytics will confer no useful advantage.  But to reject the concept without examining all of its permutations and ramifications is an ideological commitment rather than an evidence-based decision.  In football — as in the markets — evidence needs to trump ideology to maximize opportunity.

This sort of “old school” rejection of newfangled data analysis reeks of how market-making traders I knew – proud of their instincts and seat-of-the-pants judgments — reacted to and actively resisted the analytical revolution in the markets that began in the 1980s and really ramped up in the 1990s.  It should be no surprise that some of those who benefitted most from the analytical revolution in the markets have gone on to buy sports teams and use similar approaches there (John Henry of the Red Sox and Liverpool FC, for example).

Guys like Bill Polian may have to be dragged — kicking and screaming — into the 21st Century.  Even so, that he still exists and that there are others like him means that Russ Brandon and the Bills have a real shot to succeed.  It won’t be easy, but the opportunity is real.

And that’s a very good thing indeed.  Go Bills.

The NFL, Data and Player Safety

This season the National Football League introduced a full season’s worth (actually, 13 weeks, with games featuring every team) of highly valuable Thursday night games for the first time, all televised by the league-owned NFL Network.  Many players have gone on record complaining about the Thursday scheduling due to injury concerns.  Bill Simmons of Grantland, the world’s most famous sports blogger, has been the most prominent among many critics of the Thursday night match-ups, profusely criticizing NFL Commissioner Roger Goodell for putting players’ health at risk by scheduling games on such short rest. For example, Simmons cites the case of the Baltimore Ravens, who played four games in 17 days at the start of the season and subsequently suffered a rash of injuries, including the losses of star cornerback Lardarius Webb and future Hall of Fame linebacker Ray Lewis for the season (although Lewis may well return ahead of schedule). The Thursday night games have also been very sloppy, presumably due to a lack of both practice and recovery time, and last night’s match-up was no exception.

During his wide-ranging news conference earlier this week, Goodell claimed that ongoing efforts will continue to make the game safer.  Injuries have been a major focus of late, particularly as they relate to concussions and brain trauma.  But Goodell also asserted that there was no data to support the idea that the short weeks cause more or more serious injuries.  “We don’t have any information that indicates from our data that playing on Thursdays in any way decreases the safety of our players,” Goodell said.  “The injury rates do not indicate that at all over the years.  So I think we start with facts, and the facts are that that’s not a risk to the players.”  He may even be right.  Moreover, since the players’ union has approved the games, player complaints aren’t likely to make a difference any time soon (for at least as long as collective bargaining agreement remains in force).

There is pretty good reason to believe that the Commissioner is being more than a little hypocritical generally when it comes to player safety, but for the purposes of this exercise, let’s assume that his actions were and are entirely well-intentioned and that there really is no data supporting the idea of increased injury risk on Thursday nights.  As my masthead suggests and as I have written repeatedly (for example, here), I am committed to data-driven analysis — in investing and elsewhere.  But how should we deal with a lack of data?

As an initial matter, it is important to note that a lack of data is not the same as a lack of evidence.  There is a lot of testimonial evidence from the players — albeit anecdotal and incomplete evidence – that playing on Thursday is a serious risk.  The lack of data confirming that testimony may be because the player narrative is wrong.  But it may also be because there isn’t enough experience yet for the data to be meaningful or because the right measurements aren’t being taken (for example, the number of injuries may not go up on Thursday nights, but their severity might).  Remember, this is the first year that we have seen a full slate of Thursday night games.

The difficult question is how to select the appropriate default in the event that there isn’t any data or where the data remains inconclusive.  In most instances, it makes sense to maintain current practice if there is no data suggesting otherwise.  But when current practice is new or relatively new, it may be prudent to rely on anecdotal testimonial evidence alone if it sufficiently compelling.

That leads, obviously, to the issue of what kind of evidence is or should be “sufficiently compelling.”  In general, if the risks of proceeding are high, the threshold of evidence needed should be pretty low and vice versa.  Moreover, the types of risk are extremely important.  The NFL has a lot of money invested in Thursday Night Football, largely in the form of the NFL Network.  The financial stakes are enormous and they are known.  But the human injury risks are also huge, even though they remain — at least at present — somewhat speculative.

The appropriate conclusion, then, seems to depend upon how you measure and value the clear economic consequences of cancelling Thursday night games as compared to the serious but somewhat speculative injury risks alleged by many players.  There is no clear answer and the individual conclusions drawn will be predicated largely upon one’s values and priorities, not to mention any personal stake one might have in the outcome.  Roger Goodell works for the league and NFL owners, whose financial interests are at stake, which will obviously influence his conclusions (confirmation bias demands it).  The players face no financial risk, at least directly, but face a serious risk of injury (and thus career risk, with serious financial repercussions) every time they step on the field.  That reality will obviously influence their conclusions (confirmation bias demands it).

For me, the answer is clear (but remember that I’m biased too — one of my sons was a Division I college football player whose career was cut short by injury).  In my view, dynamic Baltimore Ravens safety Ed Reed said it best. “If they are really concerned about the violence and injuries . . . Why is there Thursday night football?” 

Why is there Thursday Night Football indeed?

Confirmation Bias Illustrated

As I have argued many times (here, for example), we like to think that we carefully gather and evaluate facts and data before coming to a conclusion.  But we don’t. Instead, we tend to suffer from confirmation bias and thus reach a conclusion first.  Only thereafter do we gather facts and see those facts in such a way as to support our pre-conceived conclusions.  When a conclusion fits with our desired narrative, so much the better, because narratives are crucial to how we make sense of reality.  In other words, we want to think we’re like judges searching for truth impartially when, in fact, we’re much more like attorneys running around hunting for any argument that we think might help.

With that preface, consider my friend Ben Malcolmson.  He’s a terrific guy and very good buddies with both of my sons.  He also has a great story.  Ben posted this on his Facebook page yesterday:

And on Twitter:  “It’s sad that ESPN and Packer fans have sucked all the fun out of a win that should’ve been one for us to cherish and enjoy.”

At first glance, this looks nuts, right?  Nobody really thinks the right call was made, do they? But when we consider that Ben is close to Pete Carroll and works for the Seahawks, we all nod our heads and intuitively get it — it’s confirmation bias writ large.

Unfortunately, we’re all as susceptible to it as Ben is.  Moreover, due to our overarching problem (the bias blind spot), we tend to think the problem applies only to others.  As if….

Luck, Skill and Jim Harbaugh

As I have noted before, the outcomes in many activities of life involve elements of both skill and luck. Investing is one of these.  So is football.  Understanding the relative contributions of luck and skill can help us assess past results and, more importantly, anticipate future results in many such areas.  Doing so is crucial to the investment management process and to understanding football too.

Luck (randomness) is a huge factor in investment returns, irrespective of manager.  “Most of the annual variation in [one’s investment] performance is due to luck, not skill,” according to California Institute of Technology professor Bradford Cornell in a view shared by all experts (Nobel Prize winner Daniel Kahneman talks about it in this video, for example).  Even more troublesome is our perfectly human tendency to attribute poor results to bad luck and good results to skill.  In investing, we generally underestimate luck.

Luck is really important in football too.  There is no substantive difference from a performance standpoint when a throw is tipped by a receiver and lands harmlessly on the turf, when it hits a defender who drops it, when it hits a defender who intercepts it and when it hits another receiver for a touchdown.  In those instances, luck is very important to the outcome of the game.  Similarly, while fumbles relate to skill from a defensive standpoint (stripping the ball effectively, for example), recovering fumbles is largely a matter of luck (for example, where and how the ball bounces).

It is the element of luck that has had many football geeks forecasting difficulty this season for the San Francisco 49ers. 

When Jim Harbaugh was hired to coach the 49ers before last season, the team had just suffered through yet another dreadful (6-10) season.  Harbaugh’s new players were soon locked out and pre-season training camps were dramatically curtailed, thereby limiting the players’ ability to absorb his system and precluding the new coaching staff’s ability to get to know and evaluate the players.  Perhaps most problematic in a quarterback-dominated league, Harbaugh inherited a QB in Alex Smith who was widely regarded as insufficiently talented and a total bust.

Based upon those realities and upon the related fact that a lack of turnovers and good field position correlate strongly with winning, Harbaugh established an almost painfully conservative offense designed (in a phrase that coaches love) to limit mistakes.  He also developed an aggressive and opportunistic defense as well as outstanding special teams units. Harbaugh’s focus and approach resulted in the 49ers leading the league in turnover differential (a remarkable +28), defensive starting field position (76.1 – on average, opponents started at their 24-yard line) and offensive starting field position (66.6 – on average, the 49ers started at their 33-yard line).  Largely as a consequence, the 49ers improved from 6-10 to an astonishing 13-3 and advanced to the NFC Championship Game. Harbaugh was named 2011 NFL Coach of the Year.

Most experts think that Harbaugh is a fabulous motivator and tactician, but many (and especially stat geeks) have suggested that the 49ers will slip a lot this season due to regression to the mean.  Much of what the 49ers did well last year is thought not to be repeatable.  As my mother used to say, “these things average out.” For example, the 2012 Football Outsiders Almanac projects 7.2 Niner wins this season.  That’s obviously a major fall-off from last season’s 13. 

To begin with, teams that make big jumps to the upside generally fail to sustain it.  Perhaps worse, this season’s Niner schedule seems much tougher (although the schedule for the entire NFC West is tougher).  Moreover, the 21 other teams since 1978 with a turnover margin of plus-20 or better averaged a +3 the next season – a huge drop – and won 2.5 fewer games – another big drop; the luck factor (regression to the mean) at work.  Each of the QBs with a crazy-low interception rate in 2010 (like Brady, Roethlisberger and Cassell) suffered interceptions at a rate of at least twice that in 2011; experts expect Alex Smith similarly to regress toward the league average. And, with a full year in the system under and substantial upgrades at receiver, it’s hard to expect the 49ers to remain so ultra-conservative this year, which should also mean more turnovers (Hall of Famer Jerry Rice describes this as taking the diapers off).  An increase in turnovers would also hurt San Francisco’s outstanding field position averages last season, which were (as noted above) the best in the NFL both offensively and defensively.

One of the other reasons that the Niner defense was so effective: health. Last year, San Francisco’s 11 starters on defense combined to miss a total of just eight games due to injury. That level of health – best in the NFL – is very unlikely to repeat.

There is also something to the idea of a slump for coaches and teams after a really big jump forward (Football Outsiders calls this the “plexiglass principle“).  For example, Bill Belichick followed his 5-11 debut season with the Patriots by leading New England to a stunning 11-win season and a Super Bowl win. It’s mean regression at work.  Teams that get “the breaks” in one season are not likely to get them (in an overall sense) the next.

I don’t disagree with this mean regression analysis.  But I don’t expect Harbaugh and the 49ers to underperform by much (if at all) either.  Like advocates of the efficient markets hypothesis, the stat geeks (and I use the term affectionately) significantly undervalue what skill can bring to the table – largely because real skill (and especially coaching skill) can be so hard to find.

Analysts pay lip service to the idea that Harbaugh is a terrific coach (consider the jobs he did at the University of San Diego and Stanford, for Heaven’s sake), but he’s even better than they recognize.  For example, Harbaugh made the passing game easier for his quarterback last year, particularly when it came to beating the blitz. The typical NFL offense requires that receivers break-off their routes to counter a blitz and thus demands that receivers and the quarterback consistently are “on the same page” as to what the defense is bringing and what ought to be done about it.  These are the so-called “hot routes” based upon “sight adjustments.”  Instead, Harbaugh installed a scheme with built-in hot routes by design. Thus Alex Smith knows that if the blitz comes, he throws to a particular route (which is part of the play call even if there isn’t a blitz). 

In football lingo, if Smith reads blitz, he goes through his progressions shallow-to-deep. He can throw the three-step route to the area vacated by the blitz or to the designated hot route.  If his protection holds, he can move on with his progressions for a deeper throw. The key is that Smith and the receivers don’t have to read the same thing in the same way at the line of scrimmage.  This approach gives the QB the confidence of knowing where the ball should go without having to audible. It also means he knows what to do and doesn’t have to figure it out on-the-fly.  For a beleagured quarterback lacking confidence, it was a brilliant coaching move.

Together with his tactical and motivational gifts, Harbaugh brings a lot to the table.  With a full off-season and a full training camp, I expect Harbaugh’s coaching to be even more impactful this season.

That said, players make the biggest difference in any NFL game.  This season’s 49ers return 21 of 22 starters from last year and upgraded their weakest position along with adding quality depth elsewhere. I expect quarterback Alex Smith to improve too (despite many nay-sayers), particularly because he will be in the same system for two consecutive years for the first time as a pro.  This year’s Niner roster is significantly better than last season’s.  Thus I do not expect a significant performance drop-off.

My analysis is of course supported by what has already happened this season.  Despite many projections of a tough start, the 49ers are one of only three play-off teams from a year ago to have started this season 2-0. San Francisco handily defeated Green Bay – widely touted as the best team in the league – in Green Bay and handled Detroit – also a play-off team last year – in the season’s first two games.

Alex Smith has completed over 70 percent of his passes so far this season (40 of 57) for 437 yards and four touchdowns and a 115.9 passer rating. He’s also working on a franchise-record streak of 216 passes without an interception, which is pretty impressive given some of the QBs who have played in San Francisco.  At least to this point, regression, resmeshion.  Of course, as with investing, the tide could turn in a hurry.

In football as in investing, luck matters a lot, but skill remains undervalued and underappreciated. I wish Jim Harbaugh were back in San Diego, this time with the Chargers.  Skill matters…a lot.

__________

More on luck, skill and football is available here. If you liked this post, you might want to read an article I wrote relating trading to Moneyball (here).  You might also want to read a couple of pieces I wrote on college basketball, fans and behavioral economics: