Pass or run: an empirical test of the matching pennies game using data from the national football league.

• Author: McGarrity, Joseph P.

1. Introduction

   Standard economic optimizing techniques, such as isoquant and isocost analysis, predict that when an input becomes less productive, firms will use it less and use more of their other inputs instead. Consider the case of American football. The offense has two main options. It can run the ball, or it can pass the ball. Next, suppose a starting National Football League (NFL) quarterback is injured. If we assume that the starting quarterback is more proficient than his replacement, the standard optimizing approach suggests that the coach will have the replacement quarterback throw less frequently. That is, the coach will decrease his use of the input with diminished productivity, and he will use the running back, whose productivity has not changed, more often.

   However, the standard optimizing approach does not account for the fact that the offensive output is dependent, in part, upon decisions made by the defense. The defense tries to undermine the offense whenever it can. If the defense believes the offense will run the ball, the defense will try to move its players toward the line of scrimmage, thereby closing any holes the running back can run through. If the defense believes the offense will pass, once the ball is snapped, the defense might assign more players to rush the quarterback or assign more players to cover potential receivers. The game theory approach is able to deal with this adversarial relationship. Interestingly, in the case of the injured starting quarterback, the game theory prediction is much different than the isoquant/isocost prediction.

   This article develops a game theoretic model that predicts that an offense will not change its play calling when its starting quarterback cannot play because of an injury. Instead, the defense will play against the run more often. Although the defensive play calling is not readily observable, it is very clear whether the offense calls a pass or a run. This article tests whether starting quarterbacks pass more often than their replacements. Using data from the 11 teams that replaced an injured quarterback in the 2006 NFL season, we find that an injury did not change offensive play calling.

   The matching pennies game, the mixed strategy game developed in this article, is the standard theoretical approach that one would use to explain how actors behave if they benefit when other players cannot accurately guess their actions. Therefore, it is interesting that the matching pennies game has remained so important theoretically when the empirical evidence indicates that people do not act as the model suggests that they should. However, almost all of the empirical tests have been conducted in a laboratory setting. (1) Walker and Wooders (2001) argue that the laboratory may not be the best setting for testing the applicability of mixed strategy equilibria. They note that playing a game well is difficult, and laboratory participants may not have the incentive to learn to play the game well because their payoffs are often small and because they do not get a chance to keep playing the game after the experiment is over.

   Walker and Wooders argue that sports may offer a better setting for testing whether people adopt mixed strategies in a matching pennies game. (2) Professional athletes invest heavily in learning their game and would very likely be able to take advantage of a player who did not adopt the optimal mix of strategies. They also argue that professional athletes have enough at stake so that they will learn the best strategic approaches to their game. To date, very few articles have tested the predictions of the matching pennies game outside the laboratory. We have been able to find only six such articles, and all of them use sports data to conduct the tests.

   The aforementioned Walker and Wooders (2001) is the First article to use non-laboratory data to test the predictions of a matching pennies game. In support of the game theoretic predictions, they find that tennis players mixed their serves between an opponent's backhand and forehand, so that a serve to either side had an equal probability of being a winning point. However, they found that the service choice was not serially independent as the mixed strategy equilibrium requires. They report that serves alternated sides too often. In a comment Hsu, Huang, and Tang (2007) replicate these results with a larger sample and find evidence in support of both of these two game theoretic predictions. Klaassen and Magnus (2009) find that tennis players at Wimbledon do not adopt an optimal service strategy. They suggest that optimal strategies would increase the financial winnings of men by 18.7% and increase the financial winnings of women by 32.8%.

   Chiappori, Levitt, and Groseclose (2002) find that soccer players mix the location of penalty kicks in a way consistent with the predictions of a matching pennies game. Using a larger sample of soccer penalty kicks, Palacios-Huerta (2003) reaches the same conclusion. Coloma (2007) finds evidence of mixed strategy play when using the data of Chiappori, Levitt, and Groseclose (2002) and employing an empirical strategy of estimating a simultaneous equation model. The three articles on soccer penalty kicks all find that mixed strategies, which did not seem to explain much in the laboratory, do a very good job explaining real world behavior. Taken together, the articles that analyze tennis serves also find support for mixed strategy play; although, this support is weaker than that found in the articles using soccer data. Using a different sport (football), this article adds further evidence that mixed strategies are actually played outside of the laboratory.

   One of the clear predictions of a mixed strategy equilibrium is that the expected utility of all actions should be the same. In the case of soccer penalty kicks, a goal is worth the same amount of points regardless of which side of the goal the ball enters. This implies that a kick to the right side of the goal should have an equal chance of scoring as a kick to the left side of the goal. In football we cannot assume that, in equilibrium, the expected yards from a pass yield the same number of yards as the expected yards from a run. We cannot, because a pass is riskier than a run, and the team's offense would have to expect more yards, on average, from a pass to be willing to accept the additional risk of a turnover. However, we can use comparative statics to determine whether offenses are using a rational mixed strategy. An exogenous event such as an injury to the starting quarterback makes different predictions about the new equilibrium depending upon how the team optimizes. We can differentiate whether the offense is using a mixing strategy or optimizing without considering possible defensive reaction.

2. Game Theoretic Model

   In the football play calling game, the offense and defense are adversaries. The offense can call a passing play or a running play. The defense must decide to defend against the run, or defend against the pass. The offense will do well when (i) it calls a passing play and its opponents play a run defense or (ii) it calls a running play when the defense is set up to defend a pass. Similarly, the defense does well when it tries to defend against the type of play the offense calls. The offense and defense must simultaneously decide what play to call. They will try to disguise their calls so it is not apparent what play was called, even as the play is beginning to be executed. For example, in a play action pass, a quarterback fakes a hand-off, and it is not apparent that the play is a pass until after the fake.

   The game is characterized in Table 1. The offense's payoffs for each outcome are given by the capital letters (A, B, C, D). Because football is a zero sum game, and every yard the offense gains the defense gives up, the defense's payoffs are represented by the negative of the offense's payoffs (-A, -B, -C, -D). These letters do not represent specific cardinal utilities for each outcome. Instead, this setup is much more general. The letters are variables, and any level of utility can be plugged in for a letter.

   Consider the four outcomes. The outcomes in the chart's northwest corner (A, -A) and the chart's southeast corner (D, -D) represent the times that the defense guesses correctly and defends against the type of play that the offense actually called. The outcomes in the northeast corner (B, -B) and those in the southwest corner (C, -C) represent the times that the offense fooled the defense and can run a play against a defense that is not designed to stop that type of play.

   We can say something about the offense's preference orderings. If the defense will attempt to stop the run, the offense would be better off if it passed rather than ran the ball, so we know that B > D. If the defense attempts to stop the pass, the offense would rather run than pass, so C > A. Given these preferences, the offense does not have a dominant pure strategy. Its best strategy depends upon what it believes the defense will do. (3)

   A similar story can be told for the defense. Its best strategy depends upon what the offense does. If the offense wants to pass the ball, the defense should try and stop the pass. Therefore, we know that - A > - B. In the other case, when the offense wants to run the ball, the defense is better off if it tries to stop the run, so - D > - C. This game is a matching pennies game. One player wants to pick the same option as the opponent, but the other player wants to take a different option. As with a typical matching pennies game, there is no pure strategy Nash equilibrium.

   In any given box, one player will want to move counter-clockwise to the next box. However, there is a mixed strategy Nash equilibrium. Suppose the offense passes the ball with a probability of a, and the defense defends against the pass with a probability of y. We can use the payoff-equating...