Uses and abuses of empirical evidence in the death penalty debate.

• Author: Donohue, John J.

INTRODUCTION I. THEORY: WHAT ARE THE IMPLICATIONS OF THE DEATH PENALTY FOR HOMICIDE RATES? II. A CENTURY OF MURDERS AND EXECUTIONS III. THE IMPORTANCE OF COMPARISON GROUPS A. Canada Versus the United States B. Non-Death Penalty States Versus Other States in the United States IV. PANEL DATA METHODS A. Katz, Levitt, and Shustorovich B. Dezhbakhsh and Shepherd C. Mocan and Gittings D. Other Studies V. INSTRUMENTAL VARIABLES ESTIMATES A. Problems with Invalid Instruments B. Problems with Statistical Significance VI. A PARTIAL RECONCILIATION: LACK OF STATISTICAL POWER AND REPORTING BIAS CONCLUSION INTRODUCTION

Over much of the last half-century, the legal and political history of the death penalty in the United States has closely paralleled the debate within social science about its efficacy as a deterrent. Sociologist Thorsten Sellin's careful comparisons of the evolution of homicide rates in contiguous states from 1920 to 1963 led to doubts about the existence of a deterrent effect caused by the imposition of the death penalty. (1) This work likely contributed to the waning reliance on capital punishment, and executions virtually ceased in the late 1960s. In the 1972 Furman decision, the Supreme Court ruled that existing death penalty statutes were unconstitutional. (2) In 1975, Isaac Ehrlich's analysis of national time-series data led him to claim that each execution saved eight lives. (3) Solicitor General Robert Bork cited Ehrlich's work to the Supreme Court a year later, and the Court, while claiming not to have relied on the empirical evidence, ended the death penalty moratorium when it upheld various capital punishment statutes in Gregg v. Georgia and related cases. (4) The injection of Ehrlich's conclusions into the legal and public policy arenas, coupled with the academic debate over Ehrlich's methods, led the National Academy of Sciences to issue a 1978 report which argued that the existing evidence in support of a deterrent effect of capital punishment was unpersuasive. (5) Over the next two decades, as a series of academic papers continued to debate the deterrence question, the number of executions gradually increased, albeit to levels much lower than those seen in the first half of the twentieth century.

The current state of the political debate over capital punishment is one of disagreement, controversy, and division. Governor George Ryan of Illinois suspended executions in that state in 2000 and commuted the death sentences of all Illinois death row inmates in 2003. (6) As a number of other jurisdictions were considering similar moratoria, New York's highest court ruled in 2004 that the state's death penalty statute was unconstitutional. (7) Executions in California are virtually nonexistent, although the state continues to add prisoners to death row at a rapid pace. (8) Meanwhile, executions continue apace in Texas, which accounts for over one-third of all post-Gregg executions. (9)

A host of more recent academic studies has examined the death penalty over the last decade, with mixed results. While Lawrence Katz, Steven Levitt, and Ellen Shustorovich found no robust evidence of deterrence, (10) several researchers claim to have uncovered compelling evidence to the contrary. (11) This latter research appears to have found favor with Cass Sunstein and Adrian Vermeule, who describe it as "powerful" (12) and "impressive," (13) and they refer to "many decades' worth of data about [capital punishment's] deterrent effects." (14) While Sunstein and Vermeule claim not to endorse any specific analysis, these "sophisticated multiple regression studies" (15) are "[t]he foundation for [their] argument," (16) and they specifically rely on many of the recent studies that we will reexamine as buttressing their premise that "capital punishment powerfully deters killings." (17) This empirical evidence leads to the heart of their claim that it would be irresponsible for government to fail to act upon the studies and vigorously prosecute the death penalty. Carol Steiker has offered a considered response to this claim based on moral theory; (18) by contrast, we are interested in exploring its empirical premise.

Thus, our aim in this Article is to provide a thorough assessment of the statistical evidence on this important public policy issue and to understand better the conflicting evidence. We test the sensitivity of existing studies in a number of intuitively plausible ways--testing their robustness to alternative sample periods, comparison groups, control variables, functional forms, and estimators. We find that the existing evidence for deterrence is surprisingly fragile, and even small changes in specifications yield dramatically different results. Our key insight is that the death penalty--at least as it has been implemented in the United States since Gregg ended the moratorium on executions--is applied so rarely that the number of homicides it can plausibly have caused or deterred cannot be reliably disentangled from the large year-to-year changes in the homicide rate caused by other factors. Our estimates suggest not just "reasonable doubt" about whether there is any deterrent effect of the death penalty, but profound uncertainty. We are confident that the effects are not large, but we remain unsure even of whether they are positive or negative. The difficulty is not just one of statistical significance: whether one measures positive or negative effects of the death penalty is extremely sensitive to very small changes in econometric specifications. Moreover, we are pessimistic that existing data can resolve this uncertainty.

We begin in the next Part by sketching the relevant economic theories of crime and the difficulties in identifying their effects. We then begin our tour of the statistical evidence. Part II analyzes aggregate time-series evidence. Part III analyzes first differences--the change in homicide rates that occurs following death penalty reforms. In Part IV, we turn to panel data analysis, and Part V analyzes the key instrumental variables estimates. Part VI contains our attempt at reconciling the conflicting evidence, assessing the limited precision with which we might be able to pin down the deterrent effect of the death penalty with existing data. Our organizing theme involves an attempt to examine the evidence compiled by previous scholars with the aim of highlighting the ways in which this evidence can both provide insight but also potentially mislead policy analysts.

1. THEORY: WHAT ARE THE IMPLICATIONS OF THE DEATH PENALTY FOR HOMICIDE RATES?

   The theoretical premise underlying the deterrence argument is simple: raise the price of murder for criminals, and you will get less of it. In general, the death penalty raises the price of homicide as long as execution is worse than life imprisonment for most potential murderers. (19)

   While this argument is qualitatively reasonable, its quantitative significance may be minor. In 2003, there were 16,503 homicides (including nonnegligent manslaughter), but only 144 inmates were sentenced to death. (20) Moreover, of the 3374 inmates on death row at the beginning of the year, only 65 were executed. (21) Thus, not only did very few homicides lead to a death sentence, but the prospect of execution did not greatly affect the life expectancy of death row inmates. Indeed, Katz, Levitt, and Shustorovich have made this point quite directly, arguing that "the execution rate on death row is only twice the death rate from accidents and violence among all American men" and that the death rate on death row is plausibly lower than the death rate of violent criminals not on death row. (22) As such they conclude that "it is hard to believe that in modern America the fear of execution would be a driving force in a rational criminal's calculus." (23) Moreover, even if there were a deterrent effect, capital punishment is sufficiently expensive (24) that it may potentially divert resources away from more effective crime prevention strategies.

   A more sociological approach notes that there may be social spillovers as state-sanctioned executions cheapen the value of life, potentially demonstrating that deadly retribution is socially acceptable. Thus, executions may actually stimulate more homicide through the so-called brutalization effect. (25) With theory inconclusive, we now turn to examining the data.

2. A CENTURY OF MURDERS AND EXECUTIONS

   Several of the early studies of the death penalty were based on analysis of the aggregate U.S. time-series data. Figure 1 depicts the homicide and execution rates for the United States over the last century. (26) Because data issues can be a concern with crime data, we present two series for homicides-one from the Uniform Crime Reports and the other compiled from Vital Statistics sources, based on death certificates. (27)

   [FIGURE 1 OMITTED]

   No clear correlation between homicides and executions emerges from this long time series. In the first decade of the twentieth century, execution and homicide rates seemed roughly uncorrelated, followed by a decade of divergence as executions fell sharply and homicides trended up. Then for the next forty years, execution and homicide rates again tended to move together--first rising together during the 1920s and 1930s, and then falling together in the 1940s and 1950s. As the death penalty fell into disuse in the 1960s, the homicide rate rose sharply. The death penalty moratorium that began with Furman in 1972 and ended with Gregg in 1976 appears to have been a period in which the homicide rate rose. The homicide rate then remained high and variable through the 1980s while the rate of executions rose. Finally, homicides dropped dramatically during the 1990s. By any measure, the resumption of the death penalty in recent decades has been fairly minor, and both the level of the execution rate and its year-to-year changes are tiny: since 1960 the proportion of homicides resulting in execution...