Balancing the costs of forecasting errors in parole decisions.

• Author: Berk, Richard

1. INTRODUCTION

   Parole decisions can be based on a variety of factors. Some are automatically considered and closely prescribe the actions to be token. Among these are mandatory releases after a full prison term has been served. Other factors can be discretionary. They may or may not be considered, and their use is subject to interpretation.

   Among the discretionary factors commonly weighed are two fundamental kinds of risk. One risk is that an individual will be released only to commit new crimes, sometimes very serious ones. The other risk is that an individual will not be released although behavior on parole would have been exemplary. Since the 1920s, parole boards in the United States have tried to minimize both kinds of risks by constructing forecasts of how individuals will fare under supervision in the community, and then using those forecasts to inform parole decisions. (1) The results of this enterprise are mixed. (2)

   One reason for the mixed performance is a failure to appreciate that there are inevitable tradeoffs between the two kinds of parole risks. If one is more likely, the other is less likely. A related reason is that the consequences of the competing risks have almost universally been treated symmetrically. The costs of releasing an individual who then commits crimes are implicitly assumed to equal the costs of failing to release an individual who would have succeeded in parole. An important consequence is that the forecasts that follow build in that equivalence. (3) If the costs are not truly equal, the forecasts can be misleading, often badly so. (4)

   In this paper, I will review ways to think about forecasting errors in parole decisions. The key issue will be how to properly address the consequences of the two kinds of forecasting errors, known more formally as false positives and false negatives. In the parole context, false positives can be individuals incorrectly projected to be poor parole risks. False negatives can be individuals who are incorrectly projected to be good parole risks. Over many parole decisions, both kinds of errors are virtually inevitable and when improperly considered, adversely affect the quality of those decisions.

2. FALSE POSITIVES, FALSE NEGATIVES, AND FORECASTING

   For a didactic discussion of false positives and false negatives, it is helpful to work with archetypes. Popular culture can provide them. For this paper, therefore, one of the ongoing appeals of the Star Wars movies is the presence of good and evil, starkly represented respectively by Luke Skywalker and Darth Vader. (5) Darth Vader vaporized an entire planet populated by millions of peace-loving humanoids. (6) Luke Skywalker saved the federation of such planets. (7)

   1. False Positives and False Negatives

      In the extreme, at least, parole boards may need to ascertain whether individuals whose cases are being reviewed are more like Darth Vader or more like Luke Skywalker. Parole boards are often especially sensitive to violent crimes committed by individuals on parole, so their risk assessment instruments and procedures may be tuned to identify the Darth Vaders. When they succeed, the term "true positive" can be applied. When they fail, the term "false negative" can be applied. In a parallel fashion, if the risk instrument correctly identifies a Luke Skywalker, one can use the term "true negative." And when that identification is in error, the term "false positive" can be applied. The four possibilities are shown in Table 1, with the two kinds of errors in bold type. A case can fall in any one of the four cells.

      Looking at Table 1, there is a simple way to avoid all false negatives: just proceed as if all cases were like Darth Vader. That is, use only the "Predict Darth Vader" column. One can see that all of the true Darth Vaders will be correctly identified. However, all of the true Luke Skywalkers will automatically become false positives. In a parallel fashion, one can see in Table 1 that there is a simple way to avoid all false positives: just proceed as if all cases were like Luke Skywalker. That is, use only the "Predict Luke Skywalker" column. One can see that all true Luke Skywalkers will be correctly identified. However, all true Darth Vaders will automatically become false negatives. Both solutions are simple because there is no need to do any forecasting. One rule or the other can be applied without any consideration of risk. But in practice, both solutions are likely to be unsatisfactory, in part because all differences between individuals are ignored. For example, first-time offenders are treated the same as habitual offenders. One response is to introduce forecasts of risk based on individual differences.

   2. Parole Decisions as Parole Forecasts

      At the time parole decisions are made, the consequences of parole decisions cannot be known. That is why forecasts are required even when they are not acknowledged as such. A real forecast is being made, often implicitly. If the forecasted outcome is favorable, the individual is more likely to be released. If the forecasted outcome is unfavorable, the individual is less likely to be released.

      The basis of parole forecasts is necessarily historical. In effect, past cases serve as "training data." For these cases, the parole decision and subsequent outcomes can be known. Why cases fall in any of the four cells in Table 1 can be considered. Sometimes that information is used to construct systematic risk instruments. Sometimes it is used anecdotally. In both instances, the historical information informs current decisions that are also shaped by various legal requirements, and are often combined with behavioral evaluations derived from formal clinical training, research findings, theory from the social sciences, and a range of craftlore. For example, an inmate's "motivation" can be an important factor.

      Because at the time parole decisions are made, one cannot know to which of Table 1's cells a case actually belongs, Table 1 is of no help with the instant case. Rather, Table 1 is an essential part of the process by which forecasting procedures are constructed from historical data. We turn to that next.

   3. The Relative Costs of Forecasting Errors

      In an ideal world, the forecasts would be perfect. There would be no false positives and no false negatives. In practice, there can be a substantial number of both. Moreover, there are tradeoffs between the two. If the net is cast to catch a larger number of true negatives, it will almost certainly catch a larger number of false positives as well. If the net is cast to catch a larger number of true positives, it will almost certainly catch a larger number of false negatives as well. This means that as false negatives decrease, false positives increase, and likewise, as false positives increase, false negatives decrease. How, then, should the tradeoff between false positives and false negatives be undertaken? The answer is that it depends on their relative costs.

      When a case becomes a false positive or false negative, a forecasting error has been made, and there are costs associated with each. A false negative often means that a serious crime has been committed. There are one or more victims and various criminal justice...