Predictive Validity of Pretrial Risk Assessments: A Systematic Review of the Literature
Author | Sarah L. Desmarais,Evan M. Lowder,Sarah E. Duhart Clarke,Samantha A. Zottola |
Published date | 01 April 2021 |
Date | 01 April 2021 |
DOI | http://doi.org/10.1177/0093854820932959 |
Subject Matter | Articles |
CRIMINAL JUSTICE AND BEHAVIOR, 2021, Vol. 48, No. 4, April 2021, 398 –420.
DOI: https://doi.org/10.1177/0093854820932959
Article reuse guidelines: sagepub.com/journals-permissions
© 2020 International Association for Correctional and Forensic Psychology
398
PREDICTIVE VALIDITY OF PRETRIAL
RISK ASSESSMENTS
A Systematic Review of the Literature
SARAH L. DESMARAIS
SAMANTHA A. ZOTTOLA
SARAH E. DUHART CLARKE
North Carolina State University
EVAN M. LOWDER
George Mason University
Bail reform is sweeping the nation and many jurisdictions are looking to pretrial risk assessment as one potential strategy to
support these efforts. This article summarizes the findings of a systematic review of research examining the predictive valid-
ity of pretrial risk assessments. We reviewed 11 studies (13 publications) examining the predictive validity of six pretrial risk
assessment instruments reported in the gray and peer-reviewed literature as of December, 2018. Findings typically show good
to excellent predictive validity. Differences in predictive validity for men and women were mixed and small. When it could
be examined, predictive validity was generally comparable across racial/ethnic subgroups; however, three comparisons
revealed notably lower, albeit still fair to good, predictive validity for defendants of color than White defendants. Findings
suggest that pretrial risk assessments predict pretrial outcomes with acceptable accuracy, but also emphasize the need for
continued investigation of predictive validity across gender and racial/ethnic subgroups.
Keywords: risk assessment; predictive validity; decision-making; criminal justice system; race
Across the United States, efforts are underway to maximize pretrial release rates while
minimizing pretrial misconduct, including failure to appear in court and perpetration
of new crime during the pretrial period. Since 2012, every state in the United States has
enacted some form of pretrial legal reform (National Conference on State Legislatures,
2018), with many states implementing pretrial risk assessment instruments as part of these
efforts. Briefly, pretrial risk assessment instruments are designed to forecast the likelihood
of failure to appear in court and/or perpetration of new crime during the pretrial period
AUTHORS’ NOTE: Correspondence concerning this article should be addressed to Sarah L. Desmarais,
Department of Psychology, North Carolina State University, Campus Box 7650, Raleigh, NC 27695;
e-mail: sdesmarais@ncsu.edu.
932959CJBXXX10.1177/0093854820932959Criminal Justice and BehaviorDesmarais et al. / Predictive Validity of Pretrial Risk Assessments
review-article2020
Desmarais et al. / PREDICTIVE VALIDITY OF PRETRIAL RISK ASSESSMENTS 399
through consideration of items that have been shown in research to be associated with these
outcomes. In this way, pretrial risk assessment instruments may provide some empirical
evidence to inform pretrial decisions (Desmarais & Lowder, 2019). A recent scan of pretrial
practices across the United States found that approximately two thirds of surveyed counties
reported using a pretrial risk assessment instrument (Pretrial Justice Institute, 2019b).
Despite their widespread implementation, pretrial risk assessment is the subject of consid-
erable controversy.
Some of the controversy centers on whether pretrial risk assessment instruments are
inherently biased because they are developed using data that reflect biased policing
practices that target people of color (Eckhouse et al., 2019; Mayson, 2019). Other cri-
tiques reflect beliefs that pretrial risk assessments cannot predict pretrial outcomes, that
they increase rates of detention, and that they exacerbate racial biases in pretrial deci-
sion-making (e.g., Barabas et al., 2019; Pretrial Justice Institute, 2020; Scurich &
Krauss, 2020). Behind these critiques, at least in part, are analyses of risk assessments
completed using a single instrument in a single jurisdiction, including ProPublica’s anal-
ysis of the Correctional Offender Management Profiling for Alternative Sanctions
(COMPAS) in Broward County, Florida (Angwin et al., 2016) and a subsequent study
that concluded COMPAS assessments performed no better than lay persons at predicting
recidivism (Dressel & Farid, 2018). Although these two particular studies have been
criticized and rebutted in the academic literature (e.g., Bansak, 2019; Flores et al., 2016;
Holsinger et al., 2018; Lin et al., 2020), they are among the most prominent sources
cited in calls to abolish risk assessment instruments from the criminal justice system.
Discussion of racial bias in risk assessment—pretrial or otherwise—often conflates sev-
eral issues (Desmarais & Zottola, 2020), a few of which we touch on here. A first issue is
whether Black and Brown people receive higher risk scores or classifications than White
people. A second issue is whether Black and Brown people have higher rates of criminal
behavior than others (actual or as the result of unfair police practices). A third issue is
whether Black and Brown people are over classified at higher risk levels and under classi-
fied at lower risk levels relative to their actual rates of criminal behavior. It is this third issue
that speaks to predictive validity. In particular, pretrial risk assessments should be able to
forecast pretrial outcomes with comparable predictive validity, regardless of the base rate of
offending (Lowder et al., 2019). To our knowledge, there has been no effort to date that
examines and compares the predictive validity of pretrial risk assessments as a function of
race/ethnicity, or gender for that matter, across instruments and outcomes. We sought to
address this gap.
PRETRIAL RISK ASSESSMENT INSTRUMENTS
Pretrial risk assessment instruments were designed as a strategy to help overcome some
of the limitations of human decision-making that may contribute to biased and unfair deci-
sions in the pretrial context (Desmarais & Lowder, 2019). It is well accepted that human
judgment is influenced by our personal beliefs and that increasing structure can reduce reli-
ance on heuristics in the decision-making process, thereby producing less biased and more
accurate decisions (Tversky & Kahneman, 1974). Consequently, increasing structure
through the use of risk assessment instruments may contribute to more accurate and less
biased pretrial decisions. Indeed, meta-analyses of research conducted over the past 60
To continue reading
Request your trial