Testing Predictive Biases at the Intersection of Race-Ethnicity and Sex: A Multi-Site Evaluation of a Pretrial Risk Assessment Tool

• Published date: 01 June 2024
• DOI: http://doi.org/10.1177/00938548241233934
• Author: Matthew Demichele,Ian A. Silver,Ryan M. Labrecque,Debbie Dawes,Pamela K. Lattimore,Stephen Tueller
• Date: 01 June 2024
• Subject Matter: Articles

CRIMINAL JUSTICE AND BEHAVIOR, 2024, Vol. 51, No. 6, June 2024, 850 –875.

DOI: https://doi.org/10.1177/00938548241233934

Article reuse guidelines: sagepub.com/journals-permissions

© 2024 International Association for Correctional and Forensic Psychology

850

TESTING PREDICTIVE BIASES AT THE

INTERSECTION OF RACE-ETHNICITY AND SEX

A Multi-Site Evaluation of a Pretrial Risk Assessment

Tool

MATTHEW DEMICHELE

IAN A. SILVER

RYAN M. LABRECQUE

DEBBIE DAWES

PAMELA K. LATTIMORE

STEPHEN TUELLER

RTI International

Pretrial assessment instruments provide courtroom actors with information about the likelihood that someone who has been

charged with a criminal offense would engage in criminal behavior if released prior to trial. Although prior research supports

the ability of pretrial instruments to predict pretrial outcomes, there are concerns that pretrial instruments may inadvertently

exacerbate racial-ethnic and sex disparities found in the larger criminal legal system. In the current study, we conduct multi-

site predictive bias tests of a widely used pretrial instrument—the Public Safety Assessment (PSA)—across six racial-ethnic

and sex groups (i.e., White males, White females, Black males, Black females, Hispanic males, and Hispanic females). Study

results support the PSA as a valid and consistent predictor of failure to appear, new criminal activity, and new violent crimi-

nal activity across these six racial-ethnic and sex groups. These findings support the use of the PSA and identify several areas

for future research.

Keywords: jail; sex; pretrial; PSA; race; pretrial assessment

AUTHORS’ NOTE: The manuscript is the culmination of a more than five-year study that benefited from the

work of dozens of individuals who we would like to thank. The study would not have been possible without the

patience and dedication of several local and state criminal legal system officials who supported this research by

providing access to their data and operations. Besides the legal system officials, we thank several RTI colleagues

who were responsible for data management and processing and Megan Comfort, Kristin Bechtel, Jim Trudeau,

and Jenn Rineer for their thoughtful comments on prior drafts. We appreciate Virginia Bersch of Arnold

Ventures for her support and encouragement. The authors, however, are solely responsible for all statements

within the manuscript. This project is funded by Arnold Ventures. All the conclusions derived from the current

study are of the authors alone. The authors have no conflicts of interest to report. Correspondence concerning

this article should be addressed to Matthew DeMichele, Center for Legal Systems Research, RTI International,

3040 East Cornwallis Road P.O. Box 12194, Research Triangle Park, NC 27709; email: mdemichele@rti.org.

1233934CJBXXX10.1177/00938548241233934Criminal Justice and BehaviorDeMichele et al. / SHORT TITLE

research-article2024

DeMichele et al. / TESTING PREDICTIVE BIASES 851

INTRODUCTION

The use of pretrial detention has long been criticized for its differential impact on people

of color (Ares et al., 1963; Petersen, 2020). There are concerns about the influence of a

person’s sex in combination with their race-ethnicity on judicial decision-making during the

pretrial release process (Brittain et al., 2021). Pretrial scholarship routinely shows that peo-

ple of color are more likely to be detained prior to trial than White individuals (Demuth,

2003; Schlesinger, 2005). Research suggests that while White females are the most likely to

be released during pretrial, racial-ethnic minority males are the most likely to remain

detained (Demuth, 2003; Schlesinger, 2005). One of the leading theoretical explanations for

these disparities is that pretrial decisions are governed more by informal norms and organi-

zational practices than by formal legal rules (Sutton, 2013). As judges and prosecutors

receive incomplete information and maintain broad discretion during the pretrial process,

they are believed to inadvertently base their decisions in part on stereotypes about sex and

racial-ethnic groups (Albonetti, 1991; Sutton, 2013). These stereotypes are derived from the

long-standing structural and cultural properties of American society that have designed a

justice system inherently biased against certain sex and racial-ethnic groups. If males and

people of color are perceived to pose a greater threat for failing to return to court and engag-

ing in new criminal conduct, then judges and prosecutors may be more likely to take action

for detaining individuals with these characteristics regardless of the actual risk that each

person may pose.

It is critical to ensure that pretrial processes are administered fairly and equitably (Van

Brunt & Bowman, 2018). One strategy introduced as a potential mechanism for mitigating

against the disparate treatment of people charged with crimes is the use of an actuarial pre-

trial instrument (Desmarais et al., 2022). The predictive validity of several pretrial instru-

ments in identifying one’s likelihood for new criminal activity while out on release has been

established in numerous jurisdictions across the United States (Desmarais et al., 2021).

Nevertheless, support for pretrial instruments has been hindered by the potential to exacer-

bate sex and racial-ethnic disparities (Eckhouse et al., 2019; Woldgabreal et al., 2020). This

potential to exacerbate sex and racial-ethnic disparities is a concern associated with the

long-standing history of the criminal legal system and American society. The concern is if

there are systematic differences in the predictions of an instrument across groups, then

implementing these tools could worsen the disparities observed in the criminal legal sys-

tem. Recent research on pretrial instruments has largely focused on evaluating scoring and

prediction differences between sex or racial-ethnic groups separately, with the findings

showing little evidence of bias (DeMichele et al., 2020; Desmarais et al., 2021). Although

this scholarship is integral to understanding and developing strategies to reduce predictive

bias in pretrial assessments (Cohen & Lowenkamp, 2019), the lack of research on the inter-

sectionality of sex and race-ethnicity represents a gap in the literature (see Brittain et al.,

2021). This gap in knowledge is particularly concerning given that more than half of all

pretrial agencies report using an assessment (Lattimore et al., 2020).

We extend prior research by moving away from the “tendency to use a single-axis frame-

work that treats race, ethnicity, and sex as mutually exclusive categories of experience and

analysis” (Crenshaw, 1989, p. 139). More specifically, we conduct a retrospective valida-

tion and test for differential validity and differential prediction (e.g., varying slopes by

subgroups) in a widely used pretrial instrument—the Public Safety Assessment