Probation Officer Assessments of Risk when the Youth Look Different: Contributions of Structured Professional Judgment to Concerns About Racial Bias

• Author: Gina M. Vincent,Rachael T. Perrault,Carla G. Munoz
• Published date: 01 April 2021
• Date: 01 April 2021
• DOI: 10.1177/1541204020954264
• Subject Matter: Articles

Article
Youth Violence and Juvenile Justice
2021, Vol. 19(2) 206-226
Probation Officer
ª The Author(s) 2020
Article reuse guidelines:
Assessments of Risk when the
sagepub.com/journals-permissions
DOI: 10.1177/1541204020954264
journals.sagepub.com/home/yvj
Youth Look Different:
Contributions of Structured
Professional Judgment to
Concerns About Racial Bias
Carla G. Munoz, PhD1 , Rachael T. Perrault, MA2,
and Gina M. Vincent, PhD2
Abstract
Various groups have expressed considerable concern about the potential for actuarial risk assess-
ments to exacerbate racial disparities in justice settings. This study examined that potential when
using a different approach to risk assessment, structured professional judgment (SPJ), by comparing
risk decisions made by evaluators when the examinee’s race was different versus the same as theirs.
A large sample of youth (N ¼ 1,308) evaluated on the Structured Assessment of Violence Risk in
Youth (SAVRY) by 137 juvenile probation officers (JPOs) in five states indicated the only moderation
effect for the match or mismatch between JPOs’ and youths’ race/ethnicity was in the weight JPOs
placed on five (out of 24) risk factors in their overall risk opinions. The match between JPOs’ and
youths’ race had no bearing on JPOs’ final determination of youths’ risk levels. This study lends
support for investigating the use of SPJ instruments as a method for minimizing racial bias.
Keywords
risk assessment, structured professional judgment, race/ethnicity, juvenile, probation officers
Risk assessment instruments are often used by personnel in justice settings, especially probation
officers, to inform decisions at various points in the legal process in which risk to public safety is
relevant (e.g., pretrial release, probation and parole case management strategies, National Resource
Council, 2013; Starr, 2015). Violence risk instruments are used in multiple countries (e.g., United
States, Canada, Amsterdam, England, Spain, Australia) by multiple parties in different settings (e.g.,
1 Bridgewater State Hospital, Bridgewater, MA, USA
2 University of Massachusetts Medical School, Worcester, MA, USA
Corresponding Author:
Gina M. Vincent, PhD, Department of Psychiatry, University of Massachusetts Medical School, 222 Maple Ave, Shrewsbury,
MA 01545, USA.
Email: gina.vincent@umassmed.edu

---

Munoz et al.
207
police, forensic psychologists and psychiatrists, mental health providers’, correctional staff, Singh
et al. [2014]). At least 20 states of the U.S. use risk instruments to guide sentencing decisions (Starr,
2015), up to 28 states use them to guide parole release decisions (Harcourt, 2007), and most state
juvenile justice systems have adopted risk assessment instruments to assist with dispositional plan-
ning (Wachter, 2015). The value of these instruments is to increase the accuracy of human judg-
ments regarding one’s likelihood of reoffending or being violent in the future (Grove & Meehl,
1996). For some instruments, another value-added is to identify malleable risk factors that can be
targeted for programming in order to reduce risk (Douglas & Skeem, 2005). Research indicates that
validated risk assessment instruments are less susceptible to inaccuracies about individual’s risk of
harming others than are unstructured, professional’s judgments (Ægisd´ottir et al., 2006; Hanson &
Morton-Bourgon, 2009), and, if properly implemented, may reduce unnecessary incarceration (Vil-
joen, Jonnson, et al., 2019; Vincent et al., 2016). However, recently, many groups have raised
concerns about these instruments being racially biased (e.g., Robinson & Koepke, 2019; Starr,
2015).
The recent concerns about risk instruments include views that the instruments may actually
exacerbate unwanted racial disparities. This issue gained increased attention largely due to Former
Attorney General Holder’s statement about the propensity for use of risk assessment tools in
sentencing decisions to “ . . . exacerbate unwarranted and unjust disparities that are already far
too common in our criminal justice system and in our society.” (2014). Similarly, Mr. Wroblewski
(2014), Director of the Department of Justice Office of Policy and Legislation expressed concerns
that risk assessments base individual-level decisions partially on group characteristics and static
risk factors; and therefore, instruments may be placing people of color at a disadvantage, leading
to more severe sentencing decisions. Several works have now surfaced that suggest risk instru-
ments; namely pretrial risk instruments and others that classify individuals’ risk using formulas
designed to optimize the prediction of recidivism, introduce racial bias into decisions (e.g.,
Barabas et al., 2019; Lason et al., 2016; St. John et al., 2020). However, there is another class
of valid risk assessment instruments, known as structured professional judgment instruments (e.g.,
Structured Assessment of Violence Risk in Youth [SAVRY], Borum et al., 2006; Historical
Clinical Risk-20 Scale Version 3 [HCR-20V3], Douglas et al., 2013; Short-Term Assessment of
Risk and Treatability [START], Webster et al., 2009), which are not dependent on formulas in
their design or in the manner with which risk level is determined. There is some evidence that
these instruments have equivalent accuracy in the prediction of reoffending across racial and
ethnic groups (e.g., Perrault et al., 2017; Snowden et al., 2010; Vincent et al., 2011). However,
structured professional judgment (SPJ) risk instruments may still introduce bias because they
permit more rater discretion in the determination of one’s risk level than score-based tools. The
current study examines this issue by testing whether juvenile probation officers (JPOs) introduce
biases into their risk level decisions when using a SPJ risk instrument, the SAVRY (Bartel et al.,
2002), with justice-involved youth from a different race or ethnicity than the JPO.
Risk Assessment Instruments and Racial Bias
Risk assessment instruments vary with respect to the types of risk factors included, the methods of
construction, and how the final risk decision is made. Risk factors differ in that some are highly static
(e.g., history of violence, number of prior offenses), while others are highly malleable or dynamic
(e.g., substance abuse, whether one’s primary peer group is antisocial). Risk/needs assessment
instruments used for dispositional or case planning include both static and dynamic risk factors.
Only a few include protective factors (e.g., the SAVRY and the Structured Assessment of Protective
Factors, SAPROF), which have been described as moderators or “buffers” to risk factors (Jessor

---

208
Youth Violence and Juvenile Justice 19(2)
et al., 1995). The construction of instruments differ in the level of reliance on statistical methods.
Additionally, the methods used by instruments differ in the level of rater discretion permitted.
The two primary methods for constructing an instrument and for making the final risk estimate
are actuarial and structured professional judgment (Monahan, 2008). Actuarial decision-making
means the determination of one’s risk level is formulated based on a statistical formula or a sum
of the items. These instruments are non-discretionary in terms of how risk factors are selected or
combined, and the final risk level decision is determined by a fixed formula. Many, but certainly not
all, actuarial instruments comprise primarily static risk items selected based on which factors were
associated with recidivism (generally re-arrest) in a given sample of justice-involved individuals
(the development sample). These instruments select risk items to generate a summed total score or
an algorithm (weighting risk items) to maximize the prediction of recidivism within that develop-
ment sample. This score is used to categorize future individuals into different levels based on their
likelihood of reoffending (e.g., low, moderate, high). The notion that the accuracy of objective,
actuarial instruments is superior to unstructured, professional judgment in the prediction of violence
and reoffending is fairly well-established (e.g., Grove et al., 2000). However, static actuarial instru-
ments have been criticized for being atheoretical and non-discretionary, for focusing on unchange-
able characteristics and relying on mathematical formulas without considering idiosyncratic
circumstances (Bonta, 1996; Guy et al., 2015), for doing little to guide treatment planning or risk
management, and recently, for introducing bias that is intrinsic (Hart, 2016) or statistically unde-
tectable because the outcome used to generate the formula (e.g., official re-arrest) is often racially
biased (Barbaras et al., 2019).
Structured professional judgment (SPJ) instruments on the other hand, also impose structure
with respect to which risk factors should be considered and how these items should be rated. The
risk factors are selected based on theory and research on delinquency rather than use of statistical
methods with a development sample. The final judgment of an individual’s risk level (typically
communicated as low, moderate, or high risk) is determined by a rater or evaluator based on a
combination of risk and protective factors, and idiosyncratic characteristics of the individual being
assessed. No algorithm is used; instead factors are combined or weighted based on the evaluator’s
appraisal of their relevance to a specific individual’s reoffending (Borum et al., 2006; Douglas
et al., 2013). Meta-analytic results indicate that the predictive...