Optimizing Youth Risk Assessment Performance: Development of the Modified Positive Achievement Change Tool in Washington State
| Published date | 01 August 2019 |
| DOI | 10.1177/0093854819857108 |
| Date | 01 August 2019 |
| Subject Matter | Articles |
research-article2019
Optimizing YOuth Risk Assessment
peRfORmAnce
Development of the modified positive Achievement
change tool in Washington state
ZACHARY HAMILTON
Washington State University, Spokane
MELISSA A. KOWALSKI
The College at Brockport, State University of New York
ALEX KIGERL
Washington State University, Spokane
DOUGLAS ROUTH
Center for Advancing Correctional Excellence (ACE!)
The Positive Achievement Change Tool (PACT) was developed in 1997 using a theoretical construction of items, responses,
and weights. While derived from an original tool created for a Washington State probation population, the risk-need assess-
ment is one of the most widely used youth tools utilized today. To advance the model from its theoretical construction, the
current study demonstrates tool updates making use of a large sample of Washington State youth (N = 50,862). Specifically,
several mechanisms were utilized to customize the assessment, including (a) item weighting, (b) outcome specificity, and (c)
gender responsivity. Based on the updated design, we identify improvements in predictive validity of the continuous risk
scale, accuracy of risk-level assignment, and reductions in racial/ethnic disparity. Scheduled to be implemented in the coming
year, this article describes the development of the Modified Positive Achievement Change Tool (M-PACT).
Keywords: risk assessment; predictive validity; recidivism; justice-involved youth; juvenile justice
intRODuctiOn
The primary goals of the juvenile system in the United States are prevention and rehabili-
tation, as youth are believed to be more capable of change (Platt, 1977). A first step in
AuthORs’ nOte: This article reflects only the opinions of the authors and does not necessarily reflect
the official opinion of the Administration Office of the Courts and the Washington State Juvenile Court
Administrators. Correspondence concerning this article should be addressed to Zachary Hamilton,
Department of Criminal Justice and Criminology, Washington State University, Spokane, WA 99210; e-mail:
zachary.hamilton@wsu.edu.
CRIMINAL JUSTICE AND BEHAVIOR, 2019, Vol. 46, No. 8, August 2019, 1106 –1127.
DOI: 10.1177/0093854819857108
https://doi.org/10.1177/0093854819857108
Article reuse guidelines: sagepub.com/journals-permissions
© 2019 International Association for Correctional and Forensic Psychology
1106
Hamilton et al. / M-PACT IN WASHINGTON STATE 1107
achieving said goals involves the proper identification of risk to better allocate resources.
This avenue involves the Risk principle and reserving services for high-risk offenders, as
they are more likely to benefit from such services (Andrews & Bonta, 2010).
Following the Risk-Need-Responsivity (RNR) framework, the use of risk assessment
has grown substantially in the last two decades. Since their initial expansion from adult to
youth tools, assessments have advanced to provide ever-increasing detail and prediction
(see Andrews & Bonta, 2010; Hamilton et al., 2016). However, while advanced modeling
efforts have sought to improve the accuracy of adult assessments (Hamilton, Campagna,
Tollefsbol, van Wormer, & Barnoski, 2017; Taxman, 2017), risk assessments for justice-
involved youth have notably lagged (Vincent, Laura, & Grisso, 2012). Appropriate assess-
ment of justice-involved youths’ risk and needs is a core focus in juvenile justice reform in
Washington State specifically (Hamilton, van Wormer, & Barnoski, 2015). In an effort to
advance juvenile justice assessment and prediction, the current study sought to improve one
of the first tools developed, and currently utilized, in Washington State.
DevelOpment Of the pAct
Working with the Washington Association of Juvenile Court Administrators (WAJCAs),
Barnoski (2004a) provided one of the initial advances for risk assessment generally, and
youth assessments specifically, through the creation of the Washington State Juvenile Court
Assessment (WSJCA). Originally implemented in 1997, the WSJCA was developed to
improve use of evidence-based practices and to facilitate treatment and case planning ser-
vices based on risk and needs (Hamilton, Neuilly, Lee, & Barnoski, 2015). At the time of
development, the tool was viewed as one of the most advanced instruments, pairing risk,
need, and protective items. The tool makes use of a semistructured interview with youth and
their families, identifying level of risk and a rating of needs across several dimensions.
The WSJCA includes 126 items across 12 domains and provides a detailed assessment of
youths’ risks and needs. The domains include the following: Criminal History, Demographics,
School, Use of Free Time, Employment, Relationships, Family, Alcohol and Drugs, Mental
Health, Attitudes/Behaviors, Aggression, and Skills. Each domain has a risk score, and
many also provide a protective score that results in an overall risk category. Weighted scores
on a subset of 21 items are used to create Criminal and Social History scores. Risk level is
then determined via a set of cut points, categorizing youth as Low, Moderate, or High Risk.
Many youth are also administered the Full Assessment (k = 126), which further identifies
youths’ level of need within each of the 10 dynamic domains. Reassessments are conducted
every 6 months to determine youth changes in risk and needs classification.
Due in part to its advanced status and its nonproprietary provision, the WSJCA was
developed for software applications and eventually rebranded by two companies (Barnoski,
2004a) as the Positive Achievement Change Tool (PACT) and the Youth Assessment and
Screening Instrument (YASI). The assessment was also adopted for agency-based software
applications by multiple states (e.g., Iowa, Oregon, and Utah; Baird et al., 2013). Although
“change” is in the name of the tool, the assessment itself is not intended to create change in
youth behavior. Rather, this terminology refers to change in scores across reassessments, and
variations can be identified through the needs portion of the assessment. In other words,
dynamic needs scales are assessed separately in the PACT, better specifying change over time.
In the 20 years since its establishment, the PACT has been utilized in over 20 states, with
an underlying theoretical model that is relatively unchanged. The intent at the time of
1108 CRIMINAL JUSTICE AND BEHAVIOR
construction was that items, responses, and domains would be updated once sufficient data
were collected (Barnoski, 2004a). Furthermore, RNR best practices outline the need to
reevaluate instrument performance and make modifications where needed (Andrews &
Bonta, 2010).
Like many generalized assessment tools, the PACT was adopted for a variety of popula-
tions and agencies. To provide greater optimization, we argue that the original set of items,
responses, scoring, and cut points of an assessment should be adjusted to account for local
variation. The current study serves to modify the theoretically composed tool, which
includes removing items not predictive of recidivism and rearranging current items and
scales to increase predictive performance. The updated models derived from these efforts
represent a new and improved version of the PACT for Washington State, termed the
Modified Positive Achievement Change Tool (M-PACT). The first part of this update was
outlined through preliminary analyses by Hamilton, van Wormer, and Barnoski in 2015,
exploring the potential advancement of the tool in a large sample (n = 32,699) of probation
youth using advanced statistical algorithms. Initial findings indicated substantial improve-
ments in predictive strength; however, greater methodological advancements were desired,
and subject matter experts (SMEs) requested several variations and recategorization of
study outcomes. Based on prior findings modifying adult assessment tools (Hamilton et al.,
2017; Hamilton et al., 2016) and in collaboration with the WAJCAs, the current study
sought to tailor the M-PACT for Washington State probation youth.
Several mechanisms were utilized to customize the M-PACT, including (a) item weight-
ing, (b) outcome specificity, and (c) gender responsivity. The current study thus aimed to
update the original, exploratory work completed, this time using SMEs to help identify the
categorization of crime types to be predicted and to vet the produced models for content
validity and practical utility. SME input helped further craft M-PACT items/responses
included in the analysis. Based on an updated design, the new M-PACT scoring was then
used to identify type and level of risk for youth in Washington State. Prior to the description
of analyses completed, the proceeding section explores issues relevant to the risk assess-
ment field.
liteRAtuRe RevieW
Youth risk assessments are now a common practice in the juvenile justice system (Vincent
et al., 2012), but tool updates represent a best practice needed to improve accuracy. Although
risk instruments are sometimes adjusted, many have not yet been customized to jurisdic-
tions (Hamilton et al., 2016). Furthermore, tools are often adopted “off-the-shelf” and not
modified to account for items predictive of reoffending (e.g., weighting), varying outcomes,
and gender responsivity. This section reviews issues associated with contemporary risk
assessments and ways to tailor instruments to...
To continue reading
Request your trial