Examining Optimal Weights for the Youth Assessment and Screening Instrument in North Dakota

• Published date: 01 February 2025
• DOI: http://doi.org/10.1177/00938548241291015
• Author: Sonya A. Mclaren,Danielle J. Rieger
• Date: 01 February 2025
• Subject Matter: Articles

CRIMINAL JUSTICE AND BEHAVIOR, 2025, Vol. 52, No. 2, February 2025, 181 –196.

DOI: https://doi.org/10.1177/00938548241291015

Article reuse guidelines: sagepub.com/journals-permissions

© 2024 International Association for Correctional and Forensic Psychology

181

EXAMINING OPTIMAL WEIGHTS FOR THE

YOUTH ASSESSMENT AND SCREENING

INSTRUMENT IN NORTH DAKOTA

SONYA A. MCLAREN

DANIELLE J. RIEGER

Carleton University

The purpose of deriving weights for factors is to identify and discover what items have the strongest relation to the outcome

of interest to create an assessment with the best predictive validity. The current study examined and compared weighted and

nonweighted models for the pre-screen of the Youth Assessment and Screening Instrument (YASI). The total sample of 6,175

youth from the state of North Dakota, United States, was divided into the calibration sample (33%) to build the models and

was cross-validated across five independent subsets. The results were that no method consistently nor substantially outper-

formed the original method. The one exception was the risk total score produced through the logistic regression model sub-

stantially improved the predictive accuracy for violent dispositions after the initial YASI administration in the validation

subsets. Therefore, it may be worth considering alternative models and warrants further exploration for the users of the scale.

Keywords: weighting methods; predictive accuracy; juvenile offending

Actuarial assessments are used within the criminal justice system throughout the deci-

sion-making process from pre-trial to sentencing to discharge. These types of assess-

ments use statistical models or algorithms to classify individuals (Silver, 2000; Smid, 2014).

Thus, actuarial assessments are believed to be the optimal predictive tools for decision-

making (Hannah-Moffat & Shaw, 2001). Given the significance and life-altering implica-

tions of these decisions, it is imperative that these decisions are made based on assessments

that are accurate and reliable. Thus, it is critical that the items within these scales are appro-

priately and proportionally related toward an individual’s overall risk and need to make fair,

defensible, and transparent decisions that are empirically and quantitatively supported.

Moreover, it is imperative that these assessments have high predictive accuracy, given

AUTHORS’ NOTE: The manuscript reflects the views of the authors, and not those of Orbis Partners Inc. or

the North Dakota Court System. The authors would like to thank the North Dakota Court System, Orbis

Partners Inc., and Dr. Andrea Howard for their support in this research project. Danielle Rieger is affiliated

with Orbis Partners, the developer of the Youth Assessment and Screening Instrument (YASI; Orbis Partners,

2000). Sonya McLaren was previously affiliated with Orbis Partners. Correspondence concerning this article

should be addressed to Sonya A. McLaren, Psychology Department, Carleton University, Ottawa, Ontario,

Canada K1S 5B6; e-mail: sonyamclaren@cmail.carleton.ca.

1291015CJBXXX10.1177/00938548241291015Criminal Justice and BehaviorMcLaren and Rieger / Optimal Weights for the YASI in a Midwestern State

research-article2024

182 CRIMINAL JUSTICE AND BEHAVIOR

decisions are made based on their results. However, within the field, there is little research

for the optimal derivation of weights for assessments to improve predictive accuracy.

Weights in this case are values given to risk or strength factors in actuarial models for the

evaluation of risk in recidivism (Grann & Långström, 2007). Relatedly, subpar indicators

with small to moderate predictive accuracy are used, as they are the “best” that are avail-

able. It is worthwhile to optimize the weighting scheme for items on actuarial tools because

it might improve predictive accuracy. In turn, assessments that have better predictive accu-

racy are likely to provide information on how to best allocate resources and services based

on an individual’s needs. Overall, this would lead to more data-driven decision-making that

would reduce biases while increasing public confidence and public safety. The current study

examined the optimal weights for the items on the pre-screen of the Youth Assessment and

Screening Instrument (YASI; Orbis Partners, 2000), by comparing weighting and non-

weighted models, for youth in the state of North Dakota, United States, by testing whether

alternative weighting schemes improve predictive accuracy.

WEIGHTING ITEMS

The purpose of deriving weights for items, also known as factors, is to identify and dis-

cover what items have the strongest relation to the outcome of interest to create an assess-

ment with the strongest ability to predict an outcome of interest, in this case recidivism

(predictive validity). Recidivism occurs when an individual who has previously committed

a crime re-offends. Rather than assigning each item equivalent weights (e.g., 0 when the

risk factor is absent and 1 when it is present) and then summing all the items to a total score,

each factor can be considered in terms of its relation to recidivism (Grann & Långström,

2007). That is, items that are more closely related and directly influence recidivism can be

given greater weight, compared with items that are less related. In the criminal justice field,

predictive validity needs to be optimal to be to be able to trust these assessments, given the

significance of decisions for the individuals themselves, as well as maintaining public

safety.

The current study compared weighting methods that were chosen to be consistent with

the guidelines proposed by Grann and Långström (2007) and Rieger and Serin (2021): the

original method, Burgess, Nuffield, and logistic regression. These studies found that the

weighting methods did not significantly improve predict accuracy over the nonweighted

approach. This finding is consistent with other researchers who have found no significant

improvement when weighting methods are considered, such as when using logistic regres-

sion compared with more complex methods, such as classification tree models or neural

network models (Dawes, 1979; Liu et al., 2011; Wainer, 1976; Walters, 2011). Robb (2021)

examined weighting methods for an assessment evaluating criminogenic risks, needs, and

strengths for adult women. Robb (2021) found that the gender-responsive Nuffield coding

was the superior method compared with the Burgess and traditional Nuffield method. In

comparison, others have argued that significant improvements can be garnered when the

samples sizes are large enough (Einhorn & Hogarth, 1975; Hosmer et al., 2013; Silver et al.,

2000). Therefore, given conflicting findings and the lack of evaluation of weighting meth-

ods for strength items, the current study was justified.