Professional Discretion and the Predictive Validity of a Juvenile Risk Assessment Instrument

• Author: James T. McCafferty
• Date: 01 April 2017
• Published date: 01 April 2017
• DOI: 10.1177/1541204015622255
• Subject Matter: Articles

Article

Professional Discretion

and the Predictive Validity

of a Juvenile Risk Assessment

Instrument: Exploring

the Overlooked Principle

of Effective Correctional

Classification

James T. McCafferty

1

Abstract

The ability for professionals to override the results of an actuarial risk assessment tool is an essential

part of effective correctional risk classification; however, little is known about how this important

function affects the predictive validity of these tools. Using data from a statewide sample of juveniles

from Ohio, this study examined the impact of professional adjustments on the predictive validity of a

juvenile risk assessment instrument. This study found that the original and adjusted risk levels were

significant predictors of recidivism, but the original risk levels were stronger predictors of recidivism

than the adjusted risk levels that accounted for overrides.

Keywords

override, risk assessment, predictive validity, juvenile justice

Introduction

Actuarial risk assessments are an important component of evidence-based correctional practices (see

Andrews, Bonta, & Hoge, 1990; Harris, Rice, Quinsey, & Cormier, 2015; Howell, 2009). The use of

these tools has received a large amount of empirical support from researchers for their ability to

predict juvenile recidivism (e.g., Olver, Stockdale, & Wormith, 2009; Schwalbe, 2007, 2008).

These tools can also normalize the decision-making process across justice professionals by using

a standard set of criteria on which to base decisions of risk. Without a standard set of criteria,

1

Department of Sociology and Criminal Justice, Kennesaw State University, Kennesaw, GA, USA

Corresponding Author:

James T. McCafferty, Department of Sociology and Criminal Justice, Kennesaw State University, 402 Bartow Avenue, MD

2204, Kennesaw, GA 30144, USA.

Email: jmccaff4@kennesaw.edu

Youth Violence and JuvenileJustice

2017, Vol. 15(2) 103-118

ªThe Author(s) 2015

Reprints and permission:

sagepub.com/journalsPermissions.nav

DOI: 10.1177/1541204015622255

journals.sagepub.com/home/yvj

professionals use undefined sets of criteria on which to base their decisions, which are typically

described as highly subjective due to their reliance on personal biases and beliefs (Holsinger,

Lurigio, & Latessa, 2001). Furthermore, clinical decision-making is less reliable because it does not

have strong predictive validity in comparison with actuarial instruments (Ægisdo´ttir et al., 2006;

Gottfredson & Moriarty, 2006; Hilterman, Nicholls, & van Nieuwenhuizen, 2014; Oleson, van

Benschoten, Robinson, & Lowenkamp, 2011).

Despite a plethora of evidence, which illustrates that standardized risk assessments are superior to

clinical assessments in several ways, many actuarial tools still allow the opportunity for professionals

to disagree with the tool’s conclusion by overriding an assigned risk score or level. For example, if

after completing theassessment the professional disagrees with an assigned risk level (e.g., low-risk),

the assessor has the opportunity to use their professional judgment to adjust the risk level (e.g.,

moderate- or hig h-risk) to refle ct their professio nal conclusions a bout the risk of t he juvenile. This

feature, typically referred to as professional discretionor override, is an essential partof the foundation

for effective offender classification (Andrews et al., 1990). The ability to use overrides has become a

standard feature on many juvenile risk assessments; however, the impact of these decisions on the

predictive validity of actuarial tools has received only a small amount of attention from researchers.

Supporters of overrides claim that the inclusion of professional discretion and expertise can

increase the accuracy of the tool by allowing for the inclusion of risk factors and information not

considered by the assessment instrument (e.g., disposition of the youth during interactions with

justice officials or history of parental abuse). In fact, it can be argued that the very existence of the

override process is an obvious recognition that these instruments are not completely exhaustive in

regard to the information collected about the subject. Recent commentators, however, have cau-

tioned against the likelihood that clinical decisions designed to supplement an actuarial instrument

would improve predictive accuracy (DeClue & Zavodny, 2014; Harris et al., 2015; Wormith, 2014).

Utilizing a sample of male and female youth from Ohio who were assessed with the Ohio Youth

Assessment System–Disposition Instrument (OYAS-DIS), the current study contributes to the

existing body of research on professional discretion by exploring the impact of overrides on the

predictive validity of the risk assessment tool.

Literature Review

The risk principle simply states that the level of correctional supervision and treatment should be

commensurate with the risk level of the juvenile (Lowenkamp & Latessa, 2004; Lowenkamp,

Latessa, & Holsinger, 2006). When risk is correctly identified, the juvenile justice system has a bet-

ter chance of reducing recidivism (e.g., Vincent, Guy, Gershenson, & McCabe, 2012), but, when risk

is incorrectly identified, there is a possibility for negative outcomes. For instance, if risk is overes-

timated, there is a potential for an ‘‘iatrogenic effect’’—that is, there is a possibility that a juvenile

would become more criminal than had he or she been handled at the correct, lower risk level (Gatti,

Tremblay, & Vitaro, 2009; L. B. Lovins, Lowenkamp, Latessa, & Smith, 2007; Lowenkamp &

Latessa, 2004). Conversely, if risk is underestimated, it could result in the juvenile having a greater

amount of criminal opportunity, given the less restrictive controls that may be in place as the result

of a false-negative risk assessment conclusion.

To help identifyrisk, justice officials shouldrely on assessments that canvalidly predict recidivism

(Bonta, 2002). Fortunately, there are many tools available for use by practitioners that have been

empirically supported by past research(see, e.g., Baglivio & Jackowski, 2013;Barnoski, 2004; Bech-

tel, Lowenkamp, & Latessa, 2007; Childs et al., 2013; Gretton, McBride, Hare, O’Shaughnessy, &

Kumka, 2001; McGrath & Thompson, 2012; Onifade, Davidson, & Campbell, 2009; Schwalbe,

2009; Schwalbe,Fraser, Day, & Cooley, 2006). Whilethese individual studies shed lighton the effec-

tiveness of thesetools, the overall strengthof the predictive validityof juvenile risk assessmentsis best

104 Youth Violence and Juvenile Justice 15(2)