An Examination of the Professional Override of the Level of Service Inventory–Ontario Revision

• Author: Laura C. Orton,Neil R. Hogan,J. Stephen Wormith
• Published date: 01 April 2021
• Date: 01 April 2021
• DOI: 10.1177/0093854820942270
• Subject Matter: Articles

CRIMINAL JUSTICE AND BEHAVIOR, 2021, Vol. 48, No. 4, April 2021, 421 –441.

DOI: https://doi.org/10.1177/0093854820942270

Article reuse guidelines: sagepub.com/journals-permissions

© 2020 International Association for Correctional and Forensic Psychology

421

AN EXAMINATION OF THE PROFESSIONAL

OVERRIDE OF THE LEVEL OF SERVICE

INVENTORY–ONTARIO REVISION

LAURA C. ORTON

Ministry of the Solicitor General, Government of Ontario

NEIL R. HOGAN

Alberta Ministry of Justice and Solicitor General

University of Saskatchewan

J. STEPHEN WORMITH

University of Saskatchewan

This study examined the nature and impacts of the professional override on the Level of Service Inventory–Ontario Revision

(LSI-OR), using a large archival database of 40,539 individuals’ information. Research questions focused on the predictive

validity of various LSI-OR risk metrics, including total risk/need scores, initial risk categories, and adjusted risk categories,

for various types of recidivism; how professional overrides were used; whether they were used more with some groups than

others; and whether their impacts varied depending on recidivism type. Overrides were applied in 15.4% of cases, most often

(94.1%) to increase risk levels. Override use varied based on gender, race, and the nature of index offenses. Based on receiver

operating characteristic analyses, the results generally indicated that adjusted risk levels (incorporating professional over-

rides) demonstrated inferior predictive validity relative to unadjusted metrics. The results suggest a need for increased caution

and consistency in the application of professional overrides.

Keywords: risk assessment; professional override; LSI-OR; LS/CMI; LS/RNR; recidivism

Forensic risk assessment measures are used in various criminal justice, law enforcement,

and health care contexts. These measures have been described as evolving through four

generations (Bonta, 1996), from unstructured clinical judgments (first generation), to statisti-

cally based and atheoretical actuarial tools (second generation), to theoretically informed

structured instruments comprising at least some dynamic variables (third generation), and

most recently, to tools that incorporate an ongoing case management component (fourth

AUTHORS’ NOTE: Both Laura Orton and Neil Hogan gratefully acknowledge the invaluable support and

influence of Dr. Stephen Wormith on their academic careers, and wish to dedicate this article to his memory.

This manuscript stems from a master’s thesis project undertaken by Laura Orton, under the supervision of J.

Stephen Wormith. Dr. J. Stephen Wormith received royalties from sales of the Level of Service/Case Management

Inventory from its publisher, Multi-Health Systems. The views expressed in this article do not necessarily reflect

the views of the authors’ respective organizations. Correspondence concerning this article should be addressed

to Neil R. Hogan, Integrated Threat and Risk Assessment Centre, Alberta Ministry of Justice and Solicitor

General, ALERT West Campus, Edmonton, Alberta, Canada T5S 0C2; e-mail: neil.hogan@usask.ca.

942270CJBXXX10.1177/0093854820942270Criminal Justice and BehaviorOrton et al. / Professional Override of the LSI-OR

research-article2020

422 CRIMINAL JUSTICE AND BEHAVIOR

generation). Although a large body of research has demonstrated that structured approaches

outperform first-generation approaches with regard to the prediction of recidivism (Andrews

et al., 2006; Grove et al., 2000), contemporary forensic assessors and researchers have

embraced, and debated the merits of, varied approaches (Singh et al., 2014).

THE EVOLUTION OF THE CLINICAL VERSUS ACTUARIAL DEBATE

Reviews of clinical and actuarial prediction techniques have tended to favor the latter

category, whether they have focused on wide-ranging fields (Dawes et al., 1989; Grove

et al., 2000; Meehl, 1954) or focused on the prediction of recidivism in particular (Ægisdottir

et al., 2006). Based on such findings, Ægisdottir and colleagues argued that disregarding the

higher performing actuarial approaches could be unethical and may work to the detriment

of public safety and individual rights. However, other researchers and practitioners have

challenged the purported supremacy of strict actuarial techniques. Closely aligned with

arguments on this side of the debate is the widespread use of the structured professional

judgment (SPJ) approach. SPJ measures were developed to emphasize structure, preven-

tion, and flexibility (Hart & Boer, 2009). Unlike actuarial tools that combine data in a pre-

determined fashion, SPJ tools typically require the evaluator to consider a number of risk

factors individually, and then formulate final appraisals of risk or case prioritization on the

balance of information. The manner of combining the data to formulate conclusions is left

to the discretion of the evaluator. Arguments in favor of this approach have criticized the

methodological rigor of actuarial research (e.g., Litwack, 2002), challenged the applicabil-

ity of group data to individuals (Hart et al., 2007), and emphasized the identification of risk

management targets over statistical predictors. Although these tools have been supported by

meta-analyses treating them in an actuarial manner (Campbell et al., 2009; Rossdale et al.,

2020; Yang et al., 2010) or in a binary manner (i.e., low vs. moderate/high; Fazel et al.,

2012), such evaluative strategies deviate from the tools’ clinical application. That said, a

meta-analytic review contained in an unpublished doctoral dissertation (Chevalier, 2017)

found that SPJ summary risk ratings demonstrated predictive validity for recidivism.

Additional studies that were not included in Chevalier’s review have also found support for

SPJ summary risk ratings, both in terms of overall risk/case prioritization (e.g., Hogan &

Olver, 2019; Vargen et al., 2020) and in terms of imminent risk of institutional outcomes

(e.g., Hogan & Olver, 2016, 2018).

COMBINING CLINICAL AND ACTUARIAL DATA AND THE PROFESSIONAL

OVERRIDE

In light of the aforementioned debate, still other authors have argued for a combination

of actuarial and clinical decision making. Webster et al. (2002), for instance, argued for the

use of clinical information to supplement actuarial and prediction-based assessment mea-

sures to form a more complete assessment report and integrated understanding of each case.

Evaluators may wish to increase a person’s risk level if a risk factor appears to be driving

offending behavior, but is not captured by a particular tool; alternatively, they may wish to

reduce one’s risk level if she is actuarially high risk, but physically incapacitated. Practically

speaking, professional overrides provide one mechanism for combining such data. For

instance, the principle of the professional override has been included in the level of service