A Principled Approach to the Construction of Risk Assessment Categories: The Council of State Governments Justice Center Five-Level System

• Date: 01 August 2020
• Author: Megan M. Morrison,Evan M. Lowder,Daryl G. Kroner
• Published date: 01 August 2020
• DOI: 10.1177/0306624X19870374
• Subject Matter: Articles

https://doi.org/10.1177/0306624X19870374

International Journal of

Offender Therapy and

Comparative Criminology

2020, Vol. 64(10-11) 1074 –1090

© The Author(s) 2019

Article reuse guidelines:

sagepub.com/journals-permissions

DOI: 10.1177/0306624X19870374

journals.sagepub.com/home/ijo

Original Manuscript

A Principled Approach to

the Construction of Risk

Assessment Categories:

The Council of State

Governments Justice

Center Five-Level System

Daryl G. Kroner1, Megan M. Morrison2,

and Evan M. Lowder3

Abstract

Consistent risk category placement of criminal justice clients across instruments

will improve the communication of risk. Efforts coordinated by the Council of State

Governments (CSG) Justice Center led to the development of a principled (i.e., a

system based on a given set of procedures) method of developing risk assessment

levels. An established risk assessment instrument (Level of Service Inventory–Revised

[LSI-R]) was used to assess the risk-level concordance of the CSG Justice Center

Five-Level system. Specifically, concordance was assessed by matching the defining

characteristics of the data set with its distribution qualities and by the level/category

similarity between the observed reoffending base rate and the statistical probability of

reoffending. Support for the CSG Justice Center Five-Level system was found through

a probation data set (N = 24,936) having a greater proportion of offenders in the lower

risk levels than a parole/community data set (N = 36,303). The statistical probabilities

of reoffending in each CSG Justice Center system risk level had greater concordance

to the observed Five-Level base rates than the base rates from the LSI-R original

categories. The concordance evidence for the CSG Justice Center Five-Level system

demonstrates the ability of this system to place clients in appropriate risk levels.

1Southern Illinois University Carbondale, USA

2Tennessee State University, Nashville, USA

3Indiana University, Indianapolis, USA

Corresponding Author:

Daryl G. Kroner, Department of Criminology and Criminal Justice, Southern Illinois University

Carbondale, Faner Hall, Mail Code 4504, 1000 Faner Drive, Carbondale, IL 62901-4328. USA.

Email: dkroner@siu.edu

870374IJOXXX10.1177/0306624X19870374International Journal of Offender Therapy and Comparative CriminologyKroner et al.

research-article2019

Kroner et al. 1075

Keywords

risk assessment, LSI-R, categorical risk, risk communication

Deriving meaning from an assessment score or category is at the core of conducting

assessments. With forensic and criminal justice assessments, the assigning of meaning

to a risk assessment score or category has multiple benefits. First, clients can be placed

in groups and then differentiated from other groups of clients (i.e., lower vs. higher

categories of severe problems). Second, having meaningful categories allows for the-

ory testing of crime-related etiologies (Krauss, Sales, Becker, & Figueredo, 2000).

Third, meaningful categories can guide practice. A specific category can suggest opti-

mal levels of supervision or interventions. The purpose of this article is to introduce a

general and principle-based framework for assigning client risk levels that are inde-

pendent of a specific risk instrument’s categories. In this study, the utility of the

Council of State Governments (CSG) Justice Center Five-Level system is demon-

strated through a low-risk data set having greater frequencies in the low risk levels and

greater concordance between observed and predicted recidivism at each level of risk.

In an applied risk assessment setting, multiple instruments can be available (i.e., one

for parole, one for probation), or multiple instruments required (i.e., one for violence

risk, one for general risk), or multiple instruments administered to ensure that no risk

areas are unattended. Each instrument having instrument-based categories gives rise to

a problem. The interpretation of one instrument’s results can differ from other instru-

ments that are applied to the same client. Accommodating these different risk category

interpretations is a challenging issue, especially under cross-examination. As noted by

Heilbrun, Marczyk, DeMatteo, and Mack-Allen (2007), different interpretations may

even affect the perceived accuracy of the information, stating, “Consistency across

sources makes it more likely that the agreed-upon information is accurate” (p. 53).

When multiple risk assessment instruments are administered to the same client, the

risk category concordance among the instruments can be poor. In a study using five

sexual risk assessment instruments, the top 25 percentile for an instrument was used to

identify high-risk cases (Barbaree, Langton, & Peacock, 2006). Similarly, the bottom

25 percentile rank on an instrument was used to identify low-risk cases. Only 3% of

the sample was placed in the high-risk category by all five instruments, and only 4%

of the sample was placed in the low-risk category by all five instruments. Even between

two instruments (Static-2002R, Hanson & Thornton, 2003; Static-99R, Hanson &

Thornton, 2003) with similar developmental strategies and items, the percent agree-

ment for the risk categories was 62.9% (Jung, Pham, & Ennis, 2013). Risk categories

were derived using the same percentile rank method as with the Barbaree study

(Barbaree et al., 2006), except that the boundary points were the 38th percentile for

low/moderate and the 91st percentile for moderate/high.

This lack of concordance among instruments can have implications for accurate

estimates of reoffending. In an applied setting, the accurate prediction of violence may

be compromised when a practitioner uses two risk instruments that lack concordance.

To illustrate, one study used a sample of mostly violent offenders to compare four