Developing Nonarbitrary Metrics for Risk Communication

• DOI: 10.1177/0093854816651656
• Published date: 01 December 2016
• Author: Robert J. B. Lehmann,L. Maaike Helmus,David Thornton,R. Karl Hanson
• Date: 01 December 2016
• Subject Matter: Articles

CRIMINAL JUSTICE AND BEHAVIOR, 2016, Vol. 43, No. 12, Decenber 2016, 1661 –1687.

DOI: 10.1177/0093854816651656

© 2016 International Association for Correctional and Forensic Psychology

1661

DEVELOPING NONARBITRARY METRICS FOR

RISK COMMUNICATION

Norms for the Risk Matrix 2000

ROBERT J. B. LEHMANN

Charité–University Medicine Berlin

DAVID THORNTON

Sand Ridge Secure Treatment Center

L. MAAIKE HELMUS

Global Institute of Forensic Research

R. KARL HANSON

Public Safety Canada

Nominal risk categories for actuarial risk assessment information should be grounded in nonarbitrary, evidence-based criteria.

The current study presents numeric indicators for interpreting one such tool, the Risk Matrix 2000, which is widely used to

assess the recidivism risk of sexual offenders. Percentiles, risk ratios, and 5-year recidivism rates are presented based on an

aggregated sample (N = 3,144) from four settings: England and Wales, Scotland, Germany, and Canada. The Risk Matrix

2000 Sex, Violence, and Combined scales showed moderate accuracy in assessing the risk of sexual, non-sexual violent, and

violent recidivism, respectively. Although there were some differences across samples in the distributions of risk categories,

relative increases in recidivism for ascending risk categories were remarkably consistent. Options for presenting percentiles,

risk ratios, and absolute recidivism estimates in applied evaluations are offered, with discussion of the advantages, disadvan-

tages, and limitations of these risk communication metrics.

Keywords: risk assessment; sex offenders; Risk Matrix 2000; risk communication; recidivism

Courts, police, and child protection services are frequently concerned about the risk of

future offending among individuals who are already known to have committed a sexual

offense. A number of actuarial risk tools are available (Otto & Douglas, 2010) and widely

used in forensic psychology (Archer, Buffington-Vollum, Stredny, & Handel, 2006; Neal &

Grisso, 2014; Singh et al., 2014) and corrections (McGrath, Cumming, Burchard, Zeoli, &

Ellerby, 2010).

AUTHORS’ NOTE: The views expressed are those of the authors and not necessarily those of the Wisconsin

Department of Health Services or Public Safety Canada. We would like to thank Don Grubin and Klaus-Peter

Dahle for granting us permission to use their data. Correspondence concerning this article should be addressed

to Robert J. B. Lehmann, Institute of Forensic Psychiatry, Charité–University Medicine Berlin, Oranienburger

Str. 285, 13437, Berlin, Germany; e-mail: r.lehmann@charite.de.

651656CJBXXX10.1177/0093854816651656CRIMINAL JUSTICE AND BEHAVIORLehmann et al. / RISK MATRIX 2000 NORMS

research-article2016

1662 CRIMINAL JUSTICE AND BEHAVIOR

Hanson and Morton-Bourgon (2009) identified nine risk assessment scales specifically

designed for sex offenders that have demonstrated moderate or high predictive accuracy

(d > .50) with no single risk tool showing clear superiority. Consequently, evaluators have

to choose between numerous risk tools. Considerations in choosing risk scales vary, but

may include both practical (e.g., availability of training, ease of use, types of information

frequently available) and empirical considerations (e.g., matching between scale purpose

and referral question, depth of research support, research support for particular types of

offenders or jurisdictions, and normative data). Even though the precise referral question

varies, decision makers are typically concerned about whether certain individuals should be

prioritized for special interventions or conditions (e.g., treatment, surveillance), or whether

the individual’s risk is above or below some threshold (e.g., Is the offender a tolerable risk

for community supervision? For family reintegration?). To answer these questions, actuar-

ial risk tools combine risk factors into a total score (or risk categorization) and provide

empirically derived estimates of recidivism probabilities (Dawes, Faust, & Meehl, 1989).

RISK COMMUNICATION

Although the use of risk scales is widespread (e.g., Blais & Forth, 2014; Neal & Grisso,

2014; Singh et al., 2014), there is less consensus and discussion on how to communicate the

information provided in these scales. Most of the discussions concerning the communication

of psychological test results have focused on norm-referenced measures, where scores can be

interpreted as the position of an individual within a defined group (Crawford, Garthwaite, &

Slick, 2009; Oosterhuis, van der Ark, & Sijtsma, 2016). Such an interpretation, however,

poorly expresses the information contained in criterion-referenced prediction measures,

where the goal of the assessment concerns the likelihood of a significant outcome, such as

recidivism. Consequently, common conventions for interpreting norm-referenced psycho-

logical scales may not be applicable for criterion-referenced scales (Aiken, 1985).

Practitioners and decision makers value nominal risk categories (i.e., low, moderate, high)

more than other forms of risk communication (Evans & Salekin, 2014; Grann & Pallvik,

2002; Heilbrun et al., 2004; Heilbrun, O’Neill, Strohman, Bowman, & Philipson, 2000).

Unfortunately, we (e.g., evaluators, decision makers) often infer substantially different mean-

ings from the same labels (Hilton, Carter, Harris, & Sharpe, 2008; Monahan & Silver, 2003).

These differences in interpretation persist irrespective of experience with risk assessments

(Slovic, Monahan, & MacGregor, 2000) and have been found among the general public as

well (Clarke, Ruffin, Hill, & Beamen, 1992). This lack of consistency is a serious problem

hindering the development of a common understanding of risk information.

To advance interpretation of risk assessment information, we need to have nonarbitrary

metrics to represent the information contained in risk scales (Babchishin & Hanson, 2009;

Blanton & Jaccard, 2006; Hanson, Lloyd, Helmus, & Thornton, 2012). More recent efforts

have been devoted to understanding how these nonarbitrary metrics can inform more con-

sistent and intuitive definitions of risk categories (Hanson, Babchishin, Helmus, Thornton,

& Phenix, 2016; Justice Center, 2016). Specifically, Hanson et al. (2012) suggested three

plausible quantitative anchors for risk communication (i.e., absolute recidivism rates, per-

centile ranks, and risk ratios), with each metric having its own strengths and weaknesses.

Risk communication may be maximized by using multiple types of information; however,

certain referral questions may be best addressed by a single numeric indicator.