The Development, Validity, and Reliability of the Minnesota Screening Tool Assessing Recidivism Risk (MnSTARR)

• Author: Grant Duwe
• DOI: 10.1177/0887403413478821
• Published date: 01 September 2014
• Date: 01 September 2014
• Subject Matter: Articles

Criminal Justice Policy Review

2014, Vol. 25(5) 579 –613

© 2013 SAGE Publications

Reprints and permissions:

sagepub.com/journalsPermissions.nav

DOI: 10.1177/0887403413478821

cjp.sagepub.com

Article

The Development, Validity,

and Reliability of the

Minnesota Screening Tool

Assessing Recidivism Risk

(MnSTARR)

Grant Duwe1

Abstract

This study presents findings on the development, validity, and reliability of the

Minnesota Screening Tool Assessing Recidivism Risk (MnSTARR), a prediction

instrument that assesses risk for four types of recidivism for male prisoners (sexual,

nonsexual violent, nonviolent, and felony) and three types for females prisoners

(nonsexual violent, nonviolent, and felony). Logistic regression modeling was used

to develop the MnSTARR on 11,375 male offenders and 1,100 females offenders

released from Minnesota prisons between 2003 and 2006. Bootstrap resampling

was used to not only refine the selection of predictors, but also to internally validate

the model. The optimism-corrected area under the curve (AUC) values ranged

from .73 to .80 for male offenders and .73 to .81 for female offenders. In addition

to showing a relatively high degree of calibration between MnSTARR values and

observed recidivism, the results showed the instrument can, for the most part, be

scored consistently across raters. Intraclass correlation coefficient values ranged

from .79 to .86 for male offenders and .81 to .96 for female offenders.

Keywords

recidivism, risk assessment, prisoner, calibration, predictive discrimination

1Minnesota Department of Corrections, St. Paul, MN, USA

Corresponding Author:

Grant Duwe, Research Director, Minnesota Department of Corrections, 1450 Energy Park Drive, Suite

200, St. Paul, MN 55108-5219, USA.

Email: grant.duwe@state.mn.us

478821CJP25510.1177/0887403413478821Criminal Justice Policy ReviewDuwe

research-article2013

580 Criminal Justice Policy Review 25(5)

When offenders enter prison, they are often undereducated, have little or no prior work

history, lack vocational skills, have lengthy histories of substance abuse, and are more

likely to suffer from mental illness (Petersilia, 2003). Although most state and federal

prisons offer programming opportunities within the institution, there is seldom enough

programming available for all of the offenders who need it. Indeed, research suggests

that many prisoners do not participate in programming while incarcerated (Lynch &

Sabol, 2001). Given the apparent paucity of resources available, a key question is

“How should correctional systems allocate these resources most effectively?”

The burgeoning “what works” literature, which buttresses the evidence-based prac-

tices (EBP) movement, suggests that resources are used most effectively when they

target the risk, needs, and responsivity of offenders (Cullen & Gendreau, 2000). The

risk principle holds, for example, that treatment interventions should be used primarily

with higher risk offenders (Lowenkamp, Latessa, & Holsinger, 2006). Interventions

should also target the known dynamic predictors of recidivism, which include crimi-

nogenic needs (e.g., attitudes supportive of an antisocial lifestyle, substance abuse,

companions, etc.), personal distress (e.g., anxiety, depression, schizophrenia, etc.),

and social achievement (e.g., marital status, level of education, employment, etc.;

Gendreau, Little, & Goggin, 1996). In contrast to static predictors (e.g., gender, race,

criminal history, etc.), which cannot change, targeting the criminogenic needs of

offenders is more likely to lower recidivism because these are dynamic factors in

which changes can be made. Whereas risk and need focus on who should be priori-

tized for treatment and what areas should be targeted for programing, responsivity

concentrates on how the intervention should be delivered (Dowden, Antonowicz, &

Andrews, 2003).

Because the risk principle indicates that interventions should focus on the offenders

with the greatest risk of reoffending, the development and use of risk assessment tools

is considered part and parcel of EBP. Although the emphasis placed on EBP within

corrections is relatively recent, efforts to assess offender recidivism risk are not.

During the first half of the 20th century, the “first generation” of offender risk assess-

ment was, according to Bonta and Andrews (2007), based largely on subjective, pro-

fessional judgments made by correctional staff.

Although the use of more objective, actuarial methods for assessing offender risk

had been around since at least the late 1920s (Burgess, 1928), it was not until the 1970s

that clinical judgment began to give way to the development of “second-generation”

risk assessment instruments (Bonta & Andrews, 2007). Created to provide standard-

ized predictions of risk in which items were selected based on statistical analyses,

second-generation risk assessment instruments were found to consistently outperform

clinical judgment in predicting recidivism (Brennan, Dieterich, & Ehret, 2009).

Examples of second generation instruments include the Salient Factor Score (SFS)

developed by Hoffman and Beck (1974) and the Statistical Information on Recidivism

(SIR) created by Nuffield (1982).

Despite the improvement in predictive accuracy, second-generation risk assess-

ment tools were later criticized for being comprised mainly of static risk factors, such

Duwe 581

as criminal history, which cannot change. Moreover, second generation instruments

included items regardless of whether they were consistent with existing theory

(Brennan et al., 2009). Beginning in the 1980s, third-generation risk assessment instru-

ments responded to these concerns by incorporating dynamic risk factors, selecting

items that are consistent with theory, and emphasizing the importance of risk assess-

ment for case planning purposes (Bonta & Andrews, 2007). The Level of Service

Inventory-Revised (LSI-R) is perhaps the best known example of a third-generation

risk assessment. Created by Andrews and Bonta (1995), the LSI-R is a 54-item instru-

ment that has been extensively validated and widely used within corrections.

But third-generation risk assessment instruments have also been criticized for

assuming risk factors are the same for male and female offenders, adopting a relatively

narrow theoretical focus, failing to adequately assess protective factors, and conflating

risk with need (Baird, 2009; Brennan et al., 2009). More recently, fourth-generation

risk assessment instruments have attempted to address these limitations by including a

broader selection of explanatory theories, a wider range of risk factors, and the

strengths or resiliency perspective. Designed to follow the offender from intake to case

closure, fourth-generation risk assessments have also been characterized as emphasiz-

ing the use of more advanced statistical modeling, the administration of assessments

on multiple occasions, and the integration of risk and need within the agency manage-

ment information system (Brennan et al., 2009). Examples of fourth-generation instru-

ments include the Level of Service/Case Management Inventory (LS/CMI; Andrews,

Bonta, & Wormith, 2004), the Ohio Risk Assessment System (ORAS; Latessa, Smith,

Lemke, Makarios, & Lowenkamp, 2009), the Correctional Assessment and Intervention

System (CAIS; National Council on Crime and Delinquency, 2006), and the

Correctional Offender Management Profile for Alternative Sanctions (COMPAS;

Brennan & Oliver, 2000).

Present Study

This study presents results from the development of the Minnesota Screening Tool

Assessing Recidivism Risk (MnSTARR), a risk assessment instrument designed to

predict several types of recidivism for male and female prisoners. The sample for this

study consisted of 11,375 male offenders and 1,100 female offenders released from

Minnesota prisons between 2003 and 2006. Using a 4-year follow-up period to track

recidivism, multiple logistic regression was used to develop the MnSTARR. Bootstrap

resampling was used to internally validate the MnSTARR, and an interrater reliability

assessment was conducted to estimate the consistency with which the instrument can

be scored.

In the ensuing section, this study delineates the assumptions underlying the devel-

opment of the MnSTARR, discusses its similarities with other fourth-generation

instruments, and identifies the contributions it makes to the risk assessment literature.

Next, this study describes the sample, the recidivism measures used, and the methods

used to develop and internally validate the MnSTARR. Following a presentation of