Does Treatment Quality Matter?

• Author: Kevin T. Wolff,Gabrielle Chapman,Katherine Gomez,Mark A. Greenwald,Michael T. Baglivio,Katherine Jackowski
• Date: 01 February 2018
• DOI: http://doi.org/10.1111/1745-9133.12338
• Published date: 01 February 2018

RESEARCH ARTICLE

TREATMENT QUALITY AND JUVENILE

RECIDIVISM

Does Treatment Quality Matter?

Multilevel Examination of the Effects of Intervention Quality

on Recidivism of Adolescents Completing Long-Term

Juvenile Justice Residential Placement

Michael T.Baglivio

TrueCore Behavioral Solutions, LLC

Kevin T. Wol

John Jay College of Criminal Justice

Katherine Jackowski

TrueCore Behavioral Solutions, LLC

Gabrielle Chapman

Vanderbilt University

Mark A. Greenwald

Katherine Gomez

Florida Department of Juvenile Justice

Research Summary

The Florida Department of Juvenile Justice has implemented a process to evaluate

the treatment quality of interventions provided statewide in all long-term residential

programs. In the current study, we examine the predictive validity of this treatment

quality component in the prediction of recidivism of youth (N=2,397) who com-

pleted juvenile justice residential programs from July 1, 2013 to June 30, 2014. We

use hierarchical linear modeling to assess the effects of treatment quality (as youth

are nested within programs), controlling for demographic and criminal history fac-

tors. The results indicate that higher average treatment quality scores of interventions

received within a residential program decreased the likelihood of subsequent arrest,

Direct correspondence to Michael T. Baglivio, TrueCore Behavioral Solutions, LLC, 6302 Benjamin Rd., Suite

400, Tampa, FL 323634 (e-mail: michael.baglivio@truecorebehavioral.com).

DOI:10.1111/1745-9133.12338 C2018 American Society of Criminology 147

Criminology & Public Policy rVolume 17 rIssue 1

Research Article Treatment Quality and Juvenile Recidivism

conviction, and reincarceration, whereas the highest treatment quality score of any

specific intervention provided within the program decreased the odds of reincarceration

only.

Policy Implications

In this study, we confirm the importance of fidelity and implementation quality in

the provision of crime prevention treatment interventions to serious, deep-end juvenile

offenders. Additionally, our results shed light on the ability of a juvenile justice agency

to measure treatment quality in a substantively meaningful way with the use of limited

additional resources. The services provided to keep communities safe, prevent future

offending, and rehabilitate juvenile offenders must be held accountable for producing

such outcomes, and one method of measuring such compliance is to evaluate the quality

of the interventions with respect to staff training, fidelity adherence, evaluation, and

corrective action processes.

Keywords

Treatment integrity, fidelity, program evaluation, juvenile recidivism

In 2013, one of the highest populated U.S. states, with one of the largest juvenile

justice systems in the nation, embarked on a long-term statewide initiative to monitor

treatment quality and understand its effects on youth outcomes. In this study, we de-

scribe the Florida Department of Juvenile Justice’s efforts, with extensive attention to how

treatment quality was measured, the monitoring process, the extent to which treatment

quality varies across interventions and residential facilities, and whether treatment quality

affects juvenile outcomes. The importance of proper implementation and continuous fi-

delity adherence, even for the most prolific, well-researched “evidence-based” interventions,

has been elucidated in prior work (Barnoski, 2004; Fixsen, Blas´

e, Timbers, and Wolff,

2001; Gottfredson, Gottfredson, and Czeh, 2000; Mihalic, 2004). The results of this line

of scholarship have informed practitioners that interventions selected must be carefully and

fully implemented, as well as continually maintain fidelity, in the hope of achieving results

similar to those garnered in quality empirical evaluations. Although research guides practice

demonstrating the importance of implementation quality, the monitoring and evaluation

of treatment quality on a large statewide scale overseen and administered by juvenile justice

system practitioners has been heretofore unaddressed. In this study, we describe and assess

the first attempt at statewide treatment quality monitoring and evaluation.

A dearth in the research literature remains as to whether it is possible to use a stan-

dardized metric of treatment quality to assess the individual interventions facilitated within

juvenile justice placements, as well as to whether this can be accomplished by justice sys-

tem professionals. The purpose of this study was to examine the relevance to subsequent

recidivism of a statewide evaluation of treatment quality across interventions facilitated in

148 Criminology & Public Policy

Baglivio et al.

long-term juvenile justice residential programs in Florida. As such, we examined the sub-

sequent reoffending of juveniles released from residential placement, where interventions

delivered within each facility were evaluated for treatment quality through use of a rubric

developed and administered by the Florida Department of Juvenile Justice (FDJJ). By tak-

ing a treatment quality measure from each intervention delivered within each of 56 FDJJ

residential facilities, we evaluated whether youth released from facilities delivering interven-

tions rated as higher quality evidenced lower subsequent recidivism 1 year postrelease when

controlling for initial individual differences in risk.

The policy implications if a standardized measure of treatment quality across residential

programs is related to postrelease success are tremendous. In the event juvenile justice

professionals can effectively assess treatment quality, meaning this rating is relevant to

subsequent outcomes, it will demonstrate that large-scale treatment quality evaluation

by nonresearchers is possible. Additionally, such a process would allow for identification

of specific corrective action steps that can improve quality ratings and, hence, enhance

outcomes. Such a continuous quality improvement feedback loop could drive reform,

thereby reducing reoffending, subsequent justice-system costs, and future victimization

within communities.

Characteristics and Importance of Fidelity and Treatment Quality

Approximately 54,000 juveniles were in residential placement in the United States on any

given day in 2011 (Sickmund, Sladky,Kang, and Puzzanchera, 2015). The Comprehensive

Strategy for Serious, Violent,and Chronic Juvenile Offenders and the notion of progressively

graduated sanctions (Howell, 2009; Wilson and Howell, 1993), encompassed by the “risk

principle” of matching the intensity of services to the risk to reoffend level of the youth,

provides a core model of how cases should be handled within a juvenile justice system.

Reserving the deepest end placements for the highest risk youth who have been unsuccessful

in less intensive placements is not only fiscally responsible, by targeting limited resources

to those most in need of services and who pose the highest risk to society, but has been

demonstrated to optimize overall system effectiveness (Andrews and Bonta, 2003; Andrews

and Dowden, 1999; Andrews, Zinger,Hoge, Bonta, Gendreau, and Cullen, 1990; Baglivio,

Greenwald, and Russell, 2014; Lipsey, 2009; Lowenkamp and Latessa, 2005). As poorly

implemented interventions lead to undermining the entire evidence-based approach, a

central concern in this framework is the notion of whether interventions are implemented

effectively.The question then becomes, and the one this study is designed to address, whether

the quality of the interventions or treatments delivered within juvenile justice residential

programs affects the likelihood of reoffending postcompletion, and to what degree.

Although there has been a proliferation of various registries of evidence-based practices,

such as the Office of Justice Programs’Crime Solutions (crimeSolutions.gov), the Blueprints

for Healthy Youth Developmentlist, and the Substance Abuse and Mental Health Services

Administration’s (SAMHSA) National Registry of Evidence-based Programs and Practices

Volume 17 rIssue 1 149