Revisited: Effective use of the large body of research on the effectiveness of programs for juvenile offenders and the failure of the model programs approach

• Date: 01 November 2020
• DOI: http://doi.org/10.1111/1745-9133.12530
• Published date: 01 November 2020
• Author: Mark W. Lipsey

DOI: 10.1111/1745-9133 .12530

DEBATE

EVIDENCE-BASED APPROACHES TO IMPROVING

JUVENILE JUSTICE PROGRAMMING

Revisited: Effective use of the large body of

research on the effectiveness of programs for

juvenile offenders and the failure of the model

programs approach

Mark W. Lipsey

Human & Organizational Development,

Vanderbilt University, Nashville,

Tennessee

Correspondence

MarkW. Lipsey,Department of Human

andOrganizational Development, Vander-

biltUniversity, 230 Appleton Place, PMB

181,Nashville, TN 37203-5721.

Email:mark.lipsey@vanderbilt.edu

Abstract

Elliott et al.’s essay is part of what is now a series of

papers in Criminology and Public Policy on contrasting

ways that the evidence base on effective interventions

with juvenile offenders can be used to improve juve-

nile justice programming. Their paper has two themes—

(a) a defense of the model programs approach as pro-

moted by the Blueprints for Healthy YouthDevelopment

evidence-based program registry, and (b) an attack on

the Standardized Program Evaluation Protocol scheme

for assessing programs for juvenile offenders and the

meta-analysis on which it is based. I address both

those themes in this commentary in the context of the

broader cross-disciplinary conversation about ways to

apply research evidence to improve the effectiveness of

the programs used in actual practice.

KEYWORDS

Evaluation, evidence-based program registries, evidence-based

programs, juvenile delinquency, juvenile justice system, juvenile

offenders, juvenile recidivism, meta-analysis, model programs

The Elliott, Buckley, Gottfredson,Hawkins, and Tolan (2020) essay is part of what is now a series

of papers in Criminology and Public Policy on contrasting ways that the evidence base on effec-

tive interventions with juvenile offenders can be used to improve juvenile justice programming.

The first paper (Baglivio et al., 2018) reported research showing that higher scores on a quality of

Criminology & Public Policy. 2020;19:1329–1345. © 2020 American Society of Criminology 1329wileyonlinelibrary.com/journal/capp

1330 LIPSEY

service delivery measure for interventions with juvenile offenders in residential placements were

associated with lower recidivism. That quality of service delivery measure is part of a scheme for

assessing the likelihood that a local juvenile justice program will be effective for reducing recidi-

vism that is based on a large meta-analysis my colleagues and I have conducted. In a commentary

on the Baglivio et al. paper (Lipsey, 2018), I contrasted the way that assessment scheme (called

the Standardized Program Evaluation Protocol [SPEP]) uses evidence on program effectiveness

with the approach represented in the model programs listings in the Office of Juvenile Justice

and Delinquency Prevention Model Programs Guide, CrimeSolutions, and Blueprints for Healthy

Youth Development. In particular, I noted that the model programs approach had failed to have

much actual impact on programming in juvenile justice systems due to limited uptake. I also

argued that there were good reasons to question whether the supporting research for many of the

model programs in those registries was adequate to ensure that the programs would be effective

when scaled up and used in routine practice.

The authors of Elliott et al. include the Blueprints founder, the current Blueprints director,

and several members of the Blueprints advisory board. Their paper has two themes—a defense

of the model programs approach as promoted by Blueprints, and an attack on the SPEP and the

meta-analysis on which it is based. I will address both those themes, but first it is instructive to

recognize that the issues raised in this dialogue are part of a broader cross-disciplinary conversa-

tion about ways to apply research evidence to improve the effectiveness of the programs used in

actual practice.

1 THE EVIDENCE-BASED PRACTICE MOVEMENT

The evidence-based practice (EBP) movement has revolutionized medicine and has increasing

currency in such nonmedical areas as education, mental health, social work, and criminal jus-

tice. In medicine, important developments for evidence-based treatment include recognition of

the limitations of a single study and the need for a sufficient number and scope of clinical trials

before drawing conclusions about effective treatment (Ioannidis, 2008). The associated challenge

of integrating evidence across multiple trials has propelled meta-analysis forward as a systematic

way to accomplish that integration. An especially relevant development in the present context

has been increased recognition of a category of treatments referred to as “complexinterventions.”

These are interventions with multiple constituent elements, and possibly other features that add

to their complexity,in contrast to relatively less complex pharmaceutical or surgical interventions

(Minary et al., 2019; Moore, Evans, Hawkins, Littlecott, & Turley, 2017). On this point the chal-

lenges for establishing evidence-based interventions in medicine and for social–psychological–

behavioral interventions (behavioralinterventions for short) of the sort found in education, men-

tal health, criminal justice, and so on are in close alignment. Virtually all of the behavioral inter-

ventions in these latter areas are complex interventions.

One challenge for research addressing the “what works” question for complex interventions

is defining the “what” so as to adequately identify the intervention being evaluated and guide

replication if it proves effective. The most typical approach has been to define the whole treat-

ment package with all its complexity as a single distinct treatment protocol that is then described

in an accompanying manual. To help practitioners identify and select appropriate manualized

evidence-based programs, various curatedregistries have been developed, such as the U.S. Depart-

ment of Education’s What Works Clearinghouse, the EBPs Resource Center sponsored by the

Department of Health and Human Services’ Substance Abuse and Mental Health Services Admin-

istration, and those in criminal justice mentioned earlier.