Why Getting Inside the “Black Box” Is Important

• Date: 01 March 2017
• Published date: 01 March 2017
• DOI: 10.1177/1098611116664336
• Author: Joshua C. Hinkle,Christine Famega,David Weisburd
• Subject Matter: Articles

Article

Why Getting Inside

the “Black Box”

Is Important:

Examining Treatment

Implementation and

Outputs in Policing

Experiments

Christine Famega

1

, Joshua C. Hinkle

2

,

and David Weisburd

3,4

Abstract

We propose that the causes of the effectiveness (or ineffectiveness) of police inter-

ventions can be better understood with an increased focus on the measurement of

treatment implementation and outputs, as opposed to the more common “black

box”conceptualizations of police interventions and outcome-only evaluations used

in most experimental studies. We present findings from a randomized, experimental

evaluation of broken windows policing at hotspots in three California cities. Our

analysis suggests that variation in the treatment delivered to target street segments

within and between cities limited the ability of the study to detect potential treat-

ment impacts and was due in part to the failure of the police agencies to take

ownership of the science of the intervention.

Keywords

broken windows, experimental methods, evaluation research, hot spots, policing

1

California State University – San Bernardino, CA, USA

2

Georgia State University, Atlanta, GA, USA

3

George Mason University, Fairfax, VA, USA

4

The Hebrew University, Jerusalem, Israel

Corresponding Author:

Christine Famega, Department of Criminal Justice, California State University – San Bernardino, 5500

University Pkwy, San Bernardino, CA 92407, USA.

Email: cfamega@csusb.edu

Police Quarterly

2017, Vol. 20(1) 106–132

!The Author(s) 2016

Reprints and permissions:

sagepub.com/journalsPermissions.nav

DOI: 10.1177/1098611116664336

journals.sagepub.com/home/pqx

There is now ample evidence that increased police patrol directed at hotspots is

more effective in reducing crime and disorder than random, undirected (routine

preventive) patrol (Braga, Papachristos, & Hureau, 2014; Weisburd & Eck,

2004). However, two issues remain less clear. First, we know little about the

causes of effectiveness, or conversely ineffectiveness. Aside from increased

police presence (dosage and frequency) or the use of problem-solving

approaches, which specific police actions are most effective in reducing crime

and disorder and under what conditions? Second, what actions should police

leaders take to “create and maintain the causes” of successful strategies

(Sherman et al., 2014, p. 96)?

In this article, we address two obstacles that have impeded the progress being

made in understanding both of these issues: (a) the research and evaluation

methodologies being used to study police interventions and (b) the lack of

“ownership of police science” in police agencies (Neyroud & Weisburd, 2014;

Weisburd & Neyroud, 2011). We suggest that through addressing the first obsta-

cle, police academics and researchers can better ascertain the extent to which the

second may have impacted the effectiveness of an evaluated strategy. In turn, an

increased focus on eliminating these obstacles is necessary to increase our knowl-

edge about both how police strategies are effective or ineffective, and how police

can implement interventions in ways that they are likely to have impact.

We begin by briefly discussing a common shortcoming of police evaluation

research. We then present findings from a randomized, experimental evaluation

of broken windows policing at hotspots (Weisburd, Hinkle, Famega, & Ready,

2011) to illustrate how the research methodology for evaluations of police inter-

ventions is important in assessing whether the police really took ownership of an

intervention.

Getting Inside the “Black Box”in Policing Experiments

In 1998, Lawrence Sherman advocated for evidence-based policing—that police

practices should be based on scientific evidence about what works best. He

proposed that more randomized experiments examining the outcomes of police

work (e.g., crime rates, repeat victimization, etc.) were needed, as well as

ongoing outcomes research about the results achieved by applying (or ignoring)

basic research in practice (Sherman, 1998). Today, randomized, experimental

designs have generally become accepted as the best method for generating

valid evidence about the effectiveness of criminal justice programs, practices,

and strategies (Weisburd & Hinkle, 2012). The statistical benefits of randomizing

subjects or places into treatment and control groups eliminates the possibility of

confounding variables, as everything affecting the outcomes of interest aside

from the intervention should vary randomly across the two groups (Boruch,

McSweeny, & Soderstom, 1978; Campbell & Stanley, 1963; Riecken et al.,

1974; Weisburd, 2003).

Famega et al. 107