NOT EVEN OUR OWN FACTS: CRIMINOLOGY IN THE ERA OF BIG DATA

• Date: 01 August 2018
• Author: JAMES LYNCH
• DOI: http://doi.org/10.1111/1745-9125.12182
• Published date: 01 August 2018

2017 PRESIDENTIAL ADDRESS TO THE AMERICAN

SOCIETY OF CRIMINOLOGY

NOT EVEN OUR OWN FACTS: CRIMINOLOGY

IN THE ERA OF BIG DATA∗

JAMES LYNCH

Department of Criminology and Criminal Justice, University of

Maryland—College Park

KEYWORDS: Presidential Address, big data, data quality, data access

Criminology is an applied discipline where the findings from the data collected and

analyses conducted inform debates about policy and practice. For this to happen, a

discipline must have an agreed-upon set of facts to define the problem and suggest

solutions. We can debate the soundness of fundamental data series, but these debates

must take place within the confines of scientific inquiry and all data must be subjected

to the same scrutiny. Data sources must comprise agreed-upon standards for collection

and be accessible for replication. The increasing use of “big data” has frayed this agree-

ment about quality and accessibility and has made it more difficult for criminology to

have its own facts. In this presentation, I define the term “big data” and argue it will be

difficult for big data to replace traditional data sources and to live up to their potential

for knowledge building. Finally, I suggest a few things that the discipline might do to

address these problems of access and quality.

In 1991, Al Biderman and I wrote a book about the divergence of the Uniform Crime

Reports (UCR) and National Crime Survey (NCS). Wes Skogan (1992) wrote a review.

Not surprisingly, it was a thoughtful and thorough review. He concluded with a statement

to the effect that it was too bad that these guys did not invest all their energy and skill on a

substantive issue in crime and justice instead of obsessing over data collection. Well, Wes,

I am sad to say that not much has changed in the 26 years since except that my preoc-

cupation with data collection has gotten worse and even more removed from substance.

My talk today will not only address data collection but also strategies to collect data. As I

said, Wes, things have only gotten worse.

Criminology and criminal justice are applied disciplines where we hope the findings

from the data we collect and analyses we conduct will inform the public debate about

justice policy and practice. For this to happen, certain constraints are placed on our data,

how it is collected, and how it is analyzed. Daniel Patrick Moynihan described these

constraints succinctly when he observed, “You are entitled to your own opinion, but

∗This address was delivered at the 2017 annual meeting of the American Society of Criminology in

Philadelphia, PA, on November 17, 2017. It has been adapted here for publication.

Direct correspondence to James Lynch, 2220 Samuel J. LeFrak Hall, 7251 Preinkert Drive, College

Park, MD 20742 (e-mail: jlynch14@umd.edu).

C2018 American Society of Criminology doi: 10.1111/1745-9125.12182

CRIMINOLOGY Volume 56 Number 3 437–454 2018 437

438 LYNCH

you are not entitled to your own facts” (Penny, 2003, p. 1). By this, he meant that for

meaningful policy discussions to transpire, a discipline should have an agreed-upon set

of facts or an information base that can be used to define the dimensions and nature of

the problem and to suggest strategies for addressing that problem (Mears, 2010). We

must have data that we can agree on and turn to for this purpose. We cannot simply

make things up or use an unexplored and undocumented source of data to establish

“alternative facts” (Conway, 2017).

The utility of a discipline having an agreed-upon set of facts was apparent in the debate

about the economy just prior to the 2012 presidential election. At that time, Jack Welch,

the former CEO of General Electric, cast aspersions on the validity of unemployment

statistics to claim the Obama administration’s record on employment was poor (Isidore,

2012). The response to his challenge was universal. Economists and business leaders of

every stripe defended the independence and quality of the facts emerging from the Bu-

reau of Labor Statistics (BLS; Bemis, 2012). This example clearly illustrates that if Jack

Welch could have his own facts, the discipline could not have its own and no progress in

policy making could occur.

Having agreed-upon facts does not mean that there cannot be debates about the

methodological soundness of the fundamental data series, but these debates must take

place within the confines of scientific inquiry and all data must be able to be subject to

the same scrutiny.1Alternative data sources that cannot meet these criteria should not

figure into the debate. Data sources must employ agreed-upon standards for collection

and be accessible to allow for replication. In this presentation, I will argue that the ap-

pearance of “big data” has frayed this agreement about quality at the same time that it

makes some data increasingly available. Both of these developments have made it more

difficult for criminology and criminal justice to have our own facts.

In this presentation, I will define what I mean by the term “big data.” I will argue that

it will be difficult for big data to replace traditional sources of data in criminology and

criminal justice and to live up to the traditional sources’ potential for knowledge building.

I base my argument on problems with both gaining access to big data and establishing the

quality of the data that we do access. Finally, I will suggest a few things that the discipline

might do to address these problems of access and quality.

DEFINING BIG DATA

The term “big data” has a wide variety of definitions, many of which are contradictory.

One definition is “any data that is too big to fit on your lap top” (Foster et al., 2017, p. 3).

This assessment will depend, I presume, on the capabilities of your laptop, but it gives you

a general idea. Another defines “big data” as the rich, complicated set of characteristics

and techniques and ethical issues associated with data (Japec et al., 2015). I must confess

that I am not sure what this includes and what it leaves out. Within this range of specific

and general definitions of “big data,” some commonality exists. Each implies that the

data are so voluminous we need to employ data reduction techniques just to begin to

1. There has been any number of debates about the quality of crime data, for example, when police

administrative records were challenged and victimization surveys were introduced (Biderman and

Reiss, 1967). Extensive methodological work was done on these surveys (Biderman et al., 1986;

Penick and Owens, 1976), and comparisons were made between the two modalities to establish

their validity and fitness for use (Biderman and Lynch, 1991).