Examining the Validity of Traffic Stop Data: A Mixed-Methods Analysis of Police Officer Compliance

• DOI: 10.1177/1098611120933644
• Date: 01 March 2021
• Published date: 01 March 2021
• Subject Matter: Articles

Article
Police Quarterly
Examining the Validity
2021, Vol. 24(1) 3–30
! The Author(s) 2020
of Traffic Stop Data:
Article reuse guidelines:
sagepub.com/journals-permissions
DOI: 10.1177/1098611120933644
A Mixed-Methods
journals.sagepub.com/home/pqx
Analysis of Police
Officer Compliance
Joshua Chanin1
and Megan Welsh1
Abstract
Police departments rely on administrative rules to set organizational priorities and
establish systems of accountability. To that end, several departments require officers
to submit data describing every traffic stop they conduct as a way of tracking officer
activity and identifying any race-based disparities. This paper draws on an analysis of
San Diego Police Department traffic stop records, as well as officer survey and
interview data, to examine the validity of the traffic stop data gathered and the
compliance-related motivations of officers. Findings indicate a 19 percent error
rate in stop data submitted between 2014 and 2015, amidst evidence of substantial
underreporting. Qualitative data suggest that officers see the policy as redundant and
an infringement on more pressing aspects of their job. They doubt the ability of
external stakeholders to interpret the data objectively and report a loss of morale,
largely attributed to the perception that their actions are inaccurately racialized.
Keywords
racial profiling, data validity, police compliance, police accountability, officer
motivation
1School of Public Affairs, San Diego State University
Corresponding Author:
Josh Chanin, School of Public Affairs, San Diego State University, 5500 Campanile Drive, San Diego, CA
92182, United States.
Email: jchanin@sdsu.edu

---

4
Police Quarterly 24(1)
The operation of public organizations is predicated on the order and predict-
ability that rules provide (Bozeman & Feeney, 2011; Weber,1947/2009). Rules
define individual-level professional duties and boundaries, while setting in place
the structure for organizational and individual decision-making (Burns &
Stalker, 1961; Weber, 1947/2009). They also serve as the foundation for bureau-
cratic accountability (Hupe & Hill, 2007).
There is a robust, multidisciplinary literature that examines how and why
bureaucrats respond to rules designed to shape or alter behavior in accordance
with non-mission-based values like transparency and lawfulness (e.g., Brehm &
Gates, 1999; Martin et al., 2013; Tummers et al., 2015). The research presented
here examines rule compliance in the context of police traffic stops. Compliance
can be defined as “a behavioral state in a specific time, situation, and place that
conforms (completely or partially) to behavioral directives, such as those
embodied in laws, social norms, and organizational policies” (Siddiki et al.,
2019, p. 4). We consider the motivations and behavior of San Diego Police
Department (SDPD) officers, mandated to collect data documenting the race
and gender of the drivers they stop, why each stop was initiated, and the legal
outcome of each encounter.
Traffic stop data collection is an important policy area within which to exam-
ine compliance. Well-documented racial disparities in the enforcement of traffic
stops around the country are a significant component of an ongoing discussion
of race and policing in the United States. “Driving while Black” has long been
part of the lexicon, short hand for the assumption that Black drivers are subject
to differential treatment by police (Harris, 1999; Lamberth, 1998). The costs of
this disparity are incredibly high: Several high-profile police-involved deaths
have originated with traffic stops involving Black drivers (e.g., LaFraniere &
Smith, 2016), a fact some researchers have highlighted to explain the low levels
of police trust and legitimacy in many minority communities (Tyler & Wakslak,
2004).
Traffic stops are highly discretionary; individual officers maintain substantial
authority over who is stopped and have wide legal berth in justifying their
decisions (Goldstein, 1960; Walker, 1993). As a result, both internal and exter-
nal regulation of the issue is rather difficult. Data collection has become a crit-
ical tool in this regard. In 2015, for example, President Obama’s Task Force on
21st Century Policing urged all law enforcement agencies “to collect, maintain,
and analyze demographic data on all detentions (stops, frisks, searches, sum-
mons, and arrests)” (p. 24). Several jurisdictions, including departments in
states like North Carolina, Missouri, Maryland, and Texas, are subject to state-
wide mandates (National Conference of State Legislators, 2018). Others
have adopted data collection efforts at the behest of local oversight initiatives
(e.g., St. Paul, Minnesota) (St. Paul.gov, n.d.) or voluntarily (e.g., San Diego)
(Baker, 2014).

---

Chanin and Welsh
5
Despite the proliferation of data collection efforts, little is known about the
behavior of the officers subjected to such mandates. In other words, stakehold-
ers – lawmakers, police executives, police oversight organizations, and members
of the media and the public – are underinformed about the quality of the records
being produced, with little insight into the likelihood that officers are recording
stops either incompletely or inaccurately, or simply choosing not to document
stops at all.
In an effort to help fill this gap, this study draws on an analysis of over
250,000 traffic stop records, in addition to survey and interview data gathered
from SDPD officers, to address three questions:
1. How do officers perceive of the requirement that they document and report
on the details of the traffic stops they conduct?
2. To what extent do officers comply with these requirements?
3. How do officers explain their compliance/non-compliance decision-making?
Our audit of SDPD data records shows substantial evidence of officer non-
compliance, in the form of unrecorded traffic stops, and negligence, as evidenced
by error-ridden stop records. These results are consistent with previous research.
Our primary contribution to the literature on racial profiling and bureaucratic
behavior are qualitative insights into the motivations of officers subject to data
collection mandates. The analysis suggests that officers see the Department’s
traffic stop mandate as redundant, duplicative of existing data collection efforts,
and an impediment to meaningful police work. Officers also described a lack of
trust in agency leadership and external stakeholders, including members of the
media and the public, largely attributed to a perceived ignorance of the traffic
stop process and a biased view of law enforcement.
Traffic Stop Data Collection
Information drives the practice and management of law enforcement in the
United States. Police leadership is increasingly reliant on data to set enforce-
ment priorities and allocate personnel and other organizational resources
(Ferguson, 2017; Goel et al., 2017). Data is also crucial to the oversight of
police practices. The performance of several legal and administrative account-
ability mechanisms, including civil and criminal court systems, state and federal
agencies, and civilian review boards, is predicated on this same administrative
information.
There is also a long history of research that draws on these data to gain
insight into police behavior. A deep literature exists on the enforcement of traffic
stops, with analysis of administrative records detailing the extent to which driver
race shapes the likelihood of being stopped, searched, and arrested (Pierson
et al., 2019). This work has been used as evidence in high-stakes civil litigation

---

6
Police Quarterly 24(1)
(Gelman et al., 2007), driven institutional reform efforts (U.S. DOJ, 2017), and
motivated recent legislation.
Designed to eliminate the use of race, gender, and other physical character-
istics from police enforcement decision-making, The Racial and Identity
Profiling Act (RIPA) of 2015 requires California law enforcement agencies to
collect and submit to the State’s Attorney General detailed information about
all police-initiated traffic, pedestrian, and bicycle stops, including the race and
gender of the person stopped, the reason for the stop, and the actions taken by
the officer during the stop (Sec. 3, Art. 999.226). The law also requires affected
departments to submit to the California Attorney General all data collected
(Sec. 3, Art. 999.227). Officers in at least twenty other states are required to
record and report on similar demographic and incident-related data for every
traffic stop conducted in the jurisdiction (National Conference of State
Legislators, 2018).
The central premise of such oversight efforts – that the data will deter unlaw-
ful behavior and help state regulators hold such action to account – is based on
the assumption the data collected are a valid indication of what officers do on a
daily basis. The proliferation of legislation like RIPA has no doubt increased
understanding of existing racial disparities in policing. The overwhelming
number of peer-reviewed studies using administrative data to examine traffic
enforcement is evidence of this fact.
And yet, the large majority of studies on traffic stop enforcement omit dis-
cussion of data validity. Of the over 100 papers and reports we reviewed, just 19
addressed data quality in any meaningful way. As is shown in Table 1, 17 studies
evaluated the extent to which submitted forms either included incorrect infor-
mation or lacked requisite data,1 while eight addressed the rate at which officers
failed to record traffic stops. Beyond the fact that data...