A Cautionary Note about the Use of Estimated Homicide Data for Cross-National Research
| Author | Manuel P. Eisner,Stefan Kanis,Steven F. Messner,Wilhelm Heitmeyer |
| DOI | 10.1177/1088767917715670 |
| Date | 01 November 2017 |
| Published date | 01 November 2017 |
| Subject Matter | Research Note |
research-article2017
Research Note
Homicide Studies
2017, Vol. 21(4) 312 –324
A Cautionary Note about the
© 2017 SAGE Publications
Reprints and permissions:
Use of Estimated Homicide
sagepub.com/journalsPermissions.nav
https://doi.org/10.1177/1088767917715670
DOI: 10.1177/1088767917715670
journals.sagepub.com/home/hsx
Data for Cross-National
Research
Stefan Kanis1, Steven F. Messner2,
Manuel P. Eisner3, and Wilhelm Heitmeyer1
Abstract
A major development in criminology in recent years has been the efforts by the
World Health Organization (WHO) to provide reasonably reliable estimates of
homicide rates for a large number of nations. In some instances, these estimates
entail adjustments of the records on homicide from vital statistics or criminal justice
sources submitted by participating nations. These adjustments are designed to deal
with underreporting and detected anomalies. In other instances, the estimates are
generated by regression modeling. The purpose of this research note is to raise
awareness among the community of homicide researchers of the nature of the
WHO homicide estimates and to offer caution about their appropriate use for cross-
national research.
Keywords
homicide estimates, cross-national, World Health Organization, measurement,
homicide statistics
Introduction
A long-standing problem confronting researchers interested in analyzing cross-
national variation in homicide rates has been the limited availability of data. Many
nations lack the requisite administrative agencies to compile reliable homicide
1Bielefeld University, Germany
2University at Albany, State University of New York, USA
3University of Cambridge, UK
Corresponding Author:
Stefan Kanis, Institute for Interdisciplinary Research on Conflict and Violence, Bielefeld University,
Universitätsstraße 25, Bielefeld 33615, Germany.
Email: stefan.kanis@uni-bielefeld.de
Kanis et al.
313
statistics, and as a result, researchers have had little choice but to restrict their analyses
to those nations for which data are available rather than to study the theoretical popula-
tions of interest. Moreover, as LaFree (1999) explained almost 20 years ago, reliance
on such “availability samples” (p. 135) rather than genuine probability samples results
in two interrelated problems (see also Koeppel, Rhineberger-Dunn, & Mack, 2015;
Nivette, 2011). Such samples can by no means be regarded as representative. As might
be expected, homicide data have been more readily available for the more developed
nations, nations with well-established statistical recording agencies. The numbers of
nations included in the “availability samples” employed in comparative homicide
studies have also been rather small relative to the total population of nations. Small
samples can be problematic because analyses based upon them are highly susceptible
to the impact of outliers (LaFree, 1999, p. 135).
The primary data source that has in practice determined the availability samples
used in the cross-national homicide research has changed over time. The early studies
typically relied on data from the International Criminal Police Organization
(INTERPOL), but subsequently, the publications of the World Health Organization
(WHO) emerged as the generally preferred source. Specifically, researchers have
relied mainly on the WHO Mortality Database. This database derives from the health/
vital statistics reports from participating nations on the specific causes of death, includ-
ing homicide. Homicide is defined for this purpose as “the killing of a person by
another with intent to cause death or serious injury” (WHO, Indicator and Measurement
Registry, 2017). These WHO homicide data are now generally regarded as being of
higher quality for comparative research than are other data sources (Koeppel,
Rhineberger-Dunn, & Mack, 2015, p. 51; LaFree, 1999, p. 133; Levchak, 2016, p. 8;
Messner, Pearson-Nelson, Raffalovich, & Miner, 2011, p. 67; Messner, Raffalovich, &
Shrock, 2002, p. 383; Messner, Raffalovich, & Sutton, 2010, p. 511). However,
depending on the specific year, death registration data that are taken directly from the
public health records can only be provided for approximately 70 countries. This is
only slightly over a third of the total number of nations (U.S. Department of State,
Bureau of Intelligence and Research, 2017).
Homicide researchers have accordingly been eager to locate new homicide data
sources for their analyses, especially data sources that expand and diversify the cover-
age of nations. A major development with particular relevance to this quest is the
ambitious effort by the WHO to generate Global Health Estimates (GHE) to facilitate
cross-national comparisons. These estimates are available for much larger samples of
nations than are the cause of death reports contained in the WHO Mortality Database.
WHO disseminates homicide estimates via their GHE. In addition, the United Nations
Office on Drugs and Crime (UNODC) has incorporated the WHO estimates for some
countries in their Global Studies on Homicide (GSH; UNODC, 2011, 2013, see 2013,
p. 110).1 Studies are beginning to appear in the literature that draw upon the newly
generated WHO data with estimated measures of homicide.
The purpose of our research note is to raise awareness of the nature of the WHO
homicide estimates and to highlight appropriate and inappropriate uses of these data.
We begin by describing the distinctive purposes for the development of homicide
314
Homicide Studies 21(4)
estimates by WHO and by explicating their estimation procedures. We then document
the growing use of homicide estimates in the literature and explain why the use of the
data based on estimates is potentially problematic. Finally, we offer some concluding
thoughts about the value of the WHO homicide estimates for cross-national research.
The Rationale for WHO Homicide Estimates and the
Estimation Procedures
It is important at the outset to place the WHO homicide estimates within context. As
noted above, the homicide data with estimates are in essence a by-product of a larger
effort. The analysts at WHO have been concerned primarily with the more general
issue of the relative importance of different health problems for societies across the
globe. To facilitate meaningful comparisons, they have devoted a good deal of effort
to assessing the quality of the vital statistics data that are supplied by individual
nations, and they have developed procedures to adjust the data that were submitted
when suspect, sometimes incorporating information from other data sources. In addi-
tion, they have implemented modeling procedures to generate estimates of the respec-
tive causes of death when data are lacking in part or in whole, thereby expanding the
pool of nations for which inferences about the relative importance of different health
concerns might be made.
The WHO homicide estimates are based on a rather complex process that combines
vital statistics data from the WHO Mortality Database and criminal justice data from
the UNODC (WHO, Global Status Report on Violence Prevention, 2014, p. 62).2 As
noted above, criminological researchers have generally accepted the vital statistics
data on homicide as the “gold standard” for cross-national research. However,
researchers at WHO have discovered that the reported numbers of homicide in this
source, as well as in the criminal justice data, are in some cases suspect. This has
prompted them to generate their estimates of homicides.
To explain the procedures, it is necessary to introduce some important conceptual
distinctions. There are two basic “modes” of estimation used by WHO. One mode is
grounded in data on homicides that come from the vital registration systems and/or
criminal justice sources for a particular nation. The WHO (ibid., p. 63) researchers
label the resulting estimates the directly estimated homicide rates. For some nations,
no reasonably reliable data are available from either of these two main data sources on
homicides. In these instances, the WHO researchers have relied on regression models
to predict homicides when the data are missing from a set of covariates. These are
referred to as model-based homicide rates.
An additional important conceptual distinction pertains to three types of health sta-
tistics: reported homicide deaths, adjusted homicide deaths, and comparable homicide
estimates ibid., p. 62). The reported homicide deaths are the “raw” data on homicides
that come directly from the vital registration statistics and/or the criminal justice statis-
tics of various countries. The WHO researchers have developed procedures to correct
these raw data for underreporting and misclassification. In some instances, the WHO
analysts determine after quality controls that only a specified proportion of all deaths
Kanis et al.
315
are recorded. The counts for all causes of death are accordingly adjusted upward. In
other instances, vital statistics data include a proportion of deaths that are classified as
deaths due to injuries for which the intent is unknown. These...
To continue reading
Request your trial