The illegitimacy of one-sided speculation: getting the defensive gun use estimate down.

AuthorKleck, Gary
PositionResponse to article by David Hemenway in this issue, p. 1430


It is obvious to us that David Hemenway (H) had no intention of

producing a balanced, intellectually serious assessment of our

estimates of defensive gun use (DGU). Instead, his critique serves the

narrow political purpose of "getting the estimate down," for the sake

of advancing the gun control cause. An honest, scientifically based

critique would have given balanced consideration to flaws that tend to

make the estimate too low (e.g., people concealing DGUs because

they involved unlawful behavior, and our failure to count any DGUs

by adolescents), as well as those that contribute to making them too

high. Equally important, it would have given greatest weight to

relevant empirical evidence, and little or no weight to idle speculation

about possible flaws. H's approach is precisely the

opposite -- one-sided and almost entirely speculative. Readers who have any

doubts about the degree to which H's paper is imbalanced might carry out a

simple exercise to assess our claim -- count the number of lines H

devotes to flaws tending to make the estimate too high and the number

devoted to flaws making the estimate too low. We submit that the

ratio is over 100-to-1, i.e., almost entirely devoted to speculations

about why the estimate is too high.

The political function of this advocacy scholarship is clear. While

high estimates of DGU frequency do not constitute an obstacle to

moderate controls over guns, they constitute the most serious obstacle

to advocacy of gun prohibition. Disarming the mass of noncriminal

prospective crime victims would, if high DGU estimates are even

approximately correct, result in large numbers of foregone

opportunities for uses of guns that could prevent deaths, injuries, and property

loss. To acknowledge high DGU frequency would be to concede the

most significant cost of gun prohibition. H's paper is an attempt to

neutralize concerns about such costs and to provide intellectual

respectability for positions identified with Handgun Control

Incorporated (HCI), the nation's leading gun control advocacy group.

H has close ties to HCI through two key staff members of HCI's

"educational" branch, the Center to Prevent Handgun Violence

(CPHV). His closest and most frequent collaborator on gun-related

research is Douglas Weil, currently Research Director of CPHV,(1) while

H has co-edited a strongly pro-control propaganda tract with Dennis

  1. Henigan, legal counsel to HCI and CPHV.(2)

H's political intentions and strong feelings are also evident in his

overstatements and in the grandiose conclusions he draws from weak

or irrelevant evidence and fallacious reasoning. He does not get past

his tide before making his first overstatement, claiming that he had

established, without benefit of any new empirical evidence, that our

estimates are too high and that they are "extreme overestimates."(3)

He states in his first paragraph that "it is clear that [the Kleck and

Gertz] results cannot be accepted as valid."(4) He incorrectly claims

that "all checks for external validity of the Kleck-Gertz finding confirm

that their estimate is highly exaggerated,"(5) when in fact these checks

have repeatedly confirmed our estimates.

DGUs usually involve unlawful possession of a gun by the gun-wielding

victim, and sometimes other illegalities as well,(6) a point H

does not dispute. Yet, in making the extraordinary and

counterintuitive claim that there is a social desirability bias to people

reporting their own illegal behavior,(7) H insists that such a desirability

bias is not only plausible, but that it is likely.(8) By the end, without having

provided a scintilla of credible supporting evidence, H concludes that our

research was afflicted by an "enormous problem of false positives"

(persons claiming a DGU who did not have one) and "massive

overestimation," flatly stating that "the Kleck and Gertz survey results do not

provide reasonable estimates about the total amount of self-defense

gun use in the United States."(9) It is an impressive achievement to be

able to arrive at such high-powered conclusions without the

inconvenience of gathering or even citing any new empirical evidence.

  1. The Illegitimacy of One-sided Speculation: An Ounce or Evidence

    Outweighs a Ton of Speculation

    H's critical technique is simple: one-sided, and often implausible,

    speculation about flaws that might have afflicted our research, and

    that might have been consequential enough to significantly affect our

    conclusions. H devotes his attention almost exclusively to suspected

    flaws that might have contributed to the overestimation of defensive

    gun use (DGU) frequency. He either ignores well established sources

    of underreporting, or briefly and superficially discusses them only for

    the sake of dismissing them.(10) When H speculates about sources of

    response error that are plausible, he offers no rationale for why the

    problems should lead to more false positives than false negatives.

    Instead he simply conjures up reasons they might lead to false positives.

    As support for his one-sided speculations H even cites other people

    guilty of the same dubious practice.(11)

    All research is flawed. Known flaws should be identified and

    their likely impact assessed. Speculation about flaws can play a role in

    the pursuit of truth by motivating researchers to gather better

    empirical evidence less afflicted by the flaws. Speculation by itself, however,

    should not be given any weight in assessing evidence. An ounce of

    evidence, even though flawed, outweighs a ton of speculation.

    Unfortunately, in both good research and bad, there is no upper limit on

    the amount of speculative criticism that can be directed at the work,

    and thus this sort of critique is just as easily applied to good research

    as to bad.

  2. Red Herrings and the Issue Not Addressed

    Much of H's paper is a red herring in that it implicitly misstates

    the central technical question about our estimates. Much of it is

    devoted to elaborate speculations about why people might falsely claim

    to have used a gun defensively, as if it were somehow in dispute that

    there are some false positives.(12) He inaccurately hints that we

    unreasonably ignored the possibility that some of our respondents (Rs)

    provided false positives.(13)

    We assume as a matter of course that our survey is like all other

    surveys in that some Rs give inaccurate responses to questions, and

    that these errors include both false positives and false negatives. The

    central question is not whether there are false positives, nor even how

    many false positives there are, but rather what the relative balance is

    between false positives and false negatives. Because H makes no effort

    to assess the frequency of false negatives,(14) it is logically impossible for

    him to draw meaningful conclusions about whether our estimates

    were too high or low.

  3. The Nature of False Positives

    It is hard to discern exactly what kinds of false positives H thinks

    most often show up in all these gun use surveys. He waffles on the

    issue of whether people are: (1) consciously inventing nonexistent

    events; (2) consciously but honestly misrepresenting accounts of real

    events that did not really involve DGU (e.g., they involved aggressive

    use of a gun); or (3) unconsciously distorting real events. He seems to

    have doubts himself about possibility (1) occurring very often,

    hastening to assure readers that false responders do not necessarily have to

    lie,(15) but is otherwise unwilling to commit himself to the relative

    frequency of these types of misreports.

    It is worth emphasizing how difficult it was for our Rs to falsely

    report a completely nonexistent event as a DGU. Unlike the UFO

    example that H insists is somehow parallel to reports of DGUs,(16) a

    respondent who wanted to falsely report a nonexistent DGU could not

    qualify as having had such an experience merely by saying "Yes."

    Rather, respondents had to provide as many as nineteen internally

    consistent responses covering the details of the alleged incident. In

    short, to sustain a false DGU claim, Rs had to do a good deal of agile

    mental work, and stay on the phone even longer. On the other hand,

    all it took to yield a false negative was for a DGU-involved R to speak a

    single inaccurate syllable: "No." The point is not that false positives

    were impossible, but rather that it was far harder to provide a false

    positive than a false negative.

    Consider also the context in which H imagines all these false

    reports to have occurred. Randomly selected people were called

    unexpectedly, and questioned rapidly by total strangers, for no more than

    fifteen minutes, with one question immediately following another.

    There was no prolonged opportunity to invent a nonexistent event,

    rehearse inaccurate details, or to otherwise get an false story straight.

    Rs providing a false positive had to be not only dishonest but very

    quick-witted as well.

    Regarding possibility (2), we noted that most of the DGUs were

    linked with the types of crimes -- burglaries, robberies, and sexual

    assaults -- where there is little opportunity for participants to be honestly

    confused about who was the victim and who was the offender.(17) While

    a few Rs may well have consciously misrepresented aggressive actions

    as defensive, and a very few might have consciously invented entirely

    fictitious events, it is hard to see how Rs could report an account of a

    real burglary, robbery, or sexual assault in which they were aggressors

    and somehow honestly distort it into a DGU incident.

    This kind of misunderstanding of real events in a way that falsely

    qualifies them as DGUs is more plausible in connection with

    male-against-male assault incidents, such as when people prefer to

    characterize their partly aggressive, partly defensive behavior in "mutual

    combat" incidents as purely defensive in character. We addressed this

    latter possibility in our article and showed that it could not account

    for more than a small fraction (probably less than a tenth) of the


To continue reading

Request your trial

VLEX uses login cookies to provide you with a better browsing experience. If you click on 'Accept' or continue browsing this site we consider that you accept our cookie policy. ACCEPT