Navigating expert reliability: are criminal standards of certainty being left on the dock?

AuthorRisinger, D. Michael

This article shows that, as to proffers of asserted expert testimony, civil defendants win their Daubert reliability challenges to plaintiffs' proffers most of the time, and that criminal defendants virtually always lose their reliability challenges to government proffers. And, when civil defendants' proffers are challenged by plaintiffs, those defendants usually win, but when criminal defendants' proffers are challenged by the prosecution, the criminal defendants usually lose. The article then goes on to examine, in detail, various categories of expert proffers in criminal cases, including "syndrome evidence," polygraph, bite mark, handwriting, modus operandi, and eyewitness weakness, to shed light on whether the system bias revealed in the statistical breakdown is illusory or real. Finally, an afterword analyzes the last year's cases, and makes observations on apparent trends.


    With its decision in Kumho Tire Co., Ltd. v. Carmichael(1) the United States Supreme Court has launched the lower federal courts on a voyage of discovery. The object of this quest is a set of defensible standards of threshold dependability for all expertise introduced into evidence in the trial of cases.(2) The lower courts begin this quest with only a sketchy set of sailing directions provided by the Supreme Court, and it could easily be a decade before we are in a position to evaluate the results of this enterprise. However, it is not too early to alert the fleet to one grave potential disaster, and to suggest, with more particularity than the Court has done, some navigational principles that may aid in bringing the enterprise to a respectable and productive conclusion.

    The system shipwreck I fear is that in ten years we will find that civil cases are subject to strict standards of expertise quality control, while criminal cases are not. The result would be that the pocketbooks of civil defendants would be protected from plaintiffs' claims by exclusion of undependable expert testimony, but that criminal defendants would not be protected from conviction based on similarly undependable expert testimony. Such a result would seem particularly unacceptable given the law's claim that inaccurate criminal convictions are substantially worse than inaccurate civil judgments, reflected in the different applicable standards of proof.(3)

    I believe we have reason to fear this potential result of the courts' attempts to fashion new standards of expertise dependability because, in large measure, this situation is already the operational reality (though perhaps the current scheme of things is more a result of accident than of conscious decision).

    Or perhaps, while not completely conscious, describing it as accidental goes too far. In a rough generality sufficiently precise for present purposes, we might describe the 1970s and early 1980s as a period of virtually unbridled expansion of varieties of asserted expertise in civil and criminal courtrooms, limited only by the imagination of an attorney with a point to prove and a hole in her more conventional evidence.(4) The appeal of using such experts stems, in large part, from two aspects of the law, one in regard to experts and one in regard to sufficiency of evidence. Experts were allowed to give "opinions" that could be phrased in terms of the ultimate issues in the case.(5) If the "opinion" of the expert was competent, it might be adopted in toto by the jury, hence making failure of proof on the issue legally impossible.(6) Combine this situation with lax threshold standards of admissibility for expertise, and you set the stage for the acceptance of some fairly questionable practices in the utilization of expertise by litigants. And, the threshold standards of the period were decidedly lax.(7) So, while all sides were free to play the game, the result was generally much more favorable to parties with the proof burdens (generally civil plaintiffs and the prosecution in criminal cases, though criminal defendants were substantial players in regard to various affirmative defenses).

    In the mid and late 1980s voices were beginning to be raised in protest, saying that the kind of expertise regularly accepted as admissible by courts was, frankly, "junk" of scandalous lack of dependability. Voices were raised protesting lack of dependability in both the criminal(8) and civil(9) spheres, but the voice that finally spoke loudest, and was heard most clearly, spoke almost exclusively of the injustice of "junk" expertise used against civil defendants. I refer, of course, to Peter Huber and his 1991 book Galileo's Revenge,(10) which popularized the phrase "junk science." It is unlikely to be pure coincidence that the Supreme Court chose a civil case, Daubert v. Merrell Dow Pharmaceuticals Inc.,(11) to review the appropriate criteria of dependability,(12) or that its two subsequent forays into these waters have also been in civil cases.(13) Be that as it may, the pronouncements of the Supreme Court were nominally trans-substantive constructions of the Federal Rules of Evidence, and so have application in criminal as well as civil cases.(14) And, as positions of the United States court of the highest prestige, they might be predicted to have some influence on approaches to dependability in the states, both in those jurisdictions with explicit analogues to Federal Rule of Evidence 702, and those without.

    So, what relative impact has Daubert had on civil and criminal litigation here at the beginning of the Kumho Tire era? When I first started writing this I had an idea, but I then realized that it was insufficiently based on evidence. So I have looked.


    As of August 2, 1999, there were nearly 1600 citations to Daubert in opinions issued by American courts(15) and published on Westlaw(16), including around 535 in the state courts.(17) Not all of these citations were in cases involving expert dependability issues, of course, but the great bulk of them were. This represents a truly prodigious increase in judicial examination of expert reliability, particularly in civil cases, on both a state and federal level. While we cannot, of course, compare Daubert citations post-Daubert to Daubert citations before the case was decided, we can use Frye v. United States(18) as a good proxy to get a fair idea of pre-Daubert judicial activity in this regard. Given Frye's iconic status, it is unlikely that many opinions rejecting expertise on reliability grounds prior to Daubert would have failed to cite Frye, especially in federal courts. So what do we see when we compare the universe of Daubert-citing opinions from July 1, 1993 to August 2, 1999 (the reference set) to the universe of Frye-citing opinions for similar six-year one-month periods prior to Daubert?

    Looking first at the federal court of appeals cases, 416 opinions in the reference set cite Daubert,(19) 287 of which were non-criminal. In the same length period pre-Daubert (June 1, 1987, to July 1, 1993) only twenty-one cases cited Frye, and only seven of these were noncriminal. And, in the similar period before that (May 1, 1981, to June 1, 1987) there were another twenty-two cases that cited Frye, of which only four were non-criminal. Clearly Daubert triggered a deluge, especially in regard to civil cases.

    Nor are things different in regard to the federal district courts.(20) In the post-Daubert reference set, there were 649 district court opinions, of which 584 were non-criminal. In the first pre-Daubert period there were twenty-nine cases citing Frye, of which twelve were criminal and seventeen were non-criminal. The number of district court opinions on expert reliability in criminal cases rose five fold, but similar civil cases rose thirty-four fold. And, in the period before that, there were only thirteen cases total, six criminal and seven civil. If anything, Daubert triggered a larger flood in the district courts than in the courts of appeals, especially in civil cases.

    Finally, there are the state cases.(21) In the main reference set, 528 cases cited Daubert, 288 criminal (55%) and 240 non-criminal (45%). In the first preceding comparison period there were 398 cases citing Frye, of which less than 20% were non-criminal, and in the period before that there were 190 cases citing Frye, of which about 13% were non-criminal. So, in state cases as well, the post-Daubert era has seen an explosion of civil cases, with criminal cases being relatively less affected. Now let us turn to an examination of the cases in the main reference set in more detail, beginning with the cases from the United States courts of appeals.

    1. The Court of Appeals Cases

      As already noted, there were 416 court of appeals opinions in the reference set which cited Daubert. Of these, 129 are fairly characterized as dealing with dependability issues in a criminal context, but some of these deal with ancillary proceedings such as sentencing. Excluding such peripheral cases, about 120, or roughly 30% of the total cases, deal with challenges to the dependability of expert evidence proffered for consideration on issues of guilt or innocence. (The reader will find a graphic representation, detailing the distribution of civil and criminal cases within the reference set, in Appendix Figure 1.)

      In these cases, which generally involve criminal defendants who were convicted at the district court level, the dependability of expert testimony offered by the government is attacked sixty-seven times,(22) and the exclusion of expert testimony offered by defendants and knocked out after a government dependability objection is complained of fifty-four times.

      In the sixty-seven cases of challenged government expertise, the prosecution prevailed in all but six, as seen in Appendix Figure 2A. And, in only one of those six was there a reversal because proffered expertise was found to be generally undependable, or undependable in regard to the...

To continue reading

Request your trial

VLEX uses login cookies to provide you with a better browsing experience. If you click on 'Accept' or continue browsing this site we consider that you accept our cookie policy. ACCEPT