Is expert evidence really different?

Author:Schauer, Frederick


Daubert v. Merrell Dow Pharmaceuticals, Inc. (1) has transformed much of American evidence law. In assigning judges the role of "gatekeepers" of scientific and other expert evidence, (2) and in setting out a list of factors (3) designed principally to keep so-called junk science out of the courtroom, (4) the Supreme Court launched the trial process on a course in which Daubert hearings and Daubert-inspired exclusions of expert evidence are a pervasive feature of modern federal and state litigation. (5)

Daubert has been subject to many criticisms. Some focus on the majority's clumsy philosophy of science. (6) Others note the poor fit between the Daubert criteria and the diverse types of expert evidence to which Daubert now applies. (7) Still others express the concern that Daubert's heightened evidentiary hurdles disproportionately burden the mass tort and employment discrimination plaintiffs who are more likely to rely on non-traditional experts and expertise. (8) And there is the broad worry that law should not outsource its own irreducibly legal determinations to science and scientists with different goals and consequently different standards. (9)

These criticisms are not without force, but the difficulties with Daubert go deeper. The real problem is not so much about Daubert and the subsequent cases in its line of authority (10) as it is about the very way in which evidence law treats expertise generally. Using experts is, of course, invaluable in numerous litigation settings. But as the price of allowing experts to testify about matters of which they have neither first-hand nor case-specific factual knowledge, (11) evidence law requires expert evidence to satisfy a higher threshold for admissibility than that which ordinarily applies to non-expert evidence. In other words, it is the law of expert evidence generally, and not merely Daubert's gloss on it, that excludes a great deal of otherwise relevant evidence.

Excluding relevant evidence might be thought something of a concern, (12) but of course that is what much, and perhaps even most, of evidence law does. (13) Sometimes the exclusion of relevant evidence is a function of a goal extrinsic to the truth-finding process (as with most privileges, for example). (14) More often, however, relevant evidence is excluded because of the fear that certain kinds of admittedly relevant evidence will be overvalued by the trier of fact. (15) It is this fear of overvaluation that grounds, for example, the exclusion of hearsay (16) and much of the bar on the use of character evidence. (17)

The same fear of overvaluation also explains much of the exclusion of relevant expert evidence. (18) The risk that juries will overvalue the evidence of experts has long been thought so likely that a standard for admissibility higher than mere relevance emerged as the barrier to jury overvaluation of expert testimony. (19) But the claim that juries (or even judges) will overvalue the testimony of experts is an empirical one, and may well be unsound. (20) Moreover, the contention that juries will overvalue expert evidence is based on an implicit comparison with the presumed accuracy, or at least lack of overvaluation, of more direct evidence, such as the testimony of an eyewitness observer. But that presumption may also be empirically unsound. (21) And if the empirical assumptions about overvaluation of expert opinion and accuracy of direct factual testimony are both mistaken, then the traditional restrictions on expert evidence may be mistaken as well.

Thus, the problem with expert evidence is not the inappropriateness of the Daubert approach. The narrow focus on Daubert is misplaced. The real problem is with the more deeply entrenched view that expert evidence should be excluded under circumstances in which analogous non-expert evidence would be admitted. Daubert embodies the distinction between expert and non-expert evidence, but it is that very distinction, and not just Daubert, that is the problem. Daubert has indeed transformed modern evidence law, but perhaps it has awakened us to the need for a more profound transformation, one in which the very foundations of treating expert testimony differently are undercut. This is a larger claim than that Daubert itself is a problem, and it is this larger claim we seek to advance here. (22)


    In a recent article, one of the country's most respected trial judges, Judge Jed Rakoff of the United States District Court for the Southern District of New York, offered a crisp example of the view we challenge here. (23) Writing in the Seton Hall Law Review about law and science generally, as well as about contemporary controversies over the admissibility of lie-detection evidence, (24) Judge Rakoff observed that an error rate of thirteen percent, which is on the high end of the studies examining polygraph reliability, (25) would likely be insufficiently reliable to justify admissibility. (26)

    Judge Rakoff's conclusion is unimpeachable under current doctrine. Daubert plainly imposes an additional hurdle for the admissibility of expert evidence which is more demanding than the standard applied to non-experts. Whether embodied in Daubert's focus on scientific validity or in acceptance within the relevant professional community under the partially superseded Frye test, (27) the basic idea is that expert and scientific evidence will be admitted only if there is assurance that the methods used and conclusions drawn have met a stringent standard of reliability. (28) Indeed, the fact that another highly respected federal judge initially rejected even fingerprint evidence in the wake of Daubert (29) demonstrates that the gates that Daubert's gatekeepers must guard are not easily breached.

    Technically, Daubert does not mandate a specific level of reliability for expert evidence to be admitted. The decision merely includes the existence of a known error rate as among the factors weighing in favor of admissibility, (30) and the absence of a known error rate as, conversely, inclining against. But nothing in Daubert requires that the error rate be below a certain level to make the evidence admissible. In theory, therefore, a known error rate of thirteen percent (or even higher) need not be fatal to admission. Nevertheless, the focus on a known error rate reinforces Daubert's concern with substantial reliability for scientific evidence or, now, any expert testimony (31) to be admitted, and subsequent cases have made clear that the size of the error rate is highly relevant to reliability and thus admissibility. (32)

    Judge Rakoff's conclusion is entirely consistent with both the holding of Daubert and the entire post-Daubert regime, but it is nevertheless curious. Consider the basic relevance standard, a standard which exists in canonical form for the federal courts as Rule 401 of the Federal Rules of Evidence. Under Rule 401, relevant evidence is evidence having "any tendency to make a [material] fact more or less probable than it would be without the evidence." (33) And Rule 402 goes on to say that "[r]elevant evidence is admissible" (34) except as otherwise excluded by other rules, by statute, or by the Constitution. (35)

    So consider the application of this basic relevance standard to Judge Rakoff's example and the statistics it incorporates. If some piece of evidence supports proposition P, (36) and if its support for P is eighty-seven percent reliable, then P is certainly more probable with this evidence than without. Eighty-seven percent reliability is far from perfect, but is more than enough to satisfy the minimal relevance standard embodied in Rules 401 and 402. That standard would not be satisfied were the results of a polygraph no better than random, as with, for example, the putative admissibility of astrological data or phrenological experts to provide evidence about a witness's veracity or a defendant's character for, say, violence. But even polygraphs are better than astrology and phrenology, although opinion differs on just how much better. (37) Still, an item of evidence that is eighty-seven percent likely to provide support for a material conclusion would comfortably clear the "more probable than without" threshold of minimal relevance. The curious feature of Judge Rakoff's doctrinally impeccable conclusion, therefore, is that evidence that comes from an expert or is classed as "scientific" will not be admitted even if non-expert and non-scientific evidence with no greater reliability, and often with less, would be admitted without question.

    Suppose, for example, that a witness with poor eyesight identifies the defendant, who is of a different race from the witness, as the person he observed running out of the liquor store on a dark and rainy night at a distance of two hundred feet. It is reasonable to suspect that this identification is less, probably far less, than eighty-seven percent reliable. Yet none of the grounds for doubting the reliability of the evidence would necessitate exclusion. The well known deficiencies of cross-race identification, (38) coupled with all the reasons to doubt witness perception in general and the witness's perception on this occasion, (39) would go to the weight of the evidence, but not its admissibility.

    This last example is (more or less) imaginary, but real examples of the admissibility of substantially unreliable non-expert and nonscientific evidence pervade the law of evidence. Many of the recent DNA-based exonerations of people convicted of crimes (40) have reinforced what psychologists have known for decades--eyewitness testimony is not nearly as reliable as most lawyers and laypeople believe. (41) Sometimes because of defects in perception and sometimes on account of flaws in memory, eyewitness testimony, the classic form of direct evidence, is often mistaken. (42) Indeed, numerous studies show that the accuracy rates of commonly admitted eyewitness...

To continue reading