FORENSIC SCIENCE: DAUBERT'S FAILURE.

• Author: Giannelli, Paul C.

"The man who discovers a new scientific truth has previously had to smash to atoms almost everything he had learnt, and arrives at the new truth with hands bloodstained from the slaughter of a thousand platitudes." (1)

CONTENTS INTRODUCTION A. Daubert and Rule 702 B. National Academy of Sciences Forensic Report (2009) I. DISCREDITED TECHNIQUES A. Bite Mark Comparisons 1. Foundational Research 2. Texas Forensic Science Commission (2016) 3. White House PCAST Report (2016) B. Microscopic Hair Analysis 1. FBI Hair Review 2. White House PCAST Report (2016) C. Arson Investigations 1. Willingham Case 2. Han Tak Lee Case 3. National Fire Protection Association Guidelines 4. Dog-Sniff Evidence 5. Post-Daubert Cases D. Comparative Bullet Lead Analysis 1. NAS Bullet Lead Report (2004) 2. Post-Report Developments II. Misleadingly Presented Techniques A. Firearms & Toolmark Identifications 1. Post-Daubert Cases 2. NAS Ballistic Imaging Report (2008) 3. NAS Forensic Science Report (2009) 4. White House PCAST Report (2016) B. Fingerprint Examinations 1. Post-Daubert Cases 2. Madrid Train Bombing 3. NAS Forensic Science Report (2009) 4. White House PCAST Report (2016) 5. AAAS Fingerprint Report (2017) III. FORENSIC SCIENCE RESEARCH A. National Commission on Forensic Science (2013-17) B. White House PCAST Report (2016) IV. INDEPENDENT SCIENTIFIC REVIEW CONCLUSION INTRODUCTION

In 2015, Judge Alex Kozinski of the Ninth Circuit Court of Appeals noted that "[m]any defendants have been convicted and spent countless years in prison based on evidence by arson experts who were later shown to be little better than witch doctors." (2) In the same year, Dr. Jo Handelsman, a White House science advisor, observed: "Suggesting that bite marks [should] still be a seriously used technology is not based on science, on measurement, on something that has standards, but more of a gut-level reaction." (3) According to Judge Catharine Easterly of the D.C. Court of Appeals, "[a]s matters currently stand, a certainty statement regarding toolmark pattern matching has the same probative value as the vision of a psychic." (4) A New York Times editorial echoed these sentiments:

[C]ourts have only made the problem worse by purporting to be scientifically literate, and allowing in all kinds of evidence that would not make it within shouting distance of a peer-reviewed journal. Of the 329 exonerations based on DNA testing since 1989, more than one-quarter involved convictions based on "pattern" evidence--like hair samples, ballistics, tire tracks, and bite marks--testified to by so-called experts. (5) These criticisms are valid--which raises a puzzling and consequential question: Why didn't the Supreme Court's "junk science" decision, Daubert v. Merrell Dow Pharmaceuticals, Inc., (6) prevent or restrict the admissibility of testimony based on flawed forensic techniques? After all, Daubert was decided in 1993, twenty-five years ago.

1. Daubert and Rule 702

   Daubert was considered a revolutionary decision. (7) It "radically changed the standard for admissibility of scientific testimony" (8) by sweeping away the Frye v. United States (9) "general acceptance" test, (10) which had been the majority rule in both federal and state cases. (11) The Frye standard gave great deference to the views of forensic practitioners and not to empirical testing. (12) Daubert promised to be different. The Supreme Court held that "[p]roposed testimony must be supported by appropriate validation--i.e., 'good grounds,' based on what is known. In short, the requirement that an expert's testimony pertain to 'scientific knowledge' establishes a standard of evidentiary reliability." (13) In making this reliability determination, the Daubert Court highlighted five factors: (1) empirical testing, (2) peer review and publication, (3) error rate, (4) maintenance of standards, and (5) general acceptance. (14) The first and most important factor is empirical testing. The other factors are supplementary. (15) Peer review and publication are designed to expose defects in testing. Acceptance of a technique within the scientific community is achieved through the publication of valid test results. Similarly, both error rates and standards are derived from testing.

   Daubert was followed in 1999 by Kumho Tire Co. v. Carmichael, (16) which held that Daubert's reliability standard applied to all expert testimony, not only scientific evidence. (17) By 2000, the Supreme Court was describing Daubert as establishing an "exacting" standard. (18) In the same year, Federal Rule of Evidence 702 was amended to incorporate the Daubert-Kumho standard. (19) Although a handful of jurisdictions continue to apply the Fry e test, about forty jurisdictions have adopted the Daubert standard in one form or another. (20)

   During this time, there was no shortage of commentary on the lack of empirical research in forensic science. (21) For example, shortly after Daubert was decided, Professor Margaret Berger wrote: "Considerable forensic evidence made its way into the courtroom without empirical validation of the underlying theory and/or its particular application." (22) After Kumho, two commentators--citing bite mark, hair, and firearm analysis--observed that "little rigorous, systematic research has been done to validate the discipline's basic premises and techniques, and in each area there was no evident reason why such research would be infeasible." (23)

   Notwithstanding Daubert's promise, scholars soon discerned its uneven application in civil and criminal cases: "[T]he heightened standards of dependability imposed on expertise proffered in civil cases has continued to expand, but ... expertise proffered by the prosecution in criminal cases has been largely insulated from any change in pre-Daubert standards or approach." (24) The title of a 2005 article summed up the state of the law--"The (Near) Irrelevance of Daubert to Criminal Justice. (25) In short, in the criminal context, courts applied Daubert-lite.

2. National Academy of Sciences Forensic Report (2009)

   In 2006 Congress entered the picture by authorizing the National Academy of Sciences ("NAS") to conduct a study of forensic science. After a three-year investigation, NAS issued a landmark report. One of its most riveting passages concluded: "Among existing forensic methods, only nuclear DNA analysis has been rigorously shown to have the capacity to consistently, and with a high degree of certainty, demonstrate a connection between an evidentiary sample and a specific individual or source." (26) The report went on to state that "some forensic science disciplines are supported by little rigorous systematic research to validate the discipline's basic premises and techniques." (27) Such common forensic techniques as fingerprint examinations, (28) firearm ("ballistics") and toolmark identifications, (29) handwriting examinations, (30) microscopic hair analysis, (31) and bite mark comparisons (32) fell into this category.

   Not only did the NAS report highlight flaws in forensic science, it sharply criticized the judiciary for failing to demand the validation that Daubert required: "The bottom line is simple: In a number of forensic science disciplines, forensic science professionals have yet to establish either the validity of their approach or the accuracy of their conclusions, and the courts have been utterly ineffective in addressing this problem." (33) In a later passage, the report declared that "Daubert has done little to improve the use of forensic science evidence in criminal cases." (34) The disparate treatment of civil actions and criminal prosecutions was also noted. After finding that "trial judges rarely exclude or restrict expert testimony offered by prosecutors," the report commented: "ironically, the appellate courts appear to be more willing to second-guess trial court judgments on the admissibility of purported scientific evidence in civil cases than in criminal cases." (35) Despite the NAS report, courts continued to admit the same evidence. Only a handful of courts applied the "exacting" standard that the Supreme Court said Daubert demanded. (36)

   This Article examines the justice system's failure by reviewing the status of two categories of forensic techniques. The first category involves discredited techniques: (1) bite mark analysis, (2) microscopic hair comparisons, (3) arson evidence, and (4) comparative bullet lead analysis. The second category involves techniques that have been misleadingly presented, including firearm and toolmark identifications and fingerprint examinations. Both categories present Daubert issues. The Article argues that the system's failure can be traced back to its inability to demand and properly evaluate foundational research--i.e., Daubert's first factor, empirical testing. Indeed, the justice system may be institutionally incapable of applying Daubert in criminal cases.

   A different paradigm is needed, one that assigns an independent agency the responsibility of evaluating foundational research. As discussed in Part IV, this approach was recently recommended by the National Commission on Forensic Science (37) and the President's Council of Advisors on Science and Technology ("PCAST"). (38) Both recommended that the National Institute of Standards and Technology ("NIST") evaluate all forensic disciplines on a continuing basis, thereby injecting much needed scientific expertise into the criminal justice system. The recent reports on latent fingerprints (39) and arson investigations, (40) which were published by the American Association for the Advancement of Science ("AAAS"), buttress the need for independent scientific evaluations.

   1. DISCREDITED TECHNIQUES

1. Bite Mark Comparisons

   For decades, bite-mark evidence has been admitted in hundreds of trials, (41) many of which were capital prosecutions. (42) No reported American case has rejected bite mark testimony. Moreover, it is not uncommon for courts to...