Interpreting Evidence: Evaluating Forensic Science in the Courtroom.

• Author: Friedman, Richard D.

Dwyer is accused of burglary. The police have found on his shoes several fragments of glass that have a mean refractive index close to -- but not identical to -- the mean refractive index of fragments of a broken window at the scene of the burglary.

The police also have found a blood stain near the broken window. Police investigators have subjected to DNA analysis both this stain and a sample of blood that they have taken from Dwyer. For each sample, this analysis attempts to measure the length of fragments at several identifiable sites, or loci, on the DNA molecule. The measurements for the two samples are very close -- though not identical -- for each locus.

How should the significance of this evidence be assessed?

INTRODUCTION

Over the past few decades the law reviews, and more recently the Internet, have borne extensive discussions of whether, how, and how much the conventional theory of probability and alternatives to it are useful in modeling and analyzing problems in the law of evidence.(1) The discourse often has been abstract, but, as the hypothetical above suggests, it also has important consequences for determining how evidence is presented and assessed in litigation.

David A. Schum's Evidential Foundations of Probabilistic Reasoning,(2) C.G.G. Aitken's Statistics and the Evaluation of Evidence for Forensic Scientists,(3) and Bernard Robertson and G.A. Vignaux's Interpreting Evidence: Evaluating Forensic Science in the Courtroom(4) all have something to tell us about how to use and evaluate evidence. Although the books are addressed to different primary audiences(5) and their authors come from a variety of disciplines and from distant points of the English-speaking world,(6) all three help draw the connection between underlying theory and presentation in the courtroom. Though Schum uses numerous examples from litigation and discusses the legal literature of probability and evidence, he focuses primarily not on forensic matters but on the broader question of inference "in our work and in other parts of our daily lives" (Schum, p. xiii). Accordingly, he examines in depth the structure of inference, emphasizing conventional probability theory and alternatives to it. Aitken and Robertson and Vignaux, by contrast, essentially assume that the conventional theory is valid and apposite. They concentrate on the question of how to apply that theory in assessing and presenting evidence, especially scientific evidence, in court.

These three books, very different from one another, are all important works. Anyone wishing to think carefully about the nature of inference should study Schum's book. The reader is sure to find her fundamental conceptions -- whatever they may be -- seriously challenged. Evidential Foundations of Probabilistic Reasoning represents the culmination of many years of Schum's work, and it also helpfully summarizes much other scholarship from an extraordinarily wide variety of fields.(7) Litigators and academics interested in evidence, especially scientific evidence, should absorb the arguments that Robertson and Vignaux advocate vigorously and accessibly and that Aitken supports more technically.(8) In many contexts, a lawyer presenting scientific evidence in court should be sure that his forensic scientist can apply and present the statistical techniques set out by Aitken -- or, if not, that he has a statistical expert who can.(9)

I cannot hope in this essay to summarize all the significant aspects of these three books. Rather, I will comment on selected themes. In Part I, I discuss briefly the usefulness of standard probability theory for analyzing legal inference. In Part II, I show why, as all the authors agree, the likelihood ratio is an important concept within probability theory for assessing the probative value of evidence. In Part III, I discuss theoretical problems, some highlighted by the authors, concerning the likelihood ratio. I conclude in Part IV by discussing consequences for the presentation of evidence in litigation.

1. THE CONVENTIONAL THEORY OF PROBABILITY AND ITS ALTERNATIVES

   I will begin with a primer on some principles of the conventional theory of probability and its application to problems of inference. This discussion may strike a median, not necessarily a happy one, between readers who find it unduly elementary and those who find it intimidatingly mathematical. Perhaps, though, some readers in the first group will find the diagrammatic technique interesting, even though the substance is hardly novel. I hope also that some readers in the second group will realize, with a little work, that the mathematics are really quite elementary.

   The theory begins with some basic premises. The probability of any event or proposition is assessed between 0 and 1, with 0 representing impossibility and 1 representing certainty. The probabilities of mutually exclusive events are additive -- that is, if E and H both cannot be true, P(E or H) = P(E) + P(H). Because E and Not-E both cannot be true, but one or the other must be true, P(Not-E) = 1 - P(E). And P(H|E), the probability of H given that E is true, may be expressed this way:

   P(H|E) = P(E and H)/P(E). (1)(10)

   Figure 1 may help us explore Equation 1.

   [ILLUSTRATION OMITTED]

   The figure is a box the area of which we may deem to equal 1. A portion of the box is labeled H and marked with vertical lines. Another portion, which overlaps in part, is labeled E and bears horizontal marks. Thus, the overlapping portion is labeled E and H and has crossed marks. P(H), the probability that a point picked at random from the entire box will be within H, is simply the area of H; similarly, P(E) equals the area within E, bearing horizontal marks, and P(E and H) equals the area of the overlap.

   Now suppose we want to assess the probability of a hypothesis -- that a certain point, randomly selected, falls within H -- and we are given a piece of evidence, that the point falls within E. Given this information, we need not concentrate on the entire box; rather, we should narrow our focus to the area with horizontal marks, representing the truth of E. Within that area, H is true only within the smaller area bearing crossed marks. The probability of H, given the truth of E, is thus equal to the ratio of P(E and H) to P(E) -- that is, the ratio of the probabilities of E and H and of E alone, respectively, both assessed before the truth of E was known.

   Because Equation 1 speaks in general terms, the variables E and H may be reversed. That is,

   P(H|E) = P(H and H)/P(H). (2)

   And because P(E and H) equals P(H and E), Equations 1 and 2 may be combined to yield

   P(E and H) = P(E) [??] P(H|E) = P(H and E) = P(H) [??] P(E|H),

   and so P(H|E) = P(H) [??] P(E|H)/P(E). (3)

   Equation 3 is a form of what is known as Bayes's Theorem (or Rule), and it deserves close examination. The factfinder's task is to determine P(H|E), the probability of the hypothesis given the evidence. In some cases it will be easier to assess P(E|H), the probability of the evidence given the hypothesis, which may provide a practical basis for assessing P(H|E). Bayes's Theorem thus provides a means for transposing the conditional, using P(E|H) to help determine P(H|E).

   Bayes's Theorem also may be stated in another compact form, one particularly helpful in discussing the theme of these books. Substituting Not-H for H in Equation 3, we have

   P(Not-H|E) = P(Not-H) [??] P(E|Not-H)/P(E). (4)

   Dividing Equation 3 by Equation 4, and simplifying slightly, yields

   P(H|E)/P(Not-H|E) = P(H)/P(Not-H) [??] P(E|H)/P(E|Not-H). (5)

   The odds of a proposition X are defined to be P(X)/[1 - P(X)], or P(X)/P(Not-X). Thus, a probability of .5, or 1/2, corresponds to even odds of 1. The left-hand side of Equation 5 is therefore O(H|E), the odds of H given E, and the first fraction on the right-hand side is O(H), the odds of H assessed before knowledge of whether E is true. The second fraction on the right-hand side is the likelihood ratio of E with respect to H, which I will denote as [L.sub.H](E). It indicates how much more (or less) likely the evidence E will appear given the hypothesis H than given the negation of that hypothesis. Equation 5 now may be rewritten as

   O(H|E) = O(H) [??] [L.sub.H](E). (6)

   In other words, to determine the posterior odds of H given E, we can begin with the prior odds of H assessed before we learned whether E was true, and then multiply by the likelihood ratio. If E is more likely to be true given H than given Not-H, then proof of E increases the odds of H; by contrast, if E is less likely to be true given H than given Not-H, then proof of E decreases the odds of H; and if E is equally likely to be true given H and given Not-H, then proof of E is irrelevant to H, leaving the odds of H unchanged.

   Again, Figure 1 may help demonstrate this relationship. Before we know whether E is true, the odds, of H are the ratio of the area above the border of H to the area below it, P(H)/P(Not-H). Once we know that E is true, we must confine our attention to the area to the right of the diagonal line; within that area, the odds of H are again the ratio of the area above the border of H to the area below it, P(E and H)/P(E and Not-H). That is, the proof of E means that we substitute the smaller area (E and H) for the larger area H in the numerator of the odds ratio, and the smaller area (E and Not-H) for the larger area Not-H in the denominator. Put another way, adjusting the odds of H in light of E requires us to multiply the numerator by P(E and H) and divide it by P(H), and to multiply the denominator by P(E and Not-H)/P(Not-H). By applying Equation 2 to these expressions, we can see that this means more simply that we multiply the prior odds by P(E|H)/P(E|Not-H), which is the likelihood ratio.

   [ILLUSTRATION OMITTED]

   In Figure 1, because the line marking off E runs diagonally down from left to right, a higher proportion of the H area than of the Not-H area is within E; that is, P(E|H) is greater...