When 10 trials are better than 1000: an evidentiary perspective on trial sampling.

• Author: Cheng, Edward K.

INTRODUCTION I. AVERAGING II. SHRINKAGE III. NONRANDOM SAMPLING CONCLUSION INTRODUCTION

In many mass tort cases, individual trials are simply impractical. Take, for example, Wal-Mart Stores, Inc. v. Dukes, (1) a class action employment discrimination suit that the Supreme Court reviewed last Term. With over 1.5 million women potentially involved in the litigation, (2) the notion of holding individual trials is fanciful. Other recent examples of the phenomenon include the In re World Trade Center Disaster Site Litigation (3) and the fraud litigation against light cigarette manufacturers, (4) in which Judge Weinstein colorfully noted that any "individualized process ... would have to continue beyond all lives in being." (5)

Faced with an unserviceable number of plaintiffs, courts have proposed sampling trials: rather than litigating every case, courts would litigate a small subset and award the remaining plaintiffs statistically determined amounts based on the results. But while sampling is standard statistical practice and often accepted as evidence in other legal contexts, (6) appellate courts have balked--based on due process concerns--at the notion of court-mandated, binding trial sampling. (7)

Despite this appellate reluctance, the controversy continues unabated. Trial courts have soldiered on by using nonbinding sampled trials (dubbed "bellwether trials") to induce settlement, (8) and a few brave appellate courts, including the Ninth Circuit in Dukes, have even hinted at an increased receptivity to sampling. (9) Given that trial courts have few practical alternatives, one wonders if it is just a matter of time before their appellate brethren recognize the necessity of sampling.

The most common--and most salient--argument supporting trial sampling is economic efficiency. Since the legal system lacks the resources to litigate hundreds of thousands of asbestos cases, some kind of resolution seems better than none. Otherwise, the tort system's primary goals of deterrence and compensation will be profoundly undercut. Opponents' objections predictably take a liberty- or rights-based approach: defendants are entitled to individual trials, and approximate justice will not do, no matter what the social costs.

Since sampling is nominally a species of civil procedure, the focus on efficiency and individual rights is understandable. In this Essay, however, I want to explore the sampling controversy from an evidentiary perspective. (10) Putting aside economic and liberty interests, what effect does sampling have on accuracy? Most discussions on this topic implicitly assume that sampling is a "second best" solution, contemplated only in exceptional circumstances. Individual trials are the foundation of the legal system, and though imperfect, they are presumed to be the best we can do. Thus, if we could actually try all 1.5 million cases in Dukes, we should. After all, since sampling involves estimating liability from a selected subset of cases, it would appear suboptimal to individualized adjudication. Or is it?

In the pages that follow, I offer three ways in which this "second best" assumption can be wrong. Given the right conditions, sampling can actually produce more accurate outcomes than individualized adjudication. Intuitively, sampling's advantages come from its ability to borrow strength from the different cases in the sample. Individual adjudication confines itself to a single case and factfinder; sampling does not. This basic principle recurs in each Part of this Essay, which address averaging, shrinkage, and nonrandom sampling. What's more, the good news is that the sampling procedures proposed by courts frequently capture these advantages, even if the original impetus may have been cost reduction and not accuracy. (11)

1. AVERAGING

   As noted in the Introduction, the primary motivation for sampling cases is to reduce litigation costs. Sampling is necessary in asbestos, the World Trade Center litigation, light cigarettes, or any other major mass tort because courts are not equipped to run hundreds of thousands of trials. But these arguments overlook a more fundamental question: even if we could litigate all of the pending cases individually, should we? At first glance, the answer seems an obvious "yes." The apparent problem with sampling is that it makes extrapolations from the sampled cases to the nonlitigated cases, which introduces error. If one were to litigate all of the cases individually, however, this extrapolation error would disappear.

   But extrapolation is not the only source of error when estimating damages. The jury itself is an imperfect device for measuring damages that produces error--at least in the statistical or scientific, rather than the legal, sense. On this score, sampling has distinct advantages. With individualized assessments, each case gets one jury, and absent remittitur or appellate reversal, the system is stuck with the result. Variabilities in the jury pool, mistakes made by the jury or the attorneys, or even nonjury related contingencies all become an unmitigated part of the litigation outcome. With sampling, however, case-specific contingencies even out because the case results are averaged across the sample. In short, sampling may introduce extrapolation error, but it also reduces variability.

   From an accuracy standpoint, whether one prefers sampling or individualized trials is thus a function of case homogeneity and jury variability, an observation first made by Saks and Blanck. (12) If the sampled cases are very similar (which means low extrapolation error) or juries are very flaky (which means high measurement error), then sampling and averaging will produce more stable and accurate damage assessments than case-by-case adjudication. On the flip side, if the sampled cases are appreciably different, or juries are reliable, then the conventional preference for case-by-case adjudication holds. The desirability of sampling thus rests on two empirical questions: first, a general social science question about jury behavior and reliability; and second, a litigation-specific question about the homogeneity of the cases involved. (13)

   In principle, the averaging advantage discussed above has very little to do with sampling per se. After all, one could reap the benefits of averaging merely by trying each case to multiple juries...