The Twiqbal puzzle and empirical study of civil procedure.

AuthorEngstrom, David Freeman
PositionBell Atlantic Corp. v. Twombly, Ashcroft v. Iqbal - Seventh Annual Conference on Empirical Legal Studies: The Empirical Revolution in Law

INTRODUCTION I. TWIQBAL AND ELS IN FULL FLOWER A. Gelbach and Boyd et al. on the Twiqbal Puzzle B. The Technological Flowering of ELS: Electronic Docketing and Computer Text Processing C. ELS and Methodological Cross-Pollination II. FORESTS, TREES, AND THE CHALLENGE OF ASSESSING PROCEDURAL CHANGE: THE LIMITS OF TWIQBAL EMPIRICISM A. Measurement and Methods 1. Sampling bias 2. Covariate controls B. The Elusiveness of Social Welfare 1. Unit of analysis 2. Selection and settlement 3. Salutary and non-salutary judicial merits-screening C. Does It Matter? A Twiqbal Empiricism Meta-Analysis III. Is THE BLOOM OFF THE ROSE? LESSONS FOR EMPIRICAL STUDY OF CIVIL PROCEDURE A. The Double-Edged Sword of Democratization B. The Way Forward INTRODUCTION

Few developments in civil procedure have caused anything like the furor that has greeted the Supreme Court's decisions in Bell Atlantic Corp. v. Twombly (1) and Ashcroft v. Iqbal (2) (hereinafter "Twiqbal"). (3) Indeed, earlier installments in the modern transformation of pretrial practice--from the rise of summary judgment, as symbolized by the Supreme Court's 1986 Celotex trilogy, (4) to the serial expansion of judicial case-management powers under Rules 16 and 26 and the related spread of "managerial judging" (5)--look like blips on the scholarly radar by comparison. (6) Yet the reaction to Twiqbal has not just been notable for its volume or intensity. The reaction has also, to an unusual degree, tended toward the empirical. In fact, it sometimes seems as if a hundred empirical flowers have bloomed, each purporting to capture something significant about the decisions' on-the-ground impact. (7)

Why the empirical turn? One reason is that the flimsiness of the Court's doctrinal analysis--particularly its insistence that it has not overruled Conley--(8)--offers thin gruel for serious academic commentary of the traditional sort. Part of it, too, is that Twiqbal presents correspondingly rich empirical puzzles that cry out for analysis, particularly the Court's contention that trial judges can use their "judicial experience and common sense" to efficiently cull meritless cases based on allegations alone and without the benefit of discovery. (9) But perhaps most important of all, the profusion of empirical work since Twiqbal makes clear that quantitative empirical legal studies (or "ELS" to its practitioners (10)) is no longer the province of J.D./Ph.D. types working in specialized corners of the legal academy. Rather, the systematic collection and analysis of litigation-related data is now fully within the mainstream of what civil procedure scholars do. At risk of tautology, there is more empirical work this time around--compared to, say, the period following the Court's Celotex trilogy (11)--because more people are doing it.

Questions remain, however, as to the nature, role, and desirability of this empirical turn. Just how much can we learn from the recent spate of Twiqbal empiricism, whether about pretrial practice in particular or civil procedure more generally? Is the democratization of the ELS genre a healthy development, or is empirical inquiry better left in the hands of a few increasingly sophisticated technicians? And what lessons can we draw from the recent profusion of Twiqbal studies about what empirical study of civil procedure should look like going forward? This Essay uses the Twiqbal decisions and the empirical work they have spurred as a point of entry to consider these questions and reflect upon the contribution that ELS, now in its third decade, (12) has made (and can make) to the study of civil procedure.

  1. TWIQBAL AND ELS IN FULL FLOWER

    1. Gelbach and Boyd et al. on the Twiqbal Puzzle

      Two stellar contributions to the recent Conference on Empirical Legal Studies (CELS) at Stanford Law School--both focused, more or less, on the Twiqbal puzzle--provide a useful starting point for addressing the above questions by offering a glimpse of ELS in full flower.

      In the first, Jonah Gelbach (13) offers the most ambitious and promising empirical test yet of Twiqbal's impact on pretrial practice. Interestingly, Gelbach achieves this not by studying motions to dismiss directly, as his earlier work, and nearly all other Twiqbal empiricism, does. (14) Rather, he examines summary judgment motions before and after Twiqbal on the theory that, if the Court's assumption that trial judges can reliably gauge case merit in disposing of motions to dismiss holds true, then the rate at which judges subsequently grant defense-filed summary judgment motions should decline post-Twiqbal because cases that survive beyond the pleading stage should be more meritorious. His preliminary answer based on an ongoing analysis of job discrimination and contract cases before and after Twiqbal: summary judgment grant rates have not budged, thus calling into question the merits-screening capacity of trial judges armed with new dismissal powers. (15)

      The joint contribution of Christina Boyd, David Hoffman, Zoran Obradovic, and Kosta Ristovski (hereinafter "Boyd et al.") takes a radically different, but no less illuminating, empirical tack. (16) In contrast to Gelbach's effort to isolate and quantify judicial merits-screening capacity, the Boyd et al. study offers a dazzling aerial view of pleading practice within the federal courts using spectral cluster analysis--a taxonomic tool developed in the hard sciences to characterize the relationships among different objects--to summarize the claim-level composition of lawsuits as plaintiffs plead them. (17) The result is a wonderfully revelatory portrait of pleading practice and strategy that Boyd et al. achieve by allocating civil cases to a limited number of claim "clusters" and then mapping the relationships within and between them. Among other things, we learn that understanding litigation flows requires us to know that certain types of claims are often paired together--for instance, intellectual property claims with consumer protection claims, or breach of fiduciary duty claims with tax and securities claims. (18) Yet the exercise also reveals shifts in plaintiffs' pleading strategies over time, with direct relevance to the Twiqbal puzzle. Indeed, Boyd et al. offer preliminary evidence suggesting that the number of causes of action pled per case has declined significantly post-Twombly. (19) Thus, whatever the merits-screening capacities of trial judges deploying Twiqbal's heightened pleading standard, the Court's decisions may have induced a dynamic litigant response.

    2. The Technological Flowering of ELS: Electronic Docketing and Computer Text Processing

      The sophistication and rigor of the Gelbach and Boyd et al. studies should by now be obvious. But it is also useful to step back and note some other ways in which they reflect the full flowering of ELS in the civil procedure space. Perhaps the most important is that both studies rely on electronic docket information as a data source. This has been critical to ELS's recent flowering, both in civil procedure and beyond. (20) Most obvious to anyone who regularly consumes empirical legal research, mandatory electronic docketing within the federal district courts--a process that was mostly complete by the mid-2000s on individual courts' PACER websites--has made it possible for researchers to construct something approaching a random sample of all filed cases of a given type. (21) In contrast to an earlier generation of empirical research on civil procedure and civil litigation, (22) and even some recent Twiqbal-focused empirical efforts, (23) Gelbach and Boyd et al. are studying the entire iceberg of federal litigation, not just its published-opinion or Westlaw-accessible tip. (24)

      Yet the Gelbach and Boyd et al. studies well illustrate two further, and quite divergent, effects of electronic docketing. First, the ready availability of electronic docket materials has permitted a degree of technical sophistication in the construction and analysis of datasets--and, with it, a scale of empirical inquiry--that were unheard of a decade ago, As a concrete example, Gelbach uses a text-processing computer programming language to perform a relatively basic set of sorting and search tasks across thousands of electronic docket sheets to compile his sample of summary judgment motions. (25) But the uses for such technology can also take far more complex forms. To take just one (self-promoting) example, a scholar interested in testing a version of Marc Galanter's influential theory of the advantages enjoyed by repeat players as against one-shotters within litigation regimes (26) can use automated computer methods to "scrape" party and counsel names from thousands of electronic docket sheets and deposit them into a spreadsheet to construct a precise, rolling accounting of the litigation experience and successes of all actors within the system. (27) Utilizing these and other technologies, researchers can develop large-scale, remarkably detailed datasets in a matter of weeks compared to the months or years early ELS researchers spent constructing even rudimentary "docket profiles.'' (28)

      Second, and in clear tension with increasing technical sophistication, electronic docketing has brought empirical legal research within the reach of a wider set of legal scholars. No longer are research efforts necessarily dependent upon large-scale funding to send researchers or runners to courthouses to review or collect docket materials. (29) Nowadays, any researcher with a PACER account--even if denied a statutorily provided academic fee waiver by chief district judges, a continuing embarrassment for the federal judiciary (30)--can draw a random sample of cases from one or more jurisdictions and begin the search for regularities, all without leaving the office.

      Given these technological advances, it should not be surprising to learn that Twiqbal empiricism dwarfs the empirical study performed in response to earlier tectonic...

To continue reading

Request your trial

VLEX uses login cookies to provide you with a better browsing experience. If you click on 'Accept' or continue browsing this site we consider that you accept our cookie policy. ACCEPT