Keeping Our Ambition Under Control: The Limits of Data and Inference in Searching for the Causes and Consequences of Vanishing Trials in Federal Court

• Author: Stephen B. Burbank
• Published date: 01 November 2004
• DOI: http://doi.org/10.1111/j.1740-1461.2004.00015.x
• Date: 01 November 2004

Keeping Our Ambition Under Control:

The Limits of Data and Inference

in Searching for the Causes and

Consequences of Vanishing Trials in

Federal Court

Stephen B. Burbank*

This article offers some reflections stimulated by Professor Galanter’s

materials, which were the common springboard for the Vanishing Trials

Symposium. It suggests that other data, quantitative and qualitative, may be

helpful in understanding the vanishing trials phenomenon in federal civil

cases, notably data available for years prior to 1962, and questions whether

it is meaningful to use total dispositions as the denominator in calculating

a trial termination rate. The article argues that care should be taken in using

data from state court systems, as also data from criminal cases, administra-

tive adjudication, and ADR, lest one put at risk through careless assimila-

tion of data or muddled thinking a project quite difficult enough without

additional baggage. The article describes the limitations of data previously

collected by the Administrative Office of the U.S. Courts and highlights

unique opportunities created by the AO’s switch to a new Case Manage-

ment/Electronic Case Files system. It argues that Professor Galanter may

underestimate the influence of both changing demand for court services

(docket makeup) and of changing demand for judicial services (resources)

on the trial rate. Finally, the article argues that conclusions about either the

571

©2004 American Bar Association. All rights reserved.

*David Berger Professor for the Administration of Justice, University of Pennsylvania Law School, 3400

Chestnut St., Philadelphia, PA 19104; e-mail sburbank@law.upenn.edu.

I owe a debt of gratitude to many who have assisted me in this work, including Edward Becker, Barry

Friedman, Geoffrey Hazard, Edward Ludwig, John Lungstrum, Nathaniel Persily, Chris Sanchirico, David

Shapiro, Catherine Struve, Stephen Subrin, and Stephen Wasby, who commented on a draft; Peter McCabe,

Steven Schlesinger, and their colleagues in the Administrative Office of the U.S. Courts, who very generously

met with me to explain the AO’s data-collection activities and its plans for the future and answered many

subsequent questions, and Margo Schlanger and Theodore Eisenberg, who also helped me to navigate the

AO’s data. Gregory Cooper, Penn Law Class of 2007, provided excellent assistance in preparing the manu-

script for publication.

Journal of Empirical Legal Studies

Volume 1, Issue 3, 571–590, November 2004

causes or consequences of the vanishing trials phenomenon in federal civil

cases are premature, suggesting in particular reasons to be wary of empha-

sis on “institutional factors” such as the discretionary power of first-instance

judges and the ideology of managerial judging.

I. INTRODUCTION

Americans love an empirical vacuum, and law professors are no exception. Whether

the goal is to sell a tort-reform package to a state legislature, a new or amended pro-

cedural rule or statute to the rulemakers or to Congress, a novel interpretation of

an old statute to the academy or the legal profession in general, or a model of human

behavior to the world, the task is easier if the enterprise is not encumbered by messy

facts: facts that do not support the proponent’s arguments, premises, or theory, facts

that may not provide support, or facts the meaning of which is ambiguous. Even

when unambiguous facts are available that cut against the view of reality advanced,

explicitly or implicitly, there are numerous strategies that can be, and are regularly,

deployed to neutralize their probative value. One such strategy is simply not to look

for any contrary evidence; another is to ignore it; a third is to attempt to overwhelm

it by relying on supporting data, whatever their quality, that may have greater reso-

nance with the intended audience, and a fourth (this is not an exhaustive list) is to

appeal to the need for human progress in a state of factual uncertainty. Different

strategies are likely to appeal to different actors, who come to the task with different

educational and professional backgrounds, freighted with different utility functions,

norms of argument, and, ultimately, approaches to the acquisition and dissemina-

tion of knowledge.

The reader will have noted that the preceding paragraph may be an example

of one or more of the phenomena it describes, since it offers no evidence for what

are, after all, empirical assertions. Of course, I have evidence of a sort for the propo-

sitions advanced there, but it is hardly systematic evidence, and I have not searched

to see whether such evidence is available. If pressed (and fully aware that many actors

do not fit within my generalizations), I would probably invoke some variant of the

first and fourth strategies, appealing both to the shortness of life and to my readers’

personal experience of instances where those assertions have proved accurate. I

venture out on this limb1in order to suggest both the enormous promise of the enter-

prise in which those who gathered for the Symposium on the Vanishing Trial are

engaged and the formidable obstacles that are likely to confront an attempt to trans-

572 The Limits of Data and Inference

1See Frank Cross, Michael Heise & Gregory C. Sisk, Above the Rules: A Response to Epstein and King, 69 U.

Chi. L. Rev. 135 (2002).