Legal autopsies: assessing the performance of judges and lawyers through the window of leading contract cases.

• Author: Caplan, Gerald

1. INTRODUCTION

   The medical profession enjoys ah evaluative practice for elevating physician performance. It is the autopsy. Literally translated, autopsy means to see for oneself. The pathologist observes the condition of the deceased and compares it with the diagnosis and treatment. The profession treats as a given that errors occur during the course of treatment, a judgment supported by autopsy studies of the last several decades that expose a consistent error rate hovering at forty percent. (1) The autopsy has practical value: it identifies mistakes, assesses performance, and provides a feedback loop for correction and remediation across a wide swath of treatment. (2) The legal profession has nothing like it.

   Attorneys and judges perform at a low level of visibility. Assessment is possible but forbidding because studying the relevant documents--briefs, transcripts, and lower court records--is arduous and time-consuming. Data on the incidence of indisputable error, such as timely filing, proper choice of cause of action or remedy, citation of leading cases, and the like, is non-existent. Unlike the medical profession, competence is assumed and error deemed extraordinary.

   Judge Richard Posner stands virtually alone in calling for evaluation of judicial performance. "In dealing with the work of judges," he observed in 1990,

   we inevitably take much on faith. Appellate decision making in the American legal system is characterized by a high degree of uncertainty. This makes it difficult to assess a judicial decision without access, which often is itself difficult and time-consuming to obtain, to briefs and lower-court records, and without careful study of the precedents and the other sources of law at the time.... (3) Subsequently, Posner encouraged researchers to undertake studies of judicial decision making that would apply autopsy-like objective standards for evaluating performance.

   The most illuminating kind of critical study would compare the judge's opinion in some notable case with the opinion of the lower-court judge, the record of the case, and the lawyers' briefs and oral arguments, along with any internal court memoranda written by the judge, his colleagues, or his or their law clerks. The aim would be to determine the accuracy and completeness of the judge's opinion; whether it was scrupulous in its use of precedent; the value it added to the briefs.... A series of critical judicial studies would yield insights into the methods as well as the quality of the judge. (4) Judge Posner's call for judicial studies can be extended to attorney performance. A similar set of evaluative studies could focus on client representation--the extent to which error characterizes certain aspects of law practice and is subject to remediation.

   Such studies of judicial decision making and attorney practice open a door to a novel body of research, one that more accurately describes civil legal process and stimulates self-examination. A well-designed case study may produce representative findings that impact both practice and legal education, which tends to distance itself from law in action. Perhaps researchers cannot assess performance by legal professionals with the same certitude as pathologists performing an autopsy. Yet the analogy is apt. The legal profession has well-understood performance standards and specifications.

   Evaluative studies of the order suggested by Judge Posner do not presently exist. (5) There are, however, pockets here and there in the literature that can be culled for data on attorney and judicial performance. One small but fertile database contains the studies of leading contract cases. Contract case studies are distinctive in that they are the only cluster of case studies, other than torts, that make use of the trial record; (6) and, as noted above, assessment is not possible without access to the record. As a data trove, these studies provide the largest window into how lawyers shape disputes and judges decide cases. In contrast to the factually compressed, written record, the case study is something of a tell-all. It more fully captures what happened and, importantly, positions the reader to make an independent judgment. (7) Although each case study is by definition unique, as a collective, they offer a glimpse of more general propositions that might be uncovered if autopsy-like studies were routinely undertaken.

   The contract case studies that examine performance issues are remarkably revelatory. They evidence egregious attorney errors, judicial slanting and misstating of the record, misunderstanding of the dispute by counsel and judges, instances where one or both parties withheld relevant facts from the court, and cases decided on points not argued by the parties. Every study is suggestive of more general problems and most studies exhibit more than one performance deficiency; attorney error, for example, shows up in most studies.

   Although the case study enjoys a respectable lineage in the social sciences, only in the last several decades has it captured the imagination of legal scholars. In origin, it was the domain of legal historians. Professor A. W. Brian Simpson compares his approach to that of an archeological dig in which the researcher seeks to "make sense [out] of the past." (8) The dig does not begin with a theoretical predicate. One digs and keeps digging for whatever may be relevant to understanding the dispute. Historical setting--context, place, and time--is central. This approach stands in sharp contrast to the social scientists whose studies start with a research design, are rooted and confined by a defined methodology, and are typically geared to testing hypotheses and developing a general theory.

   Current interest by legal scholars is often traced to Richard Danzig. Danzig conceptualized the case study quite differently than the legal historians. He imagined it as a vehicle for understanding the capability of the legal system to achieve its goals. Beginning with his essay on Hadley v. Baxendale (9) in 1975 and followed a few years later by an engaging anthology of well-known contract case studies, Danzig focused on legal process, rather than doctrine. (10) His work was well received and spawned the publication of other case studies. The new-found academic respect for the case study was partially a reaction to the opaqueness of the factually compressed appellate opinion. Redaction, though necessary and useful in that it tempers judicial biases, idiosyncrasies, and prejudices by channeling analysis to the elements of the rule, can mislead. Redaction is not only a synonym for edited, it also has a secondary meaning, "to select or adapt (as by obscuring or removing sensitive information) for publication or release," (11) and no doubt it is this meaning that some case study authors had in mind. (12) The appellate court opinion can blur, ignore, or dismiss relevant facts with little fear of detection. (13) Case studies provide a beam of light by illuminating what is lost through redaction. They afford a "better understanding of what 'really happened,' as well as uncertainties about what the facts 'really were' ... by a fuller telling of the story based on an examination of the trial record, the briefs, external accounts of the event, and more." (14) On occasion, they suggest a different outcome than that of the court.

   Danzig's anthology is organized around the concept of capability problems in legal process generally. He describes capability problems as systemic features, "the frictions, the ruts and the biases" that "impede and distort efforts to further preferred values through a legal system." (15) The values that Danzig seems to have had in mind relate to whether the legal process is performing according to prescribed rules and standards of the profession. These tend to be uncontroversial. Did, for example, the plaintiffs attorney sue the proper party, choose the right cause of action, and compute damages according to formula? Capability problems are not doctrinal. They are indifferent as to whether one rule is better than another in terms of fairness or efficiency or some other yardstick. The insight that Danzig brings to the fore "is the chaotic nature of the adversary process and the number of irrelevant factors that can limit attempts to advance the underlying goals (whatever they may be) of the legal system." (16)

   This is a novel focus, one that has been justly called seminal. It has received much praise, but little emulation. Part of the difficulty in building upon Danzig's work stems from the breadth and lack of clarity as to what is a capability problem. The definition is both amorphous and unbounded, running the gamut of problems "arising before, during and after trial." (17) Presumably, capability problems include such different events as death of a witness, juror misconduct, excessive judicial workloads, and the host of maladjustments that scarce resources and underfunding inject. But, however defined, the concept is of limited value because it gives no guidance as the relative importance, frequency, amenability to correction, and the like of specific problems. (18)

   Danzig, like the legal historians, starts not with a research design, but with a case, and searches somewhat non-specifically for the capability problems that might surface. No explicit working hypotheses focus the research effort. The studies follow no protocol, do not seek to test specific propositions or corroborate findings of another study; nor do they build on prior studies for findings or methodology. Each is limited to a place and time--to what happened in a unique context. The authors generally have no interest either in whether their findings are representative of some particular feature or characteristic of legal process or in abstracting more general propositions from their work. In short, they follow the historical, archeological dig model exemplified by the work of Professor...