CITATION STICKINESS.

• Author: Bennardo, Kevin

CITATION STICKINESS

Table of Contents I. Why [Read a] Study [About] Citation Stickiness? 106 A. Review of the Citation-Study Literature 107 1. Study of Citation Stickiness in the State Courts 109 2. Study of Citation Stickiness in the Federal Courts 111 B. The Utility of Studying Citation Stickiness 115 II. Our Methodology 118 III. Our Results 122 A. The Big Picture: Stickiness Percentages and Number of Citations 123 B. Stickiness by Circuit 126 C. Stickiness by Case Characteristics 129 D. Stickiness by Type of Brief 132 E. Stickiness by Winning and Losing Brief 133 F. Stickiness by Judicial Characteristics 135 1. Stickiness by Political Affiliation of Appointing President 137 2. Stickiness by Law School Attended 141 3. Stickiness by Whether Judge Sat by Designation 143 4. Stickiness by Judicial Experience 144 IV. What It All Might Mean 150 V. Conclusion 157 This article is an empirical study of what we call citation stickiness. A citation is sticky if it appears in one of the parties' briefs and then again in the court's opinion. If it helps, picture the parties tossing citations in the court's direction. Some of those citations stick and some of them don't. The ones that don't stick--that don't appear in the court's opinion--are unsticky. That covers the citations in the parties' briefs--they are either sticky or unsticky. As for the citations in a court's opinion, they are either sticky--meaning that the same source was cited in at least one brief--or they are endogenous--meaning that they appeared for the first time in the opinion. Endogenous citations spring from the court itself.

Consider a recent Tenth Circuit opinion in which the court cited thirty-three distinct cases. (1) Thirty-one of those cases were not mentioned in any of the parties' briefs. The opening brief cited twenty-nine cases, (2) the response brief cited eighteen, (3) and the reply brief cited five. (4) Out of all of the cases cited by the parties, however, the Tenth Circuit cited one from the opening brief, one from the response brief, and thirty-one that were not mentioned in any brief. On the other end of the spectrum, consider a recent opinion from the Seventh Circuit. (5) The court's opinion contains eleven unique case citations. Every single one of those eleven cases had been cited in one or more of the parties' briefs. (6)

For the Tenth Circuit case, we'd say that the cited cases in the opinion were 6% sticky and 94% endogenous. (7) We'd say that the cited cases in the opening brief were about 3% sticky and 97% unsticky. (8) For the Seventh Circuit case, the opinion contained 100% sticky case citations. (9) None of the cases cited in the opinion were endogenous. (10)

From those numbers alone, we cannot tell you whether the briefs in the Tenth Circuit case are better or worse than the briefs in the Seventh Circuit case. We cannot tell you whether the Tenth Circuit's decision is better or worse than the Seventh Circuit's decision. But what we can tell you is that there is wide variation in the percentage of cited cases in judicial opinions that originated in the parties' briefs. (11) The other thing we can tell you is that parties cite a lot of cases in their briefs that are never discussed in the resulting judicial opinions. Maybe you had some general sense of this already. Maybe not. But we bet you don't know the numbers. We do.

We traced the provenance of 7552 cited cases in 325 federal appellate opinions. We know how many of those case citations were borrowed from the parties' briefs and how many came from within the court itself. Read on to learn specifics, but here's a spoiler: most of the cases cited in the federal appellate opinions that we studied were not cited in either of the parties' briefs.

This result surprised us, as it surprised the participants in our online and conference-audience polls. (12) This finding is novel, as most citation-practice studies of judicial opinions do not trace the origins of the cited authorities. Studies that have tracked the communication of citations from the briefs to the resulting opinions have used smaller sample sizes and have been limited to only a few courts.

Additionally, we tracked the journeys of the 23,479 cases that the parties deemed worthy of citing in their briefs. We know how many of those cited cases later showed up in the resulting judicial opinions and how many did not.

Are there some characteristics that correlate with increased or decreased citation stickiness? Do some circuits tend to produce opinions with higher citation stickiness than others? Do some types of cases tend to result in higher citation stickiness? Do some judicial characteristics tend to correlate with higher citation stickiness? Fear not--we have sliced and diced the data in numerous ways.

And what does it all mean? That is a little less clear. Is it a problem that most of the cases cited in the opinions weren't mentioned by the parties? Yes, we'd say that these results indicate that something is amiss in our adversarial system. But where to point the finger? At attorneys for submitting shoddy briefs? At courts for disregarding the papers filed by the parties? Don't worry, dear reader, we devote an entire section to hypothesizing.

What follows are the results of our empirical study of citation stickiness in the federal courts of appeals. In Part I, we review the existing literature on citation studies and explain why studying citation stickiness is a worthwhile endeavor. In Part II, we lay out our research methodology. Part III reports our results, analyzing stickiness by various dimensions, such as case topic and certain characteristics of the authoring judges. Finally, Part IV hypothesizes what it all may mean and identifies some additional avenues for future research.

1. WHY [READ A] STUDY [ABOUT] CITATION STICKINESS?

   So why should we study citation stickiness? Or, more saliently at this point, why should you read our study about citation stickiness? Most critically, our study is novel. It fills a heretofore unfilled gap. While filling a gap may be a necessary reason to undertake a study, it is not itself a sufficient one. There are plenty of things that haven't been studied simply because they are not worth studying. (13) Aside from its novelty, citation stickiness is worth studying because it provides a window into judicial decisionmaking. Judges often lament the quality of attorneys' briefs. Attorneys often lament the quality of judges' decisions, especially when the opinions explaining those decisions veer away from the issues set forth in the briefs. Measuring citation stickiness will help uncover to what extent judges are conducting independent legal research. Answering that question seems foundational to determining whether judges are doing too much research, too little, or just the right amount.

   This Part will proceed with a summary of the citation studies to date and identify the precise gap that our study fills. It will then discuss the utility of studying citation stickiness.

   1. Review of the Citation-Study Literature

      While it would be nice to say that we were experts on the citation-study literature before this project began, that would not be the whole truth. (14) We had not read every citation study out there and thoughtfully noticed a gap in the citation stickiness department. Rather, as many researchers do, we started with the question and discovered the gap. We thought citation stickiness was interesting, but when we researched it, we found little data that answered the question of whether courts generally stick to the legal authorities cited by the parties.

      To be sure, there are plenty of citation-practice studies out there. (15) Many answer quite interesting questions. Given the laborious nature of the research, however, many of the studies are quite limited in scope. Many have small datasets or focus on a particular year or narrow band of years. (16) Many are limited to studying the courts of a particular state or territory. (17) On the federal side, these citation studies disproportionately focus on the United States Supreme Court. (18) Despite the quantity of existing citation studies, there have been numerous calls for expansion of this method of research. (19)

      Citation studies have largely focused exclusively on courts' opinions and ignored citation provenance. Studies have analyzed whether courts cite the same scholarship that academics do (20) and how precedents are transmitted from court to court. (21) But few have compared the sources cited in the parties' briefs to the sources cited in the resulting opinions, perhaps because this is a question that is likely more interesting to practitioners than to academics. (22) With a notable exception or two, the studies that have previously compared briefs' citations to opinions' citations have been extremely limited in scope. (23) The existing studies are summarized below.

      1. Study of Citation Stickiness in the State Courts

         The most robust citation stickiness study to date is Thomas Marvell's state-court study from over forty years ago. The court, however, was anonymous. All we know is that it was the "supreme court of a northern industrial state," referred to by Dr. Marvell as the "focal court" of his study. (24) The Marvell dataset comprised 112 cases argued during a one-year period ending in June 1972. (25) Comparing the attorneys' submissions to the opinions, Dr. Marvell found that "[a] little less than half the legal authorities cited in the majority and minority opinions in the 112 focal cases studied here were mentioned in the parties' briefs or oral arguments, and but one-sixth of the authorities mentioned by the attorneys were cited in the opinions." (26) As a sort of control study, Dr. Marvell also compared the citations in the published opinions and briefs of thirty civil cases from the United States Court of Appeals for the Sixth Circuit. (27) His results were fairly consistent with the focal court study...