A quantitative analysis of writing style on the U.S. Supreme Court.

AuthorCarlson, Keith
PositionIntroduction through II. Measuring Style, p. 1461-1486 - Author abstract

TABLE OF CONTENTS INTRODUCTION I. COMPUTATIONAL ANALYSIS OF LEGAL TEXTS A. Content Analysis of Judicial Opinions B. Non-Content Analysis: Why Style? II. MEASURING STYLE A. Data B. Preliminary Analyses 1. Productivity 2. Friendliness 3. Defensiveness C. The Stylistic Fingerprint and Similarity Scores III. TIME TRENDS IN JUDICIAL STYLE IV. POTENTIAL MECHANISMS A. Prior Decisions B. Partisan Affiliation C. Substantive Factors V. CLERK INFLUENCE A. Previous Studies B. Comparing Inter-Year Variability CONCLUSION INTRODUCTION

The written word is the medium through which the law travels: courts, agencies, and legislatures create law by producing text. In recent years, these legal texts have increasingly become available to the public in digital form. Together with advances in processing power, data storage, machine learning, and computational text analysis, the digitization of the law has opened a new frontier in empirical legal scholarship. A number of researchers have eagerly crossed over into this unexplored territory in search of new insights, methods, and questions.

This Article contributes to this exciting new enterprise by undertaking the first general quantitative investigation of writing style on the US Supreme Court. While judicial writing style often serves as fodder for commentary, it has rarely been subject to systematic study. (1) Systematic qualitative analysis is made difficult by the sheer bulk of the corpus, which prevents a human reader from digesting any more than a tiny sample. Perhaps for this reason, qualitative analysis of style tends to focus on the gems in judicial writing, examining the prominent writings of prominent Justices and neglecting the mine-run of workaday opinions. (2) Scholars have only relatively recently combined accessible digital versions of the corpus of judicial writing with the tools offered by computational text analysis to undertake quantitative analysis of style. (3)

Prior attempts to analyze judicial writing style in a quantitatve way have typically been based on relatively small datasets, and are limited to a small number of specific stylistic features. (4) This Article relies on a corpus of all cases in the US Supreme Court in the period from 1792 to 2008, compiled from publically available raw textual data that has been augmented with identifying information concerning the year, author, and opinion type. (5) In addition to examining specific stylistic features culled from the prior literature on judicial writing style, we deploy a general stylistic measure that serves as a proxy for what Judge Posner refers to as "style as signature." (6) This general style proxy is first used to examine the gradual change in writing style over time, and then to investigate potential hypotheses about sources of stylistic variation over time. Perhaps most controversially, we find that the institution of the modem clerkship appears to have had an important effect on judicial writing style on the Court, both in the consistency of writing style in individual chambers and in the consistency of writing style of the Court as an institution.

Our primary analysis relies on a commonly used measure of writing style based on the frequency of use of content-free words (also called "function words"). This measure provides a "useful stylistic fingerprint" and was used for a large-scale study of literary style executed by Hughes, Foti, Krakauer, and Rockmore. (7) This stylistic approach has its roots in statistical methods to address the problem of author attribution. (8) As will be discussed more thoroughly, our stylistic fingerprint measure allows for analysis of the similarity between texts or a group of texts, including texts that are grouped by time and by author. In our analysis, we use the stylistic fingerprint as a means of developing descriptive statistics and as the basis for testing a number of hypotheses concerning the evolution of judicial writing style in the Supreme Court.

We first address the general relationship between style and time. The starting place for this analysis is the intuitive hypothesis that there is a "style of a time" in the Court. (9) Stated somewhat more formally, the hypothesis is that as the distance in time between judicial writings increases, there is a lower likelihood that they will be stylistically similar. Our analysis finds that stylistic similarity decreases with distance in time, as expected.

We also examine potential mechanisms that could drive this robust temporal trend. We examine the possibility that the writing style of particularly influential Justices propagates over time, so that the most read and cited Justices tend to project style forward. Perhaps surprisingly, we do not generally find that being widely cited increases the stylistic similarity between a past Justice and members of the current Court. (10) We also examine the potential for party affiliation to have played a role in stylistic evolution. While initial analysis reveals some differences between Democratic-appointed and Republican-appointed Justices, these differences appear to be the result of the temporal trend (alongside the changing partisan balance on the Court over time) rather than the cause. We then examine whether substantive features of opinions influence their style. We find that there are robust stylistic differences between majority opinions and dissents, even when comparing the writings of the same Justice. The growth of dissents, and dissent-like writing styles, may account for some of the drift in writing style on the Court over time.

We finally investigate the influence that the modem institution of the judicial clerkship has on writing style on the Court. For each Justice, we define a measure of consistency as the similarity between a Justice's writings in one year and in all other years. (11) We then test the hypothesis that the number of law clerks that a Justice employs is negatively associated with intra-Justice stylistic consistency. Overall, we find evocative evidence that the substantive role that clerks now play on the Court has led to decreasing inter-year intra-Justice stylistic consistency, while leading to increasing intra-year institutional stylistic consistency on the Court.

The methodology and findings presented here are meant to serve as a starting place for future quantitative text-based analysis of the law and legal writing style. In particular, this Article serves as an invitation for researchers to engage in a broader examination of the causal factors that affect judicial writing style; of stylistic trends in other governmental institutions, including Congress and agencies; and of the dynamic relationship between writing style in political institutions and the broader culture. As discourse continues its move into the digital realm--as new content is created and earlier content is digitized--it opens the door for a rich inquiry into the relationship between literature, law, and mass media, and the relative influence (stylistic and otherwise) of each of these domains on the others.

This Article proceeds in five Parts. Part I explains how our research contributes to existing literature on judicial writing style and the growing literature using computational text analysis as the basis for legal...

To continue reading

Request your trial

VLEX uses login cookies to provide you with a better browsing experience. If you click on 'Accept' or continue browsing this site we consider that you accept our cookie policy. ACCEPT