A quantitative analysis of writing style on the U.S. Supreme Court.

• Author: Carlson, Keith
• Position: III. Time Trends in Judicial Style through Conclusion, p. 1486-1510

3. TIME TRENDS IN JUDICIAL STYLE

   This Part applies the methodology just described to examine how writing style on the Court changes over time. Specifically, we ask whether there is a "style of the time," in the sense that contemporaneous Justices tend to write more similarly than Justices who are temporally remote from one another. As will be clear from the analysis below, the answer to that question is "yes."

   To undertake our analysis of the relationship between temporal distance and writing style similarity, we first calculated feature vectors for all Justices and created similarity scores for every Justice-pair within the study period. Each Justice was also assigned a place in time, based on the mid-point of their term on the Court. (126)

   Our first analysis is a representation of similarity scores as a style "network" with Justices "linked" to each other based on stylistic similarity. In the terminology of network analysis, the Justices are "nodes" and a thresholding technique on the stylistic similarity is used to determine when "edges" (or links) are placed between the nodes. (127) Each Justice is a node in the network, and an edge was created between that Justice and the 5% of other Justices with the highest similarity scores in their set. (128) We then undertake a quantitative estimate of groups within the style network, using the methodology of spectral clustering analysis. (129) Boyd, Hoffman, Obradovic, and Ristovski describe a use of the spectral clustering methodology, which is a technique used to "classify and group" items within a dataset. (130) In essence, spectral clustering "cuts" a network into some defined number of groups (i.e., accomplishes a "clustering"), relative to the condition that similarity between members of the groups should be relatively high and the similarity between members of different groups relatively low. (131) A related (and often thorny) problem in spectral clustering is the determination of the number of clusters as based on the data. (132) We did not address that second problem--which is not necessary to our analysis--and instead set the number of clusters to be identified at seventeen, which is the number of Chief Justices that have served on the Court. That number is admittedly somewhat arbitrary, but it is sufficient for our purposes, which is generally to examine whether Justices' writing styles appear to cluster together on a temporal basis. The groups generated by the spectral clustering analysis are ordered by the median year of the Justices in the cluster, and the range of median years is presented alongside the group as well. Table 3 presents the results from the spectral clustering analysis. (133)

   The most striking observation from these analyses is the degree to which Justices are more stylistically similar to their contemporaries than to temporally distant Justices. This is especially the case in the modem era, with the Justices on the current Court quite isolated stylistically from Justices in earlier years, which can be seen quite clearly in the supplemental online figure. In general, the spectral clustering analysis displayed in Table 3 created groups that were time-based, with temporal ranges of a few decades, and some closer to a single decade. (134)

   To analyze more closely the relationship between time and stylistic similarity, we characterized every Justice by the median year of his or her term of service on the Court. For sitting Justices (including Justice Scalia), 2008 was used as the end of their tenure. We then calculated the distance in time for every pair of Justices, and related those distances to the similarity score for those Justices. The results are presented in Figure 5.

   An OLS regression generated an R-squared of 0.18 and a p-value of less than 0.01%, and a coefficient for temporal distance of 0.047. (135) These findings can be interpreted as indicating that, while there are sources of variation in the data other than time, there is also a strong trend of declining similarity in time, with a rate of decay in similarity score of roughly 4-5%. As Justices move farther apart in time, they become increasingly distinct in their writing styles.

   To examine the influence of time from a somewhat different angle, we next calculated feature vectors for all years and created similarity scores for every year pair within our study period. We then calculated average similarity scores based on temporal distance: one-year distant pairs were averaged into a single similarity score; two-year distant pairs were averaged into a second; and so on. The average similarity score for temporally matched pairs is represented in Figure 6.

   Overall, these results indicate a decline in the similarity of year feature vectors as they move farther apart in time, with a rate of decay in similarity of around 4-5%. (136) This analysis again provides strong evidence that style on the Court is not time independent, but instead changes over time. The writings of Justices working together in a given decade are far more stylistically similar to each other than they are to writings of Justices on a temporally remote Court.

4. POTENTIAL MECHANISMS

   The foregoing analysis raises an interesting question as to why stylistic similarity within judicial writing declines with temporal distance between Justices. There are a variety of potential mechanisms that could cause writing style to change with time. When examining the Court, it is perhaps most natural to look to external factors, including broader societal trends in writing style. Questions surrounding change of writing style on the Court, then, would necessarily implicate a larger set of questions concerning change of writing style outside the Court in various media, including literature, popular culture, and personal communication. (137) Those questions, while no doubt of interest, are outside the scope of this project.

   We restrict our analysis to internal factors that could help explain a change in style. In this Part, we examine three potential causal mechanisms. The first is influence by highly respected prior decisions. We do not find convincing evidence that more frequently cited prior decisions exert any particularly great influence on later style. We then examine changes in the Court's composition and do not find evidence that the partisan affiliation of Justices has an effect on style. Finally, we examine the potential influence of substance by comparing dissenting and majority opinions and find some evidence that writing style bears some relationship to opinion type.

   1. Prior Decisions

      The first mechanism that we examine is the possibility of a causal role played by influential past decisions. Just as past decisions generate legal standards and norms of judicial reasoning, they serve as the backdrop against which a Justice's writing style is perceived. While some innovation in writing style may be rewarded, Justices are likely to express some degree of conformity to prevailing conventions. Justices may also consciously model their writing style on prior Justices who they find to be particularly worthy of emulation, or may be subconsciously influenced by the decisions that they read.

      To test for the influence of prior Justices, we rely on the current Court as our baseline. To create the baseline, we used the writings of each of the currently sitting Justices in our dataset (Alito, Breyer, Ginsburg, Kennedy, Roberts, Scalia, and Thomas) to generate a single stylistic feature vector. (138) We then excluded from our analysis all of the Justices who served at the same time with any sitting Justice as a way to separate out any cross-influence between Justices and ensure that the causal relationship runs in the anticipated direction. For each remaining Justice, we constructed a feature vector for their writing, and calculated the KL divergence between that feature vector and the current Court baseline.

      For each Justice in the analysis, we then constructed a "ghost" vector made up of the texts produced by the Court in each of their years on the bench, excluding that Justice's writings. (139) We calculated the KL divergence between the ghost vectors and the current Court baseline vector. Finally, we subtracted the KL divergence for each Justice's ghost vector from their KL divergence to generate what we call "prediction scores": a difference less than zero indicated that a Justice's writing was more similar to the current Court's style than to the other writings of the Court in the years when that Justice was on the bench. Justices who perform well tend to "predict" the current style of the Court better than Justices who perform poorly (with lower numbers associated with better prediction). There were few Justices with prediction scores of less than zero, because each Justice typically authors only a small fraction of cases "first between also where who those part than him will could without whether must after before within should these only them when against same so one would their there has they other all made may if we us he under but been had his were no have are any its upon such at an with from on which this not or as for be it was by is a that in and to of the" in a given year, meaning that random sources of variation are much less likely to substantially influence the ghost vectors than an individual Justice's vector.

      We then compared the resulting prediction scores to a measure of "historical value" for each Justice, which was generated by Kosma in 1998 based on citation counts. (140) This variable is meant to capture the possibility that Justices who are widely cited exert greater stylistic pressure on subsequent Justices. We controlled for the relationship between a Justice's total production, in words, and our prediction scores. There are two potential mechanisms for this variable to affect the prediction scores. First, Justices that produce a great deal of text contribute more to the total body of the...