PREDICTING PRECEDENT: A PSYCHOLINGUISTIC ARTIFICIAL INTELLIGENCE IN THE SUPREME COURT.

• Author: Gandall, K.

CONTENTS I. Predicting Precedent: A Psycholinguistic Artificial 220 Intelligence in the Supreme Court II. Literature Review 223 III. Institutionalist Models 223 IV. Psycholinguistic Models 225 V. Predictive Algorithms in Court Behavior 233 VI. Methodology 240 VII. Variables 242 VIII. Data Collection 244 IX. Analysis 244 X. Results 249 XI. Model Performance 250 XII. Case Example Predictions 252 XIII. Discussion 256 XIV. Limitations and Future Research 263 XV. Conclusion 269 I. Predicting Precedent: A Psycholinguistic Artificial Intelligence in the Supreme Court

"The object of our study, then, is prediction, the prediction of the incidence of the public force through the instrumentality of the courts." (1)

Since the proliferation of analytic methodologies and 'big data' in the 1980s, (2) multiple studies have been developed attempting to comprehensively explain Supreme Court behavior. (3) These studies have ranged from correlating ideological measurements and voting patterns to concrete assertions by Supreme Court Justices that outcomes are almost wholly dependent on the legal context--that is, "the ordinary sense of words, history, tradition, precedents, purposes... and consequences related to those purposes." (4) Furthermore, the increasingly large social impact of Supreme Court cases has led scholars, policymakers, and political pundits alike to invest in predicting case outcomes, (5) even so that entire betting markets have emerged solely focused on predicting court outcomes. (6) As some commentators have noted, the 2022 term is likely to be one of the most impactful in recent history, with an overwhelmingly conservative court granting certiorari on a number of issues ranging from the constitutionality of affirmative action, religious freedom, free speech, the right to bear arms, due process, equal protection, and administrative deference. (7) Criticism during this term has also been emotionally explosive, with some arguing the Court has been "unhinged" and "partisan." (8) These criticisms are not particularly new. Almost immediately before Thurgood Marshall resigned, he wrote a scathing dissent in Payne v. Tennessee: "Neither the law nor the facts... underwent any change in the last four years. Only the personnel of this court did." (9) Scalia, sitting on the opposite ideological side as Marshall, would later similarly write in his dissent in Obergefell v. Hodges that the Supreme Court had "descended from the disciplined legal reasoning of John Marshall and Joseph Story to the mystical aphorisms of the fortune cookie." (10) To Marshall and Scalia, it was the individual preferences of justices sitting on the court, not the framing of arguments, that determined how the Court ruled.

While changes to the Court and its precedent are not a novel feature of the law, (11) there is evidence that suggests public perception of the Court's legitimacy is on the decline. Recent polling indicates that the public is beginning to identify the Court with broader prevailing political narratives ranging from abortion to immigration. (12) Yet, other polls suggest that, because Congressional and Executive trust is so low, the public is nevertheless more willing to trust the Supreme Court to offer final decisions on contentious areas of both policy and law. (13)

This study aims to illustrate a more universalist picture of the Supreme Court than Scalia or Marshall would propose. In short, building off Justice Barrett's quip that the Supreme Court is "not comprised of a bunch of partisan hacks," (14) we propose that legal behavior can be understood by the innate psychological framing and preferences individuals hold. To do so, we engineered a deep neural network (DNN) called SCOTUS_AI to predict Supreme Court outcomes through a sentiment analysis of government briefs. We aim to demonstrate that artificial intelligence can outperform traditional methodologies by identifying patterns in human language not readily available to even experienced scholars. Finally, this study will use SCOTUS_AI to cast predictions on several upcoming Supreme Court cases.

2. Literature Review

   Of the numerous theses of Supreme Court behavior, two tend to dominate academic studies: institutionalism and psychoanalysis. (15) These models, while not necessarily incompatible, represent a larger, ever-evolving dichotomy accelerated by new technologies. (16) The former reasserts a traditional legal framework focused on predicting court behavior through a contextual analysis of case precedent, rules, and substantive arguments. The latter asserts that human behavior can be represented in a series of advanced mathematical functions, predictable given sets of environmental externalities and psychological cues.

3. Institutionalist Models

   In his treatise on law, Holmes famously sets forth the mode of the institutionalist theory: "The means of the study are a body of reports, of treatises, and of statutes, in this country and in England, extending back for six hundred years, and now increasing annually by hundreds." (17) Along similar lines, Justice Breyer observes that the fundamental job of a judge is to "interpret or to apply the legal phrases that we find either in a statute or in the Constitution itself." (18) For Holmes and Breyer, the study of law is to reasonably apply rules through precedent and other internal, legal devices. (19) Of course, debates emerge challenging the appropriate vehicle to interpret these phrases: for the constitution specifically, conservatives tend to prefer originalism and textualism, (20) while liberals prefer seeing the Constitution as a "living document." (21) There are also exceptions to this dichotomy: "common good constitutionalism," for example, is triumphed by some conservative academics, (22) despite occasionally disputing originalism. (23)

   However, it is this basic assumption--that the legal rulings can be understood through a set of rules and precedents--that invariably binds otherwise opposed institutionalist views. (24) Such a theoretical understanding of the law also translates to similar methodological preferences, with most studies forecasting Supreme Court decisions focused on case characteristics (citations and precedence), expert opinion (the logical, legal coherence of an opinion), and institutional barriers (procedural stance). (25) Likewise, some institutional scholars focus on extralegal structural factors, ranging from economic impact, party identities, and other contextual causes (often socioeconomic). (26)

   These methodological approaches align with the claim by institutionalists that adherence to court precedence, stare decisis, is necessary to a functional society and therefore an encouraged, altogether occasionally not required, finding. (27) Even Justice Scalia, an originalist who was notable for opining against several major precedents including Roe v. Wade, consistently emphasized the importance of stare decisis: "Originalism, like any other theory of interpretation... must accommodate the doctrine of stare decisis; it cannot remake the world anew." (28)

   As a result, much effort has been focused on generalized academic opinions on the merits of legal arguments. Despite observations, such as by Ruger, that expert predictions are often either inferior or equal to simply predicting in favor of the petitioner, institutionalism nevertheless persists both for its ability to extract meaningful effects of rulings, and to understand the reasoning behind such rulings, even if such explanations are simply pretexts for larger sociological or psychological phenomena. (29) Simply put, institutionalists look at a given context, and suggest what would be a logical conclusion given the facts and rules.

3. Psycholinguistic Models

   Psychoanalysis provides a second mode of understanding the law, analyzing underlying, often subconscious, motivational incentives or preferences that can affect the perception of an individual case or policy. (30) Functionally, psychoanalysis suggests that human behavior is best understood by unconscious motivations, often created during a younger, "psychosocial" phase of childhood. (31) These unrecognized impulses, psychoanalysts argue, can reflect decision making capabilities in real world structures. (32)

   For the purposes of this paper, an important contrast between institutionalism and psychoanalysis is a distinction in methodology; institutionalists, as we noted, prefer easily articulable theories of law that reside in the legal vehicles of treaties and precedent. (33) Psychoanalysis, however, seeks to examine the underlying reason as why actors prefer vehicles of interpretation. (34) Psychoanalysis does not necessarily disregard, for example, historical precedence, but instead reinterprets it as a function of unconscious resistance to change. (35) Goldstein writes:

   there is in law... a rich residue which each generation preserves from the past, modifies for the present, and leaves for the future... The congruence of their concern for man, his mind, his behavior, and his environment may justify this assertion of mutual relevance. (36) Goldstein thus suggests that stare decisis, for example, is merely an incidental result of a deeper psychological incentive to maintain what is deemed as secure. (37) The fruit of psychoanalysis, Goldstein argues, is that it forces "into view conflicts between existing rules and preferred values which [decisionmakers] may not see or may not wish to acknowledge." (38) As early as 1881, before Freud's publications, even Oliver Wendell Holmes acknowledged the influence of emotions:

   The life of the law has not been logic: it has been experience. The felt necessities of the time, the prevalent moral and political theories, intuitions of public policy, avowed or unconscious, even the prejudices which judges share with their fellow-men, have had a good deal more to do than the syllogism in determining the rules by which men should be governed. (39) A...