The illusion of devil's advocacy: how the Justices of the Supreme Court foreshadow their decisions during oral argument.

• Author: Shullman, Sarah Levien

INTRODUCTION

The common perception about oral arguments in the United States Supreme Court is that they are colorful, entertaining, and for the lawyer who happens to be arguing at that moment, overwhelmingly nerve-racking. (1) But many appellate practitioners question whether oral arguments are at all useful, (2) and some explicitly argue that the Justices' decisions are preordained. (3) They believe that oral argument today is a mere formality, designed at a minimum to help the Court write a better opinion, although it can sometimes be used to "clarify facts and to test the vulnerability of tentative theories and approaches." (4)

But whether oral arguments are useful in the sense of being able to sway one or more of the Justices' votes, most practicing Supreme Court advocates agree on one proposition: No matter how well or poorly the argument goes, one simply cannot tell from their questions how the Justices are going to vote. (5) This article challenges that view, and suggests that oral arguments have more predictive value--and are thus more useful--than most people think.

My research indicates that by keeping track of the number of questions each Justice asks, and by evaluating the relative content of those questions, one can actually predict before the argument is over which way each Justice will vote. (6) This article also challenges the theory that the Justices use their questions to pull out the strongest and weakest points of each side equally, by showing that in the arguments I observed, they played devil's advocate much more often toward the parties with whom they disagreed than they did toward the parties they supported.

Part One of this article establishes the methodology I used to analyze the Justices' questions--specifically their content, tone, and number. Part Two summarizes my findings about oral arguments in the Supreme Court as a whole and suggests that predicting the outcome in a particular case may actually be possible. Part Three analyzes the questioning style and tendencies of each Justice. Finally, Part Four reports my predictions in three then-undecided cases, and, as a means of testing my theory, I compare my predictions to the actual outcomes.

1. METHODOLOGY

   I attended ten oral arguments at the Supreme Court during the October 2002 Term. Using the methodology described below, I tracked all of the questions asked from the bench. After the first seven of those ten cases were decided, I compared the content and tone of the Justices' questions in those cases to their decisions in each. This analysis allowed me to develop a theory about the predictive value of oral argument, which I then tested by predicting the outcomes in the three cases that had not yet been decided, and comparing my predictions to the actual results. (7)

   1. Tracking the Questions

      During oral argument, I recorded every question and noted which Justice asked it. (8) Next, I assigned a score to each question based on its content, using a scale of one (the most helpful questions) to five (the most hostile). For example, a one was assigned if the Justice asked a question that was designed to elicit the lawyer's best argument. Thus, helpful questions that began with: "Aren't you really trying to say that ...," if the lawyer really was trying to say that, would be given a one or a two. A five, on the other hand, was assigned to very hostile or argumentative questions. For example, a statement that began with: "I just don't see how your argument could possibly be correct," would receive a five. A three was assigned to completely neutral questions, or questions meant only to clarify a particular fact or minor point.

      I also tallied the number of questions each Justice asked, dividing them into categories: (1) questions asked in total; (2) questions asked per case; (3) questions asked of the party with which the Justice sided in the final decision; and (4) questions asked of the party against which the Justice sided in the final decision. To clarify, my observations show, for example, that many Justices asked far more questions of the respondent in cases in which they ultimately decided against the respondent's position.

      In addition, I made notations about the Justices' tones of voice, when they made jokes, how often they used hypotheticals, and any other potentially relevant or interesting observations that occurred to me during the oral arguments. (These notes were not given scores, but I referred to them when preparing Part III of this article, in which I analyze the styles of the individual Justices.)

   2. Analyzing the Questions

      After seven of the cases were decided, I entered the questions from all of the arguments and their corresponding scores into a spreadsheet. I tallied the numbers and calculated averages for categories such as how many questions each Justice asked per case; which Justice asked the most questions overall; how hostile or helpful each Justice's questions were on average; and how many questions were asked of one party when the Justice ultimately decided for (or against) that party, and whether those questions were hostile or helpful. I also compared the authors of the seven decided opinions to the Justices who asked the most and fewest (and most hostile and helpful) questions in those cases, to see if these factors seemed to play any role in the opinion assignments. (9) Finally, I searched for patterns in the data, essentially exploring the predictive value of the Justices' questions in hindsight. I then applied the theory I derived from the results of that analysis to predict the outcomes in the three remaining cases.

2. BRIEF SUMMARY OF CASES OBSERVED

   1. The Seven Decided Cases

      In Yellow Transportation, Inc. v. Michigan, (10) the issue was whether, under the Intermodal Surface Transportation Efficiency Act of 1991, (11) only a state's generic fee is relevant to determining the fee that it should have collected or charged as of November 15, 1991. (12) The Court reversed and remanded, holding that states may not modify a reciprocity agreement to alter any fee charged or collected as of that date. (13) The Court held that because the ICC's interpretation of the fee-cap provision was a permissible reading of the statute and reasonably resolved any ambiguities, it should have received deference under Chevron, (14) and the Michigan Supreme Court erred in declining to enforce the ICC's interpretation. (15)

      In Federal Communications Commission v. NextWave Personal Communications, Inc., (16) the Court considered whether section 525 of the Bankruptcy Code (17) prohibits the FCC from revoking licenses held by a debtor upon the debtor's failure to make timely payments for their purchase. The Court held that the FCC is prohibited under the express language of section 525 from revoking licenses, finding that whether the FCC had a valid regulatory motive for attempting to revoke them is irrelevant. (18)

      Barnhart v. Peabody Coal Co. (19) posed the question of whether the Commissioner of Social Security's failure as of October 1, 1993, to assign responsibility for eligible retired coal miners to the signatory operators that employed them voids the miners' benefits. The Court held that despite their untimeliness, initial assignments made after October 1, 1993, are valid. (20)

      In a more prominent case, Eldred v. Ashcroft, (21) the Court considered whether the twenty-year term extension set forth in the Copyright Term Extension Act of 1998 (22) violates either the Copyright Clause or the First Amendment. (23) In a much-anticipated decision, the Court upheld the CTEA. (24)

      The issue in United States v. Bean (25) was whether a federal district court has authority to exempt a convicted felon from the blanket prohibition against possessing firearms, (26) when annual appropriations limitations from Congress prevent the Bureau of Alcohol, Tobacco and Firearms from processing applications for such exemptions. The Court held that absent an actual denial by ATF of a felon's petition, judicial review is precluded under 18 U.S.C. [section] 925(c). (27)

      In Miller-El v. Cockrell, (28) the issue was whether the Court of Appeals erred in evaluating the petitioner's claim under Batson v. Kentucky, (29) and denying him a certificate of appealability (COA). In an eight-to-one opinion sharply criticizing the lower federal and Texas courts, the Court held that deference by federal judges to state-court decisions does not by definition preclude relief, and that the Fifth Circuit erred in collapsing review of the petitioner's COA request into an analysis of the merits of the case. (30) A prisoner seeking a COA need only demonstrate "a substantial showing of the denial of a constitutional right." (31)

      Finally, in Moseley v. V Secret Catalogue (32) the Court considered whether a party seeking an injunction under the Federal Trademark Dilution Act of 1995 must prove that the defendant's use of a similar mark caused it economic harm. The Court unanimously decided for the petitioner, holding that the Act requires proof of actual dilution, and that there was insufficient evidence in this case to support summary judgment for the respondent. (33)

   2. The Three Test Cases (34)

      Both Ewing v. California (35) and Lockyer v. Andrade (36) addressed the constitutionality of California's three-strikes law, and whether imposing a twenty-five-years-to-life prison term for a third-strike conviction violates the Eighth Amendment's prohibition against cruel and unusual punishment when the third strike is petty theft. The Supreme Court upheld the three-strikes law in two five-to-four opinions. (37)

      In Norfolk & Western Railway Company v. Ayers, (38) the petitioner alleged that it was error for the court below to (1) award emotional distress damages for fear of cancer under the Federal Employers' Liability Act (39) to retirees who were suffering from asbestosis, but who presented no evidence of additional physical symptoms that resulted from their fear of...