"Ideology" or "situation sense"? An experimental investigation of motivated reasoning and professional judgment.

• Author: Kahan, Dan M.
• Position: Abstract into IV. Results A. Legal Reasoning 1. Summary Data, p. 349-394

This Article reports the results of a study on whether political predispositions influence judicial decisionmaking. The study was designed to overcome the two principal limitations on existing empirical studies that purport to find such an influence: the use of nonexperimental methods to assess the decisions of actual judges; and the failure to use actual judges in ideologically-biased-reasoning experiments. The study involved a sample of sitting judges (n = 253), who, like members of a general public sample (n = 800), were culturally polarized on climate change, marijuana legalization and other contested issues. When the study subjects were assigned to analyze statutory interpretation problems, however, only the responses of the general-public subjects and not those of the judges varied in patterns that reflected the subjects' cultural values. The responses of a sample of lawyers (n = 217) were also uninfluenced by their cultural values; the responses of a sample of law students (n = 284), in contrast, displayed a level of cultural bias only modestly less pronounced than that observed in the general-public sample. Among the competing hypotheses tested in the study, the results most supported the position that professional judgment imparted by legal training and experience confers resistance to identity-protective cognition--a dynamic associated with politically biased information processing generally--but only for decisions that involve legal reasoning. The scholarly and practical implications of the findings are discussed.

INTRODUCTION I. EXISTING EMPIRICAL STUDIES AND THEIR LIMITATIONS A. Observational Studies B. Experimental Studies II. INFORMATION PROCESSING, PATTERN RECOGNITION and Professional Judgment III. STUDY DESIGN AND HYPOTHESES A. Design 1. Overview 2. Sample 3. Cultural Worldview Measures 4. Statutory Interpretation Problems 5. Risk Perception Measures B. Hypotheses 1. Four Contenders 2. Law Students C. Analytic Method and Statistical Power IV. RESULTS A. Legal Reasoning 1. Summary Data 2. Multivariate Regression B. Risk Perceptions V. TAKING STOCK A. So Are Judges Political? B. What About Law Students? C. Motivated Reasoning, Professional Judgment, and Political Conflict D. The "Neutrality Communication Problem" CONCLUSION APPENDIX A. REGRESSION MODELS A. Statutory Interpretation Problems 1. Littering Problem 2. Disclosure Problem B. Risk Perceptions Appendix B. Study Instrument A. Legal Reasoning Problems 1. Littering 2. Disclosure B. Risk Perceptions C. Cultural Worldviews INTRODUCTION

Are judges politically motivated? Do they make decisions, at least in ideologically charged cases, on the basis of "policy preferences" or predispositions?

Most members of the American public think so. Opinion polls suggest that about three-quarters of Americans believe that judges--U.S. Supreme Court Justices and lower court jurists alike--base their decisions on their "personal political views." (1) The charge that judges are "legislating from the bench" is automatic after decisions involving culturally contested matters--from gay rights to gun control, affirmative action to stem cell research. (2) If a judge rules against an African-American plaintiff suing white officers for police brutality, or a female criminal defendant asserting the "battered woman defense," the decision betrays "bias," rank or implicit; (3) if the judge rules the other way, then he or she is castigated for engaging in identity politics. (4)

Experts also share the public's assessment. Using multivariate regression models, some purport to measure the quantity of variance in case outcomes explained by judges' ideologies. (5) Others find evidence for judicial partisanship in experiments that demonstrate the impact of subconscious ideological predispositions on members of the public, including law students. (6) The evidence, according to the experts, vindicates the public's impression that judges (despite their own protestations to the contrary) are just "junior varsity politicians." (7)

We disagree--not necessarily with the conclusion that judges are "politically biased," but with the premise that existing empirical evidence furnishes secure grounds for crediting it. Popular judgments obviously are not conclusive: because members of the public lack legal training (indeed, lack comprehension of even the most rudimentary elements of the legal system), (8) they necessarily are incapable of reliably assessing the validity of judicial decisions. Inferences from experiments involving members of the public--including law students--are suspect for exactly the same reason: because the subjects in such studies are members of the public, their vulnerability to unconscious ideological predispositions begs the question of whether the training and experience that judges possess immunizes them from such influences when they are engaged in legal reasoning. (9)

Observational studies that purport to correlate the decisions of real judges with those judges' "ideologies" have also been reasonably criticized for methodological problems. One is the obvious selection bias involved in studying only cases that are actually litigated. (10) Another is the failure of researchers to specify the measured outcome variable in a manner that distinguishes the illicit contribution of political sensibilities extrinsic to law from the licit contribution of political sensibilities intrinsic to conventional legal reasoning. (11) These methodological shortcomings, critics assert, raise doubts about the strength of the inferences that can be drawn from these studies.

The most satisfactory way to overcome these limitations, we believe, is through valid experiments performed on judges. (12) In this Article, we present the results of such an experiment. In this study, judges, lawyers, and law students were instructed to assess legal problems designed to trigger unconscious political bias in members of the general public.

The experimental results furnished evidence strongly at odds with the conclusion that judges are influenced by political predispositions when they engage in legal reasoning. Judges of diverse cultural outlooks--ones polarized on their views of the risks of marijuana legalization, climate change, and other contested issues--converged on results in cases that strongly divided comparably diverse members of the public. Culturally diverse lawyers also displayed a high degree of consensus in their legal determinations. Law students, in contrast, did not; in addressing the legal problems featured in the experiment, they polarized along the same lines that divided the legally untrained members of the public, although to a lesser extent.

These results strongly support the hypothesis that professional judgment can be expected to counteract "identity-protective cognition," the species of motivated reasoning known to generate political polarization over risks and myriad policy and legally consequential facts. (13) Legal training and experience, on this view, endows judges and lawyers with a specialized form of cognitive perception--what Karl Llewellyn called "situation sense" (14)--that reliably focuses their attention on the features of a case pertinent to its valid resolution. The results of our experiment support the conclusion that "situation sense" is sufficiently robust to fix judges' attention on such decision-relevant features of a case notwithstanding the tug of influences that might systematically focus the attention of the public on facts that are irrelevant--and indeed inimical--to impartial legal decisionmaking. Indeed, this dynamic creates a source of divergence between expert legal and non-expert lay assessments of law akin to the divergence between expert and lay assessments of risk. (15)

This form of professional judgment, however, does not furnish lawyers or judges with any special immunity to the reason-disturbing effects of identity-protective cognition outside of the domain of their own expertise. The domain-specificity of judges' (and lawyers') immunity to this form of motivated reasoning furnishes insight into a variety of more general questions, including why the capacity and disposition to engage in conscious, effortful information processing does not mitigate, but rather accentuates, the polarizing consequences of identity-protective cognition in members of the public on climate change, gun control, and other culturally charged issues.

In addition to describing the design and reporting the results of our experimental study, we also offer a normative assessment of the findings. The conclusion that judges can in fact be expected to be neutral decisionmakers in many politically charged cases might be considered welcome news.

But the results also support a conclusion that ought to be a matter of deep concern for the legal profession: our system of justice lacks reliable practices for communicating courts' neutral resolution of divisive matters. As a result of identity-protective reasoning, diverse members of the public can be expected to form highly polarized perceptions of facts and highly polarized judgments about the dictates of the law in cases that resonate with contested cultural sensibilities. By virtue of their shared "situation sense," judges of comparably diverse outlooks might readily converge on outcomes that reflect legal norms understood not just by judges but by citizens generally to supply the appropriate guidance for resolving such disputes. But because members of the public lack exactly that perceptive capacity, they (or a substantial proportion of them) will predictably understand the outcome of such cases to be rooted in partisan biases nonetheless. (17) Deciding cases neutrally from the point of view of the law and communicating the neutrality of case outcomes to members of the public who are not legally trained, in other words, are completely different things. Judges might be experts at the former. But the persistent...