Reuniting 'is' and 'ought' in empirical legal scholarship.

AuthorFischman, Joshua B.
PositionIntroduction through III. Reversal Rates, p. 117-146

INTRODUCTION I. RELATING THE MEASURABLE TO THE GOOD A. Normative Metrics B. Medical Research C. Economics D. Empirical Legal Scholarship II. JUDICIAL CITATION COUNTS III. REVERSAL RATES IV. MEASURING THE RULE OF LAW: STUDIES OF INTERJUDGE DISPARITY A. The Normative Implications of Disparity B. Consistency, Predictability, and Comparative Justice C. Determinacy and Correctness D. Conclusion V. BRIDGING THE GAP BETWEEN 'IS' AND 'OUGHT' A. Prioritizing Normative Goals B. Rethinking Empirical Legal Methodology C. Accomodating Subjective Phenomena D. Emphasizing Generalizable Results CONCLUSION INTRODUCTION

A century ago, Roscoe Pound set forth his agenda for a "sociological jurisprudence" (1) that would study the "actual social effects of legal institutions and legal doctrines." (2) Pound sought to use empirical social science to advance normative goals: he "regard[ed] law as a social institution which may be improved by intelligent human effort" and proposed that social science could "discover the best means of furthering and directing such effort." (3) Two decades later, Karl Llewellyn issued his call for a "realistic jurisprudence" that would use empirical social science to study the determinants and consequences of judicial decisions. (4) Llewellyn was also motivated by normative ends, believing that in order to investigate whether the "law does what it ought," one must "first answer what it is doing now." (5) Pound and Llewellyn sparred over their respective visions, (6) but it is important to remember that they shared a common aim: to use empirical social science to improve the law.

The early legal empiricists were mindful of the challenges of connecting positive and normative approaches to legal scholarship. In his exchange with Pound, Llewellyn famously called for a "temporary divorce of Is and Ought." (7) He believed that the separation of 'is' and 'ought' was necessary for scientific credibility, but that it must be temporary in order to serve the goals of legal reform. (8) In the years that followed, however, legal empiricists struggled to balance the competing demands of social science and legal reform. (9) Some failed to separate 'is' and 'ought,' allowing their normative commitments to

influence their factual findings. (10) Others failed to reunite 'is' and 'ought,' producing "a mindless amassing of statistics without reference to any guiding theory whatsoever." (11) Years later, a disillusioned Llewellyn mocked his fellow realists for their pointless empirical projects. (12) He wrote: "I read all the results, but I never dug out what most of the counting was good for." (13)

The early legal empiricists had worthy ambitions, but their accomplishments were meager. (14) There were many reasons for their failure, (15) but prominent among them was their inability to develop any kind of theoretical framework for making their empirical findings relevant to normative legal scholarship. (16) Today, empirical legal scholarship is flourishing again, (17) and contemporary empiricists are far more sophisticated than their predecessors. Many law professors now have advanced social science training (18) and employ sophisticated methodologies from other disciplines to analyze and interpret data. Like the early empiricists, however, they are still struggling to balance the methodological imperatives of social science with the desire for legal reform. Often, the quest for scientific credibility leads contemporary empiricists to lose sight of the normative goals of legal scholarship. Some empirical studies make efforts to relate their findings to normative questions about law, and some even offer policy prescriptions, but such studies rarely explain how they derive an 'ought' from an 'is.' Even a cursory examination of the premises underlying such claims often reveals them to be untenable.

Empirical research projects need not generate immediate prescriptions, but even positive legal research should address topics that have some importance for legal scholarship. Because the law is a normative practice and exists to serve social purposes, determining what is important in legal scholarship requires some reference to the normative goals of law. (19) Thus, any empirical research that purports to be relevant to legal scholarship requires some framework for connecting 'is' and 'ought.'

As Barry Friedman has prominently argued, empirical legal scholars should "ask, at the outset of every project, why we ... might care about what is being studied." (20) Yet it is not enough to admonish legal empiricists to pay more attention to normative implications. In many settings, there are complex relationships between the phenomena that are readily measured and the values that can justify legal reform. Intuition alone cannot suffice to relate observable data to normative claims; legal scholarship needs conceptual frameworks and empirical methods that can bridge the gap between 'is' and 'ought.' Developing such frameworks will require a sustained agenda that integrates empirical methodology with legal theory.

Part I of this Article begins by considering how other disciplines have developed methods for relating quantitative empirical findings to normative claims. Typically, this is accomplished by formulating a normative metric that quantifies the goodness of the results. Using medicine and economics as examples, Part I shows how scholars in these disciplines have developed frameworks and methods for connecting the positive and the normative.

Empirical legal scholars, by contrast, often seek normative relevance by examining measureable phenomena that have some intuitive but only vaguely specified connection to a normative goal. Many studies simply conflate the measureable with the good, justifying policy proposals on the basis of the measureable objects. Parts II-IV provide illustrations of this approach for three commonly discussed judicial statistics. Part II focuses on judicial citation counts, Part III examines reversal rates, and Part IV critiques measures of interjudge disparity. These statistics are often used in empirical legal scholarship to capture conceptions of good judicial decisionmaking, and all three have been used to justify bold policy proposals.

For example, scholars have argued that judicial citation counts should be used to determine a shortlist for Supreme Court nominations, (21) to assess the merits of judicial selection procedures, (22) to determine whether judges are overpaid, (23) and even to examine whether men or women make better judges. (24) Studies documenting interjudge disparities played a prominent role in the enactment of the United States Sentencing Guidelines (25) and have also been used to justify reforms in Social Security (26) and immigration adjudication. (27) Reversal rates have been cited in debates about whether to split the Ninth Circuit (28) and used to appraise reforms in immigration adjudication. (29) Because such measures lack intrinsic normative force, however, policy arguments based on these measures alone are untenable. These measures may well have some relevance to normative concerns, but the studies are seldom explicit about their normative goals, how the data relate to these goals, and what premises are needed to justify the conclusions.

Part V discusses ways that legal empiricists can bridge the gap between 'is' and 'ought.' Most fundamentally, legal empiricists need to prioritize normative questions; research should focus on what is important, not what is easily measureable. In addition, empiricists need to rethink some aspects of empirical legal methodology. The choice of methods should be driven by questions, not the other way around. Empiricists should not try to seek objective, assumption-free conclusions, but rather should indicate how findings can be combined with assumptions to generate meaningful conclusions. Finally, due to the nature of the questions that arise in legal scholarship and the limits of experimentation, legal scholars should pay more attention to how their findings can generalize to new settings.

  1. RELATING THE MEASURABLE TO THE GOOD

    Empirical research is inherently descriptive, yet legal scholarship is predominantly normative. (30) Bridging the gap between 'is' and 'ought' therefore requires some form of normative premise. When empirical legal scholars seek to relate their empirical findings to normative claims about the law or legal institutions, however, their claims often have vague, unstated foundations. There is frequently a striking contrast between the effort devoted to making credible statistical inferences and the lax attitude toward articulating premises that can connect empirical findings to normative claims about law.

    The challenge of relating empirical findings to normative claims is hardly unique to legal scholarship. Many professional disciplines and applied sciences--such as medicine, engineering, education, and environmental studies--harness scientific knowledge in the pursuit of social purposes. Although most empirical research in the social science disciplines is positive, the research questions of these disciplines are similarly motivated by normative ends.

    This Part discusses the use of normative metrics in disciplines other than law. In some settings, the relevant metrics are directly measureable, and the results are self-interpreting. This part then examines frameworks for connecting 'is' and 'ought' in medical research and in economics, which use more sophisticated theories and methods to relate empirical findings to normative goals. In contrast to law, scholars in these disciplines are explicit about how empirical findings are used to support normative claims.

    1. Normative Metrics

      In quantitative studies, a normative premise is typically formulated in terms of a metric that maps states of the world into levels of goodness. A function f would constitute an normative metric if f(A) > f(B) whenever state A is preferred...

To continue reading

Request your trial

VLEX uses login cookies to provide you with a better browsing experience. If you click on 'Accept' or continue browsing this site we consider that you accept our cookie policy. ACCEPT