Disparate statistics.

• Author: Tobia, Kevin

ABSTRACT. Statistical evidence is crucial throughout disparate impact's three-stage analysis: during (1) the plaintiff's prima facie demonstration of a policy's disparate impact; (2) the defendant's job-related business necessity defense of the discriminatory policy; and (3) the plaintiff's demonstration of an alternative policy without the same discriminatory impact. The circuit courts arc split on a vital question about the "practical significance" of statistics at Stage 1: Are "small" impacts legally insignificant? For example, is an employment policy that causes a one percent disparate impact an appropriate policy for redress through disparate impact litigation? This circuit split calls for a comprehensive analysis of practical significance testing across disparate impact's stages. Importantly, courts and commentators use "practical significance" ambiguously between two aspects of practical significance: the magnitude of an effect and confidence in statistical evidence. For example, at Stage 1 courts might ask whether statistical evidence supports a disparate impact (a confidence inquiry) and whether such an impact is large enough to be legally relevant (a magnitude inquiry). Disparate impact's texts, purposes, and controlling interpretations are consistent with confidence inquires at all three stages, but not magnitude inquiries. Specifically, magnitude inquiries are inappropriate at Stages 1 and 3--there is no discriminatory impact or reduction too small or subtle for the purposes of the disparate impact analysis. Magnitude inquiries are appropriate at Stage 2, when an employer defends a discriminatory policy on the basis of its job-related business necessity.

AUTHOR. Yale Law School, J.D. expected; Yale Philosophy, Ph.D. expected; Rutgers University, B.A. 2012. I thank the Yale Law journal staff, especially Notes Editors Greg Cui, Joe Falvey, and Urja Mitral. This argument's examples involve impacts on communities of which I am not a member. Such advocacy is "a touchy sort of subject," in the words of SJA Germanotta: "Can you stand up for people [when] you are not necessarily fully part of that community in a way that [members] can understand?" Most special thanks to Owen Fiss and the 2016 Community of Equals seminar participants who taught me a tremendous amount, including how to approach this question.

NOTE CONTENTS INTRODUCTION 2384 1. FOUNDATIONS: DISPARATE IMPACT AND STATISTICAL CONCEPTS 2388 A. Disparate Impact: A Brief Overview 2388 B. Motivations and Purposes 2390 C. Statistical Concepts 2392 1. Statistical Significance 2392 2. Practical Significance 2394 II. DISPARATE STATISTICS 2397 A. The Statistical Standard of Prima Facie Disparate Impact 2398 B. The Statistical Standard of Job-Related Business Necessity 2407 C. The Statistical Standard of Showing a Suitable Alternative 2411 III. RECOMMENDATIONS AND IMPLICATIONS 2412 CONCLUSION 2419 INTRODUCTION

Statistical evidence is crucial in each stage of disparate impact's three-stage analysis: (1) the plaintiff's prima facie demonstration of a policy's disparate impact; (2) the defendant's job-related business necessity defense of the discriminatory policy; and (3) the plaintiff's demonstration of an alternative policy without the same discriminatory impact. There is a circuit split on the role of "practical significance" inquiries at the prima facie stage, (1) raising a fundamental question about disparate impact theory: Are such "small"--effects, about whose existence we are confident--legally insignificant? For example, is an employment policy that causes a one percent disparate impact an appropriate object of disparate impact litigation?

This question calls for a broader analysis of "practical significance" at each of disparate impact's three stages. Importantly, courts use "practical significance" in multiple ways. The present argument's primary focus is practical significance referring to the magnitude of an effect supported by statistical evidence. I call courts' evaluation of the size of an effect a "magnitude inquiry." Another sense of practical significance involves the strength of the inference from an empirical-statistical finding to the real world. I refer to a court's evaluation of this aspect of practical significance as a "confidence inquiry." This is an important distinction, and courts and commentators often use "practical significance" in ways that are ambiguous between these two aspects. (2) The second aspect--practical significance as the strength of the inference supported by statistical evidence--is obviously relevant to disparate impact analysis, in the same way that assessing the strength of the inference supported by evidence is always relevant. A debate remains regarding "magnitude inquires," evaluations of whether some effect is sufficiently large, at each stage of analysis.

I argue that such magnitude inquiries are inappropriately used to evaluate whether a "large enough" prima facie disparate impact exists or whether an alternative policy with less discriminatory impact promises a "large enough" decrease in discriminatory impact, at the first and third stages of disparate impact litigation. However, magnitude inquiries are more appropriate when an employer defends a discriminatory policy on the basis of its job-related business necessity, at the second stage of disparate impact litigation. Thus, this argument's primary contribution is an analysis of "magnitude inquiries," one aspect of practical significance, across all three stages of disparate impact.

The Note proceeds in three parts. Part I describes disparate impact theory, highlighting the logic of the shifting burden of proof, (3) and relevant statistical concepts. Part II analyzes statistics' role at three stages of disparate impact analysis : the plaintiff's establishment of prima facie disparate impact, the defendant's rebuttal of establishing a test's job-relatedness and business necessity, and the plaintiff's proposal of a less discriminatory alternative policy. I argue that disparate impact law supports the rejection of magnitude inquiries for a plaintiff's prima facie case of disparate impact and proposal of a less discriminatory alternative, but it supports a more robust magnitude inquiry during an employer's establishment of a disparity-causing test's job-relatedness and business necessity. Part III provides recommendations for improving the use of statistics in disparate impact analysis.

This Note contributes a defense of the First Circuit's decision, which has previously been subjected to critical commentary. (4) Importantly, it highlights the distinction between two aspects of "practical significance" sometimes obscured in disparate impact discussions: magnitude and confidence. The Note also contributes a comprehensive analysis of practical significance, providing recommendations for the use of statistics at all three stages of disparate impact litigation. In doing so, it calls for courts to reflect broadly about whether their use of statistics at each stage is consistent with their uses at the two other stages, their underlying theory of statistics and evidence, and their disparate impact theory.

Given the amount(5) and importance(6) of disparate impact litigation, addressing key questions that can determine the outcome of these actions, such as courts' use of magnitude inquiries, can be of great consequence. Indeed, these issues have provoked controversy. Today, the role of "practical significance" in the prima facie stage of disparate impact analysis is at the heart of a circuit split. The First, Third, and Tenth Circuits oppose practical significance inquiries; the Second, Fourth, Fifth, Sixth, Ninth, and Eleventh Circuits endorse them; and the D.C., Seventh, and Eighth Circuits have no clear precedent. (7)

Before turning to the analysis, it is worth noting that these legal questions arise against a particular scientific and cultural backdrop: the danger of relying on mere statistical significance in interpreting empirical studies is the subject of scientific and increasingly popular concern, and looking to "practical significance" is a popular remedy. (8) Calls to move science beyond simple statistical significance testing are not exclusive to the current moment, (9) nor are calls to move toward some form of practical significance testing. (10) Unreflective reliance on scientific trends might suggest that practical significance inquiries of all forms--including magnitude inquiries--are necessary parts of sound methodology, including throughout disparate impact analysis. This Note cautions otherwise. (11)

(1.) FOUNDATIONS: DISPARATE IMPACT AND STATISTICAL CONCEPTS

This Part provides an overview of disparate impact litigation and its three-stage burden-shifting framework: the plaintiff's prima facie demonstration of a disparate impact, the defendant's job-related business necessity defense, and the plaintiff's demonstration of a suitable alternative policy with less discriminatory impact. Then, I describe disparate impact theory's fundamental aims, the purpose of each stage, and the two key statistical concepts: statistical significance and practical significance. The discussion of practical significance outlines the fundamentally different aspects of practical significance testing that courts use: "magnitude inquiries" evaluate whether an effect is sufficiently large to be legally relevant, while "confidence inquiries" evaluate whether statistical evidence sufficiently supports a claim. For instance, in evaluating whether a prima facie showing of disparate impact has been made, a court might examine whether the impact is sufficiently large (for instance, is a one percent disparity legally relevant?) or whether the evidence supports the claim that the policy caused a disparity.

1. Disparate Impact: A Brief Overview

   Title VII of the Civil Rights Act of 1964 prohibits workplace discrimination on the basis of protected characteristics...