A reply to critics.

• Author: Sander, Richard H.
• Position: Response to article by Ian Ayres and others in this issue, p. 1807, 1855, 1899, 1915

INTRODUCTION I. REVISITING THE MISMATCH HYPOTHESIS A. Comparing Whites and Blacks B. The Curvilinear Effect of Grades on Outcomes C. Black-on-Black Comparisons D. The "Second-Choice" Analysis: A Valid Way to Do Black-on-Black Comparisons II. MICHELE DAUBER: THE ART OF UNINTENDED IRONY A. Blacks in the Job Market B. Peer Review C. Data Availability D. Replication III. AYRES AND BROOKS: STARING PAST THE DATA A. Making a Straw Man out of Systemic Analysis B. Empirical Evidence on Selection Bias C. Missing the "Second-Choice" Boat D. An Alternate Theory? IV. THE CHAMBERS ET AL. CRITIQUE: DEJA VU ALL OVER AGAIN A. First Round B. Arguments on the Mismatch Effect 1. Black underperformance? 2. Another black-on-black comparison 3. Grades and bar passage 4. The second-choice data C. Arguments on the Effects of Ending Preferences 1. The credentials gap in a race-blind regime 2. The surge in white applicants 3. The decline in enrollment at elite schools 4. The worsening mismatch effect V. DAVID WILKINS AND THE LONG-TERM EFFECTS OF GRADES A. Empirical Tests of the Wilkins Thesis B. Promotion to Partnership at Elite Firms C. How Rank-and-File Black Lawyers Fare over the Long Term D. The Opposite of Elite CONCLUSION: A RESEARCH AGENDA FOR THE MISMATCH DEBATE I know that most men, including those at ease with problems of the greatest complexity, can seldom accept even the simplest and most obvious truth if it be such as would oblige them to admit the falsity of conclusions which they have delighted in explaining to colleagues, which they have proudly taught to others, and which they have woven, thread by thread, into the fabric of their lives.

--Leo Tolstoy (1)

[I]t takes a theory to beat a theory.

--George J. Stigler (2)

INTRODUCTION

Although the public and academic reaction to A Systemic Analysis of Affirmative Action in American Law Schools (hereinafter Systemic Analysis) (3) has been predominantly favorable, many of my most sympathetic readers predicted a fierce reaction from what they often called "the affirmative action establishment." And although the four responses published in this issue are not the first outpourings of critical reaction, they are certainly the most concerted. When the Stanford Law Review editors sifted through the stack of prospective contributions to this issue, they specifically tried to select those that would offer the strongest critiques, bypassing several more sympathetic proposals. The distinguished authors whose responses have been selected have each focused on a different part of my article, and each suggests that he or she has found its Achilles heel, to be chewed upon over a combined 153 pages. Thinking about these critiques and trying to address them as thoroughly as possible has been both stimulating and consuming.

At the end of the day, however, the critiques in this volume are surprisingly toothless. Most of these contributors concede (and none dispute) the basic facts that frame Systemic Analysis: blacks are nearly two-and-a-half times more likely than whites not to graduate from law school, are four times more likely to fail the bar on their first attempt, and are six times more likely to fail after multiple attempts. (4) The overall lower average test scores and undergraduate grades of blacks obviously contribute to this gap--but no one disputes my finding that the black-white gap in graduation and bar passage is more than twice as large as can be explained by controlling for LSAT and undergraduate grade point average (UGPA) differences. None of these contributors offers any alternative explanation of this staggering black-white gap in graduation and bar passage, let alone a strategy for addressing it. If Stigler is right that it takes a theory to beat a theory, no one has even entered the arena. Since there is no debate that blacks' outcomes in graduation and bar passage are worse than could be possibly explained by blacks' entering credentials, we must face the fact that the legal education system is currently doing something that seriously harms blacks. Criticism and debate are important, but they must not obscure the overriding need to diagnose what we are doing wrong, and to implement solutions.

I have learned a great deal by thinking hard about the questions each of these contributors has raised. And I believe David Wilkins's article (5)--which I view as by far the strongest of these pieces--identifies several fundamental issues which merit long-term research and consideration. But none of these articles raises a serious challenge to the basic thesis of Systemic Analysis: large racial preferences, as currently practiced by American law schools, impose very large costs on blacks. Michele Dauber (6) is factually wrong on each of her principal claims. Ian Ayres and Richard Brooks (7) critique a straw man version of my theory and then construct an alternate test for the mismatch theory that is fatally compromised by methodological errors. David Chambers, Timothy Clydesdale, William Kidder, and Richard Lempert (Chambers et al.), (8) in their second attempt to mount a critique, make some valid points on the policy implications of my analysis but do not seriously address most of the article itself. Wilkins argues with great flair that my paper does not capture crucial benefits blacks gain from racial preferences, and validly identifies many relevant questions about which we know far too little, but marshals very little probative evidence supporting his arguments. I find the data that is available seriously undercuts his arguments.

Before jumping into detailed responses to each of the four critiques, it seems useful to set forth in some detail my thinking about the mismatch hypothesis at the core of Systemic Analysis. A weakness of that article, perhaps, is that the central hypothesis is presented almost as an afterthought, as a way of explaining the data patterns I found. Clearly setting out the hypothesis, and comparing several alternative ways of evaluating it, is a helpful way of laying bare exactly what I and the critics are talking about and where we disagree.

1. REVISITING THE MISMATCH HYPOTHESIS

   The premise of the mismatch theory is simple: if there is a very large disparity at a school between the entering credentials of the "median" student and the credentials of students receiving large preferences, then the credentials gap will hurt those the preferences are intended to help. A large number of those receiving large preferences will struggle academically, receive low grades, and actually learn less in some important sense than they would have at another school where their credentials were closer to the school median. The low grades will lower their graduation rates, bar passage rates, and prospects in the job market.

   Although there is a good deal of research on the mismatch theory focusing on undergraduates, Systemic Analysis is unusual in examining a fairly self-contained segment of graduate education and using databases that are sufficiently broad to permit one to examine a number of possible mismatch effects across the whole spectrum of schools, from very elite schools to night schools. (9) At the same time, the data has serious limitations. The primary database available for studying graduation and bar passage outcomes is the Law School Admission Council's Bar Passage Study (LSAC-BPS). (10) Though the LSAC-BPS is a remarkable data set in many ways (sample size, breadth of schools covered, and breadth of both subjective and objective measurements taken of participants), it was weakened terribly by the LSAC's decision to destroy all information linking students to individual schools. One can only place individual students in one of six broad "clusters" of schools, with between seven and fifty schools per cluster and with the clusters loosely but sufficiently correlated with prestige to think of them as "tiers." As we shall see, treating these clusters as perfect measures of prestige, rather than as rough and heterogeneous proxies, leads more than one critic astray.

   Even with the imperfect data currently available, there are several different ways of testing the mismatch theory for law students. Much of the emerging debate on the theory actually reflects differences in which methods scholars use and what assumptions they make with each method. In this Part, I examine each of these methods in turn to see whether a consistent underlying story emerges from the data.

   1. Comparing Whites and Blacks

      In most of Systemic Analysis, I evaluated the mismatch theory by comparing white and black experiences and outcomes in legal education. The analyses in Part II of the article showed that the vast majority of whites are admitted to law schools primarily on the basis of their quantifiable academic credentials (UGPA and LSAT), while blacks are admitted on essentially the same basis, but with a very large boost assigned to them based on race. These racial preferences for blacks have the effect of elevating them to much more elite schools, so that if we compare two students with similar credentials, one white and one black, the black student will usually be at a significantly more elite school than the white one, and the black student will usually have much lower credentials than most of his classmates. (11)

      The premise of the white-black comparison is that three things are true when we compare white law students with black law students: First, blacks tend to perform about the same in law school as do whites with similar entering credentials and are about as likely to graduate and pass the bar as are whites with similar grades from the same schools. Second, racial preferences tend to place blacks at much more elite schools than whites with similar credentials, creating the "credentials gap" between blacks and their classmates. Third, this credentials gap causes blacks to get dramatically lower grades, on average, than do their white counterparts at less elite schools, and...