Material facts in the debate over Twombly and Iqbal.

• Author: Gelbach, Jonah B.
• Position: III. Employment Discrimination Cases through Conclusion, with footnotes and tables, p. 398-424

3. Employment Discrimination Cases

   This Part of the Article reports the study's empirical findings for employment discrimination cases. Part III.A discusses the details of the employment discrimination sample, while Part III.B discusses the empirical results for cases in this sample. These results indicate that regardless of whether a plaintiff's win is defined as winning with respect to all or simply at least one challenged claim or issue, and regardless of whether one adjusts for certain potential confounding variables, the plaintiff's win rate did not change in a statistically significant way following Twiqbal.

   1. Basic Characteristics of the Employment Discrimination Sample

      Coders processed a total of 2511 employment discrimination cases. Of these, 185 were dropped because they were from districts that had not adopted the ECF system as of October 1, 2005. (75) An additional 335 were dropped because at least one pro se plaintiff was involved in at least one motion for summary judgment. (76) A further 138 were dropped because they appeared to involve ADA-related claims. The ADA Amendments Act of 2008 expanded the set of people covered by the ADA, which introduces a risk of misattributing effects of this change in ADA law to changes in pleading standards. This study therefore dropped from its sample those cases with claims that could be identified as ADA related. (77) This left a sample of 1853 employment discrimination cases with summary judgment motions in which all plaintiffs had counsel and none of their claims appeared to be ADA related. In 32 of these, there was no defendant's summary judgment motion filed. (78) This left a total sample of 1821 non-AD A employment discrimination cases in which there were no pro se plaintiffs and at least one defense summary judgment motion. (79)

      The first column of Table 2 reports the number of such cases in the sample by year of filing (Panel A) and according to whether the filing year was in the pre-Twombly or post-Iqbal period (Panel B). All told, the sample contained 1189 employment discrimination cases coded in the pre-Twombly period that had defense summary judgment motions and no pro se plaintiffs; there were 632 such cases in the post-Iqbal period. The second column of Table 2 reports the number of these cases that had at least one summary judgment motion adjudicated within 731 days of case filing. It is not an accident that there are a disproportionate number of cases included in the 2005 filing period. Law school student coders were ready to work before all years of data were loaded into the database. Thus, while database code necessary to load the other years' cases was completed, the students worked on coding cases that were filed in 2005. To avoid any unrepresentativeness in the results, this study uses weights that adjust for the difference in sample sizes. These weights appear in the final column of Table 2; each case from 2005 effectively gets just under twenty-seven percent as much weight as cases filed in other years. (80)

      Within each year, just below sixty percent of cases had motions that had been adjudicated before the 731-day cutoff. All told, the pre-Twombly period had 700 employment discrimination cases with no pro se plaintiff and an adjudicated defense summary judgment motion, while the post-Iqbal period had 368 such cases. These are the cases on which the study's primary analysis is performed below.

   2. Summary Judgment Adjudication Results for Employment Discrimination Cases

      This Subpart reports the core results for employment discrimination cases through both an unadjusted and an adjusted estimate of the change in the plaintiff's win rate following Twombly and Iqbal. The unadjusted estimates are based on the simple win rate for plaintiffs among observations in the analysis sample. The adjusted estimates are based on binary logit models. (81) In these models, the outcome variable serves as a dummy variable equal either to one for a case wherein the plaintiff wins on a defense summary judgment motion (however a win is measured) or equal to zero if the plaintiff does not. The variable of primary interest is a dummy variable indicating whether the case was filed in the post-Iqbal period. The adjusted estimate of the change in the plaintiff's win rate following Twombly and Iqbal is calculated using the following three steps. First, the binary logit model is estimated. Second, for each observation the estimated coefficients from that model are used to calculate the estimated change in the plaintiff's win probability due to Twiqbal. Such an estimated change is commonly known as a "marginal effect" of a change in circumstances--here, the change in the pleading standard. Finally, these observation-specific estimated marginal effects are averaged over all cases in the relevant sample, yielding a single adjusted effect. (82) Besides the Twiqbal dummy variable, the other predictor variables--whose inclusion in the model is the source of the adjustment--are the following:

      * A dummy variable indicating whether the judge was appointed by a Republican President. (83)

      * A dummy variable indicating whether any defendant in the case was a business organization. (84)

      * The employment-to-population ratio for the year when the case was filed, in the state where the judicial district is located.

      * The employment-to-population ratio for each of the two years preceding the year in which the case was filed, in the state where the judicial district is located.

      * The population for the year when the case was filed, for the state where the judicial district is located.

      The appointing-President variable is meant to capture the effects of any ideological variation in judging, to the extent that such a variable does so. (85) The business organization variable is included to capture any effects of differences in party type. The employment-to-population variables are designed to control for business cycle variation that could have important effects on litigation behavior independent of Twombly and Iqbal. (86) Finally, the population variable is included to control for the possibility that cases in bigger states might be systematically different from those in smaller districts.

      1. The percentage of cases in which plaintiffs win as to all challenged claims or issues

      This Subpart reports results for the percentage of cases in which plaintiffs win as to all challenged claims or issues. As the discussion to come will show, the results indicate that this measure of the plaintiff's win rate was lower following Twiqbal. However, this drop in the plaintiff's win rate is estimated imprecisely, so that the results are not statistically significantly different from a zero effect.

      The first column of Table 3 reports percentages of employment discrimination cases, among those with summary judgment motions adjudicated within 731 days of case filing, in which the plaintiff won on all challenged claims. For the pre-Twombly sample, 18.3% of motions were denied as to all claims raised in the motions. For the post-Iqbal period, this figure was slightly lower, at 17.1%. Table 3's second column reports estimated standard errors for these percentages, which are 1.8% and 2.0% respectively.

      The first column of Table 3's third row reports information for the difference in the all-claims-denied percentage. This difference is -1.2%, nominally suggesting that case quality fell. However, the second column shows that the difference in win rates has an estimated standard error of 2.7%--great enough so that the difference is far from being statistically significantly different from zero. (87) Another way to put this is via a 90% confidence interval for the change in the plaintiff's win rate against all challenged claims. This confidence interval, reported in the bottom row of Table 3, includes all values between a drop of 5.5% and an increase of 3.2%, which makes it clear that the results are not precise enough to allow us to conclude that Twiqbal caused either a drop or an increase in this measure of the plaintiff's win rate. these district courts have no variation in the plaintiff's win rate, preventing the inclusion of these observations in logit estimation. The adjusted estimates in this table are thus based on 968 observations, by comparison to 1068 observations in the unadjusted estimation. Unadjusted estimates calculated using only the 968 observations included in the logit estimation yielded a difference in the plaintiff's win rate of -0.8%, with an estimated standard error of 2.8%.

      The third and fourth columns show that adjusting for party of the appointing President, business status of defendants, state population, and trends in the state employment-to-population ratio leads to a substantially larger estimate of the change in the plaintiff's win rate--a drop of 4.4%. However, the adjusted difference is still quite far from being statistically significant: its estimated standard error is 6.1%, yielding a p-value well above conventional significance levels. (88) The greater estimated standard error leads to an even wider 90% confidence interval, which includes all values between a drop of 14.4% and an increase of 5.7%. Thus the adjusted results also are not precise enough to allow us to conclude with typical levels of confidence that Twiqbal caused either a drop or an increase in this measure of the plaintiff's win rate.

      2. The percentage of cases in which plaintiffs win on at least one challenged claim

      This Subpart reports results for the percentage of cases in which plaintiffs win as to at least one challenged claim or issue. As the discussion to come indicates, the results indicate that this measure of the plaintiff's win rate was a bit higher following Twiqbal. However, this increase in the plaintiff's win rate is estimated imprecisely, so that the results are not statistically significantly different from a zero effect. The first column of Table 4 reports the percentages of claims...