Why are there so few conservatives and libertarians in legal academia? An empirical exploration of three hypotheses.

• Author: Phillips, James C.
• Position: III. Analysis, Commentary, and Caveats through Conclusion, with footnotes and tables, p. 182-207 - Thirty-Fourth Annual Federalist Society National Student Symposium

3. Analysis, Commentary, and Caveats

   1. Analysis and Commentary

      1. Qualifications

         As can be seen in Graph 5, conservatives and libertarians tend to be more qualified than their peers of unknown or liberal political orientation. The Graph treats the qualifications of liberal law professors as the baseline with which to compare conservatives/libertarians and unknowns, since liberals are the largest group.

         Conservatives and libertarians are easily the most likely to have had Supreme Court clerkships: They are 235.5% more likely to have clerked on the Supreme Court than their peers with unknown political orientation and 68.2% more likely than liberal peers. (119) Likewise, conservatives and libertarians are more likely to have made law review in law school--39.9% more likely than unknowns and 5.4% more likely than liberals. (120) Conservatives and libertarians also on average graduated from higher-ranked law schools--6.8% more highly ranked than unknowns and 24.1% more highly ranked than liberals. (121) Finally, conservatives and libertarians are more likely to hold a J.D. and have a federal appellate clerkship as their highest clerkship/22 but less likely to have their highest clerkship be with a federal district court, state court, or foreign court, these being less prestigious/23 In one less traditional but increasingly more relevant aspect, though, conservatives and libertarians appeared less "qualified"--having a Ph.D.--as conservatives and libertarians were 44.9% less likely to have Ph.D.s compared to unknowns, and 25.4% less likely compared to liberals. (124) Overall, however, the data are more consistent with Hypothesis #3 (and maybe a version of Hypothesis #2) as conservatives and libertarians are, on average, somewhat significantly more "qualified" than their peers in the legal academy. Whether this is because of discrimination or because only the more qualified conservatives and libertarians are interested and actually seek law professor jobs is unclear. (125)

      2. Causality

         1. The Potential Outcomes Framework

            Questions of causal inference can be thought of as the task of determining counterfactuals. This is often referred to as the potential outcomes framework: what would the potential outcome have been under the alternative scenario where the unit of observation did not (or did) receive the treatment, ceteris paribus. (126)

            Of course, this is impossible outside of science fiction and creates a problem of missing data--we can never see the outcome in the alternative universe for any one individual. (127) Instead, researchers attempt to create two groups that appear to be essentially equal on factors that matter for the outcome being studied, giving one group the treatment (or intervention) and withholding it from the other. By measuring the difference between these two otherwise identical groups on the outcome being studied, one can infer that the treatment caused the difference. This is why random assignment of subjects to either a treatment (128) or control group in experimental designs is the gold standard for determining causality.

            But like our alternative universe scenario above, even this is often not fully possible since some of the most interesting or important causal questions cannot be examined under the conditions of a controlled experiment. This leaves us with the task of inferring causality from the messy data generated by the real world. And this is the scenario here.

            This far from ideal situation requires careful thinking about the potential outcomes (or counterfactual) framework, specifically the Stable Unit Treatment Value Assumption (SUTVA), (129) and the ignorable treatment assignment assumption. (130)

            SUTVA "is simply the a priori assumption that the value of [an outcome] for [a] unit [] when exposed to treatment [] will be the same no matter what mechanism is used to assign treatment [] to [the] unit [] and no matter what treatments the other units receive." (131) It has two basic principles. First, that treatment of one individual does not affect the treatment of another individual. (132) Second, that treatment is homogenous. (133) Thus, the first principle could be violated if, for example, subjects in an experiment discussed the positive effects of their treatment with those in the control group and convinced them to start taking the treatment (such as exercise). (134) The second principle would be violated if something caused the treatment to be stronger or weaker for differing individuals or under different conditions, such as more or fewer people assigned the treatment or control groups.

            Applying SUTVA to the study at hand, for it to hold, the perception of the political orientation of candidates by law schools--the treatment here (135)--cannot be dependent on such things as the pool of current candidates, the order of looking at candidates, or current composition of the legal academy's collective political orientation. Given that we are dealing with perception, which is potentially influenced by anchoring and ordering effects, this could be problematic. Thus, a candidate may appear more or less conservative (or liberal) depending on the candidates whose FAR forms or meat-market interviews came just before or after her, or the other candidates who also were called out for a job talk. Likewise, a candidate may appear more or less conservative (or liberal) when collectively viewed by a more or less conservative (or liberal) faculty or hiring committee, the latter of which serves as a gatekeeper and given its smaller size, is both more likely to fluctuate as to its collective political/ideological orientation and more likely to be subject to groupthink. Further, if one year the majority of candidates were conservative to some degree or another (a farfetched scenario, admittedly), and the next year the majority of candidates were more or less liberal, a slightly conservative candidate in the first year might appear to be in the middle or even to the left of center ideologically/politically, whereas he may appear quite conservative the next year. However, SUTVA is not necessarily problematic here just because an individual member of a hiring committee or faculty may have her perception altered through discussions with other members since it is the committee or the faculty overall that is making the collective decision to hire or not hire a candidate, not the individual. Thus, because SUTVA does not completely hold with the scenario being studied here, the ability to generalize to years outside of those being studied is limited. (136)

            The ignorable treatment assignment assumption, alternatively referred to as unconfoundedness, (137) selection on observables, (138) conditional independence, (139) and exogeneity, (140) channels the principle of random assignment in an experimental design. (141) It stands for the proposition that whether or not someone received the treatment is unrelated to the outcome being measured after taking into account the other characteristics they possess that could influence the outcome (or controlling for these other factors). Thus, overt and hidden biases are not a problem if this assumption holds. But if this assumption is violated, it is impossible to eliminate alternative, confounding explanations for the measured outcome. In the real world this assumption is violated all the time as people self-select into various "treatments," or others select to apply "treatment" outside of the neutrality of random assignment. A good research design is the best cure for this inferential ill, but statistical corrections can sometimes be a suitable fallback.

            Certainly this study, as with most observational studies that are not some kind of fortuitous natural experiment, violates this assumption and requires statistical correction since we cannot randomly assign the perception of political/ideological orientation given that is driven by (1) the actual underlying political/ideological orientation of a candidate; (2) the degree an individual chooses to publically signal such orientation; (3) the degree faculties evaluating candidates pick up on these signals; (4) the degree faculties' underlying actual political/ideological orientation colors their reading of the candidates' signals. Thus, statistical correction is necessary.

            Regression. Regression modeling, matching and propensity score analysis are all attempting to do the same thing--break the link between treatment assignment and treatment outcome. But they are not interchangeable. When "treatment groups have important covariates that are more than one-quarter or one-half of a standard deviation apart, simple regression methods are unreliable for removing biases associated with differences in covariates, a message that goes back to the early 1970s but is often ignored." (142)

            Thus, when trying to adjust for covariate imbalance, regression "is adequate in simple situations," but inadequate when "the differences between the two distributions are [too] large." (143) This is because regression estimates are sensitive to the lack of covariate overlap, often making it "impossible to arrive at a credible estimator based on simple regression methods." (144)

         2. Covariate Balance

            As noted above, conservatives/libertarian law professors and law professors with either a liberal or unknown political/ideological orientation are not similarly qualified. This is a problem since these qualifications are covariates in statistical models seeking to tease out causal effects. Without some correction so that apples are being compared to apples, any estimated causal effect will be biased. As noted in the graph below of the propensity scores for all of the data, the overlap is particularly poor when the propensity scores approach 1.

            There are several techniques to correct this that will be applied here: propensity score matching, propensity score weighting, nearest neighbor matching (NNM), and coarsened exact matching (CEM). (145)...