"Ideology" or "situation sense"? An experimental investigation of motivated reasoning and professional judgment.

• Author: Kahan, Dan M.
• Position: Continuation of IV. Results A. Legal Reasoning 1. Summary Data through Conclusion, with footnotes, p. 394-439

Cultural divisions were more modest but still apparent among members of the public in Disclosure (Figure 6). Egalitarian Individualist and Hierarchical Communitarian subjects favored finding a violation of the statute by narrow but roughly equal margins (52% to 48% and 53% to 47%, respectively) in the "pro-choice center" version. But in the "pro-life center" version, subjects with these identities diverged: whereas 60% of the Egalitarian Individualists concluded the officer had violated the statute by alerting the anti-abortion family-counseling center of its possible infiltration by a pro-choice activist, only 39% of Hierarchical Communitarian subjects did. While less dramatic than the effect in Littering, this disparity, too, fit the study predictions on how identity-protective cognition would affect the disposition of individuals with these worldviews to find violations in the two versions of Disclosure.

There was no evidence of comparable effects in the responses of the judges. In Littering, Egalitarian Communitarian judges were slightly less likely, not more, to find the defendants had violated the statute in the "immigrant aid" version than were Hierarchical Individualist judges. In addition, the proportion of Egalitarian Communitarian judges who supported finding a violation did not increase but rather decreased slightly in the "construction worker" version (Figure 5). The proportion of Hierarchical Individualist judges finding a violation in the "construction worker" version also decreased by a small amount--but judges with that cultural worldview remained more likely to find the construction workers liable than did Egalitarian Communitarian judges. None of these differences, all of which were small, displayed the relationship between worldviews and outcome judgments suggestive of identity-protective cognition.

The responses of the lawyer members of the study sample were also not suggestive of identity-protective cognition. In Littering, Hierarchical Individualist lawyers were modestly more likely to find a violation in both versions ("construction worker": 33%; "immigrant aid": 40%) than were Egalitarian Communitarian lawyers ("construction worker": 25%; "immigrant aid": 33%). The latter, moreover, were slightly less likely, not more, to find a violation in the "immigrant aid" version than they were in the "construction worker" version. In Disclosure, a higher proportion of Egalitarian Individualist lawyers (95%) than Hierarchical Communitarian ones (82%) supported finding the defendant officer violated the statute when he exposed the pro-choice activist's effort to obtain a position at the religious, pro-life family planning center. But the proportion of Egalitarian Individualist lawyers who supported finding a violation (88%) was also higher than the proportion of Hierarchical Communitarian ones who did (83%) in the "pro-choice center" version, where the police officer had tipped off the pro-choice family planning center that a job applicant had concealed his identity as a pro-life activist. In both versions, moreover, lawyers overwhelmingly construed the statute as dispensing with the need to prove the officer "knew" his conduct was illegal.

Among the students, in contrast, cultural divisions were again evident. Cultural dissensus was most pronounced in Disclosure (Figure 6). Eighty-six percent of Egalitarian Individualist students, but only 63% of Hierarchical Communitarian ones, favored finding the police officer violated the statute in the "pro-life center" version. But in "pro-choice"--the version in which the officer tipped off the pro-choice family counseling center of possible infiltration by a pro-life activist--78% of Hierarchical Communitarians, and only 65% of Egalitarian Individualists, supported finding a violation. This inversion reflects the pattern associated with identity-protective cognition.

In Littering, comparable proportions of Egalitarian Communitarian students (68%) and Hierarchical Individualist ones (64%) favored finding no violation of the statute in the "construction worker" version. But in the "immigrant aid" version, a 20% gap emerged between Hierarchal Individualist students, 63% of whom favored finding a violation, and Egalitarian Communitarian ones, only 43% of whom supported that outcome (Figure 5). This pattern was also consistent with the predicted impact of the experimental manipulation on individuals with the specified cultural outlooks.

2. Multivariate Regression

   1. Generally

      The impact of subject type, worldview, and experimental assignment was probed more systematically with multivariate logistical regression analysis. For each problem, a regression model was constructed to enable statistical estimation of the probability that different subject types (member of the public, law student, lawyer, or judge) would find a violation conditional on the subject's worldview and the version of the problem. (145)

      Monte Carlo simulations based on the regression models were performed to facilitate interpretation of the results. In a Monte Carlo simulation, the regression model outcome variable is calculated and randomly adjusted by an amount reflecting the measurement error associated with the model parameters. This process is repeated a sufficient number of times to populate the entire probability distribution for the outcome variable at specified values of the model predictors. (146) Using this technique, the probability that a decisionmaker would find a violation was computed a thousand times for each combination of subject type, cultural worldview, and experimental assignment of interest (Appendix A, Figure At, Figure A2). (147) Differences in the predicted probabilities of finding a violation conditional on worldview or experimental assignment were determined in the same fashion. (148)

   2. Judges versus Members of the Public Using Frequentist Methods

      Figure 7 reports simulated estimates of the size of the interaction between subject worldviews and the experimental assignment for a member of the public and for a judge, respectively. There are separate estimates for each problem and for the "average effect" based on responses to both problems. (149)

      Based on the multivariate regression models, the density distributions reflect the predicted difference in the probability that a decisionmaker with a particular worldview ("Hierarchical Individualist" or "Egalitarian Communitarian" in Littering, or "Hierarchical Communitarian" or "Egalitarian Individualist" in Disclosure) will find a violation in one version of the problem versus the other. Equivalently, the curves reflect estimates of how much more likely on average a decisionmaker with a particular worldview is to interpret the statute differently when finding a violation affirms rather than denigrates his or her cultural commitments. We thus characterize the model outputs as indicating the predicted "identity-protective cognition impact" (IPCI) of the experimental manipulations. (150)

      The most likely IPCI for any subject type is the mean value in the distribution for that subject type. The probability that the "true" IPCI is larger or smaller than that becomes progressively less likely, consistent with the bell shape of the probability density distribution. Ninety-five percent confidence intervals (or ones of any other size) for the "true impact" can be determined by identifying the predicted values that bound the relevant interval in the range of simulated probabilities. (151)

      The results confirm that the experimental manipulations generated the predicted identity-protective cognition effects in members of the public. Based on the regression model for Littering, for example, the predicted "identity-protective cognition impact" or IPCI for a member of the public is 29%. That is the best estimate, in other words, of how much being assigned to the "immigrant aid" version of the problem as opposed to the "construction worker" version changes the probability that either a "Hierarchical Individualist" or an "Egalitarian Communitarian" member of the public will find a violation. The 0.95 level of confidence for that estimate is [+ or -] 9%. (152) In Disclosure, the IPCI--the difference in the probability that either an "Egalitarian Individualist" or "Hierarchical Communitarian" member of the public will find a violation if assigned to the "pro-choice center" as opposed to "pro-life center" version--is 16% ([+ or -] 9%). The average IPCI for a member of the public is 22% ([+ or -] 6%) (Figure 7). (153)

      The regression model corroborates the inference that judges were not affected by identity-protective cognition. As is clear from the judge IPCI probability distributions (Figure 7), the predicted IPCI for judges was not different, statistically or practically, from zero in either Littering or Disclosure. (154)

      Moreover, the difference between the public and judge IPCIs in both problems was large and significant, statistically and practically, in both Littering (34%, [+ or -] 22%) and Disclosure (20%, [+ or -] 17%). The predicted average public IPCI exceeds the predicted judge IPCI by 27% ([+ or -] 14%). The "null hypothesis"--that there is no difference in the vulnerability of judges and members of the public to identity-protective reasoning--can thus be "rejected."

   3. Judges versus Members of the Public Using Bayesian Methods

      As an alternative to assessing the improbability of the "null hypothesis," one can use Bayesian methods to assess the strength of the evidence in relation to competing hypothesized IPCIs. (155) Under Bayes' theorem the likelihood ratio reflects how much more consistent an observed outcome is with one hypothesis than a rival one. It is the factor in proportion to which one should adjust one's assessment of the relative probability (expressed in odds) of one hypothesis in relation to the other. (156)

      Imagine, for example, that we are shown two opaque canvas bags, labeled "[B.sub.1]" and...