Let's Not Make a Deal: An Empirical Study of Decision Making in Unsuccessful Settlement Negotiations

• Author: Randall L. Kiser,Blakeley B. McShane,Martin A. Asher
• Date: 01 September 2008
• Published date: 01 September 2008
• DOI: http://doi.org/10.1111/j.1740-1461.2008.00133.x

Let’s Not Make a Deal: An Empirical

Study of Decision Making in

Unsuccessful Settlement Negotiations

Randall L. Kiser, Martin A. Asher, and Blakeley B. McShane*

This study quantitatively evaluates the incidence and magnitude of errors

made by attorneys and their clients in unsuccessful settlement negotiations.

The primary study analyzes 2,054 contested litigation cases in which the

plaintiffs and defendants conducted settlement negotiations, decided to

reject the adverse party’s settlement proposal, and proceeded to arbitration

or trial. The parties’ settlement positions are compared with the ultimate

award or verdict, revealing a high incidence of decision-making error by

both plaintiffs and defendants. This study updates and enhances three prior

studies of attorney/litigant decision making, increasing the number of cases

in the primary data sets more than threefold, adding 72 explanatory vari-

ables from 19 classes, applying a multivariate analysis, presenting an histori-

cal review of error rates during the 1964–2004 period, and comparing the

primary study error rates with error rates in cases where the parties are

represented by attorney-mediators. Notwithstanding these enhancements,

the incidence and relative cost of the decision-making errors in this study

are generally consistent with the three prior empirical studies, demonstrat-

ing the robustness of the earlier works by Samuel Gross and Kent Syverud,

and Jeffrey Rachlinski. The multivariate analysis, moreover, shows that the

incidence of decision-making error is more significantly affected by

“context” variables (e.g., case type and forum) than by “actor” variables

(e.g., attorney gender and experience level).

*Address correspondence to Randall L. Kaiser, DecisionSet, 550 Hamilton Ave., Ste. 300, Palo

Alto, CA 94301; email: rkiser@decisionset.com. Asher is Director, Research and Scholars Pro-

grams, Wharton Undergraduate Division, and Adjunct Professor of Finance at The Wharton

School, University of Pennsylvania; McShane is a graduate student in the Department of

Statistics, The Wharton School, University of Pennsylvania.

We thank Jeffrey Rachlinski, Theodore Eisenberg, and an anonymous review for their insight-

ful comments on previous versions of this article.

Journal of Empirical Legal Studies

Volume 5, Issue 3, 551–591, September 2008

© 2008, Copyright the Authors

Journal compilation © 2008, Cornell Law School and Wiley Periodicals, Inc.

551

I. Introduction

The decision to settle or litigate necessarily requires an assessment of the

likely trial outcome.1Absent extrinsic motivations, a rational litigant roughly

weighs an adversary’s settlement proposal against the likely trial outcome,

makes some adjustments for attorney fees, court costs, and the possibility of

delays and appeals, and either accepts or rejects the adversary’s settlement

proposal. For litigants unwilling to accept an adversary’s settlement offer and

intent on obtaining a judgment on the merits, trials are their best alternative

to a negotiated agreement (BATNA). As Roger Fisher and William Ury assert

in Getting to Yes, a party’s BATNA “is the standard against which any proposed

agreement should be measured. That is the only standard which can protect

you both from accepting terms that are too unfavorable and from rejecting

terms it would be in your interest to accept.”2

To test whether attorneys and litigants accurately measure trial

outcomes against settlement alternatives in adjudicated cases, this study

examines 2,054 California civil cases ultimately resolved through trial or

arbitration, following unsuccessful settlement negotiations.3The cases were

reported in a 38-month period between November 2002 and December

2005; about 20 percent of all California litigation attorneys represented the

parties in these cases. The parties’ settlement positions in those cases are

compared with the ultimate award or verdict to determine whether the

parties’ probability judgments about trial outcomes were economically

1Samuel Gross & Kent Syverud, Don’t Try: Civil Jury Verdicts in a System Geared To Settlement,

44 UCLA L. Rev. 51 (1996): “Every theory of pretrial bargaining assumes that a negotiated

settlement is determined, at least in part, by the parties’ predictions of the outcome of the case

if it did go to trial.”

2Rober Fisher & William Ury, Getting to Yes: Negotiating Agreement Without Giving In

(Penguin Books 1991).

3The vast majority of civil cases, of course, are resolved by voluntary settlements or pretrial

proceedings. It is impossible to objectively measure the economic utility of decision making in

the settled cases, as the settlement consideration cannot be compared with an actual trial

outcome. The results of this study are limited to decision making in adjudicated cases with

confirmed settlement positions and, due to this selection bias, may not have any explanatory

value in settled cases. As Ward Farnsworth explained in his study of injunctions: “I am not

purporting to ask or answer any questions about what happens in cases that settle, so excluding

them is just a limitation on what the study means.” Ward Farnsworth, Do Parties to Nuisance

Cases Bargain After Judgment? A Glimpse Inside the Cathedral, in Behavioral Law & Economics

(Cass Sunstein, ed., Cambridge University Press 2000).

552 Kiser et al.

efficacious, that is, did the parties commit a decision error by rejecting a

settlement alternative that would have been the same as or better than the

ultimate award? Employing a multivariate analysis, the study presents a quan-

titative evaluation of those attorney/client probability judgments regarding

liability and damages, the costs of inaccurate probability assessments, and the

effect of explanatory variables such as offers of compromise, case type,

nature of alleged damages, and forum.

The results of this 38-month study are complemented by a 40-year

survey of settlement decisions in adjudicated cases from 1964 to 2004. The

40-year survey indicates whether attorney/litigant decision error rates are

constant and whether the incidence of adverse outcomes in the 38-month

study is atypical. Lastly, to tentatively assess whether the decision-making

errors shown in this study may be attributable to the study attorneys’ pos-

sible risk-taking propensities and bias against negotiated resolutions, the

study results are compared with error rates in cases where the parties are

represented by attorney-mediators who meet state-mandated mediator

training requirements and have been selected to serve on their local

court’s panel of mediators. This group of attorney-mediators, skilled in

case evaluation and conflict resolution, presumably would exhibit lower

decision-making error rates if the study attorneys’ error rates resulted from

singular risk-taking propensities or anti-settlement biases. Alternatively,

similar error rates for the study attorneys and the attorney-mediators could

demonstrate that the study attorneys are not uniquely risk seeking or that

clients, not their attorneys, assume the dominant role in making settlement

decisions.4

The study serves two principal purposes. First, it is a large-scale empiri-

cal study of settlement decision error in adjudicated cases, demonstrating

the extent, costs, and persistence of attorney/litigant judgment error.

Second, it updates and evaluates the continued validity of three pioneering

empirical studies of attorney/litigant settlement decision making: Samuel

Gross and Kent Syverud’s 1991 article, “Getting to No: A Study of Settlement

Negotiations and the Selection of Cases for Trial,” their 1996 study, “Don’t

4Attorneys, of course, are required to abide by “a client’s decision whether to accept an offer of

settlement of a matter.” American Bar Association Model Rules of Professional Conduct, Rule

1.2. In referring to “attorney/litigant” decision making, we intend to convey the collaborative

nature of the attorney/client relationship while acknowledging that the client is the ultimate

decisionmaker.

Decision Making in Unsuccessful Settlement Negotiations 553