Fixing Ever-ready: Repairing and Standardizing the Traditional Survey Measure of Consumer Confusion

• Publication year: 2019

Fixing Ever-Ready: Repairing and Standardizing the Traditional Survey Measure of Consumer Confusion

Eric D. DeRosia
Brigham Young University Marriott School of Business

FIXING EVER-READY: REPAIRING AND STANDARDIZING THE TRADITIONAL SURVEY MEASURE OF CONSUMER CONFUSION

Eric D. DeRosia^*

[Page 613]

In trademark infringement litigation, courts often rely on consumer surveys that use the "Ever-Ready" method to measure consumer confusion. Courts are understandably careful to scrutinize consumer surveys for ways in which their methodology might have biased their results toward the outcome desired by their proponents. This Article strengthens and improves such examinations by empirically testing and improving the Ever-Ready method itself.
The findings of four new empirical studies reported in this Article indicate the faith placed by the courts in the Ever-Ready method is somewhat misplaced. Seemingly subtle variations in the wording of the Ever-Ready questions have a consistent and surprisingly large influence on the survey's final results.
Fortunately, the four empirical studies also give clear guidance on how to repair and standardize the question wordings. Two versions of the Ever-Ready method—one version to be used in surveys proffered by plaintiffs and another version to be used in surveys proffered by defendants—are defined and verified based on data from the four studies. The two versions are "known" in the sense that they are defined empirically and published to the community, and "conservative" in the

[Page 614]

sense that they cut against the direct interests of the survey's proponent.
The standards set by these known-conservative versions of the Ever-Ready method will empower the courts and a survey's opponents to scrutinize (in a rigorous, consistent, and systematic manner) the extent to which the question wording in an Ever-Ready survey is slanted toward the proponent's desired outcome. Anticipating such scrutiny, experts will have a powerful incentive to adopt the known-conservative versions that are identified in this Article. The expected end result is greater reliability for expert testimony, with a particular emphasis on preventing parties from bolstering weak cases with methodological artifacts.

[Page 615]

Table of Contents

I. Introduction...................................................................616

II. Background of the Ever-Ready Method....................622

A. ORIGINS OF THE EVER-READY METHOD........................... 622
B. CURRENT ACCEPTANCE OF THE EVER-READY METHOD ... 623
C. COMPARISONS TO OTHER METHODS................................626

III. Testable Propositions..................................................630

A. VARIANTS OF THE EVER-READY METHOD........................631
B. RESPONSE EFFECTS OF THE EVER-READY VARIANTS.......632

IV. Variants of the Ever-Ready Method..........................645

A. EXEMPLARS OF THE EVER-READY METHOD AS USED IN FEDERAL COURTS......................................................... 645
B. GENERALIZED FORM OF THE EVER-READY VARIANTS......649

V. Response Effects of the Ever-Ready Variants........651

A. STUDY 1..........................................................................651
B. STUDY 2..........................................................................657
C. STUDY 3..........................................................................658
D. STUDY 4..........................................................................663

VII. The "Known-Conservative" Approach........................666

A. DEFINING THE KNOWN-CONSERVATIVE APPROACH......... 666
B. BENEFITS OF A KNOWN-CONSERVATIVE APPROACH........670
C. OBJECTIONS TO A KNOWN-CONSERVATIVE APPROACH .... 675

VIII. Contributions to Broader Discussions and Conclusion......................................................................676

A. APOLOGISTS VERSUS RESTRICTIONISTS, STRONG CASES VERSUS WEAK CASES.................................................... 677
B. JUDICIAL EVALUATIONS AFTER THE DAUBERT REVOLUTION.................................................................677
C. ALLEGED INCOMPATIBILITY BETWEEN THE LAW AND SCIENCE........................................................................678
D. CONCLUSION.................................................................. 681

[Page 616]

I. Introduction

"As every trial lawyer knows, the phrasing of the question will determine the content of the answer by the witness. So also with survey questions." - J. Thomas McCarthy¹

Most trademark infringement disputes in federal court hinge on a single question: are consumers likely to believe the plaintiff is the source of the defendant's goods or services? At its core, this question is about the "perceptions"² and "beliefs"³ that are held "in the minds of consumers."⁴ As a result, both the scholarly legal⁵ literature and the scholarly consumer psychology⁶ literature have explored the topic of trademark infringement. This Article draws upon case law, the scholarly legal literature, and the scholarly consumer psychology literature to identify and fix a problem with the survey-based evidence that is often relied upon by courts in infringement litigation.

Parties to trademark infringement disputes often hire expert witnesses to conduct surveys that are designed to measure the

[Page 617]

extent of confusion among consumers.⁷ These surveys frequently play an important role in pretrial negotiations⁸ and at trial.⁹ The normative appeal of such surveys is described by Professor

[Page 618]

McCarthy: "A conscientious judge is distrustful of his or her own subjective estimations of consumer reaction and welcomes some accurate factual information on this score."¹⁰ Reviews of case records show that about two-thirds of the surveys in infringement disputes are proffered by plaintiffs.¹¹ Naturally, a plaintiff who proffers such a survey typically intends to show that consumer confusion is commonplace, whereas a defendant who proffers a survey typically intends to demonstrate that consumer confusion is rare.

The essence of such a survey is that participants are exposed to the junior product in a controlled manner, and questions are asked to measure whether participants think the senior user is the product's source.¹² The survey's results are proffered to courts as an indication of how consumers have responded (or will respond) to the junior mark in the marketplace.¹³ The courts are supportive of this basic approach, and they frequently embrace such surveys as probative of the likelihood of confusion among consumers.¹⁴

However, because a survey's methodology can strongly influence the survey's results,¹⁵ the courts carefully scrutinize the technical details of the methods used by the expert. Whether this scrutiny is part of an opponent's challenge of the admissibility of the expert's survey-based testimony or an evaluation of how much weight should

[Page 619]

be accorded to the testimony,¹⁶ the courts have been especially careful to scrutinize survey methods for ways in which they might "bias,"¹⁷ "skew,"¹⁸ and "manipulate"¹⁹ the results of the survey toward the outcome desired by the survey's proponent. For example, courts harshly criticize plaintiffs who use survey methods that the courts believe have falsely inflated the estimate rate of confusion.²⁰

When a survey's methodology receives judicial praise, other courts have shown a strong tendency to adhere to that dicta by giving credit to surveys that follow similar methodologies.²¹ The result has been a bandwagoning effect that nowadays constrains practically all confusion surveys into one of two methodologies: the Ever-Ready method²² and the Squirt method. Whereas the Squirt

[Page 620]

method has been accepted by some courts but rejected by others,²³ the Ever-Ready method has been widely accepted,²⁴ with courts calling it the "traditional"²⁵ and "gold standard"²⁶ method for measuring consumer confusion.

Because the Ever-Ready method is treated as orthodoxy by the courts, and because the fitness of the Ever-Ready method qua method has not yet been refined or even empirically tested, this Article turns a spotlight onto the Ever-Ready method itself. Part II provides the background of the Ever-Ready method. Part III sets out testable propositions concerning (1) subtly different variants in the question wording for Ever-Ready surveys that have been credited by federal courts, and (2) differential response effects for those variants, with implications for a survey's final results. To identify variants of the Ever-Ready method, Part IV reports a review of twenty surveys conducted by prominent expert witnesses. To test the response effects of the Ever-Ready variants, Part V reports the results of four new empirical studies. Collectively, the studies demonstrate that although the differences between the Ever-Ready variants may appear to be subtle and inconsequential, their response effects yield a consistent influence on the results of the surveys. The studies show that the response effects are so large that choosing an Ever-Ready variant can easily sway a survey to yield levels of confusion that are above or below the court's traditional threshold for infringement, thereby making a dispositive difference in a trademark dispute. The effects are consistent across the studies

[Page 621]

and robust to other methodological choices (e.g., whether the study uses a so-called "memory test" or a "reading test"). These empirical results indicate the faith of the courts in the Ever-Ready method is somewhat misplaced.

Rather than simply cursing the darkness, this Article lights a candle. In Part VI, the results of the four studies are drawn upon to repair and refine the Ever-Ready method. Two "known-conservative" versions of the Ever-Ready method are identified: one for plaintiffs and another for defendants. The two versions are "conservative" in the sense that they cut against the interests of the survey's proponent. The two versions are "known" in...