“Bending” Evidence for a Cause: Scholar‐Advocacy Bias in Family Law

• Published date: 01 April 2016
• Date: 01 April 2016
• Author: Michael Saini,Robert E. Emery,Marsha Kline Pruett,Janet R. Johnston,Amy Holtzworth‐Munroe,Irwin Sandler,JoAnne L. Pedro‐Carroll
• DOI: http://doi.org/10.1111/fcre.12210

RESEARCHERS’ ROUNDTABLE

“BENDING” EVIDENCE FOR A CAUSE: SCHOLAR-ADVOCACY BIAS

IN FAMILY LAW

Robert E. Emery, Amy Holtzworth-Munroe, Janet R. Johnston, JoAnne L. Pedro-Carroll,

Marsha Kline Pruett, Michael Saini, and Irwin Sandler

There are a number of salient public policy issues in the family law field that have invoked impassioned policy debates on a

recurrent basis. In the absence of a body of research to address these critical concerns, advocates under the guise of social sci-

ence scholarship have exacerbated the confusion and controversy by construing the scant available research evidence to justify

their own ends, without regard to the relevance, quality, utility, and limitations of the studies. This is one of two articles on this

problem that we have named “scholar-advocacy bias.” In this article, we discuss the difference between truth in social science

and truth in law. We identify common ways in which social science researchers and reviewers of research—wittingly or unwit-

tingly—can become advocates for ideological positions and social policies at the expense of being balanced reporters of

research evidence as illustrated by recent debates about overnight parenting of infants and toddlers. We also consider how

adherence to established scientific principles and methods prevents the misuse of research in this way.

Key Points for the Family Court Community:

Scholar-advocacy bias, the intentional or unintentional use of social science research to legitimize advocacy claims, is

a problem that practitioners and policy makers must recognize and guard against in family law.

Because of different methods of pursuing and establishing truth in science and the law, ironically, the less rhetorically

convincing argument often represents science most faithfully.

Practitioners—and scientists—must guard against using various rhetorical tactics that bend research evidence, for

example, the rules of science dictate that we must prove our hypotheses; others need not disprove them.

Keywords: Advocacy Research; Scholar-Advocacy Bias; and Social Science Research.

Thomas Huxley famously pointed to “The great tragedy of Science - the slaying of a beautiful

hypothesis by an ugly fact.” Huxley’s “tragedy” is tongue-in-cheek. He delighted in empirical scien-

ce’s triumph of evidence over ideas. Our present concern is the opposite of Huxley’s: facts murdered,

or merely maimed, in the name of a cause, even a worthy one. The name of this enterprise is not Sci-

ence, but “Scholar-Advocacy Bias” masquerading in the name of Research. We define scholar-

advocacy bias as the intentional or unintentional use of the language, methods, and approaches of

social science research, as well as one’s status as an expert, for the purpose and/or outcome of legiti-

mizing advocacy claims at the cost of misrepresenting research findings. We argue that scholar-

advocacy bias goes largely unacknowledged in family law.

Advocacy and research in family matters are terms that embody an essential tension between the

pursuit of truth in law and science. Truth in the law is about persuasion, using the legal rules of evi-

dence to advocate for your side and ultimately to convince a judge or jury of the truth of your case.

Truth in science is about public demonstration, designing research procedures that others can use

objectively to replicate your findings. Both pursuits of truth serve very worthy purposes. Establishing

what is social science evidence is complicated within an adversarial legal system where the legal

Correspondence: holtzwor@indiana.edu, mpruett@smith.edu, johnston527@sbcglobal.net, michael.saini@utoronto.ca,

jpcarroll4peace@gmail.com, ree@virginia.edu

*All authors contributed equally to this work.

FAMILY COURT REVIEW, Vol. 54 No. 2, April 2016 134–149

V

C2016 Association of Family and Conciliation Courts

approach to reaching truth is zealous advocacy to make the best case for opposing sides. The scien-

tific approach is to adhere to scientific principles and methods that guard against bias. When the legal

advocacy approach to fact-finding is applied to scientific evidence, then one-sided presentations of

data and polarization of controversial positions become more likely.

Combining the terms advocacy and research produces an oxymoron—advocacy research.

2

Research involves seeking knowledge about, or solutions to, problems that can be objectively dem-

onstrated to others; advocacy implies one already knows the solution and the task is convincing

others to mobilize resources accordingly. Although researchers can advocate for the use of their find-

ings to affect policy and practice, they still are bound by the rules of science. Scientists know they

may be wrong. They acknowledge the limitations of their research, while disputing specious chal-

lenges to their methods. Scientists must also recognize the difficulties in translating limited research

findings into real world solutions.

Cautions about scientific integrity, caveats, and limitations may not be terribly persuasive to non-

scientists, especially in contrast to the certainty of advocacy. Advocates marshal unambiguous argu-

ments, often with emotional appeal, in pursuit of persuading others of the truth and moral rectitude

of their side, including arguments about what “research shows.” Thus, ironically, a key to distin-

guishing science from advocacy researcher bias is that the scientist is likely to be rhetorically less

convincing than the researcher working from a position of advocacy bias. As psychologist Daniel

Kahneman (2011) has shown, our brains are wired to accept emotionally charged arguments over

rational ones. Our brains respond quickly and unquestioningly to powerful emotions, which may sig-

nal that our very survival is at stake, in comparison to rational arguments, which are less emotionally

appealing and require time and effort to sort through.

SCHOLAR-ADVOCACY BIAS IN FAMILY LAW

Scholar-advocacy bias is not a new concern in family law. A decade ago, the AFCC sponsored a

plenary session at its annual conference on the politics of research entitled “The Use, Abuse and Mis-

use of Social Science Research.” The presenters at this conference, along with others before and

since, emphasized the need to develop effective cross-disciplinary communication, common standards,

and ethical principles for the use of research findings to provide accurate court testimony and to build

an evidence-informed knowledge base of effective programs and policies (Cashmore & Parkinson,

2014; Cherlin, 1999; Gelles, 2007; Johnston, 2007; Kelly & Ramsey, 2007; Kelly & Ramsey, 2009;

Pruett, 2007; Ramsey & Kelly, 2004; Ramsey & Kelly, 2006).

Although there have been important contributions of science in politically sensitive areas of fam-

ily law,

3

during the past decade, unfortunately public investment in civil court matters has remained

a low priority. Consequently, research that directly addresses critical family court questions is limited

or absent, leaving the door wide open for scholar-advocates to promote false or misleading claims

within impassioned debates about what “research says” with regard to a range of family law issues,

such as liberal divorce laws and effects on children, sex abuse allegations in custody disputes, same-

sex marriage and parenting, artificial procreation and single parenthood, relocation, father involve-

ment, and child support enforcement. More recently, parenting time policies for infants and young

children, parental alienation, and intimate partner violence in family court have been at the center of

controversy.

Where does the truth lie? For advocates, truth emerges from judgements based on competing

arguments that reach beyond the available social science evidence and focus on moral and ethical

arguments, personal experiences, consideration of related laws and legal procedures, calls for the pro-

tection of civil rights, and appeals to social values and morals. By contrast, the role played by science

is more circumspect and in some ways less compelling. Given the broad reach of family law, the rare

use of random assignment studies (the “gold standard” of scientific research), and the relatively small

number of studies (and researchers) in the field, the ultimate empirical truth regarding many family

law controversies often is “more research is needed.” Of course, this answer can frustrate both

Emery et al./SCHOLAR-ADVOCACY BIAS 135