Using Field Experiments in Accounting and Finance

• Published date: 01 May 2016
• Author: JOHN A. LIST,ERIC FLOYD
• DOI: http://doi.org/10.1111/1475-679X.12113
• Date: 01 May 2016

DOI: 10.1111/1475-679X.12113

Journal of Accounting Research

Vol. 54 No. 2 May 2016

Printed in U.S.A.

Using Field Experiments in

Accounting and Finance

ERIC FLOYD

∗AND JOHN A. LIST

∗

ABSTRACT

The gold standard in the sciences is uncovering causal relationships. A grow-

ing literature in economics utilizes field experiments as a methodology to

establish causality between variables. Taking lessons from the economics lit-

erature, this study provides an “A-to-Z” description of how to conduct field

experiments in accounting and finance. We begin by providing a user’s guide

into what a field experiment is, what behavioral parameters field experiments

identify, and how to efficiently generate and analyze experimental data. We

then provide a discussion of extant field experiments that touch on important

issues in accounting and finance, and we also review areas that have ample op-

portunities for future field experimental explorations. We conclude that the

time is ripe for field experimentation to deepen our understanding of impor-

tant issues in accounting and finance.

JEL codes: C00; C9; C93; G00; M4; M5

Keywords: field experiments; causality; identification; experimental design;

replication

1. Introduction

Rolling weighted balls down a shallowly inclined ramp, Galileo used scien-

tific experiments to prove that matter moves vertically at a constant rate

∗Rice University, University of Chicago, and NBER.

Accepted by Douglas Skinner. Wewould like to thank Brian Akins, Hans Christensen, Yael

Hochberg, Michael Minnis, Patricia Naranjo, Brian Rountree, Haresh Sapra, and participants

at the JAR 50th Annual Conference for helpful comments and suggestions. We appreciate ex-

cellent research assistance from Seung Lee, Rachel Yuqi Li, Ethan Smith, and Rustam Zufarov.

437

Copyright C, University of Chicago on behalf of the Accounting Research Center,2016

438 E.FLOYD AND J.A.LIST

(regardless of mass) due to gravitational effects. Ever since, the experimen-

tal approach has been a cornerstone of the scientific method. Whether it

was Sir Isaac Newton conducting glass prism experiments to educate him-

self about the color spectrum or Charles Darwin and his son Francis us-

ing oat seedlings to explore the stimuli for phototropism, researchers have

rapidly made discoveries since Galileo laid the seminal groundwork. Sci-

entists have even taken the experimental method from the laboratory to

the field. In one classic 1882 example, Louis Pasteur designated half of a

group of 50 sheep as controls and treated the other half using vaccination.

All animals then received a lethal dose of anthrax. Two days after inocula-

tion, every one of the 25 control sheep was dead, whereas the 25 vaccinated

sheep were alive and well. Pasteur had effectively made his point!

Increasingly, social scientists have turned to the experimental model of

the physical sciences as a method to understand human behavior. Much

of this research takes the form of laboratory experiments in which volun-

teers enter a research laboratory to make decisions in a controlled environ-

ment (see Bloomfield, Nelson, and Soltes [2016] in this special issue). Over

the past two decades, economists have increasingly left the ivory tower and

made use of field experiments to explore economic phenomena, studying

actors from the farm to the factory to the boardroom (see Harrison and

List [2004]). Much different from experimentation in the hard sciences,

field experimenters in economics typically use randomization to estimate

treatment effects. And unlike laboratory experiments in the social sciences,

field experiments are typically conducted in naturally occurring settings, in

certain cases extracting data from people who might not be aware that they

are experimental participants.

In this way, field experiments provide a useful bridge between laboratory

and naturally occurring data in that they represent a mixture of control

and realism usually not achieved in the laboratory or with uncontrolled

data. This unique combination provides the researcher with an opportunity

to address questions that heretofore have been difficult to answer. Impor-

tantly, after grasping the interrelationships of factors in the chosen field

setting, the field experimenter then uses theory to understand more dis-

tant phenomena that have the same underlying structure. When this is

achieved, the deep rewards of field experimentation are realized.

In this study, we leverage what we have learned in the economics litera-

ture to provide a “how-to guide” for developing, implementing, and exe-

cuting efficient and robust field experiments in accounting and finance.

Interestingly, accounting research has traditionally focused on the mea-

surement and auditing of firm performance information that is for use

by both internal management for decision making and external users of

financial information, such as analysts and investors. Within the field of

accounting research, inquiries have ranged from the examination of ac-

counting and audit quality on a firm’s cost of capital to understanding the

effects of executive compensation contract provisions on CEO incentives.

Because of the institutional nature of accounting, research in these areas

USING FIELD EXPERIMENTS IN ACCOUNTING AND FINANCE 439

has typically been archival, focusing on the use of financial databases, such

as Compustat, to provide empirical evidence. Yet, despite the benefits of

using empirical settings that encompass a majority of the institutions of po-

tential interest, considerable challenges remain with respect to treatment

identification. Large-scale archival accounting research is often plagued by

the absence of exogenous variation, thereby limiting the degree to which

researchers can effectively demonstrate causality (Gow, Larcker, and Reiss

[2016]).

At the other end of the spectrum, experiments in accounting have tried

to address these issues in the laboratory. Laboratory experiments have ex-

tended across multiple areas of accounting research (Libby, Bloomfield,

and Nelson [2002]) and have identified several interesting phenomena

that are difficult to document empirically in archival studies. Beyond pro-

viding a playbook for conducting a field experiment, this article bridges the

gap between archival and laboratory work by showing the promise of field

experiments in accounting and finance, which, in many cases, permit the

researcher to establish a tighter link between theory and empirics.

In the next section, we begin by summarizing the various empirical

approaches one can use when trying to establish causality. We then de-

fine the various field experiments, provide a description of the behav-

ioral parameters that they estimate, and summarize how to implement a

field experiment—from theoretical construction to executing the optimal

experimental design. From there, we discuss three interconnected issues

related to the building of knowledge from field experiments. The issues

revolve around appropriate hypothesis testing, how to update one’s priors

after conducting a field experiment, and the role of replication. We con-

clude with a discussion of extant field experiments that touch on important

accounting and finance issues, and we review areas that are ripe for future

field experimental explorations.

2. Empirical Approaches

The empirical gold standard in the sciences is to identify a causal effect of

some variable (or set of variables) on another variable. For example, mea-

suring the effect of a new governmental law on the reporting of corporate

information and how various compensation schemes affect employee pro-

ductivity are queries for the scientist interested in causal relationships. The

difficulty that arises in establishing causality is that either the treatment is

given or it is not—we never directly observe what would have happened

in the alternative state. This problem, one of generating the appropriate

counterfactual, combined with the fact that, in the real world, there are

deep market complexities and simultaneously many moving parts, has led

scholars interested in empiricism to focus on the analysis of naturally occur-

ring data. This is where we begin our discussion, which draws heavily from

the related work of Harrison and List [2004], List [2006a], and Al-Ubaydli