Causal Inference in Accounting Research

• Date: 01 May 2016
• Author: IAN D. GOW,DAVID F. LARCKER,PETER C. REISS
• DOI: http://doi.org/10.1111/1475-679X.12116
• Published date: 01 May 2016

DOI: 10.1111/1475-679X.12116

Journal of Accounting Research

Vol. 54 No. 2 May 2016

Printed in U.S.A.

Causal Inference in Accounting

Research

IAN D. GOW,

∗DAVID F. LARCKER,

†AND PETER C. REISS

†

ABSTRACT

This paper examines the approaches accounting researchers adopt to draw

causal inferences using observational (or nonexperimental) data. The vast

majority of accounting research papers draw causal inferences notwithstand-

ing the well-known difficulties in doing so. While some recent papers seek to

use quasi-experimental methods to improve causal inferences, these methods

also make strong assumptions that are not always fully appreciated. We be-

lieve that accounting research would benefit from more in-depth descriptive

research, including a greater focus on the study of causal mechanisms (or

causal pathways) and increased emphasis on the structural modeling of the

phenomena of interest. We argue these changes offer a practical path forward

for rigorous accounting research.

JEL codes: C18; C190; C51; M40; M41

Keywords: Causal inference; accounting research; quasi-experimental

methods; structural modeling

1. Introduction

There is perhaps no more controversial practice in social and biomedi-

cal research than drawing inferences from observational data. Despite ...

∗Harvard Business School; †Rock Center for Corporate Governance, Stanford Graduate

School of Business.

Accepted by Philip Berger.We are grateful to our discussants, Christian Hansen and Miguel

Minutti-Meza, and participants at the 2015 JAR Conference for helpful feedback. We also

thank seminar participants at London Business School, Karthik Balakrishnan, Robert Ka-

plan, Christian Leuz, Alexander Ljungqvist, Eugene Soltes, Daniel Taylor, Robert Verrecchia,

Charles Wang, and Anastasia Zakolyukina for comments.

477

Copyright C, University of Chicago on behalf of the Accounting Research Center,2016

478 I.D.GOW,D.F.LARCKER,AND P.C.REISS

problems, observational data are widely available in many scientific fields

and are routinely used to draw inferences about the causal impact of inter-

ventions. The key issue, therefore, is not whether such studies should be

done, but how they may be done well. (Berk [1999, p.95])

Most empirical research in accounting relies on observational (or non-

experimental) data. This paper evaluates the different approaches account-

ing researchers adopt to draw causal inferences from observational data.1

Our discussion draws on developments in fields such as statistics, econo-

metrics, and epidemiology. The goal of this paper is to identify areas for

improvement and suggest how empirical accounting research can improve

inferences drawn from observational data.

The importance of causal inference in accounting research is clear from

the research questions that accounting researchers seek to answer. Most

long-standing questions in accounting research are causal: Does conser-

vatism affect the terms of loan contracts? Do higher quality earnings reports

lead to lower information asymmetry? Did International Financial Report-

ing Standards cause an increase in liquidity in the jurisdictions that adopted

them? Do managerial incentives lead to managerial misstatements in finan-

cial reports? The accounting researchers focus on causal inference, which

is consistent with the view that “the most interesting research in social sci-

ence is about questions of cause and effect” (Angrist and Pischke [2008,

p. 3]). Simply documenting descriptive correlations provides little basis for

understanding what would happen should circumstances change, whereas

using data to make inferences that support or refute broader theories could

facilitate these kinds of predictions.

To provide insights into what is actually done in empirical accounting

research, we examined all papers published in three leading accounting

journals in 2014. While accounting researchers are aware of problems that

can arise from the use of observational data to draw causal inferences, we

found that most papers still seek to draw such inferences. Making causal

inferences requires strong assumptions about the causal relations among

variables. For example, estimating the causal effect of Xon Yrequires

that the researcher has controlled for variables that could confound esti-

mates of such effects. Section 2 provides an overview of causal inference

using causal diagrams as a framework for thinking about the subtle issues

involved. We believe that these diagrams are also very useful for communi-

cating the cause-and-effect logic underlying regression analyses that use ob-

servational data. Nonetheless, difficulties identifying, measuring, and con-

trolling for all possible confounding variables have led many to question

causal inferences drawn from observational data.

Recently, some social scientists have held out hope that better re-

search designs and statistical methods can increase the credibility of causal

1Thus, our focus is on what Bloomfield, Nelson, and Soltes [2016] call “archival studies.”

Floyd and List [2016] discuss opportunities for researchers to use experiments in accounting

research.

CAUSAL INFERENCE IN ACCOUNTING RESEARCH 479

inferences. For example, Angrist and Pischke [2010] suggest that “empir-

ical microeconomics has experienced a credibility revolution, with a con-

sequent increase in policy relevance and scientific impact.” Angrist and

Pischke [2010, p. 26] argue that such “improvement has come mostly

from better research designs, either by virtue of outright experimenta-

tion or through the well-founded and careful implementation of quasi-

experimental methods.” Our survey of research published in 2014 finds 5

studies claiming to study natural experiments (or “exogenous shocks”) and

10 studies using instrumental variables (IVs). Although these numbers sug-

gest that quasi-experimental methods are infrequently used in accounting

research, we believe their use will increase in the future.2

Section 3 evaluates the use of quasi-experimental methods in account-

ing research. Quasi-experimental methods produce credible estimates of

causal effects only under very strong maintained assumptions about the

model and data. For example, variations in treatments are rarely random,

the list of controls rarely exhaustive, and instruments do not always sat-

isfy the necessary inclusion and exclusion restrictions. We explain some of

these concerns using causal diagrams. In general, it appears that the as-

sumptions required to apply quasi-experimental methods are unlikely to

be satisfied by observational data in most empirical accounting research

settings.

Ultimately, we believe that accounting research needs to recognize the

stringent assumptions that need to be maintained to apply statistical meth-

ods to derive estimates of causal effects for observational data. Statistical

methods alone cannot solve the inference issues that arise in observational

data. The second part of the paper (sections 4 and 5) identifies approaches

that can provide a plausible framework for guiding future accounting

research. Specifically:

rThere should be an increased emphasis on the study of causal mecha-

nisms, that is, the “pathways” through which claimed causal effects are

propagated. We believe that evidence on the actions and beliefs of indi-

viduals and institutions can bolster causal claims based on associations,

even absent compelling estimates of the causal effects. We also suggest

that more careful modeling of phenomena, using structural modeling

or causal diagrams, can help to identify plausible mechanisms that war-

rant further study.

rCausal diagrams are a useful tool for conveying the key elements of

a structural model and can also act as a middle-level stand-in when

structural modeling of a phenomenon is infeasible.3

2We use the term “quasi-experimental” methods to refer to those methods that have a plau-

sible claim to “as if” random assignment to treatment conditions. The term “as if” is used by

Dunning [2012] to acknowledge the fact that assignment is not random in such settings, but

is claimed to be as if random assignment had occurred.

3“Middle-level” here refers to the placement of causal diagrams between relatively informal

verbal reasoning and the rigors of a structural model.