Beyond Random Assignment: Credible Inference and Extrapolation in Dynamic Economies

• Published date: 01 April 2020
• Author: CHRISTOPHER A. HENNESSY,ILYA A. STREBULAEV
• DOI: http://doi.org/10.1111/jofi.12862
• Date: 01 April 2020

THE JOURNAL OF FINANCE •VOL. LXXV, NO. 2 •APRIL 2020

Beyond Random Assignment: Credible Inference

and Extrapolation in Dynamic Economies

CHRISTOPHER A. HENNESSY and ILYA A. STREBULAEV∗

ABSTRACT

Wederive analytical relationships between shock responses and theory-implied causal

effects (comparative statics) in dynamic settings with linear profits and linear-

quadratic stock accumulation costs. For permanent profitability shocks, responses

can have incorrect signs, undershoot, or overshoot depending on the size and sign

of realized changes. For profitability shocks that are i.i.d., uniformly distributed, bi-

nary, or unanticipated and temporary, there is attenuation bias, which exceeds 50%

under plausible parameterizations. We derive a novel sufficient condition for prof-

itability shock responses to equal causal effects: martingale profitability. We estab-

lish a battery of sufficient conditions for correct sign estimation, including stochastic

monotonicity. Simple extrapolation/error correction formulas are presented.

IN APPLIED MICROECONOMICS FIELDS such as corporate finance, theory-implied

causal effects are generally derived using comparative statics. For instance, as

Heckman (2000) writes, “Comparative statics exercises formalize Marshall’s

notion of a ceteris paribus change which is what economists mean by a causal

effect.” Athey, Milgrom, and Roberts (1998) similarly observe that “most of the

testable implications of economic theory are comparative static predictions.”

In assessing implications of tradeoff theory, Welch (2010) states that “The

comparative statics analysis are simple and obvious: The firm should have

more debt if the interest rate is higher, if the corporate tax rate is higher, and

the cost of distress is lower.”

The objective of much econometric work is to empirically estimate signs

and magnitudes of causal effects. The estimates can be compared to theory-

predicted causal effects and can also be used to inform welfare analysis.

Theories predicting incorrect signs are informally referred to as “falsified,”

while those with small estimated causal effects are informally referred to

∗Christopher Hennessy is with London Business School, CEPR, and ECGI. Ilya A. Strebulaev

is with Stanford Graduate School of Business and NBER. We thank Michael Roberts and Philip

Bond (the Editors) and two anonymous referees for valuable input. We also thank Manuel Adelino,

Antoinette Schoar, and Luke Taylor (discussants), as well as seminar participants at Stanford,

Wharton, LBS, Duke, Boston University,LSE, UNC, UBC, Maryland, NC State, Imperial College,

Simon Fraser,Koc, INSEAD, VGSF, SFI, SFS Cavalcade, the Causal Inference in Finance and Eco-

nomics Conference, and the Stanford Conference on Causality in the Social Sciences. Disclosures:

Hennessy received funding from the European Research Council related to this project. Strebulaev

received no funding related to this project. We have read The Journal of Finance’sdisclosure policy

and have no conflicts of interest to disclose.

DOI: 10.1111/jofi.12862

C2019 the American Finance Association

825

826 The Journal of Finance R

as “second-order.” Small empirical estimates of causal effects have been

used to argue that government-induced behavior distortions generate small

deadweight losses (see, for example, Slemrod (1990) and Aaron (1990)).

In their textbook, Mostly Harmless Econometrics: An Empiricist’s Com-

panion (MHE hereafter), Angrist and Pischke (2009) imply that the primary

barrier to correct empirical estimation of causal effects is endogeneity and se-

lection bias: “The goal of most empirical research is to overcome selection bias,

and therefore to have something to say about the causal effect of a variable.”

They then state that “A principle that guides our discussion is that most of

the estimators in common use have a simple interpretation that is not heavily

model dependent.” The perceived conjunction of credibility and simplicity of

interpretation has led to a surge in the popularity of the MHE methodology.

A limitation of the MHE methodology in practice, however, is that it is silent

on how to address the inherent complications arising from dynamic uncer-

tainty. Indeed, while there is much discussion about shock exogeneity, there

is little discussion, let alone formal estimation, of the data-generating process

responsible for the exploited shocks. Moreover, there are few formal attempts

to map observed shock responses back to theory-implied causal effects (theory

testing), extrapolate shock responses across stochastic environments (external

validity), or estimate causal parameters entering welfare calculations. In some

cases, researchers may have a prior suggesting that dynamic uncertainty is

a second-order concern. In other cases, researchers are aware that inference

is somehow clouded, but do not know the exact nature of the problems or

how to address them at an operational level without resorting to full-scale

structural estimation.

Issues arising from dynamic uncertainty are avoided if one claims that

an exploited shock is unanticipated and permanent. However, this common

assumption suffers from a host of problems. First, confining attention to truly

rare events means that timely evidence is unlikely to exist. Second, the notion

that an exploited shock is permanent generally contradicts the underlying

motivation for empirical work, namely, to help optimize policy in the future

and/or to forecast responses to future shocks. Third, inspection of time series

for many policy variables such as the effective corporate tax rate and the real

minimum wage reveals changes to be the norm, not the exception. Finally,

there is a fundamental difference between shocks never occurring and shocks

taking place infrequently, say, once a decade in expectation. In the latter

case, the arrival of a shock today might be casually treated as a “surprise.”

Nevertheless, as we show, the bias arising from expectations in such cases can

still be quite large.

What special challenges does dynamic uncertainty pose for researchers

performing shock-based inference, how important are they, and what can be

done to overcome them without resorting to full-scale structural estimation? To

address these questions, we develop a canonical model of firm decision making

flexible enough to capture a broad range of commonly studied dependent

variables and alternative causal theories. In our baseline model, a firm

dynamically accumulates a (divisible) stock, such as capital, inventories,

Beyond Random Assignment 827

patents, debt, cash, or employees, which provides a (linear) flow of benefits

over time while facing (linear-quadratic) accumulation costs.

Using the model as our laboratory, we consider an econometrician whose ob-

jective is to infer whether the shock responses she observes are consistent with

the theory-implied causal effects. Fortunately, we know the theory-implied

causal effects, which we compute using comparative statics. In contrast, our

econometrician is forced to rely upon shock responses for inference. Critically,

however, we consider a setting in which the endogeneity challenges to which

MHE devotes exclusive attention are a nonissue. Specifically, we assume that

shocks take the form of exogenous Markov chain transitions. Examining the

relationship between theory-implied causal effects and shock responses in this

laboratory allows us to assess whether the MHE tool kit suffices for inference

of theory-implied causal effects in dynamic settings.

Our primary focus is on inference when the econometrician exploits shocks

that bring about changes in the flow of periodic benefits generated by the

stock, for example, shocks to taxes, regulations, competition, productivity,

input prices, or yields on cash/debt. To begin, we derive a set of tractable

analytical formulas relating shock responses to theory-implied causal effects.

Each formula can be used to perform back-of-the-envelope bias calculations or

inverted to correct for bias.

The bias formulas reveal troubling implications. If the shock is, say, binary,

then shock responses can represent severely downward-biased estimates of

theory-implied causal effects. For example, under our baseline parameteriza-

tion, attenuation bias exceeds one-half the causal effect if the shock probability

exceeds just 5.26%. Worse still, shock responses can be biased upward and

even have signs opposite to that of the causal effect. For example, if a shock is

permanent, the shock response will overshoot the causal effect if the realized

change is opposite in sign to the expected change. Intuitively, objectively good

(bad) news becomes great (terrible) news if bad (good) news was expected. Even

worse still, the shock response will have the wrong sign if the realized change is

sufficiently small relative to the expected change. Intuitively, objectively good

(bad) news becomes bad (good) news if even better (worse) news was expected.

The problems that these examples highlight are not theoretical curiosities.

To illustrate, we use the time series of historical effective tax rates to estimate

a Markov chain approximating the data-generating process confronting

corporations. We then analytically compare simulated shock responses to

theory-implied causal effects. Under our baseline parameterization, we

find that attenuation bias is severe, with shock responses generally falling

just below one-half the theory-implied causal effect. Intuitively, real-world

transience in tax rates makes firms much less responsive to shocks than would

be the case if the shocks were unanticipated and permanent. Such severe

magnitude bias has important policy implications. After all, as we show, the

same 50% attenuation bias exhibited by shock responses carries over directly

to estimated tax elasticity and excess burden calculations.

To make sense of shock responses, mapping them back to theory-implied

causal effects or extrapolating them, one must move beyond exogeneity by