Man Versus Machine: Complex Estimates and Auditor Reliance on Artificial Intelligence

• Published date: 01 March 2022
• Author: BENJAMIN P. COMMERFORD,SEAN A. DENNIS,JENNIFER R. JOE,JENNY W. ULLA
• Date: 01 March 2022
• DOI: http://doi.org/10.1111/1475-679X.12407

DOI: 10.1111/1475-679X.12407

Journal of Accounting Research

Vol. 60 No. 1 March 2022

Printed in U.S.A.

Man Versus Machine: Complex

Estimates and Auditor Reliance on

Artificial Intelligence

BENJAMIN P. COMMERFORD ,∗SEAN A. DENNIS ,†

JENNIFER R. JOE ,‡AND JENNY W. ULLA §

Received 19 July 2019; accepted 26 May 2021

ABSTRACT

Audit firms are investing billions of dollars to develop artificial intelligence

(AI) systems that will help auditors execute challenging tasks (e.g., evaluating

complex estimates). Although firms assume AI will enhance audit quality, a

growing body of research documents that individuals often exhibit “algorithm

aversion”—the tendency to discount computer-based advice more heavily

than human advice, although the advice is identical otherwise. Therefore, we

conduct an experiment to examine how algorithm aversion manifests in audi-

tor judgments. Consistent with theory, we find that auditors receiving contra-

dictory evidence from their firm’s AI system (instead of a human specialist)

∗University of Kentucky; †University of Central Florida; ‡University of Delaware; §University

of Nevada, Las Vegas

Accepted by Rodrigo Verdi. This paper benefited from thoughtful comments from an

anonymous associate editor and an anonymous reviewer. We thank Sanaz Aghazadeh, Tim

Bauer,Jessica Buchanan, Jon Grenier, Rick Hatfield, Sean Hillison, Blake Holman, Khim Kelly,

Jared Koreff, Tamara Lambert, Justin Leiby, Curtis Mullis, EB Poziemski, Greg Trompeter,

and Aubrey Whitfield as well as workshop participants at the University of Alabama, Baruch

College, the University of Central Florida, the University of Kansas, and Kent State Univer-

sity for their helpful comments. We also appreciate the feedback received from participants

at the University of Waterloo Centre for Accounting Ethics’ 2019 Ethics Symposium, 2019

PCAOB/TAR Conference on Auditing and Capital Markets, 2020 Hawaii Accounting Re-

search Conference, and 2020 Auditing Section Midyear Conference. We also thank the audit

firms that provided participants for this study and gratefully acknowledge funding from the

Von Allmen School of Accountancy at the University of Kentucky.

171

© 2021 The Chookaszian Accounting Research Center at the University of Chicago Booth School of

Business

172 b. p. commerford, s. a. dennis, j. r. joe, and j. w. ulla

propose smaller adjustments to management’s complex estimates, particu-

larly when management develops their estimates using relatively objective (vs.

subjective) inputs. Our findings suggest auditor susceptibility to algorithm

aversion could prove costly for the profession and financial statements users.

JEL codes: M40, M41, M42, O30, O33

Keywords: accounting estimates; auditing; artificial intelligence; algorithm

aversion; subjectivity; competing information

1. Introduction

Some of the largest audit firms have touted plans to invest billions of dol-

lars in audit technologies in the coming years, with the goal of enhancing

the effectiveness, efficiency, and decision-usefulness of audits (Bloomberg

Tax [2020]). One of the most promising advanced technologies under con-

sideration is the application of machine learning or artificial intelligence

(AI) on audit engagements. AI can synthesize large amounts of diverse and

unstructured data, and some firms are harnessing these abilities to help

auditors perform tasks that have traditionally been performed by human

specialists, such as evaluating complex accounting estimates (e.g., commer-

cial loan grades; KPMG [2016]). In doing so, audit firms are carefully de-

veloping “narrow AI” systems to perform specific tasks within certain pro-

grammable parameters.1Currently, the goal for these AI systems is to use

machine learning to replicate specific (yet complex) judgments that audi-

tors and human specialists make (e.g., Deloitte [2014, 2016], PwC [2017]).

Firms believe that applying these advanced technologies to the audit set-

ting will enhance audit quality and provide significant benefits for auditors

and clients (FEI [2017], EY [2018]). These benefits, however, will only ma-

terialize if auditors consider and incorporate the information produced by

such systems into their evidence evaluation. Therefore, this study examines

when and how receiving contradictory evidence from a firm’s AI system–

rather than a firm’s human specialist–influences auditor judgments related

to complex estimates.

Auditors lack the requisite expertise to evaluate management’s complex

estimates on their own, and are therefore expected to rely on advice from

1According to the Brookings Institution (e.g., West and Allen [2018]), AI incorporates in-

tentionality, intelligence (i.e., “machine learning”), and adaptability. “General AI” refers to a

fully autonomous system that can learn about any problem and then solve it (e.g., Deloitte

[2018a]). With narrow AI systems (e.g., KPMG [2016]), firms can maintain more direct con-

trol over their audit methodologies and how they comply with current auditing standards. This

focus on narrow AI also is consistent with documented concerns around potential regulatory

scrutiny and legal liability that can arise from the use of autonomous, unsupervised AI (e.g.,

Christ, Emett, Summers, and Wood [2021], Emett, Kaplan, Mauldin, and Pickerd [2021]).

Importantly, while firms appear focused on narrow AI currently,they intend to use this tech-

nology to assist with some of the most challenging tasks that humans perform during audits

(e.g., KPMG [2016]).