Performance Effects of Setting a High Reference Point for Peer‐Performance Comparison

• Author: HENRY EYRING,V. G. NARAYANAN
• Date: 01 May 2018
• DOI: http://doi.org/10.1111/1475-679X.12199
• Published date: 01 May 2018

DOI: 10.1111/1475-679X.12199

Journal of Accounting Research

Vol. 56 No. 2 May 2018

Printed in U.S.A.

Performance Effects of Setting

a High Reference Point for

Peer-Performance Comparison

HENRY EYRING

∗AND V. G. NARAYANAN

†

Received 2 November 2015; accepted 2 December 2017

ABSTRACT

We conduct a field experiment, based on a registered report accepted by the

Journal of Accounting Research, to test performance effects of setting a high

reference point for peer-performance comparison. Relative to providing the

median as a reference point for online students to compare themselves to,

providing the top quartile: damps performance for those below the median,

boosts performance for those between the median and top quartile, and, in

the case of outcome but not process comparison, boosts performance for

those above the top quartile. We do not find that either reference point yields

a greater average performance effect. However, providing the more effective

reference point in each partition of initial performance yields a 40% greater

performance effect than providing either reference point uniformly. Students

∗London School of Economics and Political Science; †Harvard Business School, Harvard

University.

Accepted by Douglas Skinner. This paper is the final Registered Report resulting from

the Registration-Based Editorial Process (REP) implemented by JAR for its 2017 confer-

ence; details of the process are available here: https://research.chicagobooth.edu/arc/

journal-of-accounting-research/2017-registered-reports. The accepted proposal and an On-

line Appendix for this report are available here: https://research.chicagobooth.edu/arc/

journal-of-accounting-research/online-supplements. We thank two anonymous reviewers for

their thorough feedback. Dennis Campbell, Tatiana Sandino, Ian Gow, Rajesh Vijayaragha-

van, and participants at two Harvard Business School seminars provided helpful comments.

We also thank Paul Smith for his assistance with software development and the administrators

at HarvardX, including Heather Sternshein, Rafael Irizarry, and Glenn Lopez, who generously

hosted the field experiment.

581

Copyright C, University of Chicago on behalf of the Accounting Research Center,2018

582 H.EYRING AND V.G.NARAYANAN

access the online courses intermittently over the span of a year. Our effects

derive from small portions of our treatment groups—5% in the case of pro-

cess comparison and 26% in the case of outcome comparison—who accessed

treatment and who were, on average, more active leading up to and during

our intervention.

JEL codes: C93; D91; I21; M41

Keywords: relative performance information; reference points; perfor-

mance; social comparison

1. Introduction

We test performance effects of setting a high reference point in rela-

tive performance information (RPI). RPI, or information for comparing

one’s own performance to that of peers, elicits performance improvement

in a variety of settings including when performance is not tied to pay

(Hannan, Krishnan, and Newman [2008], Allcott [2011], Tafkov [2013]).

Theories of social comparison, reference points, expectancy, and goals

could guide inquiry into the performance effects of the height of reference

points for peer-performance comparison that are commonly displayed in

RPI.1Such inquiry could inform the many government, nonprofit, and cor-

porate administrators who use RPI (e.g., Blanes i Vidal, and Nossol [2011],

Hallsworth et al. [2017], Song et al. [2017]). However, evidence regarding

the performance effects of RPI reference point height is lacking.

We provide such evidence using a field experiment in online education.

The field experiment is based on a registered report accepted by the Journal

of Accounting Research. Our research builds on recent field studies that sug-

gest that the peer-median reference point does not motivate above-median

performers to improve (Schultz et al. [2007], Chen et al. [2010], Allcott

[2011]). We extend these studies by testing whether a higher reference

point, the top quartile, has a different performance effect. By comparing

the median and top quartile as reference points, we assess reference points

that RPI designers often provide and that RPI literature has focused atten-

tion on (Grote [2005], Chen et al. [2010], Bizjak, Lemmon, and Nguyen

[2011], Gong, Li, Shin [2011]).

In our field experiment, we offer RPI with either of these two reference

points, each correctly labeled as the median or top quartile, for a period of

two months to students in online courses. We use data on a range of actions

in the courses and a record of each time a given student accesses RPI. The

field setting and our intervention do not involve explicit incentives for per-

formance, which helps us to identify the distinct information effects of RPI

reference point height.

1We study reference points that are percentiles of the peer-performance distribution. We

use the term “RPI reference point height” to refer to how high of a percentile of the distribu-

tion is provided as a reference point in RPI.

PERFORMANCE EFFECTS OF A HIGH REFERENCE POINT 583

RPI and reference points work through multiple forces to influence per-

formance. RPI allows peer comparison and so activates innate incentives to

attain a positive self-image that results from outperforming peers (Smith

[2000], Brown et al. [2007]). Individuals are averse to feelings of loss that

come from falling short of a reference point and so try to reach it (Abeler

et al. [2011], Allen et al. [2017]). Expectancy theory adds a qualification

on the incentives that RPI and reference points produce; a level of perfor-

mance must seem attainable to motivate improvement (Atkinson [1957]).

We draw on these and related theories to predict performance effects of

providing a higher RPI reference point than the median.2

As planned in our registered report proposal, we assess both an average

performance effect as well as performance effects in cross sections of our

sample. Theory suggests that providing a higher, rather than lower, refer-

ence point will most positively affect the performance of individuals who

are initially between the two alternatives. This implies a concave relation-

ship between the positive performance effect of reference point height and

an individual’s initial performance.

We find the predicted concave relationship. Relative to providing the me-

dian in RPI as a reference point, providing the top quartile damps perfor-

mance for those below the median, boosts performance for those between

the median and top quartile, and, in the case of RPI on an outcome mea-

sure but not a process measure, boosts performance for those above the

top quartile.

To understand the mixed results for top-quartile performers, we draw on

survey evidence on the outcome and process measures in our study. The

outcome measure, Grade, is the percentage of problems a student has an-

swered correctly. The process measure, Activity Level, is a weighted sum of a

student’s actions in the course, such as logins and video views. We find per-

sistent (diminished) interest in outperforming peers toward higher levels

of Grade (Activity Level).

These responses align with social comparison theory. Outcome compar-

isons that reflect ability tend to encourage competition to rise well above

the median (Smith [2000], Tafkov [2013]). Behavior and process compar-

isons tend to discourage substantial deviation from the median (Chen et al.

[2010], Allcott [2011], Dolan et al. [2012]). Our display of the top-quartile

reference point for Grade might motivate top-quartile performers to fur-

ther exceed it because they desire to distinguish themselves significantly

by an outcome measure that reflects ability. By contrast, our display of the

top-quartile reference point for Activity Level might not yield such a perfor-

mance response because top-quartile performers feel it is not important,

and perhaps suboptimal, to be so far from the norm of a process measure.

2Festinger [1954] and Smith [2000] address foundational theory regarding social com-

parison, Kahneman and Tversky [1979] regarding reference points and loss aversion, and

Atkinson [1957] and Vroom [1964] regarding expectancy-based motivation.