What can we learn from the fifties?

• Date: 01 November 2017
• Published date: 01 November 2017
• Author: Fabian Gouret
• DOI: http://doi.org/10.1002/for.2468

Received: 20 July 2016 Revised: 7 December 2016 Accepted: 25 February 2017

DOI: 10.1002/for.2468

RESEARCH ARTICLE

What can we learn from the fifties?

Fabian Gouret

THEMA, Université de Cergy-Pontoise,

France

Correspondence

Fabian Gouret, THEMA, Université de

Cergy-Pontoise, 33 Bvd du Port, 95011

Cergy-Pontoise Cedex, France.

Email: fabiangouret@gmail.com

Funding information

Labex MME-DII ; Chaire d’Excellence

CNRS

Abstract

Economists haveincreasingly elicited probabilistic expectations from survey respon-

dents. Subjective probabilistic expectations show great promise to improve the

estimation of structural models of decision making under uncertainty. However, a

robust finding in these surveys is an inappropriate heap of responses at “50%,” sug-

gesting that some of these responses are uninformative. The waythese 50s are treated

in the subsequent analysis is of major importance. Taking the 50s at face value will

bias any aggregate statistics. Conversely, deleting them is not appropriate if some of

these answers do conveysome information. Further more, the attention of researchers

is so focused on this heap of 50s that they do not consider the possibility that other

answers may be uninformative as well. This paper proposes to take a fresh look at

these questions using a new method based on weak assumptions to identify the infor-

mativeness of an answer.Applying the method to probabilistic expectations of equity

returns in three waves of the Survey of Economic Expectations in 1999–2001, I find

that: (i) at least 65% of the 50s convey no information at all; (ii) it is the answer

most often provided among the answers identified as uninformative; (iii) but evenif

the 50s are a major contributor to noise, they represent at best 70% of the identified

uninformative answers. These findings have various implications for survey design.

KEYWORDS

epistemic uncertainty and fifty-fifty, subjective probability distribution, survey data

1INTRODUCTION

Since the 1990s, an important empirical literature has devel-

oped to measure probabilistic expectations that individuals

hold about future events (Manski, 2004). Subjective prob-

abilistic expectations show great promise to improve the

estimation of structural models of decision-making under

uncertainty.1However, researchers are particularly embar-

rassed by a seemingly inappropriate high frequency of

responses at “50%”. Table 1 provides some examples of this

heap of 50s in surveys whichhave included probabilistic ques-

tions. There are at least two interrelated problems with these

1For work in that direction, see Delavande (2008), Giustinelli (2016), Van

der Klaauw and Wolpin (2008), as well as Attanasio (2009, pp. 90–91) who

provides an overview of some of his papers in progress.

50s. The first one is that some of them are probably uninfor-

mative; that is, some respondents provide this answer when

they do not know what probability to answer in the interval

[0,100].2Fischhoff and Bruine de Bruin (1999), for instance,

argue that some respondents use this answer as a proxy for

the verbal expression “fifty-fifty”, an expression used to mean

“I really don’t know.” If so, treating all the 50s as any other

answer will bias any aggregate statistics. Conversely, purely

suppressing all of them is not appropriate, particularly if

some of them do actually convey some information. Given

2My use of the term “uninformative” deserves perhaps a comment or a justi-

fication. When a respondent feels unable to assign any subjectiveprobability

to an event, that is, he is in a state of complete ignorance, it means that

he has a total absence of relevant information. Therefore, if this respondent

provides an answer, this answer conveys no information; hence the term

“uninformative” answer.

756 Copyright © 2017 John Wiley & Sons, Ltd. wileyonlinelibrary.com/journal/for Journal of Forecasting.2017;36:756–775.

GOURET 757

that survey questions usually do not directly reveal whether

a response provides an uninformative answer, it raises the

question: How can one identify if a 50 is uninformative? The

second related problem is that the attention of researchers is so

focused on this heap of 50s that they do not consider the possi-

bility that other responses may be uninformative as well. This

raises the second question: Do the 50s represent a substantial

share of all uninformative answers?

The present paper proposes a conservative solution appli-

cable to a specific but widely used format of probabilistic

questions used to determine the distribution of the future real-

ization of a continuous variable. As argued in this paper, this

format, introduced initially in the Survey of Economic Expec-

tations (SEE) by Dominitz and Manski (1997), facilitates ex

post control on the informativeness of a probabilistic answer;

and although this method is only applicable in the specific

context of this format, its application permits to shed light

on the appropriateness of some practices in the related litera-

ture. Let me introduce this format first. When the variable is

continuous, the SEE asks each respondent iasequence of K

probabilistic questions of the type

What is the percent chance that, one year from

now, Riwould be worth over ri,k?

Hence, for each respondent i, we observe the probabilistic

answers Qi,k≡PRi>ri,k,k=1,2,…,K,whereri,1 <

ri,2 <…<ri,Kare Kthresholds about which the respondent

is queried.3Ridenotes the variable of interest; it can be the

respondent’s household income (Dominitz & Manski, 1997),

the respondent’s personal income (Dominitz, 2001), or the

stock market returns according to the respondent (Dominitz

& Manski, 2011). The responses to this sequence of Kprob-

abilistic questions enable the estimation of each respondent’s

subjective distribution. The sequence of Kspecific thresh-

olds is chosen among a finite number of possible sequences

by an algorithm that uses the average of the the respondent’s

answers ri,min and ri,max to two preliminary questions asking

for the lowest and highest possible valuest hat Rimight take.4

Table 2 provides an example that I will study in detail in

Sections 3 and 4: It considers the questions posed in three

waves of the SEE (12, 13 and 14) in July 1999 to March2001

to measure the probabilistic expectations that Americans hold

about equity returns in the year ahead. The SEE first poses a

scenario highlighting that the respondent has to consider the

performance of an investmentof $1000 in a diversified mutual

3Instead of asking Kpoints on respondent i’s subjective complementary

cumulative distribution, it is possible to ask points on respondent i’s subjec-

tive cumulative distribution; see, for example,Dominitz (2001).

4These two preliminary questions are useful to get an idea of the support of

the distribution. Dominitz and Manski (1997) also note that these prelim-

inary questions decrease overconfidence with respect to central tendencies

and anchoring problems wherein respondents’ beliefs are influenced by the

questions posed.

fund in the year ahead. Then the two preliminary questions are

posed, and the responses are used to select via the algorithm

a sequence of K=4 threshold values among a set of five pos-

sible sequences. Finally, the SEE asks the sequence of K=4

probabilistic questions. For instance, if a respondent answers

ri,min =600 and ri,max =1100, his midpoint is 850, and he

is asked the percent chance that next year’s investment in the

mutual fund would be worth over {500,900,1000,1100}.

The objective is to infer the informativeness of a subjec-

tive probability Qi,k. Note that the response Qi,1 is particularly

important because it determines the answers for the subse-

quent thresholds, while the reverse is not true: Once a respon-

dent answers Qi,1, his answers to the next thresholds have to

weakly fall to satisfy the monotonicity of the complementary

cumulative distribution function (Qi,1 ⩾Qi,2 ⩾…⩾Qi,K).5

Hence I will mainly focus on the informativeness of Qi,1.

To understand the method, consider a respondent who has

a precise subjective distribution in mind concerning Ri.Ifso,

he is able to provide informative answers to the preliminary

questions and the sequence of probabilistic questions, that is,

answers that reflect sharp expressions of beliefs. If this is

the case, he should also provide coherent answers between

the preliminary questions and the sequence of probabilistic

questions; that is, he should use the same underlying subjec-

tive distribution to answer the preliminary questions and the

sequence of probabilistic questions. Alternatively, a respon-

dent may have imprecise beliefs concerning Ri; that is, he

does not hold a unique subjective distribution but a set of sub-

jective distributions. This imprecision can occur because he

lacks some information; he may also be unable to form pre-

cise beliefs in practice, even if he is able to do so in principle,

because he lacks time and thinking is costly. Whatever the

reason for this imprecision, note that if a respondent provides

clearly incoherent answersbetween the preliminar y questions

and the sequence of probabilistic questions, that is, he does not

use the same underlying subjective distribution, then one can

be sure that the respondent has imprecise beliefs; his answers

are thus partially informative. And if the incoherence is too

high, one can be sure that the respondent has extremely impre-

cise beliefs (a case wherein he lacks any relevant information

or he does not want to put any effort into his answers). In that

case, his answers are clearly uninformative.

The general idea is thus to exploit the answers ri,min and

ri,max to the preliminary questions to make a prediction

̃

PRi>ri,1of an answer Qi,1. The distance between a pre-

dicted value and the actual value is a measure of coherence.

If this measure is too high, Qi,1 is highly incoherent with the

respondent’s answer to the pair of preliminary questions, so

the respondent has an extremely imprecise belief on the event

Ri>ri,1. One can thus infer that Qi,1 is uninformative. If this

5Logically, the interviewer informsthe respondent if a probability elicited at

threshold ri,2,ri,3,…or ri,Kis higher than one elicited previously to ensure

the monotonicity condition (Dominitz & Manski, 2004).