Revealing forecaster's preferences: A Bayesian multivariate loss function approach

• Date: 01 April 2020
• DOI: http://doi.org/10.1002/for.2636
• Published date: 01 April 2020

RESEARCH ARTICLE

Revealing forecaster's preferences: A Bayesian multivariate

loss function approach

Emmanuel C. Mamatzakis

1

| Mike G. Tsionas

2

1

School of Business, Management and

Economics, University of Sussex,

Brighton, UK

2

Lancaster University Management

School, Lancaster, UK

Correspondence

Emmanuel C. Mamatzakis, School of

Business, Management and Economics,

Jubilee Building, University of Sussex,

Falmer, Brighton BN1 9SL, UK.

Email: e.mamatzakis@sussex.ac.uk

Abstract

Revealing the underlying preferences of a forecaster has always been at the core

of much controversy. Herein, we build on the multivariate loss function concept

and propose a flexible and multivariate family of likelihoods. This allows exam-

ining whether a vector of forecast errors, along with control variables, shapes a

forecaster's preferences and, therefore, the underlying multivariate, non-

separable, loss function. We estimate the likelihood function using Bayesian

exponentially tilted empirical likelihood, which reveals the shape of the parame-

ter and the power of the multivariate loss function. In the empirical section, the

reported evidence reveals that the EU Commission forecasts are predominantly

asymmetric, leaning towards optimism in the year ahead, while a correction

towards pessimism occurs in the current year forecast. There is some variability

of this asymmetry across member states, with forecasts, i.e. gross domestic prod-

uct growth, for large Member States exhibiting more optimism

KEYWORDS

asymmetric preferences, forecasting, multivariate loss function

1|INTRODUCTION

Given the importance of forecasting in economics, in

finance and operational research, it is hardly surprising

that testing for the underlying statistical properties of

forecasts has a long tradition that can be traced back

to Theil, (1966). Mincer and Zarnowitz, (1969) offered

a way of streamlining this procedure by testing

whether forecast errors have zero mean and are

uncorrelated with information available at the time

that forecasts are formed. Such testing implicitly

assumes that the forecaster has an underlying loss

function that is symmetric and quadratic. The symme-

try of the underlying loss function of the forecaster is

a very strong assumption as in the presence of asym-

metry in the loss function previous tests are losing

validity. If indeed the underlying loss function of the

forecaster is asymmetric then the standard Theil–

Mincer–Zarnowitz tests for efficiency are biased and

thereby misleading (see also Clements, 2015). Elliott,

Komunjer, and Timmermann, (2005, 2008) propose

tests that allow the estimation of the shape parameter

of an underlying loss function, and thereby reveal

whether the latter is symmetric or not.

1

Knowing the

shape of the loss function is important, as it reveals

the forecaster's underlying preferences and thus its

behavior.

2

1

Based on such testing, previous studies emerged, which test the

underlying shape of the loss function for a variety of forecasts (see

Christodoulakis & Mamatzakis, 2008, 2009).

2

The hypothesis of rational forecasts has been prominent in economic

theory and modeling since Lucas's report (1972). Rational expectations

crucially depend on the shape of the loss function, which is assumed to

be symmetric and mostly nonlinear and specifically quadratic (Elliott

et al., 2005, 2008; Komunjer & Owyang, 2012). In the event that the

underlying loss function is asymmetric, the forecaster will show

preferences, for example, to positive forecast errors compared to

negative forecast errors. Elliott et al., (2005) argue that if such

preferences are not widely known then forecasts are not rational.

Received: 29 October 2018 Revised: 4 October 2019 Accepted: 28 October 2019

DOI: 10.1002/for.2636

412 © 2019 John Wiley & Sons, Ltd. Journal of Forecasting. 2020;39:412–437.wileyonlinelibrary.com/journal/for

The results presented in this paper complement previ-

ous studies and fill the gap in the literature by providing

estimations of the underlying multivariate loss function

parameter, including whether it is linear or nonlinear,as

well as allowing for examining the impact of control vari-

ables on the shape of the loss function. Moreover, we pro-

pose a flexible likelihood function, which is consistent

with the multivariate loss function of Komunjer and

Owyang, (2012). This likelihood function is estimated

using Bayesian techniques of the multivariate loss as a

Bayesian exponentially tilted empirical likelihood

(BETEL).

3

In addition, our proposed modeling allows for

inferences of whether the shape of the multivariate loss

function is linear or nonlinear. In some detail, our contri-

bution to the literature is fourfold: First, we opt for a

Bayesian estimation of a flexible likelihood multivariate

loss function. Second, the proposed likelihood function

accommodates the estimation of the power of the under-

lying loss function. Our modeling also allows for control

variables that would explain the shape of the multivariate

loss function in single-stage estimation. Third, we provide

Monte Carlo evidence that the proposed BETEL likeli-

hood function estimation is unbiased and consistent.

Finally, we provide empirical evidence for the underlying

properties of the EU Commission forecasts that have

been the center of much attention since the financial

meltdown in 2009 and the bailout of several euro area

member states thereafter. Our sample covers the period

from 1969 to 2014 and refers to estimations of current

year and year-ahead forecasts. We examine the EU Com-

mission forecasts using the proposed BETEL multivariate

loss function estimation, which does not rely on implic-

itly assuming additive separability across forecast errors.

The results show that EU forecasts are predominantly

asymmetric and, to an extent, in which this information

is not disclosed, they are not rational, and in addition lin-

earity is also not present. The EU forecasts lean towards

optimism in years ahead, whereas they correct somewhat

this optimism towards pessimism in the current-year

forecasts. We also reveal that for large member states

forecasts lean towards optimism, which has certain policy

implications. Our multivariate loss function analysis

reveals that EU Commission forecasts should be inter-

preted with caution, as they tend to be rather optimistic,

in particular with respect to gross domestic product

(GDP) growth. Since the GDP growth forecasts are key

for the assessment of the national economic policy of the

EU member states, and in particular in the euro area,

optimistic forecasts allow for certain leeway against

tougher fiscal consolidation in order to meet the targets

set by the EU treaty. In addition, the Commission's fore-

casts also act as a benchmark upon which the condition-

ality imposed to financial constraint EU member states is

assessed. Such forecasts form the base of measuring any

‘fiscal and financial gap’of the stressed member states

that could, in turn, lay the base for requesting corrective

actions, such as fiscal adjustment and appropriate struc-

tural reforms. Therefore, from an economic policy point

of view, it is imperative that the EU Commission fore-

casts do not suffer from deviations from asymmetry, lean-

ing towards, for example, optimism.

The rest of the paper follows with Section 2, which

presents a new family of flexible-likelihood multivariate

loss function of forecasts. Section 3 provides data sources

and discusses the EU Commission forecasts, while

Section 4 provides the empirical results. The final

section offers some concluding remarks.

2|METHODOLOGY: ASYMMETRY

IN THE LOSS FUNCTION OF

FORECAST ERRORS

The starting point of testing for asymmetry in the loss

function is the framework proposed by Elliott et al.,

(2005).

4

Based on this asymmetric loss function we opt

for the following loss function:

Le;α,pðÞ=α+1−2αðÞIe≤0ðÞ½exp −ejj

p

ðÞ,ð1Þ

where eis the forecast error, α(alpha) is the asymmetry

parameter, and pis the parameter that nests the case of

both linear and nonlinear underlying loss function.

A drawback of the above loss function is that it is uni-

variate. A univariate loss function implicitly assumes

additive separability across forecast errors. This assump-

tion could result in a bias in the estimation of the asym-

metry (Komunjer & Owyang, 2012). Komunjer and

Owyang, (2012) proposed a new family of multivariate

loss functions to test the rationality of a vector of forecast

errors without assuming additive separability across such

errors. They also derived a generalized method of

moments (GMM) test for multivariate forecast rationality

3

Bekiros and Paccagnini, (2014) develop an interesting modeling of

Bayesian forecasting based on factor-augmented vector autoregressive

DSGE models, while Groen and Kapetanios, (2016) propose principal

components and Bayesian regressions for data reach macroeconomic

forecasting.

4

Elliott et al., (2005) employ the following loss function of the forecast

error (Y

t+1

−f

t+1

), where Y

t+1

is the actual value and f

t+1

the

forecast, L(p,α,θ)=[α+(1–2α)1(Y

t+1

−f

t+1

< 0)]|Y

t+1

−f

t+1

|

p

.

Note that αwould measure the asymmetry in the underlying loss

function and ptakes into account the linear and nonlinear case and θis

an unknown k-vector of parameters. This loss function can be

simplified by defining e=Y

t+1

−f

t+1,

the error term.

MAMATZAKIS AND TSIONAS 413